1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise...
-
Upload
oliver-hensley -
Category
Documents
-
view
216 -
download
0
Transcript of 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise...
![Page 1: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/1.jpg)
1
Today’s Plan
• 900am–915am: Sort out questions regarding refund forms
• 915am–945am: Finalise today’s agenda• 945am-1045am: Breakout Groups Session 1• 1045am–1115am: Coffee + recalibration at 11• 1115am–1215pm: Breakout Groups Session 2• 1215pm–145pm: Lunch• 145pm–245pm: Report-backs• 245pm–300pm: Wrap-up
![Page 2: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/2.jpg)
2
Yesterday’s Shifting Sentiment #1
0
1
2
3
4
5
6
0 1 2 3 4 5 6 7 8 9 10
Round 1
Round 2
Round 3
![Page 3: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/3.jpg)
3
Yesterday’s Shifting Sentiment #2
0
2
4
6
8
10
12
14
16
Round 1 Round 2 Round 3
0-2
3-6
7-10
![Page 4: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/4.jpg)
4
Yesterday’s Main Themes #1
• General Methodological Concerns:– Reasons to be cautious [Scott, Moore, Di
Eugenio, McCoy, Green]• Frameworks for Evaluation:
– Frameworks for evaluation [Paris, Stent]– GENEVAL [Belz and Reiter] – Building on RAGS [Mellish + Scott]
• Resources:– A Wiki [Stent]– Shared data resources [Walker]
![Page 5: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/5.jpg)
5
Yesterday’s Main Themes #2
• Shared Tasks:– DUC or other existing STECs as a host [McKeown]
– Referring Expression Generation [Gatt, Viethen]
– Virtual Environments [Koller et al]– Question Qeneration [Rus + Graesser]– Textual Variation [McDonald]
![Page 6: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/6.jpg)
6
Today’s Goals
• Observation:– Opinion is spread between those who favour a Shared Task (one or many) and those who are cautious
• Hypothesis #1:– Those who favour a Shared Task will push ahead anyway
• Hypothesis #2:– Those who are cautious are the best people to identify the relevant desiderata
![Page 7: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/7.jpg)
7
Possible Working Groups
• Desiderata Development• Evaluation• Shared Resources• Shared Task #1: Text to Text (perhaps
via DUC?)• Shared Task #2: Referring Expression
Generation• Shared Task #3: Data to Text• Shared Task #4: Virtual Environments
![Page 8: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/8.jpg)
8
Target Outcome
• A document• How far can we get today?
– A set of PowerPoint files that plans the sections of the document to one bullet point per subsubsection or paragraph
– Report-back on planned content to the assembled group
• After the workshop:– Flesh out the text
![Page 9: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/9.jpg)
9
Task 1: Desiderata
• Main Sections:– What’s unique about NLG Evaluation– Approaches and Frameworks– A ‘due diligence checklist’ —` a clear enumeration of the things you should consider if you are going to pursue a Shared Task in NLG
![Page 10: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/10.jpg)
10
Tasks 2 through N: Shared Tasks Proposals
• Main Sections:– A definition of the Shared Task and its type– The aims of the Shared Task: what it will achieve– The subcommunity it seeks to engage, and how it will do this
– Whether the task will involve evaluation and how– The resources required, and how they will be obtained
– A plan of execution– [Later] How this Shared Task addresses the desiderata
![Page 11: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/11.jpg)
11
Today’s Plan
• 900am–915am: Sort out questions regarding refund forms
• 915am–945am: Finalise today’s agenda• 945am-1045am: Breakout Groups Session 1• 1045am–1115am: Coffee + recalibration• 1115am–1215pm: Breakout Groups Session 2• 1215pm–115pm: Lunch• 115pm–245pm: Report-backs• 245pm–300pm: Wrap-up
![Page 12: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/12.jpg)
12
Possible Working Groups
• Desiderata Development [5]• Shared Task #1: Text to Text (perhaps
via DUC?) [3]• Shared Task #2: Referring Expression
Generation [5]• Shared Task #4: Virtual Environments
[5]
![Page 13: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/13.jpg)
13
New Thoughts and Considerations
• Is this working?• Is the spec broken or can it be refined?• What have you thought of that you
think the others could benefit from in working out their plan?
![Page 14: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/14.jpg)
14
Today’s Plan
• 900am–915am: Sort out questions regarding refund forms
• 915am–945am: Finalise today’s agenda• 945am-1045am: Breakout Groups Session 1• 1045am–1115am: Coffee + recalibration• 1115am–1215pm: Breakout Groups Session 2• 1215pm–115pm: Lunch• 115pm–245pm: Report-backs• 245pm–300pm: Wrap-up
![Page 15: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/15.jpg)
15
Next Steps
• One or two people in each group commits to making sure there is follow-through
• Each group commits to writing up their section of the report and delivering a draft to MW + RD by June 1st.
• At time June 1st, all sections are redistributed to all workshop participants.
• All can send comments on any part to MW+RD by mid June.• In mid-June, comments are collated and redistributed to
authors of sections.• Revised versions are returned to MW + RD by 15th July.• Distributed to community for input to be received by mid
August.• Comments taken on board and final report made available
to the community at end August.
![Page 16: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/16.jpg)
16
The Responsible Adults
• Group 1: Donia Scott and Cécile Paris• Group 2: Marilyn Walker and Amanda
Stent• Group 3: Albert Gatt• Group 4: Johanna Moore and Alex Koller
![Page 17: 1 Today’s Plan 900am–915am:Sort out questions regarding refund forms 915am–945am:Finalise today’s agenda 945am-1045am:Breakout Groups Session 1 1045am–1115am:Coffee.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cff5503460f949d01a2/html5/thumbnails/17.jpg)
17
Report-Backs
• Desiderata Development• Shared Task #1: Text to Text• Shared Task #2: Referring Expression
Generation• Shared Task #3: Virtual Environments