error handling – Higgins / Galatea
-
Upload
abra-weaver -
Category
Documents
-
view
35 -
download
1
description
Transcript of error handling – Higgins / Galatea
![Page 1: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/1.jpg)
1
error handling – Higgins / Galatea
Dialogs on Dialogs Group
July 2005
![Page 2: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/2.jpg)
2
work by …
Gabriel Skantzeph.d. studentKTH, Stockholm
“I am doing research on spoken dialogue systems. More specifically, I am interested in studying miscommunication and error handling, but also in the representation and modelling of utterances and dialogue, as well as conducting experiments with users.“
and co-authors: J. Edlund, D. House, R. Carlson
![Page 3: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/3.jpg)
3
3 papers
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
![Page 4: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/4.jpg)
4
1st paper
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
![Page 5: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/5.jpg)
5
Higgins
practical goal of Higgins project build a collaborative dialog system in which error handling
ideas can be tested empirically
error handling issues, plus incremental dialogue processing on-line prosodic feature extraction robust interpretation flexible generation and output
![Page 6: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/6.jpg)
6
domain
pedestrian city navigation and guiding user gives system a destination system guides user by giving verbal instructions
complex large variety of error types semantic structures can be quite complex reference resolution
domain can be extended even further
![Page 7: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/7.jpg)
7
architecture
follow-up from Adapt everything is XML
domain objects utterance semantics discourse model database content system output (before surface) 3D city model
![Page 8: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/8.jpg)
8
research issues
early detection and correction late detection incrementality error recovery
![Page 9: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/9.jpg)
9
early detection and correction
KTH LVCSR – output likely to contain errors
robust interpretation Pickering: some syntactic analysis is needed
e.g. relations between objects but handles insertions and non-agreement phrases
humans - good at early detection (woz)
![Page 10: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/10.jpg)
10
late detection and correction
discourse modeller (GALATEA) joins several results from Pickering into a discourse model adds grounding information can be manipulated later
remove concepts which turn out not to be grounded
![Page 11: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/11.jpg)
11
incrementality
end-pointers cause trouble even more so in this domain
better:
![Page 12: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/12.jpg)
12
incrementality [2]
all components support incremental processing
several issues when to barge in? (semantic content and prosody) longer-than-utterance units: interpreter or dialog manager? rapid and unobtrusive feedback: challenge for synthesis
![Page 13: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/13.jpg)
13
error recovery
signaling non-understandings decreased experience of task success slower recovery
ask other task-related question
![Page 14: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/14.jpg)
14
2nd paper
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
![Page 15: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/15.jpg)
15
GALATEA
a discourse modeller for conversational spoken dialog systems builds a discourse model (what has been said during the
discourse)
resolution of ellipses & anaphora tracks the grounding status
who said what when (plus confidence information) can be used for concept-level error handling
![Page 16: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/16.jpg)
16
should do grounding at concept level
explicit and implicit verification on whole utterance can be tedious and unnatural
45% of clarifications in BNC are fragmentary / elliptical
![Page 17: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/17.jpg)
17
should do grounding at concept level
Traum (1994) – utterance level computational model of grounding
Larsson (2002) – issue-level computational model of grounding in Issue-Based DM
Rieser (2004), Schlangen (2004): systems capable of fragmentary clarification requests, but models do not handle user reactions
systems should keep grounding information at the concept level
like RavenClaw?
![Page 18: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/18.jpg)
18
semantic representation
rooted unordered trees of semantic concepts nodes: attr-value pairs, objects, relations, properties
![Page 19: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/19.jpg)
19
semantic representation
enhanced with “meta”-information confidence communicative acts info is new / given
![Page 20: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/20.jpg)
20
ellipsis resolution
transforms ellipsis into full propositions
rule based ~10 rules domain-specific
![Page 21: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/21.jpg)
21
anaphora resolution
keeps a list of entities (talked about) assigns ids
when given entities are added to the discourse, look up the antecedent
if found, unification (and move to the top of the entity list)
unification also allows entities to be referred to in new ways
how does this fare and compare?
![Page 22: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/22.jpg)
22
grounding status
who added the concept? in which turn? how confident?
may be used by the action manager for instance remove all items with high grounding status
when referring to an entity
![Page 23: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/23.jpg)
23
updating grounding status
![Page 24: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/24.jpg)
24
late error detection
discover inconsistencies in discourse model
look at grounding status to see where error may be concept can be removed
![Page 25: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/25.jpg)
25
future
methods for automatic tuning of strategy selection
extend to track confidence and grounding status at different levels
evaluate how people respond to incorrect
confirmations, and how can that information be used to update grounding status
error recovery after non-understandings other domains
![Page 26: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/26.jpg)
26
3rd paper
HigginsHiggins – a spoken dialogue system for investigating error handling techniques,
Edlund, Skantze, Carlson [2004]
GalateaGALATEA: A Discourse Modeller Supporting Concept-Level Error Handling in
Spoken Dialog Systems, Skantze [2005]
Prosody & ClarificationsThe Effects of Prosodic Features on the Interpretation of Clarification Ellipses, Edlund, House, Skantze [2004]
![Page 27: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/27.jpg)
27
prosody in clarifications
effects of prosodic features on interpretation of elliptical clarifications U: Further ahead on the right I see a red building… S: Red (?)
vary prosodic features study impact on user’s understanding
of the system’s intention
![Page 28: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/28.jpg)
28
motivation
long (whole utterance) confirmations are not good tedious, unnatural BNC corpus: 45% of clarifications are elliptical
short confirmations make dialog more efficient by focusing on the actual
problematic fragments however
interpretation depends on context and prosody
![Page 29: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/29.jpg)
29
3 readings
U: Further ahead on the right I see a red building… S: Red (?)
Ok, red [all positive]
Do you really mean red? What do you mean by red? [positive perception, negative understanding]
Did you say red? [positive contact, negative perception]
![Page 30: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/30.jpg)
30
stimuli
3 test words [red, blue, yellow] di-phone voice (MBROLA) manipulated
peak position [mid, early, late / 100ms] peak height [130Hz / 160 Hz] vowel duration [normal, long / +100ms]
![Page 31: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/31.jpg)
31
subjects + design
8 speakers: 2f / 6m, 2nn / 6n introduced to Higgins listen to all 42 (only once); random
order 3 options
Okay, X Did you really mean X? Did you say X?
![Page 32: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/32.jpg)
32
results
no effects for color, subject, duration
significant effects for peak position, peak height, & their interaction
![Page 33: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/33.jpg)
33
results
Statement: early, low peak Question: late, high peak Clear division between “did you mean” and “did you say”
![Page 34: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/34.jpg)
34
food for thought
how about English? red red? red!? how many ways can you say it?
![Page 35: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/35.jpg)
35
conclusion
strong relationship between intonation and meaning
statement: early, low peak question: late, high peak clear division between “did you mean”
and “did you say”
![Page 36: error handling – Higgins / Galatea](https://reader035.fdocuments.in/reader035/viewer/2022062321/568131c3550346895d982cb4/html5/thumbnails/36.jpg)
36
the end