Applications of distributed dialogue systems: The KTH Connector
description
Transcript of Applications of distributed dialogue systems: The KTH Connector
![Page 1: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/1.jpg)
1© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Applications of distributed dialogue systems:
The KTH Connector
Jens Edlund Anna Hjalmarsson
Aalborg, November 10th, 2005
![Page 2: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/2.jpg)
2© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Why spoken interfaces?Speech is good because it is• Hands free• Eyes free• Intuitive – already known• Robust – e.g. redundancy, grounding• Flexible• Responsive• Efficient• …
![Page 3: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/3.jpg)
3© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
What do they do?• Purpose
— Problem solving— Information seeking— Transactions— Control— …
• Initiative — System— User— Mixed
• Modality— Multimodal (input and/or output)— Unimodal (input and/or output)
• …
![Page 4: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/4.jpg)
4© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
What else do they do?
Transactions, e.g. ATIS
Web forms (and sales reps)
Call routing, e.g. HMIHY
DTMF (and operators)
Info services, e.g. weather
Web forms, DTMF
Control, e.g. smart rooms
Buttons, remote controls
Spoken interfaces often replace or complement existing automated interfaces, for example:
![Page 5: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/5.jpg)
5© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Alternative interface systems
• Speech is an alternative or substitute• Commonly built to be as good as or
better than the corresponding system• Symmetry often required – what can
be done with the original system should be doable with speech and vice versa
![Page 6: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/6.jpg)
6© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Why spoken interfaces?• Hands free• Eyes free• Intuitive – already known• Robust – e.g. redundancy,
grounding• Flexible• Responsive• Efficient• …
• Hands free• Eyes free• Intuitive – already known• Robust – e.g. redundancy,
grounding• Flexible• Responsive• Efficient• …
These aspects are often not exploited much in alternative interface systems
![Page 7: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/7.jpg)
7© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
What metaphor to rely on?• The voice as an input device?
—”You may use your voice to order”From a travel booking instruction in Swedish
—”It didn’t give me any alternatives”From a post interview with call routing user
• The computer as a human?—Problem: Turing test
![Page 8: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/8.jpg)
8© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Summary• Speech can be used successfully as
an alternative or complementary interface to other interfaces, particularly when hands and/or eyes are occupied, disabled, or otherwise impratcical to use
• The advantages of speech promised by analogies to human-human communication may not be fully exploited in such domains
![Page 9: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/9.jpg)
9© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
The KTH Connector
• Background• Domain• System
![Page 10: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/10.jpg)
10© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Background: CHIL
• CHIL: Computers in the Human Interaction Loop (EU funded, IP506909)
• The dialogue system as an unobtrusive conversational partner in a group of humans
![Page 11: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/11.jpg)
11© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
A telephony based secretaryWide range of complexity• From answer phone…
![Page 12: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/12.jpg)
12© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
A telephony based secretary• …to meeting assistant
![Page 13: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/13.jpg)
13© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Dialogue setup
Multimodal, multiparty,
system barge-in
Multipartytelephony
![Page 14: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/14.jpg)
14© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
System highlights
Interpretation Generation
Skype IP telephony
User
System
ASR /nailon/
PICKERING
GALATEA Knowledgebase
DialogueManager
TextGeneration
TTS
1a) words
2) concepts
3) discourseresolved concepts 4) discourse
resolved concepts
5) worldresolved concepts
1b) potentialend-points
6) concepts
7a) text
7b) concepts
VAC VAC0) Voice input 8) Synthesised speech output
Interpretation Generation
Skype IP telephony
User
System
ASR /nailon/
PICKERING
GALATEA Knowledgebase
DialogueManager
TextGeneration
TTS
1a) words
2) concepts
3) discourseresolved concepts 4) discourse
resolved concepts
5) worldresolved concepts
1b) potentialend-points
6) concepts
7a) text
7b) concepts
VAC VAC0) Voice input 8) Synthesised speech output
System output is included in
discourse model(incrementally)
Prosody enhanced
endpointing
System may dial users
![Page 15: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/15.jpg)
15© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Research highlight: Responsiveness
• When should we respond?
• Turn yielding & turn holding cues: —Prosody—Gaze —…
![Page 16: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/16.jpg)
16© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Research highlight: Incrementality• When can we respond?
— After a ”long enough” silence?— At some semantic or syntactic completeness?— After any ”word”?— Anytime?
• What has actually happened?— System and user barge-in— Keep track of what we say, as well as what
the user says
![Page 17: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/17.jpg)
17© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Research highlight: Unobtrusiveness
• With what should we respond?—Long prompts can be annoying—Short ones may be insufficient
• How?—Efficiently or politely?—Speech, gesture, other?
• Do we have to ”take turn”? —Backchannels—Grounding
![Page 18: Applications of distributed dialogue systems: The KTH Connector](https://reader036.fdocuments.in/reader036/viewer/2022062810/56815c02550346895dc9ea3c/html5/thumbnails/18.jpg)
18© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Thank you for your attention.