1© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Applications of distributed dialogue systems:
The KTH Connector
Jens Edlund Anna Hjalmarsson
Aalborg, November 10th, 2005
2© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Why spoken interfaces?
Speech is good because it is• Hands free• Eyes free• Intuitive – already known• Robust – e.g. redundancy, grounding• Flexible• Responsive• Efficient• …
3© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
What do they do?• Purpose
— Problem solving— Information seeking— Transactions— Control— …
• Initiative — System— User— Mixed
• Modality— Multimodal (input and/or output)— Unimodal (input and/or output)
• …
4© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
What else do they do?
Transactions, e.g. ATIS
Web forms (and sales reps)
Call routing, e.g. HMIHY
DTMF (and operators)
Info services, e.g. weather
Web forms, DTMF
Control, e.g. smart rooms
Buttons, remote controls
Spoken interfaces often replace or complement existing automated interfaces, for example:
5© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Alternative interface systems
• Speech is an alternative or substitute• Commonly built to be as good as or
better than the corresponding system• Symmetry often required – what can
be done with the original system should be doable with speech and vice versa
6© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Why spoken interfaces?
• Hands free• Eyes free• Intuitive – already known• Robust – e.g. redundancy,
grounding• Flexible• Responsive• Efficient• …
• Hands free• Eyes free• Intuitive – already known• Robust – e.g. redundancy,
grounding• Flexible• Responsive• Efficient• …
These aspects are often not exploited much in alternative interface systems
7© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
What metaphor to rely on?
• The voice as an input device?—”You may use your voice to order”
From a travel booking instruction in Swedish
—”It didn’t give me any alternatives”From a post interview with call routing user
• The computer as a human?—Problem: Turing test
8© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Summary
• Speech can be used successfully as an alternative or complementary interface to other interfaces, particularly when hands and/or eyes are occupied, disabled, or otherwise impratcical to use
• The advantages of speech promised by analogies to human-human communication may not be fully exploited in such domains
9© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
The KTH Connector
• Background• Domain• System
10© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Background: CHIL
• CHIL: Computers in the Human Interaction Loop (EU funded, IP506909)
• The dialogue system as an unobtrusive conversational partner in a group of humans
11© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
A telephony based secretary
Wide range of complexity• From answer phone…
12© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
A telephony based secretary• …to meeting assistant
13© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Dialogue setup
Multimodal, multiparty,
system barge-in
Multipartytelephony
14© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
System highlights
Interpretation Generation
Skype IP telephony
User
System
ASR /nailon/
PICKERING
GALATEAKnowledgebase
DialogueManager
TextGeneration
TTS
1a) words
2) concepts
3) discourseresolved concepts 4) discourse
resolved concepts
5) worldresolved concepts
1b) potentialend-points
6) concepts
7a) text
7b) concepts
VAC VAC
0) Voice input 8) Synthesised speech output
Interpretation Generation
Skype IP telephony
User
System
ASR /nailon/
PICKERING
GALATEAKnowledgebase
DialogueManager
TextGeneration
TTS
1a) words
2) concepts
3) discourseresolved concepts 4) discourse
resolved concepts
5) worldresolved concepts
1b) potentialend-points
6) concepts
7a) text
7b) concepts
VAC VAC
0) Voice input 8) Synthesised speech output
System output is included in
discourse model(incrementally)
Prosody enhanced
endpointing
System may dial users
15© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Research highlight: Responsiveness
• When should we respond?
• Turn yielding & turn holding cues: —Prosody—Gaze —…
16© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Research highlight: Incrementality
• When can we respond?— After a ”long enough” silence?— At some semantic or syntactic completeness?— After any ”word”?— Anytime?
• What has actually happened?— System and user barge-in— Keep track of what we say, as well as what
the user says
17© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Research highlight: Unobtrusiveness
• With what should we respond?—Long prompts can be annoying—Short ones may be insufficient
• How?—Efficiently or politely?—Speech, gesture, other?
• Do we have to ”take turn”? —Backchannels—Grounding
18© 2005 CHIL KTH ASIDE 2005, Aalborg, 2005-11-10
Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson
Thank you for your attention.
Top Related