Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction...
-
Upload
meghan-blankenship -
Category
Documents
-
view
213 -
download
0
Transcript of Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction...
![Page 1: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/1.jpg)
Towards a Theoretical Framework for the Integration of Dialogue Models
into Human-Agent Interaction
John R. LeeAssistive Intelligence Inc.
Andrew B. WilliamsSpelman College
![Page 2: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/2.jpg)
Motivation
• How should an intelligent agent incorporate communication?
• How does communication and behavior integrate within an agent model?
• How can ideas from many different dialogue models and conversation examples by incorporated?
• How can one validate the correctness of an agent conversational model?
![Page 3: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/3.jpg)
Motivation
Negotiation
Persuasion
Formal Argumentation
Informal Argumentation
Integrated Model
Cooperative Planning
Belief Grounding
![Page 4: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/4.jpg)
Dialogue Agent Paradigm
• Embedded dialogue manager– Perception processing
• Embedded behavior model
DIALOGUE CAPABLE AGENT
ActionsPercepts
DialogueManager
Behavioral Model
Conversation
Internal API or Language
![Page 5: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/5.jpg)
Goal
• A unified conversational architecture for intelligent agents and assistants– Representation of communication– Incorporation of communication within
behavior– Incorporating a variety of models and ideas
into a single integrated model– Validation of the conversational model– Independent of dialogue interpreter or agent
![Page 6: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/6.jpg)
Focus
Com
munication
Behavior
Agent Implementation
Bring behavior to communication as much as possible
![Page 7: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/7.jpg)
Focus
Com
munication
Behavior
Agent Implementation
• Separate dialogue interpreter from agent– Parallel development of each– Interchangeable components
![Page 8: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/8.jpg)
INTELLIGENT AGENTHUMAN USER
VOCAL CHORDSAND EAR DRUM
SPEAKER ANDMICROPHONE
PHYSICAL LAYER
(SOUND WAVES)
PHONEMES PHONEMESINDIVIDUAL PHONEMES
SOUND CAPTUREAND GENERATION
MUSCLE AND NERVE INFORMATION
UTTERANCE UTTERANCEUTTERANCES AND SENTENCES
(UH-HUH, “OKAY”, “LET’S DO THAT”)
SPEECH RECOGNITION AND GENERATION
PRONUNCIATION AND VOCABULARY RECOGNITION
SEMANTICS SEMANTICSUTTERANCE UNDERSTANDING
(LANGUAGE INDEPENDENT MEANING)
CONTEXTUALIZING AND REFERENCE RESOLUTION
CONTEXT AND REFERENCE RESOLUTION
PRAGMATICS PRAGMATICSDIALOGUE UNDERSTANDING
(TAKE TURN, CLARIFY, ACKNOWLEDGE)
DIALOGUE AND SPEECH ACT THEORY
INTERACTION ABSTRACTION
TASK MANIPULATION MODEL
LOW-LEVEL INTERACTION
(ADOPT, SELECT, IDENTIFY, EVALUATE)
INTERACTION RECOGNITIONAND APPLICATION
SYNTAX SYNTAXLANGUAGE DEPENDENT ANALYSIS
(CONTEXT INVARIANT MEANING)
SENTENCE FORMATION AND UNDERSTANDING
SENTENCE PARSING AND GENERATION
COMMITMENT, BELIEFS, INTENTIONS KNOWLEDGE
COMMITMENT, BELIEFS, INTENTIONS KNOWLEDGE
INTERACTION MANAGEMENT
TASK ABSTRACTIONTASK MODEL
ABSTRACTION
TASK CONTEXT UNDERSTANDING
(OBJECTIVE, RESOURCE, ACTION)
TASK RECOGNITION AND APPLICATION
HIGH-LEVEL INTERACTION
(NEGOTIATE, EXPLAIN, PERSUADE)
Hum
an Interpreter
![Page 9: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/9.jpg)
The Practical CommunicationLanguage (PCL) Hypothesis
There exists a language between that of a human conversational participant and that of an intelligent agent.
This language is capable of abstracting away the complexity of human language while yet maintaining the practical information of the conversation.
Adding to The Practical Dialogue Hypothesis and
The Domain-independence Hypothesis stated in Allen 2000.
![Page 10: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/10.jpg)
Current Utterance-Based Languages
• Application Programmer Interfaces (API)– Task Management Interface
• Specialized Languages– Artificial Discourse Language– Universal Communication Language (Interlingua)– Parameterized Action Representation
• Discourse and Speech Act Tags• Agent Communication Languages
– Foundation for Intelligent Physical Agents (FIPA-ACL)– Knowledge Query Manipulation Language (KQML)
![Page 11: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/11.jpg)
Searching for the language…
True PCL is ideal and volatileEver expanding definition of ‘practical’
PCL should be abstracted* of
1. Region and dialect aspects of language.
2. Informal, Colloquial, Slang and Idiomatic expressions.
3. Modality (Spoken, Written, Gestural, GUI)
*Translated or Incorporated not discarded.
![Page 12: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/12.jpg)
Approach
• Task Communication Language (TCL)– Messages to/from Dialogue Interpreter
Task ModelTask
Communication Model
Interaction Model
TCL Message
– Set of integrated models
![Page 13: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/13.jpg)
TCL Messages
• Header– Generator: Generated utterance or gesture
– Addressee: Intended Receivers of message
– Receiver: Participants who saw or heard
– Uncertainty in all above fields– Interpretation Stack
• Information obtained at al levels of translation– Used by feedback mechanism for improving interpreters
– Content• Meaning-Action Concept
![Page 14: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/14.jpg)
Conversational Paradigms
Human Agent
Human Observation
Manager / Assistant
Teacher / Student
Coach / Player
Peer / Peer
AgentAgent Communication
Semantic Web
![Page 15: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/15.jpg)
Conversational Paradigms
Single Agent Multiple Agent
Single
HumanCurrent Trend
Simulation and Training
Consumer Products
Multiple
Human
Mediator
Discussion Leader
Team Coordinator
Referee
Semantic Web
Marketplaces
Teamwork
![Page 16: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/16.jpg)
Conversational Paradigms
• Not just endpoint to endpoint
• Multiple segmentations– A conversation between people listening in on
another conversation
![Page 17: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/17.jpg)
Meaning-Action Concept
• Meaning of utterance or gesture• Possible association with action.
• “propose( action: )”• “propose( goal: )”• “reject( goal: )”• “counter-propose( action: )”• “query( justification( action: ) )”
![Page 18: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/18.jpg)
Meaning-Action Concepts (MAC)
• Defined in ontological format– Allows for rollback to known concepts
– Manageable growth of concept space
Proposal( )
Counter-Proposal( )
Commit( )
Commit( confidence:30 )
![Page 19: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/19.jpg)
Task Communication Expression
• First-order logic expression of MAC.– Conjunction: Multiple Meanings– Disjunction: Ambiguity– Expressiveness and complexity
![Page 20: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/20.jpg)
Focus
Com
munication
Behavior
![Page 21: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/21.jpg)
Task Model
• Task Concepts:
• Objectives • Recipes • Actions
• Resources • Situations • States
• Constraints • Beliefs • Intentions
• Metrics • Priorities
![Page 22: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/22.jpg)
Task Model
• Task Operations:
• Adoption • Selection • Deferment
• Abandonment • Release • Identification
• Evaluation • Modification
![Page 23: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/23.jpg)
Task Model
Com
munication
Behavior
Task Model
![Page 24: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/24.jpg)
Task-Communication Model
• Integrate the task concepts and operators– Communication with a dialgue interpreter– Task manipulation of an intelligent agent
• Modeling can be language independent– CFSM, CPetriNet, Inference-Based, BDI...
![Page 25: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/25.jpg)
Task-Communication Model
• Nested task operators
• Lower layers:– Persuasion, inquiry, deliberation, formal
argumentation, informal argumentation, clarification, explanation…
• Higher layers:– Negotiation, cooperative planning, learning
through orders, command and control…
![Page 26: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/26.jpg)
Task-Communication Model
Com
munication
Behavior
Task Model
Task-Communication Model
![Page 27: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/27.jpg)
Task-Communication Model
Trivial Example:
• Communicative acts– TCL Messages
• Behavioral acts– Agent integration
[IN]: Propose( Action A )
Evaluate( Action A )
[OUT]: Reject( Action A )
[OUT]: Accept( Action A )
![Page 28: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/28.jpg)
Integration Model
• Generated automatically through tracing the Task-Communication Model
• Represents incoming and outgoing message sequences and possibilities
![Page 29: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/29.jpg)
Task-Communication Model
[IN]: Propose( Action A )
Evaluate( Action A )
[OUT]: Reject( Action A )
[OUT]: Accept( Action A )
![Page 30: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/30.jpg)
Interaction Model
• Extraction of Input-Output Sequences
[IN]: Propose( Action A ) [OUT]: Accept( Action A )
[OUT]: Reject( Action A )
[OUT]: Refine( Action A )
[OUT]: Clarify( Goal G )
[OUT]: Counter( Action A )
![Page 31: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/31.jpg)
Interaction Model
Com
munication
Behavior
Task Model
Interaction Model
Task-Communication Model
![Page 32: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/32.jpg)
Interaction Model
• Validated with known TCL sequences– If sequence is covered, path validated– If sequence is missing, update model
• Assists in integration of models
• Prove various properties– Turn Taking– Liveness– Sanity checks
![Page 33: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/33.jpg)
Mixed-Initiative Control
• No longer in hands of dialogue interpreter– Also managed by ‘task communication model’
• Task-communication model must– Initiate dialogue sequences– Manage
• turn-taking• context tracking• autonomy
![Page 34: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/34.jpg)
Stratagus
• Open source real-time strategy engine– Multiple data sets for varying games
• Dynamically changing environment
• Real time resource management
![Page 35: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/35.jpg)
![Page 36: Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.](https://reader036.fdocuments.in/reader036/viewer/2022081603/56649f135503460f94c27c29/html5/thumbnails/36.jpg)
Discussion