Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.
-
Upload
leslie-warner -
Category
Documents
-
view
215 -
download
0
Transcript of Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.
![Page 1: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/1.jpg)
Interaction Challenges for Intelligent Assistants
Jim Blythe
USC Information Sciences Institute
![Page 2: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/2.jpg)
How to build “truly useful assistants”? Personalized, Learn, Engender trust, Become partners
Organizer: Neil Yorke-Smith
Committee: Pauline Berry, Timothy Bickmore, Mihai Boicu, Justine Cassell, Ed Chi, Mike Cox, John Gersh, Jihie Kim, Jay Modi, Donald Patterson, Debra Schreckenghost, Richard Simpson, Stephen Smith, Sashank Varma
28 accepted papers
![Page 3: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/3.jpg)
Topics
• Trust
• When to assist?
• Learning
• Modeling
• Desktop assistants
• Panel with symp. on multidisciplinary collaboration for socially assistive robots
• Panel with intentions in intelligent systems
![Page 4: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/4.jpg)
How To Make Users Happy
• And avoid annoying users
- Brad Myers’ invited talk
![Page 5: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/5.jpg)
User Happiness?
Hu = f (Performance)
![Page 6: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/6.jpg)
User Happiness?
Hu = f (Performance, Trust)Hu = f (Performance, Trust)
![Page 7: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/7.jpg)
User Happiness!Hu = f (EAssistant ENegative EPositive EValue EUser ECorrected
EBy-hand ECost EAvoided EApparentness ECorrect-difficulty
ESensible WQuality WCommitment TBy-hand TBy-Hand-start-
up TBy-Hand-per-unit TAssistant TTraining-start-up TAssistant-
per-unit TInteraction-per-unit TMonitoring TCorrecting
TResponsiveness TSystem-Training TUser-training
TAverage-for-each-correction AError-rate Nunits PPleasantness
UPerceive UWhy UProvenance UPredictability IAssistant-
interfere IScreen-space ICognitive IAppropriate-Time CAutonomy
CCorrecting SSensible-Actions SUser-models SLearning
RSocial-Presence DHand VImportance)
Hu = f (EAssistant ENegative EPositive EValue EUser ECorrected
EBy-hand ECost EAvoided EApparentness ECorrect-difficulty
ESensible WQuality WCommitment TBy-hand TBy-Hand-start-
up TBy-Hand-per-unit TAssistant TTraining-start-up TAssistant-
per-unit TInteraction-per-unit TMonitoring TCorrecting
TResponsiveness TSystem-Training TUser-training
TAverage-for-each-correction AError-rate Nunits PPleasantness
UPerceive UWhy UProvenance UPredictability IAssistant-
interfere IScreen-space ICognitive IAppropriate-Time CAutonomy
CCorrecting SSensible-Actions SUser-models SLearning
RSocial-Presence DHand VImportance)
![Page 8: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/8.jpg)
A Tale of Two Associates• Pilot’s Associate (1985-
1991)– Single Pilot– Direct pilot interaction with
associate meant added workload
– Design philosophy minimized direct pilot interaction with associate
– Moderate user acceptance
The Pilot is The Pilot is ALWAYS in ALWAYS in
charge.charge.
The Effort The Effort required of the required of the pilot to control pilot to control the associate the associate
must be less than must be less than the effort saved the effort saved by the associateby the associate
• Rotorcraft Pilot’s Associate (1994-1999)– Two Pilots
– 1/3 of human activity is crew coordination
– Design philosophy included some direct pilot interaction with associate
– Improved User Acceptance
![Page 9: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/9.jpg)
Why and how to model multi-modal interaction for a mobile robot companion
Shuyin Li & Britta Wrede Best paper
• Tested policies with users interacting with a robot
• Communicate pre-interaction attention
• Need to make social remarks with non-verbal methods (because people tend to reply in kind)
Biron and Barthoc
![Page 10: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/10.jpg)
Interaction Challenges for Agents with Common Sense
Invited talk from Henry Lieberman
• We now have several sources of common sense knowledge, e.g. Cyc, Open Mind, ThoughtTreasure
• Some strategies and examples of exploiting common sense to build better interfaces
![Page 11: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/11.jpg)
Strategies for using common sense in interfaces
• Find underconstrained situations
• Find situations where every little helps
• Know a little about everything, but not too much about anything
• Make better mistakes! Not just ‘right’ and ‘wrong’, being reasonable is better– Plausible mistakes can increase trust
• Set user expectations
![Page 12: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/12.jpg)
Examples of interfaces using common sense
• ARIA photo agent: more powerful matching of tags using common sense
• Predictive typing:
“I’m having landlord problems because my roommate was late with my r..”
• BEAM
(Gil & Chklovski)
![Page 13: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/13.jpg)
Trust
• Openness and understanding more important as systems become more complex.
• Methods to improve understanding: explanations [McGuinness et al.]
• HTN metamodel [Wallace]• Patterson: would I trust a fork? a bridge? a
space shuttle?– predictability, understandability, similarity, liability,
social/emotional
![Page 14: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/14.jpg)
Learning (and Trust)
• Adaptive (Learning) vs Adaptable (Instructed by user) – important for believability and trust
![Page 15: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/15.jpg)
Supporting interaction in Robocare intelligent assistant agent
Endowed with human like I/O channels by engineering state of the art components
•Face: Lucia (Piero Cosi, ISTC, Pd)
•Voice: Sonic (Univ.Colorado)
•Simple Interaction Manager
The Motion Skills
The Interaction Skills
Robust continuous behavior at home with person
Use of multiagent technologyCesta et al. Best application paper
![Page 16: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/16.jpg)
Multiple Intelligent Systems
![Page 17: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/17.jpg)
Supporting interaction in Robocare intelligent assistant agent
Integrates multiple systems to produce a socially acceptable robotic care assistant
• Interesting DCOP solution to allow multiple systems and guarantee coherent behaviour
• System follows a STN to notice deviations from expected behaviour
![Page 18: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/18.jpg)
• Experiments in face/no-face in RoboCare
• People prefer no-face – “less artificial”, “more integrated in the
domestic environment”
![Page 19: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/19.jpg)
Desktop assistants
• Many papers on desktop assistants– 6 from the Calo project
PeXA architecture
![Page 20: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/20.jpg)
Towel todo manager
• Towel [Conley et al]: taking an IM approach to give access to tasks
Inspired by Diamond Help [Rich et al. 06]
![Page 21: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/21.jpg)
Did Ken sacrifice himself to User Testing?
• Registered to give talk at AAAI Spring symposium
![Page 22: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/22.jpg)
Should Ken have worked on meeting scheduling?
• Registered to give talk at AAAI Spring symposium
• Booked another trip in same week
![Page 23: Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.](https://reader036.fdocuments.in/reader036/viewer/2022081518/55144e86550346284e8b4ffa/html5/thumbnails/23.jpg)
Should Ken have worked on meeting scheduling?
• Registered to give talk at AAAI Spring symposium
• Booked trip to Hawaii in same week
• Ultimate in user testing? You decide..