Socially Intelligent Robots Cynthia Breazeal MIT Media Lab Robotic Life Group Cynthia Breazeal MIT...
-
date post
18-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Socially Intelligent Robots Cynthia Breazeal MIT Media Lab Robotic Life Group Cynthia Breazeal MIT...
Socially Intelligent RobotsSocially Intelligent Robots
Cynthia BreazealMIT Media Lab
Robotic Life Group
Cynthia BreazealMIT Media Lab
Robotic Life Group
Robots turn 85 years oldPosted May 31st 2006 11:38PM by Ryan BlockFiled under: RobotsDear Robots,
We're very sorry. It appears we missed your 85th birthday two days ago -- the anniversary of which is marked by the date Czech writer Karel Capek debuted his play R.U.R. (Rossum's Universal Robots) to its first audience in Prague. Yes, we know the concept of the automaton dates back much further, but we think it's well agreed upon that Capek's play marks the robot's entry into mass consciousness (as well as marking the first use of the word "robot"). No matter, we're just saying happy birthday, robots -- not because we fear you'll one day you'll subsume us in some dystopian nightmare of artificial intelligence gone terribly wrong, but because from Asimov to AIBO, from Roomba to Ri-Man, from QRIO to ASIMO, we just love ya. So happy birthday, happy birthday, happy birthday, robots, and when the day of reckoning comes, please remember: Engadget and its readers are your friends.
All our love,Engadget
Robots have… explored ocean depths,mapped subterranean mines, rescued natural disaster victims,assisted surgeons with operations,driven autonomously across the desert,
And even been to Mars…
Robots have… explored ocean depths,mapped subterranean mines, rescued natural disaster victims,assisted surgeons with operations,driven autonomously across the desert,
And even been to Mars…
What’s Next?What’s Next?
The next big frontier…society at largeThe next big frontier…society at large
Everyday Life with People and Robots
…and its implication for design
Everyday Life with People and Robots
…and its implication for design
People and RobotsPeople and Robots
Robots are not perceived as pure tools or appliances, but often as social actors -- over a wide range of morphologies and behaviors
Robots are not perceived as pure tools or appliances, but often as social actors -- over a wide range of morphologies and behaviors
Robots Evoke Human Social Responses
Robots Evoke Human Social Responses
New Scientist, 2005
“The Kismet Effect”
Newsmaker: My friend, the robot
CNET news.com, May 24, 2006
Newsmaker: My friend, the robot
CNET news.com, May 24, 2006
The PackBots have almost become members of military units, Angle said, recalling an incident when a U.S. soldier begged iRobot to repair his unit's robot, which they had dubbed Scooby Doo. "Please fix Scooby Doo because he saved my life," was the soldier's plea, Angle told the Future in Review conference last week in Coronado, Calif. For many reasons, people bond with robots in a way they don't bond with their lawn mowers, televisions or regular vacuum cleaners. At some point, this could help solve the looming health care problem caused by an enormous generation of aging people. Not only could robots make sure they take their medicine and watch for early warning signs of distress, but they could also provide a companion for lonely people and extend their independence.
Social Robots Socio-emotive Factors
Social Robots Socio-emotive Factors
Interactive Toys
NEC “babysitters” OMRON “pets”
BANDAI “elder toys”Professional ServiceRobots
the socio-emotive and psychologicalaspects of people, in long-term relations
Future applicationsrequire robots to address
“Social as interface”
“Social as entertainment”
“Social as relationship”
HRI, An Emerging DisciplineHRI, An Emerging Discipline
An important goal of Human-Robot Interaction (HRI) is synergy of the human-robot system. Robots bring their own abilities that complement human strengths. It is not about equivalence (replacement), but compatibility with a typical human partner
Lastin
g R
ela
tion
ship
Four Cornerstones of Social Robotics in HRI
Four Cornerstones of Social Robotics in HRI
Team
work
Socia
l Learn
ing
Socia
l Inte
lligen
ceInterdependence
Transparent Communication
Cognitive Compatibility
Perspective Taking
User Studies,Psychology &Social Development
Today’s FocusToday’s Focus
Robots, like humans, should leverage the social and environmental constraints in the real world to foster learning new skills and knowledge from anyone.
Animal training techniques{Stern, Frank, Resner, Virtual Petz, Agents 1998}{Blumberg et al. Integrated learning for interactive characters,
SIGGRAPH 2002}{Kaplan et al., Robot clicker training, RAS 2002}
Reinforcement Learning with humans{Isbell et al. Cobot: a social reinforcement learning agent, UAI
1998}{Evans, Varieties of Learning, AI Game Programming Wisdom,
2002}{Clouse, Utgoff, Teaching a Reinforcement Learner, ICML 1992}
Active Learning, Learning with Queries{Cohn, Ghahramani, Jordan, Active learning with statistical
models, 1995}{Cohn et al., Semi-supervised clustering with user feedback,
2003}
Personalization agents, Adaptive user interfaces
{Lashkari, Metral, Maes, Collaborative Interface Agents, AAAI 1994}
{E. Horovitz et al., The Lumiere project, UAI 1998}
Learning by Demonstration, Programming by Example
{Voyles, Khosla, Programming robotic agents by demonstration, 1998}
{Lieberman, Your Wish is my Command, 2001}{A. Billard, Special Issue of RAS on Robot Programming by
Demonstration, 2006}
Learning by Imitation {S. Schaal review in TICS 1999} {K. Dautenhahn & C. Nehaniv, Imitation in Animals and Artifacts,
2002}
… and many more
Most people don’t have experience with Machine Learning techniques, they have a lifetime of experience with social learning interactions that they bring to the table.
We emphasize the need to consider and design to support the ways that people naturally approach teaching.
And then design algorithms and systems that take better advantage of this
How Do Ordinary People Teach a RL Agent?
How Do Ordinary People Teach a RL Agent?
Experiments inSophie’s KitchenExperiments in
Sophie’s Kitchen
QuickTime™ and aAnimation decompressor
are needed to see this picture.
A “computer game” - players teach a virtual robot to bake a cake, by sending various messages with a mouse interface.
A “computer game” - players teach a virtual robot to bake a cake, by sending various messages with a mouse interface.
Sophie learns via Q-Learning
30 steps~10,000 states2-7 actions/state
Allows us to run many subjects on-line
Experiments inSophie’s KitchenExperiments in
Sophie’s Kitchen
A “computer game” - players teach a virtual robot to bake a cake, by sending various messages with a mouse interface.
A “computer game” - players teach a virtual robot to bake a cake, by sending various messages with a mouse interface.
QuickTime™ and aAnimation decompressor
are needed to see this picture.
An object specific reward is about a particular part of the world
Initial ExperimentInitial ExperimentThomaz & Breazeal RO-MAN 2006
18 people trained Sophie They are given a description of the cake task,
and told they can’t do actions but can help Sophie by sending FEEDBACK messages with the mouse
System logs time of state changes, agent actions, and any human feedback. We analyze games logs to understand people’s teaching behavior
18 people trained Sophie They are given a description of the cake task,
and told they can’t do actions but can help Sophie by sending FEEDBACK messages with the mouse
System logs time of state changes, agent actions, and any human feedback. We analyze games logs to understand people’s teaching behavior
Findings: GuidanceFindings: Guidance
People tried to use the object specific rewards as FUTURE directed guidance.
People tried to use the object specific rewards as FUTURE directed guidance.
QuickTime™ and aAnimation decompressor
are needed to see this picture.
0 20 40 60 80 100%%%%%
Each player’s %Object Rewards about last object
Never About Most Recent Object
Always About Most Recent Object
Many object rewards not about the last object usedMany object rewards not about the last object used
0
2
4
6
8
10
12
14
16
Number of People
Zero rewardsto Empty Bowl
At least 1 rewardto Empty Bowl
Almost everyone gave rewards to the bowl or tray sitting empty on the shelf...a guidance reward.
Almost everyone gave rewards to the bowl or tray sitting empty on the shelf...a guidance reward.
Findings: People Adapt Teaching to their Mental
Model of Sophie
Findings: People Adapt Teaching to their Mental
Model of Sophie
People gave more rewards after realizing their feedback made a difference
Interpreted Sophie’s behavior as being a “staged” learner
Adapted their teaching strategy accordingly
People gave more rewards after realizing their feedback made a difference
Interpreted Sophie’s behavior as being a “staged” learner
Adapted their teaching strategy accordingly
human rewards : agent actions
(Avg)Individual (Avg)IndividualIndividual (Avg)
Initial Experiment
Transparency
Asymmetry
Guidance
}
Using Guidance in Sophie’s Kitchen
Using Guidance in Sophie’s Kitchen
Interactive Q-Learning Algorithm, baseline system}
slight delay to animate act and receive human reward
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
Using Guidance in Sophie’s Kitchen
Using Guidance in Sophie’s Kitchen
QuickTime™ and aAnimation decompressor
are needed to see this picture.
GuidanceExperimentGuidance
ExperimentThomaz & Breazeal, AAAI 2006
Hypothesis: Non-expert teachers can use guidance to improve agent’s performance
27 subjects trained Sophie in two groups:Using feedback only Using both feedback and guidance
Again, system logs game play and logs are analyzed to understand teaching behavior
Hypothesis: Non-expert teachers can use guidance to improve agent’s performance
27 subjects trained Sophie in two groups:Using feedback only Using both feedback and guidance
Again, system logs game play and logs are analyzed to understand teaching behavior
Effects of GuidanceEffects of Guidance
+ >> only
1-tailed T-tests show logs in guidance condition are significantly better than non-guidance 1-tailed T-tests show logs in guidance condition are significantly better than non-guidance
feedback only
guidance + feedback
effect size
Number of Trials 28.5 14.6 49%
Number of Actions 816.4 368 55%
Number of Failures 18.89 11.8 38%
Number Fails before 1st Goal
18.7 11 41%
Number Unique States Visited
124.44 62.7 50%
Initial Experiment
Transparency
Asymmetry
Guidance
}
TransparencyTransparency
How can machine learners be Transparent? How can machine learners be Transparent?
Teachers structure the environment and the task to help a learner succeed. Learners contribute by revealing internal state; helping the teacher maintain a mental model to make guidance more appropriate.
Sophie’s Gaze BehaviorSophie’s Gaze Behavior
Interactive Q-Learning Algorithm modified toincorporate Guidance
Sophie’s Gaze BehaviorSophie’s Gaze Behavior
QuickTime™ and aAnimation decompressor
are needed to see this picture.
TransparencyExperiment
TransparencyExperiment
52 subjects trained Sophie in an online version:
Feedback and guidance, no gaze Feedback and guidance, Sophie gazing
Hypothesis:
Learners can help shape their learning environment by communicating aspects of the internal process -- gaze will improve the human’s guidance instruction
52 subjects trained Sophie in an online version:
Feedback and guidance, no gaze Feedback and guidance, Sophie gazing
Hypothesis:
Learners can help shape their learning environment by communicating aspects of the internal process -- gaze will improve the human’s guidance instruction
Thomaz et al., ICDL 2006
Sophie’s Gaze BehaviorSophie’s Gaze Behavior
Results: Sophie’s gaze significantly improves the guidance received - more when uncertainty high and less when uncertainty is low.
Uncertainty high: 3 or more choices
Uncertainty low: 3 or less
0
10
20
30
40
50
60
70
80
90
uncertainty low uncertainty high
gaze
no-gaze
LessonsLessonsPeople bring their own teaching and learning experience to the task
Social factors of guidance and transparency
Collaborative process between teacher and learner improves performance
Agent can use transparency cues to improve its own learning environment by helping teacher form a better mental model
Adding gaze significantly improves the human’s Guidance
SummarySummary