HUMAN CENTERED ROBOTICS ISE/EE-4.60 - Lecture 2: Fundamentals Dr Yiannis Demiris Department of Electrical and Electronic Engineering Imperial College London y.demiris@imperial.ac.uk To make apple pie from scratch, one must first create the universe… Carl Sagan

1

Fundamentals of Human Centered Robotics (1)

Interacting agents require perception of others & context

How is the robot & context perceived

by the human participant(s)

How is the human & context perceived by the

robot participant(s)

L 3 & 4: Perception of user state through vision L 5 & 6: Perception of user state through other modalities (speech, touch) L 7 & 8: Human action understanding

L 9 & 10: Influences of robot morphology and robot actions on human perception and behaviour

2

Variables that can be manipulated

• Robot morphology • Robot movement repertoire •  Interaction modalities (vision, speech, touch…) • Robot’s role and response selection mechanisms • Context • Robot sensors and algorithms

3

Appearance and movement (1) Anthropomorphism

Hiroshi Ishiguro’s anthropomorphic robots

4

Robot Morphology and movement timing

5

Fundamentals of Human Centered Robotics (2)

Interacting agents need to factor others in their planning

How can the robot(s) help

me? How can I help the human(s)?

L 11 & 12: Interaction and joint action

6

Fundamentals of Human Centered Robotics (3)

Interacting agents need to improve interaction over time

Human and robot can learn to work together better

Humans can learn

robot abilities

Robots can learn user

preferences

7

Case study (1): ALIZ-E Adaptive Strategies for Sustainable Long-Term Social Interaction •  Interaction setting: Children in paediatric wards, admitted

for diabetes treatment, with typical duration of 5-10 days. • Problem to be addressed: high levels of anxiety for

children, due to unknown environments and constantly changing hospital personnel.

• Proposed solution: create an interactive robotic friend that will be with the child for the duration of its treatment.

• Challenges: •  Recognition of child’s state non-intrusively; body, face, speech, emotions •  Selection of appropriate robot morphology, appearance and movement

patterns to suit children •  Design appropriate interaction scenarios in consultation with doctors •  Sustainable long term social interaction

•  Memory, user models, dialogue management

8

9

Case study (1): ALIZ-E Adaptive Strategies for Sustainable Long-Term Social Interaction

• Robot morphology •  NAO humanoid robot

•  Advantages: child-friendly appearance and size, easy to program •  Disadvantages: limited onboard computational power and sensors

10

Case study (1): ALIZ-E Adaptive Strategies for Sustainable Long-Term Social Interaction

•  Engagement strategy: games! •  “Postural imitation” game, Simon game, and Dance game •  Quiz game •  (Varying) roles of robot to suit child’s varying needs:

•  educator, peer (co-trainee), listener…

11

Case study (1): ALIZ-E Adaptive Strategies for Sustainable Long-Term Social Interaction

• Significant difficulties in perceiving the state of the child in realistic, potentially crowded hospital environments

• Significant difficulties in deciding how to act to support a child’s behaviour

12

Case study(2): Children Robotic wheelchairs Supporting the development of young disabled children

•  Interaction setting: Disabled children’s homes • Problem to be addressed: children with motor disabilities

do not develop socio-cognitive skills due to the lack of mobility; lack of development forbids wheelchair provision.

• Proposed solution: create a safe, intelligent robotic wheelchair that will assist the child in mobility tasks.

• Challenges: •  Recognition of child’s intention •  Sustainable long term social interaction

•  Model of developing user skills •  Determination of stage-appropriate level of assistance. •  Consideration of long-term issues.

•  Evaluation

13

14

§ 640,000 Wheelchair users in the UK (Audit Commission, 2000)

§ 70% over the age of 60, 9% under 19 years § Particularly for children in development-critical ages

(under 5), deprivation of movement can lead to “learned helplessness” (Nisbet, 1996)

§ 40% of NHS wheelchair services reported problems in child learning initial control (Nicholson 2002).

The need for Intelligent Assistive Wheelchairs Overview

Case study(2): Children Robotic wheelchairs Supporting the development of young disabled children

15

Case study(2): Children Robotic wheelchairs Supporting the development of young disabled children

16

Case study(2): Children Robotic wheelchairs Supporting the development of young disabled children

17

Case study(3): I-CUB Imitation Learning •  Interaction setting: Tabletop environment • Problem to be addressed: complex humanoid robots are

difficult to program manually, and inaccessible to non-technical users

• Proposed solution: create algorithms for learning by imitation: user demonstrates task and robot learning by observing and imitating.

• Challenges: •  Recognition of human action and intention •  Mapping between dissimilar bodies •  Adaptation of recognized action to novel environments

18

One-Shot learning of inverse models by Imitation

n  Efficient Template-based Path Imitation by Invariant Feature Mapping

n  Extract action (movement + environment) templates by demonstration

n  Morph optimal template for new environment based on a combination of minimum distortion (local features level) and minimum energy (task level) strategies

n  Yan Wu and Yiannis Demiris, “Towards One Shot Learning by imitation for Humanoid Robots”, IEEE ICRA 2010 n  Yan Wu and Yiannis Demiris, “Efficient template-based path imitation by invariant feature mapping”, IEEE ROBIO 2009.

19

Learning by imitation

Yan Wu and Yiannis Demiris, “Towards One Shot Learning by imitation for Humanoid Robots”, IEEE ICRA 2010

20

Learning by imitation using syntactic approaches

21

Learning by imitation – EFAA project •  Incorporate these methods into a multimodal tabletop environment

•  Instrument the environment to aid perception & awareness

22

Multimodal Interaction

23