Cognitive Development and the iCub

Cognitive Developmentand

The iCub Humanoid Robot

Second EUCogII Members Conference"Development of Cognition in Artificial Agents“

Zürich29 January 2010

Giulio Sandini, IIT & U. GenoaGiorgio Metta, IIT & U. GenoaLorenzo Natale, IIT & U. GenoaFrancesco Nori, IIT & U. GenoaPaul Fitzpatrick, IIT & U. GenoaFrancesco Orabona, IIT & U. GenoaMatteo Brunettini, IIT & U. GenoaAlessandro Scalzo, IITMarco Maggiali, IITMarco Randazzo IITRoberto Puddu, IITGabriele Tabbita, IITWalter Fancellu, IITBruno Bonino, IITFabrizio Larosa, IITClaudio Lorini, IITLuciano Pittera, IITDavide Dellepiane, IITMattia Salvi, IITLuca Rivano, IITRavinder Dahiya, IITAlberto Parmiggiani, IITMatteo Fumagalli, IITAlexander Schmitz, IITDiego Torazza, IITNikos Tsagarakis, IIT & U. SheffieldDarwin Caldwell, IIT & U. SheffieldFrancesco Becchi, TeleRobotPaolo Pino, TeleRobotGiulio Maggiolo, TeleRobotGabriele Careddu TeleRobotJohn Gray, U. SheffieldDavid Vernon, U. Genoa

Rolf Pfeifer, U. ZurichHarold Martinez, U. ZurichGabriel Gomez, U. ZurichAlexandre Schmitz, U. ZurichYvonne Gustain, U. ZurichJonas Ruesch, U. Zurich Kerstin Dautenhahn, U. HertfordshireChrystopher L. Nehaniv, U. HertfordshireHatice Kose-Bagci, U. HertfordshireFrank Broz, U. HertfordshireNaeem Assif Mirza, U. HertfordshireDorothée François, U. HertfordshireLars Olsson, U. HertfordshireQiming Shen, U. HertfordshireCecilia Laschi, SSSAPaolo Dario, SSSAFernando Gamarra, SSSADavide Zambrano, SSSAEgidio Falotico, SSSAMaria Chiara Carrozza, SSSAGiovanni Stellin, SSSAGiovanni Cappiello, SSSAAude Billard, EPFLAuke Ijspeert, EPFLSarah Degallier, EPFLLudovic Righetti, EPFLS. Gay, EPFL

José Santos-Victor, ISTAlexandre Bernardino, IST Ricardo Beira, ISTBruno Damas, STJonas Hornstein, , ISTLuís Vargas, ISTRicardo Nunes, ISTHugo Alves, ISTNuno Conraria, ISTJulio Gomes, IST Matteo Tajana, IST Giovanni Saponnaro, IST Christian Wressengger, IST Dario Figueira, IST Rodrugo Ventura, IST Miguel Praça, IST Jonas Ruesch, IST Luís Montesano, IST Manuel Lopes, IST Luciano Fadiga, U. FerraraLaila Craighero, U. FerraraAndrey Olyniyck, U. FerraraLivio Finos, U. FerraraGiovanni Ottoboni, U. Ferrara Claes von Hofsten, U. UppsalaKerstin Rosander, U. UppsalaOlga Kochukova, U. UppsalaHelena Gronqvist, U. Uppsala

U. of Genoa/IIT Scuola S. Anna U. of Zurich U. of Uppsala U. of Ferrara

U. Hertfordshire IST - Lisbon U. Sheffield/IIT EPFL Telerobot S.r.l

Giulio SandiniGiorgio Metta

Cecilia LaschiPaolo Dario

Rolf Pfeifer Claes von Hofsten Luciano Fadiga

Darwin CaldwellNikos Tsagarakis

José Santos-VictorAlexandre Bernardino

Aude BillardAuke Ijspeert

Francesco BecchiKerstin DautenhahnChrystopher L. Nehaniv

Head Design (IST)

Face/Cover Design (IST)

Facial Expressions (IST)

Smooth Pursuit(U. Uppsala, IST, SSSA)

4‐week‐old

9‐week‐old

Perception and anticipation of the upcoming motion

“Uppsala drum” – where we can rotate the base and/or the target

Manuel Lopes, Alexandre Bernardino, José Santos-Victor, Claes von Hofsten and Kerstin Rosander. Biomimetic Eye-Neck Coordination. IEEE -International Conference on Development and Learning, Shanghai, China, 2009.

Smooth Pursuit(U. Uppsala, IST)

Predictive tracking with temporarily-occluded objects

(U. Uppsala, U. Zürich, SSSA)

4-month-old children move gaze ahead of time to the place where they expect the object to appear

9‐week‐old lag= 740 ms 17‐week‐old lag = 15 ms

Smooth Pursuit“Infant” Gaze (U. Uppsala & IST)

Attention

• Posner task (IST & U. Ferrara)

Attention

• Infant Gaze (U. Uppsala)

Multi-Modal Bottom-Up Attention System (IST & UZH)

Salience Egocentric

flicker faces

skin

Multimodal Saliency-Based Bottom-Up Attention A Framework for the Humanoid Robot iCub, Jonas Ruesch, et al. ICRA 2008.

Attention

• Top-down (IST)• Representation and detection familiar objects• Learning triggered by depth (proximity based) segmentation

From Pixels to Objects: Enabling a spatial model for humanoid social robots. Dario Figueira, et al. ICRA’09

Attention

Attention

(IST & UZH)

Reaching and Grasping

(U. Genoa, IIT, IST)

Crawling to sitting

Crawling

Kinematic studies of crawling children as they crawled, went from sitting to crawling, and crawling to sitting (U. Uppsala, EPFL)

Crawling with one hand occupied

Crawling


Crawling


(U. Uppsala, EPFL)

Wiki

Software

Hardware

Drawings

iCub production

15 iCubs completed and 5 more in production

Mind as Motion, Port & Van Gelder

Functionalist & Dualist

Mechanisms are independentof the instantiation

Emergent, Embodied, & Enactive

Embodiment plays a constitutive role in the process of cognition

GOFAI Cognitive Systems

Breaking the ‘here-and-now barrier’

Bond of UnionM. C. Escher, 1956

z

Breaking the ‘prior knowledge barrier’


z


The fire-hose of experience

Cognition: guide actions

– Missing information– Uncertain information– LATE information

• Adapt (make sense of the world)• Anticipate (predict what might happen)

Cognitive systems

– Anticipate– Assimilate– Adapt

– Predict future events when selecting actions

– Learn from what actually happens– Modify subsequent predictions– Autonomously

Learn & develop

• What makes an action the right one to choose?

• What type of behaviour does cognition enable?

• What motivates cognition?

• How is perception guided?

• How are actions selected?

• What makes cognition possible?

• Cognitive skills can improve, but what do you need to get started?

• What drives the developmental process?

Embodiment

Meaning(inter-agent epistemology)

Development

Meaning emerges through shared consensual experience mediated by interaction



OrganismicEmbodiment

OrganismoidEmbodiment

Physical Embodiment

Structural Coupling

Historical Embodiment

From: T. Ziemke, 2003

OrganismicEmbodimentOrganismicEmbodiment

OrganismoidEmbodimentOrganismoidEmbodiment

Physical EmbodimentPhysical Embodiment

Structural CouplingStructural Coupling

Historical EmbodimentHistorical Embodiment

From: T. Ziemke, 2003

Types of EmbodimentWhy Humanoid Robotics is Special

Shared epistemology ⇒compatible embodiment

The problem of disparate embodiment & interaction histories

Progressive ontogenetic acquisition of anticipatory capabilities

– Cognition cannot short-circuit ontogeny

– Necessarily the product of a process of embodied development

– Initially dealing with immediate events

– Increasingly acquiring a predictive capability

Cognition and perception are functionally-dependent on the richness of the action interface

t

t

Development

Luc Steels:

1. AI through design2. AI through statistical machine learning3. Self-generated AI

(AI by orchestrating the processes that generate it)

Self-Generated AI … HOW?

Phylogeny(Cognitive Architecture)

Ontogenesis(Learning & Development + Motivations)

Co-Determination / Structural Coupling

BUT … simple coupling between sensor and motor surfaces

Perturbation is only effected by the environment

[Note: this ideogram and similar ones to follow were introduced in Maturana and Varela 1987]

Cognitive system: operationally-closed system with a nervous system

Nervous system facilitates a highly-plastic mapping between sensor and motor surfaces

Perturbation by both environment and system (of receptors & NS)

“Development is the result of a process with two foci, one in the central nervous system and one in the subject’s dynamic interactions with the environment”

Claes von Hofsten

t

tAnticipation / Planning / Deliberation / Prediction

INTERACTION

A shared activity in which the actions of each agentInfluence the actions of the other agents in the same interactionResulting in a mutually-constructed pattern of shared behaviour[Ogden et al.]

COGNITION & SENSE-MAKING

Is a process whereby the issues that are important for the continued existence of the cognitive entityare brought forth: co-determined by the entity as it interactswith the environment in which it is embedded

PERCEPTION, ACTION, and COGNITION

form a single process of self-organization in the specific context of environmental perturbations of the system

THE SPACE OF PERCEPTUAL POSSIBILITIES

Is predicated not on an objective environment, but on the space of possible actionsthat the system can engage in whilst still maintaining the consistency of its coupling with the environment

EmergentSystems

Self-Organization

Co-determination

Co-development

Phylogeny Ontogeny

Embodiment

Humanoid

EmergentSystems

Self-Organization

Co-determination

Co-development

Phylogeny Ontogeny

Embodiment

Morphology is a constitutive component of both co-determination and co-developmentConsequently, a plastic morphology is important: the embodiment shouldn’t be static

Humanoid

EmergentSystems

Self-Organization

Co-determination

Co-development

Phylogeny Ontogeny

Embodiment

CognitiveDevelopment

Increasing complexity in action space

Increasing degree of Prospection

Humanoid

Self-modification

Learning: to tunephylogenetic skills

Development:change system dynamics

new action spaces

Exploratory motives

Social motives

Novel regularities in the world

Potential of own actions

agent interaction: mutually-constructed patterns of behaviour

Mechanism to rehearsehypothetical scenariosModerate

actual behaviour

Network ofcooperating/competing

circuits

Each with ownlimited encodings

LearningSupervised

Reinforcement

Unsupervised

EmbodimentAction-PerceptionCouplings

Model Generation

Cognitive ArchitectureCognitive ArchitectureThe iCub Phylogeny

Software ArchitectureMultiple YARP processes

Running on multiple processors

Gbit ethernet

Level 0 APIs: data acquisition & motor control

DSP

iCubEmbeddedSystems

HUB

DSP DSP DSP

Sensors & Actuators

Level 1 APIs: perception/action behaviours

Cognitive Architecture

Innate perception/action primitives loose federation of behaviours

Based on phylogenic

configuration

own learning model

Level 2 APIs: Prospective Action Behaviours

Coordinated operation: Ontogenic Development

Phylogenetic self-organizing perceptuo-motor skills

Modulation circuit: homeostatic action selection by disinhibition of perceptuo-motor skills

Motivation(Amygdala)

Auto-associativeMemory

(Hippocampus)

ActionSelection

(Basal Ganglia)

Motor/Sensoryhetero-associative

memory

Sensory/Motorhetero-associative

memory

simulated motor signals

simulated sensory signals

Prospection byaction simulation

Perturbation

The RobotCub Cognitive Architecture for the iCub

– Improve the predictive performance

– Increase the space of viable actions

Key Issue:Development

Self-modification

Learning: to tunephylogenetic skills

Development:change system dynamics

new action spaces

Three purposes of internal simulation:

1. Prediction: future events

2. Reconstruction: explaining observed events (imagining a causal chain leading to that event)

3. Imagination: internal simulation as a way of imagining new ideas

Key Issue: Internal Simulation

Mechanism to rehearsehypothetical scenariosModerate

actual behaviour

Key Issue: Exogenous and Endogenous

Exogenous: curiosity

Sensori-motor learning

Endogenous: experimentation

Experience-based generative development & operation

Exploratory motivesNovel regularities in the world

Potential of own actions

Aural, visual, and proprioceptive sensory data

ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience

EndogenousSalience Vergence

ActionSelection

ProceduralMemory

EpisodicMemory

A Priori Feature Values

Egosphere

Reach & Grasp

AffectiveState

Loco-motion


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Gaze Control(IST, SSSA, U. Uppsala)


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Reach& Grasp

(U. Genoa, IIT, IST)


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Crawling(EPFL)


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Attention Sub-system(IST)


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Bhattacharyya distance


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Alternative work on episodic memory at the University of Hertfordshire


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

P A P A P

P

P

A

Prediction & Reconstruction by presenting(Pi, ~, ~) or (~, ~, Pk)

Also possible(Pi, Aj, ~), (~, Aj, Pk), or (Pi, ~, Pk)


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

Interaction Histories at theUniversity of Hertfordshire

Learning affordances at IST


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

SIFT-based object salience at IST


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

x∑New Event | Expired Event ⇒ Curiosity Spike

Predicted Event ≡ Recalled Event ⇒ Experimentation Spike

Level of Curiosity & Experimentation

C, E

Curiosity & Experimentation


ControlGaze

iCubInterface

AttentionSelection

ExogenousSalience


ActionSelection

ProceduralMemory

EpisodicMemory


Egosphere

Reach & Grasp

AffectiveState

Loco-motion

argmax (Curiosity Level, Experimentation Level)

(exogenous)salience

attentionSelection

/right/view/icub/cam/right

controlGaze2

cameraCalib egoSphere

/remoteEgoSphere

/o:velocity

/o:position

/i:gaze

/i:map

/right/map /right/out /mapVisual/map_in

/mapVisual/rgb_in

/map_out

/pos

/vel /o:status

/conf

/remoteEgoSphere

/episodicMemory/action:i

EndogenousSalience

/salience:o

From /episodicMemory/image:o/logpolarImage:i

From /icub/head/state:o/head:i

/cartesianImage:i

/mapObject/bottle_in

/segmentedImage:oTo viewer

episodicMemory

proceduralMemory

/image:i/icub/cam/right

controlGaze2

logPolarTransform logPolarTransform

/imageId:i

/head:i

/weights:o

/action:o/mode:i

/imageId:i /imageId:o

/recalledImage:o

/imageId:o

From /attentionSelection/o:position

/pos

/action:i

TO /endogenousSalience/logpolarImage:i

actionSelectionaffectiveState/mode:o

/mode:i

/imageId:i /state:o /state:i

From /icub/head/state:o

To viewer

To viewer

/retrievedImage:o

A thesis for discussion:

“Cognition emerges during development in a close interplay of experience, of the social and physical environment and of the neuronal mechanisms of growth.

An understanding of cognition cannot be achieved without an understanding of the development of cognition.

It is thus an necessity for artificial cognitive systems to take development on board”

Key Research Topics for the Future

Imagination

Self-Modification

Plasticity

Redundancy

Internal Dynamics

Social motives

The value system: creating order

Thank You!

Giulio Sandini, IIT & U. GenoaGiorgio Metta, IIT & U. GenoaLorenzo Natale, IIT & U. GenoaFrancesco Nori, IIT & U. GenoaPaul Fitzpatrick, IIT & U. GenoaFrancesco Orabona, IIT & U. GenoaMatteo Brunettini, IIT & U. GenoaAlessandro Scalzo, IITMarco Maggiali, IITMarco Randazzo IITRoberto Puddu, IITGabriele Tabbita, IITWalter Fancellu, IITBruno Bonino, IITFabrizio Larosa, IITClaudio Lorini, IITLuciano Pittera, IITDavide Dellepiane, IITMattia Salvi, IITLuca Rivano, IITRavinder Dahiya, IITAlberto Parmiggiani, IITMatteo Fumagalli, IITAlexander Schmitz, IITDiego Torazza, IITNikos Tsagarakis, IIT & U. SheffieldDarwin Caldwell, IIT & U. SheffieldFrancesco Becchi, TeleRobotPaolo Pino, TeleRobotGiulio Maggiolo, TeleRobotGabriele Careddu TeleRobotJohn Gray, U. SheffieldDavid Vernon, U. Genoa

Rolf Pfeifer, U. ZurichHarold Martinez, U. ZurichGabriel Gomez, U. ZurichAlexandre Schmitz, U. ZurichYvonne Gustain, U. ZurichJonas Ruesch, U. Zurich Kerstin Dautenhahn, U. HertfordshireChrystopher L. Nehaniv, U. HertfordshireHatice Kose-Bagci, U. HertfordshireFrank Broz, U. HertfordshireNaeem Assif Mirza, U. HertfordshireDorothée François, U. HertfordshireLars Olsson, U. HertfordshireQiming Shen, U. HertfordshireCecilia Laschi, SSSAPaolo Dario, SSSAFernando Gamarra, SSSADavide Zambrano, SSSAEgidio Falotico, SSSAMaria Chiara Carrozza, SSSAGiovanni Stellin, SSSAGiovanni Cappiello, SSSAAude Billard, EPFLAuke Ijspeert, EPFLSarah Degallier, EPFLLudovic Righetti, EPFLS. Gay, EPFL

José Santos-Victor, ISTAlexandre Bernardino, IST Ricardo Beira, ISTBruno Damas, STJonas Hornstein, , ISTLuís Vargas, ISTRicardo Nunes, ISTHugo Alves, ISTNuno Conraria, ISTJulio Gomes, IST Matteo Tajana, IST Giovanni Saponnaro, IST Christian Wressengger, IST Dario Figueira, IST Rodrugo Ventura, IST Miguel Praça, IST Jonas Ruesch, IST Luís Montesano, IST Manuel Lopes, IST Luciano Fadiga, U. FerraraLaila Craighero, U. FerraraAndrey Olyniyck, U. FerraraLivio Finos, U. FerraraGiovanni Ottoboni, U. Ferrara Claes von Hofsten, U. UppsalaKerstin Rosander, U. UppsalaOlga Kochukova, U. UppsalaHelena Gronqvist, U. Uppsala

Cognitive Development and the iCub

Documents

Transcript of Cognitive Development and the iCub