Vertebrate Evolution: Testing Adaptation Hypotheses using Biorobotics.
Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own...
Transcript of Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own...
![Page 1: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/1.jpg)
Minoru AsadaGraduate School of Engineering, Osaka UniversityJST ERATO Asada Synergistic Intelligence Project
13-15 December, 2010Multipurpose Hall, Bibliotheca Alexandrina
Biorobotics Workshop
Cognitive Developmental Robotics (CDR):
What's CDR and how?
![Page 2: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/2.jpg)
![Page 3: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/3.jpg)
Outline of my talk1. Cognitive Developmental Robotics
1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Vocal imitation
2. Is paradigm shift possible?1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment2. From self/other identification to self/other
separation
![Page 4: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/4.jpg)
What’s going on in the womb (1)
Emergence of fetal movements and sense (Brain figures on the top are from Figure 22.5 in [Purves et al., 08], emergence of movements is from Figure 1 in [Vries et al., 84], and fetal senses are from [http://www.birthpsychology.com/lifebefore/fetalsense.html]
![Page 5: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/5.jpg)
Infant development and learning targetsM / behaviors / learning targets--------------------------------------------------------------------------------------------------------------------------------------------------
5 / hand regard / forward and inverse models of the hand
6 / finger the other’s face / integration of visuo-tactile sensation of the face
7 / drop objects and observe the result / causality and permanency of objects
8 / hit objects / dynamics model of objects9 / drum or bring a cup to mouth / tool use 10 / imitate movements / imitation of unseen movements11 / grasp and carry objects to others / action
recognition and generation, cooperation12 / pretend / mental simulation
![Page 6: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/6.jpg)
External world(other agents, objects)
Dynamic interaction
icon
index
symbol
Set of constraints
Social interaction
Incrementalprocess
Self-organization・ DoF and motion・ self-exploration・ categorization・ sensori-motor integration
Facets of development
Based on [Lungarella_Metta_Pfeifer_Sandini03]
Neuromodulation, value, and synaptic plasticity
![Page 7: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/7.jpg)
Outline of my talk1. Cognitive Developmental Robotics
1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Vocal imitation
2. Is paradigm shift possible?1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment2. From self/other identification to self/other
separation
![Page 8: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/8.jpg)
What is Cognitive Developmental Robotics?
• Cognitive developmental robotics [1] (in short, CDR) aims at understanding human cognitive developmental process by synthetic or constructive approaches, and its core idea is "physical embodiment” that enables information structuring through interactions with the environment, including other agents.
[1] M. Asada et al., "Cognitive developmental robotics: a survey," IEEE Transactions on Autonomous Mental Development, 1(1):12–34, 2009.
![Page 9: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/9.jpg)
Design issues of CDR• From a viewpoint of artifact design: 1. What kinds of capabilities or structures should
be embedded? → the design of self-developing structures inside the robot's brain
2. How to set up the environment so that the robots embedded therein can gradually adapt themselves to more complex tasks in more dynamic situations? → the environmental design
3. What’s the temporal development structure? Discussion on the issues of underlying
neuromechanism and behavioral verification are needed. [Asada_et_al., 09]
![Page 10: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/10.jpg)
Approaches of CDR A: construction of computational model of cognitive
development1) hypothesis generation: proposal of a computational
model or hypothesis based on knowledge from existing disciplines
2) computer simulation: simulation of the processes difficult to implement with real robots such as physical body growth
3) hypothesis verification with real agents (humans, animals, and robots), then go to 1)
B: offer new means or data to know human developmental process → mutual feedback with A
1) measurement of brain activity by imaging methods 2) verification using human subjects or animal ones3) providing the robot as a reliable reproduction tool in
(psychological) experiments
![Page 11: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/11.jpg)
Amodio, D. M. and Frith, C. D. (2006) Meeting of minds: the medial frontal cortex and social cognition. Nat. Rev. Neurosci., 7, 268-277.
A view of social brain
Posterior region of the rostral MFC (prMFC) is involved in representing and continuously updating the value of possible future actions in order to regulate behavior.
prMFC
oMFCOrbital region of the MFC(oMFC) represents and updates the value of possible future outcomes.
arMFC
Anterior region of the rostral MFC(arMFC) comprise roughly three different categories: self-knowledge,person knowledge and mentalizing.
![Page 12: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/12.jpg)
UnderstandingEmergence of Human intelligence
Design for symbiotichumanoids
SynergisticIntelligence
Physio-SIMotor emergence through Dynamic Interaction with
environment based on Artificial Muscles
Perso-SISynthetic understandingCognitive development
by Imitation Science
SI-mechanismVerification of Computational
Models by brain functional imagingand human/animal experiments
Socio-SIUnderstanding and
Realization of communication with
Androids
Koh Hosoda’sKoh Hosoda’sgroupgroup
Yasuo Yasuo Kuniyoshi’sKuniyoshi’s
groupgroupHiroshi Hiroshi
Ishiguro’sIshiguro’sgroupgroup
Toshio Inui’sToshio Inui’sgroupgroup
Synergistic Intelligence Project
![Page 13: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/13.jpg)
Amodio, D. M. and Frith, C. D. (2006) Meeting of minds: the medial frontal cortex and social cognition. Nat. Rev. Neurosci., 7, 268-277.
SI-group structure
Physio-SI
Perso-SI
Socio-SISI-Mechanism
![Page 14: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/14.jpg)
Towards the principle of cognitive development
• Rethinking "embodiment" or more correctly "Shintaisei ( 身体性 )" explained as event that the body specifies the interaction between active agent and environment, and its contents. That is also the infrastructure to form the cognition and behaviors by providing the structure to the interaction.
• Representation and architecture– Individual development: more neuroscientific
representation and more computational – Social development: opposite difficult to
formalize from case studies to formalization
![Page 15: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/15.jpg)
object permanency visuo-spatial rep.
allocentric co-system
rolling overcrawling
standing with support, early walking
walking running jumping
To intension understanding
whole body motion & assist
Dynamic Motion
holding during neonate & infant
highercognition
reflection
perceptionsensorymotor mapping
functional development
voluntarymotions
From emergence of social behavior through interactions with caregiver to development of communication
Vowel imitation Joint attention Responsive gazeSympathy development Future image: development of communication
ATR 知能ロボティクス研究所開発
body scheme & tool use
egocentric co-system
Fetus Simulation
Body representation &Spatial perception
![Page 16: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/16.jpg)
Physio-Synergistic Intelligence
• From dynamic connection between body structure and environment with artificial muscles to understanding emergence of human intelligence– Antagonistic pair of pneumatic
actuators enables biped robots to walk on unstructured terrain and also to run and jump.
– Seamless transition between dynamic motions
[Hosoda G]
![Page 17: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/17.jpg)
From motor development to cognitive one
[Prof. Koh Hosoda G]
![Page 18: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/18.jpg)
Outline of my talk1. Cognitive Developmental Robotics
1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Vocal imitation
2. Is paradigm shift possible?1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment2. From self/other identification to self/other
separation
![Page 19: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/19.jpg)
When the robot touches its own body….→ Spatial and temporal integration of each sensory information
Infants’ imitation = AIM-model
Organ Identification
( A.N.Meltzoff and M.K.Moore 1977 )
( Active Intermodal Matching Model )
Visual sense
Tactile sense
Proprioceptive sense
how do infants code the perceived and produced human acts within a common framework which enables infants to detect equivalences between the features of the different modality ??
Question?
![Page 20: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/20.jpg)
Fetal Sensori-motor Development
[Kuniyoshi and Sangawa 06]Yasuo Kuniyoshi and Shinji Sangawa, Early Motor
Development from Partially Ordered Neural-Body Dynamics -- Experiments with A Cortico-Spinal-Musculo-Skeletal Model,
Biological Cybernetics, vol. 95, no. 6, pp. 589-605, Dec., 2006.
![Page 21: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/21.jpg)
Fetal Brain Development (1)[Sangawa & Kuniyoshi 06]
Body and brain
![Page 22: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/22.jpg)
Fetal Brain Development (2)[Kuniyoshi et al., 08]
see new video
![Page 23: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/23.jpg)
Fetal Brain Development (3)[Sangawa & Kuniyoshi 06]
![Page 24: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/24.jpg)
Fetal Brain Development (4)[Sangawa & Kuniyoshi 06]
![Page 25: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/25.jpg)
Fetal Brain Development (5)[Kinjo & Kuniyoshi 09]
![Page 26: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/26.jpg)
Visuo-Tactile Face Representation through Self-induced Motor Activity
[Fuke, Ogino, Asada 07]Fuke Sawa, Masaki Ogino, and Minoru Asada. Body image constructed from motor and tactle images with visual information. International Journal of Humanoid
Robotics, Vol.4, No.2, pp.347-364, 2007.
![Page 27: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/27.jpg)
Simulation
Arrangement of the tactile units
Arm : 5 degrees of freedom
Tactile units : 21×21Hand cells : 36
<The robot can detect…>1. Angles of its arm2. Position of the hands
in the camera image3. The ID of the nearest
cell of the hand??
![Page 28: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/28.jpg)
A neural network model for image formation of the face
through double-touch without visual information
[Song, Inui, Takemura 09]Wei Song, Toshio Inui, and Naohiro Takemura.
2nd International Conference on Cognitive Neurodynamics 2009(ICON'09), 2009.
![Page 29: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/29.jpg)
tactile sensor
of finger
SI (finger)
shape coding withGabor filters
Hebbian connections
tactile sensor
of face
SI (face)
VIP
visual area
onion-like mapping
SOM with Gaussian-like local connections
SOM with Gaussian-like local connections
location pathw
ay
shape pathw
ay
Schematic diagram of the model
[Song, Inui, Takemura 09]
![Page 30: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/30.jpg)
• When a part of a fetus’s face is touched by his/her hand, the tactile receptors on the facial skin are activated. Topological information about touched area on the face is transmitted to, and represented in, the VIP (ventral intraparietal area) via an onion-like map in the primary somatosensory cortex (SI) (Moulton et al., 2009).
• Connections between SI and VIP as well as VIP and visual cortex are learned with the self-organizing map (SOM).
S1 map
finger areacorresponding
representation of onion-like map
Location pathway
[Song, Inui, Takemura 09]
![Page 31: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/31.jpg)
• Concavo-convex information of the face within the finger surface is encoded by Gabor filters in SI according to the physiological findings (Bensmaia et al., 2008). There are sixteen kinds of filters with different orientation selectivity (see right fig.) The population of finger SI units codes shape information for each location within this filter bank.
Shape pathway
[Song, Inui, Takemura 09]
![Page 32: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/32.jpg)
• After SOM, VIP units acquired facial somatotopic mapping (Fig. 5) similar to that seen in human fMRI data (Sereno and Huang, 2006).
• Right figure shows the acquired facial image in the visual cortex. This image appears when all the SI (finger) units are activated. it is a potential image formed by a number of finger tactile inputs. Retino-cortical mapping is well known as a log-polar transformation of the retinal image (e.g., Shwartz, 1977) Learning result
A
B
C q
Log(r+
1)
Results and Discussion
[Song, Inui, Takemura 09]
![Page 33: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/33.jpg)
Issues:• Myowa and Takeshita (2006): mouth opening
reflection when fetuses (25w) hear their mothers’ voice.
• Oyabu (2008): Lip motion is exceptional while body motion and sensing function are compatible two months after the birth.
• Go and Konishi (2008): Neural infrastructure for neonatal imitation is under sub-cortical structure.
To what extent and how much innate?
![Page 34: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/34.jpg)
object permanency visuo-spatial rep.
allocentric co-system
rolling overcrawling
standing with support, early walking
walking running jumping
To intension understanding
whole body motion & assist
Dynamic Motion
holding during neonate & infant
highercognition
reflection
perceptionsensorymotor mapping
functional development
voluntarymotions
From emergence of social behavior through interactions with caregiver to development of communication
body scheme & tool use
egocentric co-system
Vowel imitation Joint attention
Fetus Simulation
Responsive gazeSympathy development Future image: development of communication
ATR 知能ロボティクス研究所開発
Body representation &Spatial perception
![Page 35: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/35.jpg)
Outline of my talk1. Cognitive Developmental Robotics
1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Vocal imitation
2. Is paradigm shift possible?1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment2. From self/other identification to self/other
separation
![Page 36: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/36.jpg)
Visual attention by saliency leads cross-modal body representation
[Hikita, Fuke, Ogino, Asada 07]Mai Hikita, Fuke Sawa, Masaki Ogino, and Minoru
Asada. IEEE 7th International Conference on Development and Learning, 2008.
![Page 37: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/37.jpg)
Dynamic body representation (1)• Bimodal neuron : body image extension by
tool use. [Iriki_et_al,96,01]
• Body scheme → unconscious, dynamic process of body control.
• Body image → conscious representation of self body.
• Interaction between external environment (vision) and body scheme.
[Stamenov, 2005]
Visual
Proprioceptive
Tactile
![Page 38: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/38.jpg)
Dynamic body representation (2)
The activities of bimodal neurons change after training of tool-use.
The receptive field was extended to the tool.
Tool-use by the Japanese macaque [Iriki et al. 1996]
? How such representation is acquired ?
Before tool-use After tool-use
![Page 39: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/39.jpg)
Dynamic body representation (3)Results: Experiment with a real robot
CB2 ( Minato et al. 2007)
With hand
With a tool
The connection weights
The activation of the integration map
![Page 40: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/40.jpg)
Outline of my talk1. Robotics and RoboCup Initiative
1. What’s RoboCup?2. Video Clips
2. Cognitive Developmental Robotics1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Physical interaction5. Vocal imitation
3. RoboCity CoRE
![Page 41: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/41.jpg)
object permanency visuo-spatial rep.
allocentric co-system
rolling overcrawling
standing with support, early walking
walking running jumping
To intension understanding
whole body motion & assist
Dynamic Motion
holding during neonate & infant
highercognition
reflection
perceptionsensorymotor mapping
functional development
voluntarymotions
From emergence of social behavior through interactions with caregiver to development of communication
body scheme & tool use
egocentric co-system
Vowel imitation Joint attention
Fetus Simulation
Responsive gazeSympathy development Future image: development of communication
ATR 知能ロボティクス研究所開発
Body representation &Spatial perception
![Page 42: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/42.jpg)
Analysis of Physical Human- Robot Interaction for Motor Learning with Physical Help
Shuhei Ikemoto, Takashi Minato, and Hiroshi Ishiguro
2008 IEEE-RAS International Conference on Humanoid Robots, 2008
![Page 43: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/43.jpg)
Background• Flexibility of robot joints
– simple control system– physical interaction
• How to evaluate physical interaction?– the result (success or failure)– subjectively good interaction
for human.
Which parameters lead subjectively good physical interaction
![Page 44: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/44.jpg)
Procedure1. The human pulls up the robot’s hands.2. The Robot reacts and shrinks back.
Human assists robot uprising
3. The Robot kicks floor.
4. The human assists robot uprising
![Page 45: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/45.jpg)
Human assists robot uprising
CB2 just switches own desired postures only twice.
![Page 46: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/46.jpg)
• Time-window: approx. 1.5sec
• lag: -1.5 sec ~ 1.5sec
Cross-Correlation Value
time
post
ure
chan
ge n
orm
time
lag
Robot data are shifted
Human data are shifted
cross-correlationvalue
• Robot moves after human started moving
• The profiles of both norm data become similar each other
![Page 47: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/47.jpg)
Movies
Expert Beginner (smooth)
Beginner (nonsmooth) Beginner (failed)
![Page 48: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/48.jpg)
Successful ResultExpert Beginner (success)
An expert can start an interaction from more synchronized state because he can
predict robot’s motion
![Page 49: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/49.jpg)
Failed ResultBeginner (nonsmooth)Beginner (failed)
An interaction successfully done needs continuous positive correlation,and the lag should be smaller
![Page 50: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/50.jpg)
Results• An expert starts an interaction from more
synchronized states.• An expert is supposed to predict robot’s movements.
• An interaction successfully done needs continuous positive correlation, and the lag should be smaller.
• In failure interaction, that feature doesn’t appear in the correlation space.
![Page 51: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/51.jpg)
Physical Interaction Learning: Behavior Adaptation in Cooperative Human-Robot Tasks Involving Physical Contact [IEEE RO-MAN 2009, 2009]
![Page 52: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/52.jpg)
Physical human-robot interaction(1)• We are focusing on physical human-robot
interaction• In human-human interaction, there are
several kinds of physical interaction– Physical interaction between a mother and a baby– The baby can stand up despite she still cannot
stand up by herself
![Page 53: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/53.jpg)
Physical human-robot interaction(2)• Main technical difficulties in this challenge
are:– How to ensure that safety is always guaranteed– How to assure that the robot reacts appropriately
to the force applied by the human instructor– How to improve the robot’s behavior using a
machine learning algorithm in the PHRI
Flexible jointsare necessary!
![Page 54: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/54.jpg)
Learning to switch target pauses
• Robot motions and changes cause human’s motions and changes. Vice versa?
PHRImodulation
DB s1
s2
Switching rule
:Posture spacesuccess
![Page 55: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/55.jpg)
• In order to refine the robot’s motion, improving the human’s motion is required– The purpose is
to improve not only the robot’s motion but also the human’s motion, in other words, to improve the interaction
Human-in-the-loop Approach (1)
The human bears a part of the learning system as a critic of the interaction
The human bears a part of the dynamics as an actor of the interaction
![Page 56: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/56.jpg)
Human-in-the-loop Approach (2)Which is the target pause during motions
1
2
Colors change when the targetpauses change
These trajectories can be realizedwith human’s physical assistance
・・・
52D Pause Space
Reduction by PCA
52 → 2
Gaussian Mixture Model:
where
EMAlgorithm
![Page 57: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/57.jpg)
Results (1)
Beforelearning
Beforelearning
Afterlearning
Afterlearning
![Page 58: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/58.jpg)
Results (2)
![Page 59: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/59.jpg)
Mapping from Facial Expression to Internal State based on Intuitive Parenting
[Watanabe, Ogino, Asada 07]Ayako Watanabe, Masaki Ogino, and Minoru Asada. Mapping Facial Expression to Internal States Based on Intuitive Parenting. Journal of Robotics and Mechatronics, Vol.19, No.3, pp.315--323, 2007
![Page 60: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/60.jpg)
Introduction (I)
• Future robots entering into our daily life are expected to be endowed with the ability of sympathy.
• Understanding the developing process to acquire the sympathetic ability is one of the interesting mysteries in developmental psychology.– imitation during motor babbling [Breazeal
et al., 2005] ?
![Page 61: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/61.jpg)
• Parents often mimic and emphasize their facial expression of certain emotional stated presumed from their children’s facial expression
[ Papousek and Papousek. 1987 ]
Intuitive Parenting
Introduction (II)
![Page 62: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/62.jpg)
• Intuitive parenting is thought to help children to develop their sympathetic ability [Gergely and Watson, 1999]
Visual space for face
relation
stimulus
Internal state
• Modeling a developmental process of sympathy through intuitive parenting
Introduction (III)
![Page 63: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/63.jpg)
System Overview
![Page 64: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/64.jpg)
Internal state (I)• The internal state of the robot consists of two
kinds of variables, arousal-sleep axis and pleasure-displeasure axis.
[Russel1980 ]Arousal
Sleep
Displeasure
Surprise Elated
Happy
Content
Sleepy
Afraid
Sad
FrustratedExcitementStress
Depression Calm
BoredRelaxed
NaturalPleasure
![Page 65: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/65.jpg)
Facial expression• The robot changes its facial expression according
to the internal state, S.• Two structural variables “curving and releasing”
and “inclination” are related to the internal state. [Yamada, 1993]
![Page 66: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/66.jpg)
Learning (I)• Facial expression of the caregiver is
categorized by self organizing map (SOM)• Mapping between face SOM and internal state
is learned by Hebbian.
)(Vfij
Internal space)(Sklg
Face SOM
€
∆wijkl =αfij(V)gkl(S)
€
f i j ( V ) = e − ν | V − V i j | 2
g k l ( S ) = e − ρ | S − S k l | 2
![Page 67: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/67.jpg)
Experimental Setup
Keyboard (touch sensor)
Microphone (sound)
CameraDisplay
![Page 68: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/68.jpg)
Learning experiment
Human facial expression SOM of humanfacial expression
Robot facialexpression
p
a
Internal state space
4: human facial expressioncorresponding to robot state
and expression
3: facial expression 2: change of internal stateby the external stimulas
1: External stimulas Hebbian Learninfg
![Page 69: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/69.jpg)
Experimental result• The result of mapping between caregiver’s
facial expression and self internal state• The caregiver’s
faces are associated with the internal state, and the distribution is almost same as the Russel’s two dimensional emotional model.
![Page 70: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/70.jpg)
Experimental result Sympathetic communication after learning
• Sympathetic facial expression is possible by introducing the effect of the correspondence of internal state and other’s facial expression.
![Page 71: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/71.jpg)
Discussion (1)• In brain, which region is responsible for
sympathy? [Singer et al. 2004]• Using fMRI, brain activity while volunteers
experienced a painful stimulus and compared it to that elicited when they observed a signal indicating that their loved one. Bilateral anterior insula (AI), rostral anterior cingulate cortex (ACC), brainstem, and
cerebellum were activated when subjects received pain and also by a signal that a loved one experienced pain.
![Page 72: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/72.jpg)
Discussion (2)1. The pain area is stimulated first using self-
sensor input. 2. Another sensor input of a different modality
(e.g. visual information from a caregiver) is partly projected on to the same area.
3. Intuitive parenting strengthens the connection between the subject’s feeling based on the pain sensation of self and the feeling toward that of others based on visual information.
4. Sensor input from different modalities suppress each other, finally forming a sympathetic region.
![Page 73: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/73.jpg)
Discussion (3)• It seems that the emotional state of self and
others are represented in the different area of the same region. What switches self and others?
![Page 74: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/74.jpg)
Vowel Acquisition by Maternal Imitation
[Yoshikawa et al., 03] etc.Yuichiro Yoshikawa, Minoru Asada, Koh Hosoda, and Junpei Koga. A Constructivist approach to
infants' vowel acquisition through mother-infant interaction. Connection Science, Vol.15, No.4,
pp.245--258, 2003.
![Page 75: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/75.jpg)
Introduction• Vowel Imitation between Agents with Different
Articulation Parameters by Parrot-like Teaching
– Infants seem to acquire (imitate) phonemes:• without any explicit knowledge about the
relationship between their sensorimotor system and phonemes, and
• without a capability to reproduce the adult’s sound as they are.
• How can robots do that?
![Page 76: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/76.jpg)
2007.08.27
Human Vocalization [Deacon 98]• Vocalization the
interaction of the oral and respiratory tracts special association with midbrain systems.
• To organize vocalization coordinated activation of the cluster of motor neurons that control the muscle of breathing, the tension of the larynx, and the movement of the oral and facial muscles. the motor neurons controlling all of these are located in the upper brain stem.
![Page 77: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/77.jpg)
Human Vocalization [Deacon 98]• Two evolutionary shifts producing increasing
cortical control over motor output from brain stem articulatory and vocal systems. These shifts were produced by an increase in the proportions of the cerebral cortex in comparison to these brain stem structures.
Cortex-brain stem projection
HumanPrimateNon-primate
![Page 78: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/78.jpg)
A constructivist approach• The purpose To build a robot that
acquires the vowels of a human caregiver • Design issues:
–What kind of mechanism should be embedded?
–What should be the behavior of the caregiver?
Caregiver’s behavior?
Robot’s mechanism?
![Page 79: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/79.jpg)
Observations in human infants
• Infant’s speech-like cooing tends to make its mother utter [Masataka and Bloom’94].
• Maternal imitation of infant's cooing (i.e., parrot-like vocalization) increases vocalization rates of a three-month-infant [Pelaez-Noqueras ’96] .
infant cooing maternal imitation
Conjectures It reinforces infants’ speech-like cooing. It helps to find the correspondence between cooing and phonemes.
![Page 80: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/80.jpg)
The robotOutput sound Sound
source
Silicon tube
deformingArticulation vector
PC
5 Motors
Formant vector
Formant extractorMicrophone F1
F2 F3 F4…..
Frequency [Hz]
![Page 81: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/81.jpg)
Formant distribution of Japanese average female
• Resonant frequency changes depending on the shape of vocal tract.
• Vocal feature for vowel discrimination.
• Non-human primates and birds utilize as perceptual cues [Fitch 2000]
What’s Formant Space?
![Page 82: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/82.jpg)
A model of interaction
Randomarticulator
Parrot-like teaching
Learningmodule
motors
microphone
The robotThe caregiverRandomly cooing
Auditory layer
Articulation layer
![Page 83: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/83.jpg)
auditory layer
Learning mechanism
• Clustering the articulation parameters and the formant vectors by the SOM algorithm.
• Connections are updated based on Hebbian learning.
random articulator
SOM
Parrot-like teaching
Formant vector
input
Formant extractor
articulation layer
SOM
Hebbian learning
trigering
Articulationvector
cooing
![Page 84: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/84.jpg)
Acquired vowels• The acquired vowels can be interpreted as
Japanese vowels.
/a/, /i/,/
u/, /e/
![Page 85: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/85.jpg)
Result: how it acquire vowels?
The articulation vectors corresponding to the variation of the caregiver’s vowels
PC 1
PC 2
/a/
/i//e/
/u/
• The vowel /o/ is not acquired due to the difference in shape of vocal tracts.
• There is “arbitrariness” in correspondence.
![Page 86: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/86.jpg)
Introducing subjective criteria
• A “subjective” criterion: more facile articulation is better.– less torque ( )– less deformation change ( )
Articulation layer
Auditory layer
mj
fiij aaw ∝∆
mj
fiidctrqij aaccw ),(η∝∆
trqc
idcc
Basic Hebbian rule
Modified Hebbian rule
![Page 87: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/87.jpg)
Result: effect of subjective criteria
Comparing the distribution of the articulation vectors corresponding to the variation of the caregiver’s vowels
PC 1PC
2
/a/
/i/
/e/
/u/
Subjective criteria can reduce arbitrariness.
PC 1
PC 2
/a/
/i/
/e/
/u/
Basic Hebbian rule Subjective criteria
![Page 88: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/88.jpg)
Lip shape imitation
[Miura et al, 2006]
![Page 89: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/89.jpg)
Visual imitation , too!
![Page 90: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/90.jpg)
Caregiver’s Auto-mirroring and Infant’s Articulatory Development Enable Vowel Sharing
Hisashi Ishihara*, Yuichiro Yoshikawa**, and Minoru Asada**,** Grad. Sch. of Eng., Osaka University, Japan** Asada Synergistic Intelligence Project, ERATO, JST [ICDL08,EpiRob09]
i eau
oi e
auo
![Page 91: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/91.jpg)
Infant’s Vowel Development• Sharing process of perceptual & behavioral primitives
between a caregiver and her infant across their different bodies– Physical quantities of their producible vowels are
different [Vorperian & Kent.’07]
– Infants’ audition [Polka & Werker.’94] & articulation [Kuhl & Meltzoff.’96] adapt to mother-tongue (become native)
• Dynamic process including intrapersonal interaction & social interaction [Kuhl et al.’08]
Social engagementof caregiver Perceptual
development
Articulatorydevelopment
![Page 92: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/92.jpg)
Caregiver’s Sensorimotor Magnets Lead Infant’s Vowel Acquisition through Auto Mirroring
• A method that aids unconscious guidance in mutual imitation for infant development based on a biasing element with two different kinds of modules.
1.The normal magnet effect in perceiving heard vocal sounds as the listener’s own vowels (perceptual magnet) and also includes another magnet effect for imitating vocal sounds that resemble the imitator’s vowels (articulatory magnet).
2.What we call “auto mirroring bias,” by which the heard vowel is much closer to the expected vowel because the other’s utterance is an imitation of the listener’s own utterance.
[Ishihara et al, 2008]
![Page 93: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/93.jpg)
Caregiver’s Sensorimotor Magnets Lead Infant’s Vowel Acquisition through Auto MirroringHow humans imitate the sound?
Synthesi zedVoi ces Imitated
Voices
/a/
/i //u/
/e/
/o/2nd F
orma
nt [
mel]
1st Formant [mel ]
i mi tati on
[Ishihara et al, 2008]
![Page 94: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/94.jpg)
Psychological experiment for auto-mirroring bias
Cnt. G Exp. G
Voice-Feature Space
Voice-Feature Space
Imi (Cnt) < Imi (Exp)
Auto-Mirroringbias
HypothesisImi (Cnt)
Imi (Exr)
[Wakasa et al., unpublished]
![Page 95: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/95.jpg)
p-value 0.036 0.014 0.454
Cnt. G
Exp. G# of Data
15
11
15
14
14
14
Significant difference
Auto-mirroring bias exists!
No significant differenceConcentrationlevel down
3rd set2nd set1st set
Voice Feature D
ifference
[Hz]
※significant level 5%
Psychological experiment for auto-mirroring bias
Cnt. Exp.
[Wakasa et al., unpublished]
![Page 96: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/96.jpg)
Caregiver’s Sensorimotor Magnets Lead Infant’s Vowel Acquisition through Auto Mirroring
[Ishihara et al, 2008]
![Page 97: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/97.jpg)
Caregiver’s Sensorimotor Magnets Lead Infant’s Vowel Acquisition through Auto Mirroring
[Ishihara et al, 2008]
![Page 98: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/98.jpg)
Remarks• Our model reproduced some phenomena in vowel
sharing process of infants– Rapid separation of infant vowel clusters– Motherese– Gradual acquisition of native vowels
• Caregiver’s affirmative biases guided infant vowels to be native through mutual imitation
• Infant articulatory immaturity (noise) contributed vowel cluster separation and enables acquiring native vowels
• Stretching motherese, exaggerating vocalizations, might caused by caregiver’s underestimation of infant vocalizations
![Page 99: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/99.jpg)
Outline of my talk1. Cognitive Developmental Robotics
1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Vocal imitation
2. Is paradigm shift possible?1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment2. From self/other identification to self/other
separation
![Page 100: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/100.jpg)
Our goal is ….• To build a model of human’s cognitive
development processes.• One promising approach is “Cognitive
Developmental Robotics” that aims at not simply filling the gap between brain science and developmental psychology but more challengingly at building a new paradigm that provides new understanding of ourselves and at the same time new design theory of humanoids symbiotic with us.
• But, not yet!
![Page 101: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/101.jpg)
Is a paradigm shift possible? (1)• Analysis based approach from a God’s
viewpoint faces with twofold difficulties when it focus on humans as living things.
1.Biology more and more microscopic views with various kinds of levels and representations such as those in cell biology or molecular biology.
2.Understanding human beings as social agentshard or insufficient under a single scientific paradigm, therefore
Interdisciplinary approach seems essential.
![Page 102: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/102.jpg)
Is a paradigm shift possible? (2)• What is a sufficient condition? • Is it impossible by integrating the existing scientific
disciplines? • Is CDR completely independent from them? • Of course not! By involving them, CDR should give its
significance by prospecting the limits of the existing scientific disciplines.
• In this context, the issue to be attacked is “interaction” between neurons or brain regions or individual persons.
• Communication, a kind of interaction between subjective agents may involve the language development in the level of the individual persons, and therefore seems difficult to formalize the interaction.
![Page 103: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/103.jpg)
Is a paradigm shift possible? (3)• To summarize the CDR approach,1)integrate the knowledge, evidences, and findings
(utilize the existing paradigms and synthesize them), not to deny the existing disciplines but to involve them. Therefore, CDR researchers should have the minimum amount of knowledge in these disciplines
2) build a model or a hypothesis that have no contra- diction with the existing disciplines or resolve the contradiction or controversial issues, a key point for the CDR researchers to hit on an idea that reflects the integrated knowledge in 1) and,
3) Find a new factor that provides a solution to mystery through the verification process of the model or the hypothesis by simulations or real experiments.
![Page 104: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/104.jpg)
Is a paradigm shift possible? (4)• The common issues are body representation, rhythm
and timing, multimodal input/output (vision, auditory, touch, somatosensory, motion, vocalization etc.), self-other separation, sociality acquisition, and so on.
• If CDR can provide the constructive and unified form of the representation that can explain and simultaneously design the cognitive development of these issues, instead of representing them separately, this may lead to the creation of a new value of the paradigm shift.
• To enable this, the studies of developmental disorders in addition to the studies of normal children may help the unified model construction of cognitive development.
![Page 105: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/105.jpg)
Outline of my talk1. Cognitive Developmental Robotics
1. Human development2. What’s cognitive developmental robotics?3. Fetus/neonate development simulation4. Vocal imitation
2. Is paradigm shift possible?1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment2. From self/other identification to self/other
separation
![Page 106: Biorobotics Workshop Cognitive Developmental Robotics (CDR ... · When the robot touches its own body…. → Spatial and temporal integration of each sensory information Infants’](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec105792502f651f72627d6/html5/thumbnails/106.jpg)
Acknowledgement• Group Leaders and Research Contributors :
– Prof. Koh Hosoda (Osaka Univ.)– Prof. Yasuo Kuniyoshi (Univ. of Tokyo)– Prof. Hiroshi Ishiguro (Osaka Univ.)– Prof. Toshio Inui (Kyoto Univ.)– Dr. Masaki Ogino (Osaka Univ., body image)– Dr. Yuichiro Yoshikawa (JST ERATO, vowel imitation)– Ph.D student Hisashi Ishihara (Vowel imitation, Affetto)
Thank you for your attention!Please come to Osaka! Everything robotics and cognition is there!