The advocating of the embodied, situated, enactive characters of cognition
Adaptation in Embodied & Situated Agents
-
Upload
claudio-martella -
Category
Technology
-
view
2.936 -
download
1
description
Transcript of Adaptation in Embodied & Situated Agents
![Page 1: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/1.jpg)
Adaptation in embodied & situated agents
Author: Claudio MartellaCollaborators: Dott. Stefano Nolfi (ISTC - CNR)
Prof. N.A. Borghese (AIS Lab - UniMi)
October, 2011
1
Tuesday, October 11, 11
![Page 2: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/2.jpg)
• the behavior might be too complex for the designer to control
• the environment is noisy and not perfect
• the world is unpredictable
It is difficult to build autonomous systems through a top-down approach:
2
The problem
Tuesday, October 11, 11
![Page 3: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/3.jpg)
Evolutionary robotics is a branch of robotics that uses evolutionary methodologies
to develop controllers for autonomous robots.
Nolfi, Floreano [2004] - MIT Press
3
Tuesday, October 11, 11
![Page 4: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/4.jpg)
The objective
We wanted to analyze the possibility of applying adaptive processes to embodied & situated agents
considering evolutionary, individual and social learning.
4
Tuesday, October 11, 11
![Page 5: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/5.jpg)
E&S agents
• Embodied: the agent can exploit the characteristics of the robot (shape, sensors, actuators etc.).
• Situated: the solution can exploit the possible interactions that the environments offers.
5
Tuesday, October 11, 11
![Page 6: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/6.jpg)
The methodologyE-puck Robot Simulation
Problem: categorize 10 objects (Good, Poisonous)6
Tuesday, October 11, 11
![Page 7: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/7.jpg)
The evolutionary process
7
Tuesday, October 11, 11
![Page 8: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/8.jpg)
1st goal
8
Implement an algorithm for individual learning.
The algorithm should start with one set of candidate parameters
and it would modify them by trial & error.
Decision: start from Simulated Annealing *
* "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science
Tuesday, October 11, 11
![Page 9: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/9.jpg)
Simulated Annealing
9
Temperature:
It probabilistically accepts mutations that decrease
the fitness.
The probability decreases with time.
It allows the algorithm to jump out of local minima.
Tuesday, October 11, 11
![Page 10: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/10.jpg)
Stochasticity in E&SEvaluation depends on
the (random) initial conditions:
10
Tuesday, October 11, 11
![Page 11: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/11.jpg)
The intuition
0
0.225
0.45
0.675
0.9
100 200 300 400 500
Temperature
0
0.225
0.45
0.675
0.9
10 20 30 40 50
Stochasticity
Probability of accepting negative mutations decreases with the
increase of time
Probability of accepting negative mutations decreases with the
increase of #evaluations11
Tuesday, October 11, 11
![Page 12: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/12.jpg)
Contributions
• Simplifies the algorithm
• Better performance (~10% improvement)
• Lighter algorithm (~50% less evaluations for us)
• Remove Temperature
• Start with few evaluations and increase with time
Results
12
Substitute external stochasticity with internal:
Tuesday, October 11, 11
![Page 13: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/13.jpg)
2nd goal
13
Implement an algorithm for social learning.
The algorithm should take advantage of the interaction with an expert agent
to acquire an adaptive solutionthat is improved and/or in less time.
Decision: apply individual learning to imitation.
Tuesday, October 11, 11
![Page 14: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/14.jpg)
Why?
14
Social learning should avoid reinventing the wheel.
In principle, when guided, learning is faster & safer.
It should be the basis for cultural evolution.
Tuesday, October 11, 11
![Page 15: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/15.jpg)
How?
15
There are simpler forms of social learning:
• social facilitation
• contagious behavior
• stimulus enhancement
Tuesday, October 11, 11
![Page 16: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/16.jpg)
How (technically)?
16
Fitness function: student should learn to give outputs similar to the agent’s, given the same input.
Tuesday, October 11, 11
![Page 17: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/17.jpg)
How (technically)?
17
fit = fitsoc
· (1� ↵) + fitind
· ↵
↵ = cN
Pure imitation brings to under-fitting individuals.We introduced a hybrid approach.
Tuesday, October 11, 11
![Page 18: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/18.jpg)
Contributions
• Performance on the problem is not improved
• Adaptive behavior is acquired faster
• More agents acquire an adaptive behavior
• Modeled social learning with simple form of imitation
• Modeled hybrid social-individual learning approach
Results
18
Tuesday, October 11, 11
![Page 19: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/19.jpg)
Intuitive interpretation
19
Social learning as a method for promising initial parameters selection.
Social learning as a method for jumping out of local maxima.
parameters space solutions space
Tuesday, October 11, 11
![Page 20: Adaptation in Embodied & Situated Agents](https://reader033.fdocuments.in/reader033/viewer/2022042623/54620e37b1af9f92238b4cb4/html5/thumbnails/20.jpg)
Questions?
20
Tuesday, October 11, 11