robotics - Yonsei Universitysclab.yonsei.ac.kr/courses/07TAI/robotics.pdf · Evolutionary Activity...
Transcript of robotics - Yonsei Universitysclab.yonsei.ac.kr/courses/07TAI/robotics.pdf · Evolutionary Activity...
1
Robotics
2
Rule-based Integration of Multiple Neural Networks Evolved Based on Cellular Automata
3
Motivation
• Construction of mobile robot controller– Evolving neural networks using genetic algorithm (Floreano,
1996)– Genetic programming (Nordin, 1997)– Fuzzy (Cho, 1996)
• Integration of multi-modules– Solving complex task
→ Simpler components or subtasks– Learn each module by separate systems– Combine them to solve goal task
4
Behavior-based Robot : Khepera
8 distance sensors ( 0 ~ 1023 )8 light sensors ( 50 ~ 500 )2 motors ( -10 ~ 10 )
Battery level sensor ( 0~2500 )Floor-brightness sensor
Battery RechargeArea
LightSource
5
Evolved NNs based on CA
Generate initial population
Develop NNs based on CA from chromosome
Apply it to a problem
Evaluate fitness of each NN
Select good NNs in population
Maniplate them with genetic operater
Generate new population
Satisfied NN found ? Yes
NoStop
6
CA-Based Neural Networks
Chromosome Input
Output
Growth phase Signal phase
Transmit signals from input cells to output cells
Make neural network structurewith chromosome
Applying to Control a Mobile Robot
Growth phase Signaling phase Khepera Simulator
Fitness value
Evolution
Chromosome
Neural Networks
Sensor value
Velocity of wheels
8
Cell States
• Blank – Empty space– No participate in any cell interaction during signaling
• Neuron– Collect neural signals from surrounding dendrite cells– Send them to surrounding axon cells
• Axon– Distribute signals to neighborhoods originating from neuron
• Dendrite – Collect signals from neighborhoods – Pass them to neuron cells
Growth Phase
(a)
(d) (f)
(c)(b)
(e)
y1
x1
y2
y3
y4
x2 x3 x4
Signaling phase
Collecting signals
Distributing signals
ExcitatoryAxonInhibitory
Axon
Excitatory signalInhibitory signal
Neuron
Dendrite
11
Basis Behaviors
• Battery Recharge– If a robot arrives at battery recharge area, battery is recharged.
• Go Ahead– If there is nothing around the robot, it goes ahead.
• Follow Light– The robot goes to stronger light.
• Avoid Obstacles– If the obstacles exist around the robot, it avoids obstacles
without bumping against them.
12
“Battery Recharge”
• Programmed Module• Enables it to live for a long time• “Battery Recharge” area
– Black area in environment– Light source exists
13
“Go Ahead”
• Programmed module– The velocity of left : 5– The velocity of right : 5
• Make it to move continuously without stopping
14
“Follow Light” (1)
• The robot goes to stronger light => Go to “Battery Recharge” area• Evolved module
– Evolving CAM-Brain to follow light• Fitness Function
S : Average speed of the two wheelsV : Difference between the velocity of two wheelsc : 1 (if the robot does not bump against obstacles)
1/2 (if the robot bumps)D : Distance from the robot to the goal position
15
“Follow Light” (2)
Best Fitness Average Fitness
16
“Follow Light” (3)
0 degree
180 degrees 270 degrees
90 degrees
17
“Avoid Obstacles”
• Evolved module– Evolving CAM-Brain to avoid obstacles without bumping– Using CAM-Brain module evolved incrementally in previous work
18
Incremental Evolution
• Single system learns a succession of tasks
• Task {t1, t2, t3, …. , tn}
– tn : goal task
– ti is easier than ti+1 for all i: 0 < i <= n
– Each task is derived by transforming a goal-task
19
Environments
(a) (b) (c)
(f)(e)(d)
20
Fitness
Incremental Evolution Direct Evolution
21
Architecture of NNs
1
2
3
4
5
7
6
8
1
2
3
4
5
7
6
8
1
2
3
4
5
8
7
9
1
10
11
12
13
15
14
11
17
18
16
22
Applying to Other Environments
23
Rules of Combining (1)
IF (“Battery Recharge” area)Battery Recharge module
ELSEIF (Battery sensor < α) AND (Minimum value of light sensors ≤ γ)
IF (Maximum value of distance sensors ≤ β1)Follow Light module
ELSEAvoid Obstacles module
ELSEIF (Maximum value of distance sensors ≤ β2)
Go Ahead moduleELSE
Avoid Obstacles module
24
Rules of Combining (2)
• α : If battery sensor value is less than α, battery is needed to recharging.
• β1, β2 : If the maximum values of distance sensors are larger than β1, β2, the robot recognize as obstacles.
• γ : If the minimum values of light sensor is less than γ, the robot recognizes as light.
25
Results of Combining
Moving 5000 step Moving 14000 step Moving 10000 step
26
Coordination of Multiple Behavior Modules Evolved on CAM-Brain
27
Motivation
• Construction of mobile robot controller– Evolving neural networks using genetic algorithm (Floreano, 1996)– Genetic programming (Nordin, 1997)– Fuzzy system evolved by genetic algorithm(Cho, 1996)
• Integration of multi-modules by action selection mechanism– Solving complex task
→ Simpler components or subtasks– Learn each module by separate system– Combine them to solve goal task by action selection mechanism
28
Action Selection Mechanism
• Proposed by Maes (1989)• Idea : Integrate high-level behavior into a system using lower-level
behaviors• Distributed, non-hierarchical network• Two waves of input to the network
– Sensors of external environment– Motivations or goals
• Different types of links encoding various relationships• Components of ASM : Nodes, internal links and external links
29
Example
30
Nodes
PRECONDITIONS
ADD LIST
DELETE LIST
ACTIVATION
EXECUTABLECODE
INPUT OUTPUT
PREDECESSOR LINKS
SUCCESSOR LINKS
CONFLICTOR LINKS
GOALS
ENVIRONMENT
PREDECESSOR LINKS
SUCCESSOR LINKS
CONFLICTOR LINKS
31
Internal Links
• A→B is active– Predecessor link
If (p = false) ∧ (p ∈ precondition of A) ∧ (p ∈ add list of B)– Successor link
If (p = false) ∧ (p ∈ add list of A ) ∧ (A is executable) ∧ (p ∈ precondition of B)
– Conflictor linkIf (p = true) ∧ (p ∈ precondition of A ) ∧ (p ∈ delete list of B)
32
External Links
• p→A is active– From sensors of the environment
If (p = true) ∧ (p ∈ precondition of A) – From goals
If (p ≥ 0 ) ∧ (p ∈ add list of A) – From protected goals
If (p ≥ 0 ) ∧ (p ∈ delete list of A)
33
Action Selection Procedure
• Activation of a node is updated by excitation coming in from theenvironment and the motivations
• By the type of internal links, activation of a node is exchanged between two nodes
• Normalize the node activation AVG(SUM( (activation of node1) + (activation of node2) ….)) = π
• If (all preconditions of a node are true) and (activation of a node ≥ θ )
then the node is executable • If (executable node exist )
then execute the node else θ is reduced by 10% and repeat cycle
34
Experiment Environments
Battery RechargeArea
LightSource
Battery RechargeArea
LightSource
Simple Chaotic
35
Basic Behaviors
• Recharging batteryIf a robot is in battery recharge area, then battery is recharged
• Following light Robot goes to stronger light
• Avoiding obstacles If obstacles exist around the robot, then robot avoids them without bumping
• Going straight Robot goes ahead
36
Sensors and Goals
Sensors of environment (Binary value) Goals (Real or binary value)
“In battery recharge area”“Near battery recharge area”
“In shade area”“Nothing around the robot”
“Obstacles are near”
“Battery is not zero”“Battery is not below the half”
37
Precodition and Add List of Nodes
Precondition Add list
“In battery recharge area” “Battery is not zero”“Battery is not below the half”
Recharging battery
Followinglight
GoingStraight “Nothing around the robot”
“Obstacles are near””In battery recharge area”
“Near battery recharge area”
“Obstacles are near” “Nothing around the robot”
“In shade area”“Near battery recharge area” “In battery recharge area”
Avoiding Obstacles
38
Internal Links between Nodes
Predecessor Links Successor Links
(Recharging battery → Following light)(Recharging battery → Going straight)
(Following light → Going straight)(Going straight → Avoiding obstacles)
(Avoiding obstacles → Goinging straight)
(Following light → Recharging battery)(Going straight → Following light) (Going straight → Recharging battery)(Going straight → Avoiding obstacles)(Avoiding obstacles → Going straight)
39
Model of ASM
Battery is not zero
Battery is not below the half
In battery recharge area
In shade area
Obstacles are near
Near battery recharge area
Nothing around the robot
Rechargingbattery
Followinglight
AvoidingObstacles
Goingstraight
40
Simulation Results
Simple Chaotic
41
Action Sequence (1)
0
1
2
3
4
5
1 1001 2001 3001 4001
1 = Recharging Battery2 = Following Light3 = Avoiding Obstacle4 = Going Straight
42
Action Sequence (2)
0
1
2
3
4
5
1 501 1001 1501
A B C D
43
A Framework of Evolvable Systems and Measures for
Intelligent Agents
44
Need for Agents
• Conventional information processing– Inference, planning, and commands directed by users
• Increase of the amount of information
Need for an agent that works for a user
45
Intelligent Agent
• Intelligent agent: achieve user’s goals autonomously instead of users
• Properties of intelligent agents– Autonomy– Reactivity– Proactivity– Reasoning and learning– social ability
• Cooperation• Communication
Cooperate Learn
Autonomous
Collaborativelearning agents
Smart agents
Collaborative agents Interface agents
A category of intelligent agents
46
Related Works:Application of Soft Computing Techniques
• Soft computing techniques– Neural networks, fuzzy, probabilistic inference, evolutionary
computation– Flexible inference/random searching capability
• Type 1: application of each soft computing technique– To Simple problems– Difficult to tune internal parameters– Require expert’s knowledge/time and effort
• Type 2: combination with evolutionary algorithms– Tuning of internal parameters by evolutionary algorithms
47
AdaptiveEvolution
HighEvolvability
LowEvolvability
Non-AdaptiveEvolution
GoodSolution
BadSolution
DesirableEvolutionary Causes
and Effects
High probability
Low probability
Emergence
AdaptiveBehavior
Problems of Type 2
• Can the same results be obtained? Adaptive evolution( )• What properties are genetically preferred? Adaptive behaviors( )• How the solutions are formed? Evolutionary pathways to the solutions( )• Behavioral properties? Illustration of emergence( )
1
2
1
2
4
3
3
4
48
Research Objectives
• A framework for intelligent agents easy to – represent expert’s knowledge– analyze evolution
• Application to a real-world agent to show the usefulness
49
A Soft Computing Framework
AgentConstruction
Rule-BasedSystem
EvolutionaryAlgorithms
HardwareAgent
SoftwareAgent
EvolutionAnalysis
EvolutionaryActivity
Statistics
SchemaAnaysis
ObservationalEmergence
Analysis ofEvolution
AdaptiveEvolution
AdaptiveBehavior Emergence
Evolutionary
Pathways
Research goals How to achieve the goals What we can get or show
BehaviorAnalysis
50
Rule-based Systems
• Working memory: represents current facts• Inference engine: infer appropriate actions based on current facts• Knowledge base: a set of rules for inference
IF (condition) THEN (action)….IF (condition) THEN (action)
• Merits– Easy to represent expert’s knowledge– Easily understood by humans
• Demerits– Need knowledge base
Working Memory(Facts)
InferenceEngine
Knowledge Base(Rules)
RuseBasedSystems
51
Genetic Algorithm (GA)
CandidateSolutions
Test
Terminate ?
End
CrossoverMutation
ReproduceY
N
Genetic Algorithm Parameters
Encoding of chromosomes
Types of algorithms: Simple GA, Overlapped GA
Genetic operations: Mutation/Crossover
Others
52
Evolutionary Activity Statistics
• Attach a counter to each component (e.g., alleles, individuals, …)• Accumulate counter values over generations• Need to define increment function for the counter
MeanActivity
NewActivity
Adaptability ofeach component
Overalladaptabilityof a system
Adaptiveinnovations
EvolutionaryActivity
53
Schema
• Definition– A similarity template representing a subset of strings with
similarity at certain string positions ( Holland 1968)• Composed of a character set and a don’t care character• Examples
– Character set = {0,1}, don’t care=#– #0000 {10000, 00000}– #111# {01110, 01111, 11110, 11111}
54
Observational Emergence
• “creation of new properties” – Morgan, C.L., Emergent Evolution, Williams and Norgate, 1923
• Observational emergence– Proposed by Bass, N.A.
• S : structure - system, organization, organism, machine, …• P : a property of S observed by observational mechanism
Obs
55
Application to a Robot Agent
• Robot Agent– Khepera mobile robot: 8 proximity sensors, 2 motors
56
AgentConstruction
Rule-BasedSystem
Fuzzy Logic Controller
Evolutionary Algorithms
Genetic Algorithm
EvolutionAnalysis
EvolutionaryActivity
Schema Anaysis
ObservationalEmergence
Analysis ofEvolution
AdaptiveEvolution
AdaptiveBehavior Emergence
EvolutionaryPathways
to Solutions
Research goals How to achieve the goals What we can get or show
BehaviorAnalysis
57
Fuzzy Rule-based System
Working Memory
Inference Engine
Knowledge BaseFuzzy
Rule-basedSystem
Min-Max Correlation
Eight proximitysensors
Centroid Defuzzification
20 rules maximum
OthersNumber of fuzzy sets
Membership functions
Output:Two motors
58
Genetic Algorithm
변수의퍼지집합코딩
Parameters
Encoding of chromosomes
Types of algorithms: Simple GA, Overlapped GA
Genetic operations:Mutation/crossover
Others
Overlapped GA (50%)
Two point crossover (50%)Mutation (1%)
50 individuals
59
Adaptive Evolution
0 100 200 300 400 500 600 700 800 9000
20
40
60
80
100
120
Generation
Mea
n A
ctiv
ity
Fuzzy ModelNeutral Shadow
0 100 200 300 400 500 600 700 800 9000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Generation
New
Act
ivity
Fuzzy ModelNeutral Shadow
60
Adaptive Behaviors
+ +
61
Evolutionary Pathways
62
14th individual (780)
0111001112140000311
0003120304010102401
0304000410130204031
0002040003010411441
0400010114000110211
......
...
0111001112140410311
...
9th individual (600)
0003120304020102401
0301100410130104031
...
0111101112140411311
...
6th individual (578)
...
0003120304020102401
...
25th individual (578)
0301100410130104031
...
0111001112140000311
...
5th individual (775)
0003120304010102401
0304000410130204031
...
0002040003010411441
...
27th individual (777)
0400010114000110211
...
0002030003010411441
...
24th individual (605)
0400040014040010211
...
0400040014040010211
...
6th individual (599)
...
0302020112010300001
...
6th individual (6)
...
0114041013001300130
...
33th individual (8)
...
0403120200100210101
...
21th individual (1)
...
0002030003010411441
...
12th individual (603)
...
1111111100141100131
...
1001010001030411441
27th individual (0)
M
C
M M M
CC
M
M
M
C
M
...
0301100410130104031
...
12th individual (574)
1
4
4
1
1
1
1
4
4
4
2
2
...
0003120304020102401
...
47th individual (576)
2
2
2
2
3
3
3
3
3
3
5
5
5
5
5
M Mutation
C Crossover
Best (816th generation)
0111001102140000311
0003120304010103401
0304000410130204031
0102041003010411441
0400110114000110211
......
1
2
3
4
5
1 5~ Tags for information
63
Observational Emergence:Turning Around Behavior
First order structures Obs1
Int
Obs2
• The interactions of the three first-order structures make Obs2(S2) different from the Obs1( ), of the first-order
• This implies that
• Therefore, we can conclude that Obs2(S2) is observationally emergentbehavior
17
15
12 111
,, SSS11iS }7,5,2{∈i