Robot Control using Genetic Algorithms - UFSCmauro/ine5377/Cursos-ICA/CE-ga7_2.pdf · Robot Control...

1

Robot Control using Genetic Algorithms

Summary

• Introduction– Robot Control– Khepera Simulator

• Genetic Model for Path Planning– Chromosome Representation– Evaluation Function– Case Studies

• Conclusions

2

The Robot Controller Problem

Given a robot and a description of an environment,provide commands (motor speeds) to the robot, in

order to achieve a path between two specifiedlocations, which is collision-free and satisfies

certain optimisation criteria.

(xi, yi)

(xf, yf)

Optimisation Criteria

• Robot should:– attempt near-optimal paths– avoid obstacles– perform straight motion

• Controller should be independent of:– the robot’s environment– target location

3

The Khepera Simulator

• Freeware mobile robot simulator (designed by OlivierMichel, University of Nice Sophia-Antipolis)

• User designed worlds• Control algorithms can be written in C/C++• Robot’s position and angle reading• Colourful Graphical Interface• 8 sensors (S0-S7): [0, 1023]• 2 motors (M1, M2): [-10, +10]

S0

S1S2 S3

S4

S5

S7 S6

M1 M2

Front identification

Simulator Readings:sensors, position and angle

S0

S1S2 S3

S4

S5

S7 S6

M1 M2


S0-S7: [0, 1023]

1000

1000

X

Y

Robot’s World

S0

S1S2 S3

S4

S5

S7 S6

M1 M2


angle of the robot with the world

α : [-π, π]

x

y

0

obstacle not obstacle very

detected closed

4

Control Mode

• To evolve the robot’s attitudes as itinteracts with the external environment

• Each robot action determines:– how well the controller performs with

respect a given task;– the next input stimuli to the controller.

• The controller should learn as the robotinteracts with the environment

Controller Model

Genetic Algorithm

evolves robot’s attitudes

Sensors

Position

Robot’s Angle

Goal Location

Motor 2 Motor 1

KheperaSimulator

5

Proposed Model

IF Obstacle detectedTHEN

Avoid collision, forget target

ELSE Straight to the target according to thetarget direction

END

Sensors Reading Simplification

Sleft Sright

Sback

S0

S1S2 S3 S4

S5

S7 S6

Sleft = ( S0 + S1 + S2 ) / 3

Sright= ( S3 + S4 + S5 ) / 3

Sback = ( S6 + S7 ) / 2

6

Determining the Target Direction

(x,y)α = π/2

arrival position: (xf ,yf)

β

β = tan-1[(yf - y)/(xf - x)]

β - απ/2

π

π/4

0

− π/4

− π/2

3π/4

− 3π/4

Goal point toright side

Goal point infront

Goal point toleft side

Goal pointbehind

Direction =

Model

IF ((Sleft > L) or (Sright > L) or (Sback > L))THEN

Obstacle detected, avoid collision, forget targetProximity-sensor = highest value (Sleft, Sright, Sback)

ELSEObstacle not detected (collision-free), straight to the targetTarget direction = β - α

ENDL=collision threshold=900

7

Genetic Algorithm Modelling

• Chromosome Representation• Evaluation Function• Genetic Operators• Techniques• Parameters

Chromosome Representation

M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2

Target Direction(collision free) Obstacle detected

Left Right BackFront Left Right Back

Attitudes (Genes)

1 2 3 4 5 6 7

Which speed should be imposed to each motor ineach situation the robot is?

8

Evaluation Function

( )10;10;10

**7

1

≤≤≤≤≤≤

= ∑=

AiDiVi

AiDiViFi

• Three objectives:– (V) speed– (D) straight motion– (A) action: reach a target and avoid obstacles

• Calculated for each gene [1,7] at each step

Speed

max)*2/(|)2||1(| MMMVi +=• Normalised sum of the absolute value of the

motors speeds;• Whatever the robot does, it does quickly.

9

Straight Motion

=++=

=]7,6,5,1[,2))max)*2()21((1(

]4,3,2[,1iMMMi

Di

• It favours high positive speeds to bothmotors

• When the robot is not oriented to the target,D=1 avoids contradictory learning

Action

• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness

=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max0,)0()1(,max

iifSSAAielsegiandiifggiAAielsediandiifddi

AAi

t

t

t

t

≠=

= ∑=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt

TPi = total of steps executed by attitude i

AAi=action’s fitness at stept of attitude i

10

Action


=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max0,)0()1(,max


AAi

t

t

t

t

≠=

= ∑=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt



Rates the distance variation to the target between

two consecutive steps, and the maximum distance in one step,

for collision free/front

Action


=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max0,)0()1(,max


AAi

t

t

t

t

≠=

= ∑=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt



Rates the angle variation between two consecutive steps, and

the maximum angle in one step, for collision freeleft, right, back

11

Action


=−=≤∆=∆=≤∆=∆

=]7,6,5[,max1

0,)0(])4,3,2[(,max0,)0()1(,max


AAi

t

t

t

t

≠=

= ∑=

0,)(

0,0

1

TPiTPiAAi

TPiAi TPi

tt


AAi=action’s fitness at stept of attitude iIncreases as the distance

to the proximity-sensor increasesin the step

Improving the Target DirectionModel

Â2 - Â1 3π/4 π/4

Target at LEFT

Target Behind Target in FRONT

Target at RIGHT

-3π/4 - π/4

First Target Direction Model

4 possible target directions

0

π/4

π/2

3π/4

π/2

-3π/4 -π/4

-π/2

8 possible target directions

12

Chromosome Representations

Cada e s t ado co r re sponde a uma ún ica at i tude (par de velocidadesM1 e M2) , e c ada a t i t ude co r r e sponde a um gene do c romossoma . Target Direction (collision free) Obstacle detected

M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 Front Left Right Back Left Right Back

Chromosome with 7 attitude Genes

Target Direction (collision free) Obstacle detected

M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 Front Front Left Left Right Right Back Back Left Right Back left right front back front back left right

Chromosome with 11 attitude Genes

Genetic Algorithm

• Real number chromosome• Population Size =100• Generations = 50• Crossover Rate = 80 %• Mutation Rate = 4%• Roulette Wheel Reproduction• Elitism• Linear scaling of fitness• 300 Evaluation Steps for each chromosome• Average of 25 Experiments

13

Genetic Algorithm Performance7 Genes Chromosome

Best Chromosomesin 1 experiment

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

Number of Generations

FITN

ES

S


Average of Best Chromosomes in 25 experiments

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49


FITN

ESS

14


Best Chromosomesin 1 experiment

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49


FITN

ES

S


Average of Best Chromosomesin 25 experiments

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49


FIT

NE

SS

15

Paths Achieved in World 1Case Study 1SITUAÇÃO 1

CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome



16

Speed Comparison

0

1000

2000

3000

4000

5000

6000

1 2 3 4 5

Case Studies

Nu

mb

er

of

Ste

ps

11 Genes Chromosome

7 Genes Chromosome



17


CROMOSSOMA DE 7 GENES CROMOSSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome

0

500

1000

1500

2000

2500

3000

3500

4000

4500

1 2 3

Case Studies

Num

ber

of S

teps

Speed Comparison

11 Genes Chromosome

7 Genes Chromosome

18

84,62%

40,00%

214,29%

Speed Comparison (%)

Case Study 1

Case Study 2

Case Study 3



19





20



Speed Comparison

Speed Comparison – World 37 Genes Cromosome 11 Genes Cromosome

Case Study 1 target not reached 729Case Study 2 1124 618Case Study 3 target not reached target not reachedCase Study 4 target not reached 1655

21

Conclusions

• A simple GA could gradually evolve the robot control

• The robot achieved near optimal path towards the goal,avoiding obstacles

• Retraining is not necessary when the environment changes

• Controller improved performance with the 11 genes model

• The robot has no memory about previous unsuccessful pathsand may get lost

• Other tasks can be included in the model (e.g. energy supply)

• Chromosome codification is limited for few robot’s situations

Robot Control using Genetic Algorithms - UFSCmauro/ine5377/Cursos-ICA/CE-ga7_2.pdf · Robot Control...

Documents

Transcript of Robot Control using Genetic Algorithms - UFSCmauro/ine5377/Cursos-ICA/CE-ga7_2.pdf · Robot Control...