Approaches to Artificial Evolution

Verena Hamburger

Problems in engineering

• Simple designs are often not effective

• Highly sophisticated morphologies are hard to model

• Each motor action has various effects

• Much effort to make sensors highly precise measuring devices

• A lot of a priori and designer knowledge is needed

Benefits of evolutionary robotics (1)

• Overcoming the stiff human concepts

=> Truly new designs & innovative forms of sensory-motor-coordination

• Sensory-motor-systems acts as a whole in close coupling with the environment

=> An essential aspect of real cognition

Benefits of evolutionary robotics (2)

• Reduction of modelling process

=> Emphasis on behaviour analysis

• More, but simpler sensors

=> Non-linearities are exploited or amended

=> Sensors are combine to extract useful information

Open-ended evolution

An evolutionary process that leads to the

ongoing development of new traits

that tend

– to be retained for long evolutionary periods and

– to constitute important building blocks for further evolutionary stages.

Requirements to open-ended evolution (1)

Requirements according to Bianco and Nolfi:

● implicit and general selection criteria, but BII fitness functions may cause a “bootstrap problem”.

● favourable organisation of the evolving individuals, a good genotype-to-phenotype mapping is expressive, compact, autonomous & complete

● dynamically changing environmental conditions

Requirements to open-ended evolution (2)

Requirements according to Maley:

● Endogenous implementation of ecological niches

● Unbounded diversity during growth phase

● Selection must be embodied

● System must exhibit continuing (“positive”) new adaptive activity

Requirements to open-ended evolution (3)

Requirements according to Channon:

● Evolutionary emergence: The constant need to change one's model to keep up with a system’s behaviour.

● Natural selection: evolution as a result of the system dynamics is a prerequisite for evolutionary emergence (also Packard, Ray).

● Non-linear systems which do not obey the superposition principle (also Langton).

Stages of natural evolution

1. The creature's body plan changes

2. New sensors and actuators are explored

3. The environment changes (at least as

the organisms perceives it)

4. The nervous system adapts.

State of the ArtEvolution of complete agents are still quite restricted to:

– single tasks

– few basic shapes

– limited variety of sensors, actuators and materials

Some methods require a lot of human interference.

Modular evolution

● A simple body plan with two legs

● Robot drags itself along the ground● Development of multi-jointed legs ● ..... ● Quadruped with stable locomotion

=> Stages are induced by the designer(Muthuraman, MacLeod and Maxwell)

Evolving Neural Networks

• Support of various learning techniques• Resistant to noise • Biologically inspired • Real values for in- and output

• Graceful degradation

• Low level

• Evtl. flexible neuron model

• Recurrent NNs (internal state, rich intrinsic dynamics)

TPE learning in PNN (Hebb, post-/pre-synaptic, covariance)

• More complex skills (also sequential tasks)

• Fewer generations

• Adapting to environments never seen during evolution (even different morphology)

• Controllers transferred to reality quickly

(Floreano & Urzelai)

Evolution of learning

LAE by (Punctuated) Anytime learning

Algorithm receiving information from the real world to adapt off-board simulator or GA. Best solution is periodically sent to controller.

Parker and others: Punctuated anytime learning with “cyclic GA” for hexapod gaits:

•Incrementally evolving individual leg controllers

•Adapting to a different environments

•Evolving a simulated team of legged robots.

Importance of morphology

• Locomotion for ten legged agents with different body plans (Bongard and Pfeifer)

=> Shape and mass determine fast/slow/no success

=> Build up parameter directory

• Closed loop controller for stable biped walking (Paul and Bongard)

=> The more control over the weight distribution, the more stable gaits

Importance of environment

Fast locomotion for Sony Aibo:

• TPE on flat carpet did not generalise well to new surfaces

• TPE on uneven surface generalised well

but: it was tricky to get the level of unevenness right

(Hornby, Fujita, Takamura, Yamamoto, Hanagata)

Evolution with meta-model• Partial order via classification of genotype

• Classifier is evolved online

• Additional regular fitness based evaluation

=> Reduction of evaluation (~50%)

=> Less deterioration of the robot

=> False classification may lead to loss of best individual

(Jens Ziegler: GP for fast locomotion of Sony Aibo)

Variation of evolutionary parameters (1)

The performance of GA is sensitive to initial conditions

but: no quantitative methods are available

=> lower risk through parameter variation

Variation of evolutionary parameters (2)

• Classification, regression, robot control

• Comparing fitness expectation and variance after

200 generations

• Wilcoxon-Test (confidence niveau 0.95)

=> deterministic or adaptive variation with

mutation rate ↑ - crossover rate ↓(Jens Ziegler)

Architectures of variable complexity (1)

• Fixed architectures are unsuitable since complexity cannot be foreseen (fixation = limitation)

• Open-ended evolution gets interesting when problem-oriented evolution reaches equilibrium

• At equilibrium all individuals are similar

=> only mutation can bring forth new traits

Architectures of variable complexity (2)

• Mutation rate tied to whole genotype or single gene is both inappropriate

=> Mutation-lock of beneficial genes

• Mutation-locked genes evtl. still suboptimal

=> Mutation-lock immunises against major, but not against minor mutation

My personal idea

We need a methodology for

automatic modular (co-)evolution

of learning in

architectures of variable complexity– Mutation-lock

– Parameter variation in individual stages

– Evtl. meta-model for real world experiment

in dynamically changing environment