Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer...

Evolutionary Computation: Evolutionary Computation: A TutorialA Tutorial

September 2009September 2009

Biointelligence Laboratory

School of Computer Sci. & Eng.

Seoul National University

http://bi.snu.ac.kr/

ContentsContents

Part I: Introduction to Evolutionary Computation Part II: Genetic Programming Part III: Advanced Topics Part IV: Applications Part V: Summary and Further Info

Part I: Introduction to Evolutionary Part I: Introduction to Evolutionary ComputationComputation

OverviewOverview

The Basic Concept General Framework of Evolutionary Computation Components of Evolutionary Computation Paradigms in Evolutionary Computation

Basic ConceptBasic Concept

Use of Darwinian-like evolutionary processes to solve difficult computational problems. Hence the name, “Evolutionary Computation”

Biological basis Biological systems adapt themselves to a new environment by

evolution. Generations of descendants are produced that perform better than do

their ancestors.

Biological evolution Production of descendants changed from their parents Selective survival of some of these descendants to produce more

descendants

Basic ConceptBasic Concept

What is the Evolutionary Computation? Stochastic search (or problem solving) techniques that

mimic the metaphor of natural biological evolution.

Metaphor

EVOLUTION

Individual

Fitness

Environment

PROBLEM SOLVING

Candidate Solution

Quality

Problem

General Framework of ECGeneral Framework of EC

Generate Initial PopulationGenerate Initial Population

Evaluate FitnessEvaluate Fitness

Select ParentsSelect Parents

Generate New OffspringGenerate New Offspring

Termination Condition?Termination Condition?Yes

Fitness Function

Crossover, Mutation

Best Individual

Components of EC (1/9)Components of EC (1/9)

Representations Population Fitness function Selection mechanism Variation operators

Crossover/Mutation

Initialization / Termination

An example: the 8 queens problem Place 8 queens

on an 8x8 chessboard in such a way that they cannot check each other.

Representations How to represent the space to be searched? Genotype

Internal representation of solutions in EC. Minimum domain knowledge.

Phenotype External representation of solutions for the problem. Require domain knowledge.

Be careful so that every feasible solution can be represented in genotype space.

1 23 45 6 7 8

Genotype: a permutation of the numbers 1 - 8

Phenotype: a board configuration

Obvious mapping

Representation Example:Representation Example:8 Queens Problem8 Queens Problem

Components of EC (4/9)Components of EC (4/9) Population

Usually has a fixed size and is a multiset of genotypes Some sophisticated EAs also assert a spatial structure on the

population e.g., a grid. Diversity of a population refers to the number of different

fitnesses / phenotypes / genotypes present (note not the same thing).

Population sizing Parent population size (M), offspring population size (K). Typically M = K. In typical ES (explained later), M << K. In steady state algorithms, M > K.

Fitness function Represents the requirements that the population should

adapt to a.k.a. quality function or objective function Assigns a single real-valued fitness to each phenotype

which forms the basis for selection So the more discrimination (different values) the

better Typically we talk about fitness being maximised

Some problems may be best posed as minimisation problems, but conversion is trivial.

Fitness Function Example:Fitness Function Example:8 Queens Problem8 Queens Problem

Penalty for a queen: The number of queens she can

Penalty for a configuration: The sum of penalties of all

queens

Note: penalty is to be minimized

Fitness of a configuration: inverse penalty to be maximized

1 23 45 6 7 8

1.022110112

Parent selection mechanism Assigns probabilities for an individual to be selected as parents. Selection probabilities are relative to current population.

Different probabilities can be assigned to the same individuals. Usually depends on the individual’s fitness and probabilistic.

High quality solutions more likely to become parents than low quality ones

Selection pressure – the degree of correlation between the individual’s fitness and its selection probability.

High selection pressure results in reducing search scope. Even worst in current population usually has non-zero probability of

becoming a parent This stochastic nature can aid escape from local optima.

Selection Mechanism Example:Selection Mechanism Example:8 Queens Problem8 Queens Problem

1 23 45 6 8 7

6 83 17 5 2 4

6 31 25 7 8 4

8 26 54 1 3 7

1/9 = 0.11

1/10 = 0.1

1/11 = 0.091

1/6 = 0.17

Individual Proportional Ranking

(1) 0.23 0.3

(2) 0.21 0.2

(3) 0.19 0.1

(4) 0.36 0.4

Different selection mechanisms assign different selection probabilities for the same individual.

Components of EC (7/9)Components of EC (7/9) Crossover (Recombination)

Mix information from parents into offspring in stochastic way.

Most offspring may be worse, or the same as the parents. Hope is that some are better by combining elements of

genotypes that lead to good traits. Principle has been used for millennia by breeders of

plants and livestock

Example

8 7 6 42 5311 3 5 24 678

8 7 6 45 1231 3 5 62 874

Mutation It is applied to one genotype and delivers a (slightly)

modified mutant, the child or offspring of it. Element of randomness is essential. The role of mutation in EC is different in various EC

subtypes.

Example swapping values of two randomly chosen positions

1 23 45 6 7 8 1 23 4 567 8

Initialization usually done at random Need to ensure even spread and mixture of possible

allele values Can include existing solutions, or use problem-specific

heuristics, to “seed” the population Termination condition checked every generation

Reaching some (known/hoped for) fitness Reaching some maximum allowed number of

generations Reaching some minimum level of diversity Reaching some specified number of generations

without fitness improvement

Paradigms in ECParadigms in EC

Evolutionary Programming (EP) [L. Fogel et al., 1966] FSMs, mutation only, tournament selection

Evolution Strategy (ES) [I. Rechenberg, 1973] Real values, mainly mutation, ranking selection

Genetic Algorithm (GA) [J. Holland, 1975] Bitstrings, mainly crossover, proportionate selection

Genetic Programming (GP) [J. Koza, 1992] Trees, mainly crossover, proportionate selection

Appendix: Evolution StrategyAppendix: Evolution Strategy

Evolution Strategy (ES)Evolution Strategy (ES)

Problem of real-valued optimizationFind extremum (minimum) of function F(X): Rn ->R

Operate directly on real-valued vector X Generate new solutions through Gaussian

mutation of all components Selection mechanism for determining new parents

ES: RepresentationES: Representation

One individual:

The three parts of an individual:

Ixxa nnn

,,,,,,,, 111

: Object variables

: Standard deviations

: Rotation angles

Fitness

Variances

Covariances

, where rx, r , r {-, d, D, i, I, g, G}, e.g. rdII rrr x

GiSiTiiS

giSiTiS

IiSiTiS

iiSiTiS

teintermedia dgeneralize panmictic)(

teintermedia dgeneralize)(

teintermedia panmictic2/)(

teintermedia2/)(

discrete panmicticor

discreteor

ionrecombinat no

ES: Operator - RecombinationES: Operator - Recombination

m{,’,} : I I is an asexual operator. n = n, n = n(n-1)/2

1 < n < n, n = 0

n = 1, n = 0

)),(,0(

))1,0()1,0(exp(

0873.0

))1,0()1,0(exp(

))1,0(exp( 0

iii Nxx

ES: Operator - MutationES: Operator - Mutation

ES: Illustration of Mutation ES: Illustration of Mutation HyperellipsoidsHyperellipsoids

Line of equal probability density to place an offspring

ES: Evolution Strategy vs. Genetic ES: Evolution Strategy vs. Genetic AlgorithmAlgorithm

Create random initial population

Evaluate population

Select individuals for variation

Insert intopopulation

Create random initial population

Evaluate population

Selection

Insert intopopulation

Part II: Genetic ProgrammingPart II: Genetic Programming

OverviewOverview

Genetic Programming Tree-based Representations Setting Up a Genetic Programming Run Example: Wall-Following Robot Result

Genetic Programming (GP)Genetic Programming (GP)

GP is an domain-independent method that evolves a population of programs to solve a problem.

GP uses variable-size tree-representations rather than fixed-length strings of binary values in typical GA.

GP has been successful in many domains.

Tree-based Representations (1/6)Tree-based Representations (1/6)

Function set: internal nodes Functions, predicates, or actions which take one or

more arguments

Terminal set: leaf nodes Program constants, actions, or functions which take no

arguments

S-expression: (+ 3 (/ ( 5 4) 7))

Terminals = {3, 4, 5, 7}Functions = {+, , /}

Trees can represent: Arithmetic formula

Logical formula

Program

And many others depending on the definition of terminals and nonterminals.

15)3(2

(x true) (( x y ) (z (x y)))

i =1;while (i < 20){

i = i +1}

15)3(2

(x true) (( x y ) (z (x y)))

i =1;while (i < 20){

i = i +1}

In GA, ES, EP chromosomes are linear structures (bit strings, integer string, real-valued vectors, permutations)

Tree shaped chromosomes are non-linear structures.

In GA, ES, EP the size of the chromosomes is fixed.

Trees in GP may vary in depth and width.

Setting up a GP RunSetting up a GP Run

Users need to specify:1. The terminal / function set

2. The fitness measure

3. Algorithm parameters to control the run Population size, maximum number of generations Selection / variation operators and their probabilities The termination criterion Maximum depth of a GP tree, etc.

Example: Wall-Following Robot Example: Wall-Following Robot (Step 1)(Step 1) Program Representation in GP

Functions AND (x, y) = 0 if x = 0; else y OR (x, y) = 1 if x = 1; else y NOT (x) = 0 if x = 1; else 1 IF (x, y, z) = y if x = 1; else z

Terminals Actions: move the robot one cell to each direction

{north, east, south, west} Sensory input: its value is 0 whenever the coressponding cell is

free for the robot to occupy; otherwise, 1. {n, ne, e, se, s, sw, w, nw}

Program ExampleProgram Example

Example: Wall-Following Robot Example: Wall-Following Robot (Step 1)(Step 1) Be careful when specifying function / terminal set.

To avoid syntactic error All programs in the initial population should be valid,

executable programs. The genetic operators during the run should produce valid,

executable programs as offspring.

To avoid run-time error All functions can take any terminal or the results produced by

any other functions as input.

Example: Wall-Following Robot Example: Wall-Following Robot (Step 2)(Step 2) Fitness measures can be

Error between program’s output and the desired output. Accuracy in recognizing patterns or classifying objects into

classes. Payoff a game-playing program produces.

Often each program is executed over a representative sample of fitness cases.

In our wall-following robot problem the number of cells next to the wall that are visited during 60

steps over 10 independent runs. Perfect score (320) : One Run (32) 10 randomly chosen starting points

Example: Wall-Following Robot Example: Wall-Following Robot (Step 3)(Step 3) Genetic operators – crossover (subtree exchange)

a a b+

Example: Wall-Following Robot Example: Wall-Following Robot (Step 3)(Step 3) Genetic operators – mutation

Example: Wall-Following Robot Example: Wall-Following Robot (Step 3)(Step 3) Population size: 5,000 Termination condition: found perfect solution Creating Next Generation

500 programs (10%) are copied directly into next generation (elitism). Tournament selection

• 7 programs are randomly selected from the population 5,000.• The most fit of these 7 programs is chosen.

4,500 programs (90%) are generated by crossover. Parents are each chosen by tournament selection.

In this example, mutation was not used.

Example: Wall-Following Robot Example: Wall-Following Robot (Step 3)(Step 3)

Result (1/5)Result (1/5) Generation 0

The most fit program (fitness = 92) Starting in any cell, this program moves east until it reaches a

cell next to the wall; then it moves north until it can move east again or it moves west and gets trapped in the upper-left cell.

Result (2/5)Result (2/5)

Generation 2 The most fit program (fitness = 117)

Smaller than the best one of generation 0, but it does get stuck in the lower-right corner.

Following the wall perfectly but still gets stuck in the bottom-right corner.

Following the wall around clockwise and moves south to the wall if it doesn’t start next to it.

Result (5/5)Result (5/5) Fitness Curve

Fitness as a function of generation number The progressive (but often small) improvement from

generation to generation

Part III: Advanced Topics and Part III: Advanced Topics and ApplicationsApplications

OverviewOverview

Theoretical topics Mathematical Models of Simple Genetic Algorithms Bloat Phenomena in Genetic Programming

Applications Summary

Mathematical Models of Simple GAs (1/7)Mathematical Models of Simple GAs (1/7)

Based on (Vose & Liepins, 1991). The model simple GA

1. Start with a random population of binary strings of length l.

2. Calculate the fitness f(x) of each string x in the population.

3. Choose (with replacement) two parents from the population with probability proportional to each string’s relative fitness in the population.

4. Crossover the two parents (at single random point) with probability pc to from two offspring and discard one at random.

5. Mutate each bit in the selected offspring with probability pm and replace it in the new population.

6. Go to Step 3 until new population is complete.

7. Go to Step 1.

Notations Each string is represented by binary string between 0

and 2l-1.

)6667.0,1667.0,1667.0,0()(

)5.0,25.0,25.0,0()(

tpProportion of string i in the population

The selection probability of string i,assuming that fitness is equal to the number of 1’s in the string.

)()( l

GA with selection only. Expectation of next population is equal to the selection

probabilities.)())1(( tstpE

)(~))1((

)1(~)1(

tsFtsE

Including crossover and mutation. Define recombination matrix M for crossover and

mutation. ri,j(k) – the probability that string k will be produced,

given that string i and j are selected to mate.

Defining ri,j(k) and M is tricky. We will show defining ri,j(0) only.

jijik krtststpE,

, )()()())1((

Defining ri,j(0) Case 1: crossover does not occur and the selected

offspring is mutated to all zeros. Case 2: crossover occurs and the selected offspring is

mutated to all zeros.

Case 1 |i|: the number of 1’s in a string i. The probability that i be mutated to all zeros

|||| )1( ilm

|||||||| )1()1(12

imc ppppp

Case 2 Temporary definition

|h|=|i|-|i2|+|j2| & |k|=|j|-|j2|+|i2|.

But, gives the number of 1’s in rightmost c bits of i.

i: 0 0 0 1 0 1 0 1

j: 1 1 1 0 1 0 1 0

h: 0 0 0 0 1 0 1 0

k: 1 1 1 1 0 1 0 1

|||||||| )1()1(1

hmc pppp

ii c )^12(2 i: 0 0 0 0 1 0 1 0

25-1: 0 0 0 1 1 1 1 1

cjicji

and Then,

)^12()^12(

)1/( mm pp

)1()0(

cjicji

For the rest of derivation of ri,j(0) and M, refer to (Vose & Liepins, 1991).

Bloat Phenomena in GP (1/3)Bloat Phenomena in GP (1/3)

Bloat Program growth without (significant) return in terms of

fitness.

Fitness

Program size

Why does it happen? No certain explanation. Replication accuracy theory

GP evolves towards (bloated) representations that increase replication accuracy.

Crossover bias theory Short programs are less fit than long ones long programs

have a selective advantage bloat

Techniques limiting bloat Fixed size or depth limit: programs exceeding the limit

are discarded and a parent is kept. Dangerous: acts as advantage for programs that violate the

limit.

Parsimony pressure: a penalty term for larger programs is added to the fitness value.

(Fitness_parsimony) = (Fitness_raw) – c * (program_size)

Part IV: ApplicationsPart IV: Applications

Applications of ECApplications of EC

Numerical, Combinatorial Optimization System Modeling and Identification Planning and Control Engineering Design Data Mining Machine Learning Artificial Life

Co-evolving Soccer Softbots (1/5)Co-evolving Soccer Softbots (1/5)

At RoboCup there are two "leagues": the "real" robot league and the "virtual" softbot league

How do you do this with GP? GP breeding strategies: homogeneous and heterogeneous Decision of the basic set of function with which to evolve players Creation of an evaluation environment for our GP individuals

Application Example 1Application Example 1

Co-evolvingCo-evolving Soccer Softbots Soccer Softbots With Genetic With Genetic ProgrammingProgramming

Initial Random Population

Kiddie Soccer

Learning to Block the Goal

Becoming Territorial

Evolutionary ArtsEvolutionary Arts

Exploits the process of evolution to create an artwork according to an evolutionary algorithm.

Selection may be done by the viewer explicitly by selecting individuals which are aesthetically pleasing, or implicitly according to the length of time a viewer spends near a piece of evolving art.

EvoMUSART workshops 2009 http://evostar.na.icar.cnr.it/EvoWorkshops/

EvoMUSART/EvoMUSART.html

Part V: Summary & Further Part V: Summary & Further InformationInformation

Recapitulation of EARecapitulation of EA

EAs fall into the category of “generate and test” algorithms.

They are stochastic, population-based algorithms. Variation operators (recombination and mutation)

create the necessary diversity and thereby facilitate novelty.

Selection reduces diversity and acts as a force pushing quality.

Fundamental trade-off between exploration and exploitation

Exploration vs. ExploitationExploration vs. Exploitation

There are many possibilities to set the parameters to increase the explorative capability of an EA Increase variation probability

Increase mutation rate Increase crossover rate Increase population size

Decrease selection pressure Add noise to fitness function Add noise in selection Increase rank ratio in ranking selection Decrease tournament size in tournament selection

Important is the combined effect of various parameter settings.

Typical Behavior of an EATypical Behavior of an EA

Phases in optimizing on a 1-dimensional fitness landscape

Early phase:

quasi-random population distribution

Mid-phase:

population arranged around/on hills

Late phase:

population concentrated on high hills

Are Long Runs Beneficial?Are Long Runs Beneficial?

Time (number of generations)

Progress in 1st half

Progress in 2nd half

Answer: - It depends how much you want the last bit of progress - It may be better to do more shorter runs

Scale of “all” problems

Random search

Special, problem tailored method

Evolutionary algorithm

EAs as Problem SolversEAs as Problem Solvers

Advantages of ECAdvantages of EC

No presumptions w.r.t. problem space Widely applicable Low development & application costs Easy to incorporate other methods Solutions are interpretable (unlike NN) Can be run interactively, accommodate user

proposed solutions Provide many alternative solutions

Disadvantages of ECDisadvantages of EC

No guarantee for optimal solution within finite time

Weak theoretical basis May need parameter tuning Often computationally expensive, i.e. slow

Further Information on ECFurther Information on EC

Conferences IEEE Congress on Evolutionary Computation (CEC) Genetic and Evolutionary Computation Conference (GECCO) Parallel Problem Solving from Nature (PPSN) Foundation of Genetic Algorithms (FOGA) EuroGP, EvoCOP, and EvoWorkshops Int. Conf. on Simulated Evolution and Learning (SEAL)

Journals IEEE Transactions on Evolutionary Computation Evolutionary Computation Genetic Programming and Evolvable Machines

ReferencesReferences

Evolutionary Computation by Kenneth A. De Jong, 2006. Evolutionary Computation: Toward a New Philosophy of Machine Intel

ligence, Third Edition (IEEE Press Series on Computational Intelligence) by David B. Fogel, 2005,

Introduction to Evolutionary Computing by A. E. Eiben and J. E Smith, Springer, 2003.

Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems) by John R. Koza, 1992.

Genetic Programming: An Introduction by W. Banzhaf, P. Nordin, R. Keller, and F. Francone, Morgan Kaufmann, 1997.

A Field Guide to Genetic Programming by R. Poli and W. B. Langdon, 2008. Online free book – http://www.gp-field-guide.org.uk/

Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer...

Documents

Transcript of Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer...

Ch 4. Linear Models for Classification Adopted from Seung-Joon Yi Biointelligence Laboratory, Seoul National University

PT TOMAS Total Management Solution DiponED BioIntelligence LLP.

Biointelligence Lab School of Computer Sci. & Eng. Seoul National University Artificial Intelligence Chapter 1 Introduction.

Artificial Intelligence Chapter 21 The Situation Calculus Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.

Seoul int'l culinary academy (seoul culinary 2012)

Ch 13. Sequential Data (1/2) Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Summarized by Kim Jin-young Biointelligence Laboratory, Seoul.

Ch 8. Graphical Models Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Summarized by B.-H. Kim Biointelligence Laboratory, Seoul National.

Artificial Neural Networks Biointelligence Laboratory Department of Computer Engineering Seoul National University.

Artificial Intelligence Chapter 13 The Propositional Calculus Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.

Artificial Intelligence Chapter 16 Resolution in the Predicate Calculus Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.

Ch 9. Mixture Models and EM Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Biointelligence Laboratory, Seoul National University

FIRST Technical Colloquium Seoul - FIRST - … Technical Colloquium Seoul November 17-18, 2015 Somerset Palace Seoul Image from Seoul Tourism Organization Welcome to the FIRST TC Seoul

Ch 7. Sparse Kernel Machines Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Summarized by S. Kim Biointelligence Laboratory, Seoul National.

Artificial Intelligence Chapter 24 Communication among Agents Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.

AUTUMN SEOUL SEOUL 5 วัน 3 คืน · AUTUMN SEOUL SEOUL 5 วัน 3 คืน • ชมวิวบน สะพาน Gamaksan Suspension Bridge • สัมผัสบรรยากาศ

Online Evolutionary Collaborative Filtering RECSYS 2010 Intelligent Database Systems Lab. School of Computer Science Engineering Seoul National University.

The Biointelligence Explosion...The Biointelligence Explosion How recursively self-improving organic robots will modify their own source code and bootstrap our way to full-spectrum

Artificial Intelligence Chapter 6 Robot Vision Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.

Artificial Intelligence Chapter 25 Agent Architectures Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.

The Biointelligence Explosion