Grant Dick Department of Information Science, School of Business, University of Otago, Dunedin, NZ ...

Australasian Computational Intelligence Summer School, 2009

Spatially-Structured Evolutionary Computation

Grant DickDepartment of Information Science,

School of Business,University of Otago, Dunedin, NZ

http://www.otago.ac.nz/informationscience/

21/11/2009

2

Overview

• Basic evolutionary algorithm (EA) model• Spatial models of evolution• Theory of spatially-structured EAs (SSEAs)• SSEA examples and applications• Conclusion

21/11/2009 Australasian Computational Intelligence Summer School, 2009

Background

3

Basic Terms

• Individual – candidate solution to problem• Population – collection of individuals• Deme – subset of population

21/11/2009 4Australasian Computational Intelligence Summer School, 2009

5Australasian Computational Intelligence Summer School, 2009

Evolutionary Algorithms (EAs)

• Stochastic population-based search methods• “Generate and test” metaphor• Inspired by principles of neo-Darwinian

evolution:– Variation (e.g. mutation)– Fitness– Selection (“Survival of the fittest”)

21/11/2009


Basic EA Model

pop = generate and evaluate random populationwhile not done do

parents = selection(pop)gen = recombine(parents)evaluate(gen)pop = replacement(pop gen)

end while

21/11/2009


Concepts of Evolution

• Mutation• Selection:– Takeover time– Probability of fixation to mutant

• Genetic Drift:– “Drunkard’s Walk”– Fixation through chance in a finite population

21/11/2009

Populations in Space

• Populations evolve through time and space:– Spatial structure of populations often ignored

• Space divides a population and provides:– Clines (Gradients)– Local competition for resources– Reproductive isolation– Restricted gene flow

• Spatial structure important for species development21/11/2009 9Australasian Computational Intelligence Summer School, 2009


Spatial Models of Speciation

21/11/2009

Allopatric Parapatric Sympatric

Spatially-Structured Evolutionary

Algorithms (SSEAs)

Spatial Structure vs. Parallel Exec.

• Spatial structure does NOT imply parallel execution:– Parallel execution (e.g. Fitness evaluation)

needs no structure (e.g. master-slave model)• However, spatial structure provides logical

framework for parallelisation


SSEAs

• Inspired by models of biological evolution• Limit interactions between individuals

through topological constraints• Two main approaches:– Island models– Diffusion models

• Basic EA can be thought of as a special case SSEA (called a panmictic EA)


Island Models

Island Model EAs

• Multiple population extension to simple EA• Basic operators unchanged:– e.g. selection, recombination, mutation

• Introduction of migration operators



Island Model EAs

21/11/2009

Islands


Island Model EApop = generate and evaluate random populationwhile not done do

for each islandimmigrants = select_migrants(pop, topology)insert_immigrants(island, immigrants)

nextfor each island

/* perform simple EA iteration within island */next

end while

21/11/2009


Migration

21/11/2009


Migration

• Exchange of individuals between islands– Promote restoration of intra-island diversity

• Parameters:– Topology– Rate (how often, how many)– Selection strategy (e.g. best or random)– Replacement strategy– Copy or true exchange?

21/11/2009


Migration Topologies

21/11/2009

Fully-Connected Stepping Stone Directed


Punctuated Equilibrium

• Islands converge quickly:– Long periods of stasis

• Migration disrupts equilibrium:– Introduces diversity– Temporary drop in

fitness

• Disruption allows evolution to continue

21/11/2009


Island Model EAs

Benefits• Easily parallelised on

cluster computing platforms

• Can help prevent premature convergence

• Can implement local policies within islands:– e.g. Injection island EAs

Challenges• Many parameters:

– Topology– Migration rate– Migration size– Migration selection– Migration replacement

• Parameter interaction:– e.g. Little and frequent– e.g. Large and infrequent

21/11/2009

Diffusion Models

Diffusion Model EAs

• “Isolation by distance” extension to EAs• Basic operators unchanged:– e.g. selection, recombination, mutation

• Introduction of overlapping subpopulations:– a.k.a. demes or neighbourhoods



Diffusion Model EApop = generate and evaluate random populationwhile not done do

for each location in spacedeme = construct_deme(pop, location)parents = selection(deme)offspring = recombine(parents)evaluate(offspring)insertion(offspring, location)

nextend while

21/11/2009


Diffusion Model Topologies

• Regular (common):– e.g. Ring, ladder, torus

• Irregular (uncommon):– e.g. Scale-free, small-

world, random

• Single individual at each location:– Contrast with island

models

21/11/2009


Topology Representation

• Graph representation:– Locations as vertices– Edges relate “close” locations

• Matrix representation:– NxN matrix– Entry in matrix represent shared connection

between locations:• 1 = relationship present (connected)• 0 = no relationship (not connected)

21/11/2009


Topology Representation

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 150 1 1 0 1 1 0 0 0 0 0 0 0 1 0 0 01 1 1 1 0 0 1 0 0 0 0 0 0 0 1 0 02 0 1 1 1 0 0 1 0 0 0 0 0 0 0 1 03 1 0 1 1 0 0 0 1 0 0 0 0 0 0 0 14 1 0 0 0 1 1 0 1 1 0 0 0 0 0 0 05 0 1 0 0 1 1 1 0 0 1 0 0 0 0 0 06 0 0 1 0 0 1 1 1 0 0 1 0 0 0 0 07 0 0 0 1 1 0 1 1 0 0 0 1 0 0 0 08 0 0 0 0 1 0 0 0 1 1 0 1 1 0 0 09 0 0 0 0 0 1 0 0 1 1 1 0 0 1 0 0

10 0 0 0 0 0 0 1 0 0 1 1 1 0 0 1 011 0 0 0 0 0 0 0 1 1 0 1 1 0 0 0 112 1 0 0 0 0 0 0 0 1 0 0 0 1 1 0 113 0 1 0 0 0 0 0 0 0 1 0 0 1 1 1 014 0 0 1 0 0 0 0 0 0 0 1 0 0 1 1 115 0 0 0 1 0 0 0 0 0 0 0 1 1 0 1 1

21/11/2009

0 1 2

4 5 6

8 9 10

12 13 14

3

7

11

15

0

4

8

12

12 13 14 15


Demes (Neighbourhoods)

• Define boundaries of local interactions:– Selection/competition– Reproduction

• Each individual “belongs” to multiple demes

• Ratio between deme size and population size defines spatial characteristic

21/11/2009

0 1 2

4 5 6

8 9 10

12 13 14

3

7

11

15

Deme of location 10 = {6, 9, 10, 1, 14)(von Neumann neighbourhood)


Deme Construction

/* construct deme at location x *//* connectivity matrix M */deme = For each location y

If Mxy == 1 then deme = deme individualxy

Next

21/11/2009

31

Example Deme Structures (Ring)

21/11/2009 Australasian Computational Intelligence Summer School, 2009

D=1 D=2 D=3 D=6


Example Deme Structures (Torus)

21/11/2009

“Compact” “Diamond” “Linear”


Offspring Placement

• Offspring are placed within the population within the same deme as their parents

• Example strategies:– “Replace always” – offspring always replace

the individual that resides at the current location

– Elitism – offspring replace the current individual when they are strictly better, otherwise the current individual remains

21/11/2009

Properties ofDiffusion Models

More Basic Terms

• Gene – single coded piece of information• Locus – position of gene in individual• Allele – unique value for a gene• Ploidy – number of complete sets of genes• Homozygous – same alleles (by value) for a

gene at a given locus• Heterozygous – different alleles (by value)

for a gene at a given locus


Behavioural Changes of SSEAs

• Mutation – none, same as for simple EA• Genetic drift• Selection



Drift in SSEAs

• Local divergence:– Produces homogeneous subpopulations that

eventually grow/die out• Measurement of interest:– Time to loss of variation in population (fixation

time)

21/11/2009


Wright-Fisher Model of Drift

• Use Markov chain and first-step analysis– Compute the fundamental matrix of transition

probabilities• Assumption: populations identical by

proportion of alleles• Assumption: transition between states is

binomial:–

21/11/2009

pij= ( )Nj ( )N - i

NN-j( )i

Nj


Wright-Fisher Model and SSEAs

• Consider these two distinct populations:

• Wright-Fisher models assumes they are identical in terms of transition probability:– Need to model each unique population state!

21/11/2009


Extended Wright-Fisher Model

• Model transition from one unique (by placement) population to another:– Requires 2N 2N matrix (rather than N N)

• Gives exact model for behaviour:– Computationally prohibitive once N > 11

21/11/2009


Random Walk in Ring Structures

Raw fixation timeAfter subtracting “panmictic” fixation time

21/11/2009


Random Walk in Toroidal Structures

Raw fixation timeAfter subtracting “panmictic” fixation time

21/11/2009


Random-Walk Model(Ring Population Structure)

• Model drift behaviour on rings• Model drift as two components:– Contribution from “panmictic” fixation time– Contribution from “spatial effect”

• Time to fixation then becomes:– ring = panmixa + X

21/11/2009


Random Walk Model of Gene Flow

• Alleles enter and leave neighbouring demes

• Eventually a specific allele will enter the deme furthest away from its origin:– Distance b will be N/2– Call this an “absorbing

state”

• Model as a 1D random walk to absorbing boundary21/11/2009

Source: Dick & Whigham (2005), 2005 IEEE Congress on Evolutionary Computation , pp. 1855-1860


Drift with Multiple Loci

21/11/2009



Random Walk Model

21/11/2009



Selection in SSEAs – Takeover Time

• “The [...] time it takes for the single best individual to take over the entire population”

• SSEAs reduce selection intensity:– Quality of selection (i.e.

prob. of fixation) not measured.

21/11/2009

Source: Giaocobini et al. (2005), IEEE Transactions on Evolutionary Computation 9(5) , pp. 489-505


Probability of Mutant Fixation

• Assume that a mutant is introduced into population with fitness f=1+r (r [-1:1])– What is the chance that this mutant will fix in

the population?• Can be modelled through a Moran process:– Continuous birth/death model

21/11/2009


Moran Process

• Pick individual a at random (pa = 1/N)

• Pick individual b with probability proportional to fitness (pb = fb/f)

• Remove a from population

• Create duplicate of b

21/11/2009

ab

Source: Moran (1962), The Statistical Processes of Evolutionary Theory . Oxford, Clarendon Press


Moran Process in SSEAs

• Pick individual a at random (pa = 1/N)

• Construct deme around a• Pick individual b from

within deme with probability proportional to fitness (pb = fb/f)

• Replace a with b

21/11/2009

b

a

Source: Whigham and Dick (2008), Genetic Programming and Evolvable Machines 9(2) , pp. 157-170


Moran Process in SSEAs

21/11/2009

Source: Komarova (2006), Bulletin of Mathematical Biology 68, pp. 1573-1599


Behavioural aspects of SSEAs

• Selection quantifiably less but qualitatively similar to panmictic EA– Qualitative change minor (negligible?)

• Drift neither quantitatively nor qualitatively similar to panmictic EA– Drift promotes local divergence– May be useful for discovering multiple

solutions (more later ...)

21/11/2009

Applications of SSEAs

Classical Function OptimisationSource: Mühlenbein et al. (1991), Proc. 4th Intl. Conf. Genetic Algorithms, pp. 271-278



Multimodal Optimisation

• Problems may present several potential solutions:– Of same or varying fitness

• EAs typically return a single solution

• Ideally, EAs should return multiple useful solutions:– Within a single run!

21/11/2009


Global Multimodal Search Approach – Fitness Sharing

• Frequency-dependent selection:– e.g. Resource

contention

• Reduce fitness of common solutions

• Stabilise against selection and drift

21/11/2009


Multimodal Search using SSEAs

21/11/2009

Unequal Peaks

Equal Peaks


Parapatric Speciation Revisited

• Parapatric speciation results from:– Reproductive isolation through distance– Divergence through adaptation to local

environment

• Most SSEAs promote isolation by distance• Local environments largely ignored!

21/11/2009


Gradient-Based SSEAs

21/11/2009

Source: Dick & Whigham (2006), 6th Intl. Conf. Simulated Evolution and Learning , pp. 505-512



21/11/2009



Local Sharing Methods

• Incorporate fitness sharing within the demes of an SSEA:– Alternatively weight sh(dij) according to

topological distances between i and j• Control elitism through shared fitness• Introduce localised stabilising selection

pressure:– Help preserve lower-valued desired optima

21/11/2009



21/11/2009

Maximum Peak Ratio performance (higher=better)

Source: Dick & Whigham (2008), 7th Intl. Conf. Simulated Evolution and Learning , pp.452-461

Equal-Valued Peaks Function Unequal-Valued Peaks Function



21/11/2009

Chi-Square-Like performance (lower=better)

Source: Dick & Whigham (2008), 7th Intl. Conf. Simulated Evolution and Learning , pp.452-461

Equal-Valued Peaks Function Unequal-Valued Peaks Function


Multiobjective Optimisation

• Selection based on multiple (conflicting) objectives

• Aim: evolve a covering set of non-dominated solutions

21/11/2009


Gradient-Based EMO

• Apply gradient of objective weightings

• Fitness then becomes sum of individually weighted objectives:– f(i,x,y) = w1(x,y)o1(i)

+ w2(x,y)o2(i)

21/11/2009

10 w1

01 w2

Source: Kirley (2001), 2001 IEEE Congress on Evolutionary Computation , pp. 949-956


Algorithm 1 Algorithm 2 Comparison

T1MEA SPEA [100, 0]

MEA PAES [52.1, 0]

T4MEA SPEA [55.5, 0]

MEA PAES [96.3, 0]

MEA – Gradient-Based EMO

21/11/2009

Source: Kirley (2001), 2001 IEEE Congress on Evolutionary Computation , pp. 949-956


Bloat Control in Genetic Programming

21/11/2009


Bloat Control via SSEAs

• Bloat is increase in program size:– Without corresponding fitness increase

• Elitist replacement known to reduce bloat:– But elitism stagnates search in panmictic EAs

• Inbreeding thought to reduce bloat:– Small demes in SSEAs promote inbreeding

• Solution: Spatial Structure + Elitism (SS+E)

21/11/2009



21/11/2009

Source: Whigham and Dick. (TBA), IEEE Transactions on Evolutionary Computation (in press)


Parameter Control via SSEAs

• Terrain-based genetic algorithm (TBGA):– Gradient of parameter

space

• Each region specifies unique combination of crossover/mutation rate

• Hopefully, the ideal combination ofparameters is present in terrain

21/11/2009

2

1

3

6

4

5

7

8

0.050.00 0.10 0.250.15 0.20 0.30 0.35

Mutation Prob.#

Cros

sove

r Po

ints

Mutation rate = 0.15# crossover points = 6

Source: Gordon et al. (1999), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 1999) , pp. 229-235


Predator-Prey SSEAs

21/11/2009

• Sparsely populated space:– Individuals “roam” the

landscape

• Two “species”:– Predators, for selection– Prey, candidate solutions

for given problem


Predator-Prey SSEAs

21/11/2009

Real-parameter function optimisation

Source: Li & Sutherland (2002), 4th Intl. Conf. Simulated Evolution and Learning , pp.76-80


Predator-Prey SSEAs

21/11/2009

Multiobjective optimisation (predators targetspecific objectives, thus promoting coverage)

Source: Li (2003), Evolutionary Multi-Criterion Optimization, Second International Conference (EMO 2003) , pp. 207-221

Conclusion

Recommendations for SSEAs

• Torus topology good in general• Use strictly better elitism in all cases• Local selection method not so important• Use crossover probability of 1.0• Increase mutation probability slightly

compared to panmictic EA



Possible Research Areas in SSEAs

• Drift/Selection in Non-Linear Structures• Scalability of Gradient-Based SSEAs to

higher-dimension problems• Applicability of alternative population

structures:e.g. scale-free

• Implementation of SSEAs on GPUs

21/11/2009


Thank you

Questions?

[email protected]://www.otago.ac.nz/informationscience/

21/11/2009

Grant Dick Department of Information Science, School of Business, University of Otago, Dunedin, NZ ...

Documents

Transcript of Grant Dick Department of Information Science, School of Business, University of Otago, Dunedin, NZ ...