Genetic ProgrammingGenetic Programming
Using Simulated Natural Selection to Using Simulated Natural Selection to Automatically Write ProgramsAutomatically Write Programs
Genetic ProgrammingGenetic Programming
John Koza, Stanford UniversityJohn Koza, Stanford University Principal proponent of GPPrincipal proponent of GP Has obtained human-competitive results in a Has obtained human-competitive results in a
number of problem domainsnumber of problem domains Reproduced existing patentsReproduced existing patents Created new patentable designsCreated new patentable designs
Has written extensively on GPHas written extensively on GP Four volume set on Genetic ProgrammingFour volume set on Genetic Programming Numerous papers on the GPNumerous papers on the GP
Genetic ProgrammingGenetic Programming
Basic AlgorithmBasic Algorithm Create a population of programsCreate a population of programs Each program attempts to solve a set of problems Each program attempts to solve a set of problems
in a “training set.”in a “training set.” Program fitness is determined by success in Program fitness is determined by success in
solving training setsolving training set More fit members have better chance to produce More fit members have better chance to produce
offspring in the next generationoffspring in the next generation Offspring are produced using some form of Offspring are produced using some form of
crossovercrossover
Tree Structure of Genetic ProgramsTree Structure of Genetic Programs
Various structures are used to represent Various structures are used to represent genetic programs, but tree structures are the genetic programs, but tree structures are the most well known.most well known. Nonterminal nodes are functions that take their Nonterminal nodes are functions that take their
children as parameters.children as parameters.
+
21
Tree Structure Tree Structure
Terminal Nodes, the nodes that make up the Terminal Nodes, the nodes that make up the leaves of a program tree, provide data to the leaves of a program tree, provide data to the program.program. ConstantsConstants Parameterless functionsParameterless functions InputsInputs
Genetic Program ComponentsGenetic Program Components
Terminal SetTerminal Set Work as set of primitive data typesWork as set of primitive data types
ConstantsConstants Parameterless functionsParameterless functions Input ValuesInput Values
Function setFunction set Set of available functionsSet of available functions Often tailored specifically for the needs of the Often tailored specifically for the needs of the
program domain.program domain.
Initializing the PopulationInitializing the Population
The following two parameters are specifiedThe following two parameters are specified Maximum depth of a program treeMaximum depth of a program tree Maximum number of nodes in a program treeMaximum number of nodes in a program tree
Three methods in common use (Koza)Three methods in common use (Koza) FullFull
Nonterminals are used to build a complete tree up to the Nonterminals are used to build a complete tree up to the leaf nodes, which are then completely populated with leaf nodes, which are then completely populated with terminals. Every tree is grown to maximum depth and terminals. Every tree is grown to maximum depth and has the maximum number of nodes allowed.has the maximum number of nodes allowed.
Initializing the Population Initializing the Population (continued)(continued)
Three methods in common use (Koza)Three methods in common use (Koza) GrowGrow
The root node is chosen from the function setThe root node is chosen from the function set All nodes not at maximum depth are chosen randomly.All nodes not at maximum depth are chosen randomly.
Growth for a branch ends when a terminal is chosen.Growth for a branch ends when a terminal is chosen. Trees can have irregular shapes.Trees can have irregular shapes.
Nodes at the maximum depth are chosen from the Nodes at the maximum depth are chosen from the terminal set only.terminal set only.
Initializing the Population Initializing the Population (continued)(continued)
Three methods in common use (Koza)Three methods in common use (Koza) Ramped Half and HalfRamped Half and Half
M is the max depth of deepest partition in the M is the max depth of deepest partition in the populationpopulation
The population is separated into M partitionsThe population is separated into M partitions The ith partition, (i ranges from 0 to M-1) has a max depth of The ith partition, (i ranges from 0 to M-1) has a max depth of
M – i.M – i.
Half of each partition is populated with grow, the other Half of each partition is populated with grow, the other half is populated with full.half is populated with full.
Genetic Operators: CrossoverGenetic Operators: Crossover
CrossoverCrossover Randomly select a node in the motherRandomly select a node in the mother Randomly select a node in the fatherRandomly select a node in the father Swap the two nodes along with their subtreesSwap the two nodes along with their subtrees
Crossover ExampleCrossover Example+
1
*
2
-
2
-
13
/
4
power
abs2
-7
Parent 1 Parent 2
+
*
2
-
2
-
power
abs2
-7
1
13
/
4
Child 1 Child 2
Genetic Operations: MutationGenetic Operations: Mutation
MutationMutation Randomly select a node in the program treeRandomly select a node in the program tree Remove that node and its subtreeRemove that node and its subtree Replace the node with a new subtree, using the Replace the node with a new subtree, using the
same method used to initially instantiate the same method used to initially instantiate the population.population.
Typically, mutation is applied to a small number of Typically, mutation is applied to a small number of offspring after crossover. offspring after crossover.
Mutation ExampleMutation Example+
1 3
*
2 4
+
+
1
*
2
Left subtree is randomly selected for mutation.
-
*2
7 4
The entire subtree is replaced
Fitness-based SelectionFitness-based Selection
Gives “graded and continuous feedback about Gives “graded and continuous feedback about how well a program performs on the training how well a program performs on the training set” (Banzhaf et. al.)set” (Banzhaf et. al.) Standardized FitnessStandardized Fitness
Fitness scores are transformed so that 0 is the fitness of Fitness scores are transformed so that 0 is the fitness of the most fit member.the most fit member.
Normalized FitnessNormalized Fitness Fitness is transformed to values that always are between Fitness is transformed to values that always are between
0 and 1.0 and 1.
Different Selection AlgorithmsDifferent Selection Algorithms
GA ScenarioGA Scenario Same as that used in Genetic AlgorithmsSame as that used in Genetic Algorithms
Create gene pool by selecting parents based on fitnessCreate gene pool by selecting parents based on fitness Next generation completely replaces current generationNext generation completely replaces current generation
ES ScenarioES Scenario Same as used in Evolutionary StrategiesSame as used in Evolutionary Strategies
Generate children firstGenerate children first Apply fitness function to parents and childrenApply fitness function to parents and children Select the next generation from children (and possibly parents too)Select the next generation from children (and possibly parents too) Selection pressure can be tuned by adjusting the ratio of the Selection pressure can be tuned by adjusting the ratio of the
number of offspring to the number of parents.number of offspring to the number of parents.
Selection PressureSelection Pressure
Ratio of the best individual’s selection Ratio of the best individual’s selection probability to the average selection probabilityprobability to the average selection probability
MostFitSelectionProbability / MostFitSelectionProbability / AverageFitSelectionProbabilityAverageFitSelectionProbability
The larger this ratio, the greater the selection The larger this ratio, the greater the selection pressure.pressure.
Sample Fitness MeasuresSample Fitness Measures
Error FitnessError Fitness The sum of the absolute value of the differences between The sum of the absolute value of the differences between
the computed result and the desired result.the computed result and the desired result.
||1
i
n
iip opf
Where:
fp is the fitness of the pth individual in the population
oi is the desired output for the ith example in the training set
pi is the output from the pth individual on the ith example in the training set
* Squaring the expressing (pi-oi) can provide larger penalties for errors.
Fitness Measures can be as Varied as Fitness Measures can be as Varied as the Applicationsthe Applications
ExamplesExamples Number of correct solutionsNumber of correct solutions Number of wins competing against other members Number of wins competing against other members
of the population.of the population. Number of errors navigating a mazeNumber of errors navigating a maze Time required to solve a puzzleTime required to solve a puzzle
Truncation or (Truncation or (µ, µ, λλ) Selection) Selection
A number of parents (A number of parents (µ) are allowed to breed µ) are allowed to breed and produce (and produce (λλ) children. The µ best children ) children. The µ best children are used to produce the next generation.are used to produce the next generation. A variation, A variation, ((µ + µ + λλ) selection includes the parents ) selection includes the parents
in those considered for selection into the next in those considered for selection into the next generation. generation.
Ranking SelectionRanking Selection
Selection Based on Fitness OrderSelection Based on Fitness Order The members of the population are ranked from The members of the population are ranked from
best to worst.best to worst. The selection probability is assigned based on the The selection probability is assigned based on the
rank.rank.
Tournament SelectionTournament Selection
Select a subset of the population (the Select a subset of the population (the tournament size) randomly.tournament size) randomly.
More fit (winning) individuals are used to More fit (winning) individuals are used to generate replacements for less fit (losing) generate replacements for less fit (losing) individuals.individuals. Accelerates processing time (compared with full Accelerates processing time (compared with full
competition)competition) Facilitates parallel processingFacilitates parallel processing
The Basic GP Algorithm (from The Basic GP Algorithm (from Banzhaf, et. al)Banzhaf, et. al)
Define the terminal setDefine the terminal set Define the function setDefine the function set Define the fitness functionDefine the fitness function Define parameters such as population size, Define parameters such as population size,
maximum individual size, crossover maximum individual size, crossover probability, selection method, and termination probability, selection method, and termination criterioncriterion
Generational GPGenerational GP
Like what we have seen in GALike what we have seen in GA New generation completely replaces the previous New generation completely replaces the previous
generation.generation. Initialize the populationInitialize the population Evaluate the individual programsEvaluate the individual programs Until a new population is fully populated, repeatUntil a new population is fully populated, repeat
Select an individual or individuals in the population using selection Select an individual or individuals in the population using selection algorithmalgorithm
Perform genetic operations on the selected individual or individualsPerform genetic operations on the selected individual or individuals Insert the result of the genetic operations into the new populationInsert the result of the genetic operations into the new population
Best individual is the resulting program.Best individual is the resulting program.
Steady State GPSteady State GP
There are no generationsThere are no generations1.1. Initialize the populationInitialize the population
2.2. Randomly choose a subset of the population to take part Randomly choose a subset of the population to take part in the tournamentin the tournament
3.3. Evaluate the fitness value of each competitor in the Evaluate the fitness value of each competitor in the tournament.tournament.
4.4. Select the winner or winners from the competitors in the Select the winner or winners from the competitors in the tournament using the selection algorithm.tournament using the selection algorithm.
5.5. Apply genetic operators to the winner or winners of the Apply genetic operators to the winner or winners of the tournamenttournament
Steady State GP (continued)Steady State GP (continued)
6.6. Replace the losers in the tournament with the results Replace the losers in the tournament with the results of the application of the genetic operators to the of the application of the genetic operators to the winners of the tournament.winners of the tournament.
7.7. Repeat steps 2-6 until the termination criterion is met.Repeat steps 2-6 until the termination criterion is met.
IntronsIntrons
Code sections (functions) that provide no real Code sections (functions) that provide no real value for the problem at handvalue for the problem at hand Introns do not directly affect the fitness of the Introns do not directly affect the fitness of the
individual.individual. e.g., j = j + 0 or j = j * 1e.g., j = j + 0 or j = j * 1 Early and middle sections of GP runs might Early and middle sections of GP runs might
include 40-60% introns.include 40-60% introns. Later in the run, introns begin to dominate the Later in the run, introns begin to dominate the
code.code. Introns growth is exponential!Introns growth is exponential!
Why GP Introns EmergeWhy GP Introns Emerge
Children tend to be less fit than parentsChildren tend to be less fit than parents Crossover and mutation can be extremely Crossover and mutation can be extremely
destructivedestructive Introns reduce the destructive effects of genetic Introns reduce the destructive effects of genetic
operatorsoperators Parents generate introns when it is easier to protect Parents generate introns when it is easier to protect
what they already can do, through the creation of what they already can do, through the creation of introns, than improve on what they are currently introns, than improve on what they are currently doing.doing.
Effective FitnessEffective Fitness
Function of at least two factorsFunction of at least two factors The fitness of the parentThe fitness of the parent Likelihood that genetic operators will affect the Likelihood that genetic operators will affect the
fitness of the parent’s childrenfitness of the parent’s children
Effects of IntronsEffects of Introns
Introns may have differing effects before and Introns may have differing effects before and after exponential growth of introns beginsafter exponential growth of introns begins
Different systems may generate different types Different systems may generate different types of introns with different probabilities.of introns with different probabilities.
The extent to which genetic operatos are The extent to which genetic operatos are destructive in their effect is likely to be a very destructive in their effect is likely to be a very important initial condition in intron growth.important initial condition in intron growth.
Mutation and crossover may affect different Mutation and crossover may affect different types of introns differently.types of introns differently.
Problems Caused by IntronsProblems Caused by Introns
Run stagnation (no progress)Run stagnation (no progress) Poor results (do nothing code)Poor results (do nothing code) Drain on memory and CPU time (storing and Drain on memory and CPU time (storing and
executing unnecessary code)executing unnecessary code)
Possible Beneficial Effects of IntronsPossible Beneficial Effects of Introns
Introns might serve to isolate useful code Introns might serve to isolate useful code blocksblocks This facilitates the building block model by This facilitates the building block model by
protecting useful building blocks from disruptionprotecting useful building blocks from disruption
Methods of Handling IntronsMethods of Handling Introns
Reduce the destructiveness of genetic Reduce the destructiveness of genetic operatorsoperators Reducing destructive crossover to 0 results in hill Reducing destructive crossover to 0 results in hill
climbingclimbing Attach fitness penalty to the length of the Attach fitness penalty to the length of the
program.program. Change the fitness functionChange the fitness function
Provides the GP with a way to improve that is Provides the GP with a way to improve that is better than just insulating the current best solution.better than just insulating the current best solution.
ReferencesReferences
Genetic Programming, An IntroductionGenetic Programming, An Introduction Wolfgang Banzhaf, Peter Nordin, Robert E. Keller, Wolfgang Banzhaf, Peter Nordin, Robert E. Keller,
Frank D. FranconeFrank D. Francone Genetic Programming TutorialGenetic Programming Tutorial
John Koza, Gecco 2005John Koza, Gecco 2005 Genetic Programming: The MovieGenetic Programming: The Movie
John KozaJohn Koza
Top Related