Genetic Algorithms: A Tutorial - 하나원닷컴 (하나ONE 융합교육) … · 2019-05-16 ·...
58
Page 1 Genetic Algorithm “Genetic Algorithms are good at taking large, potentially huge search spaces and navigating them, looking for optimal combinations of things, solutions you might not otherwise find in a lifetime.” Salvatore Mangano Computer Design, May 1995 Genetic Algorithm Genetic Algorithm Structure of Biological Gen
Transcript of Genetic Algorithms: A Tutorial - 하나원닷컴 (하나ONE 융합교육) … · 2019-05-16 ·...
Page 1 Genetic Algorithm
“Genetic Algorithms are good at taking large,
potentially huge search spaces and navigating
them, looking for optimal combinations of things, solutions you might not
otherwise find in a lifetime.”
Salvatore ManganoComputer Design, May 1995
Genetic AlgorithmGenetic Algorithm
Structure of Biological Gen
Page 2 Genetic Algorithm
• Every animal cell is a complex structure where many small
“factories” are working together
• The center of this all is the cell nucleus
• The nucleus contains the genetic information
Biological Background (cell)
Page 3 Genetic Algorithm
Biological Background (chromosomes)• The genetic information is stored in the chromosomes• Each chromosome is build of DNA (deoxyribonucleic acid). • Chromosomes in humans form are pairs.• There are 23 pairs (in the human cell).• The chromosome is divided into parts: genes.
• Genes code for properties.• The posibilities of the genes for
one property is called: allele.• Every gene has an unique positionon the chromosome: locus.
Page 4 Genetic Algorithm
Biological Background (chromosomes)
On the picture we can see a human cell with all metaphase chromosome ordered by size and an other attributemetaphase: the state of gen dividing when the chromosomes are situated in the plane of the equator of cell.
Page 5 Genetic Algorithm
Biological Background (reproduction)
• Reproduction of genetical information• Mitosis
• Meiosis
• Mitosis is copying the same genetic information to newoffspring: there is no exchange of information
• Mitosis is the normal way ofgrowing of multicell structures,like organs.
Page 6 Genetic Algorithm
Biological Background (reproduction)
• Meiosis is the basis of sexual reproduction
• After meiotic division 2 gametes appear in the process
• In reproduction two gametes conjugate to a zygote wich will become the new individual
• Hence genetic information is shared between the parents in order to create new offspring
Page 7 Genetic Algorithm
Biological Background (reproduction)
• During reproduction “errors” occur
• Due to these “errors” genetic variation exists
• Most important “errors” are:
• Recombination (cross-over)
• Mutation
Page 8 Genetic Algorithm
Biological Background (natural secection)
• The origin of species: “Preservation of favourable variations and rejection of unfavourable variations.”
• There are more individuals born than can survive, so there is a continuous struggle for life.
• Individuals with an advantage have a greater chance for survive: survival of the fittest.
Page 9 Genetic Algorithm
Genetic Algoritm(in Technical Tasks)
● Directed search algorithms based on the mechanics of biological evolution.
● Developed by John Holland, University of Michigan (1970’s)♦ To understand the adaptive processes of natural
systems♦ To design artificial systems software that retains
the robustness of natural systems
Page 10 Genetic Algorithm
● Provide efficient, effective techniques for optimization and machine learning applications
● Widely-used today in business, scientific and engineering circles
Genetic Algoritm(in Technical Tasks)
Page 11 Genetic Algorithm
Classes of Search Techniques
Finonacci Newton
Direct methods Indirect methods
Calculus-based techniques
Evolutionary strategies
Centralized Distributed
Parallel
Steady-state Generational
Sequential
Genetic algorithms
Evolutionary algorithms Simulated annealing
Guided random search techniques
Dynamic programming
Enumerative techniques
Search techniques
Page 12 Genetic Algorithm
A problem to solve, and ...● Encoding technique (gene, chromosome)● Initialization procedure (creation)● Evaluation function (environment)● Selection of parents (reproduction)● Genetic operators (mutation, recombination)● Parameter settings (practice and art)
Genetic Algoritm (Components)
Page 13 Genetic Algorithm
Simple Genetic Algorithm{
_initialize population;_evaluate population;while Termination_Criteria_Not_Satisfied{
_select parents for reproduction;_perform recombination and mutation;_evaluate population;
}}
Page 14 Genetic Algorithm
Genetic Algoritm; Cycle of Reproduction
reproduction
population evaluation
modification
discard
deleted members
parents
selected parents
modifiedchildren
evaluated children
initialization
Page 15 Genetic Algorithm
Cycle of Reproduction; Population
♦ Bit strings (0110001011010011100)♦ Real numbers (43.2, -33.1, ... 0.0, 89.2) ♦ Permutations of element (E11, E3, E7, ... E1, E15)♦ Lists of rules (R1, R2, R3, ... R22, R23)♦ Program elements (genetic, programming)♦ ... any data structure ...
population
Elements of population (chromosomes) could be:
initialization
Page 16 Genetic Algorithm
Bit string chromosomeparameters
chromosome (bit string)
range of integer number
linear decoding
parameter range x=26+13*(4/31) = 27.68
a gen = 1 bit
Page 17 Genetic Algorithm
Bit string cromosome
Page 18 Genetic Algorithm
Real numbers chromosome
Real number chromosome = vector of real numbers
a gen = one real numberx0 x1
Page 19 Genetic Algorithm
Permutation of elements cromosome
chromosome (permutation of elements)
representation by graph
a gen = an order number at a given position
Page 20 Genetic Algorithm
Cycle of Reproduction; Reproduction
reproduction
population
parents
selected parents
Parents are selected at random with selection chances biased in relation to chromosome evaluations (fitness function).
Page 21 Genetic Algorithm
Reproduction; Fitness Function
Real numbers chromosome
Fitness(x,y) is any calculated value
Bit string chromosome
Page 22 Genetic Algorithm
Reproduction; Fitness Function
Fitness ( cromosome ) for example is the distance between the points represented positions
chromosome (permutation of elements)
Page 23 Genetic Algorithm
Reproduction; Rulett Whell Selection
F[pv1]
F[pv2]
F[pv3]F[pv4]
F[pv5]
F[pv6]
F[pv7] F[pv8]
”Cake” of fitness functionsin a population
area = fitness(x7,y7)
Page 24 Genetic Algorithm
Reproduction; Rank Selection
● Ranking is a parent selection method based on the rank of chromosomes.
● Each chromosome is ranked by its fitness value.● r1 is assigned to the worst; r2 to second worst;
and so on.● Higher fitness value has the higher ranking, which
means it will be chosen with higher probability.● Calculate the sum of ranks: result is Rsum.● Parent selection: Random number generating
between 0..Rsum
Page 25 Genetic Algorithm
Reproduction; Tournament Selection
● Tournament selection is one of many methods of selection in genetic algorithms which runs a "tournament" among a few individuals chosen at random from the population and selects the winner (the one with the best fitness) for crossover.
● If the tournament size is higher, weak individuals have a smaller chance to be selected.
● A 1-way tournament selection is equivalent to random selection.
Page 26 Genetic Algorithm
Cycle of Reproduction; Modification
modificationselected parents
Modifications are stochastically triggered● Operator types are:
♦ Crossover (recombination) (probability of crossover)
♦ Mutation (probability of mutation)
modified children
Page 27 Genetic Algorithm
Crossover is a critical feature of genetic algorithms:• It greatly accelerates search early in evolution of
a population• It leads to effective combination of schemata
(sub solutions on different chromosomes)
Modification; Crossover
P1 (0 1 1 0 1 0 0 0) (1 1 0 0 1 0 0 0) Ch1
P2 (1 1 0 1 1 0 1 0) (0 1 1 1 1 0 1 0) Ch2
cut
Page 28 Genetic Algorithm
Modification; Mutation
Before: (1 0 1 1 0 1 1 0)
After: (0 1 1 0 0 1 1 0)
Before: (1.38 -69.4 326.44 0.1)
After: (1.38 -67.5 326.44 0.1)
● Causes movement in the search space(local or global)
● Restores lost information to the population
Real numbers chromosome
Bit string chromosomerandom selected positions
Page 29 Genetic Algorithm
Cycle of Reproduction; Evaluation
● The evaluator decodes a chromosome and assigns it a fitness measure
● The evaluator is the only link between a classical GA and the problem it is solving
evaluation
evaluatedchildren
modifiedchildren
Page 30 Genetic Algorithm
Cycle of Reproduction; Deletion
● Generational GA:entire populations (fully) replaced with each iteration
● Steady-state GA:a few members replaced each generation
population
discard
discardedmembers
Page 32 Genetic Algorithm
“The Gene is by far the most sophisticated program around.”- Bill Gates, Business Week, June 27, 1994
Page 33 Genetic Algorithm
A Simple Example
The Travelling Salesman Problem (TSP):
We search the summa of distance between the towns with the next conditions:
♦ we can visit every town only ones♦ we have to minimize the full distance
(summa distance between the towns in a given sequence)
Page 34 Genetic Algorithm
Representation of TSPRepresentation is an ordered list of citynumbers known as an order-based GA.
1) Oulu 3) Velence 5) Peking 7) Tokio2) Budapest 4) Singapur 6) London 8) New York
CityList1 (3 5 7 2 1 6 4 8)CityList2 (2 5 7 6 8 1 3 4)
Page 35 Genetic Algorithm
Crossover of TSPCrossover combines inversion and recombination:
* *Parent1 (3 5 7 2 1 6 4 8)Parent2 (2 5 7 6 8 1 3 4)
Child (5 8 7 2 1 6 3 4)
random selected positions
Page 36 Genetic Algorithm
Mutation involves reordering of the list:
* *Before: (5 8 7 2 1 6 3 4)
After: (5 8 6 2 1 7 3 4)
Mutation of TSP
random selected positions
Page 38 Genetic Algorithm
Rucksack (Backpack) PackingThe problem:
We have N pieces of objects. Every object has a mass and a face value. We have a rucksack (backpack) which has X kg weight-bearing capacity.
The task is selecting a suitable subset of the objects, where the face value is maximal and the sum mass of objects are limited to X kg.
We solve the problem applying the genetic algoritm with bitstring chromosome. The coding in the chromosome gives whether the object is selected or not.
The upper matrix containing the parent chromosomes the lower the crossovered and mutated chromosomes. The numbers beside the matrixes are the sum of face values of selected objects with other words the fitness function.
Page 40 Genetic Algorithm
Finding the maximum value of a function (MVFN):
We finding the maximal value of an explicit function, (the global maximum):
♦ the input parameter ranges are given for the search
Maximum Value of Function
Page 41 Genetic Algorithm
The chromosome is coded with a vector of real numbers.
x = { -20.0, +12.0 } range of search x y = { -30.0, +100.0 } range of search y
c(i) = [ x(i), y(i) ] the form of chromosome
c(1) = [ 3.234, 5.111 ] chromosome(1)c(2) = [ 10.12, -3.9234 ] chromosome(2)
Representation of MVFN
Page 42 Genetic Algorithm
Parent(1)= [x(1), y(1)]; Parent(2) = [x(2), y(2)];
a = random number between {0..1} / type!!! /
Child(1) = [ a*x(1) + {1-a}*x(2), a*y(1) + {1-a}*y(2) ] / ranges!!! /Child(2) = [ {1-a}*x(1) + a*x(2), {1-a}*y(1) + a*y(2) ] / ranges!!! /
Crossover of MVFN
Page 43 Genetic Algorithm
The mutation a randomly selected element value changing randomly:
a,b = random numbers with range {0..1} / type!!! / c = constant { for example = 0.25 }Before: [ x1, y1 ]
After: [ x1 + (x1max - x1min)*c*a, / ranges!!! / y1 + (y1max - y1min)*c*a ] / ranges!!! /
Mutation of MVFN
Page 44 Genetic Algorithm
Type of Random Numbers:Uniform Distribution
≤≤
−=otherwise0
bxaif)ab(
1)x(f
Page 45 Genetic Algorithm
Type of Random Numbers:Exponential Distribution
≤⋅λ
=⋅λ−
otherwise0x0fore
)x(fx
Page 46 Genetic Algorithm
Type of Random Numbers:Normal Distribution
ye2
1)x(f −⋅
π⋅⋅σ=
2
2
2)x(y
σ⋅µ−=
Page 47 Genetic Algorithm
Type of Random Numbers:Triangular Distribution
checkpoint = ( usual -minimum ) / ( maximum - minimum );
triangledistribution = ( random <= checkpoint )?
sqrt ( random * ( maximum - minimum ) * (usual - minimum) )+minimum :
maximum - sqrt ((1-random)*(minimum-maximum)*(usual-maximum)) ;
Page 48 Genetic Algorithm
The function:f(x, y) = 20 - (x1 - x0)^2 + (y1 - y0)^2where x0 and y0 are constants
Maximum Value of Function
We can find also the minimum value, in this case we maximazing the - f(x,y) function.
Page 49 Genetic Algorithm
Maximum Value of Function[ ]( )∑
=
−−=2
1i
20.5ix20)x(f
Page 50 Genetic Algorithm
Maximum Value of Function[ ] [ ]( )∑
=
−−=N
1i
20ixix20)x(f
Page 51 Genetic Algorithm
Search the Structure of a Function
We have big number of measuring data where the input parameter(s) is(are) known to the measured data. For example the measuring data is the area of a circle and the input parameter the radius of the circle.
We want to find a suitable structure of function is giving the connection between the input parameter and measured data.
Page 53 Genetic Algorithm
Issues for GA Practitioners● Choosing basic implementation issues:
♦ representation of information♦ population size, mutation rate, ...♦ selection, deletion policies♦ crossover, mutation operators
● Termination Criteria● Performance, scalability● Solution is only as good as the evaluation
function (often hardest part)
Page 54 Genetic Algorithm
Benefits of Genetic Algorithms● Concept is easy to understand● Modular, separate from application● Supports multi-objective optimization● Good for “noisy” environments● Always have an answer; answer became
better and better with time● Inherently parallel; easily distributed
Page 55 Genetic Algorithm
Benefits of Genetic Algorithms● Many ways to speed up and improve a GA-
based application as knowledge about problem domain is gained.
● Easy to exploit previous or alternate solutions.● Flexible building blocks for hybrid applications.● Substantial history and range of use.
Page 56 Genetic Algorithm
When to Use a GA● Alternate solutions are too slow or overly
complicated● Need an exploratory tool to examine new
approaches● Problem is similar to one that has already been
successfully solved by using a GA● Want to hybridize with an existing solution● Benefits of the GA technology meet key problem
requirements
Page 57 Genetic Algorithm
Some GA Application TypesDomain Application Type
Control gas, pipeline, pole balancing, missile evasion, pursuit
Design semiconductor layout, aircraft design, keyboard configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation
Robotics trajectory planning
Machine Learning designing neural networks, improving classification algorithms, classifier systems
Signal Processing filter design
Game Playing poker, checkers, prisoner’s dilemma
Combinatorical Optimization
set covering, traveling salesman, routing, bin packing, graph coloring and partitioning
Page 58 Genetic Algorithm
Thank you for your attention
Questions?
