Genetic Algorithms Chapter 3A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsGA Quick OverviewDeveloped !SA in the "#$%&sEarly names J. 'olland, (. DeJong, D. Goldberg)ypically applied to* discrete optimi+ationAttributed ,eatures*not too ,ast*good heuristic ,or combinatorial problemsSpecial -eatures*)raditionally emphasi+es combining in,ormation ,rom good parents .crossover/*many variants, e.g., reproduction models, operatorsA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsGenetic algorithms'olland&s original GA is no0 1no0n as the simple genetic algorithm .SGA/2ther GAs use di,,erent*3epresentations*4utations*Crossovers*Selection mechanismsA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsSGA technical summary tableau3epresentation 5inary strings3ecombination 67point or uni,orm4utation 5it0ise bit7,lipping 0ith ,i8ed probability9arent selection -itness79roportionateSurvivor selection All children replace parentsSpeciality Emphasis on crossoverA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsGenotype space : ;%,"AC"BA : "$E3BA : @%E5>BA : 33EA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsAn eample a!ter Goldberg "#$ %1&Simple problem ma8 8> over ;%,",F,3",3,?P, O3,?,>,"P are 2(/Search space is 5IG ,or 3% cities there are 3%S "%3' possible toursA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms/utation operators !or permutations6ormal mutation operators lead to inadmissible solutions*e.g. bit70ise mutationlet gene ihave value j*changing to some other value k0ould mean that k occurred t0ice and j no longer occurred )here,ore must change at least t0o values4utation parameter no0 re,lects the probability that some operator is applied once to the 0hole string, rather than individually in each positionA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms.nsert /utation !or permutations9ic1 t0o allele values at random4ove the second to ,ollo0 the ,irst,shi,ting the rest along to accommodate6ote that this preserves most o, the order and the adHacency in,ormationA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsSwap mutation !or permutations9ic1 t0o alleles at random and s0ap their positions9reserves most o, adHacency in,ormation .? lin1s bro1en/, disrupts order moreA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms.nversion mutation !or permutations9ic1 t0o alleles at random and then invert the substring bet0een them.9reserves most adHacency in,ormation .only brea1s t0o lin1s/ but disruptive o, order in,ormationA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsScramble mutation !or permutations9ic1 a subset o, genes at random3andomly rearrange the alleles in those positions.note subset does not have to be contiguous/A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsK6ormalL crossover operators 0ill o,ten lead to inadmissible solutions4any specialised operators have been devised 0hich ,ocus oncombining order or adHacency in,ormation ,rom the t0o parents*rossover operators !or permutations1 2 3 4 55 4 3 2 11 2 3 2 15 4 3 4 5A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsOrder 1 crossoverIdea is to preserve relative order that elements occurIn,ormal procedure". Choose an arbitrary part ,rom the ,irst parent>. Copy this part to the ,irst child3. Copy the numbers that are not in the ,irst part, to the ,irst childstarting right ,rom cut point o, the copied part, using the order o, the second parent and 0rapping around at the end?. Analogous ,or the second child, 0ith parent roles reversedA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsOrder 1 crossover eampleCopy randomly selected set ,rom ,irst parentCopy rest ,rom second parent in order ",#,3,T,>A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsIn,ormal procedure ,or parents 9" and 9>". Choose random segment and copy it ,rom 9" >. Starting ,rom the ,irst crossover point loo1 ,or elements in that segment o, 9> that have not been copied3. -or each o, these i loo1 in the o,,spring to see 0hat element j has been copied in its place ,rom 9"?. 9lace i into the position occupied j in 9>, since 0e 1no0 that 0e 0ill not be putting j there .as is already in o,,spring/@. I, the place occupied by j in 9> has already been ,illed in the o,,spring k, put i in the position occupied by k in 9>A. 'aving dealt 0ith the elements ,rom the crossover segment, the rest o, the o,,spring can be ,illed ,rom 9>. Second child is created analogously2artially /apped *rossover %2/(&A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms2/(eampleStep "Step >Step 3A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms*ycle crossover4asic idea Each allele comes ,rom one parent together with its position.In,ormal procedure". 4a1e a cycle o, alleles ,rom 9" in the ,ollo0ing 0ay. .a/ Start 0ith the ,irst allele o, 9". .b/ =oo1 at the allele at the same position in 9>..c/ Go to the position 0ith the same allele in 9". .d/ Add this allele to the cycle..e/ 3epeat step b through d until you arrive at the ,irst allele o, 9".>. 9ut the alleles o, the cycle in the ,irst child on the positions they have in the ,irst parent.3. )a1e ne8t cycle ,rom second parentA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms*ycle crossover eampleStep " identi,y cyclesStep > copy alternate cycles into o,,springA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms5dge RecombinationGor1s by constructing a table listing 0hich edges are present in the t0o parents, i, an edge is common to both, mar1 0ith a Re.g. [1 2 3 4 5 6 7 8 9] and [9 3 7 8 2 6 5 1 4]A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms5dge Recombination 'In,ormal procedure once edge table is constructed". 9ic1 an initial element at random and put it in the o,,spring>. Set the variable current element : entry3. 3emove all re,erences to current element ,rom the table?. E8amine list ,or current element*I, there is a common edge, pic1 that to be ne8t element*2ther0ise pic1 the entry in the list 0hich itsel, has the shortest list*)ies are split at random@. In the case o, reaching an empty list*E8amine the other end o, the o,,spring is ,or e8tension*2ther0ise a ne0 element is chosen at randomA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms5dge Recombination eampleA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms/ultiparent recombination3ecall that 0e are not constricted by the practicalities o, nature6oting that mutation uses " parent, and KtraditionalL crossover >, the e8tension to aU> is natural to e8amine5een around since "#A%s, still rare but studies indicate use,ul )hree main types*5ased on allele ,reDuencies, e.g., p7se8ual voting generalising uni,orm crossover*5ased on segmentation and recombination o, the parents, e.g., diagonal crossover generalising n7point crossover*5ased on numerical operations on real7valued alleles, e.g.,center o, mass crossover, generalising arithmetic recombination operatorsA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms2opulation /odelsSGA uses a Generational model*each individual survives ,or e8actly one generation*the entire set o,parents is replaced by the o,,springAt the other end o, the scale are Steady7State models* one o,,spring is generated per generation,* one member o, population replaced,Generation Gap * the proportion o, the population replaced*".% ,or GGA,"BpopCsi+e ,or SSGAA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms0itness 4ased *ompetitionSelection can occur in t0o places*Selection ,rom current generation to ta1e part in mating .parent selection/ *Selection ,rom parents R o,,spring to go into ne8t generation .survivor selection/Selection operators 0or1 on 0hole individual*i.e. they are representation7independentDistinction bet0een selection*operators de,ine selection probabilities*algorithms de,ine ho0 probabilities are implementedA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms.mplementation eample: SGAE8pected number o, copies o, an individual i ( ni ) = Q f(i)! f . : pop.si+e, ,.i/ : ,itness o, i, , avg. ,itness in pop./3oulette 0heel algorithm*Given a probability distribution, spin a "7armed 0heel n times to ma1e n selections*6o guarantees on actual value o, ni 5a1er&s S!S algorithm*n evenly spaced arms on 0heel and spin once*Guaranteesfloor(( ni ) ) ni ceil(( ni ) )A.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms9roblems include*2ne highly ,it member can rapidly ta1e over i, rest o, population is much less ,it 9remature Convergence*At end o, runs 0hen ,itnesses are similar, lose selection pressure *'ighly susceptible to ,unction transpositionScaling can ,i8 last t0o problems*Gindo0ing f"(i) = f(i) # t 0here is 0orst ,itness in this .last n/ generations*Sigma Scaling f"(i) = max. f(i) $ . f # c Q f /, %&%/0here c is a constant, usually >.%0itness-2roportionate SelectionA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms0unction transposition !or 02SA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsRank 6 4ased SelectionAttempt to remove problems o, -9S by basing selection probabilities on relati'e rather than absol(te ,itness3an1 population according to ,itness and then base selection probabilities on ran1 0here ,ittest has ran1 and 0orst ran1 ")his imposes a sorting overhead on the algorithm, but this is usually negligible compared to the ,itness evaluation timeA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms7inear Ranking9arameterised by ,actor s: ".% V s >.%*measures advantage o, best individual*in GGA this is the number o, children allotted to it Simple 3 member e8ampleA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms5ponential Ranking=inear 3an1ing is limited to selection pressureE8ponential 3an1ing can allocate more than > copies to ,ittest individual6ormalise constant ,actor c according to population si+eA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms)ournament SelectionAll methods above rely on global population statistics*Could be a bottlenec1 esp. on parallel machines*3elies on presence o, e8ternal ,itness ,unction 0hich might not e8ist e.g. evolving game players In,ormal 9rocedure*9ic1 k membersat random then select the best o, these*3epeat to select more individualsA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms)ournament Selection '9robability o, selecting i0ill depend on*3an1 o, i*Si+e o, sample k higher k increases selection pressure*Ghether contestants are pic1ed 0ith replacement9ic1ing 0ithout replacement increases selection pressure*Ghether ,ittest contestant al0ays 0ins .deterministic/ or this happens 0ith probability p-or k : >, time ,or ,ittest individual to ta1e over population is the same as linear ran1ing 0ith s = > Q pA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic AlgorithmsSurvivor Selection4ost o, methods above used ,or parent selectionSurvivor selection can be divided into t0o approaches*Age75ased Selectione.g. SGAIn SSGA can implement as Kdelete7randomL .not recommended/ or as ,irst7in7,irst7out .a.1.a. delete7oldest/ *-itness75ased Selection!sing one o, the methods above orA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms)wo Special *asesElitism*Gidely used in both population models .GGA, SSGA/*Al0ays 1eep at least one copy o, the ,ittest solution so ,arGE6I)23 a.1.a. Kdelete70orstL*-rom Ghitley&s original Steady7State algorithm .he also used linear ran1ing ,or parent selection/*3apid ta1eoveruse 0ith large populations or Kno duplicatesL policyA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms5ample application o! order based GAs: 8SS2 9recedence constrained Hob shop scheduling problemJ is a set o, Hobs.2 is a set o, operations4 is a set o, machines )ble 2 4 de,ines 0hich machines can per,orm 0hich operationsPre 2 2 de,ines 0hich operation should precede 0hich *(r 2 4 I3 de,ines the duration o, o 2 on m 4 )he goal is no0 to ,ind a schedule that isComplete all Hobs are scheduledCorrect all conditions de,ined by )ble and Pre are satis,ied2ptimal the total duration o, the schedule is minimalA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms2recedence constrained 9ob shop scheduling GA3epresentation individuals are permutations o, operations9ermutations are decoded to schedules by a decoding procedure*ta1e the ,irst .ne8t/ operation ,rom the individual*loo1 up its machine .here 0e assume there is only one/*assign the earliest possible starting time on this machine, subHect tomachine occupationprecedence relations holding ,or this operation in the schedule created so ,ar,itness o, a permutation is the duration o, the corresponding schedule .to be minimi+ed/use any suitable mutation and crossoveruse roulette 0heel parent selection on inverse ,itnessGenerational GA model ,or survivor selectionuse random initialisationA.E. Eiben and J.E. Smith, Introduction to Evolutionary ComputingGenetic Algorithms8SS2 eample: operator comparison