Omni-Optimizer A Procedure for Single and Multi-objective Optimization Prof. Kalyanmoy Deb and...
-
Upload
antony-moody -
Category
Documents
-
view
214 -
download
0
Transcript of Omni-Optimizer A Procedure for Single and Multi-objective Optimization Prof. Kalyanmoy Deb and...
Omni-OptimizerOmni-Optimizer
A Procedure for Single and A Procedure for Single and Multi-objective OptimizationMulti-objective Optimization
Prof. Kalyanmoy Deb andProf. Kalyanmoy Deb and
Santosh TiwariSantosh Tiwari
MotivationMotivation
Generic Programming PracticesGeneric Programming Practices Unified algorithm for all types of Unified algorithm for all types of
optimization problemsoptimization problems An efficient implementation of NSGA-II An efficient implementation of NSGA-II
framework (procedure)framework (procedure) Developing an efficient and self-Developing an efficient and self-
adaptive optimization paradigmadaptive optimization paradigm
Literature SurveyLiterature Survey
CHC (Cross generation elitist selection, CHC (Cross generation elitist selection, Heterogeneous recombination, Cataclysmic Heterogeneous recombination, Cataclysmic mutation) – mutation) – Explicit DiversityExplicit Diversity
GENITOR (Steady state GA), more like GENITOR (Steady state GA), more like (µ+1)-ES so far as selection mechanism is (µ+1)-ES so far as selection mechanism is concerned. – concerned. – High selection pressureHigh selection pressure
NPGA (Niched Pareto Genetic Algorithm), NPGA (Niched Pareto Genetic Algorithm), uses sharing parameter σuses sharing parameter σshareshare – – # of niches # of niches obtained depend on the sharing parameterobtained depend on the sharing parameter
Literature Survey contd…Literature Survey contd… PESA (Pareto Envelope-based Selection Algorithm), PESA (Pareto Envelope-based Selection Algorithm),
Hyper-grid division of phenotypic space, selection Hyper-grid division of phenotypic space, selection based on crowding measurebased on crowding measure
NSGA-II (Non-dominated Sorting Genetic Algorithm)NSGA-II (Non-dominated Sorting Genetic Algorithm) SPEA2 (Strength Pareto Evolutionary Algorithm), Fine SPEA2 (Strength Pareto Evolutionary Algorithm), Fine
grained fitness assignment mechanism utilizing density grained fitness assignment mechanism utilizing density information, Only archive members participate in information, Only archive members participate in mating – mating – Excellent Diversity in phenotypic spaceExcellent Diversity in phenotypic space
NCGA (Neighborhood Cultivation Genetic Algorithm), NCGA (Neighborhood Cultivation Genetic Algorithm), used neighborhood crossover, based on NSGA-II and used neighborhood crossover, based on NSGA-II and SPEA2SPEA2
RPSGAe (Reduced Pareto Set Genetic Algorithm with RPSGAe (Reduced Pareto Set Genetic Algorithm with elitism)elitism)
ENORA (Evolutionary Algorithm of Non-dominated ENORA (Evolutionary Algorithm of Non-dominated Sorting with Radial Slots)Sorting with Radial Slots)
Salient Features of the Salient Features of the AlgorithmAlgorithm
Based on NSGA-II frameworkBased on NSGA-II framework Based on the concept of Pareto dominanceBased on the concept of Pareto dominance Incorporates elitismIncorporates elitism Explicit diversity preserving mechanismExplicit diversity preserving mechanism
Can be used for single-objective as well as Can be used for single-objective as well as multi-objective problemsmulti-objective problems
Can be used for uni-global as well as multi-Can be used for uni-global as well as multi-global problemsglobal problems
Independent of the number of niches that Independent of the number of niches that an optimization problems exhibitsan optimization problems exhibits
Moving beyond NSGA-IIMoving beyond NSGA-II
Restricted Selection SchemeRestricted Selection Scheme Tournament selection based on usual dominationTournament selection based on usual domination Non-dominated sorting based on epsilon dominanceNon-dominated sorting based on epsilon dominance
Crowding Distance AssignmentCrowding Distance Assignment Genotypic as well as Phenotypic space nichingGenotypic as well as Phenotypic space niching Choose best members from above average populationChoose best members from above average population Remove worst members from below average populationRemove worst members from below average population
More robust recombination and variation operatorsMore robust recombination and variation operators Two point crossover for binary variablesTwo point crossover for binary variables Highly disruptive real variable mutationHighly disruptive real variable mutation
Restricted SelectionRestricted Selection Helps in preserving multi-modalityHelps in preserving multi-modality Experiments show that it gives faster overall Experiments show that it gives faster overall
convergenceconvergence
Epsilon Domination PrincipleEpsilon Domination Principle
A finite percentage (based on A finite percentage (based on function value) of solutions function value) of solutions assigned a particular rankassigned a particular rank
Allows somewhat inferior Allows somewhat inferior solutions to remain in the solutions to remain in the populationpopulation
Provides guaranteed diversityProvides guaranteed diversity Helps to obtain multi-modal Helps to obtain multi-modal
solutions in case of single solutions in case of single objective problemsobjective problems
Epsilon is generally taken in the Epsilon is generally taken in the range 10range 10-3-3 ~ 10 ~ 10-6-6
Modified Crowding DistanceModified Crowding Distance Genotypic as well as Phenotypic space nichingGenotypic as well as Phenotypic space niching
Highly Disruptive Mutation Highly Disruptive Mutation OperatorOperator
1 2
11 1
1
1 1 1
2
,
2 (1 2 )*(1 ) 1 if r 0.5
1 2(1 ) 2( 0.5)*(1 ) otherwise
( )
m m
m m
p l u p
u l u l
q
c p q p l
x x x x
x x x x
r r
r r
x x x x
Computational ComplexityComputational Complexity
Restricted selection Restricted selection O O (n(nNN22)) Ranking procedure Ranking procedure O O (M(MNN22)) Crowding procedureCrowding procedure
max{ max{ O O ((MNMN log log NN), ), O O ((nNnN log log NN) }) } Overall iteration-wise complexityOverall iteration-wise complexity
max {max {O O (n(nNN22), ), O O ((MNMN22), ), O O ((nNnN log log NN)})}
Implementation DetailsImplementation Details
Code written in simple C and strictly Code written in simple C and strictly conforms to ANSI/ISO standardconforms to ANSI/ISO standard
Data structure used is a custom doubly Data structure used is a custom doubly linked list (gives O(1) insertion and linked list (gives O(1) insertion and deletion)deletion)
Randomized quick sort used for sortingRandomized quick sort used for sorting Almost all the functions perform in-place Almost all the functions perform in-place
operation (addresses are passed, operation (addresses are passed, significantly decreases stack overheads)significantly decreases stack overheads)
Simulation ResultsSimulation Results
GA parameters for all problems chosen as GA parameters for all problems chosen as followsfollows ηηcc = 20 = 20 ηηmm = 20 = 20 P (crossover) = 0.8P (crossover) = 0.8 P (mutation) = 1/n, where n = # of real variablesP (mutation) = 1/n, where n = # of real variables δδ = 0.001 = 0.001 Population size and number of generations Population size and number of generations
different for different problemsdifferent for different problems
Simulation Results contd…Simulation Results contd… 20 variable Rastrigin function20 variable Rastrigin function
# of function evaluation# of function evaluation Least = 19260Least = 19260 Median = 24660Median = 24660 Worst = 29120Worst = 29120
20 variable Schwefel function20 variable Schwefel function # of function evaluation# of function evaluation
Least = 54950Least = 54950 Median = 69650Median = 69650 Worst = 103350Worst = 103350
Other single objective problems can be found in the Other single objective problems can be found in the paperpaper
In all cases, better results are found in comparison to In all cases, better results are found in comparison to previous reported studiesprevious reported studies
Single objective multi-modal Single objective multi-modal functionfunction
f(x) = sinf(x) = sin22 ( (ππx)x) x x єє [0,20] [0,20]
Single objective multi-modal Single objective multi-modal functionfunction
Unconstrained Himmelblau’s functionUnconstrained Himmelblau’s function
Multi-objective Uni-Global Test Multi-objective Uni-Global Test ProblemsProblems
30 variable ZDT2 (100×100)30 variable ZDT2 (100×100)
Multi-objective Uni-Global Test Multi-objective Uni-Global Test ProblemsProblems
10 variable ZDT4 (100×250)10 variable ZDT4 (100×250)
Multi-objective Uni-Global Test Multi-objective Uni-Global Test ProblemsProblems
CTP4 (100×7000)CTP4 (100×7000)
Multi-objective Uni-Global Test Multi-objective Uni-Global Test ProblemsProblems
CTP8 (100×100)CTP8 (100×100)
Multi-objective Uni-Global Test Multi-objective Uni-Global Test ProblemsProblems
DTLZ4 (300×100)DTLZ4 (300×100)
Multi-objective Multi-Global Test Multi-objective Multi-Global Test ProblemProblem
FF11 (x) = summation (sin ( (x) = summation (sin (ππxxii) )) ) xxii єє [0,6] [0,6]FF22 (x) = summation (cos ( (x) = summation (cos (ππxxii) )) ) xxii єє [0,6] [0,6]
Efficient points in phenotypic space
Multi-objective Multi-Global Test Multi-objective Multi-Global Test ProblemProblem
Genotypic space plotsGenotypic space plots
Few Sample SimulationsFew Sample Simulations
F(x) = sinF(x) = sin22 (10,000*pi*x) (10,000*pi*x) Himmelblau’s FunctionsHimmelblau’s Functions ZDT Test ProblemsZDT Test Problems CTP Test ProblemsCTP Test Problems Test Problem TNKTest Problem TNK Multi-global Multi-objective Test Multi-global Multi-objective Test
ProblemProblem
Further Ideas and Future Further Ideas and Future WorkWork
Incorporating PCX instead of SBX for Incorporating PCX instead of SBX for crossovercrossover
Automatically fine-tuning mutation Automatically fine-tuning mutation index so as to achieve arbitrary index so as to achieve arbitrary precisionprecision
Self-adaptation of parameter Self-adaptation of parameter δδ Segregating population into niches Segregating population into niches
without the introduction of DMwithout the introduction of DM Dynamic population sizingDynamic population sizing Using hierarchical NDS for the Using hierarchical NDS for the
crowding distance assignmentcrowding distance assignment