ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints:...
Transcript of ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints:...
![Page 1: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/1.jpg)
ME 697: INTELLIGENT SYSTEMS
OPTIMIZATION ALGORITHMS
Daniel McArthur PhD Student
![Page 2: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/2.jpg)
EVOLUTIONARY STRATEGIES
![Page 3: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/3.jpg)
PROBLEM DEFINITION 2-INPUT NONLINEAR FITNESS FUNCTION TO MINIMIZE
Domain: โ512 โค ๐ฅ๐ฅ,๐ฆ๐ฆ โค 512
โEggholder Functionโ[1]
Constraints: ๐๐1 = ๐ฅ๐ฅ + 512 ๐๐2 = 512 โ ๐ฅ๐ฅ ๐๐3 = ๐ฆ๐ฆ + 512 ๐๐4 = 512 โ ๐ฆ๐ฆ
![Page 4: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/4.jpg)
EVOLUTIONARY STRATEGIES ALGORITHM & PARAMETERS
ฮผ = Number of parents ฮป = Number of children (offspring)
Parameters to Select โข Population Size
โข Number of parents โข Number of children
โข Maximum number of generations โข Elitism Factor
โข Which individuals reproduce? โข Include parents in next population?
โข Mutation Rate โข Cross-over Rate
Parameter Types โข Object Parameter (OP)
โข Individual = vector of real numbers โข Strategy Parameter (SP)
โข Standard Deviation for each input variable of each individual (controls individual mutation)
Standard (ฮผ,ฮป) Algorithm[2]:
![Page 5: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/5.jpg)
EVOLUTIONARY STRATEGIES MATLAB IMPLEMENTATION
Mutation โข Modify OP by adding ๐๐(๐๐,๐๐) with ๐๐=0
โข ๐๐๐๐๐๐๐๐๐๐(๐๐) = ๐๐๐๐(๐๐) + ๐๐(0,๐๐(๐๐)) โข Vary mutation step size (๐๐) by โ1/5
success ruleโ โข After k iterations, reset ๐๐ by:
๐๐ = ๐๐๐๐
๐๐๐๐ ๐๐๐ ๐ > 15
๐๐ = ๐๐ โ ๐๐ ๐๐๐๐ ๐๐๐ ๐ < 15
๐๐ = ๐๐ ๐๐๐๐ ๐๐๐ ๐ = 15
Fitness & Elitism
โข fit_fun.m & constr_fun.m โข Evaluate fitness and constraint
values for an individual โข Individuals are sorted by fitness value
using min(), and only the best individuals reproduce
โข Parents included in selection pool
Random Population โข rng()
โข Set seed for repeatable tests โข rand()
โข Select (x,y)-pairs in domain โข Set each ๐๐ between 0 & ๐๐๐ ๐ ๐๐๐ ๐ ๐ ๐ ๐๐
Recombination
โข randperm() โข Select unique parent pairs
โข Discrete โข Select ๐๐๐๐๐๐(i) from random parent
โข Intermediate: โข ๐๐๐๐๐๐(i) = average of ๐๐๐๐๐๐ ๐๐
![Page 6: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/6.jpg)
PARAMETER SELECTION POPULATION SIZE: NUMBER OF PARENTS
![Page 7: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/7.jpg)
PARAMETER SELECTION POPULATION SIZE: NUMBER OF CHILDREN
![Page 8: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/8.jpg)
PARAMETER SELECTION RECOMBINATION
Mutation Only
Discrete Recombination
Intermediate Recombination
ฯ = 5
ฯ = 20
![Page 9: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/9.jpg)
PARAMETER SELECTION MUTATION: STANDARD DEVIATION
![Page 10: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/10.jpg)
PARAMETER SELECTION MUTATION: STRATEGY PARAMETER DISTRIBUTION
ฯ Randomly Distributed
at Start (๐๐ โค ๐๐ โค ๐๐๐๐๐๐๐๐)
ฯ Same at Start (๐๐ = ๐๐๐๐๐๐๐๐)
![Page 11: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/11.jpg)
PARAMETER SELECTION SENSITIVITY TO RANDOMIZATION
Run 1
ฮผ = 5
ฮผ = 150
Run 2 Run 3
![Page 12: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/12.jpg)
FINAL PARAMETERS BEST COMBINATION
Population Size โข Number of Parents (ฮผ) = 150 โข Number of Children (ฮป) = 2 * ฮผ
Number of Generations โข 50 Generations
โข Typically converged much sooner (e.g. < 30 generations)
Recombination โข Discrete outperforms Intermediate
Mutation โข ฯ = 5 โข Reset after every 5 iterations
Elitism โข Include parents in fitness competition
![Page 13: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/13.jpg)
TEST RESULTS EVOLUTIONARY STRATEGIES OPTIMIZATION VS. ACTUAL VALUES
Actual Global Minimum: -959.6407 Location of Minimum: 512,404.2319
Evolutionary Strategies Global Minimum: -959.6308 Location of Minimum: 511.9982,404.1737
Absolute Error: 0.001 %
![Page 14: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/14.jpg)
TEST RESULTS EVOLUTIONARY STRATEGIES OPTIMIZATION VS. ACTUAL VALUES
Actual Global Minimum: 0 Location of Minimum: 0, 0
Evolutionary Strategies Global Minimum: 0.000092 Location of Minimum: โ0.000028, 0.000017
Absolute Error: 0.0092 %
![Page 15: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/15.jpg)
CONCLUSIONS EVOLUTIONARY STRATEGIES
โข ES is able to optimize complex nonlinear functions with very little error โข Each parameter in the ES method requires careful tuning, and will vary
depending on the function being optimized โข The performance of ES can vary significantly due to randomization,
but proper selection of population size (number of parents/children) can reduce this variation
โข Mutation parameters can affect the efficiency of the algorithm and the average fitness of each population, but within a reasonable range, the ES method will still come close to the global solution
โข Thus, some experimentation may be required to get the parameters into a useable range, but the parameters generally donโt need to be perfect to get a reasonable solution
![Page 16: ME 697: INTELLIGENT SYSTEMS presentation/2016...ย ยท โEggholder Functionโ[1] Constraints: ๐๐1= ๐ฅ๐ฅ+ 512 ๐๐2= 512 โ๐ฅ๐ฅ ๐๐3= ๐ฆ๐ฆ+ 512 ๐๐4=](https://reader035.fdocuments.in/reader035/viewer/2022071219/60569cd2f81b08010f55d532/html5/thumbnails/16.jpg)
REFERENCES
1. Test Functions for Optimization. (n.d.). Retrieved March 04, 2016, from https://en.wikipedia.org/wiki/Test_functions_for_optimization
2. Shin, Yung C., and Chengying. Xu. Intelligent Systems Modeling, Optimization, and Control. Automation and Control Engineering. Hoboken: Taylor and Francis, 2008.