Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009...
Transcript of Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009...
![Page 1: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/1.jpg)
Nikolaus Hansen, Inria, Université Paris-Saclay
Function-Value-Free Second-Order Stochastic Optimization with CMA-ES
Nikolaus Hansen
Inria Research Centre Saclay – Île-de-France
Université Paris-Saclay, École Polytechnique, CMAP
INRIA: The French National Institute for Research in Computer science and Control 8 research centres, 210 research teams
http://www.inria.fr http://www.lri.fr/~hansen
January 2017
![Page 2: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/2.jpg)
2
…feel free to ask questions…
![Page 3: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/3.jpg)
Black-Box Optimization (Search)
Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 4: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/4.jpg)
Landscape of Continuous Search Methods
4
Gradient-based (Taylor, local)
• Conjugate gradient methods [Fletcher & Reeves 1964]
• Quasi-Newton methods (BFGS) [Broyden et al 1970]
Derivative-free optimization (DFO)
• Trust-region methods (NEWUOA, BOBYQA) [Powell 2006, 2009]
• Simplex downhill [Nelder & Mead 1965]
• Pattern search [Hooke & Jeeves 1961, Audet & Dennis 2006]
Stochastic (randomized) search methods
• Evolutionary algorithms (broader sense, continuous domain)
– Differential Evolution [Storn & Price 1997]
– Particle Swarm Optimization [Kennedy & Eberhart 1995]
– Evolution Strategies [Rechenberg 1965, Hansen & Ostermeier 2001]
• Simulated annealing [Kirkpatrick et al 1983]
• Simultaneous perturbation stochastic approximation (SPSA) [Spall 2000]
![Page 5: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/5.jpg)
5
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
• > 3000 citations to the two main original CMA-ES articles∫ 100 published applications
• implemented in various software libraries
– evolutionary computation: Open BEAGLE, DEAP, EO, PyEC. . .
– optimization: Apache Commons Math, NOMADm, OpenOpal
– machine learning: PyBrain, Shark
– image processing, robot control & simulation: PAC, OpenSim
– PDE solver: FreeFem++
– water model calibration: PEST
– economics: AmiBroker, Dynare, parma
• ¥ 2000 monthly page views on Wikipedia and of the source code page
• used by various companies: Alstom, Astrium, Bosch, Honda,Rolls-Royce, Siemens, Storengy, Total,. . .
![Page 6: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/6.jpg)
6
Typical Applications
![Page 7: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/7.jpg)
7
An Example Application
https://youtu.be/pgaEE27nsQw
https://youtu.be/pgaEE27nsQw
![Page 8: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/8.jpg)
8
Difficulties in Black-Box Optimization
in any case, the objective function must be highly regular
![Page 9: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/9.jpg)
Nikolaus Hansen INRIA, University Paris-Sud
Rugged landscape
Func
tion
valu
e
![Page 10: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/10.jpg)
Randomized optimization template
Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 11: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/11.jpg)
Randomized optimization template
Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 12: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/12.jpg)
12
A new search problemThe template replaces the original search problem
arg min
x
(f(x)) x œ X
with a new search problem on ◊-space,
arg max
◊
(J(◊)) where J(◊) = Ex≥p(.|◊)
!W
f
◊t(f(x))
",
where W
f
◊tis monotonous decreasing.
think of W (f(x)) as ≠f(x) for the time being
Both problems have the same solution (same optimum):
P (x|◊ú) = ”(x ≠ x
ú) for all W
f
◊t
i.e., Pr(x = x
ú | ◊ = ◊
ú) = 1.
W
f◊t
(f(x)) := w(Pry≥p(.|◊t)(f(y) Æ f(x)))
![Page 13: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/13.jpg)
13
A new search problemThe template replaces the original search problem
arg min
x
(f(x)) x œ X
with a new search problem on ◊-space,
arg max
◊
(J(◊)) where J(◊) = Ex≥p(.|◊)
!W
f
◊t(f(x))
",
where W
f
◊tis monotonous decreasing.
think of W (f(x)) as ≠f(x) for the time being
Both problems have the same solution (same optimum):
P (x|◊ú) = ”(x ≠ x
ú) for all W
f
◊t
i.e., Pr(x = x
ú | ◊ = ◊
ú) = 1.
W
f◊t
(f(x)) := w(Pry≥p(.|◊t)(f(y) Æ f(x)))
Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 14: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/14.jpg)
14
Information Geometric Optimization A Gradient Method in -Space◊
Nikolaus Hansen, Inria, Université Paris-Saclay
. Taking a gradient gives an update for ◊t:
d◊t
dt
=natural gradient�̃∇◊ J(◊)�
◊=◊t
= ∇̃◊ E �“f-invariant, adaptive” objective������������������������������������������������
W
f◊t(f(x))��
◊=◊t
,
target distribution��������������������������������������x ∼ p(.�◊)
= . . . = E �W
f◊t(f(x))������������������������������������������������
preference weight
natural gradient�̃∇◊ ln p(x�◊)�◊=◊t���������������������������������������������������������������������������������������intrinsic direction
�
consistent
estimator≈ 1Z(⁄)
⁄�k=1�����������������������������������
taking the average
�⁄�2 − rank(f(xk))�����������������������������������������������������������������������������������������������������������������������������
preference weight
∇̃◊
log-likelihood����������������������������������������ln p(xk �◊)�◊=◊t�����������������������������������������������������������������������������������������������intrinsic direction
, xk ∼ p(.�◊t)��������������given distribution
• gradients are based on a metric (inner product): the natural gradient ∇̃◊ is
defined from the Fisher metric, as parametrization invariant, as compatible
with entropy and with KL-divergence
• works also in discrete ◊-spaces (maximal f -improvement under minimal
entropy change)
![Page 15: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/15.jpg)
Randomized optimization template
◊t
◊t
1. Sample distribution P (x|◊t) æ x1, . . . , x⁄ œ X2. Evaluate samples on f æ f(x1), . . . , f(x⁄)3. Update parameters
◊t+1 = ◊t + ÷
1Z(⁄)
⁄ÿ
k=1
!⁄/2 ≠ rank(f(xk))
"ÂÒ◊ ln p(xk|◊)--◊=◊t
15Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 16: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/16.jpg)
16
Instantiation in : CMA-ES in a nutshellRn
Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is
a second-order method, similar to quasi-Newton methods,
however randomized and function-value free
1. P (x|◊) is a multivariate normal distribution
◊ represents mean and covariance matrix
2. the ◊-update is a (smoothed) ML-update
• separate for mean and covariance matrix
• designed as invariant under linear coordinate system
transformations
• mainly coincides with a natural gradient ascent
3. step-size control thrives for orthogonal steps
based on non-local information, correlations
between steps
Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 17: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/17.jpg)
Covariance Matrix Adaptation2-D search space
17
![Page 18: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/18.jpg)
2-D search space
Covariance Matrix Adaptation
18
![Page 19: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/19.jpg)
2-D search space
Covariance Matrix Adaptation
19
![Page 20: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/20.jpg)
2-D search space
Covariance Matrix Adaptation
20
![Page 21: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/21.jpg)
2-D search space
Covariance Matrix Adaptation
21
![Page 22: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/22.jpg)
2-D search space
Covariance Matrix Adaptation
22
![Page 23: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/23.jpg)
2-D search space
Covariance Matrix Adaptation
23
![Page 24: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/24.jpg)
[Kjellstroem 1991, Hansen&Ostermeier 1996, Ljung 1999]
2-D search space
Covariance Matrix Adaptation
24
![Page 25: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/25.jpg)
25
Input: m œ Rn, ‡ œ R+, ⁄ œ NØ2, usually ⁄ Ø 5
Set cm = 1, cµ ¥ µw/n2, cc ¥ 4/n, c‡ ¥ 4/n, c1 ¥ 2/n2
, d‡ ¥ 1,
set ‚wi=1,...,⁄ decreasing in i,q
i|‚wi| = 1 and µ≠1
w :=
qi
‚w 2i ¥ 3/⁄
Initialize C = I, and pc = 0, p‡ = 0
While not terminatexi = m + ‡ yi ≥ N
!m, ‡2
C
", for i = 1, . . . , ⁄ sampling
m Ω m + cm‡q
i‚wfl(i) yi =: m + cm‡yw, update mean
p‡ Ω (1 ≠ c‡) p‡ +
1 ≠ (1 ≠ c‡)
2Ôµw C
≠ 12
yw path for ‡
‡ Ω ‡ ◊ exp
!c‡d‡
! Îp‡ÎEÎN(0,I)Î ≠ 1
""update of ‡
pc Ω (1 ≠ cc) pc + 1I[0,2n])Îp‡Î2
* 1 ≠ (1 ≠ cc)
2Ôµw yw path for C
C Ω C + cµ
q⁄
i=1 ‚wfl(i) (yiyTi ≠ C) + c1 (pc p
Tc ≠ C) update C
CMA-ES (Covariance Matrix Adaptation Evolution Strategy)= natural gradient ascent + cumulation + step-size control
green shade: natural gradient red shade: not explained by natural gradient
(a default is available)
![Page 26: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/26.jpg)
Step-Size Control: The Concept
too large neutral and optimal too small step-size step-size step-size
search paths in 2-Dshort expected long
If several updates go into the same or similar direction (if they havethe same sign) increase the step-size
26
![Page 27: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/27.jpg)
27
Design Principles Applied in CMA-ES
![Page 28: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/28.jpg)
28
Invariance: Function-Value Free Property
28
Three functions belonging to the same equivalence class
f = h f = g1 ¶ h f = g2 ¶ h
A function-value free search algorithm is invariant under the
transformation with any order preserving (strictly increasing) g.
Invariances make
• observations meaningful
as a rigorous notion of generalization
• algorithms predictable and/or ”robust”
![Page 29: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/29.jpg)
29
Invariance Under Rigid Search Space
related publications: [2,14,41]
−3 −2 −1 0 1 2 3−3
−2
−1
0
1
2
3
for example, invariance under search space rotation
(separable … non-separable)
f-level sets in dimension 2f = hRast f = h
![Page 30: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/30.jpg)
30
Invariance Under Rigid Search Space
−3 −2 −1 0 1 2 3−3
−2
−1
0
1
2
3f-level sets in dimension 2f = hRast ¶ R f = h ¶ R
for example, invariance under search space rotation
(separable … non-separable)
invariances make observations meaningful
![Page 31: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/31.jpg)
Landscape of Continuous Search Methods
31
Gradient-based (Taylor, local)
• Conjugate gradient methods [Fletcher & Reeves 1964]
• Quasi-Newton methods (BFGS) [Broyden et al 1970]
Derivative-free optimization (DFO)
• Trust-region methods (NEWUOA, BOBYQA) [Powell 2006, 2009]
• Simplex downhill [Nelder & Mead 1965]
• Pattern search [Hooke & Jeeves 1961, Audet & Dennis 2006]
Stochastic (randomized) search methods
• Evolutionary algorithms (broader sense, continuous domain)
– Differential Evolution [Storn & Price 1997]
– Particle Swarm Optimization [Kennedy & Eberhart 1995]
– Evolution Strategies [Rechenberg 1965, Hansen & Ostermeier 2001]
• Simulated annealing [Kirkpatrick et al 1983]
• Simultaneous perturbation stochastic approximation (SPSA) [Spall 2000]31
![Page 32: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/32.jpg)
Limitations of CMA-ES
32Nikolaus Hansen, Inria, Université Paris-Saclay
![Page 33: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/33.jpg)
33
http://cma.gforge.inria.fr http://cma.gforge.inria.fr/cmaes_sourcecode_page.html
Questions?
![Page 34: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/34.jpg)
…(experimental) validation...
34
![Page 35: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/35.jpg)
A simple unimodal test function
35
![Page 36: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/36.jpg)
Experimentum crucis
without covariance matrix adaptation it takes 1000 times longer to reach f = 10-1036
![Page 37: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/37.jpg)
Experimentum crucis
without covariance matrix adaptation it takes 1000 times longer to reach f = 10-1037
![Page 38: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/38.jpg)
Quantifying the enhancement
[Hansen & Ostermeier 2001]
38
![Page 39: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/39.jpg)
Runtime versus condition number
condition number
func
tion
eval
uatio
ns
separable & quadratic
[Auger et al 2009]
1
10 10 10 10 10 10
dimension 20
39
![Page 40: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/40.jpg)
Runtime versus condition numbernon-separable & quadratic
func
tion
eval
uatio
ns
condition number
2
[Auger et al 2009]
dimension 20
40
![Page 41: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/41.jpg)
Runtime versus condition numbernon-separable & non-convex
func
tion
eval
uatio
ns
condition number
3
[Auger et al 2009]
dimension 20
41
![Page 42: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/42.jpg)
42
Various Benchmarks
![Page 43: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/43.jpg)
43
Benchmarks and Applications• well suited for non-separable, ill-conditioned, rugged problems
considered as state-of-the-art, e.g., Scholarpedia, 2(8):1965, 2007
• Used for the RoboCup world champion 2011 & 2012 (team AustinVilla, 3D Nao simulation league)
in 2013 with a goal difference of 67:1 on 2nd place
• Benchmarks and Competitions
– 2005 IEEE-CEC (Special Session on Real-Parameter Optimization)restarts (IPOP-CMA-ES [36]), best algorithm
– 2009 ACM-GECCO (BBOB Black-Box Optimization BenchmarkingWorkshop) restarts (BIPOP-CMA-ES [80,81]),
best algorithm for large budgets
– 2013 IEEE-CEC (Competition on Real-Parameter Single ObjectiveOptimization) restarts (NBIPOPaCMA [Loshchilov]), best algorithm
– 2013 ACM-GECCO (BBOB Black-Box Optimization BenchmarkingWorkshop) portfolio (HCMA [Loshchilov et al]), best algorithm
![Page 44: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/44.jpg)
44
Empirical run-length distributions [# f-evaluations], AKA data profilesall 1224 problemsdimension 10
log10(f -evaluations / dimension)
proportion
ofsolved
problem
s
COCO/BBOB: 24 Functions and 51 Target Values
in: Hansen et al 2010
![Page 45: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/45.jpg)
in: I. Loshchilov , T. Stützle and T. Liao, 2013
45
2013 IEEE-CEC Competition
28 functions in dimension 10,30,50
![Page 46: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/46.jpg)
46in L.M. Rios and N.V. Sahinidis, 2013
![Page 47: Function-Value-Free Second-Order Stochastic Optimization with … · 2017. 6. 9. · – 2009 ACM-GECCO (BBOB Black-Box Optimization Benchmarking Workshop) restarts (BIPOP-CMA-ES](https://reader036.fdocuments.in/reader036/viewer/2022081617/6044e69bcabdab3d8b2f7348/html5/thumbnails/47.jpg)
Questions?
CMA-ES source code: http://www.lri.fr/~hansen/cmaes_inmatlab.htmlNikolaus Hansen, Inria, Univ. Paris-Sud