CMA-ES with local meta-models

Investigating the Local-Meta-Model CMA-ES forLarge Population Sizes

Zyed Bouzarkouna1,2 Anne Auger2 Didier Yu Ding1

1IFP (Institut Francais du Petrole)

2TAO Team, INRIA Saclay-Ile-de-France, LRI

April 07, 2010

Statement of the Problem

Objective

To solve a real-world optimization problem formulated in ablack-box scenario with an objective function f : Rn 7→ R.

f may be:

multimodal non-smoothnoisy non-convexnon-separable computationally expensive. . .

Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 2 of 15

A Real-World Problem in Petroleum Engineering

History Matching

The act of adjusting a reservoir model until it closely reproducesthe past behavior of a production history.

A fluid flow simulation takes several minutes to several hours !!


Statement of the Problem (Cont’d)

Difficulties

Evolutionary Algorithms (EAs) are usually able to cope withnoise, multiple optima . . .

Computational cost

build a model of f , based on true evaluations ;

use this model during the optimization to save evaluations.

⇒ How to decide whether:

the quality of the model is good enough to continueexploiting this model ?

ornew evaluations on the “true” objective function should be

performed ?


Table of Contents

1 CMA-ES with Local-Meta-ModelsCovariance Matrix Adaptation-ESLocally Weighted RegressionApproximate Ranking Procedure

2 A New Variant of lmm-CMAA New Meta-Model Acceptance Criterionnlmm-CMA Performance

3 Conclusions


Covariance Matrix Adaptation-ES

CMA-ES (Hansen & Ostermeier 2001)

Initialize distribution parameters m, σ and C, set population sizeλ ∈ N.while not terminate

Sample xi = m + σNi (0,C), for i = 1 . . . λ according to amultivariate normal distribution

Evaluate x1, . . . , xλ on f

Update distribution parameters(m, σ,C)← (m, σ,C, x1, . . . , xλ, f (x1), . . . , f (xλ))

where

m ∈ Rn: the mean of the multivariate normal distribution

σ ∈ R+: the step-size

C ∈ Rn×n: the covariance matrix.


Covariance Matrix Adaptation-ES (Cont’d)

Moving the mean

m =µ(=λ

2 )∑i=1

ωixi :λ.

where xi :λ is the i th ranked individual:

f (x1:λ) ≤ . . . f (xµ:λ) ≤ . . . f (xλ:λ) ,

ω1 ≥ . . . ≥ ωµ > 0,µP

i=1ωi = 1.

Other updates

Adapting the Covariance Matrix

Step-Size Control

⇒ Updates rely on the ranking of individuals according to f andnot on their exact values on f .


Locally Weighted Regression

q ∈ Rn: A point to evaluate

⇒ f (q) : a full quadratic meta-model on q.



A training set containing m points with their objective functionvalues (xj , yj = f (xj)) , j = 1..m



We select the k nearest neighbor data points to q according toMahalanobis distance with respect to the current covariance matrixC.



h is the bandwidth defined by the distance of the kth nearestneighbor data point to q.



Building the meta-model f on q

mink∑

j=1

[(f (xj , β)− yj

)2ωj

], w.r.t β ∈ R

n(n+3)2

+1.

f (q) = βT(q2

1 , · · · , q2n, · · · , q1q2, · · · , qn−1qn, q1, · · · , qn, 1

)T.


Approximate Ranking Procedure

Every generation g , CMA-ES has λ points to evaluate.

⇒ Which are the points that must be evaluated with:

the true objective function f ?

the meta-model f ?

Approximate ranking procedure (Kern et al. 2006)

1 approximate f and rank the µ best individuals

2 evaluate f on the ninit best individuals

3 for nic := 1 to“λ−ninit

nb

”do

4 approximate f and rank the µ best individuals

5 if (the exact ranking of the µ best individuals changes) then

6 evaluate f on the nb best unevaluated individuals

7 else

8 break

9 fi

10 od

11 adapt ninit depending on nic


A New Meta-Model Acceptance Criterion

Requiring the preservation of the exact ranking of the µ bestindividuals is a too conservative criterion to measure the quality ofthe meta-model.

New acceptance criteria (nlmm-CMA)

The meta-model is accepted if it succeeds in keeping:

the best individual and the ensemble of the µ best individualsunchanged

or

the best individual unchanged, if more than one fourth of thepopulation is evaluated.


nlmm-CMA Performance

Success Performance (SP1):

SP1 = mean (number of function evaluations for successful runs)ratio of successful runs .

Speedup (algo) = SP1(algo)SP1(CMA−ES) .

0

2

4

6

8

(Dimension, Population Size)

Sp

eed

up


nlmm-CMA Performance

4 nlmm-CMA � lmm-CMA

fSchwefel fSchwefel1/4 fNoisySphere

(2, 6) (4, 8) (8, 10) (16, 12)0

2

4

6

8


Sp

eed

up

(2, 6) (4, 8) (5, 8) (8, 10)0

2

4

6

8

Sp

eed

up

(Dimension, Population Size)(2, 6) (4, 8) (8, 10) (16, 12)0

2

4

6

8


Sp

eed

up

fRosenbrock fAckley fRastrigin

(2, 6) (4, 8) (5, 8) (8, 10)0

2

4

6

8


Sp

eed

up

(2, 5) (5, 7) (10, 10)0

2

4

6

8


Sp

eed

up

(2, 50) (5, 140)0

2

4

6

8


Sp

eed

up

⇒ nlmm-CMA outperforms lmm-CMA, on the test functions investigated

with a speedup between 1.5 and 7.


nlmm-CMA Performance for Increasing PopulationSizes

4 nlmm-CMA � lmm-CMADimension n = 5

fSchwefel1/4 fRosenbrock fRastrigin

8 16 24 32 48 960

1

2

3

4

5

Population Size

Sp

eed

up

8 16 24 32 48 960

1

2

3

4

5

Population Size

Sp

eed

up

70 140 2800

1

2

3

4

5

Population Size

Sp

eed

up

⇒ nlmm-CMA maintains a significant speedup,between 2.5 and 4, when

increasing λ while the speedup of lmm-CMA drops to one.


Impact of the Recombination Type

nlmm-CMA

a default weighted recombination type

ωi = ln(µ+1)−ln(i)µ ln(µ+1)−ln(µ!) , for i = 1 . . . µ.

nlmm-CMAI

an intermediate recombination type

ωi = 1µ , for i = 1 . . . µ.


Impact of the Recombination Type (Cont’d)

4 nlmm-CMA � nlmm-CMAI (with equal RT)

fSchwefel fSchwefel1/4 fNoisySphere

(2, 6) (4, 8) (8, 10) (16, 12)0

2

4

6

8


Sp

eed

up

(2, 6) (4, 8) (8, 10)0

2

4

6

8

Sp

eed

up

(Dimension, Population Size)(2, 6) (4, 8) (8, 10) (16, 12)0

2

4

6

8


Sp

eed

up

fRosenbrock fAckley fRastrigin

(2, 6) (4, 8) (8, 10)0

2

4

6

8


Sp

eed

up

(2, 5) (5, 7) (10, 10)0

2

4

6

8


Sp

eed

up

(2, 50) (5, 140)0

2

4

6

8


Sp

eed

up

⇒ nlmm-CMA outperforms nlmm-CMAI .

⇒ The ranking obtained with the new acceptance criterion still has an amount

of information to guide CMA-ES.Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 14 of 15

Summary

CMA-ES with meta-modelsThe speedup of lmm-CMA with respect to CMA-ES drops to one when the populationsize λ increases.

⇒ The meta-model acceptance criterion is too conservative.

New variant of CMA-ES with meta-modelsA new meta-model acceptance criterion: It must keep:

the best individual and the ensemble of the µ best individuals unchanged

the best individual unchanged, if more than one fourth of the population

is evaluated.

nlmm-CMA outperforms lmm-CMA on the test functions investigated with aspeedup in between 1.5 and 7.

nlmm-CMA maintains a significant speedup, between 2.5 and 4, when

increasing the population size on tested functions.


Thank You For Your Attention


Investigating the Local-Meta-Model CMA-ES forLarge Population Sizes

Zyed Bouzarkouna1,2 Anne Auger2 Didier Yu Ding1

1IFP (Institut Francais du Petrole)

2TAO Team, INRIA Saclay-Ile-de-France, LRI

April 07, 2010

CMA-ES with local meta-models

Technology

Transcript of CMA-ES with local meta-models