Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i)...
Transcript of Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i)...
![Page 1: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/1.jpg)
Multifidelity importance sampling methodsfor rare event simulation
Benjamin PeherstorferUniversity of Wisconsin-Madison
Karen Willcox and Boris KramerMIT
Max GunzburgerFlorida State University
July 2017
1 / 24
![Page 2: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/2.jpg)
Introduction
High-fidelity model with costs w1 � 0
f (1) : D → Y
Random variable Z , estimate
s = E[f (1)(Z )]
Monte Carlo estimator of s with samples Z1, . . . ,Zn
y (1)n =
1n
n∑i=1
f (1)(Zi )
Computational costs highI Many evaluations of high-fidelity modelI Typically 103 − 106 evaluationsI Intractable if f (1) expensive
high-fidelitymodel
uncertaintyquantification
outp
uty in
pu
tz
2 / 24
![Page 3: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/3.jpg)
Surrogate modelsGiven is a high-fidelity model f (1) : D → Y
I Large-scale numerical simulationI Achieves required accuracyI Computationally expensive
Additionally, often have surrogate models
f (i) : D → Y , i = 2, . . . , k
I Approximate high-fidelity f (1)
I Often orders of magnitudes cheaperExamples of surrogate models
cost
s
error
high-fidelitymodel
surrogatemodel
surrogatemodel
surrogatemodel
surrogatemodel
data-fit models,response surfaces,machine learning
coarse-gridapproximations
RN
u(z1)
u(z2)
u(zM)
{u(z) | z ∈ D}
reduced basis,proper orthogonaldecomposition
simplified models,linearized models
3 / 24
![Page 4: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/4.jpg)
Surrogate models in uncertainty quantification
Replace f (1) with a surrogate modelI Costs of uncertainty quantification reducedI Often orders of magnitude speedups
Estimate depends on surrogate accuracyI Control with error bounds/estimatorsI Rebuild if accuracy too lowI No guarantees without bounds/estimators
IssuesI Propagation of surrogate error on estimateI Surrogates without error controlI Costs of rebuilding a surrogate model
surrogatemodel
uncertaintyquantification
outp
uty in
pu
tz
4 / 24
![Page 5: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/5.jpg)
Our approach: Multifidelity methodsCombine high-fidelity and surrogate models
I Leverage surrogate models for speedupI Recourse to high-fidelity for accuracy
Multifidelity speeds up computationsI Balance #solves among modelsI Adapt, fuse, filter with surrogate models
Multifidelity guarantees high-fidelity accuracyI Occasional recourse to high-fidelity modelI High-fidelity model is kept in the loopI Independent of error control for surrogates
[P., Willcox, Gunzburger, Survey of multifidelity methods in uncertainty propagation, inference, and opti-mization; available online as technical report, MIT, 2016]
high-fidelitymodel
surrogatemodel
uncertaintyquantification
outp
uty in
pu
tz
.
.
5 / 24
![Page 6: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/6.jpg)
Our approach: Multifidelity methodsCombine high-fidelity and surrogate models
I Leverage surrogate models for speedupI Recourse to high-fidelity for accuracy
Multifidelity speeds up computationsI Balance #solves among modelsI Adapt, fuse, filter with surrogate models
Multifidelity guarantees high-fidelity accuracyI Occasional recourse to high-fidelity modelI High-fidelity model is kept in the loopI Independent of error control for surrogates
[P., Willcox, Gunzburger, Survey of multifidelity methods in uncertainty propagation, inference, and opti-mization; available online as technical report, MIT, 2016]
high-fidelitymodel
...
surrogatemodel
surrogatemodel
uncertaintyquantification
outp
uty in
pu
tz
.
.
5 / 24
![Page 7: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/7.jpg)
Multifidelity Monte Carlo (MFMC)MFMC use control variates for var reduction
I Derives control variates from surrogatesI Uses number of samples that minimize error
[P., Willcox, Gunzburger: Optimal model management for multi-fidelity Monte Carlo estimation. SIAM Journal on Scientific Com-puting, 2016]
Multifidelity sensitivity analysisI Identify the parameters of model with largest
influence on quantity of interestI Elizabeth Qian (MIT)/Earth Science (LANL)
Asymptotic analysis of MFMC[P., Gunzburger, Willcox: Convergence analysis of multifidelityMonte Carlo estimation, submitted. 2016]
MFMC with information reuse[Ng, Willcox: Monte Carlo Information-Reuse Approach to Air-craft Conceptual Design Optimization Under Uncertainty. 2015]
MFMC with optimally-adapted surrogates100 101 102
Computational budget (s)
10-6
10-5
10-4
10-3
10-2
10-1
Vari
ance
of
mean e
stim
ate
(MF)MC hydraulic conductivity estimation
MCMFMC
Figures: Elizabeth Qian (MIT)
6 / 24
![Page 8: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/8.jpg)
Multifidelity rare event simulation based onimportance sampling
with Karen Willcox and Boris Kramer
7 / 24
![Page 9: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/9.jpg)
MFCE: Problem setup
Threshold 0 < t ∈ R and random variable
Z ∼ p
Estimate rare event probability
Pf = Pp[f (1) ≤ t]
Can be reformulated to estimating E
Pf = Ep[I(1)t ]
with indicator function
I(1)t (z) =
{1, f (1)(z) ≤ t ,
0, f (1)(z) > t
If Pf � 1, very unlikely that we hit f (1) ≤ t
-0.5 0 0.5 1 1.5outputs f (1)(z)
0
1
2
3
4
5
dens
ity
realizationsdensity
8 / 24
![Page 10: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/10.jpg)
MFCE: Importance sampling
Consider a biasing density q with
supp(p) ⊆ supp(q)
Reformulate estimation problem
Pf = Ep[I(1)t ] = Eq
[I(1)t
p
q
]Goal is constructing suitable q with
Varq
[I(1)t
p
q
]� Varp[I(1)t ]
⇒ Use surrogate models -0.5 0 0.5 1 1.5outputs f (1)(z)
0
1
2
3
4
5
dens
ity
realizationsnominalbiasing
9 / 24
![Page 11: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/11.jpg)
MFCE: Literature review
Two-fidelity approachesI Switch between models [Li, Xiu et al., 2010, 2011, 2014]
I Reduced basis models with error estimators [Chen and Quarteroni, 2013]
I Kriging models and importance sampling [Dubourg et al., 2013]
I Subset method with machine-learning-based models [Bourinet et al., 2011],
[Papadopoulos et al., 2012]
I Surrogates and importance sampling [P., Cui, Marzouk, Willcox, 2016]
Multilevel methods for rare event simulationI Variance reduction via control variates [Elfverson et al., 204, 2016], [Fagerlund et al., 2016]
I Subset method with coarse-grid approximations [Ullmann and Papaioannou, 2015]
Combining multiple general types of surrogatesI Importance sampling + control variates [P., Kramer, Willcox, 2017]
10 / 24
![Page 12: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/12.jpg)
MFCE: Direct sampling of surrogate models
Directly sampling surrogate models to construct biasing densityI Reduces costs per sampleI Number of samples to construct biasing density remains the sameI Works well for probabilities > 10−5
⇒ Insufficient for very rare event probabilities in range of 10−9
00.51
1.52
2.53
3.54
t mean
density
realizations of f (1)(Z )
[P., Cui, Marzouk, Willcox, 2016], [P., Kramer, Willcox, 2017]
11 / 24
![Page 13: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/13.jpg)
MFCE: Direct sampling of surrogate models
Directly sampling surrogate models to construct biasing densityI Reduces costs per sampleI Number of samples to construct biasing density remains the sameI Works well for probabilities > 10−5
⇒ Insufficient for very rare event probabilities in range of 10−9
00.51
1.52
2.53
3.54
t mean
density
realizations of f (1)(Z )
[P., Cui, Marzouk, Willcox, 2016], [P., Kramer, Willcox, 2017]
11 / 24
![Page 14: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/14.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 15: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/15.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 16: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/16.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 17: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/17.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t t3 t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 18: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/18.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t t4 t3 t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 19: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/19.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t t5 t4 t3 t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 20: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/20.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
t t6 t5 t4 t3 t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 21: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/21.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
tt7 t6 t5 t4 t3 t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 22: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/22.jpg)
MFCE: Construct biasing density iteratively
Threshold t controls “rareness” of event
I(1)t (z) =
{1 , f (1)(z) ≤ t ,
0 , f (1)(z) > t
Cross-entropy method constructs densities iteratively t1 > t2 > · · · > t
00.51
1.52
2.53
3.54
tt7 t6 t5 t4 t3 t2 t1 mean
density
realizations of f (1)(Z )
[Rubinstein, 1999], [Rubinstein, 2001]
12 / 24
![Page 23: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/23.jpg)
MFCE: Cross-entropy method – in each step
Need to find biasing density in each step i = 1, . . . ,TI Optimal biasing density that reduces variance to 0
qi (z) ∝ I(1)ti (z)p(z)
⇒ Unknown normalizing constant (quantity we want to estimate)I Find qv i ∈ Q = {qv : v ∈ P} with min Kullback-Leibler distance to qi
minv i∈P
DKL(qi ||qv i )
I Reformulate as (independent of normalizing constant of qi )
maxv i∈P
Ep[I(1)ti log(qv i )]
I Solve approximately by replacing Ep with Monte Carlo estimator
maxv i∈P
1m
m∑i=1
I(1)ti (Zi ) log(qv i (Zi )) , Z1, . . . ,Zm ∼ p
⇒ Optimization problems affected by rareness of event I(1)ti (Z )[Rubinstein, 1999], [Rubinstein, 2001]
13 / 24
![Page 24: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/24.jpg)
MFCE: Iterative cross-entropy procedureReuse qv i−1 as biasing density for constructing qv i
maxv i∈P
Eqv i−1[I(1)ti
p
qv i−1
log(qv i )]
I Choose t1 � t, solve for v1 ∈ P with
maxv1∈P
Ep[I(1)t1 log(qv1)]
I Select t2 < t1, solve for v2 ∈ P with
maxv2∈P
Eqv1[I(1)t2
p
qv1
log(qv2)]
I Repeat until threshold t is reached and parameter v∗ is obtainedI Reweighed optimization problems have same optimum as original problems
Once qv∗ has been found, cross-entropy estimator of Pf is
PCEt =
1m
m∑i=1
I(1)t (Z∗i )p(Z∗i )
qv∗(Z∗i ), Z∗1 , . . . ,Z
∗m ∼ qv∗
14 / 24
![Page 25: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/25.jpg)
MFCE: Iterative cross-entropy procedureReuse qv i−1 as biasing density for constructing qv i
maxv i∈P
Eqv i−1[I(1)ti
p
qv i−1
log(qv i )]
I Choose t1 � t, solve for v1 ∈ P with
maxv1∈P
Ep[I(1)t1 log(qv1)]
I Select t2 < t1, solve for v2 ∈ P with
maxv2∈P
Eqv1[I(1)t2
p
qv1
log(qv2)]
I Repeat until threshold t is reached and parameter v∗ is obtainedI Reweighed optimization problems have same optimum as original problems
Once qv∗ has been found, cross-entropy estimator of Pf is
PCEt =
1m
m∑i=1
I(1)t (Z∗i )p(Z∗i )
qv∗(Z∗i ), Z∗1 , . . . ,Z
∗m ∼ qv∗
14 / 24
![Page 26: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/26.jpg)
MFCE: Costs of the cross-entropy method
Step ti defined by quantile parameter ρI Quantile parameter typically ρ ∈ [10−2, 10−1]
I Step ti is ρ quantile corresponding to qv i−1
Number of steps T
I Introduce minimal step size δ > 0I Number steps T is bounded as
T ≤ t1 − t
δ
Costs of cross-entropy method bounded as
c(PCEt ) ≤ (t1 − t)mw1
δ
00.51
1.52
2.53
3.54
tt7 t6 t5 t4 t3 t2 t1 meandensity
realizations of f (1)(Z )
⇒ Critically depends on t1 and thus on density used in first step
15 / 24
![Page 27: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/27.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
t mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 28: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/28.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
t t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 29: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/29.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
t t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 30: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/30.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
t t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 31: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/31.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
t t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 32: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/32.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
t t5 t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 33: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/33.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
tt6 t5 t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 34: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/34.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
tt6 t5 t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 35: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/35.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
tt6 t5 t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
t meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 36: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/36.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
tt6 t5 t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
tt1 meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 37: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/37.jpg)
MFCE: Multifidelity-preconditioned cross-entropy method
Our approach: Use surrogates to reduce t1 and thus #iters with f (1)
f (2), . . . , f (k) : Z → Y
Multifidelity-preconditioned cross-entropy (MFCE) methodsurrogate model high-fidelity model
00.51
1.52
2.53
3.54
tt6 t5 t4 t3 t2 t1 mean
density
realizations of f (2)(Z )
00.51
1.52
2.53
3.54
tt1 meandensity
realizations f (1)(Z )
I Find biasing density q(k)v∗ with surrogate f (k)
I Start with q(k)v∗ to find biasing density q
(k−1)v∗ with f (k−1)
I Repeat until q(1)v∗ is found with f (1)
16 / 24
![Page 38: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/38.jpg)
MFCE: Analysis for two models
Consider high-fidelity f (1) and low-fidelity model f (2)
I Let q(2)v∗ be the biasing density found with f (2)
I Set t(1)1 to be the ρ-quantile corresponding to q(2)v∗
I Set tp to be the ρ-quantile corresponding to the nominal density pI Need to show
Pq(2)v∗
[f (1) ≤ t
(1)1
]− Pp
[f (1) ≤ t
(1)1
]≥ 0
to have t(1)1 ≤ tp
Make two assumptionsI A “local” bound 0 < α on the error |f (1)(z)− f (2)(z)|I Lipschitz continuous distribution functions F (i)
q (t) = Pq[f (i) ≤ t]
Proposition 2 in [P., Kramer, Willcox, 2017]
Pq(2)v∗
[f (1) ≤ t
(1)1
]− Pp
[f (1) ≤ t
(1)1
]& −α
17 / 24
![Page 39: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/39.jpg)
MFCE: Numerical examples: Heat transfer
Consider
−∇ · (a(ω, ξ)∇u(ω, ξ)) = 1 , ξ ∈ (0, 1) , (1)u(ω, 0) = 0 , (2)
∂nu(ω, 1) = 0 , (3)
I Coefficient a is given as
a(ω, ξ) = ez1(ω)e−0.5|ξ−0.5|0.0225 + ez2(ω)e−0.5
|ξ−0.8|0.0225
I Random vector Z = [z1, z2] normally distributedI System response
f (Z (ω)) = u(ω, 1)
I Discretize with varying mesh width h ∈ {2−8, 2−7, . . . , 2−4}I Obtain models f (1), . . . , f (5)
Goal is to estimate Pf = Pp[f (1) ≤ 1.4] with reference Pf ≈ 6× 10−9
18 / 24
![Page 40: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/40.jpg)
MFCE: Numerical examples: Speedup
1e-06
1e-05
1e-04
1e-03
1e-02
1e-01
1e+00 1e+01 1e+02 1e+03 1e+04 1e+05
est.squaredcoeff
.ofvar.
runtime biasing construction [s]
CE, single modelMFCE
1e-06
1e-05
1e-04
1e-03
1e-02
1e-01
1e+00 1e+01 1e+02 1e+03 1e+04 1e+05
est.squaredcoeff
.ofvar.
total runtime [s]
CE, single modelMFCE
(a) biasing runtime (b) total runtime
I Single-fidelity approach uses f (1)
I MFCE uses f (1), . . . , f (5)
I MFCE reduces runtime by almost 2 orders of magnitude
19 / 24
![Page 41: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/41.jpg)
MFCE: Numerical examples: Number of iterations
0
2
4
6
8
10
level 4level 5
level 6level 7
level 8
avgnu
mberof
CEite
ratio
nsCE, single model
MFCE
I Number of iterations averaged over 30 runsI MFCE performs most iterations with coarse-grid models
20 / 24
![Page 42: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/42.jpg)
MFCE: Numerical examples: Reacting flow
x1 [cm]
x 2 [cm
]
0 0.5 1 1.5
0.2
0.4
0.6
0.8
tem
p [K
]
0
500
1000
1500
2000
1e-04
1e-03
1e-02
1e-01
1e+00 1e+01 1e+02 1e+03 1e+04 1e+05
est.squaredcoeff
.ofvar.
runtime biasing construction [s]
CE, single modelMFCE
1e-04
1e-03
1e-02
1e-01
1e+01 1e+02 1e+03 1e+04 1e+05est.squaredcoeff
.ofvar.
total runtime [s]
CE, single modelMFCE
(a) biasing runtime (b) total runtime
Reacting-flow problemI Three models: data-fit, projection-based, and high-fidelity modelI Estimate probability that temperature is below threshold, ref Pf ≈ 10−6I MFCE reduces iterations with high-fidelity model from ≈ 5 to ≈ 1
21 / 24
![Page 43: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/43.jpg)
MFCE: Apply to OpenAeroStruct
Consider baseline UAV definition in OpenAeroStructI Design variables are thickness and position of control pointsI Uncertain flight conditions (angle of attack, air density, Mach number)I Output is fuel burn
Estimate 10−6-quantile γ (value-at-risk)
Pp[f (1) ≤ γ] = 10−6
Derive a data-fit surrogateI Take a 3× 3× 3 equidistant grid in stochastic domainI Evaluate high-fidelity model at those 27 pointsI Derive linear interpolant of output
Apply multifidelity pre-conditioned cross-entropy method
22 / 24
![Page 44: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/44.jpg)
MFCE: OpenAeroStruct: Value-at-Risk computation
1e-08
1e-07
1e-06
1e-05
1e-04
1e+03 1e+04 1e+05 1e+06
est.mean-squarederror
total runtime [s]
CE, single modelmultifidelity
I Computing 10−6-quantile for a fixed design pointI Multifidelity approach achieves up to one order of magnitude speedup
23 / 24
![Page 45: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/45.jpg)
Conclusions
1e-06
1e-05
1e-04
1e-03
1e-02
1e-01
1e+00 1e+01 1e+02 1e+03 1e+04 1e+05
est.squaredcoeff
.ofvar.
total runtime [s]
CE, single modelMFCE
0
2
4
6
8
10
level 4level 5
level 6level 7
level 8
avgnu
mberof
CEite
ratio
ns CE, single modelMFCE
Multifidelity rare event simulationI Leverage surrogate models for runtime speedupI Recourse to high-fidelity model for accuracy guarantees
Our references1 P., Cui, Marzouk, Willcox: Multifidelity Importance Sampling. Computer Methods in Applied
Mechanics and Engineering, 300:490-509, 2016.2 P., Kramer, Willcox: Combining multiple surrogate models to accelerate failure probability
estimation with expensive high-fidelity models. J. of Computational Physics, 341:61-75, 2017.3 P., Kramer, Willcox: Multifidelity preconditioning of the cross-entropy method for rare event
simulation and failure probability estimation. submitted, 2017.24 / 24
![Page 46: Multifidelity importance sampling methods for rare event ... · n y(1) n = 1 n Xn i=1 f(1)(Z i) Computationalcostshigh I Manyevaluationsofhigh-fidelitymodel I Typically103 106 evaluations](https://reader034.fdocuments.in/reader034/viewer/2022050122/5f5237105612d44aa91ba2df/html5/thumbnails/46.jpg)
PostDoc positionavailable• data-driven model reduction• uncertainty propagation• Bayesian inverse problems
https://[email protected]
24 / 24