Post on 24-May-2020
Discrete optimality in nonlinear model reduction:analysis and application to CFD
Kevin Carlberg1, Matthew Barone1, Harbir Antil2
Sandia National Laboratories1
George Mason University2
1st Pan-American Congress on Computational MechanicsBuenos Aires, Argentina
April 27, 2015
Discrete optimality Carlberg, Barone, Antil 1 / 37
Model reduction at Sandia
CFD model
100 million cells200,000 time steps
High simulation costs
6 weeks, 5000 cores6 runs maxes out Cielo
Barrier
Design engineers requirefaster simulations
Uncertainty quantification
Objective: break barrier
Discrete optimality Carlberg, Barone, Antil 2 / 37
Discrete optimality outperforms Galerkin, but why?
Discrete-optimal ROM outperforms Galerkin on large-scalecompressible-flow problems [Carlberg, 2011, Carlberg et al., 2013]
K.$Carlberg /25GNAT$nonlinear$model$reduction
Ahmed$body:$MinimumRresidual$ROM$is$accurate
10
0 0.02 0.04 0.06 0.08 0.10.345
0.35
0.355
time$(s)
drag
coefficient FOM$(82,074$dof)
MinimumRres.$ROM$
(127$POD$vectors)
Galerkin$ROM
$(127$POD$vectors)
discrete(optimal.ROM.(127.POD.vectors)
Galerkin.ROM.(127.POD.vectors)
full(order.model.(82,000.dof)
time.(s)
drag
.coe
fficien
t
Strong performance attributed to discrete optimality
Limited comparative analysis of the two approaches
Goal: Deeper understanding of Galerkin v.discrete-optimal projection for general time integrators
Discrete optimality Carlberg, Barone, Antil 3 / 37
Outline
Time-continuous and time-discrete representations
Equivalence conditions
Discrete-time error bounds
Discrete optimality Carlberg, Barone, Antil 4 / 37
Outline
Time-continuous and time-discrete representations
Equivalence conditions
Discrete-time error bounds
Discrete optimality Carlberg, Barone, Antil 5 / 37
Continuous and discrete representations
Full-order modelODE
time discretization
Full-order modelO∆E
Discrete optimality Carlberg, Barone, Antil 6 / 37
Full-order model
ODE (initial value problem)
dx
dt= f(x, t), x(0) = x0,
O∆E, linear multistep schemes: rn (wn) = 0
rn (w) := α0w −∆tβ0f(w, tn) +k∑
j=1
αjxn−j −∆t
k∑j=1
βj f(
xn−j , tn−j)
xn = wn (explicit state update)
O∆E, Runge–Kutta: rni (wn1, . . . ,w
ns ) = 0 , i = 1, . . . , s
rni (w1, . . . ,ws) := wi − f(xn−1 + ∆ts∑
j=1
aijwj , tn−1 + ci∆t)
xn = xn−1 + ∆ts∑
i=1
biwni (explicit state update)
Discrete optimality Carlberg, Barone, Antil 7 / 37
Continuous and discrete representations
Full-order modelODE
Galerkinprojection
Continuous-optimal ROMODE
time discretization
Full-order modelO∆E
Discrete optimality Carlberg, Barone, Antil 8 / 37
Galerkin ROM: continuous representation
ODE: Galerkin projection on FOM ODE
1 x(t) ≈ x(t) = Φx(t)⇡ =
2 ΦT (f(x, t)− d xdt ) = 0
⇡ =
( (=
d x
dt= ΦT f(Φx, t), x(0) = ΦTx0.
Theorem
The Galerkin ROM velocity minimizes the error in the FOM velocity fover range (Φ):
d x
dt(Φx, t) = arg min
v∈range(Φ)‖v − f(Φx, t)‖2
2.
Discrete optimality Carlberg, Barone, Antil 9 / 37
Continuous and discrete representations
Full-order modelODE
Galerkinprojection
Continuous-optimal ROMODE
time discretization
Continuous-optimal ROMO∆E
time discretization
Full-order modelO∆E
Discrete optimality Carlberg, Barone, Antil 10 / 37
Galerkin ROM: discrete representation
O∆E, linear multistep schemes: rn (wn) = 0
rn (w) := α0w − ∆tβ0ΦT f(Φw, tn) +k∑
j=1
αj xn−j − ∆t
k∑j=1
βjΦT f(
Φxn−j , tn−j)
xn = wn (explicit state update)
O∆E, Runge–Kutta: rni (wn1, . . . , w
ns ) = 0 , i = 1, . . . , s.
rni (w1, . . . , ws) := wi − ΦT f(Φxn−1 + ∆ts∑
j=1
aijΦwj , tn−1 + ci∆t)
xn = xn−1 + ∆ts∑
i=1
bi wni (explicit state update)
Discrete optimality Carlberg, Barone, Antil 11 / 37
Galerkin ROM: Commutativity
Theorem
Projection and time discretization are commutative for Galerkin ROMs:
rn (w) = ΦT rn (Φw)
rni (w1, . . . , ws) = ΦT rni (Φw1, . . . ,Φws) , i = 1, . . . , s,
Full-order modelODE
Galerkinprojection
Continuous-optimal ROMODE
time discretizationtime discretization
Full-order modelO∆E
Galerkinprojection
Continuous-optimal ROMO∆E
Discrete optimality Carlberg, Barone, Antil 12 / 37
Continuous and discrete representations
Full-ordermodelODE
Galerkinprojection
Continuous-optimal ROM
ODE
time discretizationtime discretization
Full-ordermodelO∆E
Galerkinprojection
Continuous-optimal ROM
O∆E
Residualminimization
Discrete-optimal ROM
O∆E
Discrete optimality Carlberg, Barone, Antil 13 / 37
Discrete-optimal ROM: discrete representation
O∆E, linear multistep schemes:
wn = arg minz∈Rp‖Arn (Φz) ‖2
2.
m
Ψn(wn)T rn(Φwn) = 0, Ψn(w) := ATA
∂rn
∂w(Φw)
A = I: Least-squares Petrov–Galerkin [LeGresley, 2006, Carlberg et al., 2011]
A = (PΦr )+ P: GNAT [Carlberg et al., 2013]
Alternative norm: `1[Abgrall and Amsallem, 2015]
O∆E, Runge–Kutta:(wn
1, . . . , wns
)= arg min
(z1,...,zs )∈Rp×s
s∑i=1
‖Ai rni (Φz1, . . . ,Φzs) ‖2
2
ms∑
j=1
Ψnij(wn
1, . . . , wns )T rnj
(Φwn
1, . . . ,Φwns
)= 0, i = 1, . . . , s
Ψnij(w1, . . . , ws) := AT
i Ai∂rni∂wj
(w1, . . . , ws)
Discrete optimality Carlberg, Barone, Antil 14 / 37
Continuous and discrete representations
Full-ordermodelODE
?Galerkin
projection
Continuous-optimal ROM
ODE
time discretizationtime discretization
Full-ordermodelO∆E
Galerkinprojection
Continuous-optimal ROM
O∆E
Residualminimization
Discrete-optimal ROM
O∆E
Discrete optimality Carlberg, Barone, Antil 15 / 37
Continuous and discrete representations
The discrete-optimal ROM sometimes has atime-continuous representation.
Full-ordermodelODE
Petrov–Galerkinprojection
Discrete-optimal ROM
ODE
time discretization
Galerkinprojection
Continuous-optimal ROM
ODE
time discretizationtime discretization
Full-ordermodelO∆E
Galerkinprojection
Continuous-optimal ROM
O∆E
Residualminimization
Discrete-optimal ROM
O∆E
Discrete optimality Carlberg, Barone, Antil 16 / 37
Discrete-optimal ROM: continuous representation
Theorem (Linear multistep schemes)
The discrete-optimal ROM is equivalent to applying a Petrov–Galerkinprojection to the ODE with test basis
Ψ(x, t) = ATA
(α0I−∆tβ0
∂f
∂x(x0 + Φx, t)
)Φ
and subsequently applying time integration with time step ∆t if
1 βj = 0, j ≥ 1 (e.g., a single-step method),
2 the velocity f is linear in the state, or
3 β0 = 0 (i.e., explicit schemes).
Discrete optimality Carlberg, Barone, Antil 17 / 37
Discrete-optimal ROM: continuous representation
Theorem (Runge–Kutta schemes)
The discrete-optimal ROM is equivalent to applying a Petrov–Galerkinprojection to the ODE with test basis
Ψ(x, t) = ATA
(I−∆ta11
∂f
∂x(x0 + Φx, t)
)Φ
and subsequently applying time integration if either
1 aij = 0 ∀i 6= j and aii = ajj ∀i , j , or
2 the scheme is explicit, i.e., aij = 0, ∀j ≥ i .
Discrete optimality Carlberg, Barone, Antil 18 / 37
Outline
Time-continuous and time-discrete representations
Equivalence conditions
Discrete-time error bounds
Discrete optimality Carlberg, Barone, Antil 19 / 37
Equivalence
Ψn(w) := ATA∂rn
∂w(Φw) = ATA
(α0I−∆tβ0
∂f
∂x(Φw, tn)
)Φ
Theorem (Linear multistep schemes)
Galerkin projection is discrete-optimal (Ψn(w) = Φ)
1 in the limit of ∆t → 0 with A = 1/√α0I,
2 if the scheme is explicit (β0 = 0) with A = 1/√α0I, or
3 if ∂rn
∂w is positive definite with A the Cholesky factor of [∂rn
∂w ]−1
Discrete optimality Carlberg, Barone, Antil 20 / 37
Outline
Time-continuous and time-discrete representations
Equivalence conditions
Discrete-time error bounds
Discrete optimality Carlberg, Barone, Antil 21 / 37
Discrete-time error bound
Theorem (Linear multistep schemes)
If the following conditions hold:
1 f(·, t) is Lipschitz continuous with Lipschitz constant κ, and
2 ∆t is such that 0 < h := |α0| −|β0|κ∆t,
then∥∥δxnG∥∥ ≤ ∆t
h
k∑`=0
|β`|∥∥∥∥(I− V) f
(x0 + Φxn−`
G
)∥∥∥∥+1
h
k∑`=1
(|β`|κ∆t +|α`|
)∥∥∥δxn−`G
∥∥∥∥∥δxnD
∥∥ ≤ ∆t
h
k∑`=0
|β`|∥∥∥∥(I− Pn) f
(x0 + Φxn−`
D
)∥∥∥∥+1
h
k∑`=1
(|β`|κ∆t +|α`|
)∥∥∥δxn−`D
∥∥∥ ,with
δxnG := xn? −ΦxnG .
δxnD := xn? −ΦxnD
V := ΦΦT
Pn := Φ(
(Ψn)TΦ)−1
(Ψn)T
Discrete optimality Carlberg, Barone, Antil 22 / 37
Discrete-time error bound
Theorem (Backward Euler)
If conditions (1) and (2) hold, then∥∥δxnG∥∥ ≤ ∆t
n−1∑j=0
1
(h)j+1
∥∥∥∥(I− V) f(
x0 + Φxn−jG
)∥∥∥∥︸ ︷︷ ︸εn−jG∥∥δxnD
∥∥ ≤ ∆t
n−1∑j=0
1
(h)j+1
∥∥∥∥(I− Pn−j)
f(
x0 + Φxn−jD
)∥∥∥∥︸ ︷︷ ︸εn−jD
εkG =
∥∥∥∥ΦxkG −∆tf(
x0 + ΦxkG
)−Φxk−1
G
∥∥∥∥εkD =
∥∥∥∥ΦxkD −∆tf(
x0 + ΦxkD
)−Φxk−1
D
∥∥∥∥ = miny
∥∥∥Φy −∆tf (x0 + Φy)−Φxk−1D
∥∥∥Corollary (Discrete-optimal smaller error bound)
If xk−1D = xk−1
G , then εkD ≤ εkG .
Discrete optimality Carlberg, Barone, Antil 23 / 37
Discrete-optimal time-step dependence
Corollary
Define xj as the full-space solution centered at the discrete-optimalsolution:
xj = ∆tf(
x0 + xj)
+ Φxj−1D , j = 1, . . . , n.
Then, the discrete-optimal error can be bounded as∥∥δxnD∥∥ ≤ ∆t(1 + κ∆t)n−1∑j=0
µn−j
(h)j+1‖f(xn−j)‖
with∆xj := xj −Φxj−1
D
∆xjD := xjD − xj−1D
µj :=∥∥∥Φ∆xjD −∆xj
∥∥∥ /∆xj
Effect of decreasing ∆t:
+ The terms ∆t(1 + κ∆t) and 1/(h)j+1 decrease- The number of total time instances n increases? The term µn−j may increase or decrease, depending on the
spectral content of the basis ΦDiscrete optimality Carlberg, Barone, Antil 24 / 37
Example: Cavity-flow problem
Unsteady Navier–Stokes
DES turbulence model
1.2 million degrees offreedom
Linear multistep: BDF2
∆t? = 1.5× 10−3 sec bytime-step verification study
Re = 6.3× 106
M∞ = 0.6
Discrete optimality Carlberg, Barone, Antil 25 / 37
FOM responses
Figure : vorticity field
Figure : pressure fieldDiscrete optimality Carlberg, Barone, Antil 26 / 37
Galerkin and discrete-optimal responses for basis dimension p = 204
0 1 2 3 4 5 61.4
1.6
1.8
2
2.2
2.4
2.6
2.8
t ime
pre
ssure
FOM, ∆t = 0.0015∆t=9.375e-05∆t=0.0001875∆t=0.000375∆t=0.00075∆t=0.0015∆t=0.003∆t=0.006∆t=0.012∆t=0.015∆t=0.024
0 2 4 6 8 10 12 141.6
1.8
2
2.2
2.4
2.6
2.8
3
t ime
pre
ssure
FOM, ∆t = 0.0015∆t=0.00015∆t=0.0001875∆t=0.000375∆t=0.00075∆t=0.0015∆t=0.003∆t=0.006∆t=0.012∆t=0.015∆t=0.024
(a) Galerkin
0 2 4 6 8 10 12 141.6
1.8
2
2.2
2.4
2.6
2.8
3
t ime
pre
ssure
FOM, ∆t = 0.0015∆t=0.00015∆t=0.0001875∆t=0.000375∆t=0.00075∆t=0.0015∆t=0.003∆t=0.006∆t=0.012∆t=0.015∆t=0.024
(b) Discrete optimal
- Galerkin ROMs unstable for all time steps. Consistent withprevious results [Carlberg et al., 2013, Carlberg et al., 2011, Carlberg, 2011]
+ Discrete-optimal ROMs accurate and stable, with a cleardependence on the time step ∆t.
Discrete optimality Carlberg, Barone, Antil 27 / 37
Superior performance (p = 204)
10−5
10−4
10−3
10−2
10−1
∆t
errorin
time-averagedpressure
1
GalerkinMinimum residual
(c) 0 ≤ t ≤ 0.55
10−5
10−4
10−3
10−2
10−1
∆t
errorin
time-averagedpressure
1
GalerkinMinimum residual
(d) 0 ≤ t ≤ 1.1
10−4
10−2
10−4
10−3
10−2
10−1
∆t
errorin
tim
e-averagedpressure
1
GalerkinMinimum residual
(e) 0 ≤ t ≤ 1.54
X When Galerkin is stable, the discrete-optimal ROM yields asmaller error for all time intervals and time steps.
Discrete optimality Carlberg, Barone, Antil 28 / 37
Limiting equivalence
tim
e-av
erag
edpre
ssure
diff
eren
ce
∆t10−4 10−3 10−2 10−1
10−3
10−2
(f) p = 204
tim
e-av
erag
edpre
ssure
diff
eren
ce∆t
10−4 10−3 10−2 10−110−3
10−2
tim
e-av
erag
edpre
ssure
diff
eren
ce∆t
10−4 10−3 10−2 10−110−4
10−2
(g) p = 368
tim
e-av
erag
edpre
ssure
diff
eren
ce
∆t10−4 10−3 10−2 10−1
10−3
10−2
tim
e-av
erag
edpre
ssure
diff
eren
ce
∆t10−4 10−3 10−2 10−1
10−4
10−2
tim
e-av
erag
edpre
ssure
diff
eren
ce
∆t10−4 10−3 10−2 10−1
10−4
10−2
(h) p = 564
Figure : Galerkin/discrete-optimal difference in the stable Galerkininterval 0 ≤ t ≤ 1.1.
X The discrete-optimal ROM converges to Galerkin as ∆t → 0.
Discrete optimality Carlberg, Barone, Antil 29 / 37
POD basis spectral analysis
(a) mode 1 (b) mode 21 (c) mode 101
Mode number0 100 200 300 400 500 600
τ 95
10-2
10-1
100
Higher modes numbers associate with smaller spatial andtemporal scales.
Discrete optimality Carlberg, Barone, Antil 30 / 37
Backward Euler error bound
∥∥δxnD∥∥ ≤ ∆t(1 + κ∆t)n−1∑j=0
µn−j
(h)j+1‖f(xn−j)‖
Approximate µk :=∥∥∥Φ∆xkD −∆xk
∥∥∥ /∆xk with relative projection error.
maxim
um
relativeprojectionerror
∆t
p = 564p = 368p = 204
0 0.04 0.08 0.12 0.16 0.20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
errorbound
∆t
p = 564
p = 368
p = 204
0 0.04 0.08 0.12 0.16 0.210
3
104
105
106
1 Adding basis vectors for larger time steps yields little improvement
2 Approximated error bound: intermediate time step → lowest boundDiscrete optimality Carlberg, Barone, Antil 31 / 37
Discrete-optimal performance in 0 ≤ t ≤ 2.5
∆t
p = 564p = 368p = 204
errorin
time-averaged
pressure
10−5 10−4 10−3 10−2 10−1 10010−4
10−3
10−2
10−1
100
p = 564p = 367p = 204
simulation
time(seconds)
∆t10−4 10−3 10−2 10−1
103
104
105
106
107
1 Adding basis vectors for larger time steps yields little improvement
2 Intermediate time step → lowest error
p = 564 case:
∆t = 1.875× 10−4 sec: relative error = 1.40%, time = 289 hrs
∆t = 1.5× 10−3 sec: relative error = 0.095%, time = 35.8 hrsDiscrete optimality Carlberg, Barone, Antil 32 / 37
GNAT model
wn = arg minz∈Rp‖ (PΦr )+ Prn (Φz) ‖2
2
Sample mesh [Carlberg et al., 2013]: 4.1% nodes, 3.0% original cells
+ Allows GNAT to run on 2 cores instead of 48 cores
Discrete optimality Carlberg, Barone, Antil 33 / 37
GNAT performance
errorin
time-averaged
pressure
∆t10−3 10−2 10−1
10−2
10−1
100
computationalsavings
∆t
10−3 10−2 10−1101
102
103
1.5× 10−3 sec: relative error = 3.32%, cpu savings = 14.9
6.0× 10−3 sec: relative error = 2.25%, cpu savings = 55.7
Discrete optimality Carlberg, Barone, Antil 34 / 37
Conclusions
Time-continuous and time-discrete representations
Galerkin: projection and time-discretization are commutativeDiscrete-optimal: a continuous representation sometimes exists
Equivalence conditions
1 Limit of ∆t → 02 Explicit schemes3 Positive definite residual Jacobians
Discrete-time error bounds
Discrete-optimal ROM yields smaller error bound than GalerkinAmbiguous role of time step
Numerical experiments
Discrete-optimal ROM always yields a smaller error thanGalerkinEquivalent as ∆t → 0Approximated error bound and actual error minimized forintermediate ∆t
Discrete optimality Carlberg, Barone, Antil 35 / 37
Acknowledgments
Charbel Farhat: permitting the open use of AERO-F
Julien Cortial, David Amsallam, Charbel Bou-Mosleh:contributing to implementation of model reduction in AERO-F
Stephen Pope: insightful conversations that inspired this work
This research was supported in part by an appointment to theSandia National Laboratories Truman Fellowship in NationalSecurity Science and Engineering, sponsored by SandiaCorporation (a wholly owned subsidiary of Lockheed MartinCorporation) as Operator of Sandia National Laboratoriesunder its U.S. Department of Energy Contract No.DE-AC04-94AL85000.
Discrete optimality Carlberg, Barone, Antil 36 / 37
Questions?
K. Carlberg, M. Barone, H. Antil. “Galerkin v. discrete-optimalprojection in nonlinear model reduction,” arXiv e-Print 1504.03749(2015).
∆t
p = 564p = 368p = 204
errorin
time-averaged
pressure
10−5 10−4 10−3 10−2 10−1 10010−4
10−3
10−2
10−1
100
p = 564p = 367p = 204
simulation
time(seconds)
∆t10−4 10−3 10−2 10−1
103
104
105
106
107
Figure : Discrete-optimal ROM performance.
Discrete optimality Carlberg, Barone, Antil 37 / 37
Bibliography I
Abgrall, R. and Amsallem, D. (2015).Robust model reduction by l1-norm minimization andapproximation via dictionaries: Application to linear andnonlinear hyperbolic problems.Stanford University Preprint.
Carlberg, K. (2011).Model Reduction of Nonlinear Mechanical Systems viaOptimal Projection and Tensor Approximation.PhD thesis, Stanford University.
Discrete optimality Carlberg, Barone, Antil 38 / 37
Bibliography II
Carlberg, K., Bou-Mosleh, C., and Farhat, C. (2011).Efficient non-linear model reduction via a least-squaresPetrov–Galerkin projection and compressive tensorapproximations.International Journal for Numerical Methods in Engineering,86(2):155–181.
Carlberg, K., Farhat, C., Cortial, J., and Amsallem, D. (2013).The GNAT method for nonlinear model reduction: effectiveimplementation and application to computational fluiddynamics and turbulent flows.Journal of Computational Physics, 242:623–647.
Discrete optimality Carlberg, Barone, Antil 39 / 37