A micro/macro algorithm to accelerate MonteCarlo simulation of stochastic differential

equations

Kristian Debrabant

Scientific Computing Research Group,Katholieke Universiteit Leuven, Belgium

Innsbruck, October 29, 2010

1

Outline

1 Introduction

2 Accelerated Monte Carlo simulation

3 A convergence result

joint work with Giovanni Samaey (K.U. Leuven)

2

Introduction Accelerated Monte Carlo simulation A convergence result

Stochastic differential equations (SDEs)

dX (t) = g0(X (t)

)dt +

m∑l=1

gl(X (t)

)? dWl(t), X (t0) = X0

X (t) =X0 +

∫ t

t0g0(X (s)

)ds +

m∑

l=1

∫ t

t0gl(X (s)

)? dWl(s)

︸ ︷︷ ︸, t0 ≤ t ≤ T

0 t

W (t)lim

∆s→0

∑j

gl

(X(ξj )

)(Wl (sj+1)−Wl (sj )

)

ξj =sj : Itô-integral,∫ tt0

gl

(X(s))

dWl (s)

ξj =12 (sj +sj+1): Stratonovich-integral,∫ t

t0gl

(X(s))◦dWl (s)

W (t): standard Wiener-processW (0) = 0 a. s.W (t2)−W (t1) ∼ N(0, t2 − t1) for 0 ≤ t1 < t2 ≤ TW (t2)−W (t1) and W (t4)−W (t3) are independent for0 ≤ t1 < t2 ≤ t3 < t4 ≤ T

4

Introduction Accelerated Monte Carlo simulation A convergence result

Model problem: Immersed polymers

dX (t) =

[κ(t) X (t)− 1

2WeF(X (t)

)]dt +

1√We

dW (t)

X - polymer’s length vectorκ(t) - fluid’s velocity gradientF (X ) - entropic force, here:finitely extensible nonlinearlyelastic (FENE),

F (X ) =X

1− ‖X‖2/γ

Function of interest: non-Newtonian stress tensor

τp(t) =ε

We

(E(

X (t)⊗ F(X (t)

))− 1l

)

5

Introduction Accelerated Monte Carlo simulation A convergence result

Euler-Maruyama method

Aim: discrete approximation Y ∆t =(Y ∆t (t)

)t∈I∆t on

I∆t = {t0, t1, . . . , tN}, t0 < t1 < · · · < tN ≤ T ,

such that Yn = Y ∆t (tn) ≈ X (tn)

X (tn+1) = X (tn) +

∫ tn+1

tng0(X (s)

)ds +

m∑

l=1

∫ tn+1

tngl(X (s)

)dWl(s)

Yn+1 = Yn + g0(Yn)

∫ tn+1

tnds +

m∑

l=1

gl(Yn)

∫ tn+1

tndWl(s)

= Yn + g0(Yn) (tn+1 − tn)︸ ︷︷ ︸=∆nt

+m∑

l=1

gl(Yn)(Wl(tn+1)−Wl(tn)

)︸ ︷︷ ︸

=∆nWl

6

Introduction Accelerated Monte Carlo simulation A convergence result

Convergence

0 0.2 0.4 0.6 0.80

1

2

3

4

5

t

individual paths

expectation

Strong convergence:

maxt∈I∆t

E ‖Y ∆t (t)−X (t)‖ ≤ C∆tp

Weak convergence:∀f ∈ C2(p+1)

P (IRd , IR)

maxt∈I∆t|E(

f(Y ∆t (t)

)−f(X (t)

))| ≤ Cf ∆tp

Euler-Maruyama: Strong concergence order 0.5, weakconvergence order 1

7

Introduction Accelerated Monte Carlo simulation A convergence result

Discretization of the model problem

E. g. Euler-Maruyama scheme:

Yk+1 = Yk +

[κ(tk ) Yk −

12We

F(Yk)]

∆t +1√We

∆kW

Accept-reject strategy (e.g. Öttinger):

‖Yk+1‖ >√

(1−√

∆t)γ ⇒ reject Yk+1 and try again

⇒ ∆t has to be chosen very small

8

Introduction Accelerated Monte Carlo simulation A convergence result

Idea of accelerated Monte Carlo simulation

t∗ t∗ + Kδt t∗ + ∆t

1. simulate

4. project

5. simulate

t

Y(t,ω

)

t∗ t∗ + Kδt t∗ + ∆t

2.restrict

3. extrapolate

4.pr

ojec

t

t

Mac

rosc

opic

stat

es

10

Introduction Accelerated Monte Carlo simulation A convergence result

Simulation and restriction

SimulationDo K ≥ 1 microsteps with one-step method ϕ: For k = 1, . . . ,K

Y (j)(t? + kδt) = ϕ(t? + (k − 1)δt ,Y (j)(t? + (k − 1)δt); δt)

Restriction

Map ensemble Y = (Y (j))Jj=1 to a number L of (macroscopic)

state variables U = (Ul)Ll=1,

U(t) = R(Y(t)

), with Rl

(Y(t)

)=

1J

J∑

j=1

ul(Y (j)(t)

).

Example: ul(x) = x l yields standard empirical moments of thedistribution in a one-dimensional setting.

11

Introduction Accelerated Monte Carlo simulation A convergence result

Extrapolation and Projection

Extrapolation

U(t? + ∆t) =K∑

k=0

lkU(t? + kδt)

Simplest form: linear extrapolation,

U(t? + ∆t) =

U(t? + K δt) + (∆t − K δt)U(t? + K δt)− U

(t? + (K − 1)δt

)

δt

ProjectionE. g. by

Y(t? + ∆t) = argminZ: R(Z)=U(t?+∆t)

‖Z − Y(t? + K δt)‖2

13

Introduction Accelerated Monte Carlo simulation A convergence result

Projection

Corresponding Lagrange equations:

Y(t? + ∆t) = Y(t? + K δt) +L∑

l=1

λl∇YRl(Y(t? + ∆t)

),

with Λ = {λl}Ll=1 such that R(Y(t? + ∆t)

)= U(t? + ∆t)

⇒ Expensive. Cheap alternative:

Y(t? + ∆t) = Y(t? + K δt) +L∑

l=1

λl∇YRl(Y(t? + K δt)

),

with Λ = {λl}Ll=1 such that R(Y(t? + ∆t)

)= U(t? + ∆t)

14

Introduction Accelerated Monte Carlo simulation A convergence result

Projection

Lemma (Conditions for local solvability)For standard empirical moments Ul :

det(Ui+j−2

)i,j=1,...,L 6= 0

Neglecting statistical error:

det(

E X i+j−2)

i,j=1,...,L= 0

only possible if pdf has finite support

15

Introduction Accelerated Monte Carlo simulation A convergence result

Projection - numerical results for FENE dumbbells I

0

0.2

0.4

0.6

ϕ(x)

0 2 4 6

x

ϕ[2]

ϕ[5]

ϕ[8]

ϕ∗

ϕ−

10−16

10−12

10−8

10−4

100

(Ul−U

∗ l)/U

∗ l0 5 10 15 20

l

L = 2

L = 5

L = 8

L = 10

1d, κ = 2, γ = 49, δt = 2 · 10−4, J = 105, t− = 1, t∗ = 1.15

16

Introduction Accelerated Monte Carlo simulation A convergence result

Projection - numerical results for FENE dumbbells II

10−5

10−4

10−3

10−2

rel.errorin

τ p

0.001 0.01

∆t

L = 3

L = 4

L = 5

O(∆t)

1d, κ = 2, γ = 49, δt = 2 · 10−4, J = 105, t− = 1.5, 100 realizations

17

Introduction Accelerated Monte Carlo simulation A convergence result

Numerical example: FENE dumbbells

0

10

20

Eτ p(t)−Eτ p(t)

0 1 2 3 4 5 6

t

0

200

400

Eτ p(t),Eτ p(t)

0

1

2

3

4

Stdev(τ

p(t))

0 1 2 3 4 5 6

t

L = 2

L = 3

L = 4

reference

1d, κ(t) = 2 ·(1.1 + sin(πt)

), γ = 49, δt = 2 · 10−4, ∆t = 1 · 10−3, J = 5000, 500

realizations

18

Introduction Accelerated Monte Carlo simulation A convergence result

Idealized restriction and projection operators

limit J →∞restriction:

U(t) = R(Y (t)

), with Rl

(Y (t)

)= E ul

(Y (j)(t)

).

projection:Y ∗ = P(Y ,U∗) with R(Y ∗) = U∗.

Self-consistency: Y = P(Y , R(Y )

)

Sequences of projection and restriction operators

U[L] = (Ul)Ll=1

Corresponding projection and restriction operators: P [L] andR[L].

20

Introduction Accelerated Monte Carlo simulation A convergence result

Properties of projection operators

Uniform continuity in U∗:

|E g(P [L](Z ,U∗[L])

)− E g

(P [L](Z ,U

+[L]))| ≤ Cg‖U∗[L] − U+

[L]‖

Consistency:

|E g(P [L](Z ∗,U[L])

)− E g

(P [L](Z +,U[L])

)|

≤ Cg,L|E g(Z ∗)− E g(Z +)|with Cg,L → 0 for L→∞

LemmaFor normally distributed random variables and sequences(

U∗[L]

)L=1,2,...

of (centralized) moment values consistent with

normal distributions, the mentioned projection operators arecontinuous and consistent.

21

Introduction Accelerated Monte Carlo simulation A convergence result

Convergence

TheoremSuppose the following conditions hold:

(i) The SDE-coefficients are sufficiently smooth.

(ii) The one step method ϕ is weakly consistent of order pϕ.(iii) The sequence of (self-consistent) projection operators is

continuous and consistent for the numerical approximationprocess.

(iv) The extrapolation is consistent of order pe ≥ 1.Then for all t ∈ I∆t , all L ≥ L0, and all ∆t ∈ [0,∆t0]

|E f(Y[L](t)

)− E f

(X (t)

)| ≤ CL + CL(∆t)min{pe,pϕ}

with CL → 0 for L→∞.

22

Introduction Accelerated Monte Carlo simulation A convergence result

Conclusion

acceleration technique for Monte-Carlo simulationconvergence in the absence of statistical errorfor more details and references, see arXiv:1009.3767

Open problems:prove consistency and convergence of projection step forgeneral random variables,study stability and propagation of statistical error,study possibilities for variance reduction,construct an efficient adaptive error control, controlling thenumber of moments to extrapolate, the microscopic andmacroscopic time step, and the number of SDE realizations,couple FENE-SDE to Navier-Stokes equations

Thank you very much for your attention!23