Stratified Monte Carlo Integration and Applications

Stratified samplingIntegration of indicator functions

Numerical results

Stratified Monte Carlo Integrationand Applications

R. El Haddad, R. Fakhereddine, C. Lecot, G. Venkiteswaran

Universite Saint-Joseph, Beirut, Lebanon&

Universite de Savoie, Le Bourget-du-Lac, France&

Birla Institute of Technology and Science, Pilani, India

MCQMC 2012February 13 – 17, 2012, Sydney, Australia

R. El Haddad, R. Fakhereddine, C. Lecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications


Numerical results

Plan of the talk

1 Stratified sampling

2 Integration of indicator functions

3 Numerical results



Numerical results



3 Numerical results



Numerical results

Numerical integration

I := [0, 1),

for s ≥ 1, λs is the s-dimensional Lebesgue measure,

g : I s → R is square-integrable.

We want to approximate

J :=

∫I sg(x)dλs(x).



Numerical results

Monte Carlo approximation

{U1, . . . ,UN} i.i.d. random variables uniformly distributedover I s ,

X :=1

N

N∑`=1

g ◦ U`.

E [X ] = J and Var(X ) =σ2(g)

N,

where

σ2(g) :=

∫I s

(g(x)

)2dλs(x)−

(∫I sg(x)dλs(x)

)2.



Numerical results

Stratified sampling

{D1, . . . ,Dp} a partition of I s ,

N1, . . . ,Np integers,

{V k1 , . . . ,V

kNk} i.i.d. random variables uniformly distributed

over Dk .

Tk :=1

Nk

Nk∑`=1

g ◦ V k` T :=

p∑k=1

λs(Dk)Tk .

E [T ] = J and (proportional allocation 1) Var(T ) ≤ Var(X ).

1G.S. Fishman, Monte Carlo, Springer, New York (1996)R. El Haddad, R. Fakhereddine, C. Lecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications


Numerical results

Simple stratified sampling

{D1, . . . ,DN} a partition of I s ,

λs(D1) = · · · = λs(DN) =1

N,

{V1, . . . ,VN} independent random variables,V` uniformly distributed over D`.

Y :=1

N

N∑`=1

g ◦ V`.

E [Y ] = J and 2 3 Var(Y ) ≤ Var(X )

2S. Haber, A modified Monte Carlo quadrature, Math. Comput. 20, 361–368(1966)

3R.C.H. Cheng, T. Davenport, The problem of dimensionality in stratifiedsampling, Manage. Sci. 35, 1278–1296 (1989)



Numerical results


0 1

1



Numerical results

Latin hypercube sampling

I` := [(`− 1)/N, `/N), 1 ≤ ` ≤ N

{V i1, . . . ,V

iN} independent random variables,

V i` uniformly distributed over I`

{π1, . . . , πs} independent random permutations of {1, . . . ,N}.W` := (V 1

π1(`), . . . ,Vsπs(`))

Z :=1

N

N∑`=1

g ◦W`.

E [Z ] = J and 4 Var(Z ) ≤ Var(X )

4M.D. McKay, R.J. Beckman, W.J. Conover, A comparison of threemethods for selecting values of input variables in the analysis of output from acomputer code, Technometrics, 21, 239–245 (1979)



Numerical results

Latin hypercube sampling



Numerical results



3 Numerical results



Numerical results

Monte Carlo approximation

A ⊂ I s , g := 1A, J = λs(A),

{U1, . . . ,UN} i.i.d. random variables uniformly distributedover I s ,

X :=1

N

N∑`=1

1A ◦ U`.

Var(X ) =1

Nλs(A)

(1− λs(A)

)≤ 1

4N.



Numerical results

ε-collars

‖x‖∞ := max1≤i≤s

|xi |.

For ε > 0,

A−ε := {x ∈ I s : ∀y ∈ I s \ A ‖x − y‖∞ ≥ ε},Aε := {x ∈ I s : ∃y ∈ A ‖x − y‖∞ < ε}.

A−ε ⊂ A ⊂ Aε.

Suppose there exists a nondecreasing γ : [0,+∞)→ [0,+∞)such that

∀ε > 0 max(λs(Aε \ A), λs(A \ A−ε)

)≤ γ(ε),



Numerical results

ε-collars

0 1

1

A



Numerical results

ε-collars

0 1

1

A

A-ε



Numerical results

ε-collars

0 1

1

A

A-ε

Aε



Numerical results


N = ns ; for k = (k1, . . . , ks) with 1 ≤ ki ≤ n,

Ck :=s∏

i=1

[ki − 1

n,kin

),

{Vk : 1 ≤ ki ≤ n} independent random variables,

Vk uniformly distributed over Ck .

Y :=1

N

∑k

1A ◦ Vk .



Numerical results




Numerical results

Simple stratified sampling : Jordan measurable set

Proposition 1. If ∀ε > 0 max(λs(Aε \ A), λs(A \ A−ε)

)≤ γ(ε),

then

Var(Y ) ≤ 1

2Nγ

(1

N1/s

).

Proof. We have

Var(Y ) ≤ 1

4N2#{k : Ck ∩ A 6= ∅ and Ck 6⊂ A}.

Since ⋃Ck∩A6=∅ and Ck 6⊂A

Ck ⊂ A1/n \ A−1/n,

we have1

N#{k : Ck ∩ A 6= ∅ and Ck 6⊂ A} ≤ 2γ

(1

n

).

for linear γ : Var(Y ) ≤ O(

1N1+1/s

).



Numerical results

Under the hypersurface

f : Is−1 → I and Af := {(x ′, xs) ∈ I s : xs < f (x ′)}.

0 1

1

Af

We want to approximate

I :=

∫I s−1

f (x ′)dλs−1(x ′) =

∫I s

1Af(x)dλs(x).



Numerical results

Simple stratified sampling : under the hypersurface

Proposition 2. If f is of bounded variation V (f ) on Is−1

, then

Var(Y ) ≤(s − 1

4V (f ) +

1

2

)1

N1+1/s.

Proof. For k = (k1, . . . , ks) with 1 ≤ ki ≤ n,

k ′ = (k1, . . . , ks−1) and C ′k ′ :=s−1∏i=1

[ki − 1

n,kin

).

Var(Y ) ≤ 1

4N2

∑k ′

#{ks : C(k ′,ks) ∩ Af 6= ∅ and C(k ′,ks) 6⊂ Af }

if C(k ′,ks) ∩Af 6= ∅ then ∃x ′k ′ ∈ C ′k ′ such that ks < nf (x ′k ′) + 1,if C(k ′,ks) 6⊂ Af then ∃y ′k ′ ∈ C ′k ′ such that nf (y ′k ′) < ks .

Var(Y ) ≤ 1

4N2

∑k ′

(n(f (x ′k ′)− f (y ′k ′)

)+ 2).



Numerical results

Variation

The result follows from Lemma 1.

Lemma 1. 5 Let f be a function of bounded variation V (f ) on Is.

Let n1, . . . , ns be integers. For k = (k1, . . . , ks) with 1 ≤ ki ≤ ni ,

denote Ck :=∏s

i=1

[ki−1ni, kini

)and xk , yk ∈ Ck . Then

∑k

|f (xk)− f (yk)| ≤ V (f )s∏

i=1

ni

s∑i=1

1

ni.

5C. Lecot, Error bounds for quasi-Monte Carlo integration with nets. Math.Comput. 65, 179–187 (1996)



Numerical results



3 Numerical results



Numerical results

Special latin hypercube sampling

N = ns

W` := (W 1` , . . . ,W

s` )

W i` :=

ì − 1

n+πi (i )− 1

N+

U i

N` := (`1, . . . , `s)i := (`1, . . . , ì−1, ì+1, . . . , `s)π1, . . . , πs independent random bijections

{1, . . . , n}s−1 → {1, . . . ,N/n}

U = (U1, . . . ,Us) random variable uniformly distributed overI s



Numerical results

Special latin hypercube sampling



Numerical results

Measure of the unit ball

Q := {x ∈ I s : ‖x‖2 < 1} then λs(Q) =πs/2

2sΓ( s2 + 1).

We compare MC, stratified MC and LHS.

s = 2 : N = 102, 202, 302, . . . , 4002 = 160 000 points,

s = 3 : N = 103, 203, 303, . . . , 2003 = 8 000 000 points,

s = 4 : N = 104, 124, 144, . . . , 404 = 2 560 000 points.

In order to estimate the variance, we replicate the quadratureindependently M times and compute the sample variance,We use M = 100, 200, . . . , 1 000.



Numerical results

Measure of the unit ball : variance order

Assuming Var = O(N−β), linear regression (M = 1 000) gives

Table: Order β of the variance

dimension MC stratified MC LHS

2 0.99 1.49 1.503 1.00 1.33 1.324 1.00 1.25 1.25



Numerical results

Simulation of diffusion

We solve the pure initial-value problem (here∫ +∞−∞ c0(x)dx = 1).

∂c

∂t(x , t) = a

∂2c

∂x2(x , t), x ∈ R, t > 0,

c(x , 0) = c0(x), x ∈ R

Random walk

Sample X0,1, . . . ,X0,N from c0, e.g., X`,N := C−10 ( 2`−1

2N ),

Choose ∆t, let tn := n∆t,

If Xn,1, . . . ,Xn,N are known,

Xn+1,` = Xn,` + f (Un,`)

where

f (u) :=√

4a∆t erf−1(2u − 1)

Un,` i.i.d. random variables uniformly distributed over I



Numerical results

Random walk using special LHS

Sample X0,1, . . . , X0,N from c0, e.g., X`,N := C−10 ( 2`−1

2N ),

Choose ∆t, let tn := n∆t,

If Xn,1, . . . , Xn,N are known,

1 Sort : Xn,1 ≤ . . . ≤ Xn,N ,2 Xn+1,bNW 1

n,`c = Xn,bNW 1n,`c + f (W 2

n,`)

where

f (u) :=√

4a∆t erf−1(2u − 1)

(W 1n,1,W

2n,1), . . . , (W 1

n,N ,W2n,N) independent special LHS in I 2



Numerical results

Test problem

∂c

∂t(x , t) = a

∂2c

∂x2(x , t), x ∈ R, t > 0,

c(x , 0) = c0(x), x ∈ R

Take c0(x) = 1√2πe−x

2

Compute∫ a

0 c(x ,T )dx for a = 4, T = 1 (with ∆t = 1/100)

Compare MC, stratified MC and LHS,with N = 102, . . . , 2002 particles,

Compute the sample variance (M = 1 000 replications)

Assuming Var = O(N−γ), linear regression gives

MC stratified MC LHS

γ 1.00 1.44 1.43



Numerical results

Random walk for 1-D diffusion : computation time

Figure: Computation time for the sample variance of 100 independentcopies of the approximation of

∫ a

0c(x , t)dx as a function of N (from 102

to 2002). MC (+) stratified MC (�) and LHS (?).



Numerical results

Thank You


Stratified Monte Carlo Integration and Applications

Documents

Transcript of Stratified Monte Carlo Integration and Applications