Monte Carlo Integration
-
Upload
sarah-jacobs -
Category
Documents
-
view
60 -
download
2
description
Transcript of Monte Carlo Integration
![Page 1: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/1.jpg)
Monte Carlo Integration
Digital Image SynthesisYung-Yu Chuang11/30/2006
with slides by Pat Hanrahan and Torsten Moller
![Page 2: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/2.jpg)
Introduction
• The integral equations generally don’t have analytic solutions, so we must turn to numerical methods.
• Standard methods like Trapezoidal integration or Gaussian quadrature are not effective for high-dimensional and discontinuous integrals.
)ωp,( ooL )ω,p( oeL
iiiio ωθcos)ωp,()ω,ωp,(2
dLf is
![Page 3: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/3.jpg)
Numerical quadrature • Suppose we want to calculate , but can
’t solve it analytically. The approximations through quadrature rules have the form
which is essentially the weighted sum of samples of the function at various points
b
adxxfI )(
n
iii xfwI
1
)(ˆ
![Page 4: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/4.jpg)
Midpoint rule
convergence
![Page 5: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/5.jpg)
Trapezoid rule
convergence
![Page 6: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/6.jpg)
Simpson’s rule
• Similar to trapezoid but using a quadratic polynomial approximation
convergence
assuming f has a continuous fourth derivative.
![Page 7: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/7.jpg)
Curse of dimensionality and discontinuity• For a sd function f,• If the 1d rule has a convergence rate of O(n-r), t
he sd rule would require a much larger number (ns) of samples to work as well as the 1d one. Thus, the convergence rate is only O(n-r/s).
• If f is discontinuous, convergence is O(n-1/s) for sd.
![Page 8: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/8.jpg)
Randomized algorithms
• Las Vegas v.s. Monte Carlo• Las Vegas: gives the right answer by using rand
omness.• Monte Carlo: gives the right answer on the ave
rage. Results depend on random numbers used, but statistically likely to be close to the right answer.
![Page 9: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/9.jpg)
Monte Carlo integration
• Monte Carlo integration: uses sampling to estimate the values of integrals. It only requires to be able to evaluate the integrand at arbitrary points, making it easy to implement and applicable to many problems.
• If n samples are used, its converges at the rate of O(n-1/2). That is, to cut the error in half, it is necessary to evaluate four times as many samples.
• Images by Monte Carlo methods are often noisy.
![Page 10: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/10.jpg)
Monte Carlo methods
• Advantages– Easy to implement– Easy to think about (but be careful of statistical
bias)– Robust when used with complex integrands
and domains (shapes, lights, …)– Efficient for high dimensional integrals
• Disadvantages– Noisy– Slow (many samples needed for convergence)
![Page 11: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/11.jpg)
Basic concepts
• X is a random variable• Applying a function to a random variable gives
another random variable, Y=f(X).
• CDF (cumulative distribution function)
• PDF (probability density function): nonnegative, sum to 1
• canonical uniform random variable ξ (provided by standard library and easy to transform to other distributions)
dx
xdPxp
)()(
}Pr{)( xXxP
![Page 12: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/12.jpg)
Discrete Probability Distributions
• Discrete events Xi with probability pi
• Cumulative PDF (distribution)
• Construction of samples• To randomly select an event,
• Select Xi if
0ip 1
1n
ii
p
1i iP U P
ip
1
j
j ii
P p
U
1
0Uniform random variable 3X
iP
![Page 13: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/13.jpg)
Continuous Probability Distributions• PDF
• CDF1
0
( ) Pr( )P x X x
Pr( ) ( )
( ) ( )
X p x dx
P P
( )p x
10
Uniform
( ) 0p x
0
( ) ( )x
P x p x dx(1) 1P
( )P x
![Page 14: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/14.jpg)
Expected values
• Average value of a function f(x) over some distribution of values p(x) over its domain D
• Example: cos function over [0, π], p is uniform
Dp dxxpxfxfE )()()(
00
1cos)cos( dxxxE p
![Page 15: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/15.jpg)
Variance
• Expected deviation from the expected value• Fundamental concept of quantifying the error
in Monte Carlo methods
2)()()]([ xfExfExfV
![Page 16: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/16.jpg)
Properties
)()( xfaExafE
ii
ii XfEXfE )()(
)()( 2 xfVaxafV
22 )()()( xfExfExfV
![Page 17: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/17.jpg)
Monte Carlo estimator
• Assume that we want to evaluate the integral of f(x) over [a,b]
• Given a uniform random variable Xi over [a,b], Monte Carlo estimator
says that the expected value E[FN] of the estimator FN equals the integral
N
iiN Xf
N
abF
1
)(
![Page 18: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/18.jpg)
General Monte Carlo estimator
• Given a random variable X drawn from an arbitrary PDF p(x), then the estimator is
• Although the converge rate of MC estimator is O(N1/2), slower than other integral methods, its converge rate is independent of the dimension, making it the only practical method for high dimensional integral
N
i i
iN Xp
Xf
NF
1 )(
)(1
![Page 19: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/19.jpg)
Convergence of Monte Carlo
• Chebyshev’s inequality: let X be a random variable with expected value μ and variance σ2. For any real number k>0,
• For example, for k= , it shows that at least half of the value lie in the interval
• Let , the MC estimate FN becomes)(/)( iii XpXfY
N
iiN Y
NF
1
1
2
1}|Pr{|
kkX
2)2,2(
![Page 20: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/20.jpg)
Convergence of Monte Carlo
• According to Chebyshev’s inequality,
• Plugging into Chebyshev’s inequality,
So, for a fixed threshold, the error decreases at the rate N-1/2.
YVN
YVN
YVN
YN
VFVN
ii
N
ii
N
iiN
1111][
12
12
1
}
][|][Pr{|
21
NNN
FVFEF
}
][1|Pr{|
21
YV
NIFN
![Page 21: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/21.jpg)
Properties of estimators
• An estimator FN is called unbiased if for all N
That is, the expected value is independent of N.
• Otherwise, the bias of the estimator is defined as
• If the bias goes to zero as N increases, the estimator is called consistent
QFE N ][
QFEF NN ][][
0][lim N
NF
QFE NN
][lim
![Page 22: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/22.jpg)
Example of a biased consistent estimator• Suppose we are doing antialiasing on a 1d pixe
l, to determine the pixel value, we need to evaluate , where is the filter function with
• A common way to evaluate this is
• When N=1, we have
1
0)()( dxxfxwI )(xw
1
01)( dxxw
N
i i
N
i iiN
Xw
XfXwF
1
1
)(
)()(
IdxxfXfEXw
XfXwEFE
1
011
111 )()]([
)(
)()(][
![Page 23: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/23.jpg)
Example of a biased consistent estimator• When N=2, we have
• However, when N is very large, the bias approaches to zero
Idxdxxwxw
xfxwxfxwFE
1
0
1
0 2121
22112 )()(
)()()()(][
N
i i
N
i ii
N
XwN
XfXwNF
1
1
)(1
)()(1
Idxxfxwdxxw
dxxfxw
XwN
XfXwNFE
N
i iN
N
i iiN
NN
1
01
0
1
0
1
1)()(
)(
)()(
)(1
lim
)()(1
lim][lim
![Page 24: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/24.jpg)
Choosing samples
•
• Carefully choosing the PDF from which samples are drawn is an important technique to reduce variance. We want the f/p to have a low variance. Hence, it is necessary to be able to draw samples from the chosen PDF.
• How to sample an arbitrary distribution from a variable of uniform distribution?– Inversion– Transform– Rejection
N
i i
iN Xp
Xf
NF
1 )(
)(1
![Page 25: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/25.jpg)
Inversion method
• Cumulative probability distribution function
• Construction of samplesSolve for X=P-1(U)
• Must know:1. The integral of p(x)
2. The inverse function P-1(x)
U
X
1
0
( ) Pr( )P x X x
![Page 26: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/26.jpg)
Proof for the inversion method
• Let U be an uniform random variable and its CDF is Pu(x)=x. We will show that Y=P-1(U) has the CDF P(x).
because P is monotonic Thus, Y’s CDF is exactly P(x).
)())(()(Pr)(PrPr 1 xPxPPxPUxUPxY u
![Page 27: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/27.jpg)
Inversion method
• Compute CDF P(x)
• Compute P-1(x)
• Obtain ξ
• Compute Xi=P-1(ξ)
![Page 28: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/28.jpg)
Example: Power Function
• Assume( ) ( 1) np x n x
11 1
0 0
1
1 1
nn x
x dxn n
1( ) nP x x
1 1~ ( ) ( ) nX p x X P U U
1 2 1max( , , , , )n nY U U U U 1
1
1
Pr( ) Pr( )n
n
i
Y x U x x
Trick
It is used in sampling Blinn’s microfacet model.
(It only works for sampling power distribution)
Similarly, a trick to obtain a Gaussian distribution is to take average.
![Page 29: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/29.jpg)
• Compute CDF P(x)
• Compute P-1(x)
• Obtain ξ• Compute Xi=P-1(ξ)
Example: exponential distribution
useful for rendering participating media.
![Page 30: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/30.jpg)
Transformation of variables
• Given a random variable X from distribution px
(x) to a random variable Y=y(X), where Y=y(X) and y is one-to-one, i.e. monotonic. We want to derive the distribution of Y, py(y).
• • PDF:
)(}Pr{)}(Pr{)( xPxXxyYyP xy
dx
xdP
dx
ydPxy )()(
)(xpx
dx
dy
dy
ydP
dx
dyyp y
y
)()(
)()(1
xpdx
dyyp xy
![Page 31: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/31.jpg)
Example
XY
xxpx
sin
2)(
2
11
1
sin2
cos
2)()(cos)(
y
y
x
xxpxyp xy
![Page 32: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/32.jpg)
Transformation method
• A problem to apply the above method is that we usually have some PDF to sample from, not a given transformation.
• Given a source random variable X with px(x) and a target distribution py(y), try transform X into to another random variable Y so that Y has the distribution py(y).
• We first have to find a transformation y(x) so that Px(x)=Py(y). Thus,
))(()( 1 xPPxy xy
![Page 33: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/33.jpg)
Transformation method• Let’s prove that the above transform works. We first prove that the random variable Z= Px(x)
has a uniform distribution. If so, then should have distribution Px(x) from the inversio
n method.
Thus, Z is uniform and the transformation works.
• It is an obvious generalization of the inversion method, in which X is uniform and Px(x)=x.
)(1 ZPy
xxPPxPXxXPxZ xxxx ))(()(Pr)(PrPr 11
![Page 34: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/34.jpg)
Thus, if X has the distribution , then the random variable has the distribution
Example
xxpx )( yy eyp )(
2)(
2xxPx y
y eyP )(
2lnln2)2
ln())(()(2
1 xx
xPPxy xy
yyPy ln)(1
2lnln2 XYxxpx )(
yy eyp )(
![Page 35: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/35.jpg)
Multiple dimensions
We often need the other way around,
![Page 36: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/36.jpg)
Spherical coordinates
• The spherical coordinate representation of directions is
cos
sinsin
cossin
rz
ry
rx
sin|| 2rJT
),,(sin),,( 2 zyxprrp
![Page 37: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/37.jpg)
Spherical coordinates
![Page 38: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/38.jpg)
Rejection method
• AlgorithmPick U1 and U2
Accept U1 if U2 < f(U1)
• Wasteful?
1
0
( )
( )
y f x
I f x dx
dx dy
( )y f x
Efficiency = Area / Area of rectangle
• Sometimes, we can’t integrate into CDF or invert CDF
![Page 39: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/39.jpg)
Rejection method
• Rejection method is a dart-throwing method without performing the above steps
1. Find q(x) so that p(x)<Mq(x)
2. Dart throwinga. Choose a pair (X, ξ), where X is sampled from q(x)
b. If (ξ<p(X)/Mq(X)) return X• Essentially, we pick a point (X, ξMq(X)). If it lies beneath p(X) then we are fine.
![Page 40: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/40.jpg)
Why it works
• For each iteration, we generate Xi from q. The sample is returned if ξ<p(X)/Mq(X), which happens with probability p(X)/Mq(X).
• So, the probability to return x is
• Thus, when a sample is returned (probability 1/M), then Xi is distributed according to p(x).
M
xp
xMq
xpxq
)(
)(
)()(
![Page 41: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/41.jpg)
Example: sampling a unit sphere
void RejectionSampleDisk(float *x, float *y) {
float sx, sy;
do {
sx = 1.f -2.f * RandomFloat();
sy = 1.f -2.f * RandomFloat();
} while (sx*sx + sy*sy > 1.f)
*x = sx; *y = sy;
}
π/4 ~ 78.5% good samples, gets worse in higher dimensions, for example, for sphere, π/8 ~ 39.3%
![Page 42: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/42.jpg)
Multidimensional sampling
![Page 43: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/43.jpg)
Sampling a hemisphere
2
1)( p
2
sin),( p
![Page 44: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/44.jpg)
Sampling a hemisphere
![Page 45: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/45.jpg)
Sampling a hemisphere
![Page 46: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/46.jpg)
Sampling a disk
1
2
2 U
r U
1
2
2 U
r U
RIGHT Equi-ArealWRONG Equi-Areal
![Page 47: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/47.jpg)
Sampling a disk
RIGHT Equi-ArealWRONG Equi-Areal
1
2
2 U
r U
1
2
2 U
r U
![Page 48: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/48.jpg)
Sampling a disk
![Page 49: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/49.jpg)
Shirley’s mapping
1
2
14
r U
U
U
![Page 50: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/50.jpg)
Sampling a Triangle
121 1 1
0 0 00
(1 ) 1(1 )
2 2
u uA dv du u du
( , ) 2p u v
0
0
1
u
v
u v
u
v
1u v
( ,1 )u u
![Page 51: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/51.jpg)
)(
),()|(
vp
vupvup
Sampling a Triangle
• Here u and v are not independent!• Conditional probability
( , ) 2p u v
00 0 00 0
0 0
1( | ) ( | )
(1 ) (1 )o ov v v
P v u p v u dv dvu u
1( | )
(1 )p v u
u
0 20 00
( ) 2(1 ) (1 )u
P u u du u
1
0
( ) 2 2(1 )u
p u dv u
0 11u U
0 1 2v U U
dvvupup ),()(
![Page 52: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/52.jpg)
Cosine weighted hemisphere
cos)( p
dp )(1
2
0 0
2 sincos1 ddc
ddd sin
2
0sincos21
dc
1c
sincos1
),( p
![Page 53: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/53.jpg)
Cosine weighted hemisphere
sincos1
),( p
2sinsincos2sincos1
)(2
0 dp
2
1
)(
),()|(
p
pp
12
12cos
2
1)( P
22)|(
P
)21(cos2
11
1
22
![Page 54: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/54.jpg)
Cosine weighted hemisphere
• Malley’s method: uniformly generates points on the unit disk and then generates directions by projecting them up to the hemisphere above it.
Vector CosineSampleHemisphere(float u1,float u2){
Vector ret;
ConcentricSampleDisk(u1, u2, &ret.x, &ret.y);
ret.z = sqrtf(max(0.f,1.f - ret.x*ret.x -
ret.y*ret.y));
return ret;
}
![Page 55: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/55.jpg)
Cosine weighted hemisphere
• Why Malley’s method works?• Unit disk sampling• Map to hemisphere here
•
rrp ),(),(sin),( r
),(),,( XrY
![Page 56: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/56.jpg)
Sampling Phong lobe np cos)(
ncp cos)(
2
0
2/
0
1sincos ddc n
0
1cos
1coscos2
dc n 11
2
n
c
21
n
c
sincos2
1),( nn
p
![Page 57: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/57.jpg)
Sampling Phong lobe
'cos1
1
cos)1(coscos)1(
sincos)1()'(
sincos)1(sincos2
1)(
1
'cos
1cos
1'
0
'
0
2
0
n
nn
n
nn
nndn
dnP
ndn
p
11
1cos n
sincos2
1),( nn
p
![Page 58: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/58.jpg)
Sampling Phong lobe
2
'
2
1)|'(
2
1
sincos)1(
sincos
)(
),()|(
'
0
21
dP
np
pp
n
nn
22
sincos2
1),( nn
p
![Page 59: Monte Carlo Integration](https://reader035.fdocuments.in/reader035/viewer/2022081421/56812c33550346895d90b810/html5/thumbnails/59.jpg)
Sampling Phong lobe
)2,(cos),( 1,n 211
211 2),21(cos
2
1),(
cosine-weighted hemisphere
2
12cos
2
1)( P 21 cos1cos1)( nP
222 cos1sin2
1)sin21(
2
1
2
12cos
2
1
When n=1, it is actually equivalent to