Markov chain Monte Carlo and
its Application to some Engineering Problems
Konstantin Zuev
Department of Computing & Mathematical Sciences
California Institute of Technology
http://www.its.caltech.edu/∼zuev
May 24, 2011
Mechanical & Civil Engineering Seminar, Caltech
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 1 / 32
MCMC Revolution
P. Diaconis (2009), “The Markov chain Monte Carlo revolution”:
...asking about applications of Markov chain Monte Carlo (MCMC)
is a little like asking about applications of the quadratic formula...
you can take any area of science, from hard to social, and find a
burgeoning MCMC literature specifically tailored to that area.
decouplingblog.comslowslowmuse.wordpress.com www-thphys.physics.ox.ac.uk computerweekly.com
The main goal of this talk: To demonstrate how MCMC algorithms can be used
for solving engineering problems
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 2 / 32
Outline
1 What problems is MCMC meant to solve?
2 Why is MCMC useful in Engineering?
3 How does MCMC work?
4 MCMC applications to Reliability Problem
5 Summary
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 3 / 32
Outline
1 What problems is MCMC meant to solve?
2 Why is MCMC useful in Engineering?
3 How does MCMC work?
4 MCMC applications to Reliability Problem
5 Summary
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 3 / 32
Purpose of MCMC
Problem: To estimate
I =
∫Θ
h(θ)π(θ)dθ
= Eπ[h]
Θ ⊆ Rd parameter space
h : Θ→ R function of interest
π(θ) “target” PDF on Θ
“Easy” Cases:
d is small (d = 1, 2, 3) ⇒ numerical integration
π(θ) is easy to sample from ⇒ Monte Carlo method
Typical Case in Applications:
d is large (d ∼ 102 − 103)
π(θ) is known only up to a constant, π(θ) = cf(θ)
Solution: Use an appropriate MCMC method
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 4 / 32
Purpose of MCMC
Problem: To estimate
I =
∫Θ
h(θ)π(θ)dθ = Eπ[h]
Θ ⊆ Rd parameter space
h : Θ→ R function of interest
π(θ) “target” PDF on Θ
“Easy” Cases:
d is small (d = 1, 2, 3) ⇒ numerical integration
π(θ) is easy to sample from ⇒ Monte Carlo method
Typical Case in Applications:
d is large (d ∼ 102 − 103)
π(θ) is known only up to a constant, π(θ) = cf(θ)
Solution: Use an appropriate MCMC method
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 4 / 32
Purpose of MCMC
Problem: To estimate
I =
∫Θ
h(θ)π(θ)dθ = Eπ[h]
Θ ⊆ Rd parameter space
h : Θ→ R function of interest
π(θ) “target” PDF on Θ
“Easy” Cases:
d is small (d = 1, 2, 3) ⇒ numerical integration
π(θ) is easy to sample from ⇒ Monte Carlo method
Typical Case in Applications:
d is large (d ∼ 102 − 103)
π(θ) is known only up to a constant, π(θ) = cf(θ)
Solution: Use an appropriate MCMC method
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 4 / 32
Purpose of MCMC
Problem: To estimate
I =
∫Θ
h(θ)π(θ)dθ = Eπ[h]
Θ ⊆ Rd parameter space
h : Θ→ R function of interest
π(θ) “target” PDF on Θ
“Easy” Cases:
d is small (d = 1, 2, 3) ⇒ numerical integration
π(θ) is easy to sample from ⇒ Monte Carlo method
Typical Case in Applications:
d is large (d ∼ 102 − 103)
π(θ) is known only up to a constant, π(θ) = cf(θ)
Solution: Use an appropriate MCMC method
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 4 / 32
Purpose of MCMC
Problem: To estimate
I =
∫Θ
h(θ)π(θ)dθ = Eπ[h]
Θ ⊆ Rd parameter space
h : Θ→ R function of interest
π(θ) “target” PDF on Θ
“Easy” Cases:
d is small (d = 1, 2, 3) ⇒ numerical integration
π(θ) is easy to sample from ⇒ Monte Carlo method
Typical Case in Applications:
d is large (d ∼ 102 − 103)
π(θ) is known only up to a constant, π(θ) = cf(θ)
Solution: Use an appropriate MCMC method
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 4 / 32
Outline
1 What problems is MCMC meant to solve?
2 Why is MCMC useful in Engineering?
I Bayesian Inference
I Optimal Stochastic Design
I Reliability Problem
3 How does MCMC work?
4 MCMC applications to Reliability Problem
5 Summary
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 5 / 32
Bayesian Inference
M the assumed model class for the target dynamic system:
I set of I/O probability models p(y|θ, u)u−→ System
y−→I θ ∈ Θ the uncertain model parameters
I prior PDF π0(θ) over Θ
Bayesian approach:
Update π0(θ) to posterior PDF π(θ|D) via Bayes’ theorem:
π(θ|D) = L(D|θ)π0(θ)/Z
D the measured data from the system
L(D|θ) the likelihood function
Problems:
Evidence
Z =
∫Θ
L(D|θ)π0(θ)dθ
Posterior expectations
Eπ[h] =
∫Θ
h(θ)π(θ|D)dθ
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 6 / 32
Bayesian Inference
M the assumed model class for the target dynamic system:
I set of I/O probability models p(y|θ, u)u−→ System
y−→I θ ∈ Θ the uncertain model parameters
I prior PDF π0(θ) over Θ
Bayesian approach:
Update π0(θ) to posterior PDF π(θ|D) via Bayes’ theorem:
π(θ|D) = L(D|θ)π0(θ)/Z
D the measured data from the system
L(D|θ) the likelihood function
Problems:
Evidence
Z =
∫Θ
L(D|θ)π0(θ)dθ
Posterior expectations
Eπ[h] =
∫Θ
h(θ)π(θ|D)dθ
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 6 / 32
Bayesian Inference
M the assumed model class for the target dynamic system:
I set of I/O probability models p(y|θ, u)u−→ System
y−→I θ ∈ Θ the uncertain model parameters
I prior PDF π0(θ) over Θ
Bayesian approach:
Update π0(θ) to posterior PDF π(θ|D) via Bayes’ theorem:
π(θ|D) = L(D|θ)π0(θ)/Z
D the measured data from the system
L(D|θ) the likelihood function
Problems:
Evidence
Z =
∫Θ
L(D|θ)π0(θ)dθ
Posterior expectations
Eπ[h] =
∫Θ
h(θ)π(θ|D)dθ
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 6 / 32
Optimal Stochastic Design Problem
M the assumed model class for the target dynamic system
θ ∈ Θ the uncertain model parameters
ϕ ∈ Φ design variables: controllable parameters that define the system design
The objective function for a robust-to-uncertainties design:
Eπ[h(ϕ, θ)] =
∫Θ
h(ϕ, θ)π(θ|ϕ)dθ
h(ϕ, θ) a performance function of the system, e.g. stress in a component
π(θ|ϕ) a PDF, which incorporates available knowledge about the system
Problem:
ϕ∗ = arg minϕ∈Φ
Eπ[h(ϕ, θ)]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 7 / 32
Optimal Stochastic Design Problem
M the assumed model class for the target dynamic system
θ ∈ Θ the uncertain model parameters
ϕ ∈ Φ design variables: controllable parameters that define the system design
The objective function for a robust-to-uncertainties design:
Eπ[h(ϕ, θ)] =
∫Θ
h(ϕ, θ)π(θ|ϕ)dθ
h(ϕ, θ) a performance function of the system, e.g. stress in a component
π(θ|ϕ) a PDF, which incorporates available knowledge about the system
Problem:
ϕ∗ = arg minϕ∈Φ
Eπ[h(ϕ, θ)]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 7 / 32
Optimal Stochastic Design Problem
M the assumed model class for the target dynamic system
θ ∈ Θ the uncertain model parameters
ϕ ∈ Φ design variables: controllable parameters that define the system design
The objective function for a robust-to-uncertainties design:
Eπ[h(ϕ, θ)] =
∫Θ
h(ϕ, θ)π(θ|ϕ)dθ
h(ϕ, θ) a performance function of the system, e.g. stress in a component
π(θ|ϕ) a PDF, which incorporates available knowledge about the system
Problem:
ϕ∗ = arg minϕ∈Φ
Eπ[h(ϕ, θ)]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 7 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd represents the uncertain input load
I θ is a stochastic vector and has joint PDF π
F ⊂ Rd a failure domain (unacceptable performance)
F = {θ : G(θ) ≥ b∗}
G(θ) a performance function
b∗ a critical threshold for performance
IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 8 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd represents the uncertain input load
I θ is a stochastic vector and has joint PDF π
F ⊂ Rd a failure domain (unacceptable performance)
F = {θ : G(θ) ≥ b∗}
G(θ) a performance function
b∗ a critical threshold for performance
IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 8 / 32
Outline
1 What problems is MCMC meant to solve?
2 Why is MCMC useful in Engineering?
3 How does MCMC work?
I Markov chains
I Markov chain Monte Carlo
I Metropolis-Hastings algorithm
4 MCMC applications to Reliability Problem
5 Summary
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 9 / 32
MCMC: The Main Idea
Monte Carlo method∫Θ
h(θ)π(θ)dθ ≈ 1
N
N∑i=1
h(θi), θii.i.d∼ π
How to obtain i.i.d. samples from π ?
MCMC samples from π and computes integrals using Markov chains:
∫Θ
h(θ)π(θ)dθ ≈ 1
N −N0
N∑i=N0+1
h(xi)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 10 / 32
MCMC: The Main Idea
Monte Carlo method∫Θ
h(θ)π(θ)dθ ≈ 1
N
N∑i=1
h(θi), θii.i.d∼ π
How to obtain i.i.d. samples from π ?
MCMC samples from π and computes integrals using Markov chains:
∫Θ
h(θ)π(θ)dθ ≈ 1
N −N0
N∑i=N0+1
h(xi)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 10 / 32
MCMC: The Main Idea
Monte Carlo method∫Θ
h(θ)π(θ)dθ ≈ 1
N
N∑i=1
h(θi), θii.i.d∼ π
How to obtain i.i.d. samples from π ?
MCMC samples from π and computes integrals using Markov chains:
∫Θ
h(θ)π(θ)dθ ≈ 1
N −N0
N∑i=N0+1
h(xi)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 10 / 32
Markov chains
K(x,Rd) ≡ 1
K(x, {x}) is not necessarily zero
DefinitionMarkov chain is a sequence of stochastic vectors x0, x1, x2, . . . such that
P (xn+1 ∈ A|x0, . . . , xn) = P (xn+1 ∈ A|xn) ≡ K(xn, A)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 11 / 32
Markov chains
K(x,Rd) ≡ 1
K(x, {x}) is not necessarily zero
DefinitionMarkov chain is a sequence of stochastic vectors x0, x1, x2, . . . such that
P (xn+1 ∈ A|x0, . . . , xn) = P (xn+1 ∈ A|xn) ≡ K(xn, A)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 11 / 32
Markov chains
K(x,Rd) ≡ 1
K(x, {x}) is not necessarily zero
DefinitionMarkov chain is a sequence of stochastic vectors x0, x1, x2, . . . such that
P (xn+1 ∈ A|x0, . . . , xn) = P (xn+1 ∈ A|xn) ≡ K(xn, A)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 11 / 32
Markov chains
K(x,Rd) ≡ 1
K(x, {x}) is not necessarily zero
DefinitionMarkov chain is a sequence of stochastic vectors x0, x1, x2, . . . such that
P (xn+1 ∈ A|x0, . . . , xn) = P (xn+1 ∈ A|xn) ≡ K(xn, A)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 11 / 32
Markov chains
Markov chain theory: To describe the distribution of xn when n→∞
DefinitionA probability distribution π is called stationary distribution for K if
π(A) =
∫K(x,A)π(x)dx, for all measurable sets A
The central result: Let K be a transition kernel with a stationary distribution
π where K satisfies certain “ergodic conditions”, then
π is the unique stationary distribution
xn ∼ π, when n→∞
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 12 / 32
Markov chains
Markov chain theory: To describe the distribution of xn when n→∞
DefinitionA probability distribution π is called stationary distribution for K if
π(A) =
∫K(x,A)π(x)dx, for all measurable sets A
The central result: Let K be a transition kernel with a stationary distribution
π where K satisfies certain “ergodic conditions”, then
π is the unique stationary distribution
xn ∼ π, when n→∞
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 12 / 32
Markov chains
Markov chain theory: To describe the distribution of xn when n→∞
DefinitionA probability distribution π is called stationary distribution for K if
π(A) =
∫K(x,A)π(x)dx, for all measurable sets A
The central result: Let K be a transition kernel with a stationary distribution
π where K satisfies certain “ergodic conditions”, then
π is the unique stationary distribution
xn ∼ π, when n→∞
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 12 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Markov chain Monte Carlo
MCMC methods: The stationary distribution π is given:
it is the target distribution we want to sample from
The main goal: To find an appropriate transition kernel K
K(x, dy) = k(x, y)dy + r(x)δx(dy)
Jumps from x to y 6= x occur
according to k(x, y)
r(x) = 1−∫k(x, y)dy
probability that the chain remains at x
δx(dy) = 1 if x ∈ dy and 0 otherwise
The central result: If k satisfies the detailed balance equation
π(x)k(x, y) = π(y)k(y, x)
then π is the stationary distribution for K
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 13 / 32
Metropolis-Hastings algorithm
kMH(x, y) = S(y|x)a(x, y)
S(y|x) proposal distribution
a(x, y) acceptance probability
a(x, y) = min
{1,π(y)S(x|y)
π(x)S(y|x)
}
N. Metropolis et al, 1953
I Statistical mechanics
I Boltzmann distribution
I S(x|y) = S(y|x)
I Absence of Markov chains
W.K. Hastings, 1970
I Generalization of the M-algorithm
I Markov chains come into play
I Non-symmetric proposals
I Different acceptance probabilities
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 14 / 32
Metropolis-Hastings algorithm
kMH(x, y) = S(y|x)a(x, y)
S(y|x) proposal distribution
a(x, y) acceptance probability
a(x, y) = min
{1,π(y)S(x|y)
π(x)S(y|x)
}
N. Metropolis et al, 1953
I Statistical mechanics
I Boltzmann distribution
I S(x|y) = S(y|x)
I Absence of Markov chains
W.K. Hastings, 1970
I Generalization of the M-algorithm
I Markov chains come into play
I Non-symmetric proposals
I Different acceptance probabilities
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 14 / 32
Metropolis-Hastings algorithm
kMH(x, y) = S(y|x)a(x, y)
S(y|x) proposal distribution
a(x, y) acceptance probability
a(x, y) = min
{1,π(y)S(x|y)
π(x)S(y|x)
}
N. Metropolis et al, 1953
I Statistical mechanics
I Boltzmann distribution
I S(x|y) = S(y|x)
I Absence of Markov chains
W.K. Hastings, 1970
I Generalization of the M-algorithm
I Markov chains come into play
I Non-symmetric proposals
I Different acceptance probabilities
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 14 / 32
Metropolis-Hastings algorithm
kMH(x, y) = S(y|x)a(x, y)
S(y|x) proposal distribution
a(x, y) acceptance probability
a(x, y) = min
{1,π(y)S(x|y)
π(x)S(y|x)
}
N. Metropolis et al, 1953
I Statistical mechanics
I Boltzmann distribution
I S(x|y) = S(y|x)
I Absence of Markov chains
W.K. Hastings, 1970
I Generalization of the M-algorithm
I Markov chains come into play
I Non-symmetric proposals
I Different acceptance probabilities
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 14 / 32
Outline
1 What problems is MCMC meant to solve?
2 Why is MCMC useful in Engineering?
3 How does MCMC work?
4 MCMC applications to Reliability Problem
I Subset Simulation
I Enhancements for Subset Simulation
5 Summary
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 15 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000 ⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6 ⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000
⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6 ⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000 ⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6 ⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000 ⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6 ⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000 ⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6
⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000 ⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6 ⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Reliability Problem
Reliability Problem: To estimate the probability of failure pF
pF = P (θ ∈ F ) =
∫Rd
π(θ)IF (θ)dθ
Notation:
θ ∈ Rd is a stochastic vector with the joint PDF π
F ⊂ Rd is a failure domain, F = {θ : G(θ) ≥ b∗}IF (θ) = 1 if θ ∈ F and IF (θ) = 0 if θ /∈ F
Typically in Applications:
d is very large, d ∼ 1000 ⇒ numerical quadrature methods are not suitable
We can calculate IF (θ) for any θ, but it is expensive (dyn. system analysis)
pF is very small, pF ∼ 10−3 − 10−6 ⇒ Monte Carlo method is not suitable
We need MCMC based simulation methods
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 16 / 32
Subset SimulationS.K. Au & J.L. Beck (PEM, 2001)
Rd = F0 ⊃ F1 ⊃ . . . ⊃ Fm = F
F = {θ : G(θ) ≥ b∗}
Fi = {θ : G(θ) ≥ b∗i }
b∗1 < b∗2 < . . . < b∗m = b∗
pF =
m−1∏k=0
P (Fk+1|Fk)
P (Fk+1|Fk) ≈ 1
N
N∑i=1
IFk+1(θ
(i)k )
θ(i)k ∼ π(θ|Fk) =
π(θ)IFk(θ)
P (Fk)
How to sample from π(·|Fk)?
Use an appropriate MCMC alg
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 17 / 32
Subset SimulationS.K. Au & J.L. Beck (PEM, 2001)
Rd = F0 ⊃ F1 ⊃ . . . ⊃ Fm = F
F = {θ : G(θ) ≥ b∗}
Fi = {θ : G(θ) ≥ b∗i }
b∗1 < b∗2 < . . . < b∗m = b∗
pF =
m−1∏k=0
P (Fk+1|Fk)
P (Fk+1|Fk) ≈ 1
N
N∑i=1
IFk+1(θ
(i)k )
θ(i)k ∼ π(θ|Fk) =
π(θ)IFk(θ)
P (Fk)
How to sample from π(·|Fk)?
Use an appropriate MCMC alg
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 17 / 32
Subset SimulationS.K. Au & J.L. Beck (PEM, 2001)
Rd = F0 ⊃ F1 ⊃ . . . ⊃ Fm = F
F = {θ : G(θ) ≥ b∗}
Fi = {θ : G(θ) ≥ b∗i }
b∗1 < b∗2 < . . . < b∗m = b∗
pF =
m−1∏k=0
P (Fk+1|Fk)
P (Fk+1|Fk) ≈ 1
N
N∑i=1
IFk+1(θ
(i)k )
θ(i)k ∼ π(θ|Fk) =
π(θ)IFk(θ)
P (Fk)
How to sample from π(·|Fk)?
Use an appropriate MCMC alg
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 17 / 32
Subset SimulationS.K. Au & J.L. Beck (PEM, 2001)
Rd = F0 ⊃ F1 ⊃ . . . ⊃ Fm = F
F = {θ : G(θ) ≥ b∗}
Fi = {θ : G(θ) ≥ b∗i }
b∗1 < b∗2 < . . . < b∗m = b∗
pF =
m−1∏k=0
P (Fk+1|Fk)
P (Fk+1|Fk) ≈ 1
N
N∑i=1
IFk+1(θ
(i)k )
θ(i)k ∼ π(θ|Fk) =
π(θ)IFk(θ)
P (Fk)
How to sample from π(·|Fk)?
Use an appropriate MCMC alg
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 17 / 32
Subset SimulationS.K. Au & J.L. Beck (PEM, 2001)
Rd = F0 ⊃ F1 ⊃ . . . ⊃ Fm = F
F = {θ : G(θ) ≥ b∗}
Fi = {θ : G(θ) ≥ b∗i }
b∗1 < b∗2 < . . . < b∗m = b∗
pF =
m−1∏k=0
P (Fk+1|Fk)
P (Fk+1|Fk) ≈ 1
N
N∑i=1
IFk+1(θ
(i)k )
θ(i)k ∼ π(θ|Fk) =
π(θ)IFk(θ)
P (Fk)
How to sample from π(·|Fk)?
Use an appropriate MCMC alg
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 17 / 32
Modified Metropolis-Hastings algorithm
S.K. Au & J.L. Beck (PEM, 2001): Standard Metropolis-Hastings algorithm
is not efficient in high dimensions
Generate candidate state y
For each j = 1 . . . N :
I Simulate yj ∼ Sj(·|xjn)
I Compute the acceptance probability
aj(xjn, yj) = min
{1,πj(y
j)
πj(xjn)
}
I Accept/Reject yj
yj =
yj , with prob. aj(xjn, yj)
xjn, with prob. 1− aj(xjn, yj)
Accept/Reject y
xn+1 =
y, if y ∈ Fixn, if y /∈ Fi
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 18 / 32
Modified Metropolis-Hastings algorithm
S.K. Au & J.L. Beck (PEM, 2001): Standard Metropolis-Hastings algorithm
is not efficient in high dimensionsGenerate candidate state y
For each j = 1 . . . N :
I Simulate yj ∼ Sj(·|xjn)
I Compute the acceptance probability
aj(xjn, yj) = min
{1,πj(y
j)
πj(xjn)
}
I Accept/Reject yj
yj =
yj , with prob. aj(xjn, yj)
xjn, with prob. 1− aj(xjn, yj)
Accept/Reject y
xn+1 =
y, if y ∈ Fixn, if y /∈ Fi
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 18 / 32
Efficiency of Subset Simulation
Linear Problem
I F ⊂ Rd is a hyperplane
I d = 1000
I pF = 10−k, k = 3, 4, 5, 6
What computational effort is required to achieve the COV δ = 30% ?
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 19 / 32
Efficiency of Subset Simulation
Linear Problem
I F ⊂ Rd is a hyperplane
I d = 1000
I pF = 10−k, k = 3, 4, 5, 6
What computational effort is required to achieve the COV δ = 30% ?
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 19 / 32
Efficiency of Subset Simulation
Linear Problem
I F ⊂ Rd is a hyperplane
I d = 1000
I pF = 10−k, k = 3, 4, 5, 6
What computational effort is required to achieve the COV δ = 30% ?
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 19 / 32
Outline
1 What problems is MCMC meant to solve?
2 Why is MCMC useful in Engineering?
3 How does MCMC work?
4 MCMC applications to Reliability Problem
I Subset SimulationI Enhancements for Subset Simulation
F Modified Metropolis-Hastings algorithm with Delayed RejectionF Bayesian post-processor for Subset Simulation
5 Summary
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 20 / 32
Modifications of the Metropolis-Hastings algorithm
Modified MH (MMH) algorithm
I S.K. Au & J.L. Beck, 2001
MH algorithm with delayed rejection (MHDR)
I L. Tierney & A. Mira, 1999
Modified Metropolis-Hastings algorithm with delayed rejection (MMHDR)
I K.M. Zuev & L.S. Katafygiotis, 2011
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 21 / 32
Metropolis-Hastings algorithm with Delayed Rejection
L. Tierney & A. Mira (Statistics in Medicine, 1999):
a1(xn, y1) = min
{1,π(y1)
π(xn)IF (y1)
}a2(xn, y1, y2) = min
{1,
π(y2)S1(y1|y2)(1− a1(y2, y1))
π(xn)S1(y1|xn)(1− a1(xn, y1))IF (y2)
}
Drawback: Inefficient in high dimensions
Reason: S1(·|xn) and S2(·|xn, y1) are d-dimensional PDFs
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 22 / 32
Metropolis-Hastings algorithm with Delayed Rejection
L. Tierney & A. Mira (Statistics in Medicine, 1999):
a1(xn, y1) = min
{1,π(y1)
π(xn)IF (y1)
}a2(xn, y1, y2) = min
{1,
π(y2)S1(y1|y2)(1− a1(y2, y1))
π(xn)S1(y1|xn)(1− a1(xn, y1))IF (y2)
}
Drawback: Inefficient in high dimensions
Reason: S1(·|xn) and S2(·|xn, y1) are d-dimensional PDFs
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 22 / 32
Metropolis-Hastings algorithm with Delayed Rejection
L. Tierney & A. Mira (Statistics in Medicine, 1999):
a1(xn, y1) = min
{1,π(y1)
π(xn)IF (y1)
}a2(xn, y1, y2) = min
{1,
π(y2)S1(y1|y2)(1− a1(y2, y1))
π(xn)S1(y1|xn)(1− a1(xn, y1))IF (y2)
}
Drawback: Inefficient in high dimensions
Reason: S1(·|xn) and S2(·|xn, y1) are d-dimensional PDFs
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 22 / 32
Metropolis-Hastings algorithm with Delayed Rejection
L. Tierney & A. Mira (Statistics in Medicine, 1999):
a1(xn, y1) = min
{1,π(y1)
π(xn)IF (y1)
}a2(xn, y1, y2) = min
{1,
π(y2)S1(y1|y2)(1− a1(y2, y1))
π(xn)S1(y1|xn)(1− a1(xn, y1))IF (y2)
}
Drawback: Inefficient in high dimensions
Reason: S1(·|xn) and S2(·|xn, y1) are d-dimensional PDFs
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 22 / 32
Metropolis-Hastings algorithm with Delayed Rejection
L. Tierney & A. Mira (Statistics in Medicine, 1999):
a1(xn, y1) = min
{1,π(y1)
π(xn)IF (y1)
}a2(xn, y1, y2) = min
{1,
π(y2)S1(y1|y2)(1− a1(y2, y1))
π(xn)S1(y1|xn)(1− a1(xn, y1))IF (y2)
}
Drawback: Inefficient in high dimensions
Reason: S1(·|xn) and S2(·|xn, y1) are d-dimensional PDFs
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 22 / 32
Modified MH algorithm with Delayed Rejection
Features of the Algorithm:
Samples generated by MMHDR are less correlated
then samples generated by MMH.
MMHDR needs more computational effort than MMH
for generating the same number of samples.
Whether MMHDR is useful for reliability problems depends on whether the
gained reduction in variance compensates for the additional cost.
With fixed computational effort:
I MMH: more Markov chains with more correlated states
I MMHDR: fewer Markov chains with less correlated states
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 23 / 32
Modified MH algorithm with Delayed Rejection
Features of the Algorithm:
Samples generated by MMHDR are less correlated
then samples generated by MMH.
MMHDR needs more computational effort than MMH
for generating the same number of samples.
Whether MMHDR is useful for reliability problems depends on whether the
gained reduction in variance compensates for the additional cost.
With fixed computational effort:
I MMH: more Markov chains with more correlated states
I MMHDR: fewer Markov chains with less correlated states
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 23 / 32
Modified MH algorithm with Delayed Rejection
Features of the Algorithm:
Samples generated by MMHDR are less correlated
then samples generated by MMH.
MMHDR needs more computational effort than MMH
for generating the same number of samples.
Whether MMHDR is useful for reliability problems depends on whether the
gained reduction in variance compensates for the additional cost.
With fixed computational effort:
I MMH: more Markov chains with more correlated states
I MMHDR: fewer Markov chains with less correlated states
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 23 / 32
Modified MH algorithm with Delayed Rejection
Features of the Algorithm:
Samples generated by MMHDR are less correlated
then samples generated by MMH.
MMHDR needs more computational effort than MMH
for generating the same number of samples.
Whether MMHDR is useful for reliability problems depends on whether the
gained reduction in variance compensates for the additional cost.
With fixed computational effort:
I MMH: more Markov chains with more correlated states
I MMHDR: fewer Markov chains with less correlated states
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 23 / 32
Modified MH algorithm with Delayed Rejection
Features of the Algorithm:
Samples generated by MMHDR are less correlated
then samples generated by MMH.
MMHDR needs more computational effort than MMH
for generating the same number of samples.
Whether MMHDR is useful for reliability problems depends on whether the
gained reduction in variance compensates for the additional cost.
With fixed computational effort:
I MMH: more Markov chains with more correlated states
I MMHDR: fewer Markov chains with less correlated states
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 23 / 32
Modified MH algorithm with Delayed Rejection
Features of the Algorithm:
Samples generated by MMHDR are less correlated
then samples generated by MMH.
MMHDR needs more computational effort than MMH
for generating the same number of samples.
Whether MMHDR is useful for reliability problems depends on whether the
gained reduction in variance compensates for the additional cost.
With fixed computational effort:
I MMH: more Markov chains with more correlated states
I MMHDR: fewer Markov chains with less correlated states
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 23 / 32
ExampleLinear problem
Geometry
I d = 1000
I pF = 10−5, β = 4.265
Proposal PDFs
I MMH: Sj(·|xj0) = N (xj0, 1)
I MMHDR: Sj1,2(·|xj0) = N (xj0, 1)
Subset Simulation
MMH(1): SS + MMH, n = 103
MMHDR(1.4): SS + MMHDR, n = 103
MMH(1.4): SS + MMH, n = 1450
Reduction in CV is 11%
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 24 / 32
ExampleLinear problem
Geometry
I d = 1000
I pF = 10−5, β = 4.265
Proposal PDFs
I MMH: Sj(·|xj0) = N (xj0, 1)
I MMHDR: Sj1,2(·|xj0) = N (xj0, 1)
Subset Simulation
MMH(1): SS + MMH, n = 103
MMHDR(1.4): SS + MMHDR, n = 103
MMH(1.4): SS + MMH, n = 1450
Reduction in CV is 11%
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 24 / 32
ExampleLinear problem
Geometry
I d = 1000
I pF = 10−5, β = 4.265
Proposal PDFs
I MMH: Sj(·|xj0) = N (xj0, 1)
I MMHDR: Sj1,2(·|xj0) = N (xj0, 1)
Subset Simulation
MMH(1): SS + MMH, n = 103
MMHDR(1.4): SS + MMHDR, n = 103
MMH(1.4): SS + MMH, n = 1450
Reduction in CV is 11%
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 24 / 32
ExampleLinear problem
Geometry
I d = 1000
I pF = 10−5, β = 4.265
Proposal PDFs
I MMH: Sj(·|xj0) = N (xj0, 1)
I MMHDR: Sj1,2(·|xj0) = N (xj0, 1)
Subset Simulation
MMH(1): SS + MMH, n = 103
MMHDR(1.4): SS + MMHDR, n = 103
MMH(1.4): SS + MMH, n = 1450
Reduction in CV is 11%
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 24 / 32
ExampleLinear problem
Geometry
I d = 1000
I pF = 10−5, β = 4.265
Proposal PDFs
I MMH: Sj(·|xj0) = N (xj0, 1)
I MMHDR: Sj1,2(·|xj0) = N (xj0, 1)
Subset Simulation
MMH(1): SS + MMH, n = 103
MMHDR(1.4): SS + MMHDR, n = 103
MMH(1.4): SS + MMH, n = 1450
Reduction in CV is 11%
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 24 / 32
“Bayesianization” of Subset Simulation
The key idea of SS:
pF =
m∏k=1
pk, pk = P (Fk|Fk−1)
Original (“frequentist”) SS:
pk ≈ pk =1
N
N∑i=1
IFk(θ
(i)k−1) =
nkN, pF ≈ pSSF =
m∏k=1
nkN
Bayesian SS:
1 Specify prior PDFs p(pk) for all pk = P (Fk|Fk−1), k = 1, . . . ,m.
2 Find the posterior PDFs p(pk|Dk−1) via Bayes’ theorem,
using new data Dk−1 = {θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1)}
3 Obtain the posterior PDF p(pF | ∪m−1k=0 Dk) of pF =
∏mk=1 pk
from p(p1|D0), . . . , p(pm|Dm−1).
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 25 / 32
“Bayesianization” of Subset Simulation
The key idea of SS:
pF =
m∏k=1
pk, pk = P (Fk|Fk−1)
Original (“frequentist”) SS:
pk ≈ pk =1
N
N∑i=1
IFk(θ
(i)k−1) =
nkN, pF ≈ pSSF =
m∏k=1
nkN
Bayesian SS:
1 Specify prior PDFs p(pk) for all pk = P (Fk|Fk−1), k = 1, . . . ,m.
2 Find the posterior PDFs p(pk|Dk−1) via Bayes’ theorem,
using new data Dk−1 = {θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1)}
3 Obtain the posterior PDF p(pF | ∪m−1k=0 Dk) of pF =
∏mk=1 pk
from p(p1|D0), . . . , p(pm|Dm−1).
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 25 / 32
“Bayesianization” of Subset Simulation
The key idea of SS:
pF =
m∏k=1
pk, pk = P (Fk|Fk−1)
Original (“frequentist”) SS:
pk ≈ pk =1
N
N∑i=1
IFk(θ
(i)k−1) =
nkN, pF ≈ pSSF =
m∏k=1
nkN
Bayesian SS:
1 Specify prior PDFs p(pk) for all pk = P (Fk|Fk−1), k = 1, . . . ,m.
2 Find the posterior PDFs p(pk|Dk−1) via Bayes’ theorem,
using new data Dk−1 = {θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1)}
3 Obtain the posterior PDF p(pF | ∪m−1k=0 Dk) of pF =
∏mk=1 pk
from p(p1|D0), . . . , p(pm|Dm−1).
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 25 / 32
“Bayesianization” of Subset Simulation
The key idea of SS:
pF =
m∏k=1
pk, pk = P (Fk|Fk−1)
Original (“frequentist”) SS:
pk ≈ pk =1
N
N∑i=1
IFk(θ
(i)k−1) =
nkN, pF ≈ pSSF =
m∏k=1
nkN
Bayesian SS:
1 Specify prior PDFs p(pk) for all pk = P (Fk|Fk−1), k = 1, . . . ,m.
2 Find the posterior PDFs p(pk|Dk−1) via Bayes’ theorem,
using new data Dk−1 = {θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1)}
3 Obtain the posterior PDF p(pF | ∪m−1k=0 Dk) of pF =
∏mk=1 pk
from p(p1|D0), . . . , p(pm|Dm−1).
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 25 / 32
“Bayesianization” of Subset Simulation
The key idea of SS:
pF =
m∏k=1
pk, pk = P (Fk|Fk−1)
Original (“frequentist”) SS:
pk ≈ pk =1
N
N∑i=1
IFk(θ
(i)k−1) =
nkN, pF ≈ pSSF =
m∏k=1
nkN
Bayesian SS:
1 Specify prior PDFs p(pk) for all pk = P (Fk|Fk−1), k = 1, . . . ,m.
2 Find the posterior PDFs p(pk|Dk−1) via Bayes’ theorem,
using new data Dk−1 = {θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1)}
3 Obtain the posterior PDF p(pF | ∪m−1k=0 Dk) of pF =
∏mk=1 pk
from p(p1|D0), . . . , p(pm|Dm−1).
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 25 / 32
“Bayesianization” of Subset Simulation
The key idea of SS:
pF =
m∏k=1
pk, pk = P (Fk|Fk−1)
Original (“frequentist”) SS:
pk ≈ pk =1
N
N∑i=1
IFk(θ
(i)k−1) =
nkN, pF ≈ pSSF =
m∏k=1
nkN
Bayesian SS:
1 Specify prior PDFs p(pk) for all pk = P (Fk|Fk−1), k = 1, . . . ,m.
2 Find the posterior PDFs p(pk|Dk−1) via Bayes’ theorem,
using new data Dk−1 = {θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1)}
3 Obtain the posterior PDF p(pF | ∪m−1k=0 Dk) of pF =
∏mk=1 pk
from p(p1|D0), . . . , p(pm|Dm−1).
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 25 / 32
Prior and Posterior for pk = P (Fk|Fk−1)
1 Prior PDF p(pk)
Principle of Maximum Entropy:
p(pk) = 1, 0 ≤ pk ≤ 1.
2 Posterior PDF p(pk|Dk−1)
I If θ(1)k−1, . . . , θ
(N)k−1 are i.i.d. according to π(·|Fk−1)
⇒ IFk (θ(1)k−1), . . . , IFk (θ
(N)k−1) can be interpreted as Bernoulli trials
⇒ Bayes’ Theorem (1763):
p(pk|Dk−1) =pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
I In fact, θ(1)k−1, . . . , θ
(N)k−1 are MCMC samples (for k ≥ 2)
⇒ θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1), however, they are not independent
p(pk|Dk−1) ≈pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 26 / 32
Prior and Posterior for pk = P (Fk|Fk−1)
1 Prior PDF p(pk)
Principle of Maximum Entropy:
p(pk) = 1, 0 ≤ pk ≤ 1.
2 Posterior PDF p(pk|Dk−1)
I If θ(1)k−1, . . . , θ
(N)k−1 are i.i.d. according to π(·|Fk−1)
⇒ IFk (θ(1)k−1), . . . , IFk (θ
(N)k−1) can be interpreted as Bernoulli trials
⇒ Bayes’ Theorem (1763):
p(pk|Dk−1) =pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
I In fact, θ(1)k−1, . . . , θ
(N)k−1 are MCMC samples (for k ≥ 2)
⇒ θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1), however, they are not independent
p(pk|Dk−1) ≈pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 26 / 32
Prior and Posterior for pk = P (Fk|Fk−1)
1 Prior PDF p(pk)
Principle of Maximum Entropy:
p(pk) = 1, 0 ≤ pk ≤ 1.
2 Posterior PDF p(pk|Dk−1)
I If θ(1)k−1, . . . , θ
(N)k−1 are i.i.d. according to π(·|Fk−1)
⇒ IFk (θ(1)k−1), . . . , IFk (θ
(N)k−1) can be interpreted as Bernoulli trials
⇒ Bayes’ Theorem (1763):
p(pk|Dk−1) =pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
I In fact, θ(1)k−1, . . . , θ
(N)k−1 are MCMC samples (for k ≥ 2)
⇒ θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1), however, they are not independent
p(pk|Dk−1) ≈pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 26 / 32
Prior and Posterior for pk = P (Fk|Fk−1)
1 Prior PDF p(pk)
Principle of Maximum Entropy:
p(pk) = 1, 0 ≤ pk ≤ 1.
2 Posterior PDF p(pk|Dk−1)
I If θ(1)k−1, . . . , θ
(N)k−1 are i.i.d. according to π(·|Fk−1)
⇒ IFk (θ(1)k−1), . . . , IFk (θ
(N)k−1) can be interpreted as Bernoulli trials
⇒ Bayes’ Theorem (1763):
p(pk|Dk−1) =pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
I In fact, θ(1)k−1, . . . , θ
(N)k−1 are MCMC samples (for k ≥ 2)
⇒ θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1), however, they are not independent
p(pk|Dk−1) ≈pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 26 / 32
Prior and Posterior for pk = P (Fk|Fk−1)
1 Prior PDF p(pk)
Principle of Maximum Entropy:
p(pk) = 1, 0 ≤ pk ≤ 1.
2 Posterior PDF p(pk|Dk−1)
I If θ(1)k−1, . . . , θ
(N)k−1 are i.i.d. according to π(·|Fk−1)
⇒ IFk (θ(1)k−1), . . . , IFk (θ
(N)k−1) can be interpreted as Bernoulli trials
⇒ Bayes’ Theorem (1763):
p(pk|Dk−1) =pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
I In fact, θ(1)k−1, . . . , θ
(N)k−1 are MCMC samples (for k ≥ 2)
⇒ θ(1)k−1, . . . , θ
(N)k−1 ∼ π(·|Fk−1), however, they are not independent
p(pk|Dk−1) ≈pnkk (1− pk)N−nk
B(nk + 1, N − nk + 1)
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 26 / 32
Posterior PDF for pF
Last step: To find the PDF of pF =∏mk=1 pk, given the PDFs of all factors
pk ∼ Be(nk + 1, N − nk + 1)
Idea: To approximate pF by a single beta variable
Theorem (Da-Yin Fan, 1991)
Let X1, . . . , Xm be beta variables, Xk ∼ Beta(ak, bk), and Y = X1X2 . . . Xm.
Then Y is approximately distributed as Y ∼ Beta(a, b), where
a = µ1µ1 − µ2
µ2 − µ21
, b = (1− µ1)µ1 − µ2
µ2 − µ21
,
µ1 =
m∏k=1
akak + bk
, µ2 =
m∏k=1
ak(ak + 1)
(ak + bk)(ak + bk + 1).
Nice property of this approximation: E[Y ] = E[Y ], E[Y 2] = E[Y 2]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 27 / 32
Posterior PDF for pF
Last step: To find the PDF of pF =∏mk=1 pk, given the PDFs of all factors
pk ∼ Be(nk + 1, N − nk + 1)
Idea: To approximate pF by a single beta variable
Theorem (Da-Yin Fan, 1991)
Let X1, . . . , Xm be beta variables, Xk ∼ Beta(ak, bk), and Y = X1X2 . . . Xm.
Then Y is approximately distributed as Y ∼ Beta(a, b), where
a = µ1µ1 − µ2
µ2 − µ21
, b = (1− µ1)µ1 − µ2
µ2 − µ21
,
µ1 =
m∏k=1
akak + bk
, µ2 =
m∏k=1
ak(ak + 1)
(ak + bk)(ak + bk + 1).
Nice property of this approximation: E[Y ] = E[Y ], E[Y 2] = E[Y 2]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 27 / 32
Posterior PDF for pF
Last step: To find the PDF of pF =∏mk=1 pk, given the PDFs of all factors
pk ∼ Be(nk + 1, N − nk + 1)
Idea: To approximate pF by a single beta variable
Theorem (Da-Yin Fan, 1991)
Let X1, . . . , Xm be beta variables, Xk ∼ Beta(ak, bk), and Y = X1X2 . . . Xm.
Then Y is approximately distributed as Y ∼ Beta(a, b), where
a = µ1µ1 − µ2
µ2 − µ21
, b = (1− µ1)µ1 − µ2
µ2 − µ21
,
µ1 =
m∏k=1
akak + bk
, µ2 =
m∏k=1
ak(ak + 1)
(ak + bk)(ak + bk + 1).
Nice property of this approximation: E[Y ] = E[Y ], E[Y 2] = E[Y 2]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 27 / 32
Posterior PDF for pF
Last step: To find the PDF of pF =∏mk=1 pk, given the PDFs of all factors
pk ∼ Be(nk + 1, N − nk + 1)
Idea: To approximate pF by a single beta variable
Theorem (Da-Yin Fan, 1991)
Let X1, . . . , Xm be beta variables, Xk ∼ Beta(ak, bk), and Y = X1X2 . . . Xm.
Then Y is approximately distributed as Y ∼ Beta(a, b), where
a = µ1µ1 − µ2
µ2 − µ21
, b = (1− µ1)µ1 − µ2
µ2 − µ21
,
µ1 =
m∏k=1
akak + bk
, µ2 =
m∏k=1
ak(ak + 1)
(ak + bk)(ak + bk + 1).
Nice property of this approximation: E[Y ] = E[Y ], E[Y 2] = E[Y 2]
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 27 / 32
Bayesian post-processor for Subset Simulation
Point estimate pSSF PDF pSS+(pF ) = Be(pF |a, b)
a =
∏mk=1
nk+1N+2
(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
b =
(1−
∏mk=1
nk+1N+2
)(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
What is the relationship between pSS+ and pSSF ?
EpSS+ [pF ]→ pSSF , as N →∞
EpSS+ [pF ] ≈ pSSF , when N is large
The PDF pSS+ can be fully used for life-cost analyses, decision making, etc.
E[Loss(pF )] =
∫Loss(pF )pSS+(pF )dpF
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 28 / 32
Bayesian post-processor for Subset Simulation
Point estimate pSSF PDF pSS+(pF ) = Be(pF |a, b)
a =
∏mk=1
nk+1N+2
(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
b =
(1−
∏mk=1
nk+1N+2
)(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
What is the relationship between pSS+ and pSSF ?
EpSS+ [pF ]→ pSSF , as N →∞
EpSS+ [pF ] ≈ pSSF , when N is large
The PDF pSS+ can be fully used for life-cost analyses, decision making, etc.
E[Loss(pF )] =
∫Loss(pF )pSS+(pF )dpF
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 28 / 32
Bayesian post-processor for Subset Simulation
Point estimate pSSF PDF pSS+(pF ) = Be(pF |a, b)
a =
∏mk=1
nk+1N+2
(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
b =
(1−
∏mk=1
nk+1N+2
)(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
What is the relationship between pSS+ and pSSF ?
EpSS+ [pF ]→ pSSF , as N →∞
EpSS+ [pF ] ≈ pSSF , when N is large
The PDF pSS+ can be fully used for life-cost analyses, decision making, etc.
E[Loss(pF )] =
∫Loss(pF )pSS+(pF )dpF
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 28 / 32
Bayesian post-processor for Subset Simulation
Point estimate pSSF PDF pSS+(pF ) = Be(pF |a, b)
a =
∏mk=1
nk+1N+2
(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
b =
(1−
∏mk=1
nk+1N+2
)(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
What is the relationship between pSS+ and pSSF ?
EpSS+ [pF ]→ pSSF , as N →∞
EpSS+ [pF ] ≈ pSSF , when N is large
The PDF pSS+ can be fully used for life-cost analyses, decision making, etc.
E[Loss(pF )] =
∫Loss(pF )pSS+(pF )dpF
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 28 / 32
Bayesian post-processor for Subset Simulation
Point estimate pSSF PDF pSS+(pF ) = Be(pF |a, b)
a =
∏mk=1
nk+1N+2
(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
b =
(1−
∏mk=1
nk+1N+2
)(1−
∏mk=1
nk+2N+3
)∏mk=1
nk+2N+3 −
∏mk=1
nk+1N+2
What is the relationship between pSS+ and pSSF ?
EpSS+ [pF ]→ pSSF , as N →∞
EpSS+ [pF ] ≈ pSSF , when N is large
The PDF pSS+ can be fully used for life-cost analyses, decision making, etc.
E[Loss(pF )] =
∫Loss(pF )pSS+(pF )dpF
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 28 / 32
Elasto-Plastic Structure Subjected to Ground Motion
S.K. Au (Computers & Structures, 2005):
2D moment-resisting steel frame
Synthetic ground motion a = a(Z)
I Z = (Z1, . . . , Zd)i.i.d∼ N (0, 1)
IZ−→ Filter
a(Z)−−−→I d = 1001
Failure domain:
F = {Z ∈ Rd : δmax(Z) > b}
δmax = maxi=1,...,6
δi
δi is the maximum absolute
interstory drift ratio of the ith story
within the duration of study, 30 s
b = 0.5%⇒ pF ≈ 8.9× 10−3
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 29 / 32
Elasto-Plastic Structure Subjected to Ground Motion
S.K. Au (Computers & Structures, 2005):
2D moment-resisting steel frame
Synthetic ground motion a = a(Z)
I Z = (Z1, . . . , Zd)i.i.d∼ N (0, 1)
IZ−→ Filter
a(Z)−−−→I d = 1001
Failure domain:
F = {Z ∈ Rd : δmax(Z) > b}
δmax = maxi=1,...,6
δi
δi is the maximum absolute
interstory drift ratio of the ith story
within the duration of study, 30 s
b = 0.5%⇒ pF ≈ 8.9× 10−3
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 29 / 32
Summary
MCMC algorithms computes multi-dimensional integrals using Markov chains
MCMC-based methods can be very useful for solving engineering problems
I Reliability Engineering
Subset Simulation (Au & Beck, 2001)
I a very efficient MCMC method for estimation of small failure probabilities
Enhancements for Subset Simulation
I MMHDR = MMH (Au & Beck, 2001) + MHDR (Tierney & Mira, 1999)
I Bayesian post-processor for Subset Simulation
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 30 / 32
Summary
MCMC algorithms computes multi-dimensional integrals using Markov chains
MCMC-based methods can be very useful for solving engineering problems
I Reliability Engineering
Subset Simulation (Au & Beck, 2001)
I a very efficient MCMC method for estimation of small failure probabilities
Enhancements for Subset Simulation
I MMHDR = MMH (Au & Beck, 2001) + MHDR (Tierney & Mira, 1999)
I Bayesian post-processor for Subset Simulation
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 30 / 32
Summary
MCMC algorithms computes multi-dimensional integrals using Markov chains
MCMC-based methods can be very useful for solving engineering problems
I Reliability Engineering
Subset Simulation (Au & Beck, 2001)
I a very efficient MCMC method for estimation of small failure probabilities
Enhancements for Subset Simulation
I MMHDR = MMH (Au & Beck, 2001) + MHDR (Tierney & Mira, 1999)
I Bayesian post-processor for Subset Simulation
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 30 / 32
Summary
MCMC algorithms computes multi-dimensional integrals using Markov chains
MCMC-based methods can be very useful for solving engineering problems
I Reliability Engineering
Subset Simulation (Au & Beck, 2001)
I a very efficient MCMC method for estimation of small failure probabilities
Enhancements for Subset Simulation
I MMHDR = MMH (Au & Beck, 2001) + MHDR (Tierney & Mira, 1999)
I Bayesian post-processor for Subset Simulation
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 30 / 32
Acknowledgements
Jim Beck (Caltech)
I for teaching me Bayesian way of thinking
I for mentoring my research at Caltech
Lambros Katafygiotis (HKUST)
I for introducing me to the world of MCMC algorithms
I for supervising me at HKUST
Ivan Au (CityU)
I for providing Matlab code for the nonlinear example
I for being always open for constructive discussions
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 31 / 32
Thank you for attention!
Konstantin Zuev (Caltech) MCMC and its Application Mechanical & Civil Engineering Sem. 32 / 32
Top Related