MCMC and likelihood-free methods

MCMC and Likelihood-free Methods


Christian P. Robert

Universite Paris-Dauphine & CRESThttp://www.ceremade.dauphine.fr/~xian

November 2, 2010

http://www.ceremade.dauphine.fr/~xian


Outline

Computational issues in Bayesian statistics

The Metropolis-Hastings Algorithm

The Gibbs Sampler

Population Monte Carlo

Approximate Bayesian computation

ABC for model choice



Motivation and leading example



The Gibbs Sampler






Latent variables

Latent structures make life harder!

Even simple models may lead to computational complications, asin latent variable models

f(x|θ) =∫f?(x, x?|θ) dx?

If (x, x?) observed, fine!If only x observed, trouble!



Latent variables



f(x|θ) =∫f?(x, x?|θ) dx?

If (x, x?) observed, fine!

If only x observed, trouble!



Latent variables



f(x|θ) =∫f?(x, x?|θ) dx?

If (x, x?) observed, fine!If only x observed, trouble!



Latent variables

Example (Mixture models)

Models of mixtures of distributions:

X ∼ fj with probability pj ,

for j = 1, 2, . . . , k, with overall density

X ∼ p1f1(x) + · · ·+ pkfk(x) .

For a sample of independent random variables (X1, · · · , Xn),sample density

n∏i=1

{p1f1(xi) + · · ·+ pkfk(xi)} .

Expanding this product of sums into a sum of products involves kn

elementary terms: too prohibitive to compute in large samples.



Latent variables

Simple mixture (1)

−1 0 1 2 3

−1

01

23

µ1

µ 2

Case of the 0.3N (µ1, 1) + 0.7N (µ2, 1) likelihood



Latent variables

Simple mixture (2)

For the mixture of two normal distributions,

0.3N (µ1, 1) + 0.7N (µ2, 1) ,

likelihood proportional to

n∏i=1

[0.3ϕ (xi − µ1) + 0.7 ϕ (xi − µ2)]

containing 2n terms.



Latent variables

Complex maximisation

Standard maximization techniques often fail to find the globalmaximum because of multimodality or undesirable behavior(usually at the frontier of the domain) of the likelihood function.

Example

In the special case

f(x|µ, σ) = (1− ε) exp{(−1/2)x2}+ε

σexp{(−1/2σ2)(x− µ)2}

with ε > 0 known,

whatever n, the likelihood is unbounded:

limσ→0

L(x1, . . . , xn|µ = x1, σ) =∞



Latent variables

Complex maximisation

Standard maximization techniques often fail to find the globalmaximum because of multimodality or undesirable behavior(usually at the frontier of the domain) of the likelihood function.

Example

In the special case

f(x|µ, σ) = (1− ε) exp{(−1/2)x2}+ε

σexp{(−1/2σ2)(x− µ)2}

with ε > 0 known, whatever n, the likelihood is unbounded:

limσ→0

L(x1, . . . , xn|µ = x1, σ) =∞



Latent variables

Unbounded likelihood

−2 0 2 4 6

12

34

µ

n= 3

−2 0 2 4 6

12

34

µ

σ

n= 6

−2 0 2 4 6

12

34

µ

n= 12

−2 0 2 4 6

12

34

µ

σ

n= 24

−2 0 2 4 6

12

34

µ

n= 48

−2 0 2 4 6

12

34

µ

σ

n= 96

Case of the 0.3N (0, 1) + 0.7N (µ, σ) likelihood



Latent variables

Example (Mixture once again)

press for MA Observations from

x1, . . . , xn ∼ f(x|θ) = pϕ(x;µ1, σ1) + (1− p)ϕ(x;µ2, σ2)

Prior

µi|σi ∼ N (ξi, σ2i /ni), σ2

i ∼ I G (νi/2, s2i /2), p ∼ Be(α, β)

Posterior

π(θ|x1, . . . , xn) ∝n∏j=1

{pϕ(xj ;µ1, σ1) + (1− p)ϕ(xj ;µ2, σ2)}π(θ)

=n∑`=0

∑(kt)

ω(kt)π(θ|(kt))

[O(2n)]



Latent variables

Example (Mixture once again (cont’d))

For a given permutation (kt), conditional posterior distribution

π(θ|(kt)) = N

(ξ1(kt),

σ21

n1 + `

)×I G ((ν1 + `)/2, s1(kt)/2)

×N

(ξ2(kt),

σ22

n2 + n− `

)×I G ((ν2 + n− `)/2, s2(kt)/2)

×Be(α+ `, β + n− `)



Latent variables

Example (Mixture once again (cont’d))

where

x1(kt) = 1`

∑`t=1 xkt , s1(kt) =

∑`t=1(xkt − x1(kt))2,

x2(kt) = 1n−`

∑nt=`+1 xkt , s2(kt) =

∑nt=`+1(xkt − x2(kt))2

and

ξ1(kt) =n1ξ1 + `x1(kt)

n1 + `, ξ2(kt) =

n2ξ2 + (n− `)x2(kt)n2 + n− `

,

s1(kt) = s21 + s2

1(kt) +n1`

n1 + `(ξ1 − x1(kt))2,

s2(kt) = s22 + s2

2(kt) +n2(n− `)n2 + n− `

(ξ2 − x2(kt))2,

posterior updates of the hyperparameters



Latent variables

Example (Mixture once again)

Bayes estimator of θ:

δπ(x1, . . . , xn) =n∑`=0

∑(kt)

ω(kt)Eπ[θ|x, (kt)]

Too costly: 2n terms



The AR(p) model

AR(p) model

Auto-regressive representation of a time series,

xt|xt−1, . . . ∼ N

(µ+

p∑i=1

%i(xt−i − µ), σ2

)

I Generalisation of AR(1)I Among the most commonly used models in dynamic settings

I More challenging than the static models (stationarityconstraints)

I Different models depending on the processing of the startingvalue x0



The AR(p) model

Unknown stationarity constraints

Practical difficulty: for complex models, stationarity constraints getquite involved to the point of being unknown in some cases

Example (AR(1))

Case of linear Markovian dependence on the last value

xt = µ+ %(xt−1 − µ) + εt , εti.i.d.∼ N (0, σ2)

If |%| < 1, (xt)t∈Z can be written as

xt = µ+∞∑j=0

%jεt−j

and this is a stationary representation.



The AR(p) model

Stationary but...

If |%| > 1, alternative stationary representation

xt = µ−∞∑j=1

%−jεt+j .

This stationary solution is criticized as artificial because xt iscorrelated with future white noises (εt)s>t, unlike the case when|%| < 1.Non-causal representation...



The AR(p) model

Stationarity+causality

Stationarity constraints in the prior as a restriction on the values ofθ.

Theorem

AR(p) model second-order stationary and causal iff the roots of thepolynomial

P(x) = 1−p∑i=1

%ixi

are all outside the unit circle



The AR(p) model

Stationarity constraints

Under stationarity constraints, complex parameter space: eachvalue of % needs to be checked for roots of correspondingpolynomial with modulus less than 1

E.g., for an AR(2) process withautoregressive polynomialP(u) = 1− %1u− %2u

2, constraint is

%1 + %2 < 1, %1 − %2 < 1

and |%2| < 1

●

−2 −1 0 1 2

−1.

0−

0.5

0.0

0.5

1.0

θ1

θ 2



The MA(q) model

The MA(q) model

Alternative type of time series

xt = µ+ εt −q∑j=1

ϑjεt−j , εt ∼ N (0, σ2)

Stationary but, for identifiability considerations, the polynomial

Q(x) = 1−q∑j=1

ϑjxj

must have all its roots outside the unit circle.



The MA(q) model

Identifiability

Example

For the MA(1) model, xt = µ+ εt − ϑ1εt−1,

var(xt) = (1 + ϑ21)σ2

can also be written

xt = µ+ εt−1 −1ϑ1εt, ε ∼ N (0, ϑ2

1σ2) ,

Both pairs (ϑ1, σ) & (1/ϑ1, ϑ1σ) lead to alternativerepresentations of the same model.



The MA(q) model

Properties of MA models

I Non-Markovian model (but special case of hidden Markov)

I Autocovariance γx(s) is null for |s| > q



The MA(q) model

Representations

x1:T is a normal random variable with constant mean µ andcovariance matrix

Σ =

σ2 γ1 γ2 . . . γq 0 . . . 0 0γ1 σ2 γ1 . . . γq−1 γq . . . 0 0

. . .

0 0 0 . . . 0 0 . . . γ1 σ2

,

with (|s| ≤ q)

γs = σ2

q−|s|∑i=0

ϑiϑi+|s|

Not manageable in practice [large T’s]



The MA(q) model

Representations (contd.)

Conditional on past (ε0, . . . , ε−q+1),

L(µ, ϑ1, . . . , ϑq, σ|x1:T , ε0, . . . , ε−q+1) ∝

σ−TT∏

t=1

exp

−xt − µ+

q∑j=1

ϑj εt−j

2 /2σ2

,

where (t > 0)

εt = xt − µ+q∑j=1

ϑj εt−j , ε0 = ε0, . . . , ε1−q = ε1−q

Recursive definition of the likelihood, still costly O(T × q)



The MA(q) model

Representations (contd.)

Encompassing approach for general time series modelsState-space representation

xt = Gyt + εt , (1)

yt+1 = Fyt + ξt , (2)

(1) is the observation equation and (2) is the state equation

Note

This is a special case of hidden Markov model



The MA(q) model

MA(q) state-space representation

For the MA(q) model, take

yt = (εt−q, . . . , εt−1, εt)′

and then

yt+1 =

0 1 0 . . . 00 0 1 . . . 0

. . .0 0 0 . . . 10 0 0 . . . 0

yt + εt+1

00...01

xt = µ−

(ϑq ϑq−1 . . . ϑ1 −1

)yt .



The MA(q) model

MA(q) state-space representation (cont’d)

Example

For the MA(1) model, observation equation

xt = (1 0)yt

withyt = (y1t y2t)′

directed by the state equation

yt+1 =(

0 10 0

)yt + εt+1

(1ϑ1

).



The MA(q) model

c© A typology of Bayes computational problems

(i). latent variable models in general

(ii). use of a complex parameter space, as for instance inconstrained parameter sets like those resulting from imposingstationarity constraints in dynamic models;

(iii). use of a complex sampling model with an intractablelikelihood, as for instance in some graphical models;

(iv). use of a huge dataset;

(v). use of a complex prior distribution (which may be theposterior distribution associated with an earlier sample);

(vi). use of a particular inferential procedure as for instance, Bayesfactors

Bπ01(x) =

P (θ ∈ Θ0 | x)P (θ ∈ Θ1 | x)

/π(θ ∈ Θ0)π(θ ∈ Θ1)

.





The Metropolis-Hastings AlgorithmMonte Carlo basicsImportance SamplingMonte Carlo Methods based on Markov ChainsThe Metropolis–Hastings algorithmRandom-walk Metropolis-Hastings algorithmsExtensions

The Gibbs Sampler






Monte Carlo basics

General purpose

Given a density π known up to a normalizing constant, and anintegrable function h, compute

Π(h) =∫h(x)π(x)µ(dx) =

∫h(x)π(x)µ(dx)∫π(x)µ(dx)

when∫h(x)π(x)µ(dx) is intractable.



Monte Carlo basics

Monte Carlo 101

Generate an iid sample x1, . . . , xN from π and estimate Π(h) by

ΠMCN (h) = N−1

N∑i=1

h(xi).

LLN: ΠMCN (h) as−→ Π(h)

If Π(h2) =∫h2(x)π(x)µ(dx) <∞,

CLT:√N(

ΠMCN (h)−Π(h)

)L N

(0,Π

{[h−Π(h)]2

}).

Caveat announcing MCMC

Often impossible or inefficient to simulate directly from Π



Importance Sampling

Importance Sampling

For Q proposal distribution such that Q(dx) = q(x)µ(dx),alternative representation

Π(h) =∫h(x){π/q}(x)q(x)µ(dx).

Principle of importance

Generate an iid sample x1, . . . , xN ∼ Q and estimate Π(h) by

ΠISQ,N (h) = N−1

N∑i=1

h(xi){π/q}(xi).

return to pMC



Importance Sampling

Properties of importance

ThenLLN: ΠIS

Q,N (h) as−→ Π(h) and if Q((hπ/q)2) <∞,

CLT:√N(ΠIS

Q,N (h)−Π(h)) L N

(0, Q{(hπ/q −Π(h))2}

).

Caveat

If normalizing constant of π unknown, impossible to use ΠISQ,N

Generic problem in Bayesian Statistics: π(θ|x) ∝ f(x|θ)π(θ).



Importance Sampling

Self-Normalised Importance Sampling

Self normalized version

ΠSNISQ,N (h) =

(N∑i=1

{π/q}(xi)

)−1 N∑i=1

h(xi){π/q}(xi).

LLN : ΠSNISQ,N (h) as−→ Π(h)

and if Π((1 + h2)(π/q)) <∞,

CLT :√N(ΠSNIS

Q,N (h)−Π(h)) L N

(0, π {(π/q)(h−Π(h)}2)

).

c© The quality of the SNIS approximation depends on thechoice of Q



Monte Carlo Methods based on Markov Chains

Running Monte Carlo via Markov Chains (MCMC)

It is not necessary to use a sample from the distribution f toapproximate the integral

I =∫

h(x)f(x)dx ,

We can obtain X1, . . . , Xn ∼ f (approx) without directlysimulating from f , using an ergodic Markov chain withstationary distribution f



Monte Carlo Methods based on Markov Chains

Running Monte Carlo via Markov Chains (2)

Idea

For an arbitrary starting value x(0), an ergodic chain (X(t)) isgenerated using a transition kernel with stationary distribution f

I Insures the convergence in distribution of (X(t)) to a randomvariable from f .

I For a “large enough” T0, X(T0) can be considered asdistributed from f

I Produce a dependent sample X(T0), X(T0+1), . . ., which isgenerated from f , sufficient for most approximation purposes.

Problem: How can one build a Markov chain with a givenstationary distribution?



The Metropolis–Hastings algorithm


BasicsThe algorithm uses the objective (target) density

f

and a conditional densityq(y|x)

called the instrumental (or proposal) distribution




The MH algorithm

Algorithm (Metropolis–Hastings)

Given x(t),

1. Generate Yt ∼ q(y|x(t)).

2. Take

X(t+1) =

{Yt with prob. ρ(x(t), Yt),x(t) with prob. 1− ρ(x(t), Yt),

where

ρ(x, y) = min{f(y)f(x)

q(x|y)q(y|x)

, 1}.




Features

I Independent of normalizing constants for both f and q(·|x)(ie, those constants independent of x)

I Never move to values with f(y) = 0I The chain (x(t))t may take the same value several times in a

row, even though f is a density wrt Lebesgue measure

I The sequence (yt)t is usually not a Markov chain




Convergence properties

1. The M-H Markov chain is reversible, withinvariant/stationary density f since it satisfies the detailedbalance condition

f(y)K(y, x) = f(x)K(x, y)

2. As f is a probability measure, the chain is positive recurrent

3. If

Pr

[f(Yt) q(X(t)|Yt)f(X(t)) q(Yt|X(t))

≥ 1

]< 1. (1)

that is, the event {X(t+1) = X(t)} is possible, then the chainis aperiodic




Convergence properties (2)

4. Ifq(y|x) > 0 for every (x, y), (2)

the chain is irreducible

5. For M-H, f -irreducibility implies Harris recurrence6. Thus, for M-H satisfying (1) and (2)

(i) For h, with Ef |h(X)| <∞,

limT→∞

1T

T∑t=1

h(X(t)) =∫h(x)df(x) a.e. f.

(ii) and

limn→∞

∥∥∥∥∫ Kn(x, ·)µ(dx)− f∥∥∥∥

TV

= 0

for every initial distribution µ, where Kn(x, ·) denotes thekernel for n transitions.







5. For M-H, f -irreducibility implies Harris recurrence

6. Thus, for M-H satisfying (1) and (2)(i) For h, with Ef |h(X)| <∞,

limT→∞

1T

T∑t=1


(ii) and

limn→∞

∥∥∥∥∫ Kn(x, ·)µ(dx)− f∥∥∥∥

TV

= 0








5. For M-H, f -irreducibility implies Harris recurrence6. Thus, for M-H satisfying (1) and (2)

(i) For h, with Ef |h(X)| <∞,

limT→∞

1T

T∑t=1


(ii) and

limn→∞

∥∥∥∥∫ Kn(x, ·)µ(dx)− f∥∥∥∥

TV

= 0




Random-walk Metropolis-Hastings algorithms

Random walk Metropolis–Hastings

Use of a local perturbation as proposal

Yt = X(t) + εt,

where εt ∼ g, independent of X(t).The instrumental density is of the form g(y − x) and the Markovchain is a random walk if we take g to be symmetric g(x) = g(−x)




Algorithm (Random walk Metropolis)

Given x(t)

1. Generate Yt ∼ g(y − x(t))2. Take

X(t+1) =

Yt with prob. min{

1,f(Yt)f(x(t))

},

x(t) otherwise.




Example (Random walk and normal target)

forget History! Generate N (0, 1) based on the uniform proposal [−δ, δ][Hastings (1970)]

The probability of acceptance is then

ρ(x(t), yt) = exp{(x(t)2 − y2t )/2} ∧ 1.




Example (Random walk & normal (2))

Sample statistics

δ 0.1 0.5 1.0

mean 0.399 -0.111 0.10variance 0.698 1.11 1.06

c© As δ ↑, we get better histograms and a faster exploration of thesupport of f .




-1 0 1 2

050

100

150

200

250

(a)

-1.5

-1.0

-0.5

0.0

0.5

-2 0 2

010

020

030

040

0

(b)

-1.5

-1.0

-0.5

0.0

0.5

-3 -2 -1 0 1 2 3

010

020

030

040

0

(c)

-1.5

-1.0

-0.5

0.0

0.5

Three samples based on U [−δ, δ] with (a) δ = 0.1, (b) δ = 0.5and (c) δ = 1.0, superimposed with the convergence of themeans (15, 000 simulations).




Example (Mixture models)

π(θ|x) ∝n∏j=1

(k∑`=1

p`f(xj |µ`, σ`)

)π(θ)

Metropolis-Hastings proposal:

θ(t+1) ={θ(t) + ωε(t) if u(t) < ρ(t)

θ(t) otherwise

where

ρ(t) =π(θ(t) + ωε(t)|x)

π(θ(t)|x)∧ 1

and ω scaled for good acceptance rate




p

thet

a

0.0 0.2 0.4 0.6 0.8 1.0

-10

12

tau

thet

a

0.2 0.4 0.6 0.8 1.0 1.2

-10

12

p

tau

0.0 0.2 0.4 0.6 0.8 1.0

0.2

0.4

0.6

0.8

1.0

1.2

-1 0 1 2

0.0

1.0

2.0

theta

0.2 0.4 0.6 0.8

02

4

tau

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

6

p

Random walk sampling (50000 iterations)

General case of a 3 component normal mixture[Celeux & al., 2000]




−1 0 1 2 3

−1

01

23

µ1

µ 2

X

Random walk MCMC output for .7N (µ1, 1) + .3N (µ2, 1)





Uniform ergodicity prohibited by random walk structure

At best, geometric ergodicity:

Theorem (Sufficient ergodicity)

For a symmetric density f , log-concave in the tails, and a positiveand symmetric density g, the chain (X(t)) is geometrically ergodic.

[Mengersen & Tweedie, 1996]

no tail effect





Uniform ergodicity prohibited by random walk structureAt best, geometric ergodicity:

Theorem (Sufficient ergodicity)

For a symmetric density f , log-concave in the tails, and a positiveand symmetric density g, the chain (X(t)) is geometrically ergodic.

[Mengersen & Tweedie, 1996]

no tail effect




Example (Comparison of taileffects)

Random-walkMetropolis–Hastings algorithmsbased on a N (0, 1) instrumentalfor the generation of (a) aN (0, 1) distribution and (b) adistribution with densityψ(x) ∝ (1 + |x|)−3

(a)

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

(a)

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

(b)

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

0 50 100 150 200

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

90% confidence envelopes ofthe means, derived from 500parallel independent chains

1 + ξ2

1 + (ξ′)2∧ 1 ,




Further convergence properties

Under assumptions skip detailed convergence

I (A1) f is super-exponential, i.e. it is positive with positivecontinuous first derivative such thatlim|x|→∞ n(x)′∇ log f(x) = −∞ where n(x) := x/|x|.In words : exponential decay of f in every direction with ratetending to ∞

I (A2) lim sup|x|→∞ n(x)′m(x) < 0, wherem(x) = ∇f(x)/|∇f(x)|.In words: non degeneracy of the countour manifoldCf(y) = {y : f(y) = f(x)}

Q is geometrically ergodic, andV (x) ∝ f(x)−1/2 verifies the drift condition

[Jarner & Hansen, 2000]




Further [further] convergence properties

skip hyperdetailed convergence

If P ψ-irreducible and aperiodic, for r = (r(n))n∈N real-valued nondecreasing sequence, such that, for all n,m ∈ N,

r(n+m) ≤ r(n)r(m),

and r(0) = 1, for C a small set, τC = inf{n ≥ 1, Xn ∈ C}, andh ≥ 1, assume

supx∈C

Ex

[τC−1∑k=0

r(k)h(Xk)

]<∞,




then,

S(f, C, r) :=

{x ∈ X,Ex

{τC−1∑k=0

r(k)h(Xk)

}<∞

}

is full and absorbing and for x ∈ S(f, C, r),

limn→∞

r(n)‖Pn(x, .)− f‖h = 0.

[Tuominen & Tweedie, 1994]




Comments

I [CLT, Rosenthal’s inequality...] h-ergodicity implies CLTfor additive (possibly unbounded) functionals of the chain,Rosenthal’s inequality and so on...

I [Control of the moments of the return-time] Thecondition implies (because h ≥ 1) that

supx∈C

Ex[r0(τC)] ≤ supx∈C

Ex

{τC−1∑k=0

r(k)h(Xk)

}<∞,

where r0(n) =∑n

l=0 r(l) Can be used to derive bounds forthe coupling time, an essential step to determine computablebounds, using coupling inequalities

[Roberts & Tweedie, 1998; Fort & Moulines, 2000]




Alternative conditions

The condition is not really easy to work with...[Possible alternative conditions]

(a) [Tuominen, Tweedie, 1994] There exists a sequence(Vn)n∈N, Vn ≥ r(n)h, such that

(i) supC V0 <∞,(ii) {V0 =∞} ⊂ {V1 =∞} and(iii) PVn+1 ≤ Vn − r(n)h+ br(n)IC .




(b) [Fort 2000] ∃V ≥ f ≥ 1 and b <∞, such that supC V <∞and

PV (x) + Ex

{σC∑k=0

∆r(k)f(Xk)

}≤ V (x) + bIC(x)

where σC is the hitting time on C and

∆r(k) = r(k)− r(k − 1), k ≥ 1 and ∆r(0) = r(0).

Result (a) ⇔ (b) ⇔ supx∈C Ex{∑τC−1

k=0 r(k)f(Xk)}<∞.



Extensions

Langevin Algorithms

Proposal based on the Langevin diffusion Lt is defined by thestochastic differential equation

dLt = dBt +12∇ log f(Lt)dt,

where Bt is the standard Brownian motion

Theorem

The Langevin diffusion is the only non-explosive diffusion which isreversible with respect to f .



Extensions

Discretization

Instead, consider the sequence

x(t+1) = x(t) +σ2

2∇ log f(x(t)) + σεt, εt ∼ Np(0, Ip)

where σ2 corresponds to the discretization step

Unfortunately, the discretized chain may be be transient, forinstance when

limx→±∞

∣∣σ2∇ log f(x)|x|−1∣∣ > 1



Extensions

Discretization

Instead, consider the sequence

x(t+1) = x(t) +σ2

2∇ log f(x(t)) + σεt, εt ∼ Np(0, Ip)

where σ2 corresponds to the discretization stepUnfortunately, the discretized chain may be be transient, forinstance when

limx→±∞

∣∣σ2∇ log f(x)|x|−1∣∣ > 1



Extensions

MH correction

Accept the new value Yt with probability

f(Yt)f(x(t))

·exp

{−∥∥∥Yt − x(t) − σ2

2 ∇ log f(x(t))∥∥∥2/

2σ2

}exp

{−∥∥∥x(t) − Yt − σ2

2 ∇ log f(Yt)∥∥∥2/

2σ2

} ∧ 1 .

Choice of the scaling factor σShould lead to an acceptance rate of 0.574 to achieve optimalconvergence rates (when the components of x are uncorrelated)

[Roberts & Rosenthal, 1998; Girolami & Calderhead, 2011]



Extensions

Optimizing the Acceptance Rate

Problem of choice of the transition kernel from a practical point ofviewMost common alternatives:

(a) a fully automated algorithm like ARMS;

(b) an instrumental density g which approximates f , such thatf/g is bounded for uniform ergodicity to apply;

(c) a random walk

In both cases (b) and (c), the choice of g is critical,



Extensions

Case of the random walk

Different approach to acceptance ratesA high acceptance rate does not indicate that the algorithm ismoving correctly since it indicates that the random walk is movingtoo slowly on the surface of f .

If x(t) and yt are close, i.e. f(x(t)) ' f(yt) y is accepted withprobability

min(f(yt)f(x(t))

, 1)' 1 .

For multimodal densities with well separated modes, the negativeeffect of limited moves on the surface of f clearly shows.



Extensions

Case of the random walk

Different approach to acceptance ratesA high acceptance rate does not indicate that the algorithm ismoving correctly since it indicates that the random walk is movingtoo slowly on the surface of f .If x(t) and yt are close, i.e. f(x(t)) ' f(yt) y is accepted withprobability

min(f(yt)f(x(t))

, 1)' 1 .

For multimodal densities with well separated modes, the negativeeffect of limited moves on the surface of f clearly shows.



Extensions

Case of the random walk (2)

If the average acceptance rate is low, the successive values of f(yt)tend to be small compared with f(x(t)), which means that therandom walk moves quickly on the surface of f since it oftenreaches the “borders” of the support of f



Extensions

Rule of thumb

In small dimensions, aim at an average acceptance rate of 50%. Inlarge dimensions, at an average acceptance rate of 25%.

[Gelman,Gilks and Roberts, 1995]

This rule is to be taken with a pinch of salt!



Extensions

Example (Noisy AR(1))

Hidden Markov chain from a regular AR(1) model,

xt+1 = ϕxt + εt+1 εt ∼ N (0, τ2)

and observablesyt|xt ∼ N (x2

t , σ2)

The distribution of xt given xt−1, xt+1 and yt is

exp−12τ2

{(xt − ϕxt−1)2 + (xt+1 − ϕxt)2 +

τ2

σ2(yt − x2

t )2

}.



Extensions

Example (Noisy AR(1) continued)

For a Gaussian random walk with scale ω small enough, therandom walk never jumps to the other mode. But if the scale ω issufficiently large, the Markov chain explores both modes and give asatisfactory approximation of the target distribution.



Extensions

Markov chain based on a random walk with scale ω = .1.



Extensions

Markov chain based on a random walk with scale ω = .5.



Extensions

MA(2)

Since the constraints on (ϑ1, ϑ2) are well-defined, use of a flatprior over the triangle as prior.Simple representation of the likelihood

library(mnormt)ma2like=function(theta){n=length(y)sigma = toeplitz(c(1 +theta[1]^2+theta[2]^2,

theta[1]+theta[1]*theta[2],theta[2],rep(0,n-3)))dmnorm(y,rep(0,n),sigma,log=TRUE)}



Extensions

Basic RWHM for MA(2)

Algorithm 1 RW-HM-MA(2) sampler

set ω and ϑ(1)

for i = 2 to T dogenerate ϑj ∼ U(ϑ(i−1)

j − ω, ϑ(i−1)j + ω)

set p = 0 and ϑ(i) = ϑ(i−1)

if ϑ within the triangle thenp = exp(ma2like(ϑ)−ma2like(ϑ(i−1)))

end ifif U < p thenϑ(i) = ϑ

end ifend for



Extensions

Outcome

Result with a simulated sample of 100 points and ϑ1 = 0.6,ϑ2 = 0.2 and scale ω = 0.2

● ●

−2 −1 0 1 2

−1.

0−

0.5

0.0

0.5

1.0

θ1

θ 2

●●●●

●

●●●

●●●●●

●

●●●

●● ●●●●

●●

●●●●●●●●●●

● ●●●●●●●●

●●●

●

●●●●●●●●

●●●

●●●●●●●

●●●

●●●●●●●●●●●

●

●●●●●

●

●●

●●●●

●●●●●●●

●●●●●

●●●

●●●●●●●●●

●

●

●●●●●●●●●●●●

●●

●●

●

●●●

●●●●●●●●

●●●●●●●

●●●●●

●●●●

●

●●●●●●

●●●●●●●

●●●●●

●●●

●●

●

●●●

●

●

●●

●●●

●●●

●●

●●

●●●●

●●

●

●●●

●●●

●●

●●

●●●

●●●

●●

●●

●●●●●

●●●

●●●●●●

●●●

●●●●●

●●

●●●●

●●●●

●●

●●●●●●●●

●●●●●●●●●

●●

●

●●

●

●

●●●●●●●●

●●●●●●●●

●●●

●●●●●●●●●●●●

●●●●

●●

●●●●●●●●●●●●●

●●●●

●●●

●

●●

●

●●

●●●●

●

●●

●●

●●●●●

●

●● ●●

●●●●●

●

●●●●

●●●●●●●●●

●

●●

●

●●

●●●●

●●●●

●

●●

●●

● ●●●

●●

●●●●●●●

●●●

●●●●●

●

●●●●●

●●

●●●

●●●

●

●

●

●●

●●

●●●●●●●●

●●●●●●●●●●●

●●

●●

●●●●●●●●

●●●

●

●●●●●●

●●●

●●

●●

●●●

●●●

●●

●●●●●●●

●

●●●

●●●●

●

●●●

●●

●

●●●●●●●●●●●●

●●

●●

●

●●●●

●

●●●

●●●●●

●

●●●●●●

●●

●●●●●●

●●●

●●●

●●

●●

●

●●●●

●●

●

●●●

●●●●

●●●●●●●

●

●●●●

●●●

●●

●●●●●●●●●●●

●

●●●●

●●●●

●

●●●●

●●

●●●●●●●

●

●●●●●

●●●●●●

●●

●

●

●

●

●●●

●

●●●●

●●

●●

●●●●●●●●●●●

●●●●

●

●●

●●●●

●●●●●●●●

●●

●●●

●●●●

●

●●

●●

●●

●●●

●●●●●●●

●

●

●

●●●●

●●●●

●●●●

●●●

●●

●

●●●●●

●●●●●

●●●

●●●●●

●

●

●●

●●●●

●●●●●●

●●

●●●●●●●●●●●

●

●●●

●

●

●●●

●

●●

●●

●●●

●●●

●

●

●●●●●●●●●●●●

●●●

●●●●●

●

●

●●

●

●●●●●

●

●

●●●●●

● ●

●

●●●

●●●●●●●●●●●●●●●●●●●

●●●●

●

●

●

●●

●●

●

●●

●●

●

●

●

●●●●●

●

●

●●●●●

●

●●

●

●●●

●

●●●

●

●

●

●

●

●●

●●●●

●

●●

●

●

●

●●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●

●●

●●●●●●●●●

●●●●●●●●●●●●●●

●●●

●● ●

●

●

●

●●●●●●●●●

●

●●

●

●●●●

●

●

●●●

●●●●●

●

●

●

●

●●●●●●

●●

●●●

●

●●●●●●

●

●●●●●●●

●●

●

●●

●●●●●●●●●●

●

●●●●●

●●●●●

●●●●

●●●●●

●●●●●

●●

●●●●●●

●

●

●●

●

●●●●

●●●

●●

●●●●

●●●

●●

●●

●

●● ●●●●

●●●

●

●●

●

●

●●

●

●●●●

●●●●●●●●●●●●●●●●●

●●●

●

●●

●●●

●●●

●●●●

●●●●

●●●

●●●●●●

●●

●●●●

●●●●●

●●●

●

●●

●●●●●●●

●

●●

●●●

●●●●

●●

●●

●

●●

●●●●

●●

●●

●●●●

●●●●●●●●●

●●●●●●●●●●

●●

●

●●

●●●●●●●

●●●

●●

●●●

●

●●

●●

●●●●●

●●

●

●

●

●●●●

●●●●●●●●

●●●

●●

●

●

●●●●●

●●

●●●●

●

●●●●●●●●

●●●

●●

●●●●

●●

●●

●

●●●●●●●

●●●

●●

●

●●●

●

●●

●

●●●

●

●

●

●●

●●

●●●●●●

●●●●●

●●

●●●●

●●●●

●●●●

●

●

●●●

●

●

●●●●●

●

●●●

●●

●

●●● ●

●●●

●

●●●

●●●

●

●●●●●●

●

●

●●●

●

●●●

●

●●●●

●●

●●●

●

●●

●●

●

●●●●●

●

●

●●●

●●●●

●●●

●●●

●●●●●●●●●●

●●●●●

●

●

●

●●●●

●●●●●●●●●●●●●●●●●●

●●●

●●

●●

●

●●●●●

●●●

●●●

●●●

●●●●●

●

●●●●●●●●●●●●

●

●●

●●●●●●●●●

●●●●●

●●●●●●●

●●

●●●●

●

●●●●

●

●●

●

●●

●

●

●

●●●

●

●

●●

●●●●●●

●

●

●

●

●●

●●●●●

●●●

●●

●●●●●

●●●●●●●

●

●●

●●●●●●●●●

●●●●●●●●

●●

●●●●●

●●

●●●●●●

●●●●●●●●●●●●●●

●●●

●●●●●●●● ●●

●●●

●●●●●●●●●

●

●

●

●

●

●●●

●●

●● ●●●

●

●

●●

●

●●

●

●●●●●

●●●●

●●●● ●

●

●

●

●●

●

●●●

●

●●●●●●

●

● ●

●●●●●●●●●

●●● ●●●●●

●●●●●●●●●●

●

●

●● ●●●●

●

●●●●

●●

●●

●●●●●

●●

●

●

●

●

●

●●●

●●

●●●●

●●

● ● ●

●●●●

●●●

●

●●●●

●●

●

●●

●●

●●●●●

●

●●

●

●

●

●●●

●●●●●●

●

●

●●●●●●●●●●●●●● ●●

●●

●●

●●●●

●●●●●●●●

●●●

●●

●●

●●●●●●●

●

●●●

● ●●

●

●●

●

●●

●●●●●●●●●●●●●

●●

●

●

●●●●●●●●●●

●●●

●●●

●●●●●●

●●

●●

●●●●●●●

●●●●●●●●●●●●●

●

●●●●

●●

●●

●

●●

●

●

●●●●

●●

●

●

●●●●

●

●●●●

●●

●●●●

●●

●●

●

●

●●●●●

●

●●

●●●●●●●●

●

●●●●●●●●

●

●●

●●●●●●●

●●●●●

●●●●

●●●●●

●●●

●

●●●●●

●●●●●●●

●●●●●

●

●●●●

●●

●●●●●●●●

●●●●●●●

●●●●

●●●●

●

●●●●

●●●

●

●●

●●●

●

●●●●●

●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●

●●●●●●●●●●

●●●

●

●

●●●

●

●

●

●●●

●●

●●●●●●●●●

●●●●

●●

●●●

●●●●

●●●

●●

●

●●

●●●●●●●●●

●

●●

●

●●●●●●

●

●●●●●●

●●●●●●●●●●●●●●

●●●●

●

●●●●●●●

●●●●●●

●●●

●

●●

●

●

●●

●

●

●●

●●●●

●

●

●●●

●●●

●

●

●●●●●●●

●

●

●●●●●●●●●

●●●

●●

●●●●●●

●●●●

●●●●●

●●

●●●

●

●●●

●

●●●●●

●

●●●●

●●

●

●●●

●●●

●●●

●

●●●

●

●●●●

●●

●

●●●●

●

●●●●●

●●

●●

●● ●

●

●●

●●●

●●●●●●

●●

●●●

●

●●

●

●

●●●●●

●

●●●●●●●●●●●●●●

●●●●●

●●●

●

●●●

●●●●●●●

●

●●

●

●●

●●●

●●●

●

●

●●

●●

●●

●●●●

●

●●●●●●●

●●●● ●

●●●●

●●

●●●●

●

●

●●

●●●

●●●

●●●

●●●●●

●●

●●

●

●●

●

●●●

●●●●●

●●●

●●

●

●

●●

●●

●

●

●

●●

●●●●

●

●●

●

●●

●

●●●●●

●

●●●●●●●●●

●●

●

●●●

●●

●●●●●●●●●●●

●

●●●

●●●●

●

●●●●

●●●●●●●●●

●

●

●●

●

●

●●●●

●●●●●●

●●

●●

●

●●

●●●●●●●●●●●

●●

●●

●●

●●●●

●●

●

●●●

●●

●●

●●

●●

●●●

●●

●●●●

●●

●

●●

●●●●●●

●●●●●●

●

●●● ●●●

●●● ●●

●●●●●●●●●●●●●●●●●●●●

●

●

●

●●●

●

●●●●●●

●●

●

●

●●●●

●●

●

●●●●

●

●●

●●●●

●●

●●

●●●●●●●●●

●

●

●●●●

●

●●● ●●●

●●●●

●

●●●●●●

●●●●●●

●●●●●

●●●●●●● ●

●●●●

●●●●●●●

●●

●

●

●●●●●●●

●●●●●

●

●●●

●

●

●●●●●●

●●●

●●● ●

●●

●●

●●

●●●●●

●●●●●●●●●●●●

●

●●●

●●●

●●●

●

●●●

●●●

●●

●●●●●●●●

●●

●●●●●●

●

●●●●●

●●

●●

●

●●●●

●●●●

●

●●●●●

●

●

●

●●●●

●

●●

●

●●●●●

●

●●●

●

●●●●●●●●●●

●●

●●

●

●

●

●●●

●

●

●●

●

●●

●●●

●

●●●●

●●●●●●

●●●

●●●●

●●●●●●●●●

●●●●

●

●●●●

●●●●

●

●●●

●●●●●

●●●●

●

●

●●

●●●●●●●

●

●

●●●

● ●●●●

●●●●●●●●●

●

●●●

●●

●●●●

●●

●●●

●●●

●●●●

●●

●●●●

●

●

●●●

●

●●●●●●

●●●●●●

●●●

●●●●●●●●●●●●●

●● ●

●●

●●●●●●●●●●●●●

●●●●

●

●

●●

●●●●

●●●

●●●●

●●●

●●●●●

●●

●●

●

●●

●●

●●●

●●●●●

●

●●●●●●●●●●●●●●●●●

●

●●

●●●

●

●●

●●●●●●●●

●

●●●●●●

●●

●●

●●

●

●●●●●●●●●●●

●

●●●●●●●

●●

●●●●●●

●●●●●●

●

●

●●●●●●

●●

●●●●●

●

●●●

●●

●●●●●

●●●●●●●

●●

●●●

●

●

●

●●

●●

●●●●●●●

●●●●

●

●

●●●●●

●

●●●

●

●

●●●

●●

●●●●

●●●●●●●

●●●

●●● ●●●●●●

●

●●

●●●

●●●●●●

●●

●

●

●

●●●●●●●●●●

●●●●●●

●

●

●●●

●●●●●●

●●●●●●

●●●●●●●●

●●

●●●

●●

●●●●●●●

●●●●

●●●●●

●●

●●●●●●●●

●●●●

●

●●●

●●●● ●●●●●●●●●●●●●●●

●●●●

●●

●●●●●

●●●●●

●●

●

●●●●●●●●

●●

●●●

●●●●●●

●●

●●

●

●●

●●●●●

●●●●●●●●

●

●●●●●●●

●●

●●●

●

●●

●●●●●●●●●●

●

●●

●●●●●

●●●

●●●

●

●●

●

●

●●

●

●●

●●

●●●●

●

●●●●●●

●●●

●●●●

●●●●●●●

●●●●●●

●

●●●

●●●

●●●●●

●●

●●●●

●●●●●

●●

●●

●●●

●

●●●

●

●●

●●

●●

●

●

●●

●

●●●●●

●

●●

●●

●

●●●●●●●●

●●●●●

●●●●●

●●●

●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●

●●

●

●●

●●●

●

●

●

●●●

●●●●

●●●

●

●●●●●

●

●●

●●

●●

●

●●●●●

●●

●

●

●

●

●●●●

●

●

●●

●

●●

●●●●

●●●●●

●●

● ●

●●

●

●●●●●●●●●●●●

●●●

●●

●●●●●●

●

●

●●

●●

●●●●●●●●●

●●

●

●

●●

●

●●

●●●●●●

●●

●●●●●●●●

●

●●

●

●

●

●●●

●●

●

●

●●●

●●

●●●

●●

●●●

●●●

●

●

●●●

●●●●●

●●●●●●

●●●●●●

●●

●●●●●●●

●

●●

●●●●

●●●●●●

●●●●●●●

●●

●

●

●●

●●●

●

●●

●

●●●

●

●● ●●

●

●●●●●●

●●●●●●●●●●●

● ●●●●

●●

●

●●●●

●●●

●●

●●●●

●

●●

●●

●

●●

●●●

●

●●●

●●●●●●●●●●●●●●●

●●●●●

●●●●

●●●●●● ●●●●●●●●●

●●●

●

●●

●●

●●

●●

●

●

●●●●●●

●

●

●

●●

●●●●

●●●●●

●●●●●●●●●●

●●●●●●●●●●

●●●●

●

●●

●●

●

●●●●

●●

●

●

●●●●●

●●

●●●

●●

●●

●●●

●

●

●●

●●●●●●●●●

●

●●●●

●●●

●

●●●

●●

●●●●●

●●●●●●●●●●●●●●● ●● ●●

●●●

●●●●●

●●

●

●

●

●●●●

●●

●●●●●

●●

●●●

●●●●●●●●●

●●

●

●

●

●

●●●

●●●●●

●

●

●●

●●●●●

●●●●

●●●●●

●

●●

●●●●●

●●

●●●

●

●

●●●●●●●●●●●●●●

●●●●

●●●●●

●●

●●●

●●●

●●●●

●●

●●●●●

●

●

●

●●

●●●

●●●●●

●●●●●

●●●●●●●●

●●

●

●●●●●

●

●●●●●●●●●●●●●●

●●

●●

●

●

●

●●

●

●

●●●●●

●●●●●●●

●

●●

●●

●●●●●●●

●

●●●●●●●●●●

●

●

●●●

●●

●●●

●●●●●●●●●●●●●●●●●●●

●●●

●

●●●●●

●●●●

●●

●●

●●●●

●●

●

●●

●

●

●●●●

●

●●●●●●

●●●●●●●●

●●●

●●

●●

●●

●●

●●

●

●●

●

●

●

●●●●

●

●●

●●●

●●●

●●

●●

●

●

●●

●●●

●●●●●●●●●●●●

●

●

●●●

●●●●●

●●

●

●●

●

●●●

●

●●

●●

●

●

●●●●●

●●●●●

●●●●●

●

●

●●●

●

●●

●

●

●

●

●●●

●●

●

●●●●

●

●

●●●●

●●●●●●

●●●

●●

●●●●

●

●●

●

●

●●

●

●●●●

●

●

●

●●●

●●●●

●●●

●●●●

●●

●

●●●

●●

●●●●●●●●●

●●

●●●●●●

●●●●

●

●●●

●

●

●

●

●●●●●●●

●●●●

●●●●●

●

●●●●●

●●●●●●●●●●●●●●

●●●

●

●●

●●●●●●●●●

●●●

●●●

●●

●

●●●●

●●●●●

●●●

●●●●●●●

●●

●●

●●●●

●●

●●●●●●

●●●

●●●

●●●

●●●●●

●

●●●

●●●●

●●●●

●●●●

●●●●●●●●●

●●

●

●

●●●●

●●

●

●●●●

●●

●●●●●

●●●

●

●●●●●

●

●●●●●●

●

●

●

●●●●

●

●

●●●

●●●●●

●

●

●●●● ●●

●●●●

●●

●

●●

●●●●

●●

●●●●●●●●

●

●●●●

● ●●●●

●●

●●●●●●●●●●

●●●

●

●●●●●

●●

●●

●

●●●●●●

●●●

●●●●●●

●●●●

●●●

●●●●●

●

●●●

●●● ●

●●

●●

●

●●

●●

●

●

●

●●●

●●●●●●●●

●●

●●●●

●●●●● ●●

●●

●

●●●

●●

●

●

●●●●

●●●

●●●●●●

●●●●●●

●●

●

●●●●

●●●●●●●

●●

●

●

●●●●

●●●●

●

●

●●●●●

●

●●●●

●●

●●●

●

●●

●●●●●●●●●●●●●●●

●●●●●●

●●

●●●

●●●●●●●●

●●●●

●●

●●●●

●

●●●●●●●

●

●●●●●●●

●

●●●

●

●●●●

●●●

●

●

●

●●●●

●●

●●●●●●●●●●●●●●●●

● ●●●●●●

●●

●●●

●

●●

●

●●●●

●

●●

●●●●●

●●

●●●●●●●●●●●

●

●

●●

●

●●●●

●●●

●●●●●

●●

●●●●●●●●●●

●●

●●●●●●●

●

●●

●●●●●●

●

●●●●●

●

●●●

●

●

●●

●

●●●

●●

●●

●

●

●

●●

●

●●●

●●●

●

●●

●●●●●

●

●●●

●●●

●

●●●●●●●

●

●●●

●

●

●●●●●

●

●●●●●●●

●●●●

●●●

●●●●●

●●●●●

●

●●●●●●●●●●●

●●●

●

●●●●●

●●●●●●

●●●

●●●●●●

●●●●●●●●●●

●

●

●

●

●●●

●●

●●●●●●●●●

●●●

●

●●●●●●

●

●

●

●●●●

●●

●●

●●●●●●●●

●

●

●

●●

●●

●

●●●

●●●●●●●

●

●●●●●●●●●●●●●

●●●●●●●●

●

●●●

●●

●●●

●●●

●

●

●●

●●●●●●

●

●●●●●●●●

●

●

●

●●

●●●●

●●●

●●●●●●●●

●

●●●

●

●

●●

●●●●●●●●

●●●● ●

●

●●●

●●●●●●●●

●●●●●

●

●●●●●

●

●●●

●

●●●

●

●

●●

●

●●●●●●

●●

●●●●●

●●●

●●●●

●●●●●●●

●●

●●●●

●●●

●●●

●●

●

●●●●●

●●●●●●●

●●●●●●

●●●●●●

●●

●●●●

●

●

●●●●●●●●● ●●●●

●●●●●●

●

●

●●

●●

●

●●●●

●●●

●

●

●●

●●

●

●●●●●●●

●

●●●●●●

●

●●●●●●●

●●●

●●●●●●

●

●●●●●●●●●

●●●

●●●

●

●●●●

●●●●●●

●●●●●●

●● ●

●●●●●

●●●

●

●●

●●

●●●●

●●

●●●●●●● ●

●

●

●

●

●●●●

●

●●●●●●●

●●●

●●

●●●●●

●

●●●●●●●

●●●●●

●●●

●●

●●● ●●●●●

●●●●●●●●

●●●

●

●●●

●●●●●●●●●●●

●

●●●●●

●●●●●●●●●

●●●●

●●●

●●

●●●●●●●●●●●●

●●●●●●●●●●

●●●●●

●●

●●●● ●●

●●●

●

●

●

●●

●●●●●

●●●●●●●●●●

●

●●●

●●●

●●●● ●●

●●●●

●●●●

●●

●●●●

●●●

●●●

●●

●●●

●●●●●

●●●●●●

●●

●●●●●●●●●●●●●

●●

●●

●●●●

●

●

●●●●●●

●

●

●●

●●●●●●

●●●●●●

●●●●●

● ●●●

●●●●●

●

●●●●●

●●

●●●●

●

●●●●

●●●●

●●

●

●●●●●●●

●●

●

●

●●●●

●

●●●●●

●●●●●

●●●●●●

●

●

●●●

●●

●●●

●●●●●●●

●●●

●

●

●●●●●●●●

●

●●●●●

●●●

●●

●●●●●●●●●●

●●

●

●●●●

●

●●●●●●●●

●●

●●●●

●●●

●●

●●●

●●●●

●●●●●●●●●●

●●●

●●●●

●●●●●

●●●

●●●●● ●

●

●

●●●●

●

●●●●●●●

●

●

●●

●

●●●●●

●●●

●●

●●

●●

●

●●●

●●

●

●●●●●●●

●●●●

●●●

●●●

●

●

●●●

●

●

●●

●●●●●

●

●●●●●●●

●●●●●●

●●●●●●●

●●●●●●●●

●●

●●●

●●

●

●●●●

●

●●●●

●●

●●●●

●

●●

●

●

●●●●●●

●

●●

●

●

●

●

●

●●

●

●

●●●●●

●

●

●●

● ●●●●

●●

●

●●●

●●●●

●●●

●●

●●●

●●●●●●●●●●

●

●●●

●●●

●●●●

●

●●●●●●

●

●

●●●●●●●

●●●

●●●●●●●

●●●●●

●●●

●●

●

●●●●●●●●●●

●●●●●

●

●●

●●●●

●

●

●●●●●

●●

●●●●

●●●●●●

●●

●●●●

●

●●●●●●●●

●●

●●●●●

●●

●● ●

●

●

●●

●●●●●

●

●

●●●●●●●●●●●

●

●

●

●●●

●●●●●

●●

●

●●●●

●●●

●●●●●●●●●●●

●

●

●●●●

●

●●●●

●●

●●

●●

●●

●●●●●●

●●

●

●●●●●

●●●●

●●●●

●

●●●●●●●●● ●●

●●●

●●

●

●●

●●●●●●●●●

●

●●●

●●●

●●●●

●●●

●●●●

●●●●●

●●●●

●

●●●●

●

●

●

●●●●●●●●●

●●●

●●

●

●

●

●

●●

●●●●●

●

●●●●●

●

●●●

●●●●● ●

●

●●

●●

●●

●●

●●●●

●

●

●●

●●●●

●●

●

●●

●

●●●●●●●●●●●

●●●●●●●

●●●●●

●●

●●

●●●

●●●●

●●●●

●

●

●●●

●

●●●

●

●●

●

●●●●●●

●●●

●●

●●●●●●●●●●

●●●

●●

●●

●●

●●

●●●●

●

●●

●

●●●

●

●●●●

●

●

●●

●●●●●

●●●●●●

●●●●●●

●●● ●

●●●●

●

●●

●●

●

●

●●●

●

●●

●●●●●●

●

●●●●●●

●

●

●

●●●●●●●

●●

●

●

●●

●●●●●●●●●●●●●●●●●●

●

●●

●●●

●●

●●●

●●

●●●

●●

●●●● ●●●●●

●●●●●●●●

●●●

●●●●●●

●●●

●●●●●

●●●●●

● ●●●●●

●●●

●

●

●

●●●●●●

●●

●

●

●●

●●

● ●●●●●

●●

●●

●●●●●●●

●●●●●●

●●●

●●

●●

●●●

● ●●●●●●●

●●●●●●●●●●

●

●●●●●●●●●

●

●●

●●●●●

●●●●●●

●●

●●●

●●●●●●

●●●●●● ●●●●●●

●

●●

●●●●

●●●●●●●●

●

●

●

●●●●●●●

●●

●

●

●●

●●●●

●●●●● ●●●

●●●●●●●

●

●●●●●

●●●●●●●

●●

●●

●●

●●●●

●●●

●

● ●●●●●

●●

●

●

●●●●●●

●●●

●●●●

●

●●●

●●

●●

●●

●●●●●

●●

●●

●

●●●●

●●

●●●●● ●●●●●●●●

●●

●●

●●●●

●

●

●●

●●●

●●●●

●●●●●●

●●●●●

●●●●

●●●●●●●

●

●●●

●●

●

●●●

●

●●●

●●●●●●●●●●●●●●●●●●

●●●

●

●●●●●●●

●●●●

●●●●●●●●●●●●●●●●

●●●

●

●●●●●

●●●●

●●

●

●

●●

●●●●

●

●●

●●●●●●●

●●●

●●●●●●●●

●

●

●●●●●

●

●●

●

●●●

●●

●●●●●●

●

●●●●●

●

●●●

●●●

●●●

●●

●

●●

●

●

●●

●●●●●●●

●

●●●●●●●●

●

●

●

●●●●

●

●

●

●

●●●

●●●●

●●

●

●●●●●●●●

●●●●●●●●

●●

●●●

●●

●

●●●●●

●●●

●●●

●

●●●●

●●●●

●●●●●●

●●●●●

●●●

●

●

●●●

●●●●●●●

●

●●●

●●●●

●●●

●●

●●●●●●●

●

●

●●●●●●●●●●

●●

●●●

●●●●●●●

●●●●

●●

●●

●

●●

●●●●●●●

●●●●

●●

●●●

●●●●●●●

●●●●●

●●●●●●

●

●●

●●●●●●●

●●

●

●

●●●

●●●●●●●

●●●●●

●●●●

●

●

●

●●

●●●●

●●

●

●

●●

●●●

●●

●●●●●●

●●●

●

●●●

●●●●●

●●

●

●●●●●●●●●●

●●●

●●●●●●●●●●●●●

●

●●

●●●●●

●●

●

●●

●

●●●

●

●●●●●●●●

●●

●●●●●●●●

●●●●●●●●●●●●●●●●

●

●●●

●●●●●

●

●●●●●●

●

●

●●

●●●●

●●

●

●●●●●

●●●●

●●●●●

●

●

●●●●

●●

●

●●●

●●

●●

●●

●●

●

●●

●●●●●●

●

●

●●

●●

●●●●

●●●

●

●●●

●●●●●

●

●●

●

●●●●

●

●●

●●●●●

●

●

●●●

●

●

●●●

●●●●●

●●●●●●●●●●●●●

●

●

●●●

●●●

●●

●●●●●

●●●●

●

●●

●●

●●●

●

●

●

●

●●

●●●●●●●●●●

●●●●

●

●●

●●●●●●●

● ●●

●

●●

●

●●

●●●●

●●

●●●●●

●●●

●●●●●●●●●

●●●●●●●●●●

●

●

●

●●

●●●●●●● ●●●●

●●●

●●●●●●

●

●●●●

●●

●●●●●●

●●●●●

●

●●

●

●

●●●●

●●●

● ●●●●●●●●●●●

●●

●●

●●●●●

●

●●

●

●●●●

●●

●●●●●●●

●●

●●●

●●●●

●●

●

●

●

●●●●●

●●●

●●●

●●●●●●●●●●●

●●●●●●

●●●●●●●●

●●

●●

●●

●●●●●●

●●●

●●●

●●

●●●●

●●●●●●●●●●●

●●●●●●●●

●

●●●

●

●

●●●

●●●●●●●●

●●●

●

●●

●●

●●●●●

●●●●

●●●●●

●

●●●●

●●●●●●●●●●●

●●

●●●●

●

●●

●

●●●

●

●

●

●●●●●●

●●

●●●

●

●

●

●

●

●

●●●●●

●

●●

●●●●●●●●●

●●

●

●●●

●●●●●

●●●●●●●●●●

●●

●●●●●

●●●●

●●●●

●

●●●

●●

●●

●●●●●●●●●●●

●●●

●●●●●●●

●●●●●●

●●

●●

●●●●●●●●●

●●●●●

●

●●●●●●

●

●●●●●●●●●●●●

●●●●●

●

●●

●●

●●●●●●●

●●●●●●●●●

●●

●

●●

●

●●

●

●

●●

●

●

●

●●●●●●●●●

●●

●

●●●●

●●●●

●

●●●●

●●

●●●●●●●●●●●●

●●

●●●●●●

●

●●●●●●●●

●●●●

●

●●●●●●

●●

●

●●●●

●

●●

●●●●●●●●

●●●

●●●●

●●

●●

●

●

●

●

●●

●●●

●●●

●

●●●●●●●

●●●●●

●●

●

●

●●●●●●

●●●

●

●

●●●●●

●●

●●

●

●●●●●

●●

●

●●

●

●

●●●

●●●

●●

●●●

●●●●●●●●●

●

●●●●●

●●●●●●●●●●●●●

●●●

●

●●

●●●●●●●●

●

●●●

●●●

●●●●●

●

●●●

●●

●●●●●●

●●●●●●●●

●

●●●●

●●

●●

●

●●

●

●●●●●●●●

●●●

●●●●●●●●●

●●

●●●●●

●●●●

●

●

●●●●●

●●●●●●●

●

●●●

●●●●

●

●●

●

●●

●

●●●●●●●●●●●●

●●●●●

●

●

●

●●●●●●●

●

●●●●

●●●●

●

●●

●

●●●●

●●●●●●

●

●

●

●●

●●●●

●●

●●

●●●●●

●●

●

●

●

●●●●

●●●●●●●

●●●●●

●●●●●●●

●

●●●

●●●

●

●●

●●●

●●●

●●●●●

●

●●●

●

●

●

● ●●

●●●●●●●

●●●●●

●

●

●●●●

●●●●

●

●●●

●

●

●●●●

●

●●● ●

●●●●●●●

●●●●

●●●●●

●

●●

●●

●●●

●

●

●●●●

●

●●

●●

●

●

●

●●

●●

●●●●●

●●

●

●●

●

●●

●●●

●●●●

●●●

●●●●●●●●●●●●●

●

●●●●●●●●●●●●●

●

●●●

●●●●●●●●●●●●●

●

●

●●●

●

●●●●

●●●

●●

●●

●●

●●

●

●

●●●

●

●●●●

●●●●●●

●

●

●●●●●●

●●

●●

●

●

●●●●●●

●●●●●●●

●

●

●●

●●●

●●●

●●●●●●●●●●●●●●

●●●●●

●●●●●●●●

●●●●

●●

●

●●●●●●●●

●

●●

●●●●●●

●

●●

●

●●

●

●

●●

●●●●

●

●

●

●●

●●●●

●●●●

●●●●●

●

●●●

●

●●●●●●●

●●●●

●

●●●●●●●● ●●

●

●

●●●

●●

●●●●●

●●●●●●●

●●

●●●●●●

●●●

●●

●

●

●●●●●●

●

●●●●●●

●●●●●●●●●

●

●●

●

●●●●●

●

●

●●●●●●●

●●●●●

●

●●

●●●

●●●

●

●●

●●

●●

●●●

●●●●

●●●

●●●

●●●

●●●●●

●●

●●

●●

●

●●

●●●

●

●

●

●●●●●●●●

●●●

●

●●●●●●●●●●

●●

●●●●●

●●●●●

●●

●●

●●●

●●●

●

●

●

●

●●

●

●●●●●●●●

●●●

●

●●●

●

●●●●●●●●●●●●●

●

●●

●●●●●

●●

●●●●●●●●●●

●●●

●●●●

●●●●●●

●●

●●●●●●●●●

●

●

●●●

●

●●

●●

●●●●●●● ●●●●●

●

●●●● ●●

●

●●

●

●

●

● ●●●●

●●●

●

●

●●●●

●

●

●

●●

●●●●

●●●

●●●●

●●●●●●●●●●●●●●

●●●

●

●●●

●●

●●

●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●

●●●

●●

●●●●●●

●●●●●●

●●●●●●

●●

● ●● ●

●●

●●●●●

●

●●●●●

●●

●●

●●●●●●●●●●●●●

●●

●

●●

●●●●●●● ●●●

●●●●

●●●

●●●●●

●

●

●●●●●●●●

●

●

●●

●

●●

●●●

●●●

●●●●●●●

●●●●●●●●

●●●●

●●●●

●

●

●●●●●●●●●●

●●●●●●

●●●●●●

●●

●

●

●●

●

● ●●

●●●●●●●●●●●●●●

●●

●●●

●



Extensions

Outcome


● ●

−2 −1 0 1 2

−1.

0−

0.5

0.0

0.5

1.0

θ1

θ 2

●●●●●●●

●●●●

●●●●●

●●

●●●

●

●

●

●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●●●●

●●●●●●●●●●●●●

●●

●●●●●●●●●

●

●●

●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●

●●●●●●●●●

●●●●●●

●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●

●

●●●●●●●●●●

●●●●●●

●●●●

●

●●●●●●●●

●●●

●●●●●●●●●●

●●●●●●

●●●●●●●●●●●●●●

●●●

●

●●

●●●●●●●●●●●●●●●●

●●●●

●●●●●

●●

●●●

●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●

●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●●

●●●●●

●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●●●●

●●●

●●●●●●●●●●●●●●

●●●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●

●●●●●●●●●

●●●●●●

●

●●●●●●●●●●●

●●●●●●●●●●

●●●

●●●●●●●●●●●●●●●●

●●●

●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●

●●●●

●●●●●●

●●●

●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●●●

●●●●●●

●●●●●

●●●●●

●

●●

●●●●●●●●

●

●●●●●●●

●●

●●●●●●●●●●●●●

●●

●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●

●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●

●●

●●●

●

●●●●●●●

●●●●●●●●●●●●●●●

●●●●

●

●●●●

●

●●●●●

●●●●●●●●●●

●●

●●●●

●

●●●●●●

●●●

●●●

●

●●●●●

●●●●●●●●●●●●●●●●

●●●●

●●●●●●

●●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●

●●

●

●

●●●

●●●●●●●●●●●●●

●●●●●●

●●●●●

●●●●●●

●●●

●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●

●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●

●●

●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●

●●●●●●●●●●

●●

●●●●●●

●●

●●●●●●●●●●●

●●

●●●●●

●●●●

●

●●●

●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●

●

●●●●●●●●●●●●●●●

●●●●●●●

●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●

●●●

●●●●●

●●●●●●●●●●●●●●

●●

●●●●

●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●

●●●●●●●●●●●●●●●

●●●

●

●

● ●●●●●●●●

●●●●●●●

●●●●●●●●●●●

●●●●●

●●●●●●●●

●●

●

●●●●●

●●●●●●

●●●●●●●●●●

●●●●●●●●●

●●●●●●●●

●●●●●●●

●●●

●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●

●●●

●

●●

●●●

●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●

●●●●●●

●●●

●●●●●●●

●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●● ●●●●

●●●●●●●●●●

●●●●●●●

●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●●●

●●●●●●●●●●●

●●●●●

●●●●●●●●

●●●

●●●●●●●

●●

●●●●

●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●● ●●●

●●●●●

●●●●●●●●●●●

●

●●●●●●●

●●●●●●●●●●●●●

●●●

●●●

●●●●●●●●

●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●

●●

●●

●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●

●●●●

●

●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●

●

●●●●●●

●●●●●

●●

●●●

●

●●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●

●●●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●

●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●

●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●

●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●

●

●●●●

●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●

●

●●●●●●●

●●●●●●●●●●●●

●●●

●●●●●●●●●●●●

●●●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●

●

●●●●●

●●●

●●●●●

●●●●●●

●●●●●●●●●●●●●

●●●●●●●

●●

●●●●●●●●●●●

●●●●●

●●●●●●●●

●●●●●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●

●

●●●

●●●●●●●●●●

●

●●

●●●●●●●●●●

●●

●●●●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●

●●●

●

●●●●●

●●●●

●●●●

●●

●

●

●

●●●●●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●●

●●●●●●●●●

●●

●●

●●●●

●

●●●

●●●●●●●

●●●●●

●●●

●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●●●●●●●

●●●●●●●●●●●●●

●●●●

●●

●●●

●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●

●●●●●●●

●●●

●●

●●●●

●●●

●●●●●●●●●●●●●

●●●●●●●

●●

●●●●●●

●●●●●●●

●●●●●●●●●●●●● ●

●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●

●

●●

●●●●●●●●

●●●●●●●

●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●

●

●●●●●●●●●●●●●●●●●●

●●●

●●●

●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●

●●

●●●●●●

●●●

●

●●

●●●●

●●●●●

●●●

●●●●●●●●●●●

●●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●

●

●●●●●

●●●●

●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●

●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●

●●●●●●●●●●●●●●

●●●●

●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●

●●●●●●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●

●●●●●

●●●●

●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●

●●●

●●●

●●●

●●●●●●●●●

●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●●●

●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●

●●●

●

●●●●●●●●●●●●●●

●●

●●●●●●●

●●●●●

●●●●●●●●●●

●●

●●●●●●

●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●

●●●●●●●●●●

●●●●●●●●●●●●

●●●●

●

●●●●●●●

●

●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●

●●

●●

●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●

●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●

●●●●●●

●●●●●●

●●●

●

●●●●●●●

●●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●

●●●●●●●●●●

●●●

●●

●●●●●●●●●●●●●

●●●●●●●

●●●●●●

●●

●

●●●

●●●●●●●

●●●●●●●●●

●●●●

●●●

●●●●●●●●●

●●●● ●

●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●

●●●●●●

●●●●●

●

●●●●●●●●

●●●●●●●●

●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●

●●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●

●●●●●●●●

●●●●●●●●

●●●●●●●

●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●

●●

●●

●●●●●●●●●●●●●●

●●●●

●●

●

●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●

●

●●●

●●●●●●

●●●●●

●●

●●●●●●●

●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●

●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●

● ●●

●●

●

●●●

●

●

●●●●●●●

●●

●●●●●●●●●

●●●●

●●●●●

●●●●●

●●●●●●●●●●

●

●●●●

●●●●●●●●●●●●●●●●

●●●●

●●●●●

●●●●●

●●●●●●●●●●●●

●●●●●●

●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●●●●●●●●●

●

●●●●

●●●●

●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●

●●●●●

●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●

●●●

●●●●●●●●●●●

●●●●●

●●●●●

●●●

●●●●●●●●●●●●

●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●●●●●

●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●●●●●●●

●

●●●●●

●●●●●●

●●●●●●●

●●●●

●

●

●●●●

●●●●●●●●

●●●●●●●●

●●●●●●●●

●

●●●●●●●●●

●●●●●●●●●

●●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●●

●●

●●●●●●●●●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●●●

●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●

●●●●●●●●●●●●●●

●●●

●● ●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●

●●●●●●●●

●●●●●

●●●●●●●

●●●●

●●●●●

●●

●●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●

●●●●●●●

●●●

●●●●●●●●●

●

●●●●

●●

●●●●●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●

●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●

●●

●●●

●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●

●

●●●●●

●●●

●●●

●●●●●

●●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●

●●●●●

●●●●●

●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●

●

●

●

●●●●●

●●●●●

●●●●

●●●●● ●●●●●●●●●●●●

●●●●●●●

●

●

●●●●●●●

●●●●●●●●

●●●●●●

●●●●●●

●●●●●

●●●●●●●

●●●●●●●

●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●

●●●●

●

●●●●●●●●●●

●●●●●●●●●●●●●

●●●

●●

●

●●●●●

●

●●●●●●●●●●●●●●

●●●●

●●●

●●●●●●●

●●●●●●●

●●

●●●●●●●●●●●●●●

●●●●●●●●●●

●

●●●●●

●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

●

●●

●●●●●●●●●

●●●●●●●●●●●●●●●

●●

●●●●●●●●

●●●●●●

●

●●●●

●●●●●●

●●●●

●●

●●●

●

●

●●●●●●●

●●●●●●●●

●●●●●

●●●●●

●●●●●●

●●●●●

●●●●

●

●●●● ●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●

●

●●●

●●●●●●●●●●●●●●●●●

●

●

●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●

●●●

●●●●

●●●●●●●●●●●●●●●●

●

●●●●●●●

●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●●●●

●●●●●

●●●●●●●

●●●●●●

●●●●●

●

●●●●●●

●

●●●●●●●●●●●

●

●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●●●

●●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●

●●●●●●●●

●●●

●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●●●●●●

●●●

●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●

●●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●●●

●●●●●●●●●

●●●●●●●

●●●●

●

●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●

●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●

●●

●●●●●●●●●●●●

●●●●●●●●●●

●

●●

●●●●●●●●●

●

●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●

●●●

●●●●●●●●●●

●●●

●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●

●

●

●

●●

●●●●●●

●●

●

●●●●●●●●●●●●●

●●●

●●●●●●●●●●●●●

●●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●

●●●●●●

●●●●●●●●●●●●●●●●●

●

●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●

●

●●●●●●

●●●●●●

●

●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●

●●●●●●●●●●

●●●●●

●●●●●

●●●

●●●●●●●

●



Extensions

Outcome


● ●

−2 −1 0 1 2

−1.

0−

0.5

0.0

0.5

1.0

θ1

θ 2

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●


The Gibbs Sampler

The Gibbs Sampler

skip to population Monte Carlo

The Gibbs SamplerGeneral PrinciplesCompletionConvergenceThe Hammersley-Clifford theorem


The Gibbs Sampler

General Principles

General Principles

A very specific simulation algorithm based on the targetdistribution f :

1. Uses the conditional densities f1, . . . , fp from f

2. Start with the random variable X = (X1, . . . , Xp)3. Simulate from the conditional densities,

Xi|x1, x2, . . . , xi−1, xi+1, . . . , xp

∼ fi(xi|x1, x2, . . . , xi−1, xi+1, . . . , xp)

for i = 1, 2, . . . , p.


The Gibbs Sampler

General Principles

General Principles



2. Start with the random variable X = (X1, . . . , Xp)

3. Simulate from the conditional densities,

Xi|x1, x2, . . . , xi−1, xi+1, . . . , xp

∼ fi(xi|x1, x2, . . . , xi−1, xi+1, . . . , xp)

for i = 1, 2, . . . , p.


The Gibbs Sampler

General Principles

General Principles



2. Start with the random variable X = (X1, . . . , Xp)3. Simulate from the conditional densities,

Xi|x1, x2, . . . , xi−1, xi+1, . . . , xp

∼ fi(xi|x1, x2, . . . , xi−1, xi+1, . . . , xp)

for i = 1, 2, . . . , p.


The Gibbs Sampler

General Principles

Algorithm (Gibbs sampler)

Given x(t) = (x(t)1 , . . . , x

(t)p ), generate

1. X(t+1)1 ∼ f1(x1|x(t)

2 , . . . , x(t)p );

2. X(t+1)2 ∼ f2(x2|x(t+1)

1 , x(t)3 , . . . , x

(t)p ),

. . .

p. X(t+1)p ∼ fp(xp|x(t+1)

1 , . . . , x(t+1)p−1 )

X(t+1) → X ∼ f


The Gibbs Sampler

General Principles

Properties

The full conditionals densities f1, . . . , fp are the only densities usedfor simulation. Thus, even in a high dimensional problem, all ofthe simulations may be univariate

The Gibbs sampler is not reversible with respect to f . However,each of its p components is. Besides, it can be turned into areversible sampler, either using the Random Scan Gibbs sampler

see section or running instead the (double) sequence

f1 · · · fp−1fpfp−1 · · · f1


The Gibbs Sampler

General Principles

Properties

The full conditionals densities f1, . . . , fp are the only densities usedfor simulation. Thus, even in a high dimensional problem, all ofthe simulations may be univariateThe Gibbs sampler is not reversible with respect to f . However,each of its p components is. Besides, it can be turned into areversible sampler, either using the Random Scan Gibbs sampler

see section or running instead the (double) sequence

f1 · · · fp−1fpfp−1 · · · f1


The Gibbs Sampler

General Principles

A Very Simple Example: Independent N (µ, σ2)Observations

When Y1, . . . , Yniid∼ N (y|µ, σ2) with both µ and σ unknown, the

posterior in (µ, σ2) is conjugate outside a standard familly

But...

µ|Y 0:n, σ2 ∼ N

(µ∣∣∣ 1n

∑ni=1 Yi,

σ2

n )

σ2|Y 1:n, µ ∼ IG(σ2∣∣n

2 − 1, 12

∑ni=1(Yi − µ)2

)assuming constant (improper) priors on both µ and σ2

I Hence we may use the Gibbs sampler for simulating from theposterior of (µ, σ2)


The Gibbs Sampler

General Principles

R Gibbs Sampler for Gaussian posterior

n = length(Y);S = sum(Y);mu = S/n;for (i in 1:500)

S2 = sum((Y-mu)^2);sigma2 = 1/rgamma(1,n/2-1,S2/2);mu = S/n + sqrt(sigma2/n)*rnorm(1);


The Gibbs Sampler

General Principles

Example of results with n = 10 observations from theN (0, 1) distribution

Number of Iterations 1

, 2, 3, 4, 5, 10, 25, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2

, 3, 4, 5, 10, 25, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3

, 4, 5, 10, 25, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4

, 5, 10, 25, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4, 5

, 10, 25, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4, 5, 10

, 25, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4, 5, 10, 25

, 50, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50

, 100, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50, 100

, 500


The Gibbs Sampler

General Principles


Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50, 100, 500


The Gibbs Sampler

General Principles

Limitations of the Gibbs sampler

Formally, a special case of a sequence of 1-D M-H kernels, all withacceptance rate uniformly equal to 1.The Gibbs sampler

1. limits the choice of instrumental distributions

2. requires some knowledge of f

3. is, by construction, multidimensional

4. does not apply to problems where the number of parametersvaries as the resulting chain is not irreducible.


The Gibbs Sampler

Completion

Latent variables are back

The Gibbs sampler can be generalized in much wider generalityA density g is a completion of f if∫

Zg(x, z) dz = f(x)

Note

The variable z may be meaningless for the problem


The Gibbs Sampler

Completion

Purpose

g should have full conditionals that are easy to simulate for aGibbs sampler to be implemented with g rather than f

For p > 1, write y = (x, z) and denote the conditional densities ofg(y) = g(y1, . . . , yp) by

Y1|y2, . . . , yp ∼ g1(y1|y2, . . . , yp),Y2|y1, y3, . . . , yp ∼ g2(y2|y1, y3, . . . , yp),

. . . ,

Yp|y1, . . . , yp−1 ∼ gp(yp|y1, . . . , yp−1).


The Gibbs Sampler

Completion

The move from Y (t) to Y (t+1) is defined as follows:

Algorithm (Completion Gibbs sampler)

Given (y(t)1 , . . . , y

(t)p ), simulate

1. Y(t+1)

1 ∼ g1(y1|y(t)2 , . . . , y

(t)p ),

2. Y(t+1)

2 ∼ g2(y2|y(t+1)1 , y

(t)3 , . . . , y

(t)p ),

. . .

p. Y(t+1)p ∼ gp(yp|y(t+1)

1 , . . . , y(t+1)p−1 ).


The Gibbs Sampler

Completion

Example (Mixtures all over again)

Hierarchical missing data structure:If

X1, . . . , Xn ∼k∑i=1

pif(x|θi),

then

X|Z ∼ f(x|θZ), Z ∼ p1I(z = 1) + . . .+ pkI(z = k),

Z is the component indicator associated with observation x


The Gibbs Sampler

Completion

Example (Mixtures (2))

Conditionally on (Z1, . . . , Zn) = (z1, . . . , zn) :

π(p1, . . . , pk, θ1, . . . , θk|x1, . . . , xn, z1, . . . , zn)∝ pα1+n1−1

1 . . . pαk+nk−1k

×π(θ1|y1 + n1x1, λ1 + n1) . . . π(θk|yk + nkxk, λk + nk),

withni =

∑j

I(zj = i) and xi =∑j; zj=i

xj/ni.


The Gibbs Sampler

Completion

Algorithm (Mixture Gibbs sampler)

1. Simulate

θi ∼ π(θi|yi + nixi, λi + ni) (i = 1, . . . , k)(p1, . . . , pk) ∼ D(α1 + n1, . . . , αk + nk)

2. Simulate (j = 1, . . . , n)

Zj |xj , p1, . . . , pk, θ1, . . . , θk ∼k∑i=1

pijI(zj = i)

with (i = 1, . . . , k)pij ∝ pif(xj |θi)

and update ni and xi (i = 1, . . . , k).


The Gibbs Sampler

Completion

A wee problem

−1 0 1 2 3 4

−1

01

23

4

µ1

µ2

Gibbs started at random

Gibbs stuck at the wrong mode

−1 0 1 2 3

−1

01

23

µ1

µ2


The Gibbs Sampler

Completion

Slice sampler as generic Gibbs

If f(θ) can be written as a product

k∏i=1

fi(θ),

it can be completed as

k∏i=1

I0≤ωi≤fi(θ),

leading to the following Gibbs algorithm:


The Gibbs Sampler

Completion

Algorithm (Slice sampler)

Simulate

1. ω(t+1)1 ∼ U[0,f1(θ(t))];

. . .

k. ω(t+1)k ∼ U[0,fk(θ(t))];

k+1. θ(t+1) ∼ UA(t+1) , with

A(t+1) = {y; fi(y) ≥ ω(t+1)i , i = 1, . . . , k}.


The Gibbs Sampler

Completion

Example of results with a truncated N (−3, 1) distribution

0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2

, 3, 4, 5, 10, 50, 100


The Gibbs Sampler

Completion


0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2, 3

, 4, 5, 10, 50, 100


The Gibbs Sampler

Completion


0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2, 3, 4

, 5, 10, 50, 100


The Gibbs Sampler

Completion


0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2, 3, 4, 5

, 10, 50, 100


The Gibbs Sampler

Completion


0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2, 3, 4, 5, 10

, 50, 100


The Gibbs Sampler

Completion


0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2, 3, 4, 5, 10, 50

, 100


The Gibbs Sampler

Completion


0.0 0.2 0.4 0.6 0.8 1.0

0.00

00.

002

0.00

40.

006

0.00

80.

010

x

y

Number of Iterations 2, 3, 4, 5, 10, 50, 100


The Gibbs Sampler

Completion

Good slices

The slice sampler usually enjoys good theoretical properties (likegeometric ergodicity and even uniform ergodicity under bounded fand bounded X ).As k increases, the determination of the set A(t+1) may getincreasingly complex.


The Gibbs Sampler

Completion

Example (Stochastic volatility core distribution)

Difficult part of the stochastic volatility model

π(x) ∝ exp−{σ2(x− µ)2 + β2 exp(−x)y2 + x

}/2 ,

simplified in exp−{x2 + α exp(−x)

}

Slice sampling means simulation from a uniform distribution on

A ={x; exp−

{x2 + α exp(−x)

}/2 ≥ u

}=

{x;x2 + α exp(−x) ≤ ω

}if we set ω = −2 log u.Note Inversion of x2 + α exp(−x) = ω needs to be done bytrial-and-error.


The Gibbs Sampler

Completion

Example (Stochastic volatility core distribution)

Difficult part of the stochastic volatility model

π(x) ∝ exp−{σ2(x− µ)2 + β2 exp(−x)y2 + x

}/2 ,

simplified in exp−{x2 + α exp(−x)

}Slice sampling means simulation from a uniform distribution on

A ={x; exp−

{x2 + α exp(−x)

}/2 ≥ u

}=

{x;x2 + α exp(−x) ≤ ω

}if we set ω = −2 log u.Note Inversion of x2 + α exp(−x) = ω needs to be done bytrial-and-error.


The Gibbs Sampler

Completion

0 10 20 30 40 50 60 70 80 90 100−0.1

−0.05

0

0.05

0.1

Lag

Cor

rela

tion

−1 −0.5 0 0.5 1 1.5 2 2.5 3 3.50

0.2

0.4

0.6

0.8

1D

ensi

ty

Histogram of a Markov chain produced by a slice samplerand target distribution in overlay.


The Gibbs Sampler

Convergence

Properties of the Gibbs sampler

Theorem (Convergence)

For(Y1, Y2, · · · , Yp) ∼ g(y1, . . . , yp),

if either[Positivity condition]

(i) g(i)(yi) > 0 for every i = 1, · · · , p, implies thatg(y1, . . . , yp) > 0, where g(i) denotes the marginal distributionof Yi, or

(ii) the transition kernel is absolutely continuous with respect to g,

then the chain is irreducible and positive Harris recurrent.


The Gibbs Sampler

Convergence

Properties of the Gibbs sampler (2)

Consequences

(i) If∫h(y)g(y)dy <∞, then

limnT→∞

1T

T∑t=1

h1(Y (t)) =∫h(y)g(y)dy a.e. g.

(ii) If, in addition, (Y (t)) is aperiodic, then

limn→∞

∥∥∥∥∫ Kn(y, ·)µ(dx)− f∥∥∥∥TV

= 0

for every initial distribution µ.


The Gibbs Sampler

Convergence

Slice sampler

fast on that slice

For convergence, the properties of Xt and of f(Xt) are identical

Theorem (Uniform ergodicity)

If f is bounded and suppf is bounded, the simple slice sampler isuniformly ergodic.

[Mira & Tierney, 1997]


The Gibbs Sampler

Convergence

A small set for a slice sampler

no slice detail

For ε? > ε?,C = {x ∈ X ; ε? < f(x) < ε?}

is a small set:Pr(x, ·) ≥ ε?

ε?µ(·)

where

µ(A) =1ε?

∫ ε?

0

λ(A ∩ L(ε))λ(L(ε))

dε

if L(ε) = {x ∈ X ; f(x) > ε}‘[Roberts & Rosenthal, 1998]


The Gibbs Sampler

Convergence

Slice sampler: drift

Under differentiability and monotonicity conditions, the slicesampler also verifies a drift condition with V (x) = f(x)−β, isgeometrically ergodic, and there even exist explicit bounds on thetotal variation distance

[Roberts & Rosenthal, 1998]

Example (Exponential Exp(1))

For n > 23,

||Kn(x, ·)− f(·)||TV ≤ .054865 (0.985015)n (n− 15.7043)


The Gibbs Sampler

Convergence

Slice sampler: convergence

no more slice detail

Theorem

For any density such that

ε∂

∂ελ ({x ∈ X ; f(x) > ε}) is non-increasing

then||K523(x, ·)− f(·)||TV ≤ .0095

[Roberts & Rosenthal, 1998]


The Gibbs Sampler

Convergence

A poor slice sampler

Example

Consider

f(x) = exp {−||x||} x ∈ Rd

Slice sampler equivalent toone-dimensional slice sampler on

π(z) = zd−1 e−z z > 0

or on

π(u) = e−u1/d

u > 0

Poor performances when d large(heavy tails)

0 200 400 600 800 1000

-2-1

01

1 dimensional run

co

rre

latio

n

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

1 dimensional acf

0 200 400 600 800 1000

10

15

20

25

30

10 dimensional run

co

rre

latio

n

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

10 dimensional acf

0 200 400 600 800 1000

02

04

06

0

20 dimensional run

co

rre

latio

n

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

20 dimensional acf

0 200 400 600 800 1000

01

00

20

03

00

40

0

100 dimensional run

co

rre

latio

n

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

100 dimensional acf

Sample runs of log(u) andACFs for log(u) (Roberts& Rosenthal, 1999)


The Gibbs Sampler

The Hammersley-Clifford theorem

Hammersley-Clifford theorem

An illustration that conditionals determine the joint distribution

Theorem

If the joint density g(y1, y2) have conditional distributionsg1(y1|y2) and g2(y2|y1), then

g(y1, y2) =g2(y2|y1)∫

g2(v|y1)/g1(y1|v) dv.

[Hammersley & Clifford, circa 1970]


The Gibbs Sampler

The Hammersley-Clifford theorem

General HC decomposition

Under the positivity condition, the joint distribution g satisfies

g(y1, . . . , yp) ∝p∏j=1

g`j (y`j |y`1 , . . . , y`j−1, y′`j+1

, . . . , y′`p)

g`j (y′`j|y`1 , . . . , y`j−1

, y′`j+1, . . . , y′`p)

for every permutation ` on {1, 2, . . . , p} and every y′ ∈ Y .



Sequential importance sampling



The Gibbs Sampler






Importance sampling (revisited)

basic importance

Approximation of integrals

I =∫h(x)π(x)dx

by unbiased estimators

I =1n

n∑i=1

%ih(xi)

when

x1, . . . , xniid∼ q(x) and %i

def=π(xi)q(xi)



Iterated importance sampling

As in Markov Chain Monte Carlo (MCMC) algorithms,introduction of a temporal dimension :

x(t)i ∼ qt(x|x

(t−1)i ) i = 1, . . . , n, t = 1, . . .

and

It =1n

n∑i=1

%(t)i h(x(t)

i )

is still unbiased for

%(t)i =

πt(x(t)i )

qt(x(t)i |x

(t−1)i )

, i = 1, . . . , n



Fundamental importance equality

Preservation of unbiasedness

E[h(X(t))

π(X(t))qt(X(t)|X(t−1))

]

=∫h(x)

π(x)qt(x|y)

qt(x|y) g(y) dx dy

=∫h(x)π(x) dx

for any distribution g on X(t−1)



Sequential variance decomposition

Furthermore,

var(It

)=

1n2

n∑i=1

var(%

(t)i h(x(t)

i )),

if var(%

(t)i

)exists, because the x

(t)i ’s are conditionally uncorrelated

Note

This decomposition is still valid for correlated [in i] x(t)i ’s when

incorporating weights %(t)i



Simulation of a population

The importance distribution of the sample (a.k.a. particles) x(t)

qt(x(t)|x(t−1))

can depend on the previous sample x(t−1) in any possible way aslong as marginal distributions

qit(x) =∫qt(x(t)) dx(t)

−i

can be expressed to build importance weights

%it =π(x(t)

i )

qit(x(t)i )



Special case of the product proposal

If

qt(x(t)|x(t−1)) =n∏i=1

qit(x(t)i |x

(t−1))

[Independent proposals]then

var(It

)=

1n2

n∑i=1

var(%

(t)i h(x(t)

i )),



Validation

skip validation

E[%

(t)i h(X(t)

i ) %(t)j h(X(t)

j )]

=∫h(xi)

π(xi)qit(xi|x(t−1))

π(xj)qjt(xj |x(t−1))

h(xj)

qit(xi|x(t−1)) qjt(xj |x(t−1)) dxi dxj g(x(t−1))dx(t−1)

= Eπ [h(X)]2

whatever the distribution g on x(t−1)



Self-normalised version

In general, π is unscaled and the weight

%(t)i ∝

π(x(t)i )

qit(x(t)i )

, i = 1, . . . , n ,

is scaled so that ∑i

%(t)i = 1



Self-normalised version properties

I Loss of the unbiasedness property and the variancedecomposition

I Normalising constant can be estimated by

$t =1tn

t∑τ=1

n∑i=1

π(x(τ)i )

qiτ (x(τ)i )

I Variance decomposition (approximately) recovered if $t−1 isused instead



Sampling importance resampling

Importance sampling from g can also produce samples from thetarget π

[Rubin, 1987]

Theorem (Bootstraped importance sampling)

If a sample (x?i )1≤i≤m is derived from the weighted sample(xi, %i)1≤i≤n by multinomial sampling with weights %i, then

x?i ∼ π(x)

Note

Obviously, the x?i ’s are not iid



Iterated sampling importance resampling

This principle can be extended to iterated importance sampling:After each iteration, resampling produces a sample from π

[Again, not iid!]

Incentive

Use previous sample(s) to learn about π and q



Generic Population Monte Carlo

Algorithm (Population Monte Carlo Algorithm)

For t = 1, . . . , TFor i = 1, . . . , n,

1. Select the generating distribution qit(·)2. Generate x

(t)i ∼ qit(x)

3. Compute %(t)i = π(x(t)

i )/qit(x(t)i )

Normalise the %(t)i ’s into %

(t)i ’s

Generate Ji,t ∼M((%(t)i )1≤i≤N ) and set xi,t = x

(t)Ji,t



D-kernels in competition

A general adaptive construction:

Construct qi,t as a mixture of D different transition kernels

depending on x(t−1)i

qi,t =D∑`=1

pt,`K`(x(t−1)i , x),

D∑`=1

pt,` = 1 ,

and adapt the weights pt,`.

Darwinian example

Take pt,` proportional to the survival rate of the points

(a.k.a. particles) x(t)i generated from K`



Implementation

Algorithm (D-kernel PMC)

For t = 1, . . . , Tgenerate (Ki,t)1≤i≤N ∼M ((pt,k)1≤k≤D)for 1 ≤ i ≤ N , generate

xi,t ∼ KKi,t(x)

compute and renormalize the importance weights ωi,t

generate (Ji,t)1≤i≤N ∼M ((ωi,t)1≤i≤N )

take xi,t = xJi,t,t and pt+1,d =∑N

i=1 ωi,tId(Ki,t)



Links with particle filters

I Sequential setting where π = πt changes with t: PopulationMonte Carlo also adapts to this case

I Can be traced back all the way to Hammersley and Morton(1954) and the self-avoiding random walk problem

I Gilks and Berzuini (2001) produce iterated samples with (SIR)resampling steps, and add an MCMC step: this step must usea πt invariant kernel

I Chopin (2001) uses iterated importance sampling to handlelarge datasets: this is a special case of PMC where the qit’sare the posterior distributions associated with a portion kt ofthe observed dataset



Links with particle filters (2)

I Rubinstein and Kroese’s (2004) cross-entropy method isparameterised importance sampling targeted at rare events

I Stavropoulos and Titterington’s (1999) smooth bootstrap andWarnes’ (2001) kernel coupler use nonparametric kernels onthe previous importance sample to build an improvedproposal: this is a special case of PMC

I West (1992) mixture approximation is a precursor of smoothbootstrap

I Mengersen and Robert (2002) “pinball sampler” is an MCMCattempt at population sampling

I Del Moral, Doucet and Jasra (2006, JRSS B) sequentialMonte Carlo samplers also relates to PMC, with a Markoviandependence on the past sample x(t) but (limited) stationarityconstraints



Things can go wrong

Unexpected behaviour of the mixture weights when the number ofparticles increases

N∑i=1

ωi,tIKi,t=d−→P1D

Conclusion

At each iteration, every weight converges to 1/D:the algorithm fails to learn from experience!!



Saved by Rao-Blackwell!!

Modification: Rao-Blackwellisation (=conditioning)

Use the whole mixture in the importance weight:

ωi,t = π(xi,t)D∑d=1

pt,dKd(xi,t−1, xi,t)

instead of

ωi,t =π(xi,t)

KKi,t(xi,t−1, xi,t)



Adapted algorithm

Algorithm (Rao-Blackwellised D-kernel PMC)

At time t (t = 1, . . . , T ),

Generate(Ki,t)1≤i≤N

iid∼ M((pt,d)1≤d≤D);

Generate(xi,t)1≤i≤N

ind∼ KKi,t(xi,t−1, x)

and set ωi,t = π(xi,t)/∑D

d=1 pt,dKd(xi,t−1, xi,t);

Generate(Ji,t)1≤i≤N

iid∼ M((ωi,t)1≤i≤N )

and set xi,t = xJi,t,t and pt+1,d =∑N

i=1 ωi,tpt,d.




Theorem (LLN)

Under regularity assumptions, for h ∈ L1Π and for every t ≥ 1,

1N

N∑k=1

ωi,th(xi,t)N→∞−→P Π(h)

andpt,d

N→∞−→P αtd

The limiting coefficients (αtd)1≤d≤D are defined recursively as

αtd = αt−1d

∫ (Kd(x, x′)∑D

j=1 αt−1j Kj(x, x′)

)Π⊗Π(dx, dx′).



Recursion on the weights

Set F as

F (α) =

(αd

∫ [Kd(x, x′)∑D

j=1 αjKj(x, x′)

]Π⊗Π(dx, dx′)

)1≤d≤D

on the simplex

S =

{α = (α1, . . . , αD); ∀d ∈ {1, . . . , D}, αd ≥ 0 and

D∑d=1

αd = 1

}.

and define the sequence

αt+1 = F (αt)



Kullback divergence

Definition (Kullback divergence)

For α ∈ S,

KL(α) =∫ [

log

(π(x)π(x′)

π(x)∑D

d=1 αdKd(x, x′)

)]Π⊗Π(dx, dx′).

Kullback divergence between Π and the mixture.

Goal: Obtain the mixture closest to Π, i.e., that minimises KL(α)



Connection with RBDPMCA ??

Theorem

Under the assumption

∀d ∈ {1, . . . , D},−∞ <

∫log(Kd(x, x′))Π⊗Π(dx, dx′) <∞

for every α ∈ SD,

KL(F (α)) ≤ KL(α).

Conclusion

The Kullback divergence decreases at every iteration of RBDPMCA



An integrated EM interpretation

skip interpretation

We have

αmin = arg minα∈S

KL(α) = arg maxα∈S

∫log pα(x)Π⊗Π(dx)

= arg maxα∈S

∫log∫pα(x,K)dK Π⊗Π(dx)

for x = (x, x′) and K ∼M((αd)1≤d≤D). Then αt+1 = F (αt)means

αt+1 = arg maxα

∫∫Eαt(log pα(X,K)|X = x)Π⊗Π(dx)

andlimt→∞

αt = αmin



Illustration

Example (A toy example)

Take the target

1/4N (−1, 0.3)(x) + 1/4N (0, 1)(x) + 1/2N (3, 2)(x)

and use 3 proposals: N (−1, 0.3), N (0, 1) and N (3, 2)[Surprise!!!]

Then

1 0.0500000 0.05000000 0.90000002 0.2605712 0.09970292 0.63972596 0.2740816 0.19160178 0.534316610 0.2989651 0.19200904 0.509025916 0.2651511 0.24129039 0.4935585

Weight evolution



Target and mixture evolution



c© Learning scheme

The efficiency of the SNIS approximation depends on the choice ofQ, ranging from optimal

q(x) ∝ |h(x)−Π(h)|π(x)

to uselessvar ΠSNIS

Q,N (h) = +∞

Example (PMC=adaptive importance sampling)

Population Monte Carlo is producing a sequence of proposals Qtaiming at improving efficiency

Kull(π, qt) ≤ Kull(π, qt−1) or var ΠSNISQt,∞ (h) ≤ var ΠSNIS

Qt−1,∞(h)

[Cappe, Douc, Guillin, Marin, Robert, 04, 07a, 07b, 08]






The Gibbs Sampler


Approximate Bayesian computationABC basicsAlphabet soupCalibration of ABC




ABC basics

Untractable likelihoods

Cases when the likelihood function f(y|θ) is unavailable and whenthe completion step

f(y|θ) =∫

Zf(y, z|θ) dz

is impossible or too costly because of the dimension of zc© MCMC cannot be implemented!



ABC basics

Illustrations

Example

Stochastic volatility model: fort = 1, . . . , T,

yt = exp(zt)εt , zt = a+bzt−1+σηt ,

T very large makes it difficult toinclude z within the simulatedparameters

0 200 400 600 800 1000

−0.

4−

0.2

0.0

0.2

0.4

t

Highest weight trajectories



ABC basics

Illustrations

Example

Potts model: if y takes values on a grid Y of size kn and

f(y|θ) ∝ exp

{θ∑l∼i

Iyl=yi

}

where l∼i denotes a neighbourhood relation, n moderately largeprohibits the computation of the normalising constant



ABC basics

Illustrations

Example

Inference on CMB: in cosmology, study of the Cosmic MicrowaveBackground via likelihoods immensely slow to computate (e.gWMAP, Plank), because of numerically costly spectral transforms[Data is a Fortran program]

[Kilbinger et al., 2010, MNRAS]



ABC basics

Illustrations

Example

Phylogenetic tree: in populationgenetics, reconstitution of a commonancestor from a sample of genes viaa phylogenetic tree that is close toimpossible to integrate out[100 processor days with 4parameters]

[Cornuet et al., 2009, Bioinformatics]



ABC basics

The ABC method

Bayesian setting: target is π(θ)f(x|θ)

When likelihood f(x|θ) not in closed form, likelihood-free rejectiontechnique:

ABC algorithm

For an observation y ∼ f(y|θ), under the prior π(θ), keep jointlysimulating

θ′ ∼ π(θ) , z ∼ f(z|θ′) ,

until the auxiliary variable z is equal to the observed value, z = y.

[Tavare et al., 1997]



ABC basics

The ABC method

Bayesian setting: target is π(θ)f(x|θ)When likelihood f(x|θ) not in closed form, likelihood-free rejectiontechnique:

ABC algorithm

For an observation y ∼ f(y|θ), under the prior π(θ), keep jointlysimulating

θ′ ∼ π(θ) , z ∼ f(z|θ′) ,

until the auxiliary variable z is equal to the observed value, z = y.

[Tavare et al., 1997]



ABC basics

Why does it work?!

The proof is trivial:

f(θi) ∝∑z∈D

π(θi)f(z|θi)Iy(z)

∝ π(θi)f(y|θi)= π(θi|y) .

[Accept–Reject 101]



ABC basics

Earlier occurrence

‘Bayesian statistics and Monte Carlo methods are ideallysuited to the task of passing many models over onedataset’

[Don Rubin, Annals of Statistics, 1984]

Note Rubin (1984) does not promote this algorithm forlikelihood-free simulation but frequentist intuition on posteriordistributions: parameters from posteriors are more likely to bethose that could have generated the data.



ABC basics

A as approximative

When y is a continuous random variable, equality z = y is replacedwith a tolerance condition,

%(y, z) ≤ ε

where % is a distance

Output distributed from

π(θ)Pθ{%(y, z) < ε} ∝ π(θ|%(y, z) < ε)



ABC basics

A as approximative

When y is a continuous random variable, equality z = y is replacedwith a tolerance condition,

%(y, z) ≤ ε

where % is a distanceOutput distributed from

π(θ)Pθ{%(y, z) < ε} ∝ π(θ|%(y, z) < ε)



ABC basics

ABC algorithm

Algorithm 2 Likelihood-free rejection sampler 2

for i = 1 to N dorepeat

generate θ′ from the prior distribution π(·)generate z from the likelihood f(·|θ′)

until ρ{η(z), η(y)} ≤ εset θi = θ′

end for

where η(y) defines a (not necessarily sufficient) statistic



ABC basics

Output

The likelihood-free algorithm samples from the marginal in z of:

πε(θ, z|y) =π(θ)f(z|θ)IAε,y(z)∫

Aε,y×Θ π(θ)f(z|θ)dzdθ,

wheere Aε,y = {z ∈ D|ρ(η(z), η(y)) < ε}.

The idea behind ABC is that the summary statistics coupled with asmall tolerance should provide a good approximation of theposterior distribution:

πε(θ|y) =∫πε(θ, z|y)dz ≈ π(θ|y) .



ABC basics

MA example

Back to the MA(q) model

xt = εt +q∑i=1

ϑiεt−i

Simple prior: uniform over the inverse [real and complex] roots in

Q(u) = 1−q∑i=1

ϑiui

under the identifiability conditions



ABC basics

MA example

Back to the MA(q) model

xt = εt +q∑i=1

ϑiεt−i

Simple prior: uniform prior over the identifiability zone, e.g.triangle for MA(2)



ABC basics

MA example (2)

ABC algorithm thus made of

1. picking a new value (ϑ1, ϑ2) in the triangle

2. generating an iid sequence (εt)−q<t≤T3. producing a simulated series (x′t)1≤t≤T

Distance: basic distance between the series

ρ((x′t)1≤t≤T , (xt)1≤t≤T ) =T∑t=1

(xt − x′t)2

or distance between summary statistics like the q autocorrelations

τj =T∑

t=j+1

xtxt−j



ABC basics

Comparison of distance impact

Evaluation of the tolerance on the ABC sample against bothdistances (ε = 100%, 10%, 1%, 0.1%) for an MA(2) model



ABC basics

Comparison of distance impact

0.0 0.2 0.4 0.6 0.8

01

23

4

θ1

−2.0 −1.0 0.0 0.5 1.0 1.5

0.00.5

1.01.5

θ2

Evaluation of the tolerance on the ABC sample against bothdistances (ε = 100%, 10%, 1%, 0.1%) for an MA(2) model



ABC basics

Homonomy

The ABC algorithm is not to be confused with the ABC algorithm

The Artificial Bee Colony algorithm is a swarm based meta-heuristicalgorithm that was introduced by Karaboga in 2005 for optimizingnumerical problems. It was inspired by the intelligent foragingbehavior of honey bees. The algorithm is specifically based on themodel proposed by Tereshko and Loengarov (2005) for the foragingbehaviour of honey bee colonies. The model consists of threeessential components: employed and unemployed foraging bees, andfood sources. The first two components, employed and unemployedforaging bees, search for rich food sources (...) close to their hive.The model also defines two leading modes of behaviour (...):recruitment of foragers to rich food sources resulting in positivefeedback and abandonment of poor sources by foragers causingnegative feedback.

[Karaboga, Scholarpedia]



ABC basics

ABC advances

Simulating from the prior is often poor in efficiency

Either modify the proposal distribution on θ to increase the densityof x’s within the vicinity of y...

[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

...or by viewing the problem as a conditional density estimationand by developing techniques to allow for larger ε

[Beaumont et al., 2002]

.....or even by including ε in the inferential framework [ABCµ][Ratmann et al., 2009]



ABC basics

ABC advances

Simulating from the prior is often poor in efficiencyEither modify the proposal distribution on θ to increase the densityof x’s within the vicinity of y...

[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

...or by viewing the problem as a conditional density estimationand by developing techniques to allow for larger ε

[Beaumont et al., 2002]

.....or even by including ε in the inferential framework [ABCµ][Ratmann et al., 2009]



Alphabet soup

ABC-NP

Better usage of [prior] simulations byadjustement: instead of throwing awayθ′ such that ρ(η(z), η(y)) > ε, replaceθs with locally regressed

θ∗ = θ − {η(z)− η(y)}Tβ[Csillery et al., TEE, 2010]

where β is obtained by [NP] weighted least square regression on(η(z)− η(y)) with weights

Kδ {ρ(η(z), η(y))}

[Beaumont et al., 2002, Genetics]



Alphabet soup

ABC-MCMC

Markov chain (θ(t)) created via the transition function

θ(t+1) =

θ′ ∼ Kω(θ′|θ(t)) if x ∼ f(x|θ′) is such that x = y

and u ∼ U(0, 1) ≤ π(θ′)Kω(θ(t)|θ′)π(θ(t))Kω(θ′|θ(t)) ,

θ(t) otherwise,

has the posterior π(θ|y) as stationary distribution[Marjoram et al, 2003]



Alphabet soup

ABC-MCMC (2)

Algorithm 3 Likelihood-free MCMC sampler

Use Algorithm 2 to get (θ(0), z(0))for t = 1 to N do

Generate θ′ from Kω

(·|θ(t−1)

),

Generate z′ from the likelihood f(·|θ′),Generate u from U[0,1],

if u ≤ π(θ′)Kω(θ(t−1)|θ′)π(θ(t−1)Kω(θ′|θ(t−1))

IAε,y(z′) then

set (θ(t), z(t)) = (θ′, z′)else

(θ(t), z(t))) = (θ(t−1), z(t−1)),end if

end for



Alphabet soup

ABCµ

[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]

Use of a joint density

f(θ, ε|y) ∝ ξ(ε|y, θ)× πθ(θ)× πε(ε)

where y is the data, and ξ(ε|y, θ) is the prior predictive density ofρ(η(z), η(y)) given θ and x when z ∼ f(z|θ)

Warning! Replacement of ξ(ε|y, θ) with a non-parametric kernelapproximation.



Alphabet soup

ABCµ

[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]

Use of a joint density

f(θ, ε|y) ∝ ξ(ε|y, θ)× πθ(θ)× πε(ε)

where y is the data, and ξ(ε|y, θ) is the prior predictive density ofρ(η(z), η(y)) given θ and x when z ∼ f(z|θ)Warning! Replacement of ξ(ε|y, θ) with a non-parametric kernelapproximation.



Alphabet soup

ABCµ details

Multidimensional distances ρk (k = 1, . . . ,K) and errorsεk = ρk(ηk(z), ηk(y)), with

εk ∼ ξk(ε|y, θ) ≈ ξk(ε|y, θ) =1

Bhk

∑b

K[{εk−ρk(ηk(zb), ηk(y))}/hk]

then used in replacing ξ(ε|y, θ) with mink ξk(ε|y, θ)

ABCµ involves acceptance probability

π(θ′, ε′)π(θ, ε)

q(θ′, θ)q(ε′, ε)q(θ, θ′)q(ε, ε′)

mink ξk(ε′|y, θ′)mink ξk(ε|y, θ)



Alphabet soup

ABCµ details

Multidimensional distances ρk (k = 1, . . . ,K) and errorsεk = ρk(ηk(z), ηk(y)), with

εk ∼ ξk(ε|y, θ) ≈ ξk(ε|y, θ) =1

Bhk

∑b

K[{εk−ρk(ηk(zb), ηk(y))}/hk]

then used in replacing ξ(ε|y, θ) with mink ξk(ε|y, θ)ABCµ involves acceptance probability

π(θ′, ε′)π(θ, ε)

q(θ′, θ)q(ε′, ε)q(θ, θ′)q(ε, ε′)

mink ξk(ε′|y, θ′)mink ξk(ε|y, θ)



Alphabet soup

Questions about ABCµ

For each model under comparison, marginal posterior on ε used toassess the fit of the model (HPD includes 0 or not).

I Is the data informative about ε? [Identifiability]

I How is the prior π(ε) impacting the comparison?

I How is using both ξ(ε|x0, θ) and πε(ε) compatible with astandard probability model? [remindful of Wilkinson]

I Where is the penalisation for complexity in the modelcomparison?

[X, Mengersen & Chen, 2010, PNAS]



Alphabet soup

ABC-PRC

Another sequential version producing a sequence of Markov

transition kernels Kt and of samples (θ(t)1 , . . . , θ

(t)N ) (1 ≤ t ≤ T )

ABC-PRC Algorithm

1. Pick a θ? is selected at random among the previous θ(t−1)i ’s

with probabilities ω(t−1)i (1 ≤ i ≤ N).

2. Generateθ

(t)i ∼ Kt(θ|θ?) , x ∼ f(x|θ(t)

i ) ,

3. Check that %(x, y) < ε, otherwise start again.

[Sisson et al., 2007]



Alphabet soup

Why PRC?

Partial rejection control: Resample from a population of weightedparticles by pruning away particles with weights below threshold C,replacing them by new particles obtained by propagating anexisting particle by an SMC step and modifying the weightsaccordinly.

[Liu, 2001]

PRC justification in ABC-PRC:

Suppose we then implement the PRC algorithm for somec > 0 such that only identically zero weights are smallerthan c

Trouble is, there is no such c...



Alphabet soup

Why PRC?

Partial rejection control: Resample from a population of weightedparticles by pruning away particles with weights below threshold C,replacing them by new particles obtained by propagating anexisting particle by an SMC step and modifying the weightsaccordinly.

[Liu, 2001]PRC justification in ABC-PRC:

Suppose we then implement the PRC algorithm for somec > 0 such that only identically zero weights are smallerthan c

Trouble is, there is no such c...



Alphabet soup

ABC-PRC weight

Probability ω(t)i computed as

ω(t)i ∝ π(θ(t)

i )Lt−1(θ?|θ(t)i ){π(θ?)Kt(θ

(t)i |θ

?)}−1 ,

where Lt−1 is an arbitrary transition kernel.

In caseLt−1(θ′|θ) = Kt(θ|θ′) ,

all weights are equal under a uniform prior.Inspired from Del Moral et al. (2006), who use backward kernelsLt−1 in SMC to achieve unbiasedness



Alphabet soup

ABC-PRC weight


ω(t)i ∝ π(θ(t)


(t)i |θ

?)}−1 ,

where Lt−1 is an arbitrary transition kernel.In case

Lt−1(θ′|θ) = Kt(θ|θ′) ,

all weights are equal under a uniform prior.

Inspired from Del Moral et al. (2006), who use backward kernelsLt−1 in SMC to achieve unbiasedness



Alphabet soup

ABC-PRC weight


ω(t)i ∝ π(θ(t)


(t)i |θ

?)}−1 ,

where Lt−1 is an arbitrary transition kernel.In case

Lt−1(θ′|θ) = Kt(θ|θ′) ,

all weights are equal under a uniform prior.Inspired from Del Moral et al. (2006), who use backward kernelsLt−1 in SMC to achieve unbiasedness



Alphabet soup

A mixture example (1)

Toy model of Sisson et al. (2007): if

θ ∼ U(−10, 10) , x|θ ∼ 0.5N (θ, 1) + 0.5N (θ, 1/100) ,

then the posterior distribution associated with y = 0 is the normalmixture

θ|y = 0 ∼ 0.5N (0, 1) + 0.5N (0, 1/100)

restricted to [−10, 10].Furthermore, true target available as

π(θ||x| < ε) ∝ Φ(ε−θ)−Φ(−ε−θ)+Φ(10(ε−θ))−Φ(−10(ε+θ)) .



Alphabet soup

“Ugly, squalid graph...”

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

θθ

−3 −1 1 3

0.00.2

0.40.6

0.81.0

Comparison of τ = 0.15 and τ = 1/0.15 in Kt



Alphabet soup

A PMC version

Use of the same kernel idea as ABC-PRC but with IS correctionGenerate a sample at iteration t by

πt(θ(t)) ∝N∑j=1

ω(t−1)j Kt(θ(t)|θ(t−1)

j )

modulo acceptance of the associated xt, and use an importance

weight associated with an accepted simulation θ(t)i

ω(t)i ∝ π(θ(t)

i )/πt(θ

(t)i ) .

c© Still likelihood free[Beaumont et al., 2008, arXiv:0805.2256]



Alphabet soup

The ABC-PMC algorithm

Given a decreasing sequence of approximation levels ε1 ≥ . . . ≥ εT ,

1. At iteration t = 1,

For i = 1, ..., NSimulate θ

(1)i ∼ π(θ) and x ∼ f(x|θ(1)i ) until %(x, y) < ε1

Set ω(1)i = 1/N

Take τ2 as twice the empirical variance of the θ(1)i ’s

2. At iteration 2 ≤ t ≤ T ,

For i = 1, ..., N , repeat

Pick θ?i from the θ(t−1)j ’s with probabilities ω

(t−1)j

generate θ(t)i |θ

?i ∼ N (θ?i , σ

2t ) and x ∼ f(x|θ(t)i )

until %(x, y) < εt

Set ω(t)i ∝ π(θ(t)i )/

∑Nj=1 ω

(t−1)j ϕ

(σ−1

t

{θ(t)i − θ

(t−1)j )

})Take τ2

t+1 as twice the weighted empirical variance of the θ(t)i ’s



Alphabet soup

Sequential Monte Carlo

SMC is a simulation technique to approximate a sequence ofrelated probability distributions πn with π0 “easy” and πT astarget.Iterated IS as PMC : particles moved from time n to time n viakernel Kn and use of a sequence of extended targets πn

πn(z0:n) = πn(zn)n∏j=0

Lj(zj+1, zj)

where the Lj ’s are backward Markov kernels [check that πn(zn) isa marginal]

[Del Moral, Doucet & Jasra, Series B, 2006]



Alphabet soup

Sequential Monte Carlo (2)

Algorithm 4 SMC sampler

sample z(0)i ∼ γ0(x) (i = 1, . . . , N)

compute weights w(0)i = π0(z(0)

i ))/γ0(z(0)i )

for t = 1 to N doif ESS(w(t−1)) < NT then

resample N particles z(t−1) and set weights to 1end ifgenerate z

(t−1)i ∼ Kt(z

(t−1)i , ·) and set weights to

w(t)i = W

(t−1)i−1

πt(z(t)i ))Lt−1(z(t)

i ), z(t−1)i ))

πt−1(z(t−1)i ))Kt(z

(t−1)i ), z(t)

i ))

end for

[Del Moral, Doucet & Jasra, Series B, 2006]



Alphabet soup

ABC-SMC

[Del Moral, Doucet & Jasra, 2009]

True derivation of an SMC-ABC algorithmUse of a kernel Kn associated with target πεn and derivation of thebackward kernel

Ln−1(z, z′) =πεn(z′)Kn(z′, z)

πn(z)

Update of the weights

win ∝ wi(n−1)

∑Mm=1 IAεn (xmin)∑M

m=1 IAεn−1(xmi(n−1))

when xmin ∼ K(xi(n−1), ·)



Alphabet soup

ABC-SMCM

Modification: Makes M repeated simulations of the pseudo-data zgiven the parameter, rather than using a single [M = 1]simulation, leading to weight that is proportional to the number ofaccepted zis

ω(θ) =1M

M∑i=1

Iρ(η(y),η(zi))<ε

[limit in M means exact simulation from (tempered) target]



Alphabet soup

Properties of ABC-SMC

The ABC-SMC method properly uses a backward kernel L(z, z′) tosimplify the importance weight and to remove the dependence onthe unknown likelihood from this weight. Update of importanceweights is reduced to the ratio of the proportions of survivingparticlesMajor assumption: the forward kernel K is supposed to beinvariant against the true target [tempered version of the trueposterior]

Adaptivity in ABC-SMC algorithm only found in on-lineconstruction of the thresholds εt, slowly enough to keep a largenumber of accepted transitions



Alphabet soup

Properties of ABC-SMC

The ABC-SMC method properly uses a backward kernel L(z, z′) tosimplify the importance weight and to remove the dependence onthe unknown likelihood from this weight. Update of importanceweights is reduced to the ratio of the proportions of survivingparticlesMajor assumption: the forward kernel K is supposed to beinvariant against the true target [tempered version of the trueposterior]Adaptivity in ABC-SMC algorithm only found in on-lineconstruction of the thresholds εt, slowly enough to keep a largenumber of accepted transitions



Alphabet soup

A mixture example (2)

Recovery of the target, whether using a fixed standard deviation ofτ = 0.15 or τ = 1/0.15, or a sequence of adaptive τt’s.

θθ

−3 −2 −1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

θθ

−3 −2 −1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

θθ

−3 −2 −1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

θθ

−3 −2 −1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

θθ

−3 −2 −1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0



Alphabet soup

Wilkinson’s exact BC

Wilkinson (2008) replaces the ABC approximation error(i.e. non-zero tolerance) in with an exact simulation from acontrolled approximation to the target, a convolution of the trueposterior with an arbitrary kernel function

πε(θ, z|y) =π(θ)f(z|θ)Kε(y − z)∫π(θ)f(z|θ)Kε(y − z)dzdθ

,

where Kε is a kernel parameterised by a bandwidth ε.

I Interesting perspective in that outcome is completelycontrolled

I Link with ABCµ and assuming y is observed with ameasurement error with density Kε

I Relates to the theory of model approximation[Kennedy & O’Hagan, 2001]



Alphabet soup

Wilkinson’s exact BC

Wilkinson (2008) replaces the ABC approximation error(i.e. non-zero tolerance) in with an exact simulation from acontrolled approximation to the target, a convolution of the trueposterior with an arbitrary kernel function

πε(θ, z|y) =π(θ)f(z|θ)Kε(y − z)∫π(θ)f(z|θ)Kε(y − z)dzdθ

,

where Kε is a kernel parameterised by a bandwidth ε.

I Requires Kε to be bounded

I True approximation error never assessed

I Requires a modification of the standard ABC algorithms



Alphabet soup

Semi-automatic ABC

Fearnhead and Prangle (2010) study ABC and the selection of thesummary statistic in close proximity to Wilkinson’s proposal

ABC then considered from a purely inferential viewpoint andcalibrated for estimation purposes.Use of a randomised (or ‘noisy’) version of the summary statistics

η(y) = η(y) + τε

Derivation of a well-calibrated version of ABC, i.e. an algorithmthat gives proper predictions for the distribution associated withthis randomised summary statistic.

[calibration constraint: ABCapproximation with same posterior mean as the true randomisedposterior.]



Alphabet soup

Semi-automatic ABC

Fearnhead and Prangle (2010) study ABC and the selection of thesummary statistic in close proximity to Wilkinson’s proposal

ABC then considered from a purely inferential viewpoint andcalibrated for estimation purposes.Use of a randomised (or ‘noisy’) version of the summary statistics

η(y) = η(y) + τε

Derivation of a well-calibrated version of ABC, i.e. an algorithmthat gives proper predictions for the distribution associated withthis randomised summary statistic. [calibration constraint: ABCapproximation with same posterior mean as the true randomisedposterior.]



Alphabet soup

Summary statistics

Optimality of the posterior expectations of the parameters ofinterest as summary statistics!

Use of the standard quadratic loss function

(θ − θ0)TA(θ − θ0) .



Alphabet soup

Summary statistics

Optimality of the posterior expectations of the parameters ofinterest as summary statistics!Use of the standard quadratic loss function

(θ − θ0)TA(θ − θ0) .



Calibration of ABC

Which summary?

Fundamental difficulty of the choice of the summary statistic whenthere is no non-trivial sufficient statistics [except when done by theexperimenters in the field]

Starting from a large collection of summary statistics is available,Joyce and Marjoram (2008) consider the sequential inclusion intothe ABC target, with a stopping rule based on a likelihood ratiotest.

I Does not taking into account the sequential nature of the tests

I Depends on parameterisation

I Order of inclusion matters.



Calibration of ABC

Which summary?

Fundamental difficulty of the choice of the summary statistic whenthere is no non-trivial sufficient statistics [except when done by theexperimenters in the field]Starting from a large collection of summary statistics is available,Joyce and Marjoram (2008) consider the sequential inclusion intothe ABC target, with a stopping rule based on a likelihood ratiotest.

I Does not taking into account the sequential nature of the tests

I Depends on parameterisation

I Order of inclusion matters.



Calibration of ABC

A connected Monte Carlo study

I Repeating simulations of the pseudo-data per simulated parameterdoes not improve approximation

I Tolerance level ε does not seem to be highly influential

I Choice of distance / summary statistics / calibration factorsare paramount to successful approximation

I ABC-SMC outperforms ABC-MCMC

[Mckinley, Cook, Deardon, 2009]



Calibration of ABC

A Brave New World?!






The Gibbs Sampler



ABC for model choiceModel choiceGibbs random fieldsModel choice via ABCIllustrations



Model choice

Bayesian model choice

Several models M1,M2, . . . are considered simultaneously for adataset y and the model index M is part of the inference.Use of a prior distribution. π(M = m), plus a prior distribution onthe parameter conditional on the value m of the model index,πm(θm)Goal is to derive the posterior distribution of M , challengingcomputational target when models are complex.



Model choice

Generic ABC for model choice

Algorithm 5 Likelihood-free model choice sampler (ABC-MC)

for t = 1 to T dorepeat

Generate m from the prior π(M = m)Generate θm from the prior πm(θm)Generate z from the model fm(z|θm)

until ρ{η(z), η(y)} < εSet m(t) = m and θ(t) = θm

end for



Model choice

ABC estimates

Posterior probability π(M = m|y) approximated by the frequencyof acceptances from model m

1T

T∑t=1

Im(t)=m .

Issues with implementation:

I should tolerances ε be the same for all models?

I should summary statistics vary across models (incl. theirdimension)?

I should the distance measure ρ vary as well?

Extension to a weighted polychotomous logistic regression estimateof π(M = m|y), with non-parametric kernel weights

[Cornuet et al., DIYABC, 2009]



Model choice

The Great ABC controversy

On-going controvery in phylogeographic genetics about the validityof using ABC for testing

Against: Templeton, 2008,2009, 2010a, 2010b, 2010cargues that nested hypothesescannot have higher probabilitiesthan nesting hypotheses (!)



Model choice

The Great ABC controversy

On-going controvery in phylogeographic genetics about the validityof using ABC for testing

Against: Templeton, 2008,2009, 2010a, 2010b, 2010cargues that nested hypothesescannot have higher probabilitiesthan nesting hypotheses (!)

Replies: Fagundes et al., 2008,Beaumont et al., 2010, Berger etal., 2010, Csillery et al., 2010point out that the criticisms areaddressed at [Bayesian]model-based inference and havenothing to do with ABC...



Gibbs random fields

Gibbs random fields

Gibbs distribution

The rv y = (y1, . . . , yn) is a Gibbs random field associated withthe graph G if

f(y) =1Z

exp

{−∑c∈C

Vc(yc)

},

where Z is the normalising constant, C is the set of cliques of G

and Vc is any function also called potentialU(y) =

∑c∈C Vc(yc) is the energy function

c© Z is usually unavailable in closed form



Gibbs random fields

Potts model

Potts model

Vc(y) is of the form

Vc(y) = θS(y) = θ∑l∼i

δyl=yi

where l∼i denotes a neighbourhood structure

In most realistic settings, summation

Zθ =∑x∈X

exp{θTS(x)}

involves too many terms to be manageable and numericalapproximations cannot always be trusted

[Cucala, Marin, CPR & Titterington, 2009]



Model choice via ABC

Bayesian Model Choice

Comparing a model with potential S0 taking values in Rp0 versus amodel with potential S1 taking values in Rp1 can be done throughthe Bayes factor corresponding to the priors π0 and π1 on eachparameter space

Bm0/m1(x) =

∫exp{θT

0 S0(x)}/Zθ0,0π0(dθ0)∫

exp{θT1 S1(x)}/Zθ1,1

π1(dθ1)

Use of Jeffreys’ scale to select most appropriate model




Neighbourhood relations

Choice to be made between M neighbourhood relations

im∼ i′ (0 ≤ m ≤M − 1)

withSm(x) =

∑im∼i′

I{xi=xi′}

driven by the posterior probabilities of the models.




Model index

Formalisation via a model index M that appears as a newparameter with prior distribution π(M = m) andπ(θ|M = m) = πm(θm)

Computational target:

P(M = m|x) ∝∫

Θm

fm(x|θm)πm(θm) dθm π(M = m) ,




Model index

Formalisation via a model index M that appears as a newparameter with prior distribution π(M = m) andπ(θ|M = m) = πm(θm)Computational target:

P(M = m|x) ∝∫

Θm

fm(x|θm)πm(θm) dθm π(M = m) ,




Sufficient statistics

By definition, if S(x) sufficient statistic for the joint parameters(M, θ0, . . . , θM−1),

P(M = m|x) = P(M = m|S(x)) .

For each model m, own sufficient statistic Sm(·) andS(·) = (S0(·), . . . , SM−1(·)) also sufficient.For Gibbs random fields,

x|M = m ∼ fm(x|θm) = f1m(x|S(x))f2

m(S(x)|θm)

=1

n(S(x))f2m(S(x)|θm)

wheren(S(x)) = ] {x ∈ X : S(x) = S(x)}

c© S(x) is therefore also sufficient for the joint parameters[Specific to Gibbs random fields!]






P(M = m|x) = P(M = m|S(x)) .

For each model m, own sufficient statistic Sm(·) andS(·) = (S0(·), . . . , SM−1(·)) also sufficient.

For Gibbs random fields,


m(S(x)|θm)

=1


wheren(S(x)) = ] {x ∈ X : S(x) = S(x)}







P(M = m|x) = P(M = m|S(x)) .

For each model m, own sufficient statistic Sm(·) andS(·) = (S0(·), . . . , SM−1(·)) also sufficient.For Gibbs random fields,


m(S(x)|θm)

=1


wheren(S(x)) = ] {x ∈ X : S(x) = S(x)}





ABC model choice Algorithm

ABC-MCI Generate m∗ from the prior π(M = m).

I Generate θ∗m∗ from the prior πm∗(·).

I Generate x∗ from the model fm∗(·|θ∗m∗).

I Compute the distance ρ(S(x0), S(x∗)).

I Accept (θ∗m∗ ,m∗) if ρ(S(x0), S(x∗)) < ε.

Note When ε = 0 the algorithm is exact




ABC approximation to the Bayes factor

Frequency ratio:

BFm0/m1(x0) =

P(M = m0|x0)P(M = m1|x0)

× π(M = m1)π(M = m0)

=]{mi∗ = m0}]{mi∗ = m1}

× π(M = m1)π(M = m0)

,

replaced with

BFm0/m1(x0) =

1 + ]{mi∗ = m0}1 + ]{mi∗ = m1}

× π(M = m1)π(M = m0)

to avoid indeterminacy (also Bayes estimate).



Illustrations

Toy example

iid Bernoulli model versus two-state first-order Markov chain, i.e.

f0(x|θ0) = exp

(θ0

n∑i=1

I{xi=1}

)/{1 + exp(θ0)}n ,

versus

f1(x|θ1) =12

exp

(θ1

n∑i=2

I{xi=xi−1}

)/{1 + exp(θ1)}n−1 ,

with priors θ0 ∼ U(−5, 5) and θ1 ∼ U(0, 6) (inspired by “phasetransition” boundaries).



Illustrations

Toy example (2)

−40 −20 0 10

−50

5

BF01

BF01

−40 −20 0 10−10

−50

510

BF01

BF01

(left) Comparison of the true BFm0/m1(x0) with BFm0/m1

(x0)(in logs) over 2, 000 simulations and 4.106 proposals from theprior. (right) Same when using tolerance ε corresponding to the1% quantile on the distances.



Illustrations

Protein folding

Superposition of the native structure (grey) with the ST1structure (red.), the ST2 structure (orange), the ST3 structure(green), and the DT structure (blue).



Illustrations

Protein folding (2)

% seq . Id. TM-score FROST score

1i5nA (ST1) 32 0.86 75.31ls1A1 (ST2) 5 0.42 8.91jr8A (ST3) 4 0.24 8.91s7oA (DT) 10 0.08 7.8

Characteristics of dataset. % seq. Id.: percentage of identity withthe query sequence. TM-score: similarity between predicted andnative structure (uncertainty between 0.17 and 0.4) FROST score:quality of alignment of the query onto the candidate structure(uncertainty between 7 and 9).



Illustrations

Protein folding (3)

NS/ST1 NS/ST2 NS/ST3 NS/DT

BF 1.34 1.22 2.42 2.76P(M = NS|x0) 0.573 0.551 0.708 0.734

Estimates of the Bayes factors between model NS and modelsST1, ST2, ST3, and DT, and corresponding posteriorprobabilities of model NS based on an ABC-MC algorithm using1.2 106 simulations and a tolerance ε equal to the 1% quantile ofthe distances.

MCMC and likelihood-free methods

Education

Transcript of MCMC and likelihood-free methods