Dynamic Hamiltonian Monte Carlo in Stan

Hamiltonian Monte Carlouse of gradient information and dynamic simulation reducerandom walk

Dynamic HMCadaptive simulation time

Adaptation of algorithm parametersmass matrix and step size adaptation during warm-up

Dynamic HMC specific diagnostics

Aki.Vehtari@aalto.fi – @avehtari

Extra material for dynamic HMC

Michael Betancourt (2018). Scalable Bayesian Inferencewith Hamiltonian Monte Carlohttps://www.youtube.com/watch?v=jUSZboSq1zgMichael Betancourt (2018). A Conceptual Introduction toHamiltonian Monte Carlo. https://arxiv.org/abs/1701.02434http://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/Cole C. Monnahan, James T. Thorson, and Trevor A.Branch (2016) Faster estimation of Bayesian models inecology using Hamiltonian Monte Carlo.https://dx.doi.org/10.1111/2041-210X.12681

Aki.Vehtari@aalto.fi – @avehtari

Demos

https://github.com/avehtari/BDA_R_demos/tree/master/demos_ch12

demos_ch12/demo12_1.R

http://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●●

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●●

●

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●●

●

●

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

●

●

●

● ●●

●

●

●

●

●

−2

0

2

−2 0 2

theta1

thet

a2

● Samples Steps of the sampler 90% HPD

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

−2

0

2

0 250 500 750 1000iter

valu

e

theta1 theta2

Trends

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

0.00

0.25

0.50

0.75

1.00

0 5 10 15 20iter

valu

e

theta1 theta2

Autocorrelation function

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte CarloUses gradient information for more efficient samplingAugments parameter space with momentum variables

−1

0

1

0 250 500 750 1000iter

valu

e

theta1 theta2 95% interval for MCMC error 95% interval for independent MC

Cumulative averages

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte Carlo

Uses gradient information for more efficient samplingAugments parameter space with momentum variablesSimulation of Hamiltonian dynamics reduces random walk

http://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte Carlo

Uses gradient information for more efficient samplingAlternating dynamic simulation and sampling of the energylevel

Parameters: step size, number of steps in each chainNo U-Turn Sampling (NUTS) and dynamic HMC

adaptively selects number of steps to improve robustnessand efficiencydynamic HMC refers to dynamic trajectory lengthto keep reversibility of Markov chain, need to simulate intwo directionshttp://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/

Dynamic simulation is discretizedsmall step size gives accurate simulation, but requires morelog density evaluationslarge step size reduces computation, but increasessimulation error which needs to be taken into account in theMarkov chain

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte Carlo

Uses gradient information for more efficient samplingAlternating dynamic simulation and sampling of the energylevelParameters: step size, number of steps in each chain

No U-Turn Sampling (NUTS) and dynamic HMCadaptively selects number of steps to improve robustnessand efficiencydynamic HMC refers to dynamic trajectory lengthto keep reversibility of Markov chain, need to simulate intwo directionshttp://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/

Dynamic simulation is discretizedsmall step size gives accurate simulation, but requires morelog density evaluationslarge step size reduces computation, but increasessimulation error which needs to be taken into account in theMarkov chain

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte Carlo

Uses gradient information for more efficient samplingAlternating dynamic simulation and sampling of the energylevelParameters: step size, number of steps in each chainNo U-Turn Sampling (NUTS) and dynamic HMC

adaptively selects number of steps to improve robustnessand efficiencydynamic HMC refers to dynamic trajectory lengthto keep reversibility of Markov chain, need to simulate intwo directionshttp://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/

Dynamic simulation is discretizedsmall step size gives accurate simulation, but requires morelog density evaluationslarge step size reduces computation, but increasessimulation error which needs to be taken into account in theMarkov chain

Aki.Vehtari@aalto.fi – @avehtari

Hamiltonian Monte Carlo

Uses gradient information for more efficient samplingAlternating dynamic simulation and sampling of the energylevelParameters: step size, number of steps in each chainNo U-Turn Sampling (NUTS) and dynamic HMC

adaptively selects number of steps to improve robustnessand efficiencydynamic HMC refers to dynamic trajectory lengthto keep reversibility of Markov chain, need to simulate intwo directionshttp://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/

Dynamic simulation is discretizedsmall step size gives accurate simulation, but requires morelog density evaluationslarge step size reduces computation, but increasessimulation error which needs to be taken into account in theMarkov chain

Aki.Vehtari@aalto.fi – @avehtari

Adaptive dynamic HMC in Stan

Dynamic HMC using growing tree to increase simulationtrajectory until no-U-turn criterion stopping

max treedepth to keep computation in controlpick a draw along the trajectory with probabilities adjustedto take into account the error in the discretized dynamicsimulation

Mass matrix and step size adaptation in Stanmass matrix refers to having different scaling for differentparameters and optionally also rotation to reducecorrelationsmass matrix and step size adjustment and are estimatedduring initial adaptation phasestep size is adjusted to be as big as possible while keepingdiscretization error in control

After adaptation the algorithm parameters are fixed andsome further iterations included in the warmupAfter warmup store iterations for inferenceSee more details in Stan reference manual

Aki.Vehtari@aalto.fi – @avehtari

Adaptive dynamic HMC in Stan

Dynamic HMC using growing tree to increase simulationtrajectory until no-U-turn criterion stopping

max treedepth to keep computation in controlpick a draw along the trajectory with probabilities adjustedto take into account the error in the discretized dynamicsimulation

Mass matrix and step size adaptation in Stanmass matrix refers to having different scaling for differentparameters and optionally also rotation to reducecorrelationsmass matrix and step size adjustment and are estimatedduring initial adaptation phasestep size is adjusted to be as big as possible while keepingdiscretization error in control

After adaptation the algorithm parameters are fixed andsome further iterations included in the warmupAfter warmup store iterations for inferenceSee more details in Stan reference manual

Aki.Vehtari@aalto.fi – @avehtari

Adaptive dynamic HMC in Stan

Dynamic HMC using growing tree to increase simulationtrajectory until no-U-turn criterion stopping

max treedepth to keep computation in controlpick a draw along the trajectory with probabilities adjustedto take into account the error in the discretized dynamicsimulation

Mass matrix and step size adaptation in Stanmass matrix refers to having different scaling for differentparameters and optionally also rotation to reducecorrelationsmass matrix and step size adjustment and are estimatedduring initial adaptation phasestep size is adjusted to be as big as possible while keepingdiscretization error in control

After adaptation the algorithm parameters are fixed andsome further iterations included in the warmup

After warmup store iterations for inferenceSee more details in Stan reference manual

Aki.Vehtari@aalto.fi – @avehtari

Adaptive dynamic HMC in Stan

Dynamic HMC using growing tree to increase simulationtrajectory until no-U-turn criterion stopping

max treedepth to keep computation in controlpick a draw along the trajectory with probabilities adjustedto take into account the error in the discretized dynamicsimulation

Mass matrix and step size adaptation in Stanmass matrix refers to having different scaling for differentparameters and optionally also rotation to reducecorrelationsmass matrix and step size adjustment and are estimatedduring initial adaptation phasestep size is adjusted to be as big as possible while keepingdiscretization error in control

After adaptation the algorithm parameters are fixed andsome further iterations included in the warmupAfter warmup store iterations for inference

See more details in Stan reference manual

Aki.Vehtari@aalto.fi – @avehtari

Adaptive dynamic HMC in Stan

Dynamic HMC using growing tree to increase simulationtrajectory until no-U-turn criterion stopping

max treedepth to keep computation in controlpick a draw along the trajectory with probabilities adjustedto take into account the error in the discretized dynamicsimulation

Mass matrix and step size adaptation in Stanmass matrix refers to having different scaling for differentparameters and optionally also rotation to reducecorrelationsmass matrix and step size adjustment and are estimatedduring initial adaptation phasestep size is adjusted to be as big as possible while keepingdiscretization error in control

After adaptation the algorithm parameters are fixed andsome further iterations included in the warmupAfter warmup store iterations for inferenceSee more details in Stan reference manual

Aki.Vehtari@aalto.fi – @avehtari

Dynamic HMC

Comparison of algorithms on highly correlated 250-dimensional Gaussian distribution

•Do 1,000,000 draws with both Random Walk Metropolis and Gibbs, thinning by 1000

•Do 1,000 draws using Stan’s NUTS algorithm (no thinning)

•Do 1,000 independent draws (we can do this for multivariate normal)

Source: Jonah GabryAki.Vehtari@aalto.fi – @avehtari

Max tree depth diagnostic

Dynamic HMC specific diagnosticIndicates inefficiency in sampling leading to higherautocorrelations and lower neff

Different parameterizations matter

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari

DivergencesHMC specific: Indicates that Hamiltonian dynamicsimulation has problems going to narrow places

indicates possibility of biased estimatesDifferent parameterizations matterhttp://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html

Aki.Vehtari@aalto.fi – @avehtari