REDUCING MCMC COMPUTATIONAL COST WITH A TWO LAYERED BAYESIAN APPROACH

Ramin Madarshahian, Doctoral Candidate,

mdrshhn@email.sc.edu

Juan M. Caicedo, Associate Professor

http://sdii.ce.sc.edu/

2

Outline • Introduction

• Methodology

• Example

• Future work

3

INTRODUCTION

4

Markov Chain Monte Carlo (MCMC)

• A very powerful algorithm for getting sample from high dimensional and complicated probability distribution function.

• It gets more sample from high probability regions, and with enough number of samples, histogram of samples take similar shape as probability distribution of interest.

5

Markov Chain Monte Carlo (MCMC)Von Neumann he contributed to the development of the Monte Carlo

method, which allowed solutions to complicated problems to be approximated using random numbers.

MetropolisUlam

Paper 1949: Using Markov chain for Monte Carlo approximation

Rossenburg Teller

Paper 1953: They applied MCMC for a chemical problem

6

Bayesian modeling• Bayesian modeling is good method to deal with

uncertainty. • Bayesian modeling update our belief about the model and

its parameters by considering evidences.• Evidence comes from inputs and outputs.

7

Problem?• With a Bayesian model we would like to make an

inference about a model and its parameters. MCMC can be used to sample the posterior, but for each sample we need to run the model.

• What if our model is very

computational expensive?

8

Metamodeling• Metamodel: Simply an approximation of the

computationally expensive model.

• Also known as : Response surface, emulators, auxiliary models, etc.

• Computationally expensive models: Models of multi-scale problems like shear band, models of complicated structures like airplane, modelling of physical and biological phenomena like protein folding, etc.

9

General approach

• Few numbers of input samples.• Using the expensive model to obtain corresponding output

samples.• Using these I/O relationships to fit the metamodel.• For some methods, using obtained metamodel to select

next input sample to better fit the metamodel.• Replacing the expensive model with obtained metamodel,

and using this new model in the Bayesian process and etc.

10

Proposed method

• In our proposed method, the surface of interest is obtained from the posterior of the Bayesian model, instead of direct approximation of the expensive model by the surrogate.

11

Motivation

• A posterior is a probability distribution function with all common characteristics of that. It is usually more well-defined in comparison to the expensive model itself.

• Depending upon the type of study, a researcher can focus on high probability regions (like model updating problems) or focus on tails (Reliability problems). This makes sampling more efficient.

12

METHODOLOGY

13

Bayesian modelling formulation

Model’s parameters

Data

Posterior

Likelihood

14

Proposed method formulation

15

EXAMPLE

16

SDOF

then what is using the data in the table?

Simulated by assuming normal distribution with and for K

17

Inference without using the metamodelAssuming uniform prior for K from Zero to 2000:

18

Inference without using the metamodel

MCMC: • Selecting the prior:

Assuming uniform prior for K from Zero to 2000 :

• A total of 10000 samples are generated and the first 2000 are discarded.

19

Inference without using the metamodel

20

Proposed method

21

Proposed method• Priors:

For : a uniform distribution with the lower bound of 0.001 and the upper bound of 1.0.

For : a uniform distribution with the lower bound of zero and the upper bound of 2000.

For : a normal distribution with , and • A total of 12000 samples were obtained, 4000 of them

were considered burning samples and were discarded.

22

Proposed method

23

Proposed methodParameter Mean 95% HPD

24

Comparison• Using obtained mean and standard deviation, 95% HPD

will be which is comparable with results without the metamodel, i.e. .

25

FUTURE WORK

26

Future work

• Considering different types of metamodels like polynomial, kriging, etc.

• Development of sampling strategies.

• Considering expensive models and study of computational power of method.

REDUCING MCMC COMPUTATIONAL COST WITH A TWO LAYERED BAYESIAN APPROACH

Ramin Madarshahian, Doctoral Candidate,

mdrshhn@email.sc.edu

Juan M. Caicedo, Associate Professor

http://sdii.ce.sc.edu/

Thank you!