Bayesian Computation Approach to Inverse Problems in Heat Conduction Materials Process Design and...

Bayesian Computation Approach to Inverse Bayesian Computation Approach to Inverse Problems in Heat ConductionProblems in Heat Conduction

Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory

CCOORRNNEELLLL U N I V E R S I T Y


Principal investigator: Prof. Nicholas ZabarasPrincipal investigator: Prof. Nicholas ZabarasPresenter: Jingbo WangPresenter: Jingbo Wang

Materials Process Design and Control LaboratorySibley School of Mechanical and Aerospace Engineering

188 Frank H. T. Rhodes HallCornell University

Ithaca, NY 14853-3801

Email: [email protected]: (607) 255-9104

URL: http://www.mae.cornell.edu/zabaras/

mailto:[email protected]




Presentation outline

• Introduction to inverse problems in heat conductionIntroduction to inverse problems in heat conduction

• Introduction to Bayesian computationIntroduction to Bayesian computation

----- Fundamentals of Bayesian statistical inference----- Fundamentals of Bayesian statistical inference

----- Markov random field (MRF) and discontinuity adaptive Markov ----- Markov random field (MRF) and discontinuity adaptive Markov

random field (DAMRF)random field (DAMRF)

----- Markov chain Monte Carlo (MCMC) simulation----- Markov chain Monte Carlo (MCMC) simulation

• A few examplesA few examples

----- Inverse heat conduction problem (IHCP)----- Inverse heat conduction problem (IHCP)

----- Heat source reconstruction----- Heat source reconstruction

• Apply Bayesian to computationally intensive problems --- an Apply Bayesian to computationally intensive problems --- an

extension to inverse radiationextension to inverse radiation

• ConclusionsConclusions

Γo

Γg

Γh

* ***

****

** unknown heat flux

known temperature

known heat flux

thermocouples

Inverse problems in heat conduction



Materials Process Design and Control Laboratory

***

** **

**

Heat sources

)( TktTC P

in ,

,),( gTtxT ,g

],,0[ maxtt

,),(

hqntxT

k

on

,h),()0,( 0 xTxT

,),(

0qntxT

k

)(unknown ,0

],,0[ maxtt

on ],,0[ maxtt

in ,

on ].,0[ maxtt

+ f

Features of the inverse problem




• ill-posedness well-posedness: --- existence --- uniqueness --- continuous dependence of solutions on measurements

stability

A problem is ill-posed if it is not well-posed

identifiability

• uncertainties

random errors

solution errors

measurement errors

model uncertainties

numerical errors

incomplete mathematical representation

perturbations in systemparameters, initial or boundaryconditions …

discretization error

roundoff error

unsolved length scales

Approaches to the inverse problems




optimizationobjective

minimum least-squares error

maximum entropy

Maximum Likelihood

Bayesian inference

Gradient methods

(sensitivity and/or adjoint problems need to be solved)

Monte Carlo methods (importance sampling, rejection sampling, Markov chain MC, simulated annealing)

function specification

Tikhonov regularization

(S) future information

iterative regularization

--- conjugate gradient

--- EM method

convexification method

mollification method

regularization

Bayesian computation approach




• probabilistic description of inverse solutions

• quantification of various uncertainties

• data driven in nature

• direct simulation in deterministic space

• prior distribution regularization (spatial statistics

models)

• sampling schemes (MCMC, Latin hypercube …)

Advantages:

)|()()|( YppYp

Fundamentals of Bayesian statistics




• Bayesian statistics

)|( YθP)(

)()|(YP

θPθYP

• Bayesian inference to estimation (regression)

- interested in distribution of random quantities θ=[θ1, θ2, … θm]T

- ‘a priori beliefs’ about P(θ) - data Y=[Y1, Y2, … Yn]T relevant to θ

Priori pdf

Likelihood

Posterior pdf

• Bayes’ formula

prior + evidence => posterior probability of a hypothesis

The likelihood




Y = F(θ) + ω

For a typical system as,

the likelihood is determined by distribution of ω.eg: ω ~ N(0, σ2)

FYθp T

2))((

21exp{)|( Y FY ))}((

• conditional probability of observation (data) Y given the parameter θ• interface to data

prior posterior

The prior




• role of a prior pdf --- incorporate known to a priori information --- regularize the likelihood

• a prior distribution can be “informative” or “improper”

• techniques of prior distribution modeling --- accumulated distribution information --- conjugate prior distributions --- physical constraints --- local uniforms --- spatial statistics models

An example of conjugate prior:if Y|θ,σ ~ N(θ, σ), then θ ~ N(θo, σo), σ ~ inv-Gamma (a,b).

A class π of prior distributions is said to form a conjugate family if the posterior density p(θ|X) is in the class for all X whenever the prior density is in π.

More complicated models




* A hierarchical structure

)()(),|()|,( ppYpYp

)()|(),|()|,( ppYpYp

)()|( pYp )|( p Ym Yo , )|( Yop

diminish the effect of poor knowledge on hyper-parameters.

* An augmented formulation

quantifying measurement error from data

* A Expectation-Maximization formulation

more robust formulation (iterative regularization) when there are missing data

! Bayesian is adaptive model --- posteriors can be treated as priors for new data

Spatial statistics models




Markov random field (MRF)

• having exponential form• explore the spatial and temporal dependence• close related to Tikhonov regularization

}))((exp{)( ~ ji jiijWp

q

x

t

θi

wi

Neighbors of θ

u,

0

)(2

21u

u, else

DAMRF

Monte Carlo simulation




Monte Carlo Principle

1. draw an i.i.d. set of samples {x(i)} i = 1:N from a target density p(x)

2. approximate the target density with the following empirical point-mass function

3. approximate the integral (expectation) I(f) with tractable sums IN( f )

N

ixN x

Nxp

i1

)(1

)(

N

iX

NiN dxxpxffIxf

NfI

1

)()()()(1

)(

Markov chain Monte Carlo (MCMC)




Initialize xInitialize x00

For i=0:N-1For i=0:N-1

sample u~U(0,1)sample u~U(0,1)

sample sample xx** ~ ~ q(q(xx**|x|xii) )

if u < A(xif u < A(xii, , xx**)=min)=min{1, p(x{1, p(x**)q(x)q(xii|x|x**)/(p(x)/(p(xii)q(x)q(x**|x|xii))}))}

xxi+1i+1=x=x**

else xelse xi+1i+1=x=xii

• Sampling from a complex distribution using Markov chain mechanism

Metropolis-Hastings algorithm

Gibbs sampler

)|(~ 11

ijj

ij xxpx

Initialize x0

For i = 0:N-1 For j = 1:m sample

Bayesian formulation for IHCP




• Parameterization of unknown heat flux q0

m

iii txwq

10 ),( Unknown vector θ

Input θInput θ direct numericalsolver F

direct numericalsolver F

Measurement YMeasurement Y

simulationnoise

Y = F(θ) + ω

• System input and output relation

random

• Likelihood function

FYθp T2

))((2

1exp{)|(

Y FY ))}(( --- known σ

--- unknown σ FYθ,p T2

))((2

1(σ2)-n/2exp{σ2)|(

Y FY ))}((

Assumptions• numerical error much less then measurement noise• ω iid ~ N(0, σ2)

Bayesian formulation for IHCP (cont.)




Markov Random Field (MRF)

}))((exp{)(~ ji

jiijWp

}2

exp{)( 2/ Wp Tm

2

21

)( uu

else

ji

jin

Wi

ij ~

,0

,1

,

Prior distribution modeling

--- Single layer posterior:

--- hierarchical posterior:

TWYYp T2 )

2

1exp(})(

2

1exp{)|(

Y )(

}2

1exp{}

2

)()(exp{),,( 2//2 W

v

YYvvp Tm

T

Tn

TT

}exp{}exp{ 11

)1(0

1 10 TT vv

H H

H H

• Gibbs sampler

Gibbs sampler and modified Gibbs sampler




1

2i

i ii i

b

a a

2

2 21 1

2N N

si s sii ii i p

s s

H Ha W b

s s st t p ji j ik kt i j i k i

Y H W W

),(~| 2iiii N

)|(~ 11

ijj

ij θθpθ

Initialize θ0

For i = 0:N-1 For j = 1:m sample

• modified Gibbs sampler

(i) Initialize θ(0), λ(0) and vT(0)

(ii) For i = 0:N-1 ---For j = 1:m sample ---sample u ~ U(0,1) ---sample λ(*) ~ qλ(λ(*) | λ(i)) ---if u < A(λ(*) , λ(i) ) λ(I+1) = λ(*) ---else λ(I+1) = λ(i) ---sample u ~ U(0,1) ---sample vT

(*) ~ qv(vT(*) | vT

(i)) ---if u < A(vT

(*) , vT(i) )

vT(I+1) = vT

(*) ---else vT

(I+1) = vT(i)

)|(~ 11

ijj

ij θθpθ

xq

dL

Y (d,iΔt)

--- True q in simulationq

0 0.5 0.9

1.0

--- Normalized governing equation

2

2

xT

tT

1t 0 ,0 1x

0),0( xT 1x0

0

LxxT

)(0

tqxTx

,

,

, 1t 0

1t 0,

t1.0

--- Discretization of q(t)

θiθi-1 θi+1

dt

1D IHCP example




0.1

1D IHCP example (results)




d=0.5 σT=0.01 d=0.5 σT=0.001

d=0.1 σT=0.01 d=0.1 σT=0.001

d=0.5 σT=0.01(assume σT unknown )

Reconstruction of piecewise continuous heat source




),())(( txfTxktT

x

5.1

23

4

1

x

xk

0<=x<0.25

0.25<=x<0.5

0.5<=x<0.75

0.75<=x<=1.0

tx

tx

tx

tx

e

ex

ex

e

txf

5.2

)22(

)45(

2

),(

0<=x<0.25

0.25<=x<0.5

0.5<=x<0.75

0.75<=x<=1.0

txetxT),(

with

analytical solution to the direct problem:

2D heat source reconstruction




• Normalized governing equations

,2

2

2

2

fy

T

x

T

t

T

,10 x ,10 y ,0t

,01010

yyxx yT

yT

xT

xT

0T , t = 0.

,0t

unknown

insulated

insulated

insulated

insulated

*

*

*

**

**

*

* **

* **

*

*

*

*

*

*

*

*

*

*

*

2125.02

20)10exp(),,(

ttyxf

2

22

125.02

)725.0()75.0(exp{

yx

2D heat source reconstruction




true heat source

reconstructed heat source when σT=0.005

reconstructed heat source when σT=0.02

Inverse Heat Radiation Problem




S(gray

boundary)thermocouple

participatingmedia

Vheat source

*

**

*

*

What g(t) causes measured T?

2 ( ) ( )p r

TC k T g t G x x y y z zq

t

4( ) ( )

4 bs I I I r d Is

4b

b

TI

4

14 ( ( ) )

4br I I r s dq

0

1( ) ( ) 0b n sI r s I n I r d n ss s

wT T

Direct simulation




A finite element (FE) + S4 method framework

1. Set T(i)guess = T(I-1);

2. Substitute T(i)guess to compute Ib ;

3. Solve intensity eq for I(i);

4. Compute ;

5. Solve temperature eq to update T(i)guess;

6. If the solution converged, set T(i)guess

as T(i) and save I(i); otherwise, go to 2;

7. Go to next time step.

rq

Reduced order modeling --- A POD based approach




• POD eigenfunction problem• reduced order models

( ) ( )

1

1 eNi i

Vie

U U dvN

1

( ) ( ) ( )TK

h Ti i

i

T t r a t r

1

( ) ( ) ( )IK

h Ii i

i

I t r s b t r s

1

( ) 1TK

jj ji i j j T

i

daM H a S Q g t j K

dt

1 1

1I IK K

ji i ji i j Ii i

A b B b D j K

2( )Tj p jV

M C dv T T

ji j iVH k dv

Tj jrVS dvq

( )Tj jV

Q G x x y y z z dv 4

{ ( ) }I I I Iji i j i jVA s d dv

4 4{( ) }I I

ji i jVB d d dv

4

( )I Ij b b jV

D I I d dv

• Solution as linear combination of POD basis

MCMC algorithm --- a cycle design of single component update




• implicit likelihood MH sampler• increasing acceptance probability single component update

Algorithm:

( ) ( 1) ( ) ( ) ( )2

1 1( ) exp{ ( )}

22i i i

j j j j j jqjqj

q

},...,,,...,,{ )()(1

)1(1

)1(2

)1(1

)1( im

ij

ij

iiij

Initialize Initialize θθ00

For i=0:N-1For i=0:N-1

For j=1:mFor j=1:m

sample u~U(0,1)sample u~U(0,1)

sample sample θθ**jj ~ ~ qqjj((θθ **

jj | | θθ ii

-j-j , , θθ ii

jj ) )

if u < A(if u < A(θθ iijj , , θθ **

jj ))

θθi+1i+1==θθ**

else else θθi+1i+1==θθii

A testing example




x

z y

1m

1m

1m

g(t)(0.5m, 0.5m, 0.5m)

800K

800K

800K

800K

800K

O

800K

***

12

3

o t

g(t)

400kW/m3

0.05s0.01s 0.04s

80kW/m3

o t

g(t)

0.02s 0.04s 0.05s

160kW/m3

80kW/m3

g1(t)

g2(t)

Schematic of the example

Profile of testing heat sources

Frame 001 09 Mar 2004 Frame 001 09 Mar 2004

Basis fields




Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004

Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004

Frame 001 09 Mar 2004 Frame 001 09 Mar 2004

Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004

1st, 3rd and 6th Basis of Th

1st, 3rd and 6th Basis of Ih

along direction s =[0.9082483 0.2958759 0.2958759]

1st, 3rd and 6th Basis of Ih

along direction s =[-0.9082483 0.2958759 0.2958759]

Homogeneous temperature solution




Th computed by full model Th computed by reduced order model

Comparison of reduced order solutions atthermocouple locations

Heat source reconstruction




MAP estimates of g1 at different magnitude of noise

MAP estimates of g2 at different magnitude of noise

Posterior mean of g1 when σT =0.005

Posterior mean of g2 when σT =0.005

• Bayesian inference treats the inverse problem as a well-posed problem in an expanded stochastic space• Bayesian inference approach provides statistical distribution as well as point estimates of inverse solution• In seeking point estimates, Bayesian approach regularizes the ill-posedness of inverse problem through prior distribution modeling• MCMC samplers provide accurate estimates for the statistics of inverse solution• Bayesian computation is applicable to complex inverse problems via reduced-order modeling

Conclusions




Futures--- Sequential Bayesian filter--- Stochastic upscaling via Bayesian computation--- Uncertainty quantification in multiscale simulation

Bayesian Computation Approach to Inverse Problems in Heat Conduction Materials Process Design and...

Documents

Transcript of Bayesian Computation Approach to Inverse Problems in Heat Conduction Materials Process Design and...