Learning Summary Statistics for Approximate Bayesian...
Transcript of Learning Summary Statistics for Approximate Bayesian...
Purpose• Thepurposeofmystudyistousevariousmachinelearningmethodstofacilitatethegenera6onofsummarysta6s6csinApproximateBayesianComputa6on(ABC).Specifically,Iwillusethefollowingmethods– Momentes)ma)onandlinearregression– Radiusbasisfunc)on– Neuralnetwork
WhatisanABCalgorithm?• InBayesianEs6ma6on,oneofthemostimportantstepistocalculatethelikelihoodfunc6on
• However,inmanyimportantapplica6ons,thisisinfeasible.ABCalgorithmcangetposteriores6ma6onwithoutknowingthelikelihoodfunc6on.
• Sothecentralques6onistofindagoodsummarysta6s6cusingmachinelearningmethodstoreducethedimension.
• Thefollowingtheoremguaranteesthevalidity.
Learning Summary Statistics for Approximate Bayesian Computation
Yiwen Chen, Department of Management Science and Engineering,
Stanford University
1. Background Information
• Blum,M.G.(2010).Choosingthesummarysta6s6csandtheacceptancerateinapproximatebayesiancomputa6on.InProceedingsofCOMPSTAT'2010,pages47-56.Springer.
• Fearnhead,P.andPrangle,D.(2012).Construc6ngsummarysta6s6csforapproximatebayesiancomputa6on:semi-automa6capproximatebayesiancomputa6on.JournaloftheRoyalSta6s6calSociety:SeriesB(Sta6s6calMethodology),74(3):419-474.
• Jiang,B.,Wu,T.-y.,Zheng,C.,andWong,W.H.(2015).Learningsummarysta6s6cforapproximatebayesiancomputa6onviadeepneuralnetwork.arXivpreprintarXiv:1510.02175.
3. Evaluate with Moving Average Model 4. Evaluate with Hidden MC model
5 Conclusions and Insights
6. Reference
• ThehiddenMarkovChainmodelisasfollows
• Automa6csummarysta6s6cgenera6onisagoodwaytodoABCalgorithmwhenwedon’tknowtheexactsufficientsta6s6cs.
• Morecomplexmodelstendtogeneratemorescaberedposteriordistribu6on,butthemeanandstandarddevia6onseembeber.
• Inprac6ce,theefficiencyofabovemethodsare
momentmethod>linearregression>NN>RBF
dsf
p(X |!θ = θ )
• Assumingtheunderlyingmodel
• Andunderlyingparameter• Atypical6meserieswiththeaboveparameters
0
0
1
1
θ
1−θ
1−θ θ
System
Measurements
γ 11−γ
2. Method
Xi = ε i +θε i−1 + γε i−2(θ ,γ ) = (1,−1)
(a)LinearRegression (b)Moment
(c)RBF (d)NeuralNetwork
• Linearregressionandneuralnetworkresultsinbeberpredic6onthantheothertwomethods.
• Momentes6ma6ontendstohaveamorecenteredposterior,whileRBFresultsinamorescaberedone.
(a)RBF (b)NeuralNetwork
• Allthreemethodstendtohavesimilarresultsinthiscase,whilecomplexmethodshavebeberpredic6ons.