Soaring Skyward Select Questions Wrong OUCH! WRONG ANSWER: You’re headed for a Crash.
You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An...
Transcript of You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An...
![Page 1: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/1.jpg)
You’re Doing It Wrong!An Introduction to Bayesian Inference
Michael BetancourtPhysics Diversity Summit
January 25, 2010
![Page 2: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/2.jpg)
Introduction to Bayesian Inference
Letting the Data Speak
** *
**
*
*
*
*
* *
**
**
*
xi
x j
D, p(D|!) , !?
2
![Page 3: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/3.jpg)
Introduction to Bayesian Inference
Letting the Data Speak
** *
**
*
*
*
*
* *
**
**
*
xi
x j
D, p(D|!) , !?
2
![Page 4: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/4.jpg)
Introduction to Bayesian Inference
Letting the Data Speak
** *
**
*
*
*
*
* *
**
**
*
xi
x j
D, p(D|!) , !?
2
![Page 5: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/5.jpg)
Introduction to Bayesian Inference
Letting the Data Speak
** *
**
*
*
*
*
* *
**
**
*
xi
x j
D, p(D|!) , !?
2
![Page 6: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/6.jpg)
Introduction to Bayesian Inference
Estimators
!(D) , !! "
3
![Page 7: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/7.jpg)
Introduction to Bayesian Inference
Estimators
!(D) , !! "
!(!"")2# = (!!#"")2 +!!!2#"!!#2"
3
![Page 8: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/8.jpg)
Introduction to Bayesian Inference
Estimators
!(D) , !! "
!(!"")2# = (!!#"")2 +!!!2#"!!#2"
3
![Page 9: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/9.jpg)
Introduction to Bayesian Inference
Estimators
!(D) , !! "
!(!"")2# = (!!#"")2 +!!!2#"!!#2"
3
![Page 10: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/10.jpg)
Introduction to Bayesian Inference
Estimators: Gaussian Example
p(D|!) = "i
N (xi|µ,#)
4
![Page 11: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/11.jpg)
Introduction to Bayesian Inference
Estimators: Gaussian Example
p(D|!) = "i
N (xi|µ,#)
µ(D) = x =1n!
ixi !2 (D) =
1n!1"
i(xi! x)2
4
![Page 12: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/12.jpg)
Introduction to Bayesian Inference
Estimators: Gaussian Example
p(D|!) = "i
N (xi|µ,#)
µ(D) = x =1n!
ixi
!µ" = µ
!2 (D) =1
n!1"i
(xi! x)2
!!2" = !2
4
![Page 13: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/13.jpg)
Introduction to Bayesian Inference
Estimators: The Dirty Details
!!" =Z
dD p(D|")!(D)
5
![Page 14: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/14.jpg)
Introduction to Bayesian Inference
Estimators: The Dirty Details
!!" =Z
dD p(D|")!(D)
var(!) = !!2"D #!!"2D
5
![Page 15: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/15.jpg)
Introduction to Bayesian Inference
Maximum Likelihood
! = argmax! p(D|!)
6
![Page 16: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/16.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: χ2
p(D|!) = "i
N (xi|µi,#i)
7
![Page 17: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/17.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: χ2
p(D|!) = "i
N (xi|µi,#i)
p(D|!) " exp
!!1
2#i
"xi!µi
$i
#2$
7
![Page 18: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/18.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: χ2
p(D|!) = "i
N (xi|µi,#i)
p(D|!) " exp
!!1
2#i
"xi!µi
$i
#2$
p(D|!) " exp!!1
2#2
"
7
![Page 19: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/19.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: χ2
p(D|!) = "i
N (xi|µi,#i)
p(D|!) " exp
!!1
2#i
"xi!µi
$i
#2$
p(D|!) " exp!!1
2#2
"
argmax! p(D|!)! argmin
! "2
7
![Page 20: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/20.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
8
![Page 21: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/21.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
!L!µ
=!2"#2"!m
2 exp
#!$i (xi!µ)2
2#2
$$i (xi!µ)
#2
8
![Page 22: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/22.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
0 = !i (xi!µML)"2
ML= !i xi!mµML
"2ML
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
!L!µ
=!2"#2"!m
2 exp
#!$i (xi!µ)2
2#2
$$i (xi!µ)
#2
8
![Page 23: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/23.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
0 = !i (xi!µML)"2
ML= !i xi!mµML
"2ML
µML = !i xi
m
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
!L!µ
=!2"#2"!m
2 exp
#!$i (xi!µ)2
2#2
$$i (xi!µ)
#2
8
![Page 24: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/24.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
9
![Page 25: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/25.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
!L!"
=!2#"2"!m
2 exp
#!$i (xi!µ)2
2"2
$1"
%!m+ $i (xi!µ)2
"2
&
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
9
![Page 26: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/26.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
!L!"
=!2#"2"!m
2 exp
#!$i (xi!µ)2
2"2
$1"
%!m+ $i (xi!µ)2
"2
&
0 = !m+ !i (xi!µML)2
"2ML
= !m+ !i (xi!"x#)2
"2ML
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
9
![Page 27: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/27.jpg)
Introduction to Bayesian Inference
Maximum Likelihood: Gaussian
!L!"
=!2#"2"!m
2 exp
#!$i (xi!µ)2
2"2
$1"
%!m+ $i (xi!µ)2
"2
&
0 = !m+ !i (xi!µML)2
"2ML
= !m+ !i (xi!"x#)2
"2ML
!2ML = "i (xi!"x#)2
m
L = !i
1!2"#2
exp
!"(xi"µ)2
2#2
"=
#2"#2$"m
2 exp
!"$i (xi"µ)2
2#2
"
9
![Page 28: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/28.jpg)
Introduction to Bayesian Inference
Multimodal Distributions
10
![Page 29: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/29.jpg)
Introduction to Bayesian Inference
A New Approach
p(!|D) =p(D,!)
p(D)
11
![Page 30: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/30.jpg)
Introduction to Bayesian Inference
A New Approach
p(!|D) =p(D,!)
p(D)
11
![Page 31: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/31.jpg)
Introduction to Bayesian Inference
A New Approach
p(!|D) =p(D,!)
p(D)
p(!|D) =p(D|!) p(!)
p(D)
11
![Page 32: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/32.jpg)
Introduction to Bayesian Inference
Probability As Frequency
P(xi) =N (xi)
N! !(xi)
"
12
![Page 33: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/33.jpg)
Introduction to Bayesian Inference
Probability In Science
What is the probability that...
13
![Page 34: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/34.jpg)
Introduction to Bayesian Inference
Probability In Science
‣ gravitational waves exist?
What is the probability that...
13
![Page 35: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/35.jpg)
Introduction to Bayesian Inference
Probability In Science
‣ gravitational waves exist?
What is the probability that...
‣ the Higgs will be found at the LHC?
13
![Page 36: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/36.jpg)
Introduction to Bayesian Inference
Probability In Science
‣ gravitational waves exist?
‣ dark matter is a composite particle?
What is the probability that...
‣ the Higgs will be found at the LHC?
13
![Page 37: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/37.jpg)
Introduction to Bayesian Inference
Probability In Science
‣ any of us graduate?
‣ gravitational waves exist?
‣ dark matter is a composite particle?
What is the probability that...
‣ the Higgs will be found at the LHC?
13
![Page 38: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/38.jpg)
Introduction to Bayesian Inference
The Cox Axioms
14
![Page 39: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/39.jpg)
Introduction to Bayesian Inference
The Cox Axioms
One : P(A) > P(B) > P(C)! P(A) > P(C)
14
![Page 40: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/40.jpg)
Introduction to Bayesian Inference
The Cox Axioms
Two : P!A|B
"= f (P(A|B))
One : P(A) > P(B) > P(C)! P(A) > P(C)
14
![Page 41: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/41.jpg)
Introduction to Bayesian Inference
The Cox Axioms
Two : P!A|B
"= f (P(A|B))
Three : P(A1A2|B) = g(P(A1|B) ,P(A2|A1B))
One : P(A) > P(B) > P(C)! P(A) > P(C)
14
![Page 42: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/42.jpg)
Introduction to Bayesian Inference
Bayes Rule
p(!|D) =p(D|!) p(!)
p(D)
15
![Page 43: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/43.jpg)
Introduction to Bayesian Inference
Bayes Rule
p(!|D) =p(D|!) p(!)
p(D)
15
![Page 44: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/44.jpg)
Introduction to Bayesian Inference
Bayes Rule
p(!|D) =p(D|!) p(!)
p(D)
15
![Page 45: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/45.jpg)
Introduction to Bayesian Inference
Bayes Rule
p(!|D) =p(D|!) p(!)
p(D)
15
![Page 46: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/46.jpg)
Introduction to Bayesian Inference
Bayes Rule
p(!|D) =p(D|!) p(!)
p(D)
15
![Page 47: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/47.jpg)
Introduction to Bayesian Inference
Make Inferences, Not War
p(!) . . .
16
![Page 48: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/48.jpg)
Introduction to Bayesian Inference
Make Inferences, Not War
‣ But frequentists don’t have to make assumptions!
17
![Page 49: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/49.jpg)
Introduction to Bayesian Inference
Bayesian Example
p(x|!) = !exp [!!x]
18
![Page 50: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/50.jpg)
Introduction to Bayesian Inference
Bayesian Example
p(!|x1, . . . ,xm) =p(x1, . . . ,xm|!) p(!)
p(x1, . . . ,xm)=
!m exp [!!"i xi] p(!)p(x1, . . . ,xm)
p(x|!) = !exp [!!x]
18
![Page 51: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/51.jpg)
Introduction to Bayesian Inference
Bayesian Example
p(!|x1, . . . ,xm) =p(x1, . . . ,xm|!) p(!)
p(x1, . . . ,xm)=
!m exp [!!"i xi] p(!)p(x1, . . . ,xm)
p(x|!) = !exp [!!x]
p(!|x1, . . . ,xm) =!m exp [!!"i xi]1/!
p(x1, . . . ,xm)
18
![Page 52: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/52.jpg)
Introduction to Bayesian Inference
Bayesian Example
p(!|x1, . . . ,xm) =p(x1, . . . ,xm|!) p(!)
p(x1, . . . ,xm)=
!m exp [!!"i xi] p(!)p(x1, . . . ,xm)
p(x|!) = !exp [!!x]
p(!|x1, . . . ,xm) =!m exp [!!"i xi]1/!
p(x1, . . . ,xm)
p(!|x1, . . . ,xm) =("i xi)m
(m!1)!!m!1 exp
!!!"
ixi
"
18
![Page 53: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/53.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 54: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/54.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 55: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/55.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 56: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/56.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 57: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/57.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 58: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/58.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 59: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/59.jpg)
Introduction to Bayesian Inference
Bayesian Example
19
![Page 60: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/60.jpg)
Introduction to Bayesian Inference
Marginalization
p(!|D,") =p(D|!,") p(!) p(")
p(D)
20
![Page 61: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/61.jpg)
Introduction to Bayesian Inference
Marginalization
p(!|D,") =p(D|!,") p(!) p(")
p(D)
p(!|D) =Z
d" p(!|D,")
20
![Page 62: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/62.jpg)
Introduction to Bayesian Inference
Model Comparison
21
![Page 63: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/63.jpg)
Introduction to Bayesian Inference
References
‣ Information Theory, Inference, and Learning Algorithms,
MacKay, http://www.inference.phy.cam.ac.uk/mackay/itila/
‣ Data Analysis,
Sivia with Skilling, Oxford 2006
‣ Lectures on Probability, Entropy, and Statistical Physics,
Caticha, http://arxiv.org/abs/0808.0012v1
‣ Probability Theory: The Logic of Science,
Jaynes, Cambridge 2003
22
![Page 64: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/64.jpg)
Introduction to Bayesian Inference
Backups
23
![Page 65: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/65.jpg)
Introduction to Bayesian Inference
A Consistent Decision Theory
R(!) =Z
dD L(!,") p(D|!)
24
![Page 66: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/66.jpg)
Introduction to Bayesian Inference
A Consistent Decision Theory
R(!) =Z
dD L(!,") p(D|!)
!R" =Z
d!g(!)R(!)
24
![Page 67: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/67.jpg)
Introduction to Bayesian Inference
A Consistent Decision Theory
R(!) =Z
dD L(!,") p(D|!)
!R" =Z
d!g(!)R(!)
L(!,") = (!!")2" " =R
d!! p(D|!)g(!)Rd! p(D|!)g(!)
24
![Page 68: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/68.jpg)
Introduction to Bayesian Inference
A Consistent Decision Theory
R(!) =Z
dD L(!,") p(D|!)
!R" =Z
d!g(!)R(!)
L(!,") = (!!")2" " =R
d!! p(D|!)g(!)Rd! p(D|!)g(!)
24
![Page 69: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/69.jpg)
Introduction to Bayesian Inference
Backup: Priors
‣ What does it mean to be ignorant?
25
![Page 70: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/70.jpg)
Introduction to Bayesian Inference
Choosing Priors
‣ Transformation Invariance
‣ Translation Invariant: Uniform Prior
‣ Scale Invariant: Jeffries Prior
‣ Maximum Entropy
‣ Marginalization
‣ Hierarchal Models
26
![Page 71: You’re Doing It Wrong!web.mit.edu/edbert/betancourtInference.pdf · You’re Doing It Wrong! An Introduction to Bayesian Inference Michael Betancourt Physics Diversity Summit January](https://reader035.fdocuments.in/reader035/viewer/2022081615/5fdb54659fe12d2da100207d/html5/thumbnails/71.jpg)
Introduction to Bayesian Inference
Unnormalizable Priors?
p(!) = 1/",!"/2 < ! < "/20 , else
lim!!"
p(#) =?
27