Review of Bayes’ Rule
23
BIOL 301 Guest Lecture: Reasoning Under Uncertainty (Intro to Bayes Networks) Simon D. Levy CSCI Department 8 April 2010
description
BIOL 301 Guest Lecture: Reasoning Under Uncertainty (Intro to Bayes Networks) Simon D. Levy CSCI Department 8 April 2010. Rev. Thomas Bayes (1702-1761). Review of Bayes’ Rule. From the Product Rule:. P( A | B ) = P( A & B ) / P(B). P( A & B ) = P( A | B ) * P( B ). - PowerPoint PPT Presentation
Transcript of Review of Bayes’ Rule
BIOL 301 Guest Lecture: Reasoning Under Uncertainty
(Intro to Bayes Networks)
Simon D. LevyCSCI Department
8 April 2010
Review of Bayes’ Rule
• From the Product Rule:
• P(A|B) = P(A & B) / P(B)• P(A &B) = P(A|B) * P(B) = P(B|A) * P(A)
Rev. Thomas Bayes (1702-1761)• We derive Bayes’ Rule by
substitution:• P(A|B) = P(A & B) / P(B) = P(B|A) * P(A) / P(B)
Real-World Problems May Involve Many Variables
http://www.bayesia.com/assets/images/content/produits/blab/tutoriel/en/chapitre-3/image026.jpg
Real-World Problems May Involve Many Variables
Variables Are TypicallyOberserved Simultaneously
(Confounded)Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001
So how do we compute P(V=Yes), P(F=Yes & A=No), etc.?
Marginalization
Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001 ______ Sum = .006
P(V=Yes):
Marginalization
Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001 ______ Sum = .003
P(F=Yes & A=No):
Combinatorial Explosion(The “Curse of Dimensionality”)
Assuming (unrealistically) only two values (Yes/No) per variable:
# Variables # of Rows in Table1 22 43 84 165 326 64: :20 1,048,576
Solution: Local Causality + Belief Propagation
Local Causality
Recover Joint From Prior & Posterior
B E P(A)T T .95T F .94F T .29F F .001
P(B).001
P(E).002
B E A ProbT T T .001*.002*.95 = .000001900T T F .001*.002*.05 = .000000100T F T .001*.998*.94 = .000938120T F F .001*.998*.06 = .000059880F T T .999*.002*.29 = .000579420F T F .999*.002*.71 = .001418580F F T .999*.998*.001 = .000997002F F F .999*.998*.999 = .996004998
Belief Propagation
• Consider just B → A → J
• P(J=T | B=T) = P(J=T | A=T) * P(A=T | B=T) • Then use Bayes’ Rule and marginalization to answer more sophisticated queries like
P(B=T | J=F & E=F & M=T)
Multiply-Connected Networks
Wet Grass
Cloudy
Rain Sprinkler
Clustering (“Mega Nodes”)
Cloudy Sprinkler Rain
Sprinkler Rain Wet Grass
Sprinkler Rain
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
H
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
“Moralize””
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
HTriangulate
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
H
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
“Message-Passing”
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
Observe A=T
“Message-Passing”
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
Pick a cluster containing A:
“Message-Passing”
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
Pass messages to propagate evidence:
