Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for...
-
Upload
claude-benson -
Category
Documents
-
view
218 -
download
1
Transcript of Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for...
![Page 1: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/1.jpg)
Bayesian networks
Chapter 14
Section 1 – 2
![Page 2: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/2.jpg)
Bayesian networks• A simple, graphical notation for conditional
independence assertions and hence for compact specification of full joint distributions
• Syntax:– a set of nodes, one per variable– a directed, acyclic graph (link ≈ "directly
influences")– a conditional distribution for each node given its
parents:P (Xi | Parents (Xi))
![Page 3: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/3.jpg)
Example• Topology of network encodes conditional independence
assertions:
• Weather is independent of the other variables• Toothache and Catch are conditionally independent
given Cavity
![Page 4: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/4.jpg)
Example
• I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar?
• Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls
• Network topology reflects "causal" knowledge:– A burglar can set the alarm off– An earthquake can set the alarm off– The alarm can cause Mary to call– The alarm can cause John to call
![Page 5: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/5.jpg)
Example contd.
![Page 6: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/6.jpg)
Compactness
• If each variable has no more than k parents, the complete network requires O(n · 2k) numbers
• I.e., grows linearly with n, vs. O(2n) for the full joint distribution
• For burglary net, 1 + 1 + 4 + 2 + 2 = 10 numbers (vs. 25-1 = 31)
![Page 7: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/7.jpg)
Semantics
![Page 8: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/8.jpg)
Constructing Bayesian networks
• 1. Choose an ordering of variables X1, … ,Xn
• 2. For i = 1 to n– add Xi to the network– select parents from X1, … ,Xi-1 such that
P (Xi | Parents(Xi)) = P (Xi | X1, ... Xi-1)
This choice of parents guarantees:
P (X1, … ,Xn) = πi =1 P (Xi | X1, … , Xi-1) (chain rule)
= πi =1P (Xi | Parents(Xi)) (by construction)
n
n
![Page 9: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/9.jpg)
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?
•
Example
![Page 10: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/10.jpg)
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)?
No
•
Example
![Page 11: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/11.jpg)
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)? No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)?
P(B | A, J, M) = P(B)?
•
Example
![Page 12: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/12.jpg)
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)? Yes
P(B | A, J, M) = P(B)? No
P(E | B, A ,J, M) = P(E | A)?
P(E | B, A, J, M) = P(E | A, B)?
Example
![Page 13: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/13.jpg)
• Suppose we choose the ordering M, J, A, B, E
P(J | M) = P(J)?No
P(A | J, M) = P(A | J)? P(A | J, M) = P(A)? No
P(B | A, J, M) = P(B | A)? Yes
P(B | A, J, M) = P(B)? No
P(E | B, A ,J, M) = P(E | A)? No
P(E | B, A, J, M) = P(E | A, B)? Yes
•
Example
![Page 14: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/14.jpg)
Example contd.
• Deciding conditional independence is hard in non-causal directions• (Causal models and conditional independence seem hardwired for
humans!)• Network is less compact
![Page 15: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/15.jpg)
Example contd.
• Deciding conditional independence is hard in non-causal directions• (Causal models and conditional independence seem hardwired for
humans!)• Network is less compact: 1 + 2 + 4 + 2 + 4 = 13 numbers needed
![Page 16: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/16.jpg)
Another example
![Page 17: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/17.jpg)
Another example
![Page 18: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/18.jpg)
![Page 19: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/19.jpg)
Example from Medical Diagnostics
• Network represents a knowledge structure that models the relationship between medical difficulties, their causes and effects, patient information and diagnostic tests
Visit to Asia
Tuberculosis
Tuberculosisor Cancer
XRay Result Dyspnea
BronchitisLung Cancer
Smoking
Patient Information
Medical Difficulties
Diagnostic Tests
![Page 20: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/20.jpg)
Example from Medical Diagnostics
• Relationship knowledge is modeled by deterministic functions, logic and conditional probability distributions
Patient Information
Diagnostic Tests
Visit to Asia
Tuberculosis
Tuberculosisor Cancer
XRay Result Dyspnea
BronchitisLung Cancer
SmokingTuber
Present
Present
Absent
Absent
Lung Can
Present
Absent
Present
Absent
Tub or Can
True
True
True
False
Medical DifficultiesTub or Can
True
True
False
False
Bronchitis
Present
Absent
Present
Absent
Present
0.90
0.70
0.80
0.10
Absent
0.l0
0.30
0.20
0.90
Dyspnea
![Page 21: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/21.jpg)
Example from Medical Diagnostics
• Propagation algorithm processes relationship information to provide an unconditional or marginal probability distribution for each node
• The unconditional or marginal probability distribution is frequently called the belief function of that node
TuberculosisPresentAbsent
1.0499.0
XRay ResultAbnormalNormal
11.089.0
Tuberculosis or CancerTrueFalse
6.4893.5
Lung CancerPresentAbsent
5.5094.5
DyspneaPresentAbsent
43.656.4
BronchitisPresentAbsent
45.055.0
Visit To AsiaVisitNo Visit
1.0099.0
SmokingSmokerNonSmoker
50.050.0
![Page 22: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/22.jpg)
• As a finding is entered, the propagation algorithm updates the beliefs attached to each relevant node in the network
• Interviewing the patient produces the information that “Visit to Asia” is “Visit”• This finding propagates through the network and the belief functions of several nodes are updated
TuberculosisPresentAbsent
5.0095.0
XRay ResultAbnormalNormal
14.585.5
Tuberculosis or CancerTrueFalse
10.289.8
Lung CancerPresentAbsent
5.5094.5
DyspneaPresentAbsent
45.055.0
BronchitisPresentAbsent
45.055.0
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
50.050.0
![Page 23: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/23.jpg)
TuberculosisPresentAbsent
5.0095.0
XRay ResultAbnormalNormal
18.581.5
Tuberculosis or CancerTrueFalse
14.585.5
Lung CancerPresentAbsent
10.090.0
DyspneaPresentAbsent
56.443.6
BronchitisPresentAbsent
60.040.0
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
100 0
• Further interviewing of the patient produces the finding “Smoking” is “Smoker”• This information propagates through the network
![Page 24: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/24.jpg)
TuberculosisPresentAbsent
0.1299.9
XRay ResultAbnormalNormal
0 100
Tuberculosis or CancerTrueFalse
0.3699.6
Lung CancerPresentAbsent
0.2599.8
DyspneaPresentAbsent
52.147.9
BronchitisPresentAbsent
60.040.0
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
100 0
• Finished with interviewing the patient, the physician begins the examination• The physician now moves to specific diagnostic tests such as an X-Ray, which results in a
“Normal” finding which propagates through the network• Note that the information from this finding propagates backward and forward through the arcs
![Page 25: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/25.jpg)
TuberculosisPresentAbsent
0.1999.8
XRay ResultAbnormalNormal
0 100
Tuberculosis or CancerTrueFalse
0.5699.4
Lung CancerPresentAbsent
0.3999.6
DyspneaPresentAbsent
100 0
BronchitisPresentAbsent
92.27.84
Visit To AsiaVisitNo Visit
100 0
SmokingSmokerNonSmoker
100 0
• The physician also determines that the patient is having difficulty breathing, the finding “Present” is entered for “Dyspnea” and is propagated through the network
• The doctor might now conclude that the patient has bronchitis and does not have tuberculosis or lung cancer
![Page 26: Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.](https://reader031.fdocuments.in/reader031/viewer/2022032605/56649e6f5503460f94b6ca5e/html5/thumbnails/26.jpg)
Summary
• Bayesian networks provide a natural representation for (causally induced) conditional independence
• Topology + conditional probability = compact representation of joint distribution
• Generally easy for domain experts to construct