ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504:...
Transcript of ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504:...
![Page 1: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/1.jpg)
ECE 6504: Advanced Topics in Machine Learning
Probabilistic Graphical Models and Large-Scale Learning
Dhruv Batra Virginia Tech
Topics: – Bayes Nets: Representation/Semantics
– v-structures – Probabilistic influence, Active Trails
Readings: Barber 3.3; KF 3.3.1-3.3.2
![Page 2: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/2.jpg)
Plan for today • Notation Clarification
• Errata #1: Number of parameters in disease network
• Errata #2: Car start v-structure example
• Bayesian Networks – Probabilistic influence & active trails – d-separation – General (conditional) independence assumptions in a BN
(C) Dhruv Batra 2
![Page 3: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/3.jpg)
A general Bayes net • Set of random variables
• Directed acyclic graph – Encodes independence assumptions
• CPTs – Conditional Probability Tables
• Joint distribution:
(C) Dhruv Batra 3
![Page 4: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/4.jpg)
Factorized distributions • Given
– Random vars X1,…,Xn
– P distribution over vars – BN structure G over same vars
• P factorizes according to G if
Flu Allergy
Sinus
Headache Nose
(C) Dhruv Batra 4 Slide Credit: Carlos Guestrin
![Page 5: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/5.jpg)
How many parameters in a BN? • Discrete variables X1, …, Xn
• Graph – Defines parents of Xi, PaXi
• CPTs – P(Xi| PaXi)
(C) Dhruv Batra 5
![Page 6: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/6.jpg)
Independencies in Problem
BN:
Graph G encodes local
independence
assumptions
World, Data, reality:
True distribution P contains independence
assertions
(C) Dhruv Batra 6 Slide Credit: Carlos Guestrin
![Page 7: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/7.jpg)
Bayes Nets • BN encode (conditional) independence assumptions.
– I(G) = {X indep of Y given Z}
• Which ones? • And how can we easily read them?
(C) Dhruv Batra 7
![Page 8: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/8.jpg)
Local Structures • What’s the smallest Bayes Net?
(C) Dhruv Batra 8
![Page 9: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/9.jpg)
Local Structures
(C) Dhruv Batra 9
Z
Y X
Z Y X
Z Y X
Z Y X
Indirect causal effect:
Indirect evidential effect:
Common cause:
Common effect:
![Page 10: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/10.jpg)
Car starts BN
• 18 binary attributes
• Inference – P(BatteryAge|Starts=f)
• 218 terms, why so fast?
(C) Dhruv Batra 10 Slide Credit: Carlos Guestrin
![Page 11: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/11.jpg)
Bayes Ball Rules • Flow of information
– on board
(C) Dhruv Batra 11
![Page 12: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/12.jpg)
Active trails formalized • Let variables O ⊆ {X1,…,Xn} be observed
• A path X1 – X2 – · · · –Xk is an active trail if for each consecutive triplet:
– Xi-1→Xi→Xi+1, and Xi is not observed (Xi∉O)
– Xi-1←Xi←Xi+1, and Xi is not observed (Xi∉O)
– Xi-1←Xi→Xi+1, and Xi is not observed (Xi∉O)
– Xi-1→Xi←Xi+1, and Xi is observed (Xi∈O), or one of its descendents is observed
Slide Credit: Carlos Guestrin (C) Dhruv Batra 12
![Page 13: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/13.jpg)
Active trails and Independence
• Theorem: Variables Xi and Xj are independent given Z if – no active trail between Xi and Xj
when variables Z⊆{X1,…,Xn} are observed
A
H
C E
G
D
B
F
K
J
I
(C) Dhruv Batra 13 Slide Credit: Carlos Guestrin
![Page 14: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/14.jpg)
Naïve Bayes:
Slide Credit: Erik Sudderth
Name That Model
(C) Dhruv Batra 14
![Page 15: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/15.jpg)
Slide Credit: Erik Sudderth
Name That Model
Tree-Augmented Naïve Bayes (TAN)
(C) Dhruv Batra 15
![Page 16: ECE 6504: Advanced Topics in Machine Learnings14ece6504/slides/L3_bn_semantics.pptx.pdfECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale](https://reader034.fdocuments.in/reader034/viewer/2022052518/5f0b80567e708231d430d49f/html5/thumbnails/16.jpg)
Name That Model
Hidden Markov Model (HMM)
Y1 = {a,…z}
X1 =
Y5 = {a,…z} Y3 = {a,…z} Y4 = {a,…z} Y2 = {a,…z}
X2 = X3 = X4 = X5 =
Figure Credit: Carlos Guestrin (C) Dhruv Batra 16