Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees...
Transcript of Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees...
![Page 1: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/1.jpg)
Graphical Models
Lecture 11:
Clique TreesAndrew McCallum
Thanks to Noah Smith and Carlos Guestrin for some slide materials.1
![Page 2: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/2.jpg)
Inference
• Last two lectures: variable eliminaDon– Sum out variables one at a Dme.
• Today: alternaDve algorithms.– Also based on factors.– Central data structure: clique tree.
![Page 3: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/3.jpg)
Formalisms
Bayesian Network
Markov Network
pairwise Markov Network
factor graph
moralize skeleton
triangulateextra variables per factor
nothing
essenDally equivalent
helpful for approximate inference
clique tree
helpful for exact inference
triangulatepick root, add direcDons one
factor per clique
Lecture 5
![Page 4: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/4.jpg)
Formalisms
Markov Network
clique tree
Lecture 5
triangulate
![Page 5: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/5.jpg)
Clique Tree
A
B C
D E
F
ABC
BCD
CDE
DEF
Every maximal clique becomes a vertex.
Tree structure?then “Clique Tree”
(If graph was triangulated, always get a tree structure.)
Connect verDceswith overlappingvariables
Lecture 5
![Page 6: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/6.jpg)
Clique Tree
A
B C
D E
F
ABC
BCD
CDE
DEF
For each edge, intersecDon of r.v.s separates the rest in H.
sepH(A, D | B, C)
sepH(B, E | C, D)
sepH(C, F | D, E)
Lecture 5
“Sep-‐set” = {A, D}
![Page 7: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/7.jpg)
“Cluster Graph”• A cluster graph for a set of factors Φ over X is an undirected graph, each of whose nodes i is associated with a subset Ci ⊆ X.
• A cluster graph must be “family preserving”: each factor ϕ ∈ Φ must be associated with a cluster Ci (denoted α(ϕ)), such that Scope[ϕ] ⊆ Ci.
• Each edge between a pair of clusters Ci and Cj is associated with a sepset Si,j ⊆ Ci ∩ Cj.
• “Cluster Tree” a cluster graph that is a tree.7
![Page 8: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/8.jpg)
“Running Intersec3on” Property
• Cluster tree T (with verDces Ci…) has the running intersec3on property if, whenever there is a variable X such as X ∈ Ci and X ∈ Cj, then X is also in every cluster in the (unique) path in T between Ci and Cj.
8
ABC BCD CDE
C C C
![Page 9: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/9.jpg)
“Clique Tree” DefiniDon
• Given an undirected graph H, a tree T is a clique tree for H if:– Each node in T corresponds to a clique in H
– Each maximal clique in H is a node in T
– Each sepset Si,j separates the variables strictly on one side of the edge from the variables on the other side.
Si,j =
Ci ∩ Cj
aka “Junc0on Tree,” or “Join Tree”
Clique Tree = cluster tree that sa.sfies running intersec.on property.
Ci Cj
In other
words
![Page 10: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/10.jpg)
How to Obtain a Clique TreeOp,on 1
• Start with factor graph or undirected graph.
• Do variable eliminaDon. On iteraDon i:– Let ψi be the intermediate factor
– Let τi be the marginalized factor
– Let Ci be the set of variables involved in ψi
• One node per Ci, edge Ci-‐Cj when τi gets used in compuDng ψj.
![Page 11: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/11.jpg)
Example
• Eliminate S. Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
![Page 12: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/12.jpg)
Example
• Eliminate S.
• ψ1 = ϕFAS ·∙ ϕSR ·∙ ϕSH
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
FASRH
![Page 13: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/13.jpg)
Example
• Eliminate S.
• ψ1 = ϕFAS ·∙ ϕSR ·∙ ϕSH
Flu All.
S.I.
R.N. H.
ψ1
ϕAϕF
FASRH
![Page 14: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/14.jpg)
Example
• Eliminate S.
• ψ1 = ϕFAS ·∙ ϕSR ·∙ ϕSH
• τ1 = ∑S ψ1
Flu All.
S.I.
R.N. H.
ψ1
ϕAϕF
FASRH
![Page 15: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/15.jpg)
Example
• Eliminate S.
• ψ1 = ϕFAS ·∙ ϕSR ·∙ ϕSH
• τ1 = ∑S ψ1
Flu All.
R.N. H.
τ1
ϕAϕF
FASRH
![Page 16: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/16.jpg)
Example
• Eliminate R.
• ψ2 = τ1
• τ2 = ∑R ψ2
Flu All.
R.N. H.
τ1
ϕAϕF
FASRH FARH
![Page 17: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/17.jpg)
Example
• Eliminate R.
• ψ2 = τ1
• τ2 = ∑R ψ2
Flu All.
H.
τ2
ϕAϕF
FASRH FARH
![Page 18: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/18.jpg)
Example
• Eliminate H.
• ψ3 = τ2
• τ3 = ∑H ψ3
Flu All.
H.
τ2
ϕAϕF
FASRH FARH FAR
![Page 19: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/19.jpg)
Example
• Eliminate H.
• ψ3 = τ2
• τ3 = ∑H ψ3
Flu All.
τ3
ϕAϕF
FASRH FARH FAR
![Page 20: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/20.jpg)
Example
• Eliminate A.
• ψ4 = τ3 ϕA
• τ4 = ∑A ψ3
Flu All.
τ3
ϕAϕF
FASRH FARH FAR FA
![Page 21: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/21.jpg)
Example
• Eliminate A.
• ψ4 = τ3 ϕA
• τ4 = ∑A ψ3
Flu τ4
ϕF
FASRH FARH FAR FA
![Page 22: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/22.jpg)
Example
• Eliminate F.
• ψ5 = τ4 ϕF
• τ5 = ∑F ψ5
Flu τ4
ϕF
FASRH FARH FAR FA
F
![Page 23: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/23.jpg)
Example
• Eliminate F.
• ψ5 = τ4 ϕF
• τ5 = ∑F ψ5
τ5
FASRH FARH FAR FA
F
![Page 24: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/24.jpg)
AlternaDve Ordering
• {H, R, S, A, F}
SR
HS
SAF FA F
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
![Page 25: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/25.jpg)
How to Obtain a Clique TreeOp,on 1
• Start with factor graph or undirected graph.
• Do variable eliminaDon.
• One node per Ci, edge Ci-‐Cj when τi gets used in compuDng ψj.
• Result is always a clique tree!– Running intersecDon property.– Different orderings will give different clique trees.
![Page 26: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/26.jpg)
Reducing a Clique Tree
• Given a clique tree T for a set of factors Φ, there is always a reduced clique tree T’ such that– all clique-‐verDces in T’ are in T
– no clique vertex in T’ is a subset of another.
• ConstrucDon exploits the running intersecDon property.
![Page 27: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/27.jpg)
Reduced Clique Tree
HS
SR
SAF FA F
![Page 28: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/28.jpg)
Reduced Clique Tree
HS
SR
SAF FA F
HS
SR
SAF
![Page 29: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/29.jpg)
Reduced Clique Tree
FASRH FARH FAR FA
F
![Page 30: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/30.jpg)
Reduced Clique Tree
FASRH FARH FAR FA
F
FASRH
![Page 31: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/31.jpg)
How to Obtain a Clique TreeOp,on 2
1. Given factors, construct the undirected graph.
2. Triangulate to get a chordal graph.
3. Find maximal cliques in the chordal graph.
4. Construct a tree from the “cluster graph” of maximal cliques.
Actually the method we discussed in Lecture 5
![Page 32: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/32.jpg)
Step 2: Triangulate
• NP-‐hard in general to get the smallest one.
• Use heurisDcs.
![Page 33: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/33.jpg)
Step 3: Find Maximal Cliques
• Maximum cardinality search: gives an induced graph with no fill edges (assuming chordal graph as input, which we have).
• We proved last Dme that every maximal clique in the induced graph equates to the scope of an intermediate factor from VE.
• Other approaches exist.
![Page 34: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/34.jpg)
Maximum Cardinality Searchfor VE Ordering
• Start with undirected graph on X, all nodes unmarked.
• For i = |X| to 1:– Let Y be the unmarked variable in X with the largest number of marked neighbors
– π(Y) = i–Mark Y.
• Eliminate using permutaDon π.Lecture 10
![Page 35: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/35.jpg)
Example
• First node is arbitrary. Flu All.
S.I.
R.N. H.
![Page 36: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/36.jpg)
Example
• First node is arbitrary. Flu All.
S.I.
R.N. H.
5
![Page 37: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/37.jpg)
Example
• 1: A, S
• 0: R, H
Flu All.
S.I.
R.N. H.
5
![Page 38: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/38.jpg)
Example
• 2: A
• 1: R, H
Flu All.
S.I.
R.N. H.
5
4
![Page 39: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/39.jpg)
Example
• 1: R, H Flu All.
S.I.
R.N. H.
5
4
3
![Page 40: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/40.jpg)
Example
• 1: H Flu All.
S.I.
R.N. H.
5
4
3
2
![Page 41: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/41.jpg)
Example
• Order: {H, R, A, S, F} Flu All.
S.I.
R.N. H.
5
4
3
2 1
![Page 42: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/42.jpg)
How to Obtain a Clique TreeOp,on 2
1. Given factors, construct the undirected graph.2. Triangulate to get a chordal graph.3. Find maximal cliques in the chordal graph.4. Construct a tree from the “cluster graph” of
maximal cliques.– Results from VE with the ordering from maximum
cardinality search; or use maximum spanning tree (edge weights = sepset sizes).
![Page 43: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/43.jpg)
Example
• Order: {H, R, A, S, F} Flu All.
S.I.
R.N. H.
5
4
3
2 1
RS
HS
SF ASF
F
![Page 44: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/44.jpg)
Example (Reduce)
• Order: {H, R, A, S, F} Flu All.
S.I.
R.N. H.
5
4
3
2 1
RS
HS
ASF
![Page 45: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/45.jpg)
Now You Know …
• How to construct a clique tree.
• Also called a “juncDon tree” or a “join tree.”
![Page 46: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/46.jpg)
Running IntersecDon Property
• For connected clique-‐verDces Ci and Cj, the sepset Si,j is Ci ∩ Cj, and it
separates the variables on the Ci side from those on the Cj side.
RS
HS
ASF FT
![Page 47: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/47.jpg)
Running IntersecDon Property
A
B C
D E
F
ABC
BCD
CDE
DEF
For each edge, intersecDon of r.v.s separates the rest in H.
sepH(A, D | B, C)
sepH(B, E | C, D)
sepH(C, F | D, E)
Lecture 5
![Page 48: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/48.jpg)
Family PreservaDon
• Every factor in the original graph is associated with one clique-‐vertex in the clique tree.
![Page 49: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/49.jpg)
Family PreservaDon Example
HS
SR
SAF FA F
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH ϕFAS ϕFϕA
![Page 50: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/50.jpg)
Inference
• Last two lectures: variable eliminaDon– Sum out variables one at a Dme.
• Today: alternaDve algorithms.– Also based on factors.– Central data structure: clique tree.
• What are these algorithms?
![Page 51: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/51.jpg)
Clique Trees and Variable EliminaDon
• Each product-‐factor ψ belongs to a clique-‐vertex.
• Each τ is a “message” from one clique-‐vertex to another.
![Page 52: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/52.jpg)
Example
HS
SR
SAF FA F
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH ϕFAS ϕFϕA
• {H, R, S, A, F}
![Page 53: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/53.jpg)
Example
HS
SR
SAF FA F
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH ϕFAS ϕFϕA
• {H, R, S, A, F}
ψSR
ψSH
ψFAS ψFAψF
Just happens to be that in this graph with this ordering there is a one-‐to-‐one correspondence between original phis and product factors psi, because there was only one factor touching each variable-‐to-‐eliminate.
![Page 54: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/54.jpg)
Messages
• NoDce that the variable eliminaDon ordering implies a direcDon to the edges.
• Messages go “upstream.”
• This leads to a clique-‐tree-‐centric view of VE, in which the same calculaDons happen as in VE.
![Page 55: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/55.jpg)
Example
HS
SR
SAF FA F
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH ϕFAS ϕFϕA
• {H, R, S, A, F}
τS
τS
τAF τF
ψSR
ψSH
ψFAS ψFAψF
![Page 56: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/56.jpg)
Example
HS
SR
SAF
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH ϕFAS
ϕA
• {H, R, S, A, F}
• Reduced version
τS
τS
ψSR
ψSH
ψFAS
ϕF
![Page 57: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/57.jpg)
Example
HS
SR
SAF
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH νFAS
• {H, R, S, A, F}
• Reduced version
• IniDal potenDals
τS
τS
ψSR
ψSH
ψFAS
νFAS = φFASφAφF
Go through math that gets done for VE here.
![Page 58: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/58.jpg)
Message Passing in the Clique Tree
• Calculate ini7al poten7als for each clique-‐vertex.
• Choose an arbitrary root clique Cr. This imposes a direcDonality on the graph (think of the root as a sink).
• Pass messages “upstream”:
νj(Cj) =�
φ∈Φj
φ
δi→j =�
Ci\Si,j
νi
�
k∈Neighborsi\{j}
δk→i
Ordering can be different from the VE ordering that might have created the clique tree.
![Page 59: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/59.jpg)
Example
HS
SR
SAF FA F
Flu All.
S.I.
R.N. H.
ϕSR ϕSH
ϕAϕF
ϕFAS
ϕSR
ϕSH ϕFAS ϕFϕA
• {H, R, S, A, F}
τS’
τS
τAF τF
ψSR
ψSH
ψFAS ψFAψF
Go through math that gets done for VE here:EliminaDng S.Factor product of τS , τS’ and ϕFAS to form ψFAS.Marginalize out S
![Page 60: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/60.jpg)
Message Passing in the Clique Tree
δi→j =�
Ci\Si,j
νi
�
k∈Neighborsi\{j}
δk→i
incomingmessagesfrom otherneighbors of i
original factorsinvolving the familyof Ci
sum out variables that arenot involved in Cj
![Page 61: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/61.jpg)
Message Passing in the Clique Tree
δi→j =�
Ci\Si,j
νi
�
k∈Neighborsi\{j}
δk→i
ψ
τ
![Page 62: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/62.jpg)
Message Passing in the Clique Tree
• NoDce that we have a recursive structure here: message-‐sum of products of message-‐sums of products of message-‐sums of …
δi→j =�
Ci\Si,j
νi
�
k∈Neighborsi\{j}
δk→i
![Page 63: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/63.jpg)
Message Passing in the Clique Tree
• NoDce that you have to work “upward.”
• Root will be last.
δi→j =�
Ci\Si,j
νi
�
k∈Neighborsi\{j}
δk→i
![Page 64: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/64.jpg)
Message Passing in the Clique Tree
• NoDce that you have to work “upward.”
• Root r will be last.
βr = νr
�
k∈Neighborsr
δk→r
=�
X\Ci
�
φ∈Φ
φ
= Z · P (Cr)
Product
of ini.al
poten.
als
Messages
into roo
t
Unnormalized marginal distribuDon of root variable
s in Ci
![Page 65: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/65.jpg)
The Algorithm
• Input: clique tree T, factors Φ, root Cr• For each clique Ci, calculate νi
• While Cr is sDll waiDng on incoming messages:
– Choose a Ci that has received all of its incoming messages.
– Calculate and send the message from Ci to Cupstream-‐neighbor(i)
• Return βr
![Page 66: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/66.jpg)
Obtaining Marginals
• Given a set of query variables Q that is contained in some clique C:–Make that clique the root.– Run the upward pass of clique tree VE, resulDng in factor β.
– Return ∑C\Q β.
• Later: How to get marginals for Q that are not contained in one clique.
![Page 67: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/67.jpg)
Going Farther
• We can modify this algorithm to give us the marginal probability of every random variable in the network.– Naively: run clique tree VE with each clique as the root.
![Page 68: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/68.jpg)
HS
SR
SAF
δHS→SAF
δSR→SAF
root
![Page 69: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/69.jpg)
HS
SR
SAF
δSAF→HS
δSR→SAF
root
Same message as needed for previous root!
![Page 70: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/70.jpg)
HS
SR
SAF
δHS→SAF
δSAF→SR
root
![Page 71: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/71.jpg)
HS
SR
SAF
δHS→SAF
δSR→SAF
root
δSAF→SR
δSAF→HS
![Page 72: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/72.jpg)
Going Farther
• We can modify this algorithm to give us the marginal probability of every random variable in the network.– Naively: run clique tree VE with each clique as the root.
– Be~er: noDce that the same messages get used on different runs; only two messages per clique tree edge.
![Page 73: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/73.jpg)
Sum-‐Product Message Passing
• Each clique tree vertex Ci passes messages to each of its neighbors once it’s ready to do so.– This is asynchronous; might want to be careful about scheduling.
• At the end, for all Ci:
– This is the unnormalized marginal for Ci.
βi = νi
�
k∈Neighborsi
δk→i
![Page 74: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/74.jpg)
HS
SR
SAF
δHS→SAF
δSR→SAF
root
δSAF→SR
δSAF→HS
βSR
βHS βSAF
![Page 75: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/75.jpg)
CalibraDon
• Two adjacent cliques Ci and Cj are calibrated when: �
Ci\Si,j
βi =�
Cj\Si,j
βj
Ci CjSi,j =
Ci ∩ Cj
βi βj
![Page 76: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/76.jpg)
CalibraDon
• Two adjacent cliques Ci and Cj are calibrated when:
Ci CjSi,j =
Ci ∩ Cj
βi βj
�
Ci\Si,j
βi =�
Cj\Si,j
βj
= µi,j(Si,j)
μi,j
![Page 77: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/77.jpg)
Sum-‐Product Message Passing
• Computes the marginal probability of all variables using only twice the computaDon of the upward pass.
• Results in a calibrated clique tree.
• A~racDve if we expect different kinds of queries.
![Page 78: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/78.jpg)
Calibrated Clique Tree as a Graphical Model
• Original (unnormalized) factor model and calibrated clique tree represent the same (unnormalized) measure:
�
φ∈Φ
φ =
�
C∈Vertices(T )
βC
�
S∈Edges(T )
µS
unnormalized GibbsdistribuDon from originalfactors Φ
clique beliefs
sepset beliefs
![Page 79: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/79.jpg)
Calibrated Clique Tree as a Graphical Model
�
C∈Vertices(T )
βC
�
S∈Edges(T )
µS
=
�
C∈Vertices(T )
νC
�
C�∈Neighbors(C)
δC�→C
�
S=∈Edges(T ):S=C∩C�
µC∩C�
=
�
C∈Vertices(T )
νC
�
C�∈Neighbors(C)
δC�→C
�
S=∈Edges(T ):S=C∩C�
�
C\S
βC
=
�
C∈Vertices(T )
νC
�
C�∈Neighbors(C)
δC�→C
�
S=∈Edges(T ):S=C∩C�
�
C\S
νC
�
C��Neighbors(C)
δC��→C
![Page 80: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/80.jpg)
Calibrated Clique Tree as a Graphical Model
µi,j = δCi→Cj δCj→Ci
�
C∈Vertices(T )
βC
�
S∈Edges(T )
µS
= . . .
=
�
C∈Vertices(T )
νC
�
C�∈Neighbors(C)
δC�→C
�
S=∈Edges(T ):S=C∩C�
δC�→C
�
C\S
νC
�
C��∈Neighbors(C)\{C�}
δC��→C
=
�
C∈Vertices(T )
νC
�
C�∈Neighbors(C)
δC�→C
�
S=∈Edges(T ):S=C∩C�
δC�→CδC→C�
=�
C∈Vertices(T )
νC
=�
φ∈Φ
φ
![Page 81: Graphical)Modelsmccallum/courses/gm2011/11...Graphical)Models Lecture)11:) CliqueTrees Andrew)McCallum mccallum@cs.umass.edu Thanks)to)Noah)Smith)and))Carlos)Guestrin)for)some)slide)materials.1](https://reader034.fdocuments.in/reader034/viewer/2022042621/5f67d194febf52303c2a0ef5/html5/thumbnails/81.jpg)
What You Now Know
• Can reinterpret variable eliminaDon as message passing in a clique tree.
• Can share computaDon to get lots of marginals with only double the cost of VE.– Sum-‐product belief propagaDon
• Calibrated clique tree is an alternaDve representaDon of the original Gibbs distribuDon.