SNML-TNG: Linked Media Interfaces. Graphical User Interfaces for Search and Annotation
AND/OR search spaces for graphical models
description
Transcript of AND/OR search spaces for graphical models
![Page 1: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/1.jpg)
AND/OR search spaces for graphical models
Vibhav Gogate(Slides courtesy of Prof. Rina Dechter, UC Irvine
and her past and present students)
![Page 2: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/2.jpg)
Probabilistic Inference Tasks Probability of evidence and Partition function
Belief updating:
Finding most probable explanation (MPE)
Finding maximum a-posteriory hypothesisX/Aa
*k
*1 e),xP(maxarg)a,...,(a
evidence)|xP(X)BEL(X iii
e),xP(maxarg*xx
variableshypothesis: XA
𝑷 (𝒆𝒗𝒊𝒅𝒆𝒏𝒄𝒆 )
![Page 3: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/3.jpg)
Belief Updating and Probability of Evidence
lung Cancer
Smoking
X-ray
Bronchitis
Dyspnoea
P(lung cancer=yes | smoking=no, dyspnoea=yes ) = ?P(smoking=no, dyspnoea=yes ) = ?
![Page 4: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/4.jpg)
Recap
• Bucket elimination (BE)• Junction tree propagation• Treewidth (w*) and Tree Decompositions
– BE is time and space exponential in (w*+1).• Why is it cool?
– The complexity of exact inference can be determined by manipulating the (undirected) interaction graph
• also called primal graph, moral graph and markov network
![Page 5: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/5.jpg)
Exact Graphical Models Inference
Eliminationand propagation
Bucket eliminationJunction tree propagationDynamic Programming
Time: exp(w*)Space: exp(w*)
Hybrids:AND/OR search
spaces
Search: ConditioningTime: exp(n)Space: linear
Depth-first search
![Page 6: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/6.jpg)
Conditioning generates the probability tree
0
),|(),|()|()|()()0,(ebcb
cbePbadPacPabPaPeaP
Complexity of conditioning: exponential time, linear space
![Page 7: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/7.jpg)
Conditioning+Elimination
0
),|(),|()|()|()()0,(edcb
cbePbadPacPabPaPeaP
Idea: conditioning until of a (sub)problem gets small*w
![Page 8: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/8.jpg)
OR search spaceA
D
B C
E
F
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
E
C
F
D
B
A 0 1
Ordering: A B E C D F
![Page 9: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/9.jpg)
AND/OR search space
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
A
D
B C
E
F
A
D
B
CE
F
Primal graph DFS tree
A
D
B C
E
F
A
D
B C
E
F
![Page 10: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/10.jpg)
OR vs AND/OR AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
1
1
1
0
1
0
Zero probability(A=1,B=1)(B=0,C=0)
![Page 11: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/11.jpg)
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
AND/OR size: exp(4), OR size exp(6)
Zero probability(A=1,B=1)(B=0,C=0)
![Page 12: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/12.jpg)
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
Zero probability(A=1,B=1)(B=0,C=0)
![Page 13: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/13.jpg)
AND/OR vs. OR
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AOR
0AND 1
BOR B
0AND 1 0
EOR C E C E C
OR D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 10 1 0 10 1 10 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
(A=1,B=1)(B=0,C=0)
![Page 14: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/14.jpg)
OR space vs. AND/OR space
width heightOR space AND/OR space
time(sec.) nodes backtracks time(sec.) AND nodes OR nodes
5 10 3.154 2,097,150 1,048,575 0.03 10,494 5,247
4 9 3.135 2,097,150 1,048,575 0.01 5,102 2,551
5 10 3.124 2,097,150 1,048,575 0.03 8,926 4,463
4 10 3.125 2,097,150 1,048,575 0.02 7,806 3,903
5 13 3.104 2,097,150 1,048,575 0.1 36,510 18,255
5 10 3.125 2,097,150 1,048,575 0.02 8,254 4,127
6 9 3.124 2,097,150 1,048,575 0.02 6,318 3,159
5 10 3.125 2,097,150 1,048,575 0.02 7,134 3,567
5 13 3.114 2,097,150 1,048,575 0.121 37,374 18,687
5 10 3.114 2,097,150 1,048,575 0.02 7,326 3,663
![Page 15: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/15.jpg)
AND/OR search tree for graphical models
• The AND/OR search tree of R relative to a spanning-tree, T, has:– Alternating levels of: OR nodes (variables) and AND nodes (values)
• Successor function:– The successors of OR nodes X are all its consistent values along its path– The successors of AND <X,v> are all X child variables in T
• A solution is a consistent subtree• Task: compute the value of the root node
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
B
0
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
A
D
B C
E
F
A
D
B
CE
F
![Page 16: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/16.jpg)
From DFS trees to pseudo-trees (Freuder 85, Bayardo 95)
(a) Graph
4 61
3 2 7 5
(b) DFS treedepth=3
(c) pseudo- treedepth=2
(d) Chaindepth=6
4 6
1
3
2 7
5 2 7
1
4
3 5
6
4
6
1
3
2
7
5
![Page 17: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/17.jpg)
From DFS trees to Pseudo-trees
a
1
2
a
3
a
4
OR
AND
OR
AND
OR
AND
AND
OR
ba
b c
c
b
4
ba c
c
4
ba c
b
3
a
4
ba c
b
4
ba c
c
4
ba c
c
3
a
4
ba c
b
4
ba c
c
4
ba c
7
a
5
a
6
ba c
b
6
ba c
c
6
ba c
b
5
a
6
ba c
b
6
ba c
c
6
ba c
c
5
a
6
ba c
b
6
ba c
c
6
ba c
a
1
3
a
4
b c
OR
AND
OR
AND
OR
AND
b c
a
2
b ca
b
4
b ca
2
b ca
c
4
b ca
2
b ca
5
a
7
b ca
6
b ca
b
7
b ca
6
b ca
c
7
b ca
6
b ca
DFS treedepth = 3
pseudo- treedepth = 2
![Page 18: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/18.jpg)
Finding min-depth backbone trees
• Finding min depth DFS, or pseudo tree is NP-complete, but:
• Given a tree-decomposition whose tree-width is w*, there exists a pseudo tree T of G whose depth, satisfies (Bayardo and Mirankar, 1996, bodlaender and Gilbert, 91):
m <= w* log n,
![Page 19: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/19.jpg)
Generating pseudo-trees from Bucket trees
FAC
B DE
E
A
C
B
D
F (AF) (EF)
(A)
(AB)
(AC) (BC)
(AE)
(BD) (DE)
Bucket-tree based on dreverse elim order: A B C E D F
E
A
C
B
D
F
Induced graph
E
A
C
B
D F
Bucket-tree used as pseudo-tree
AND
AND
AND
AND
0 1
BOR B
0 1 0 1
COR E C E C E C E
OR D F D F D F D F D F D F D F D F
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 10 1 0 10 1 0 10 1 0 10 1
AOR
AND/OR search tree
Bucket-tree
ABE
A
ABC
AB
BDE AEF
bucket-A
bucket-E
bucket-B
bucket-C
bucket-D bucket-F
(AE) (BE)
![Page 20: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/20.jpg)
AND/OR Search-tree properties (k = domain size, m = pseudo-tree depth. n = number of variables)
• Theorem: Any AND/OR search tree based on a pseudo-tree is sound and complete (expresses all and only solutions)
• Theorem: Size of AND/OR search tree is O(n km)Size of OR search tree is O(kn)
• Theorem: Size of AND/OR search tree can be bounded by O(exp(w* log n))
• Related to: (Freuder 85; Dechter 90, Bayardo et. al. 96, Darwiche 1999, Bacchus 2003)
• Generalizes OR search: When the pseudo-tree is a chain we get an OR space
![Page 21: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/21.jpg)
AND/OR Tree DFS Algorithm (Probability of evidence)
AND node: Combination operator (product)
OR node: Marginalization operator (weighted summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
A
D
B C
E
A
D
B
CE
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.8 .9
.8 .9
.7 .5
.7 .5
.8 .9
.8 .9
.7 .5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
![Page 22: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/22.jpg)
Complexity of AND/OR Tree Search
AND/OR tree OR treeSpace O(n) O(n)
TimeO(n km)
O(n kw* log n)[Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95], [Darwiche01]
O(kn)
k = domain sizem = depth of pseudo-treen = number of variablesw*= treewidth
![Page 23: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/23.jpg)
From Search Trees to Search Graphs
• Any two nodes that root identical subtrees (subgraphs) can be merged
![Page 24: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/24.jpg)
From Search Trees to Search Graphs
• Any two nodes that root identical subtrees (subgraphs) can be merged
![Page 25: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/25.jpg)
AND/OR TreeA
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
![Page 26: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/26.jpg)
An AND/ORgraph
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
![Page 27: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/27.jpg)
Merging Based on Context
One way of recognizing nodes that can be merged:
context (X) = ancestors of X in pseudo tree that are connected to X, or to descendants of X
[ ]
[A]
[AB]
[AE][BC]
[AB]
A
D
B
EC
F
pseudo tree
A
E
C
B
F
D
A
E
C
B
F
D
![Page 28: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/28.jpg)
AND/OR Tree DFS Algorithm
.8 .9 .7.7.8
AND node: Combination operator (product)OR node: Marginalization operator (summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9
.5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
ContextA
D
B C
E
![Page 29: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/29.jpg)
AND/OR Graph DFS Algorithm
.7.8
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0 10 1
1
E C
0 10 1
A
D
B C
E
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP.7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9 .7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
Context
B C Value0 0 .80 1 .91 0 .71 1 .1
Cache table for D
Result: P(D=1,E=0)
![Page 30: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/30.jpg)
AND/OR context minimal graph C
0
K0
H0
L0 1
N N
0 1 0 1
F F F
1 1
0 1 0 1
F
G
0 1
1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
G
0 1
G
0 1
G
0 1
M
0 1
M
0 1
M
0 1
M
0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
K0
H0
L0 1
N N
0 1 0 1
1 1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
B A
C
E
F G
HJ
D
K ML
N
OP
C
HK
D
M
F
G
A
B
E
J
O
L
N
P[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
C
HK
D
M
F
G
A
B
E
J
O
L
N
P[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
![Page 31: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/31.jpg)
How Big Is the Context?
Theorem: The maximum context size for a pseudo tree is equal to the treewidth of the graph along the pseudo tree.
C
HK
D
M
F
G
A
B
E
J
O
L
N
P[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
B A
C
E
F G
HJ
D
K ML
N
OP
max context size = treewidth
![Page 32: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/32.jpg)
AND/OR search algorithms
• AO(i) – i = the max size of a cache table (i.e. number of
variables in a context)
i = 0 i = w* treewidth
Space:Time:
O(n exp w* )
O(n exp w* )
O( n )
O( exp (w* log n) )
![Page 33: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/33.jpg)
Searching AND/OR Graphs
• AO(i): searches depth-first, cache i-context – i = the max size of a cache table (i.e. number of
variables in a context)
i=0
i=w*
Space: O(n) Time: O(exp(w* log n))
Space: O(exp w*) Time: O(exp w*)
Space: O(exp(i) )Time: O(exp(m_i+i )
i
![Page 34: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/34.jpg)
CachingA
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}context(F)={F}
![Page 35: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/35.jpg)
CachingA
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}context(F)={F}
![Page 36: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/36.jpg)
All four search spaces
EC
FD
BA
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
EC
FD
BA 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1
AND 0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
![Page 37: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/37.jpg)
All four search spaces
EC
FD
BA
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
EC
FD
BA 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1
AND 0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
Time-space
![Page 38: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/38.jpg)
Available code
• http://graphmod.ics.uci.edu/group/Software
![Page 39: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/39.jpg)
SummaryType AND/OR Tree AND/OR graphStructure Pseudo tree Pseudo tree with
contextComputations Assign labels,
compute valuesAssign labels, compute values
Time Complexity O(n exp(h)) [max h is w*logn]
O(n exp(w*))
Space Complexity O(h) O(n exp(w*))Caching No Yes
![Page 40: AND/OR search spaces for graphical models](https://reader036.fdocuments.in/reader036/viewer/2022062305/568164cf550346895dd6f901/html5/thumbnails/40.jpg)
Research issues
• Orderings– Heuristically Constructing a best pseudo tree
• Caching strategies• Exploiting Symmetries
– Exploiting Determinism and context-specific independence
• Approximations• Dynamic vs Static Search