1 Inferring structure to make substantive conclusions: How does it work? Hypothesis testing...
-
Upload
adele-mcgee -
Category
Documents
-
view
214 -
download
0
Transcript of 1 Inferring structure to make substantive conclusions: How does it work? Hypothesis testing...
1
Inferring structure to make substantive conclusions:How does it work?
Hypothesis testing approaches:
Tests on deviances, possibly penalised (AIC/BIC, etc.), MDL, cross-validation...
Problem is how to search model space when dimension is large
2
Inferring structure to make substantive conclusions:How does it work? Bayesian approaches:
Typically place prior on all graphs, and conjugate prior on parameters (hyper-Markov laws, Dawid & Lauritzen), then use MCMC to update both graphs and parameters to simulate posterior distribution
3
Graph moves
Giudici & Green (Biometrika, 1999) develop a full Bayesian methodology for model selection in Gaussian models, assuming
decomposability
(= graph triangulated
= no chordless
-cycles)
7 6 5
2 3 414
4
• How many graphs are decomposable?
• Models using decomposable graphs are ‘dense’
Number ofvertices
Proportion of graphsthat aredecomposable
3 all 4 61/64 – all but: 6 ~80% 16 ~45%
Is decomposability a serious constraint?
22n
out of
5
Is decomposability any use?
• Maximum likelihood estimates can be computed exactly in decomposable models
• Decomposability is a key to the ‘message passing’ algorithms for probabilistic expert systems (and peeling genetic pedigrees)
lj
jkllijijkl n
nnNE
)(ˆ1 2
4 3
6
Graph moves
We can traverse graph space by adding and deleting single edges
Some are OK,but others makegraphnon-decomposable
7 6 5
2 3 41
7
Graph moves
Frydenberg & Lauritzen (1989) showed that all decomposable graphs are connected by single-edge moves
Can we test formaintaining decomposabilitybefore committing tomaking the change?
7 6 5
2 3 41
8
Cliques
A clique is a maximal complete subgraph: here the cliques are {1,2},{2,6,7}, {2,3,6}, and {3,4,5,6}
7 6 5
2 3 41
9
Deleting edges?
Deleting an edge maintains decomposability if and only if it is contained in exactly one clique of the current graph (Frydenberg & Lauritzen)
7 6 5
2 3 41
10
7 6 5
2 3 41
12
267 236 345626 36
2
a cliqueanother cliquea separator
The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree
A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)
11
7 6 5
2 3 41
12
267 236 345626 36
2
a cliqueanother cliquea separator
A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)
The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree
14
Adding edges? (Giudici & Green)
Adding an edge (a,b) maintains decomposability if and only if either:
7 6 5
2 3 41
• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them
• a and b are in different connected components, or
15
You can add edge (1,7) since 1R and 7T are cliques (with R={2} and T={2,6}) and RT={2} is a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
16
You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
17
Adding edges? (Giudici & Green)
Adding an edge (a,b) maintains decomposability if and only if either:
7 6 5
2 3 41
• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them
• a and b are in different connected components, or
18
Proof (in connected case)
First suppose that there are no such sets R and T. We have to show that adding edge (a,b) makes graph non-deomposable.Let aR and bT be the cliques containing a and b that have shortest connecting path in the junction tree: by assumption, RT is not a separator (it may be empty): so all separators on the path are proper supersets of RT. So there is a shortest path in the original graph: arv1...vktb with k0, rR\T, tT\R and all v’s RT. Joining (a,b) will make a chordless (k+4)-cycle, making the graph non-decomposable.
19
You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
20
Proof (in connected case)
S bSaS
abS
Conversely, suppose such sets R and T do exist.We can suppose aR and bT are adjacent in the junction tree (otherwise it is quite easy to show that the junction tree can be manipulated until this is true).Let S=RT, P=R\T and Q=T\R. There are 4 cases according to whether P and Q are empty or not.
Both P and Q empty:
(it is easy to see that you still have a tree & that running intersection property is maintained)
21
S bSaSP
abS bS bSQaSaSP
S bSQaS
S bSQaSP
aS abSaSP bS bSQabS
Neither P nor Q empty:
Only Q empty: Only P empty:
22
Once the test is complete, actually committing to adding or deleting the edge is little work
7 6 5
2 3 41
12
267 236 345626 36
2
23
7 6 5
2 3 41
127
267 236 345626 36
27
12
2
It makes onlya (relatively)local change to the junction tree
Once the test is complete, actually committing to adding or deleting the edge is little work
24
7 6 5
2 3 41
127
267 236 345626 36
27
It makes onlya (relatively)local change to the junction tree
6
Once the test is complete, actually committing to adding or deleting the edge is little work