Graphical Models for chains, trees and grids
description
Transcript of Graphical Models for chains, trees and grids
![Page 1: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/1.jpg)
Graphical Models for Chains, Trees, and Grids
Gabriel Brostow UCL
![Page 2: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/2.jpg)
Sources
• Book and slides by Simon Prince: “Computer vision: models, learning
and inference” (June 2012) • See more on
www.computervisionmodels.com
2
![Page 3: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/3.jpg)
Part 1: Graphical Models for Chains and Trees
3
![Page 4: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/4.jpg)
Part 1 Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
4 4 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Extra
Extra
![Page 5: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/5.jpg)
Example Problem: Pictorial Structures
5 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 6: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/6.jpg)
Chain and tree models • Given a set of measurements and world states , infer the world states from the measurements.
• Problem: if N is large, then the model relaOng the two will have a very large number of parameters.
• SoluOon: build sparse models where we only describe subsets of the relaOons between variables.
6 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 7: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/7.jpg)
Chain and tree models
7 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Chain model: only model connecOons between a world variable and its 1 preceeding and 1 subsequent variables
Tree model: connecOons between world variables are organized as a tree (no loops). Disregard direcOonality of connecOons for directed model
![Page 8: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/8.jpg)
AssumpOons
8 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
We’ll assume that – World states are discrete
– Observed data variables for each world state – The nth data variable is condi&onally
independent of all other data variables and world states, given associated world state
![Page 9: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/9.jpg)
See also: Thad Starner’s work
Gesture Tracking
9 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 10: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/10.jpg)
Directed model for chains (Hidden Markov model)
10 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
CompaObility of measurement and world state
CompaObility of world state and previous world state
![Page 11: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/11.jpg)
Undirected model for chains
11 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
CompaObility of measurement and world state
CompaObility of world state and previous world state
![Page 12: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/12.jpg)
Equivalence of chain models
12 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Directed:
Undirected:
Equivalence:
![Page 13: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/13.jpg)
Chain model for sign language applicaOon
13 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
ObservaOons are normally distributed but depend on sign k
World state is categorically distributed, parameters depend on previous world state
![Page 14: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/14.jpg)
Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
14 14 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 15: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/15.jpg)
MAP inference in chain model
15 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
MAP inference:
SubsOtuOng in :
Directed model:
![Page 16: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/16.jpg)
MAP inference in chain model
16 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Takes the general form:
Unary term:
Pairwise term:
![Page 17: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/17.jpg)
Dynamic programming
17 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Maximizes funcOons of the form:
Set up as cost for traversing graph – each path from le` to right is one possible configuraOon of world states
![Page 18: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/18.jpg)
Dynamic programming
18 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Algorithm: 1. Work through graph compuOng minimum possible cost to reach each node 2. When we get to last column, find minimum 3. Trace back to see how we got there
![Page 19: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/19.jpg)
Worked example
19 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Unary cost Pairwise costs: • Zero cost to stay at same label • Cost of 2 to change label by 1 • Infinite cost for changing by more
than one (not shown)
![Page 20: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/20.jpg)
Worked example
20 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Minimum cost to reach first node is just unary cost
![Page 21: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/21.jpg)
Worked example
21 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Minimum cost is minimum of two possible routes to get here Route 1: 2.0+0.0+1.1 = 3.1 Route 2: 0.8+2.0+1.1 = 3.9
![Page 22: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/22.jpg)
Worked example
22 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Minimum cost is minimum of two possible routes to get here Route 1: 2.0+0.0+1.1 = 3.1 -‐-‐ this is the minimum – note this down Route 2: 0.8+2.0+1.1 = 3.9
![Page 23: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/23.jpg)
Worked example
23 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
General rule:
![Page 24: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/24.jpg)
Worked example
24 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Work through the graph, compuOng the minimum cost to reach each node
![Page 25: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/25.jpg)
Worked example
25 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Keep going unOl we reach the end of the graph
![Page 26: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/26.jpg)
Worked example
26 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Find the minimum possible cost to reach the final column
![Page 27: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/27.jpg)
Worked example
27 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Trace back the route that we arrived here by – this is the minimum configuraOon
![Page 28: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/28.jpg)
Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
28 28 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 29: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/29.jpg)
MAP inference for trees
29 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 30: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/30.jpg)
MAP inference for trees
30 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 31: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/31.jpg)
Worked example
31 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 32: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/32.jpg)
Worked example
32 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Variables 1-‐4 proceed as for the chain example.
![Page 33: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/33.jpg)
Worked example
33 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
At variable n=5 must consider all pairs of paths from into the current node.
![Page 34: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/34.jpg)
Worked example
34 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Variable 6 proceeds as normal. Then we trace back through the variables, splilng at the juncOon.
![Page 35: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/35.jpg)
Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
35 35 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Extra
Extra
Jump there
![Page 36: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/36.jpg)
Marginal posterior inference
36 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Start by compuOng the marginal distribuOon over the Nth variable
• Then we`ll consider how to compute the other marginal distribuOons
![Page 37: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/37.jpg)
CompuOng one marginal distribuOon
37 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Compute the posterior using Bayes` rule:
We compute this expression by wriOng the joint probability :
![Page 38: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/38.jpg)
CompuOng one marginal distribuOon
38 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problem: CompuOng all NK states and marginalizing explicitly is intractable. SoluOon: Re-‐order terms and move summaOons to the right
![Page 39: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/39.jpg)
CompuOng one marginal distribuOon
39 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Define funcOon of variable w1 (two rightmost terms)
Then compute funcOon of variables w2 in terms of previous funcOon
Leads to the recursive relaOon
![Page 40: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/40.jpg)
CompuOng one marginal distribuOon
40 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
We work our way through the sequence using this recursion. At the end we normalize the result to compute the posterior
Total number of summaOons is (N-‐1)K as opposed to KN for brute force approach.
![Page 41: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/41.jpg)
Forward-‐backward algorithm
41 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• We could compute the other N-‐1 marginal posterior distribuOons using a similar set of computaOons
• However, this is inefficient, as much of the computaOon is duplicated
• The forward-‐backward algorithm computes all of the marginal posteriors at once
SoluOon:
Compute all first term using a recursion
Compute all second terms using a recursion
... and take products
![Page 42: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/42.jpg)
Forward recursion
42 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Using condiOonal independence relaOons
CondiOonal probability rule
This is the same recursion as before
![Page 43: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/43.jpg)
Backward recursion
43 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Using condiOonal independence
relaOons
CondiOonal probability rule
This is another recursion of the form
![Page 44: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/44.jpg)
Forward backward algorithm
44 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Compute the marginal posterior distribuOon as product of two terms
Forward terms: Backward terms:
![Page 45: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/45.jpg)
Belief propagaOon
45 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Forward backward algorithm is a special case of a more general technique called belief propagaOon
• Intermediate funcOons in forward and backward recursions are considered as messages conveying beliefs about the variables.
• We’ll examine the Sum-‐Product algorithm.
• The sum-‐product algorithm operates on factor graphs.
![Page 46: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/46.jpg)
Sum product algorithm
46 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Forward backward algorithm is a special case of a more general technique called belief propagaOon
• Intermediate funcOons in forward and backward recursions are considered as messages conveying beliefs about the variables.
• We’ll examine the Sum-‐Product algorithm.
• The sum-‐product algorithm operates on factor graphs.
![Page 47: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/47.jpg)
Factor graphs
47 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• One node for each variable • One node for each funcOon relaOng variables
![Page 48: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/48.jpg)
Sum product algorithm
48 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Forward pass • Distribute evidence through the graph
Backward pass • Collates the evidence
Both phases involve passing messages between nodes: • The forward phase can proceed in any order as long
as the outgoing messages are not sent unOl all incoming ones received
• Backward phase proceeds in reverse order to forward
![Page 49: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/49.jpg)
Sum product algorithm
49 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Three kinds of message • Messages from unobserved variables to funcOons • Messages from observed variables to funcOons • Messages from funcOons to variables
![Page 50: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/50.jpg)
Sum product algorithm
50 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message type 1: • Messages from unobserved variables z to funcOon g
• Take product of incoming messages • InterpretaOon: combining beliefs
Message type 2: • Messages from observed variables z to funcOon g
• InterpretaOon: conveys certain belief that observed values are true
![Page 51: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/51.jpg)
Sum product algorithm
51 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message type 3: • Messages from a funcOon g to variable z
• Takes beliefs from all incoming variables except recipient and uses funcOon g to a belief about recipient
CompuOng marginal distribuOons: • A`er forward and backward passes, we compute the
marginal dists as the product of all incoming messages
![Page 52: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/52.jpg)
Sum product: forward pass
52 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from x1 to g1: By rule 2:
![Page 53: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/53.jpg)
Sum product: forward pass
53 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from g1 to w1: By rule 3:
![Page 54: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/54.jpg)
Sum product: forward pass
54 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from w1 to g1,2: By rule 1: (product of all incoming messages)
![Page 55: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/55.jpg)
Sum product: forward pass
55 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from g1,2 from w2: By rule 3:
![Page 56: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/56.jpg)
Sum product: forward pass
56 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Messages from x2 to g2 and g2 to w2:
![Page 57: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/57.jpg)
Sum product: forward pass
57 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from w2 to g2,3:
The same recursion as in the forward backward algorithm
![Page 58: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/58.jpg)
Sum product: forward pass
58 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from w2 to g2,3:
![Page 59: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/59.jpg)
Sum product: backward pass
59 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from wN to gN,N-1:
![Page 60: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/60.jpg)
Sum product: backward pass
60 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from gN,N-1 to wN-1:
![Page 61: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/61.jpg)
Sum product: backward pass
61 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Message from gn,n-1 to wn-1:
The same recursion as in the forward backward algorithm
![Page 62: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/62.jpg)
Sum product: collaOng evidence
62 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Marginal distribuOon is products of all messages at node
• Proof:
![Page 63: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/63.jpg)
Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
63 63 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 64: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/64.jpg)
Marginal posterior inference for trees
64 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Apply sum-‐product algorithm to the tree-‐structured graph.
![Page 65: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/65.jpg)
Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
65 65 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 66: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/66.jpg)
Tree structured graphs
66 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
This graph contains loops But the associated factor graph has structure of a tree
Can sOll use Belief PropagaOon
![Page 67: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/67.jpg)
Learning in chains and trees
67 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Supervised learning (where we know world states wn) is relaOvely easy.
Unsupervised learning (where we do not know world states wn) is more challenging. Use the EM algorithm: • E-‐step – compute posterior marginals over
states • M-‐step – update model parameters
For the chain model (hidden Markov model) this is known as the Baum-‐Welch algorithm.
![Page 68: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/68.jpg)
Grid-‐based graphs
68 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
O`en in vision, we have one observaOon associated with each pixel in the image grid.
![Page 69: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/69.jpg)
Why not dynamic programming?
69 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
When we trace back from the final node, the paths are not guaranteed to converge.
![Page 70: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/70.jpg)
Why not dynamic programming?
70 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 71: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/71.jpg)
Why not dynamic programming?
71 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
But:
![Page 72: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/72.jpg)
Approaches to inference for grid-‐based models
72 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
1. Prune the graph.
Remove edges unOl an edge remains
![Page 73: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/73.jpg)
73 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
2. Combine variables. Merge variables to form compound variable with more states unOl what remains is a tree.
Not pracOcal for large grids
Approaches to inference for grid-‐based models
![Page 74: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/74.jpg)
74 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Approaches to inference for grid-‐based models
3. Loopy belief propagaOon.
Just apply belief propagaOon. It is not guaranteed to converge, but in pracOce it works well.
4. Sampling approaches
Draw samples from the posterior (easier for directed models) 5. Other approaches
• Tree-‐reweighted message passing • Graph cuts
![Page 75: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/75.jpg)
Structure
• Chain and tree models • MAP inference in chain models • MAP inference in tree models • Maximum marginals in chain models • Maximum marginals in tree models • Models with loops • ApplicaOons
75 75 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 76: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/76.jpg)
76 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Gesture Tracking
![Page 77: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/77.jpg)
Stereo vision
77 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Two image taken from slightly different posiOons • Matching point in image 2 is on same scanline as image 1 • Horizontal offset is called disparity • Disparity is inversely related to depth • Goal – infer dispariOes wm,n at pixel m,n from images x(1) and x(2) Use likelihood:
![Page 78: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/78.jpg)
Stereo vision
78 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 79: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/79.jpg)
Stereo vision
79 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
1. Independent pixels
![Page 80: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/80.jpg)
Stereo vision
80 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
2. Scanlines as chain model (hidden Markov model)
![Page 81: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/81.jpg)
Stereo vision
81 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
3. Pixels organized as tree (from Veksler 2005)
![Page 82: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/82.jpg)
Pictorial Structures
82 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 83: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/83.jpg)
SegmentaOon
83 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 84: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/84.jpg)
Part 1 Conclusion
84 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• For the special case of chains and trees we can perform MAP inference and compute marginal posteriors efficiently.
• Unfortunately, many vision problems are defined on pixel grids – this requires special methods
![Page 85: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/85.jpg)
Part 2: Graphical Models for Grids
85
![Page 86: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/86.jpg)
86 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Stereo vision
Example ApplicaOon
![Page 87: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/87.jpg)
Part 2 Structure
• Denoising problem • Markov random fields (MRFs) • Max-‐flow / min-‐cut • Binary MRFs -‐ submodular (exact soluOon) • MulO-‐label MRFs – submodular (exact soluOon) • MulO-‐label MRFs -‐ non-‐submodular (approximate)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 87
![Page 88: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/88.jpg)
Models for grids
88 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Consider models with one unknown world state at each pixel in the image – takes the form of a grid.
• Loops in the graphical model, so cannot use dynamic programming or belief propagaOon
• Define probability distribuOons that favor certain configuraOons of world states – Called Markov random fields – Inference using a set of techniques called graph cuts
![Page 89: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/89.jpg)
89 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Binary Denoising
Before A`er Image represented as binary discrete variables. Some proporOon of pixels
randomly changed polarity.
![Page 90: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/90.jpg)
90 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
MulO-‐label Denoising
Before A`er Image represented as discrete variables represenOng intensity. Some
proporOon of pixels randomly changed according to a uniform distribuOon.
![Page 91: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/91.jpg)
91 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Denoising Goal
Observed Data Uncorrupted Image
![Page 92: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/92.jpg)
• Most of the pixels stay the same • Observed image is not as smooth as original Now consider pdf over binary images that encourages smoothness – Markov random field
92 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Denoising Goal
Observed Data Uncorrupted Image
![Page 93: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/93.jpg)
93 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Markov random fields
This is just the typical property of an undirected model. We’ll conOnue the discussion in terms of undirected models
![Page 94: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/94.jpg)
94 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Markov random fields
Normalizing constant (parOOon funcOon) PotenOal funcOon
Returns posiOve number
Subset of variables (clique)
![Page 95: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/95.jpg)
95 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Markov random fields
Normalizing constant (parOOon funcOon)
Cost funcOon Returns any number
Subset of variables (clique) RelaOonship
![Page 96: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/96.jpg)
96 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Smoothing Example
![Page 97: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/97.jpg)
97 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Smoothing Example
Smooth soluOons (e.g. 0000,1111) have high probability Z was computed by summing the 16 un-‐normalized probabiliOes
![Page 98: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/98.jpg)
98 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Smoothing Example
Samples from larger grid -‐-‐ mostly smooth Cannot compute parOOon funcOon Z here -‐ intractable
![Page 99: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/99.jpg)
99 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Denoising Goal
Observed Data Uncorrupted Image
![Page 100: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/100.jpg)
100 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Denoising overview Bayes’ rule:
Likelihoods:
Prior: Markov random field (smoothness)
MAP Inference: Graph cuts
Probability of flipping polarity
![Page 101: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/101.jpg)
101 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Denoising with MRFs
Observed image, x
Original image, w
MRF Prior (pairwise cliques)
Inference :
Likelihoods
![Page 102: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/102.jpg)
MAP Inference
102 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Unary terms (compatability of data with label y)
Pairwise terms (compatability of neighboring labels)
![Page 103: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/103.jpg)
103 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Graph Cuts Overview
Unary terms (compatability of data with label y)
Pairwise terms (compatability of neighboring labels)
Graph cuts used to opOmise this cost funcOon:
Three main cases:
![Page 104: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/104.jpg)
104 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Graph Cuts Overview
Unary terms (compatability of data with label y)
Pairwise terms (compatability of neighboring labels)
Graph cuts used to opOmise this cost funcOon:
Approach: Convert minimizaOon into the form of a standard CS problem,
MAXIMUM FLOW or MINIMUM CUT ON A GRAPH Polynomial-‐Ome methods for solving this problem are known
![Page 105: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/105.jpg)
105 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Max-‐Flow Problem
Goal: To push as much ‘flow’ as possible through the directed graph from the source to the sink. Cannot exceed the (non-‐negaOve) capaciOes cij associated with each edge.
![Page 106: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/106.jpg)
106 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Saturated Edges
When we are pushing the maximum amount of flow: • There must be at least one saturated edge on any path from source to sink
(otherwise we could push more flow) • The set of saturated edges hence separate the source and sink
![Page 107: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/107.jpg)
107 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
Two numbers represent: current flow / total capacity
![Page 108: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/108.jpg)
Choose any route from source to sink with spare capacity, and push as much flow as you can. One edge (here 6-‐t) will saturate. 108
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
![Page 109: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/109.jpg)
109 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
Choose another route, respecOng remaining capacity. This Ome edge 6-‐5 saturates.
![Page 110: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/110.jpg)
110 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
A third route. Edge 1-‐4 saturates
![Page 111: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/111.jpg)
111 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
A fourth route. Edge 2-‐5 saturates
![Page 112: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/112.jpg)
112 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
A fi`h route. Edge 2-‐4 saturates
![Page 113: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/113.jpg)
There is now no further route from source to sink – there is a saturated edge along every possible route (highlighted arrows) 113
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
![Page 114: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/114.jpg)
114 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AugmenOng Paths
The saturated edges separate the source from the sink and form the min-‐cut soluOon. Nodes either connect to the source or connect to the sink.
![Page 115: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/115.jpg)
Graph Cuts: Binary MRF
Unary terms (compatability of data with label w)
Pairwise terms (compatability of neighboring labels)
Graph cuts used to opOmise this cost funcOon:
First work with binary case (i.e. True label w is 0 or 1) Constrain pairwise costs so that they are “zero-‐diagonal”
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 115
![Page 116: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/116.jpg)
Graph ConstrucOon • One node per pixel (here a 3x3 image) • Edge from source to every pixel node • Edge from every pixel node to sink • Reciprocal edges between neighbours
Note that in the minimum cut EITHER the edge connecOng to the source will be cut, OR the edge connecOng to the sink, but NOT BOTH (unnecessary). Which determines whether we give that pixel label 1 or label 0. Now a 1 to 1 mapping between possible labelling and possible minimum cuts
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 116
![Page 117: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/117.jpg)
Graph ConstrucOon Now add capaciOes so that minimum cut, minimizes our cost funcOon Unary costs U(0), U(1) avached to links to source and sink. • Either one or the other is paid. Pairwise costs between pixel nodes as shown. • Why? Easiest to understand
with some worked examples. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
117
![Page 118: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/118.jpg)
Example 1
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 118
![Page 119: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/119.jpg)
Example 2
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 119
![Page 120: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/120.jpg)
Example 3
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 120
![Page 121: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/121.jpg)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 121
![Page 122: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/122.jpg)
Graph Cuts: Binary MRF
Unary terms (compatability of data with label w)
Pairwise terms (compatability of neighboring labels)
Graph cuts used to opOmise this cost funcOon:
Summary of approach
• Associate each possible soluOon with a minimum cut on a graph • Set capaciOes on graph, so cost of cut matches the cost funcOon • Use augmenOng paths to find minimum cut • This minimizes the cost funcOon and finds the MAP soluOon
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 122
![Page 123: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/123.jpg)
General Pairwise costs
Modify graph to • Add P(0,0) to edge s-‐b
• Implies that soluOons 0,0 and 1,0 also pay this cost
• Subtract P(0,0) from edge b-‐a • SoluOon 1,0 has this cost
removed again
Similar approach for P(1,1)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 123
![Page 124: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/124.jpg)
ReparameterizaOon
The max-‐flow / min-‐cut algorithms require that all of the capaciOes are non-‐negaOve. However, because we have a subtracOon on edge a-‐b we cannot guarantee that this will be the case, even if all the original unary and pairwise costs were posiOve. The soluOon to this problem is reparamaterizaOon: find new graph where costs (capaciOes) are different but choice of minimum soluOon is the same (usually just by adding a constant to each soluOon)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 124
![Page 125: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/125.jpg)
ReparameterizaOon 1
The minimum cut chooses the same links in these two graphs Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 126: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/126.jpg)
ReparameterizaOon 2
The minimum cut chooses the same links in these two graphs Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
126
![Page 127: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/127.jpg)
Submodularity
Adding together implies
Subtract constant β Add constant, β
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 127
![Page 128: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/128.jpg)
Submodularity
If this condiOon is obeyed, it is said that the problem is “submodular” and it can be solved in polynomial Ome. If it is not obeyed then the problem is NP hard. Usually it is not a problem as we tend to favour smooth soluOons.
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 128
![Page 129: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/129.jpg)
Denoising Results
Original Pairwise costs increasing
Pairwise costs increasing Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
129
![Page 130: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/130.jpg)
Plan of Talk
• Denoising problem • Markov random fields (MRFs) • Max-‐flow / min-‐cut • Binary MRFs – submodular (exact soluOon) • MulO-‐label MRFs – submodular (exact soluOon) • MulO-‐label MRFs -‐ non-‐submodular (approximate)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 130
![Page 131: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/131.jpg)
ConstrucOon for two pixels (a and b) and four labels (1,2,3,4) There are 5 nodes for each pixel and 4 edges between them have unary costs for the 4 labels. One of these edges must be cut in the min-‐cut soluOon and the choice will determine which label we assign.
MulOple Labels
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 131
![Page 132: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/132.jpg)
Constraint Edges
The edges with infinite capacity poinOng upwards are called constraint edges. They prevent soluOons that cut the chain of edges associated with a pixel more than once (and hence given an ambiguous labelling)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 132
![Page 133: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/133.jpg)
MulOple Labels
Inter-‐pixel edges have costs defined as:
Superfluous terms :
For all i,j where K is number of labels Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
133
![Page 134: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/134.jpg)
Example Cuts
Must cut links from before cut on pixel a to a`er cut on pixel b. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
134
![Page 135: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/135.jpg)
Pairwise Costs
If pixel a takes label I and pixel b takes label J
Must cut links from before cut on pixel a to a`er cut on pixel b. Costs were carefully chosen so that sum of these links gives appropriate pairwise term.
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 135
![Page 136: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/136.jpg)
ReparameterizaOon
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 136
![Page 137: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/137.jpg)
Submodularity We require the remaining inter-‐pixel links to be posiOve so that or
By mathemaOcal inducOon we can get the more general result
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 137
![Page 138: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/138.jpg)
Submodularity
If not submodular, then the problem is NP hard. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
138
![Page 139: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/139.jpg)
Convex vs. non-‐convex costs
QuadraOc • Convex • Submodular
Truncated QuadraOc • Not Convex • Not Submodular
Povs Model • Not Convex • Not Submodular
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 139
![Page 140: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/140.jpg)
What is wrong with convex costs?
• Pay lower price for many small changes than one large one • Result: blurring at large changes in intensity
Observed noisy image Denoised result
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 140
![Page 141: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/141.jpg)
Plan of Talk
• Denoising problem • Markov random fields (MRFs) • Max-‐flow / min-‐cut • Binary MRFs -‐ submodular (exact soluOon) • MulO-‐label MRFs – submodular (exact soluOon) • MulO-‐label MRFs -‐ non-‐submodular (approximate)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 141
![Page 142: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/142.jpg)
Alpha Expansion Algorithm • break mulOlabel problem into a series of binary problems • at each iteraOon, pick label α and expand (retain original or change to α)
IniOal labelling
IteraOon 1 (orange)
IteraOon 3 (red)
IteraOon 2 (yellow)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 142
![Page 143: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/143.jpg)
Alpha Expansion Ideas • For every iteraOon – For every label – Expand label using opOmal graph cut soluOon
Co-‐ordinate descent in label space. Each step opOmal, but overall global maximum not guaranteed Proved to be within a factor of 2 of global opOmum. Requires that pairwise costs form a metric:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 143
![Page 144: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/144.jpg)
Alpha Expansion ConstrucOon
Binary graph cut – either cut link to source (assigned to α) or to sink (retain current label) Unary costs avached to links between source, sink and pixel nodes appropriately. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
144
![Page 145: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/145.jpg)
Alpha Expansion ConstrucOon
Graph is dynamic. Structure of inter-‐pixel links depends on α and the choice of labels. There are four cases.
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 145
![Page 146: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/146.jpg)
Alpha Expansion ConstrucOon
Case 1: Adjacent pixels both have label α already. Pairwise cost is zero – no need for extra edges.
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 146
![Page 147: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/147.jpg)
Alpha Expansion ConstrucOon
Case 2: Adjacent pixels are α,β. Result either
• α,α (no cost and no new edge). • α,β (P(α,β), add new edge).
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 147
![Page 148: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/148.jpg)
Alpha Expansion ConstrucOon
Case 3: Adjacent pixels are β,β. Result either • β,β (no cost and no new edge). • α,β (P(α,β), add new edge). • β,α (P(β,α), add new edge). Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
148
![Page 149: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/149.jpg)
Alpha Expansion ConstrucOon
Case 4: Adjacent pixels are β,γ. Result either • β,γ (P(β,γ), add new edge). • α,γ (P(α,γ), add new edge). • β,α (P(β,α), add new edge). • α,α (no cost and no new edge).
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 149
![Page 150: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/150.jpg)
Example Cut 1
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 150
![Page 151: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/151.jpg)
Example Cut 1 Important!
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 151
![Page 152: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/152.jpg)
Example Cut 2
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 152
![Page 153: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/153.jpg)
Example Cut 3
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 153
![Page 154: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/154.jpg)
Denoising Results
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 154
![Page 155: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/155.jpg)
155 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
CondiOonal Random Fields
![Page 156: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/156.jpg)
156 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Directed model for grids
Cannot use graph cuts as three-‐wise term. Easy to draw samples.
![Page 157: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/157.jpg)
157 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Background subtracOon
ApplicaOons
![Page 158: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/158.jpg)
158 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Grab cut
ApplicaOons
![Page 159: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/159.jpg)
159 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Shi`-‐map image ediOng
ApplicaOons
![Page 160: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/160.jpg)
160 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 161: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/161.jpg)
161 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Shi`-‐map image ediOng
ApplicaOons
![Page 162: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/162.jpg)
162 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Super-‐resoluOon
ApplicaOons
![Page 163: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/163.jpg)
163 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Texture synthesis
ApplicaOons
![Page 164: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/164.jpg)
164 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Image QuilOng
![Page 165: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/165.jpg)
165 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Synthesizing faces
ApplicaOons
![Page 166: Graphical Models for chains, trees and grids](https://reader034.fdocuments.in/reader034/viewer/2022051818/54bfd0cd4a79595d628b457b/html5/thumbnails/166.jpg)
Further resources
• hvp://www.computervisionmodels.com/ – Code – Links + readings (for these and other topics)
• Conference papers online: BMVC, CVPR, ECCV, ICCV, etc.
• Jobs mailing lists: Imageworld, Visionlist
166