Improved Moves for Truncated Convex Models
description
Transcript of Improved Moves for Truncated Convex Models
![Page 1: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/1.jpg)
Improved Moves for Truncated Convex Models
M. Pawan Kumar
Philip Torr
![Page 2: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/2.jpg)
AimEfficient, accurate MAP for truncated convex models
V1 V2 … … …
… … … … …
… … … … …
… … … … Vn
Random Variables V = { V1, V2, …, Vn}
Edges E define neighbourhood
![Page 3: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/3.jpg)
Aim
Va Vb
li
lkab;ik
Accurate, efficient MAP for truncated convex models
ab;ik = wab min{ d(i-k), M }
ab;ik
i-k
wab is non-negative
Truncated Linear
i-k
ab;ik
Truncated Quadratic
d(.) is convexa;i b;k
![Page 4: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/4.jpg)
MotivationLow-level Vision
• Smoothly varying regions
• Sharp edges between regions
min{ |i-k|, M}
Boykov, Veksler & Zabih 1998
Well-researched !!
![Page 5: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/5.jpg)
Things We Know• NP-hard problem - Can only get approximation
• Best possible integrality gap - LP relaxation
Manokaran et al., 2008
• Solve using TRW-S, DD, PP
Slower than graph-cuts
• Use Range Move - Veksler, 2007
None of the guarantees of LP
![Page 6: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/6.jpg)
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2 + √2
O(√M)
Chekuri et al., 2001
![Page 7: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/7.jpg)
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2 2M
O(√M) -
Boykov, Veksler and Zabih, 1999
![Page 8: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/8.jpg)
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2 4
O(√M) -
Gupta and Tardos, 2000
![Page 9: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/9.jpg)
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2 4
O(√M) 2M
Komodakis and Tziritas, 2005
![Page 10: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/10.jpg)
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2
O(√M)
2 + √2
O(√M)
![Page 11: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/11.jpg)
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
![Page 12: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/12.jpg)
Move Space
Va Vb
• Initialize the labelling
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Iterate over intervals
![Page 13: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/13.jpg)
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
![Page 14: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/14.jpg)
Two Problems
Va Vb
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Large L’ => Non-submodular
Non-submodular
![Page 15: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/15.jpg)
First Problem
Va Vb Submodular problem
Ishikawa, 2003; Veksler, 2007
![Page 16: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/16.jpg)
First Problem
Va Vb Non-submodularProblem
![Page 17: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/17.jpg)
First Problem
Va Vb Submodular problem
Veksler, 2007
![Page 18: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/18.jpg)
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 19: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/19.jpg)
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 20: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/20.jpg)
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 21: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/21.jpg)
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 22: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/22.jpg)
First Problem
Va Vb
Model unary potentials exactly
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 23: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/23.jpg)
First Problem
Va Vb
Similarly for Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 24: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/24.jpg)
First Problem
Va Vb
Model convex pairwise costs
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
![Page 25: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/25.jpg)
First Problem
Va Vb
Overestimated pairwise potentials
Wanted to model
ab;ik = wab min{ d(i-k), M }
For all li, lk I
Have modelled
ab;ik = wab d(i-k)
For all li, lk I
![Page 26: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/26.jpg)
Second Problem
Va Vb
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Non-submodular problem !!
![Page 27: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/27.jpg)
Second Problem - Case 1
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
s∞ ∞
Both previous labels lie in interval
![Page 28: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/28.jpg)
Second Problem - Case 1
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
s∞ ∞
wab d(i-k)
![Page 29: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/29.jpg)
Second Problem - Case 2
Va Vb
Only previous label of Va lies in interval
am+1
am+2
an
t
bm+1
bm+2
bn
s∞ ub
![Page 30: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/30.jpg)
Second Problem - Case 2
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
ub : unary potential of previous label of Vb
M
s∞ ub
![Page 31: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/31.jpg)
Second Problem - Case 2
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
M
wab d(i-k)
s∞ ub
![Page 32: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/32.jpg)
Second Problem - Case 2
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
M
wab ( d(i-m-1) + M )
s∞ ub
![Page 33: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/33.jpg)
Second Problem - Case 3
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
Only previous label of Vb lies in interval
![Page 34: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/34.jpg)
Second Problem - Case 3
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
sua
∞
ua : unary potential of previous label of Va
M
![Page 35: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/35.jpg)
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
Both previous labels do not lie in interval
![Page 36: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/36.jpg)
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
sua ub
Pab : pairwise potential for previous labels
ab
Pab
MM
![Page 37: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/37.jpg)
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
wab d(i-k)
sua ub
ab
Pab
MM
![Page 38: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/38.jpg)
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
wab ( d(i-m-1) + M )
sua ub
ab
Pab
MM
![Page 39: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/39.jpg)
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
Pab
sua ub
ab
Pab
MM
![Page 40: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/40.jpg)
Graph Construction
Va Vb
Find st-MINCUT. Retain old labellingif energy increases.
am+1
am+2
an
bm+1
bm+2
bn
t
ITERATE
![Page 41: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/41.jpg)
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
![Page 42: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/42.jpg)
Analysis
Va Vb
Current labelling f(.)
QC ≤ Q’C
Va Vb
Global Optimum f*(.)
QP
Previous labelling f’(.)
Va Vb
![Page 43: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/43.jpg)
Analysis
Va Vb
Current labelling f(.)
QC ≤ Q’C
Va Vb
Partially Optimal f’’(.) Previous labelling f’(.)
Va Vb
Q’0≤
![Page 44: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/44.jpg)
Analysis
Va Vb
Current labelling f(.)
QP - Q’C
Va Vb
Partially Optimal f’’(.) Previous labelling f’(.)
Va Vb
QP- Q’0≥
![Page 45: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/45.jpg)
Analysis
Va Vb
Current labelling f(.)
QP - Q’C
Va Vb
Partially Optimal f’’(.) Local Optimal f’(.)
Va Vb
QP- Q’0≤ 0 ≤ 0
![Page 46: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/46.jpg)
Analysis
Va Vb
Current labelling f(.)
Va Vb
Partially Optimal f’’(.) Local Optimal f’(.)
Va Vb
QP- Q’0 ≤ 0Take expectation over all intervals
![Page 47: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/47.jpg)
AnalysisTruncated Linear
QP ≤ 2 + max 2M , L’L’ MQ*
L’ = M 4Gupta and Tardos, 2000
L’ = √2M 2 + √2
Truncated Quadratic
QP ≤ O(√M)Q*
L’ = √M
![Page 48: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/48.jpg)
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
![Page 49: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/49.jpg)
Synthetic Data - Truncated Linear
Faster than TRW-S Comparable to Range Moves
With LP Relaxation guarantees
Time (sec)
Energy
![Page 50: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/50.jpg)
Synthetic Data - Truncated Quadratic
Faster than TRW-S Comparable to Range Moves
With LP Relaxation guarantees
Time (sec)
Energy
![Page 51: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/51.jpg)
Stereo Correspondence
Disparity Map
Unary Potential: Similarity of pixel colour
Pairwise Potential: Truncated convex
![Page 52: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/52.jpg)
Stereo Correspondence
Algo Energy1 Time1 Energy2 Time2
Swap 3678200 18.48 3707268 20.25
Exp 3677950 11.73 3687874 8.79
TRW-S 3677578 131.65 3679563 332.94
BP 3789486 272.06 5180705 331.36
Range 3686844 97.23 3679552 141.78
Our 3613003 120.14 3679552 191.20
Teddy
![Page 53: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/53.jpg)
Stereo Correspondence
Algo Energy1 Time1 Energy2 Time2
Swap 3678200 18.48 3707268 20.25
Exp 3677950 11.73 3687874 8.79
TRW-S 3677578 131.65 3679563 332.94
BP 3789486 272.06 5180705 331.36
Range 3686844 97.23 3679552 141.78
Our 3613003 120.14 3679552 191.20
Teddy
![Page 54: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/54.jpg)
Stereo Correspondence
Algo Energy1 Time1 Energy2 Time2
Swap 645227 28.86 709120 20.04
Exp 634931 9.52 723360 9.78
TRW-S 634720 94.86 651696 226.07
BP 662108 170.67 2155759 244.71
Range 634720 39.75 651696 80.40
Our 634720 66.13 651696 80.70
Tsukuba
![Page 55: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/55.jpg)
Summary
• Moves that give LP guarantees
• Similar results to TRW-S
• Faster than TRW-S because of graph cuts
![Page 56: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/56.jpg)
Questions Not Yet Answered
• Move-making gives LP guarantees– True for all MAP estimation problems?
• Huber function? Parallel Imaging Problem?
• Primal-dual method?
• Solving more complex relaxations?
![Page 57: Improved Moves for Truncated Convex Models](https://reader036.fdocuments.in/reader036/viewer/2022062520/56815a91550346895dc8081c/html5/thumbnails/57.jpg)
Questions?
Improved Moves for Truncated Convex Models
Kumar and Torr, NIPS 2008
http://www.robots.ox.ac.uk/~pawan/