Post on 21-Dec-2015
Compact Windows for Visual Correspondence via Minimum
Ratio Cycle Algorithm
Olga Veksler
NEC Labs America
Local Approach
• Look at one image patch at at time
• Solve many small problems independently
• Fast, sufficient for some problems
Global Approach
• Look at the whole image • Solve one large problem• Slower, more accurate
Local Approach
• Sufficient for some problems
• Central problem: window shape selection
• Efficiently solved using Minimum Ratio Cycle algorithm for graphs
stereo
left image right image disparity = x1-x2
Visual Correspondence
(x1,y) (x2,y)
motion
first image second image
vertical motion
horizontal motion(x1,y1) (x2,y2)
Local Approach [Levine’73]
left image right image
p
1C
1
2C
2
3C
3
++
+2 2
2 2=Common C
pd = i which gives best iC
Fixed Window Shape Problems
true disparities
fixed small window fixed large window
left image
nee
d d
iffe
ren
t w
ind
ow
sh
apes
• Two inefficient methods proposed previously 1. Local greedy search [Levine CGIP’73, Kanade’PAMI94]
2. Direct search [Intille ECCV94,Geiger IJCV95]
Variable Window: Previous Work
…….
• Need efficient optimization algorithm over sizes and shapes
Minimum Ratio Cycle
• G(V,E) and w(e), t(e): E R 0Ce
et
image pixels
• Find cycle C which minimizes:
WC
Ce
ew
Ce
et=
W
t
From Area to Cycle
blue edge
red edge+
5
-2sum up terms inside
using weights of edges
1
111
11
1
Window Cost
C(W) = =size of W+ … +
2 2
WC
Ce
ew
Ce
et
OK for any graphs
not OK for any graph 0Ce
et
positive negative
Compact Windows
p
simple graph cycles C
one-to-one
correspondencecompact windows W
we construct graph s.t. 0Ce
et
only clockwise cycles
Cycle which is not Simple
C Cin this case:
cycle C
cycle C
Solving MRC
• search for smallest s.t. there is negative cycle on graph with edge weights:
• negative cycle detection takes time due to the special structure of our graphs
find smallest s.t. for some cycle ew
et
ew et-
nnO
ew et -
• there are are compact windows, if the largest allowed window is n by n
• Contains all possible rectangles but much more general than just rectangles
• Find optimal window in in theory, linear ( ) in practice
• Search over in time
nO 2
4nO
3n 2n
perimeterarea
examples of compact windows
(small )
312O 231O
Sample Compact Windows
Speedup
for pixel p, the algorithm extends windows over pixels which are likely to have the same disparity as p
use the optimal window computed for p to approximate for pixels inside that window
Comparison to Fixed Window
Compact windows:16% errorstrue disparities
fixed small window: 33% errors fixed large window: 30% errors
motion
motion
Algorithm Tsukuba Venus Sawtooth Map
Layered 1.58 1.52 0.34 0.37Graph cuts 1.94 1.79 1.30 0.31Belief prop 1.15 1.00 0.98 0.84GC+occl. 1.27 2.79 0.36 1.79
Graph cuts 1.86 1.69 0.42 2.39Multiw. Cut 8.08 0.53 0.61 0.26
Comp. win. 3.36 1.67 1.61 0.33
Results
13 other algorithms, local and global
all
glo
bal
Running time: 8 to 22 seconds
Future Work
• Generalize the window class
• Generalize objective function – mean?– variance?