Minimal Surfaces for Stereo
Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan
MIT, HarvardMicrosoft Research, MIT
Motivation
• Optimization based stereo over greed based– No early commitment– Enforce interactions: each pixel sees unique item– Penalize interactions: non-smoothness
Stereo by Optimization
• Early algorithms: dynamic programming– (Baker ‘81, Belumeur & Mumford ‘92…)– Don’t generalize beyond 2 camera, single scanline
Stereo by Optimization
• Recent Algorithms: iterative expansion– (… Kolmogorov & Zabih ‘01)– very general– NP-Complete
• Local opt found quickly in practice
• Recent algorithms: MIN-CUT– (Roy & Cox ‘96, Ishikawa & Geiger ‘98) – Polynomial time global optimum– New interpretation to such methods
Contributions
• Stereo as a discrete minimal surface problem• Algorithms: Polynomial time globally optimal
surface– Using MIN-CUT (Sullivan ‘90)– Build from shortest path
• Applications to stereo vision– Rederive previous MIN-CUT stereo approaches– New 3-camera stereo formulation (Ayache ‘88)
Planar Graph Shortest Path
• Given: an embedded planar graph– faces,
edges, vertices
Planar Graph Shortest Path
• A non negative cost on each edge
57
Planar Graph Shortest Path
• Two boundary points on the exterior of the complex
Planar Graph Shortest Path
• Find minimal curve: (collection of edges) with given boundary
Planar Graph For stereo
Camera Left Camera Right
Selected MatchSelected Occlusion
Algorithms
• Classic: Dijkstra’s– Works even for non-planar graphs
• Wacky: use duality– But this will generalize to higher dimension
Duality
Duality
• face vertex • edge cross edge
- same cost57
Duality
• Split exterior
Source
Sink
Source
Duality
• Add source and sink
Cuts
Source
Sink
• Cuts of dual graph = partitions of dual verts• Cost = sum of dual edges spanning the partition• MIN-CUT can be found in polynomial time
Source
SinkCuts
• Claim: Primalization of MIN-CUT will be shortest path
Sink
SourceSource
SinkWhy this works
• Cuts of dual graph = partitions of dual verts
Sink
SourceSource
SinkWhy this works
• Partition of dual verts = partition of primal faces
Sink
SourceSource
Sink
Source
SinkWhy this works
• Partition of primal faces = primal path
Sink
SourceSource
Sink
Source
SinkWhy this works
• Cuts in dual correspond to paths in primal
• MIN-CUT in dual corresponds to shortest path in primal
Same idea works for surfaces!
Increasing the dimension
Planar graph: verts, edges, faces cost on edges boundary: 2 points on exterior sol: min path
Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface
Increasing the dimension
Planar graph: verts, edges, faces boundary: 2 points on exterior sol: min path
Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface
Increasing the dimension
Planar graph: verts, edges, faces boundary: 2 points on exterior sol: min path
Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface
Dual construction for min surf
• face vertex • edge cross edge
Sink
Source
• cell vertex • face cross edge
MIN-CUT primalizes to min surf
Checkpoint
• Solve for minimal paths and surfaces– MIN-CUT on dual graph
• Apply these algorithms to stereo vision
Flatland Stereo
Camera Left Camera Right
Geometric interpretation of Cox et al. 96
pixel
Flatland Stereo
Camera Left Camera Right
Geometric interpretation of Cox et al. 96
pixel
Flatland Stereo
Camera Left Camera Right
Cost: unmatched/discontinuity, β
Flatland Stereo
Camera Left Camera Right
Cost: correspondence quality
Flatland Stereo
Camera Left Camera Right
Camera Left Camera Right
MatchUnmatched
Uniqueness & monotonicity solution is directed pathFlatland Stereo
Camera Left Camera Right
MatchOcclusion, discontinuity
Note: unmatched pixels also function as discontinuities
Flatland Stereo
Flatland to Fatland
Camera Left Camera Right
Flatland to Fatland
Camera Left Camera Right
2 cameras, 3d
2 cameras, 3d
One Cuboid Among Many
Solve for minimal surface
Geometric interpretation IG98
Three Camera
Rectification (Ayache ‘88)
Three Camera
Three Camera
Three Camera
Three Camera
One cuboid
Dual graph of one cuboid
One Cuboid Among Many
Solve for minimal surface
More divisions of middle cell
More expressive decomposition
Complexity
• Vertices and edges: 20 n d– n: pixels per image– d: max disparity
• Time complexity O((nd)2 log(nd))
• About 1 min
ResultsLL image
RC
KZ01
MS
LL image
RC
KZ01
MS
Future
• Application of MS to n cameras– Monotonicity/oriented manifold enforces more
than uniqueness– see Kolmogorov & Zabih (today 11:00am)
• Other applications of MS
Top Related