Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is...
Transcript of Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is...
![Page 1: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/1.jpg)
Shape from XHaoqiang Fan
Some figures adapted from http://cvg.ethz.ch/teaching/2012spring/3dphoto/Slides/3dphoto12_shapeFromX.pdf
![Page 2: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/2.jpg)
Perception / Measurement of 3D3D is vital for survival
![Page 3: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/3.jpg)
How to reconstruct / perceive 3DBy means of visual information
-> optical, 2D array of input
![Page 4: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/4.jpg)
Structure from MotionThe most easy-to-understand approach
Triangulation
https://cn.mathworks.com/help/vision/ug/structure-from-motion.html
![Page 5: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/5.jpg)
TriangulationThe epipolar constraint
Stereo and kinect fusion for continuous 3D reconstruction and visual odometry
![Page 6: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/6.jpg)
Stereo, rectification, disparityrow-to-row correspondence
https://www.slideshare.net/DngNguyn43/stereo-vision-42147593
![Page 7: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/7.jpg)
Disparity, depthd=y_right - y_left
z=B*F/d
OpenCV: Depth Map from Stereo Images Middlebury Stereo Evaluation
![Page 8: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/8.jpg)
3D Point Cloudx=x_screen/F*z
y=y_screen/F*z
Bundler: Structure from Motion (SfM) for Unordered Image Collections
![Page 9: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/9.jpg)
Surface ReconstructionIntegration of oriented point
![Page 10: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/10.jpg)
Laplacian and NormalLaplacian = Normal * Mean Curvature
![Page 11: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/11.jpg)
SfM ScanningSLAM based positioning
![Page 12: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/12.jpg)
Depth Sensing: Active SensorsStructured Light
Time of Flight(ToF)
![Page 13: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/13.jpg)
Structured LightStatic pattern & dynamic pattern
![Page 14: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/14.jpg)
Time of Flight (ToF)Pulsed modulation
![Page 15: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/15.jpg)
Short Baseline StereoPhase Detection Autofocus
![Page 16: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/16.jpg)
Shape from XStructure from Motion: 3D geometry
Are there other possibilities?
![Page 17: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/17.jpg)
Shape from ShadingShading as a cue of 3D shape
![Page 18: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/18.jpg)
The Lambertian Law
![Page 19: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/19.jpg)
Shape from ShadingSolve for gradient
Assuming constant albedo
![Page 20: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/20.jpg)
Is Shape Uniquely Determined?bas-relief ambiguity
![Page 21: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/21.jpg)
Shape from ShadingData term + Prior
![Page 22: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/22.jpg)
Shape from ShadingExample
![Page 23: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/23.jpg)
Photometric Stereo
![Page 24: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/24.jpg)
Photometric StereoMeasure the normal direction: the chrome sphere
![Page 25: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/25.jpg)
Depth from Normals
![Page 26: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/26.jpg)
ExampleGood for near Lambertian material
![Page 27: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/27.jpg)
Shape from TextureSolving normal from texture
![Page 28: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/28.jpg)
Depth from FocusFocus sweep
![Page 29: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/29.jpg)
Depth from DefocusMeasure blur, solve depth
![Page 30: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/30.jpg)
Shape from ShadowsShadow carving
3D Reconstruction by Shadow Carving: Theory and Practical Evaluation”
![Page 31: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/31.jpg)
Shape from SpecularitiesSolve deformation of mirrors.
Toward a Theory of Shape from Specular Flow
![Page 32: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/32.jpg)
Shape from ?Shape from Nothing?
Object priors!
![Page 33: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/33.jpg)
3D Reconstruction from Single Imageinfer a whole shape, from a single image
![Page 34: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/34.jpg)
3D Reconstruction from Single Image
![Page 35: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/35.jpg)
The ShapeNet Dataset
![Page 36: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/36.jpg)
3D Reconstruction from Single Image
![Page 37: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/37.jpg)
3D Reconstruction from Single Image
![Page 38: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/38.jpg)
The issue of representation
![Page 39: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/39.jpg)
Depth map
![Page 40: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/40.jpg)
Depth map
![Page 41: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/41.jpg)
Second depth map
![Page 42: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/42.jpg)
Second depth map
![Page 43: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/43.jpg)
The problem of discontinuity
![Page 44: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/44.jpg)
Volumetric Occupancy
![Page 45: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/45.jpg)
Problem of viewpoint
![Page 46: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/46.jpg)
Canonical View
![Page 47: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/47.jpg)
Volumetric Occupancy
![Page 48: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/48.jpg)
XML file
![Page 49: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/49.jpg)
XML file
![Page 50: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/50.jpg)
XML file
![Page 51: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/51.jpg)
XML file
![Page 52: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/52.jpg)
Can we find a representation that is..flexible
structural
natural
![Page 53: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/53.jpg)
![Page 54: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/54.jpg)
Point-based representationflexible
structural
natural
![Page 55: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/55.jpg)
Implementation details
![Page 56: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/56.jpg)
Results
![Page 57: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/57.jpg)
Results
![Page 58: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/58.jpg)
Results
![Page 59: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/59.jpg)
Human Performance
![Page 60: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/60.jpg)
![Page 61: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/61.jpg)
![Page 62: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/62.jpg)
A Neural Method to Stereo Matching
![Page 63: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/63.jpg)
Flownet & Dispnet Using raw left and right images as input
Output disparity map
End-to-End training
![Page 64: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/64.jpg)
Using two stacked images as inputFlownetSimple
![Page 65: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/65.jpg)
Adding Correlation Layer Using correlation layer to explicitly provide cross view communication ability
FlownetCorr
![Page 66: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/66.jpg)
Stereo Matching Cost Convolutional Neural NetworkUsing CNN to calculate stereo matching cost between patches from different view
Following with several post-process:
Cross-based cost aggregation
Semiglobal matching
Left-right consistency check
Disparity <-> Depth
![Page 67: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/67.jpg)
MRF Stereo methods
We estimate f by minimizing the following energy function based on pairwise MRF
Data term
Smoothness term
![Page 68: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/68.jpg)
Global Local Stereo Neural NetworkFeature visualization
![Page 69: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/69.jpg)
results
![Page 70: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/70.jpg)
results
![Page 71: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/71.jpg)
results
![Page 72: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/72.jpg)
Implementation detailsEntangle two view feature inside network.
![Page 73: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/73.jpg)
Large Receptive Field Neural NetworkSimpleConv
Encoder-Decoder
ResConv
blindingly increasing the receptive
field of feature networks may not
Improve the performance
simple convSimpleConv
![Page 74: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/74.jpg)
PatchMatch Communication LayerDirectly provide the ability of
communicating across two views
![Page 75: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/75.jpg)
Multi-staged Cascade
![Page 76: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/76.jpg)
ThanksQ/A
![Page 77: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/77.jpg)
单击以结束放映
![Page 78: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/78.jpg)
SemiGlobal Matchingwe define an energy function E(D) that depends on the disparity map D
NP-Hard !!! But we can solve it through each directions to get an approximate solution by using Dynamic Programming(DP)
![Page 79: Shape from X - GitHub Pages · Slanted patch matching The disparity d_p of each pixel p is over-parameterized by a local disparity plane Each pixels in the same plane has the same](https://reader033.fdocuments.in/reader033/viewer/2022042300/5ecaa0d63d077d7ca90ecd7e/html5/thumbnails/79.jpg)
Slanted patch matching
The disparity d_p of each pixel p is over-parameterized by a local disparity plane
Each pixels in the same plane has the same parameter (a_p, b_p, c_p)
The true disparity maps are approximately piecewise linear
We can estimate (a_p, b_p, c_p) for each pixel p instead of directly estimate d_p