Bag-of-words models CS4670 / 5670: Computer Vision Noah Snavely Object Bag of ‘words’
Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.
-
Upload
randolf-rice -
Category
Documents
-
view
226 -
download
0
Transcript of Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.
![Page 1: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/1.jpg)
Lec 22: Stereo
CS4670 / 5670: Computer VisionKavita Bala
![Page 2: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/2.jpg)
Road map
• What we’ve seen so far:– Low-level image processing: filtering, edge detecting,
feature detection– Geometry: image transformations, panoramas,
single-view modeling Fundamental matrices• What’s next:
– Finishing up geometry: multi view stereo, structure from motion
– Recognition– Image formation
![Page 3: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/3.jpg)
Announcements
• Wed: photometric stereo
• No class Dragon Day
![Page 4: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/4.jpg)
Fundamental matrix result
![Page 5: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/5.jpg)
5
Properties of the Fundamental Matrix
• is the epipolar line associated with
• is the epipolar line associated with
• and
• is rank 2
T
0
![Page 6: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/6.jpg)
Estimating F
• If we don’t know K1, K2, R, or t, can we estimate F for two images?
• Yes, given enough correspondences
![Page 7: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/7.jpg)
Estimating F – 8-point algorithm
• The fundamental matrix F is defined by
0Fxx'
for any pair of matches x and x’ in two images.
• Let x=(u,v,1)T and x’=(u’,v’,1)T,
333231
232221
131211
fff
fff
fff
F
each match gives a linear equation
0'''''' 333231232221131211 fvfuffvfvvfuvfufvufuu
![Page 8: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/8.jpg)
8-point algorithm
0
1´´´´´´
1´´´´´´
1´´´´´´
33
32
31
23
22
21
13
12
11
222222222222
111111111111
f
f
f
f
f
f
f
f
f
vuvvvvuuuvuu
vuvvvvuuuvuu
vuvvvvuuuvuu
nnnnnnnnnnnn
• In reality, instead of solving , we seek f to minimize , least eigenvector of .
0Af
Af AA
![Page 9: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/9.jpg)
8-point algorithm – Problem?• F should have rank 2• To enforce that F is of rank 2, F is replaced by F’ that
minimizes subject to the rank constraint. 'FF
• This is achieved by SVD. Let , where
, let
then is the solution.
VUF Σ
3
2
1
00
00
00
Σ
000
00
00
Σ' 2
1
VUF Σ''
![Page 10: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/10.jpg)
8-point algorithm
• Pros: it is linear, easy to implement and fast• Cons: susceptible to noise
• Normalized 8-point algorithm: Hartley
![Page 11: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/11.jpg)
What about more than two views?
• The geometry of three views is described by a 3 x 3 x 3 tensor called the trifocal tensor
• The geometry of four views is described by a 3 x 3 x 3 x 3 tensor called the quadrifocal tensor
• After this it starts to get complicated…– Structure from motion
![Page 12: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/12.jpg)
Stereo reconstruction pipeline
• Steps– Calibrate cameras– Rectify images– Compute correspondence (and hence
disparity)– Estimate depth
![Page 13: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/13.jpg)
Correspondence algorithms
Algorithms may be classified into two types:1. Dense: compute a correspondence at every
pixel
2. Sparse: compute correspondences only for features
![Page 14: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/14.jpg)
Example image pair – parallel cameras
![Page 15: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/15.jpg)
First image
![Page 16: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/16.jpg)
Second image
![Page 17: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/17.jpg)
Dense correspondence algorithm
Search problem (geometric constraint): for each point in left image, corresponding point in right image lies on the epipolar line (1D ambiguity)
Disambiguating assumption (photometric constraint): the intensity neighborhood of corresponding points are similar across images
Measure similarity of neighborhood intensity by cross-correlation
epipolar line
![Page 18: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/18.jpg)
Intensity profiles
• Clear correspondence, but also noise and ambiguity
![Page 19: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/19.jpg)
region A
Normalized Cross Correlation
region B
vector a vector b
regions as vectors
a
b
![Page 20: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/20.jpg)
Cross-correlation of neighborhood
epipolar line
translate so that mean is zero
![Page 21: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/21.jpg)
left image band
right image band
cross correlation
1
0
0.5
x
![Page 22: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/22.jpg)
left image band
right image band
cross correlation
1
0
x
0.5
target region
![Page 23: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/23.jpg)
Why is cross-correlation such a poor measure?
1. The neighborhood region does not have a “distinctive” spatial intensity distribution
2. Foreshortening effects
fronto-parallel surface
imaged length the same
slanting surface
imaged lengths differ
![Page 24: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/24.jpg)
Limitations of similarity constraint
Textureless surfaces Occlusions, repetition
Non-Lambertian surfaces, specularities
![Page 25: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/25.jpg)
Other approaches to obtaining 3D structure
![Page 26: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/26.jpg)
Active stereo with structured light
• Project “structured” light patterns onto the object– simplifies the correspondence problem– Allows us to use only one camera
camera
projector
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured Light and Multi-pass Dynamic Programming. 3DPVT 2002
![Page 27: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/27.jpg)
Active stereo with structured light
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured Light and Multi-pass Dynamic Programming. 3DPVT 2002
![Page 28: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/28.jpg)
Microsoft Kinect
![Page 29: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/29.jpg)
Laser scanning
• Optical triangulation– Project a single stripe of laser light– Scan it across the surface of the object– This is a very precise version of structured light scanning
Digital Michelangelo Projecthttp://graphics.stanford.edu/projects/mich/
Source: S. Seitz
![Page 30: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/30.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 31: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/31.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 32: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/32.jpg)
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 33: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/33.jpg)
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 34: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/34.jpg)
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 35: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/35.jpg)
Aligning range images
• A single range scan is not sufficient to describe a complex surface
• Need techniques to register multiple range images
• … which brings us to multi-view stereo
![Page 36: Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.](https://reader035.fdocuments.in/reader035/viewer/2022062321/56649f1c5503460f94c3217d/html5/thumbnails/36.jpg)
Quiz