Lecture 10: Robust fitting CS4670: Computer Vision Noah Snavely.
Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala
-
Upload
bruce-richards -
Category
Documents
-
view
215 -
download
0
Transcript of Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala
![Page 1: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/1.jpg)
Lecture 12: Image alignment
CS4670/5760: Computer VisionKavita Bala
http://www.wired.com/gadgetlab/2010/07/camera-software-lets-you-see-into-the-past/
![Page 2: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/2.jpg)
Announcements
• Project 1 Artifact voting closes tomorrow. Please vote.
• Project 2 released– Due Monday, March 9 2015 (by 11:59pm)
• Prelim next Thu– Send me mail if you have a conflict– All material till end of this week– Closed book– Will release last year’s midterm. But it was a take home.
![Page 3: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/3.jpg)
PA2: Feature detection and matching
![Page 4: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/4.jpg)
Reading
• Appendix A.2, 6.1
![Page 5: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/5.jpg)
Fitting: find the parameters of a model that best fit the data
Alignment: find the parameters of the transformation that best align matched points
Common problems in vision
![Page 6: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/6.jpg)
Alignment• Alignment: find parameters of model that
maps one set of points to another
• Typically want to solve for a global transformation that accounts for *most* true correspondences
• Difficulties– Noise (typically 1-3 pixels)– Outliers (often 50%)
![Page 7: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/7.jpg)
Computing transformations
?
![Page 8: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/8.jpg)
Simple case: translations
How do we solve for ?
![Page 9: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/9.jpg)
Mean displacement =
Simple case: translations
Displacement of match i =
![Page 10: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/10.jpg)
Another view
• System of linear equations– What are the knowns? Unknowns?– How many unknowns? How many equations (per match)?
![Page 11: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/11.jpg)
Another view
• Problem: more equations than unknowns– “Overdetermined” system of equations– We will find the least squares solution
![Page 12: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/12.jpg)
Least squares formulation
• For each point
• we define the residuals as
![Page 13: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/13.jpg)
Least squares formulation
• Goal: minimize sum of squared residuals
• “Least squares” solution• For translations, is equal to mean displacement
![Page 14: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/14.jpg)
Least squares formulation
• Can also write as a matrix equation
2n x 2 2 x 1 2n x 1
![Page 15: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/15.jpg)
Least squares
• Find t that minimizes
![Page 16: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/16.jpg)
Least squares: find t to minimize
• To solve, form the normal equations• Differentiate and equate to 0 to minimize
![Page 17: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/17.jpg)
Affine transformations
• How many unknowns?• How many equations per match?• How many matches do we need?
![Page 18: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/18.jpg)
Affine transformations
• Residuals:
• Cost function:
![Page 19: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/19.jpg)
Affine transformations
• Matrix form
2n x 6 6 x 1 2n x 1
![Page 20: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/20.jpg)
Homographies
To unwarp (rectify) an image• solve for homography H given p and p’• solve equations of the form: p’ = Hp
pp’
![Page 21: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/21.jpg)
Alternate formulation for homographies
where the length of the vector [h00 h01 … h22] is 1
![Page 22: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/22.jpg)
Solving for homographies
Linear or non-linear?
![Page 23: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/23.jpg)
Solving for homographies
![Page 24: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/24.jpg)
Solving for homographies
![Page 25: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/25.jpg)
Solving for homographies
Defines a least squares problem:
2n × 9 9 2n
• Since is only defined up to scale, solve for unit vector
![Page 26: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/26.jpg)
Homographies
To unwarp (rectify) an image• solve for homography H given p and p’• solve equations of the form: p’ = Hp
– linear in unknowns: coefficients of H– H is defined up to an arbitrary scale factor– how many points are necessary to solve for H?
pp’
![Page 27: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/27.jpg)
Solving for homographies
Defines a least squares problem:• Since is only defined up to scale, solve for unit vector• Solution: = eigenvector of with smallest eigenvalue• Works with 4 or more points
2n × 9 9 2n
![Page 28: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/28.jpg)
Recap: Two Common Optimization Problems
Problem statement Solution
1 s.t. minimize xxAxAx TTT
0 osolution tlsq trivial-non Ax
1..21 :
)eig(],[
vx
AAv
n
T
Problem statement Solution
bAx osolution t squaresleast bAx \
2 minimize bAx bAAAx TT 1
(matlab)
![Page 29: Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649e185503460f94b04962/html5/thumbnails/29.jpg)