Advanced Computer Vision Feature-based Alignment Lecturer: Lu Yi & Prof. Fuh [email protected]...
-
Upload
emery-allen -
Category
Documents
-
view
221 -
download
0
Transcript of Advanced Computer Vision Feature-based Alignment Lecturer: Lu Yi & Prof. Fuh [email protected]...
Advanced Computer Vision
Feature-based AlignmentLecturer: Lu Yi & Prof. Fuh
Content
• Pre-requisites: mainly about camera model• Alignment Algorithm• RANdom Sample Consensus (RANSAC)• Pose Estimation
Sources
• EGGN 512 – Computer Vision course by Prof. Willliam Hoff• http://inside.mines.edu/~whoff/courses/EENG512/• Playlist@Youtube:• http://www.youtube.com/playlist?list=PL4B3F8D4A5CAD8DA3
• Digital Visual Effects course by Prof. Yung-Yu Chuang• http://www.csie.ntu.edu.tw/~cyy/courses/vfx/14spring/news/
• Both are highly recommended
Taught Way
• Cover the main parts of textbook chapter 6• Lecture mostly depended on EGGN 512 course videos
Frames of Reference
• Image frames are 2D; other are 3D
• The “pose” (position and orientation) of a 3D rigid body has 6 degrees of freedom
Pre-requisites
• Image Formation• Geometric Transformations (2D-2D, 3D-3D, 3D-2D)• Camera Model (Camera Matrix = Intrinsic + Extrinsic camera matrix)
• All related contents are in textbook chapter 2: Image formation
Pre-requisites
• Review1. Projective Transformation2. Camera Matrix• EGGN 512 – Lecture 5-1 3D-2D Transforms• http://
www.youtube.com/watch?v=DFNjOUMuecU&index=11&list=PL4B3F8D4A5CAD8DA3• EGGN 512 – Lecture 5-2 3D-2D Transforms• http://
www.youtube.com/watch?v=5gesrLgNuQo&list=PL4B3F8D4A5CAD8DA3
Alignment Algorithm
• EGGN 512 – Lecture 14-1 Alignment• http://www.youtube.com/watch?v=UcU4814hvR8&list=PL4B3F8D4A5CAD8DA3• EGGN 512 – Lecture 14-2 Alignment• http://
www.youtube.com/watch?v=XxEKMecNZk0&list=PL4B3F8D4A5CAD8DA3• EGGN 512 – Lecture 14-3 Alignment• http://www.youtube.com/watch?v=lSXfv4baMwk&list=PL4B3F8D4A5CAD8DA3• Slides • http://inside.mines.edu/~
whoff/courses/EENG512/lectures/15-AlignmentNonlinear.pdf
Alignment Algorithm
• Feature-based methods: only use feature points to estimate parameters• Features: SIFT, SURF, Harris .etc.
Determine pairwise alignment
• p’=Mp, where M is a transformation matrix, p and p’ are feature matches• It is possible to use more complicated models such as affine or
perspective• For example, assume M is a 2x2 matrix
• Find M with the least square error
y
x
mm
mm
y
x
2221
1211
'
'
n
i
pMp1
2'
Determine pairwise alignment
• Overdetermined system
y
x
mm
mm
y
x
2221
1211
'
''1221211
'1121111
ymymx
xmymx
'
'
'2
'1
'1
22
21
12
11
22
11
11
00
00
00
00
00
n
n
nn
nn
y
x
x
y
x
m
m
m
m
yx
yx
yx
yx
yx
Normal Equation
• Given an overdetermined system
• the normal equation is that which minimizes the sum of the square differences between left and right sides
• Why?
bAx
bAAxA TT
Normal Equation
2)( bAxx E
n
m
nmn
m
b
b
x
x
aa
aa
:
:
:
:
...
::
::
::
... 1
1
1
111
nxm, n equations, m variables
Normal Equation
n
m
jjnj
i
m
jjij
m
jjj
n
i
m
jjnj
m
jjij
m
jjj
bxa
bxa
bxa
b
b
b
xa
xa
xa
1
1
11
1
1
1
1
11
:
:
:
:
:
:
bAx
n
ii
m
jjij bxaE
1
2
1
2)( bAxx
Normal Equation
n
ii
m
jjij bxaE
1
2
1
2)( bAxx
1
0x
E
n
iii
n
ij
m
jiji baxaa
11
1 11 22
11 1
2 i
n
ii
m
jjij abxa
)(20 bAAxAx
TT
E
bAAxA TT →
Normal Equation
bbxbAxbAAxAx
bbbAxAxbAxAx
bAxbAx
bAxbAx
bAxbAx
bAx
TTTTTTT
TTTTTT
TTT
TT
T
2
bAAxAx
TT 22 E
Determine pairwise alignment
• p’=Mp, where M is a transformation matrix, p and p’ are feature matches• For translation model, it is easier.
• What if the match is false? Avoid impact of outliers. (Ans: RANSAC)
n
iiiii yymxxmE
1
2'2
2'1
1
0m
E
Application: Image Stitching
• Stitching = alignment + blendinggeometricalregistration
photometricregistration
Application: Image Stitching
• Panorama – the whole picture• Compact Camera FOV = 50 x 35°• Human FOV = 200 x 135°
Application: Image Stitching
• Panorama – the whole picture• Compact Camera FOV = 50 x 35°• Human FOV = 200 x 135°• Panoramic Mosaic = 360 x 180°
Application: Image Stitching
• Example:• http://graphics.stanford.edu/courses/cs178/applets/projection.html
RANSAC Algorithm
• RANSAC = RANdom Sample Consensus• An iterative method to estimate parameters of a mathematical model
from a set of observed data which contains outliers [Wiki]• Compare to robust statistics• Given N data points xi, assume that majority of them are generated
from a model with parameters , try to recover .
• Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381-395.
RANSAC Algorithm
Run k times: (1) draw n samples randomly (2) fit parameters with these n samples (3) for each of other N-n points, calculate its distance to the fitted model, count the number of inlier points, cOutput with the largest c
How many times?How big? Smaller is better
How to define?Depends on the problem.
RANSAC Algorithm (How to determine K)
p: probability of real inliers
P: probability of success after k trials
knpP )1(1 n samples are all inliers
a failure
failure after k trials
)1log(
)1log(np
Pk
n p k
3 0.5 35
6 0.6 97
6 0.5 293
for P=0.99
RANSAC Algorithm
• EGGN 512 – Lecture 27-1 RANSAC• http://www.youtube.com/watch?v=NKxXGsZdDp8&list=PL4B3F8D4A5
CAD8DA3&index=54
Pose Estimation
• EGGN 512 – Lecture 16-1 Pose Estimation• http://
www.youtube.com/watch?v=kq3c6QpcAGc&list=PL4B3F8D4A5CAD8DA3
• EGGN 512 – Lecture 17-1 Pose from Lines• http://
www.youtube.com/watch?v=D_4eUoqgWdc&list=PL4B3F8D4A5CAD8DA3
Pose Estimation
• EGGN 512 – Lecture 18-1 SVD• http://
www.youtube.com/watch?v=C852P1JrHXI&list=PL4B3F8D4A5CAD8DA3
• EGGN 512 – Lecture 18-2 SVD• http://
www.youtube.com/watch?v=aIkzK4CdYes&list=PL4B3F8D4A5CAD8DA3
Pose Estimation
• EGGN 512 – Lecture 19-1 Linear Pose Estimation• http://
www.youtube.com/watch?v=HojSSrxsB4Q&list=PL4B3F8D4A5CAD8DA3• EGGN 512 – Lecture 19-2 Linear Pose Estimation• http://
www.youtube.com/watch?v=ik8dFybnyPY&list=PL4B3F8D4A5CAD8DA3• EGGN 512 – Lecture 19-3 Linear Pose Estimation• http://
www.youtube.com/watch?v=OS5b-3Xfn1M&list=PL4B3F8D4A5CAD8DA3