Feature Matching

Feature Space Outlier Rejection

After Outlier Rejection

RANdom SAmple Consensus

RANSAC for Homography

Example: Panorama

Homography Transform,Warping

Image Warping

Forward Warping

Forward Warping

Inverse Warping

Inverse Warping

Forward vs. Inverse Warping

Two View Geometry

When a camera changes position and orientation, the scene moves rigidly relative to the camera

3-D Scene u

u’

X

Y

Z

d

p

Rotation + Translation

Two View Geometry (simple cases)

In two cases this results in homography:1. Camera rotates around its focal point

2. The scene is planar

Then: Point correspondence forms 1:1mapping depth cannot be recovered

The General Case: Epipolar Lines

epipolar lineepipolar line

Epipolar Plane

epipolar plane

epipolar lineepipolar lineepipolar lineepipolar line

BaselineBaseline

PP

OO O’O’

Epipole

Every plane through the baseline is an epipolar plane It determines a pair of epipolar lines (one in each image)

Two systems of epipolar lines are obtained Each system intersects in a point, the epipole The epipole is the projection of the center of the other

camera

epipolar linesepipolar linesepipolar linesepipolar lines

BaselineBaselineOO O’O’

Epipolar Lines

epipolar plane

epipolar lineepipolar lineepipolar lineepipolar line

BaselineBaseline

PP

OO O’O’

To define an epipolar plane, we define the plane through the two camera centers O and O’ and some point P. This can be written algebraically (in some worldcoordinates) as follows:

' ' 0T

OP OO O P

Essential Matrix (algebraic constraint between corresponding image points)

Set world coordinates around the first camera

What to do with O’P? Every rotation changes the observed coordinate in the second image

We need to de-rotate to make the second image plane parallel to the first

Replacing by image points

' ' 0T

OP OO O P

' 0TP t RP

, 'P OP t OO

' 0Tp t Rp

Essential Matrix (cont.)

Denote this by:

Then

Define

E is called the “essential matrix”

t p t p

' ' 0T Tp t Rp p t Rp

E t R

' 0Tp Ep

' 0Tp t Rp

Properties of the Essential Matrix

E is homogeneous 9 parameters E can be recovered up to scale using 8 points.

The constraint det E=0 7 points suffices

In fact, there are only 5 degrees of freedom in E, 3 for rotation 2 for translation (up to scale), determined by epipole

' 0Tp Ep

BackgroundThe lens optical axis does not coincide with

the sensor We model this using a 3x3 matrix the

Calibration matrix

Camera Internal Parameters or Calibration matrix

Camera Calibration matrix

The difference between ideal sensor and the real one is modeled by a 3x3 matrix K:

(cx,cy) camera center, (ax,ay) pixel dimensions, b skew

We end with

0

0 0 1

x x

y y

a b c

K a c

q Kp

Fundamental Matrix

F, is the fundamental matrix.

1 1

1

1

' 0 ( ) ( ') 0

( ) ' 0

T T

T T

T

p Ep K q E K q

q K EK q

F K EK

Properties of the Fundamental Matrix F is homogeneous 9 parameters

F can be recovered up to scale using 8 points.

The constraint det F=0 7 points suffices

0'Fpp t

Epipolar Plane

l’l’ ll

BaselineBaseline

PP

OO O’O’

Other derivations Hartley & Zisserman p. 223

x X’

ee e’e’

Homography Epipolar

Form

Shape One-to-one map Concentric epipolar lines

D.o.f. 8 8/5 F/E

Eqs/pnt 2 1

Minimal configuration

4 5+ (8, linear)

Depth No Yes, up to scale

Scene Planar

(or no translation)

3D scene

Two-views geometry Summary:

0'Fpp tHpp '

Stereo Vision

Objective: 3D reconstruction Input: 2 (or more) images taken with

calibrated cameras Output: 3D structure of scene Steps:

Rectification Matching Depth estimation

Rectification

Image Reprojection reproject image planes onto common

plane parallel to baseline Notice, only focal point of camera

really matters(Seitz)

Rectification

Any stereo pair can be rectified by rotating and scaling the two image planes (=homography)

Images have to be rectified so that Image planes of cameras are parallel. Focal points are at same height. Focal lengths same.

Then, epipolar lines fall along the horizontal scan lines of the images

References http://web.me.com/dellaert/07F-Vision/Schedule.html http://cseweb.ucsd.edu/classes/wi07/cse252a/homography_estimation/ho

mography_estimation.pdf http://en.wikipedia.org/wiki/Homography http://www.andrew.cmu.edu/course/16-720/lectures/figs1.pdf http://www.cs.utoronto.ca/~strider/vis-notes/tutHomography04.pdf http://people.scs.carleton.ca/~c_shu/Courses/comp4900d/notes/ http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/

RANSAC.pdf http://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/lec/warping.pdf http://www.wisdom.weizmann.ac.il/~bagon/CVSpring08/files/

2ViewsPart2.ppt