lecture2 camera models - Stanford...

Lecture 2 -Silvio Savarese 10-Jan-18

ProfessorSilvioSavareseComputationalVisionandGeometryLab

Lecture2CameraModels


• Pinholecameras• Cameras&lenses• Thegeometryofpinholecameras


Reading: [FP]Chapter1,“GeometricCameraModels”[HZ]Chapter6 “CameraModels”

Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li

How do we see the world?

•Let’s design a camera– Idea 1: put a piece of film in front of an object– Do we get a reasonable image?

Pinhole camera

• Add a barrier to block off most of the rays– This reduces blurring– The opening known as the aperture

Aperture

Milestones: • Leonardo da Vinci (1452-1519): first record of camera obscura (1502)

Some history…

Milestones: • Leonardo da Vinci (1452-1519): first record of camera obscura

• Johann Zahn (1685): first portable camera

Some history…

Photography (Niépce, “La Table Servie,” 1822)



• Joseph Nicéphore Niépce(1822): first photo - birth of photography

Some history…

Photography (Niépce, “La Table Servie,” 1822)



• Joseph Nicéphore Niépce(1822): first photo - birth of photography

•Daguerréotypes (1839)• Photographic Film (Eastman, 1889)• Cinema (Lumière Brothers, 1895)• Color Photography (Lumière Brothers, 1908)

Some history…

Let’s also not forget…

Motzu(468-376 BC)

Aristotle(384-322 BC)

Also: Plato, Euclid

Al-Kindi (c. 801–873) Ibn al-Haitham

(965-1040)Oldest existent book on

geometry in China

Pinhole perspective projectionPinhole cameraf

f = focal lengtho = aperture = pinhole = center of the camera

o

ïïî

ïïí

ì

=

=

zyf'y

zxf'x

úû

ùêë

é¢¢

=¢®yx

Púúú

û

ù

êêê

ë

é=zyx

P

Pinhole camera

Derived using similar triangles

[Eq. 1]

f

O

P = [x, z]

P’=[x’, f ]

f

zx

fx=¢

i

k

Pinhole camera

[Eq. 2]

f

Kate lazuka ©

Pinhole cameraIs the size of the aperture important?

Shrinkingaperture

size

Adding lenses!-What happens if the aperture is too small?

-Less light passes through

Cameras & Lenses

• A lens focuses light onto the film

imageP

P’

• A lens focuses light onto the film– There is a specific distance at which objects are “in

focus”– Related to the concept of depth of field

Out of focus

Cameras & Lenses

imageP

P’

• A lens focuses light onto the film– There is a specific distance at which objects are “in

focus”– Related to the concept of depth of field

Cameras & Lenses

• A lens focuses light onto the film– All rays parallel to the optical (or principal) axis converge to

one point (the focal point) on a plane located at the focal length f from the center of the lens.

– Rays passing through the center are not deviated

focal point

f

Cameras & Lenses

f

Paraxial refraction model

zy'z'y

zx'z'x

ïïî

ïïí

ì

=

= ozf'z +=

)1n(2Rf -

=

zo-z

Z’

From Snell’s law:

[Eq. 3]

[Eq. 4]

p’ = [x’,y’]

No distortion

Pin cushion

Barrel (fisheye lens)

Issues with lenses: Radial Distortion– Deviations are most noticeable for rays that pass through

the edge of the lens

Image magnification decreases with distance from the optical axis



• Intrinsic• Extrinsic


Pinhole perspective projectionPinhole camera

f = focal lengtho = center of the camera

23 Â®ÂE

ïïî

ïïí

ì

=

=

zyf'y

zxf'x

úû

ùêë

é¢¢

=¢®yx

Púúú

û

ù

êêê

ë

é=zyx

P

[Eq. 1]

f

From retina plane to images

Pixels, bottom-left coordinate systems

fRetina plane

Digital image

Coordinate systems

f

x

y

xc

yc

C’’=[cx, cy]

)czyf,c

zxf()z,y,x( yx ++®

1. Off set

[Eq. 5]

Converting to pixels

)czylf,c

zxkf()z,y,x( yx ++®

1. Off set2. From metric to pixels

x

y

xc

yc

C=[cx, cy] Units: k,l : pixel/m

f : m : pixel

a b

,a bNon-square pixels

f

[Eq. 6]

• Is this a linear transformation?

x

y

xc

yc

C=[cx, cy]

P = (x, y, z)→ P ' = (α xz+ cx, β

yz+ cy )

Is this projective transformation linear?

f

[Eq. 7]

No — division by z is nonlinear

• Can we express it in a matrix form?

Homogeneous coordinates

homogeneous image coordinates

homogeneous scene coordinates

•Converting back from homogeneous coordinates

EàH

HàE

Projective transformation in the homogenous coordinate system

xyz1

!

"

####

$

%

&&&&

M =

α 0 cx 00 β cy 0

0 0 1 0

!

"

####

$

%

&&&&

[Eq.8]

→ P ' = (α xz+ cx, β

yz+ cy )Ph '

Homogenous Euclidian

Ph ' =α x + cx zβ y+ cy z

z

!

"

####

$

%

&&&&

=

α 0 cx 00 β cy 0

0 0 1 0

!

"

####

$

%

&&&&

Ph

P ' =M P

= K I 0!"

#$P

The Camera Matrix

P ' =α 0 cx 00 β cy 0

0 0 1 0

!

"

####

$

%

&&&&

xyz1

!

"

####

$

%

&&&&

Camera matrix K

f

[Eq.9]

Camera Skewness

x

y

xc

yc

C=[cx, cy]θHow many degree does K have?5 degrees of freedom!

f

!P =

α −α cotθ cx 0

0 βsinθ

cy 0

0 0 1 0

#

$

%%%%%

&

'

(((((

xyz1

#

$

%%%%

&

'

((((

Canonical Projective Transformation

P ' =xyz

!

"

####

$

%

&&&&

=1 0 0 00 1 0 00 0 1 0

!

"

###

$

%

&&&

xyz1

!

"

####

$

%

&&&&

xzyz

!

"

####

$

%

&&&&

Pi ' =

P ' =M P

M3

H4 Â®Â

[Eq.10]



• Intrinsic• Extrinsic

• Othercameramodels


World reference system

Ow

iw

kw

jwR,T

•The mapping so far is defined within the camera reference system•What if an object is represented in the world reference system?•Need to introduce an additional mapping from world ref system to camera ref system

f

3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:

úúú

û

ù

êêê

ë

é -=

úúú

û

ù

êêê

ë

é

-=

úúú

û

ù

êêê

ë

é-=

1000cossin0sincos

)(

cos0sin010

sin0cos)(

cossin0sincos0001

)(

gggg

g

bb

bbb

aaaaa

z

y

x

R

R

R

p

x

Y’p’

g

x’

y

zP '→ R 0

0 1

"

#$

%

&'4×4

xyz1

"

#

$$$$

%

&

''''

A rotation matrix in 3D has 3 degree of freedom

2DTranslation

P

P'

t

Please refer to CA session on transformations for

more details

2DTranslationEquation

P

x

y

tx

tyP’

t

)ty,tx(' yx ++=+= tPP

),(),(

yx ttyx

==

tP

2DTranslationusingHomogeneousCoordinates

úúú

û

ù

êêê

ë

é×úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é++

®1100

1001

1' y

xtt

tytx

y

x

y

x

P

P = (x, y)→ (x, y,1)

= I t0 1

!

"#

$

%&⋅

xy1

!

"

###

$

%

&&&=T ⋅

xy1

!

"

###

$

%

&&&

P

x

y

tx

tyP’

t

Scaling

P

P'

ScalingEquation

P

x

y

sx x

P’sy y

úúú

û

ù

êêê

ë

é×úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é®

11000000

1' y

xs

sysxs

y

x

y

x

P

P = (x, y)→ (x, y,1)

S

= S ' 00 1

!

"#

$

%&⋅

xy1

!

"

###

$

%

&&&= S ⋅

xy1

!

"

###

$

%

&&&

)ys,xs(')y,x( yx=®= PP

Rotation

P

P'

RotationEquations• Counter-clockwiserotationbyanangle𝜃

úû

ùêë

éúû

ùêë

é -=ú

û

ùêë

éyx

yx

qqqq

cossinsincos

''P

x

y’P’

q

x’y PRP' =

' cos sinx x yq q= -

' cos siny y xq q= +

Howmanydegreesoffreedom? 1 P '→cosθ −sinθ 0sinθ cosθ 00 0 1

#

$

%%%

&

'

(((

xy1

#

$

%%%

&

'

(((

Scale+Rotation+Translation

P '→

1 0 tx0 1 ty0 0 1

"

#

$$$$

%

&

''''

cosθ −sinθ 0sinθ cosθ 00 0 1

"

#

$$$

%

&

'''

sx 0 0

0 sy 0

0 0 1

"

#

$$$$

%

&

''''

xy1

"

#

$$$

%

&

'''

cos in 0 0sin cos 0 00 0 1 0 0 1 1

x x

y y

s t s xt s y

q qq q

-é ù é ù é ùê ú ê ú ê ú= ê ú ê ú ê úê ú ê ú ê úë û ë û ë û

= R t0 1

!

"#

$

%& S 00 1

!

"#

$

%&xy1

!

"

###

$

%

&&&= R S t

0 1

!

"##

$

%&&

xy1

!

"

###

$

%

&&&

Ifsx =sy,thisisasimilaritytransformation

3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:

úúú

û

ù

êêê

ë

é -=

úúú

û

ù

êêê

ë

é

-=

úúú

û

ù

êêê

ë

é-=

1000cossin0sincos

)(

cos0sin010

sin0cos)(

cossin0sincos0001

)(

gggg

g

bb

bbb

aaaaa

z

y

x

R

R

R

P

x

y’P’

x’

y

zP '→ R 0

0 1

"

#$

%

&'4×4

xyz1

"

#

$$$$

%

&

''''

A rotation matrix in 3D has 3 degrees of freedom

R = Rx (α) Ry (β) Rz (γ )

3D Translation of Points

P '→ I T0 1

"

#$

%

&'4×4

xyz1

"

#

$$$$

%

&

''''

A translation vector in 3D has 3 degrees of freedom

P

x

y

Tx

Ty

P’

T

z

T =

TxTyTz

!

"

####

$

%

&&&&

3D Translation and Rotation

P '→ R T0 1

"

#$

%

&'4×4

xyz1

"

#

$$$$

%

&

''''

T =

TxTyTz

!

"

####

$

%

&&&&

R = Rx (α) Ry (β) Rz (γ )

= K R T!"

#$Pw

World reference system

Ow

iw

kw

jwR,T

P = R T0 1

!

"#

$

%&4×4

PwIn 4D homogeneous coordinates:

Internal parameters External parameters

P ' = K I 0!"

#$P =K I 0!

"#$

R T0 1

!

"%

#

$&4×4

Pw

M

P

P’

f

xwywzw1

!

"

#####

$

%

&&&&&

[Eq.11]

P '3×1 =M3x4 Pw = K3×3 R T"#

$% 3×4

Pw4×1

The projective transformation

Ow

iw

kw

jwR,T

How many degrees of freedom?5 + 3 + 3 =11!

P

P’

f

The projective transformation

Ow

iw

kw

jwR,T

P '3×1 =M Pw = K3×3 R T"#

$% 3×4

Pw4×1úúú

û

ù

êêê

ë

é=

3

2

1

mmm

M

=

m1

m2

m3

!

"

####

$

%

&&&&

PW =m1PWm2PWm3PW

!

"

####

$

%

&&&&

→ (m1Pwm3Pw

, m2Pwm3Pw

)E

P

P’

f

[Eq.12]

Theorem (Faugeras, 1993)

úúú

û

ù

êêê

ë

é=

3

2

1

aaa

A[ ] [ ] ][ bATKRKTRKM ===[Eq.13]

Properties of projective transformations• Points project to points• Lines project to lines• Distant objects look smaller

Properties of Projection•Angles are not preserved•Parallel lines meet!

Parallel lines in the world intersect in the image at a “vanishing point”

Horizon line (vanishing line)

One-point perspective• Masaccio, Trinity,

Santa Maria Novella, Florence, 1425-28

Credit slide S. Lazebnik

Next lecture

•How to calibrate a camera?

Supplemental material

Thin Lenses

Snell’s law:n1 sin a1 = n2 sin a2

Small angles:n1 a1 » n2 a2

n1 = n (lens)n1 = 1 (air)

zo

zy'z'y

zx'z'x

ïïî

ïïí

ì

=

=

ozf'z +=

)1n(2Rf -

=

Focal length

[FP] sec 1.1, page 8.

Horizon line (vanishing line)

horizon

¥l

lecture2 camera models - Stanford...

Documents

Transcript of lecture2 camera models - Stanford...