Cameras and Projections

Dan Witzner Hansen

Course web page:www.itu.dk/courses/MCV

Email:witzner@itu.dk

Previously in Computer Vision….

• Homographies• Estimating homographies• Applications (Image rectification)

Outline• Projections• Pinhole cameras• Perspective projection

– Camera matrix– Camera calibration matrix

• Affine Camera Models

Single view geometry

Camera model

Camera calibration

Single view geom.

Pinhole camera model

TT ZfYZfXZYX )/,/(),,(

101

0

0

1

Z

Y

X

f

f

Z

fY

fX

Z

Y

X

Pinhole camera model

101

0

0

Z

Y

X

f

f

Z

fY

fX

101

01

01

1Z

Y

X

f

f

Z

fY

fX

PXx

0|I)1,,(diagP ff

Principal point offset

Tyx

T pZfYpZfXZYX )/,/(),,(

principal pointT

yx pp ),(

101

0

0

1

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

Z

Y

X

y

x

x

x

Principal point offset

101

0

0

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

y

x

x

x

camX0|IKx

1y

x

pf

pf

K calibration matrix

Camera rotation and translation

C~

-X~

RX~

cam

X10

RCR

1

10

C~

RRXcam

Z

Y

X

camX0|IKx XC~

|IKRx

t|RKP C~

Rt PXx

CCD camera

1yx

xx

p

p

K

11y

x

x

x

pf

pf

m

m

K

Finite projective camera

1yx

xx

p

ps

K

1yx

xx

p

p

K

C~

|IKRP

non-singular

11 dof (5+3+3)

decompose P in K,R,C?

4p|MP 41pMC

~ MRK, RQ

{finite cameras}={P4x3 | det M≠0}

If rank P=3, but rank M<3, then cam at infinity

Camera anatomy

Camera centerColumn pointsPrincipal planeAxis planePrincipal pointPrincipal ray

Camera center

0PC

null-space camera projection matrix

λ)C(1λAX

λ)PC(1λPAPXx

For all A all points on AC project on image of A,

therefore C is camera center

Image of camera center is (0,0,0)T, i.e. undefined

Finite cameras:

1

pM 41

C

Infinite cameras: 0Md,0

d

C

Column vectors

0

0

1

0

ppppp 43212

Image points corresponding to X,Y,Z directions and origin

Row vectors

1p

p

p

0 3

2

1

Z

Y

X

y

x

T

T

T

1p

p

p0

3

2

1

Z

Y

X

w

yT

T

T

note: p1,p2 dependent on image reparametrization

The principal point

principal point

0,,,p̂ 3332313 ppp

330 Mmp̂Px

Action of projective camera on point

PXx

MdDp|MPDx 4

Forward projection

Back-projection

xPX 1PPPP

TT IPP

(pseudo-inverse)

0PC

λCxPλX

1

p-μxM

1

pM-

0

xMμλX 4

-14

-1-1

xMd -1

CD

Camera matrix decomposition

Finding the camera center

0PC (use SVD to find null-space)

432 p,p,pdetX 431 p,p,pdetY

421 p,p,pdetZ 321 p,p,pdetTFinding the camera orientation and internal parameters

KRM (use RQ decomposition ~QR)

Q R=( )-1= -1 -1QR

(if only QR, invert)

Euclidean vs. projective

homography 44

0100

0010

0001

homography 33P

general projective interpretation

Meaningful decomposition in K,R,t requires Euclidean image and space

Camera center is still valid in projective space

Principal plane requires affine image and space

Principal ray requires affine image and Euclidean space

Cameras at infinity

00

dP

Camera center at infinity

0Mdet

Affine and non-affine cameras

Definition: affine camera has P3T=(0,0,0,1)

Affine cameras

Summary parallel projection

1000

0010

0001

P canonical representation

10

0KK 22 calibration matrix

principal point is not defined

A hierarchy of affine cameras

Orthographic projection

Scaled orthographic projection

1000

0010

0001

P

10

tRH

10rr

P 21T

11T

tt

ktt

/10rr

P 21T

11T

(5dof)

(6dof)

A hierarchy of affine cameras

Weak perspective projection

ktt

y

x

/10rr

1α

αP 2

1T1

1T

(7dof)

1. Affine camera= proj camera with principal plane coinciding with P∞

2. Affine camera maps parallel lines to parallel lines3. No center of projection, but direction of

projection PAD=0(point on P∞)

A hierarchy of affine camerasAffine camera

ktts

y

x

A

/10rr

1α

αP 2

1T1

1T

(8dof)

1000P 2232221

1131211

tmmmtmmm

A

affine 44100000100001

affine 33P

A

Next: Camera calibration

The principal axis vector

3m

camcamcam X0|IKXPx T1,0,0mMdetv 3

camcam PP k vv 4k

4p|MC~

|IKRP k

0)Rdet(

vector defining front side of camera

(direction unaffected)

vmMdetv 43 kkk camcam PP k

because

Depth of points

C~

X~

mCXPXPT3T3T3 w

(dot product)(PC=0)

1m;0det 3 MIf , then m3 unit vector in positive direction

3m

)sign(detMPX;depth

T

w

TX X,Y,Z,T

When is skew non-zero?

1yx

xx

p

ps

K

1 g

arctan(1/s)

for CCD/CMOS, always s=0

Image from image, s≠0 possible(non coinciding principal axis)

HPresulting camera: