CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt....P c M P w K >R T @ P w Internal...

CEE598 - Visual Sensing for

Civil Infrastructure Eng. & Mgmt.

Session 4 – Camera Calibration

Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign

Mani Golparvar-Fard Department of Civil and Environmental Engineering

3129D, Newmark Civil Engineering Lab

e-mail: [email protected]

Outline

Camera Calibration

• Review Camera Parameters

• Camera Calibration Problem

• Example

2

CEE598 Visual Sensing for Civil Infrastructure Eng. & Mgmt. © Mani Golparvar-Fard, 2013 Some slides in this lecture are courtesy to Profs. S. Savarese, J.

Ponce & F-F Li

Reading: [FP] Chapter 3

[HZ] Chapter 7

[S] Chapter 6

Pinhole perspective projection f

Oc

f = focal length

Projective camera

CEE598 Visual Sensing for Civil Infrastructure Eng. & Mgmt. © Mani Golparvar-Fard, 2013

3

Pinhole perspective projection

x

y

xc

yc

C=[uo, vo]

f

Oc

f = focal length

uo, vo = offset

Projective camera


4

f

Oc

Units: k,l [pixel/m]

f [m]

[pixel] , Non-square pixels

f = focal length

uo, vo = offset

non-square pixels ,

Projective camera


5

x

y

xc

yc

C=[uo, vo]

f

Oc

f = focal length

uo, vo = offset

non-square pixels , = skewness

Projective camera


6

f

Oc

1

z

y

x

0100

0v0

0ucot

P o

o

sin

K has 5 degrees of freedom!

Pc

P’

f = focal length

uo, vo = offset


Projective camera


7

f

Oc

Pc

Ow

iw

kw

jw

R,T

wc PTRP

P’

f = focal length

uo, vo = offset


R,T = rotation, translation

Projective camera


8

f = focal length

uo, vo = offset

non-square pixels ,

f

Oc

P

Ow

iw

kw

jw

R,T

wPMP

wPTRK

Internal parameters

External parameters

= skewness

R,T = rotation, translation

P’

Projective camera


9

wPMP wPTRK

Internal parameters

External parameters

Goal of calibration

Estimate intrinsic and extrinsic parameters

from 1 or multiple images


10

wPMP wPTRK

Note: To simplify

notation let P = Pw

100

v0

ucot

K o

o

sin

T

3

T

2

T

1

R

r

r

r

z

y

x

t

t

t

T

Goal of calibration

Estimate intrinsic and extrinsic parameters

from 1 or multiple images


11

•P1… Pn with known positions in [Ow,iw,jw,kw]

•p1, … pn known positions in the image

Goal: compute intrinsic and extrinsic parameters

jC

Calibration rig

Calibration Problem


12

jC

Calibration rig

How many correspondences do we need?

•P has 11 unknown • We need 11 equations • 6 correspondences would do it

Calibration Problem


13

image jC

Calibration rig

In practice: user may need to look at the

image and select the n>=6 correspondences

Calibration Problem


14

jC

ii PMP

i

i

iv

up

3

2

1

M

m

m

m

i3

i2

i3

i1

P

P

P

P

m

m

m

m

in pixels

Calibration Problem


15

i3

i1i

P

Pu

m

m

i2i3i P)P(v mm

i1i3i P)P(u mm

i3

i2i

P

Pv

m

m

i

i

v

u

i3

i2

i3

i1

P

P

P

P

m

m

m

m

0Pv

ui

3i2

3i1

mm

mm

Calibration Problem


16

0Pv

ui

3i2

3i1

mm

mm

0Pv

un

3n2

3n1

mm

mm

…

…

0Pv

u1

312

311

mm

mm

Calibration Problem


17

2221

1211

2221

1211

BB

BBB

AA

AAA

What is AB ?

2222122121221121

2212121121121111

BABABABA

BABABABAAB

Review on Block Matrix Multiplication


18

2n x 12 12x1

1x4

0Pv

ui

3i2

3i1

mm

mm

T

3

T

2

T

1def

m

m

m

m

4x1

0Pv

un

3n2

3n1

mm

mm

…

…

Homogenous linear system

known unknown

Calibration Problem


19


Homogeneous M x N Linear Systems

A

A

x

x 0

0 =

=

Square system (M=N):

Rectangular system (M>N)

• 0 is always a solution

• Rank(A) = N

Minimize |Ax|2

under the constraint |x|2 =1

M x N

M=number of equations

N=number of unknown

• noisy measurements

20

How do we solve this homogenous linear system?

Calibration Problem


21

ixix

H

ii xHx

0xHx ii 0Ai h

Function of measurements

unknown

DLT algorithm (Direct Linear Transformation)


22

1212T

121212n2 VDU

Last column of V gives m

iPM ip

General Calibration Problem


23

A

T

3

T

2

T

1

A

a

a

a

TRK

3

1

a

3

2

1

b

b

b

b

Estimated values

)(u 21

2

o aa

)(v 32

2

o aa

3231

3231cosaaaa

aaaa

Intrinsic

b

100

v0

ucot

K o

o

sin

Extracting camera parameters


24

Theorem (Faugeras, 1993)


25

A

T

3

T

2

T

1

A

a

a

a

TRK

3

2

1

b

b

b

b

Estimated values

Intrinsic

sin31

2aa

sin32

2aa

b

f



26

A

T

3

T

2

T

1

A

a

a

a

b

TRK

3

2

1

b

b

b

b

Estimated values

Extrinsic

32

321

aa

aar

3

3

1

ar

132 rrr b1KT



27

•Pi’s cannot lie on the same plane!

• Points cannot lie on the intersection curve of two

quadric surfaces

Degenerate cases


28

Taking lens distortions into account

Chromatic Aberration

Spherical Aberration

Radial Distortion


29

No distortion

Pin cushion

Barrel

Radial Distortion

• Caused by imperfect lenses

• Deviations are most noticeable for rays that pass through the

edge of the lens


30

Radial Distortion


31

i

i

i

i pv

uPM

100

00

00

1

1

d

v

vucvbuad 222

u

3

1p

2p

pdκ1λ

Polynomial function

Distortion coefficient

To model radial behavior

Radial Distortion


32

Estimating m1 and m2…

i

i

iv

up

i3

i2

i3

i1

P

P

P

P

1

m

m

m

m

How to do that?

d

v

u Hint: slopev

u

i

i

Radial Distortion


33

Estimating m1 and m2…

i

i

iv

up

i3

i2

i3

i1

P

P

P

P

1

m

m

m

m

0)()( 121111 PuPv mm

0)()( 21 iiii PuPv mm

0)()( 21 nnnn PuPv mm…

0Q n

2

1

m

mn

Tsai technique [87]

i

i

i

i

i

i

i

i

P

P

P

P

P

P

v

u

2

1

3

2

3

1

)(

)(

)(

)(

m

m

m

m

m

m

Radial Distortion


34

Once that m1 and m2 are estimated…

i

i

iv

up

i3

i2

i3

i1

P

P

P

P

1

m

m

m

m

3m is non linear function of 1m

2m

There are some degenerate configurations for which m1 and m2 cannot be computed

Radial Distortion


35

i

i

iv

up

3

2

1

Q

q

q

q

i3

i2

i3

i1

P

P

P

P

q

q

q

q

i

i

i

i pv

uPM

100

00

00

1

1

Q

PPv

PPu

2i3i

i1i3i

qq

qq

Non-linear system of equations

Radial Distortion


36

Radial Distortion

Conversion from distorted coordinates to

undistorted coordinates

• p' = f * r(p) * p

(conversion to pixel coordinates)

r(p) is a function that computes a scaling factor to

undo the radial distortion:

• r(p) = 1.0 + k1 * ||p||^2 + k2 * ||p||^4.


37

)(PfX

measurement parameter

f( ) is nonlinear

-Newton Method

-Levenberg-Marquardt Algorithm

• Iterative, starts from initial solution

• May be slow if initial solution far from real solution

• Estimated solution may be function of the initial solution

• Newton requires the computation of J, H

• Levenberg-Marquardt doesn’t require the computation of H



38

A possible algorithm

1. Solve linear part of the system to find approximated solution

2. Use this solution as initial condition for the full system

3. Solve full system using Newton or L.M.

)(PfX


f( ) is nonlinear



39

Typical assumptions:

- zero-skew, square pixel

- uo, vo = known center of the image

- no distortion

Just estimate f

and R, T

)(PfX


f( ) is nonlinear



40

Camera Calibration Toolbox for Matlab

J. Bouguet – [1998-2000]

http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples

Calibration Procedure


41



42



43



44



45


46



47



48


Next lecture Single view reconstruction

• Assignment 1 will be online Next Tuesday


49

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt....P c M P w K >R T @ P w Internal...

Documents

Transcript of CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt....P c M P w K >R T @ P w Internal...