1

Formation et Analyse d’ImagesSession 2

Daniela Hall

26 September 2005

2

Course Overview

• Session 1: – Overview– Human vision – Homogenous coordinates– Camera models

• Session 2:– Tensor notation– Image transformations– Homography computation

• Session 3:– Reflection models– Color spaces

• Session 4:– Pixel based image analysis

• Session 5:– Gaussian filter operators– Scale Space

3

Course overview

• Session 6:– Contrast description– Hough transform

• Session 7:– Kalman filter

• Session 8:– Tracking of regions, pixels, and lines

• Session 9:– Stereo vision

• Session 10:– Epipolar geometry

• Session 11: exercises and questions

4

Trifocal Tensor

• A tensor is used in 3d position estimation with multiple cameras. The (trifocal) tensor encapsulates all the (projective) geometric relations between 3 camera views independent of the scene.

• Reference– book: R. Hartley, A.Zisserman: Multiple view geometry in

computer vision, Appendix 1, Cambridge University Press, 2000

– Exists on-line http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html

5

Tensor notation

• In tensor notation a superscript stands for a column vector

• a subscript for a row vector (useful to specify lines)

• A matrix is written as

3

2

1

3,2,1)(

p

p

p

pP iii

),,( 321 lllLi

),,( 321

3

2

1

33

32

31

23

22

21

13

12

11

jjj

i

i

iji

MMM

M

M

M

mmm

mmm

mmm

M

6

Tensor notation

• Tensor summation convention:– an index repeated as sub and superscript in a

product represents summation over the range of the index.

• Example: 3

32

21

1 plplplPL ii

7

Tensor notation

• Scalar product can be written as

• where the subscript has the same index as the superscript. This implicitly computes the sum.

• This is commutative

• Multiplication of a matrix and a vector

• This means a change of P from the coordinate system i to the coordinate system j (transformation).

33

22

11 plplplPL i

i

iii

i LPPL

iji

j PMP

8

Line equation

• In R2 a line is defined by the equation

• In homogenous coordinates we can write this as

• In tensor notation we can write this as

0 cbyax

0

1

),,(

y

x

cbaPLT

0iiPL

9

The tensor operator Eijk and Eijk

• The tensor Eijk is defined for i,j,k=1,...,3 as

123 ofn permutatio oddan isrst if 1,-

123 ofn permutatioeven an isrst if 1,

distinct are t ands,r, unless 0

rstE

3 2 1 1 3 22 1 3

1 2 3 2 3 1 3 1 2

evenodd

10

Determinant in tensor notation

jjjji mmmM 321 ,,

kjiijk

ji mmmEM 321)det(

11

Cross product in tensor notation

bac

kjijkii baEbac )(

12

Example

• Line equation in tensor notation

kjijki QPEL

1

,

1

2

1

2

1

q

q

Qp

p

P kj

0

),,(

321

12213

1131132

1

321

222332

lylxl

qpqpl

qpqpqpl

qpqpqpl

lllLi

13

Example

• Intersection of two lines

• L: l1x+l2y+l3=0, M: m1x+m2y+m3=0

• Intersection:

• Tensor:

• Result:

1221

3113,

1221

2332

mlml

mlmly

mlml

mlmlx

kjijki MLEP

12213

31132

23321

mlmlp

mlmlp

mlmlp

14

Translation

• Classic

• Tensor notation

• T is a transformation from the system A to B

• Homogenous coordinates

y

x

tyy

txx

12

12

1

1

1

100

10

01

1

2

2

y

x

ty

tx

y

x

1,2,3 BA, with ABA

B PTP

15

Rotation

• Homogenous coordinates

1)cos(1)sin(

1)sin(1)cos(

2

2

yaxay

yaxax

1

1

1

100

0)cos()sin(

0)sin()cos(

1

2

2

y

x

aa

aa

y

x

• Classic

• Tensor notation1,2,3 BA, with AB

AB PRP

16

Image transformation

For each position Pd in the destination image we searchthe pixel color I(Pd).

Source image Destination image

Tsd

17

Image transformation

First we compute a position Ps in the source image.

dsd

s PTP

Source image Destination image

Tsd

1100

)cos()sin(

)sin()cos(

1d

d

yyx

xyx

s

s

y

x

tasas

tasas

y

x

18

Image transformation

• P is not integer.• How do we compute I(Pd)=I(Ps)?• Answer: by a linear combination of the

neighboring pixels I(Psi) (interpolation).

Ps0 Ps1

Ps3Ps2

Ps

Pd

Tsd

19

Interpolation methods

• 0th order: take value of closest neighbor– fast, applied for binary images

• 1st order: linear interpolation and bi-linear interpolation

• 3rd order: cubic spline interpolation

20

1D linear interpolation

Ps0 Ps1

Ps3Ps2

Ps

position P

intensity I(P)

Ps0 Ps Ps1

)()()())(1(

)()()()(

)()(

1000

00

01

ssssss

ssssd

ss

PIPPPIPP

PIPPmPIPI

PIPIm

Gradient

Pixel color

21

2D linear interpolation

Ps0 Ps1

Ps3Ps2

Ps

x

intensity I(P)

Ps0

Ps

Ps1

)()()()()(

)()(

)()(

000

02

01

sssyy

ssxx

sd

ssy

ssx

PIPPmPPmPIPI

PIPIm

PIPIm

Gradient

Pixel color

I(Ps)

y

Psx

Psy

22

Bi-linear interpolationPs0 Ps1

Ps3Ps2

Ps

)()()()()(

,)1(

),1(),1)(1(

3210

0

0

sssss

yssy

xssx

PDIPCPBPAIPI

dxdyDdydxC

dydxBdydxA

PPdy

PPdx

The bilinear approach computes the weighted average of the four neighboring pixels. The pixels are weighted according tothe area.

AB

CD

23

Higher order interpolation

• Cubic spline interpolation takes into account more than only the closest pixels.

• Result: more expensive to compute, but image has less artefacts, image is smoother.

24

Homographie: projection from one plane to another

• Homographie HBA is bijective QB = HB

A PA

jj

jjB

jj

jjB

A

j

j

jAB

AB

B

PHPHy

PHPHx

P

H

H

H

PH

w

wy

wx

32

31

3

2

1

/

/

25

Homography computation

• H can be computed from 4 point correspondences.

Ps1 Ps2

Ps3Ps4

Rd1 Rd2

Rd3Rd4Source image (observed)

Destination image (rectified)

4,3,2,1 and 2,1 with ,0 iSRHPRHP DiSD

SiDiSD

Si

26

Homography computation

• H is 3x3 matrix and has 8 degrees of freedom (homogenous coordinates)

• gives 8 equations and one solution for H.

4,3,2,1 and 2,1 with ,0 iSRHPRHP DiSD

SiDiSD

Si

27

Application: Rectifying images

28

Applications