1

Formation et Analyse d’Images

Daniela Hall

30 Septembre 2004

2

Course material

• course slides (.pdf)– http://www-prima.inrialpes.fr/perso/Hall/Courses/FAI0

4/• last years documents

– http://www-prima.inrialpes.fr/Prima/Homepages/jlc/Courses/2003/ENSI3/FAI/ENSI3.FAI.html

• Tensor notation– book: R. Hartley, A.Zisserman: Multiple view

geometry in computer vision, Appendix 1, Cambridge University Press, 2000

3

Course Overview

• Session 1: – Homogenous coordinates and tensor notation– Image transformations– Camera models

• Session 2:– Camera models– Reflection models– Color spaces

• Session 3:– Review color spaces– Pixel based image analysis– Gaussian filter operators

• Session 4:– Scale Space

4

Course overview

• Session 5:– Contrast description– Hough transform

• Session 6:– Kalman filter– Tracking of regions, pixels, and lines

• Session 7:– Stereo vision – Epipolar geometry

• Session 8: exam

5

Session Overview

1. Homogeneous coordinates

2. Tensor notation

3. Image transformations

4. Camera models

6

Motivation

How can we transform images?Apply transformation to all pixelsFirst do translation, then rotation, then scaling

tSRvv

7

Motivation

• Transformation in 2D

• Transformation using homogenous coordinatestSRvv

110012221

1211

y

x

taa

taa

y

x

y

x

8

Session Overview

1. Homogeneous coordinates

2. Tensor notation

3. Image transformations

4. Camera models

9

Homogenous coordinates

• Allow to manipulate n-dim vectors in a n+1-dim space

• A point p can be written as vector • In homogenous coordinates we add a scaling factor

• To transform the homogenous coordinates in normal coordinate, divide by the n+1 coordinate.

y

xp

1

y

x

w

wy

wx

ph

10

Homogenous coordinates

• we note

• Proof:

11

:, y

x

by

x

aba

1/

/

/

1

y

x

aa

aay

aax

a

ay

ax

y

x

a

11

Translation

• Classic • Homogenous coordinates

y

x

tyy

txx

12

12

1

1

1

100

10

01

1

2

2

y

x

ty

tx

y

x

12

Rotation (clockwise)

• Classic • Homogenous coordinates

1)cos(1)sin(

1)sin(1)cos(

2

2

yaxay

yaxax

1

1

1

100

0)cos()sin(

0)sin()cos(

1

2

2

y

x

aa

aa

y

x

x

y

p

pl

5

7.8

087.0105.0

05.01087.0,

0

1030lpp

13

Translation and rotation

• Classic • Homogenous coordinates

tyyaxay

txyaxax

1)cos(1)sin(

1)sin(1)cos(

2

2

1

1

1

100

)cos()sin(

)sin()cos(

1

2

2

y

x

tyaa

txaa

y

x

14

Translation, rotation and scaling

• Classic • Homogenous coordinates

tyyasyxasxy

txyasyxasxx

1)cos(1)sin(

1)sin(1)cos(

2

2

1

1

1

100

)cos()sin(

)sin()cos(

1

2

2

y

x

tyasyasx

txasyasx

y

x

15

Session Overview

1. Homogeneous coordinates

2. Tensor notation

3. Image transformations

4. Camera models

16

Tensor notation

• In tensor notation a superscript stands for a column vector

• a subscript for a row vector (useful to specify lines)

• A matrix is written as

3

2

1

3,2,1)(

p

p

p

pP iii

),,( 321 lllLi

),,( 321

3

2

1

33

32

31

23

22

21

13

12

11

jjj

i

i

iji

MMM

M

M

M

mmm

mmm

mmm

M

17

Tensor notation

• Tensor summation convention:– an index repeated as sub and superscript in a

product represents summation over the range of the index.

• Example: 3

32

21

1 plplplPL ii

18

Tensor notation

• Scalar product can be written as

• where the subscript has the same index as the superscript. This implicitly computes the sum.

• This is commutative

• Multiplication of a matrix and a vector

• This means a change of P from the coordinate system i to the coordinate system j (transformation).

33

22

11 plplplPL i

i

iii

i LPPL

iji

j PMP

19

Line equation

• In R a line is defined by the equation

• In homogenous coordinates we can write this as

• In tensor notation we can write this as

0 cbyax

0

1

),,(

y

x

cbaPLT

0iiPL

20

The tensor operator Eijk and Eijk

• The tensor Eijk is defined for i,j,k=1,...,3 as

123 ofn permutatio oddan isrst if 1,-

123 ofn permutatioeven an isrst if 1,

distinct are t ands,r, unless 0

rstE

3 2 1 1 3 22 1 3

1 2 3 2 3 1 3 1 2

evenodd

21

Determinant in tensor notation

jjjji mmmM 321 ,,

kjiijk

ji mmmEM 321)det(

22

Cross product in tensor notation

bac

kjijkii baEbac )(

23

Example

• Line equation in tensor notation

kjijki QPEL

1

,

1

2

1

2

1

q

q

Qp

p

P kj

0

),,(

321

12213

1131132

1

321

222332

lylxl

qpqpl

qpqpqpl

qpqpqpl

lllLi

24

Example

• Intersection of two lines

• L: l1x+l2y+l3=0, M: m1x+m2y+m3=0

• Intersection:

• Tensor:

• Result:

1221

3113,

1221

2332

mlml

mlmly

mlml

mlmlx

kjijki MLEP

12213

31132

23321

mlmlp

mlmlp

mlmlp

25

Translation

• Classic

• Tensor notation

• T is a transformation from the system A to B

• Homogenous coordinates

y

x

tyy

txx

12

12

1

1

1

100

10

01

1

2

2

y

x

ty

tx

y

x

1,2,3 BA, with ABA

B PTP

26

Rotation

• Homogenous coordinates

1)cos(1)sin(

1)sin(1)cos(

2

2

yaxay

yaxax

1

1

1

100

0)cos()sin(

0)sin()cos(

1

2

2

y

x

aa

aa

y

x

• Classic

• Tensor notation1,2,3 BA, with AB

AB PRP

27

Session Overview

1. Homogeneous coordinates

2. Tensor notation

3. Image transformations

4. Camera models

28

Image transformation

For each position Pd in the destination image we searchthe pixel color I(Pd).

Source image Destination image

Tsd

29

Image transformation

First we compute a position Ps in the source image.

dsd

s PTP

Source image Destination image

Tsd

1100

)cos()sin(

)sin()cos(

1d

d

yyx

xyx

s

s

y

x

tasas

tasas

y

x

30

Image transformation

• P is not integer.• How do we compute I(Pd)=I(Ps)?• Answer: by a linear combination of the

neighboring pixels I(Psi) (interpolation).

Ps0 Ps1

Ps3Ps2

Ps

Pd

Tsd

31

Interpolation methods

• 0th order: take value of closest neighbor– fast, applied for binary images

• 1st order: linear interpolation and bi-linear interpolation

• 3rd order: cubic spline interpolation

32

1D linear interpolation

Ps0 Ps1

Ps3Ps2

Ps

position P

intensity I(P)

Ps0 Ps Ps1

)()()())(1(

)()()()(

)()(

1000

00

01

ssssss

ssssd

ss

PIPPPIPP

PIPPmPIPI

PIPIm

Gradient

Pixel color

33

2D linear interpolation

Ps0 Ps1

Ps3Ps2

Ps

x

intensity I(P)

Ps0

Ps

Ps1

)()()()()(

)()(

)()(

000

02

01

sssyy

ssxx

sd

ssy

ssx

PIPPmPPmPIPI

PIPIm

PIPIm

Gradient

Pixel color

I(Ps)

y

Psx

Psy

34

Bi-linear interpolation

Ps0 Ps1

Ps3Ps2

Ps

x

intensity I(P)

Ps0

Ps

Ps1

)())((

)()()(

)()(

)()(

)()(

000

00

03

02

01

sssyssxxy

ssyy

ssxx

s

ssxy

ssy

ssx

PIPPPPm

PPmPPmPI

PIPIm

PIPIm

PIPIm

I(Ps)

y

Ps3

The bilinear approach computes the weighted average of thefour neighboring pixels.

Psx

Psy

Bilinear formula:dcxybyaxyxp ),(

35

Higher order interpolation

• Cubic spline interpolation takes into account more than only the closest pixels.

• Result: more expensive to compute, but image has less artefacts, image is smoother.

36

Session Overview

1. Homogeneous coordinates

2. Tensor notation

3. Image transformations

4. Camera model

37

Camera model

• Physical geometry

38

Camera model

• Projective model

Ti

Trr

r

Tccc

c

Tsss

s

jiP

yxP

zyxP

zyxP

)1,,(

)1,,(

)1,,,(

)1,,,(

Scene coordinates

Camera coordinates

Image coordinates

39

Change of coordinate systems

• Transformation from scene to camera coordinates

• Projection of camera coordinates to retina coordinates

• Transformation from retina coordinates to image coordinates

• Composition sis

scs

rc

ir

i

rir

i

crc

r

scs

c

PMPTMCP

wPCP

PMP

PTP

40

Transformation Scene - Camera

• (xs,ys,zs) is position of the origin of the camera system with respect to the scene coordinates (translation).

• R is the orientation of the camera system with respect to the scene system (3d rotation).

1000s

s

scsc

sz

y

x

RT

41

3d rotation

• Around x-axis (counter-clockwise)

• Around y-axis

• Around z-axis

• General )()()(,

1000

1000

0100

00)cos()sin(

00)sin()cos(

)(

1000

0)cos(0)sin(

0010

0)sin(0)cos(

)(

1000

0)cos()sin(0

0)sin()cos(0

0001

)(

aRbRcRRtz

ty

tx

RT

cc

cc

cR

bb

bb

bR

aa

aaaR

xyz

z

y

x