1

Formation et Analyse d’Images

Daniela Hall

19 Septembre 2005

Daniela.Hall@inrialpes.fr

2

Course material

• course slides (.pdf)– http://www-prima.inrialpes.fr/perso/Hall/Courses/FAI0

5/• References on the slides• last years documents

– http://www-www-prima.inrialpes.fr/perso/Hall/Courses/FAI04/

• CVonline:– http://homepages.inf.ed.ac.uk/rbf/CVonline/

• Contact: email Daniela.Hall@inrialpes.fr

3

Course Overview

• Session 1: – Overview– Human vision – Homogenous coordinates– Camera models

• Session 2:– Tensor notation– Image transformations

• Session 3:– Reflection models– Color spaces

• Session 4:– Pixel based image analysis

• Session 5:– Gaussian filter operators– Scale Space

4

Course overview

• Session 6:– Contrast description– Hough transform

• Session 7:– Kalman filter

• Session 8:– Tracking of regions, pixels, and lines

• Session 9:– Stereo vision

• Session 10:– Epipolar geometry

• Session 11: exercises and questions

5

Session Overview

1. Course overview2. Image formation in the eye and the camera

1. The eye2. The retina3. Capacities of the eye

3. Homogenous coordinates4. Camera models

1. Pinhole camera2. Perspective projection3. Mathematical formulation

6

Image formation in the eye

• Biological vision: process of using light reflected from surrounding world for modifying behaviour.

• Humans require conscious understanding of the 3d world from 2d projections on the retina.

• Surrounding environment is interpreted by visual input.

Ref: CVonline/LOCAL_COPIES/OWENS/LECT1/

7

The eye

aqueous humor

8

The eye

• Light enters the eye through the transparent cornea, passes through the aqueous humor, the lens and the vitreous humor, where it forms the image on the retina.

• Accomodation: muscular adjustment of the lens that focuses the image directly on the retina.

9

The retina

• Complex tiling of photoreceptors (rods and cones).

• Photoreceptors stimulated by light transmit electrical signals to the brain via the optic nerve.

• Location of the optic nerve on the retina has no photoreceptors. No light is perceived within this region (blind spot).

10

The retina

• Rods and cones communicate through several layers of cells to the ganglion cells

• Synapses: junctions between layers

• Rods and cones are situated at the back of the retina.

• Light passes through the different layers and the signal is transmitted back via synaptic junctions to the optic nerve.

11

The retina

• Ganglion cell responds to the photostimulus according to a receptive field.

• The spatial organization of the receptive field on the retinal ganglion cell is circular symmetric (either excitatory center and inhibitory suround or inverse). Such cells are known as mexican hat operators

• Other spatial organization are – simple cells (orientation sensitive receptive fields),– complex cells (non-linear combination of even and odd responses)

and – end-stopped cells (simple differentiation operators).

12

Capacities of the eye

• Sensitivity and performance approach the limits set by quantum physics.

• Eye can detect a single photon• Eye adapts to ranges in light of many orders of magnitude• No camera can even partially match this performance• Inputs from left and right are processed in the optic

chiasma.• The slight difference of viewpoint of the eyes is used to

deduce depth. • Nerve fibers lead from the optic chiasma to the striate

cortex (the seat of visual processing in the brain).

13

Session Overview

1. Course overview2. Image formation in the eye and the camera

1. The eye2. The retina3. Capacities of the eye

3. Camera models1. Pinhole camera2. Homogenous coordinates3. Perspective projection4. Mathematical formulation

14

Camera model

• Physical geometry

The pinhole camera is the simplest. It has a infinitesimally smallhole through which light enters and forming an inverted image on the camera surface (retina). To simplify things, we model a pinhole camera by placing the retina between the camera center and the object (projective model)

15

Camera model

• Projective model

Ti

Trr

r

Tccc

c

Tsss

s

jiP

yxP

zyxP

zyxP

)1,,(

)1,,(

)1,,,(

)1,,,(

Scene coordinates

Camera coordinates

Image coordinates

16

Change of coordinate systems

• Transformation from scene to camera coordinates

• Projection of camera coordinates to retina coordinates

• Transformation from retina coordinates to image coordinates

• Composition

• This mapping from 3 dimensions to 2 is called perspective projection

sis

scs

rc

ir

i

rir

i

crc

r

scs

c

PMPTMCP

wPCP

PMP

PTP

17

Camera parameters

• 4 intrinsic parameters:– 2 for the origin of the image coordinate frame– 2 for the scale of the axes– Focal length F

• 6 extrinsic parameters:– 3 for the 3D position of the center of projection– 3 for the orientation of the image plane

18

Intrinsic camera parameters

• F: focal distance

• Ci , Cj : Optical image center (in pixels)

• Di , Dj : Physical size of the pixel on the retina (in pixel/mm)

• i, j : image coordinates (in pixels)• Transformation Retina-Image

jjr

iir

CDyj

CDxi

1100

0

0

1r

r

jj

ii

y

x

CD

CD

j

i

19

Homogenous coordinates

• Allow to manipulate n-dim vectors in a n+1-dim space

• A point p can be written as vector • In homogenous coordinates we add a scaling factor

• To transform the homogenous coordinates in normal coordinate, divide by the n+1 coordinate w.

y

xp

1

y

x

w

wy

wx

ph

20

Homogenous coordinates

• we note

• Proof:

11

:, y

x

by

x

aba

1/

/

/

1

y

x

aa

aay

aax

a

ay

ax

y

x

a

21

Translation

• Classic • Homogenous coordinates

y

x

tyy

txx

12

12

1

1

1

100

10

01

1

2

2

y

x

ty

tx

y

x

22

Scaling

• Classic • Homogenous coordinates

12

12

ryy

sxx

1

1

1

100

00

00

1

2

2

y

x

r

s

y

x

x

y

ppl

5.7

6

55.1

32,

5

3 lpp

23

Rotation (clockwise)

• Classic • Homogenous coordinates

1)cos(1)sin(

1)sin(1)cos(

2

2

yaxay

yaxax

1

1

1

100

0)cos()sin(

0)sin()cos(

1

2

2

y

x

aa

aa

y

x

x

y

p

pl

5

7.8

087.0105.0

05.01087.0,

0

1030lpp

24

Translation and rotation

• Classic • Homogenous coordinates

tyyaxay

txyaxax

1)cos(1)sin(

1)sin(1)cos(

2

2

1

1

1

100

)cos()sin(

)sin()cos(

1

2

2

y

x

tyaa

txaa

y

x

25

Translation, rotation and scaling

• Classic • Homogenous coordinates

tyyarxasy

txyarxasx

1)cos(1)sin(

1)sin(1)cos(

2

2

1

1

1

100

)cos()sin(

)sin()cos(

1

2

2

y

x

tyaras

txaras

y

x

26

3d rotation

• Around x-axis (counter-clockwise)

• Around y-axis

• Around z-axis

• General )()()(,

1000

1000

0100

00)cos()sin(

00)sin()cos(

)(

1000

0)cos(0)sin(

0010

0)sin(0)cos(

)(

1000

0)cos()sin(0

0)sin()cos(0

0001

)(

aRbRcRRtz

ty

tx

RT

cc

cc

cR

bb

bb

bR

aa

aaaR

xyz

z

y

x

27

Transformation Scene - Camera

• (xs,ys,zs) is position of the origin of the camera system with respect to the scene coordinates (translation).

• R is the orientation of the camera system with respect to the scene system (3d rotation).

1000s

s

scsc

sz

y

x

RT

28

Transformation Camera-Retina

• Imagine a 1D camera in a 2D space.

• The transformation MRc can be found by considering

similar triangles

z(xc ,zc )

F

x

xr

O

)()(

)()(

c

cr

c

cr

c

cr

c

cr

zF

Fyy

zF

y

F

y

zF

Fxx

zF

x

F

x

29

Transformation Camera-Retina

1

0

0

/1

0

0

0

1

0

0

0

1

1

,

F

M

z

y

x

M

w

wy

wx

PMP

RC

c

c

c

RCr

rRR

CR

30

Transformation Retina-Image• A frame: the image is composed of pixels (picture

elements)

• Pixels are in general not squared. There physical sizes depends on the used material.

i columns

j rows

(0,0)

(i-1,j-1)

31

Exercise

• Soit une camera avec un axe optique a la position (Ci,Cj) [pixels] et une taille de pixel de Di [mm/ligne], Dj [mm/colonne]. L'horloge du numerisateur est mal regle: chaque ligne d l'image est decalee vers la droite par k pixels par ligne.

• Ecrivez la matrice de projection, Cir de l'image

vers la retine en coordonees homogenes.