ComputerVision

Cameras, lenses and sensors

Marc PollefeysCOMP 256

ComputerVision

• Camera Models– Pinhole Perspective Projection– Affine Projection

• Camera with Lenses• Sensing• The Human Eye

Reading: Chapter 1.

Cameras, lenses and sensors

ComputerVision

Images are two-dimensional patterns of brightness values.

They are formed by the projection of 3D objects.

Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.

ComputerVision

Animal eye:

a looonnng time ago.

Pinhole perspective projection: Brunelleschi, XVth Century.Camera obscura: XVIth Century.

Photographic camera:Niepce, 1816.

ComputerVision Distant objects appear smaller

ComputerVision Parallel lines meet

• vanishing point

ComputerVision Vanishing points

VPL VPRH

VP1VP2

VP3

To different directions correspond different vanishing points

ComputerVision Geometric properties of projection

• Points go to points• Lines go to lines• Planes go to whole image

or half-plane• Polygons go to polygons

• Degenerate cases:– line through focal point yields point– plane through focal point yields line

ComputerVision

Pinhole Perspective Equation

z

yfy

z

xfx

''

''

ComputerVision

Affine projection models: Weak perspective projection

0

'where'

'z

fmmyy

mxx

is the magnification.

When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.

ComputerVision

Affine projection models: Orthographic projection

yy

xx

'

' When the camera is at a(roughly constant) distancefrom the scene, take m=1.

ComputerVision

Planar pinhole perspective

Orthographicprojection

Spherical pinholeperspective

ComputerVision Limits for pinhole cameras

ComputerVision Camera obscura + lens

ComputerVision

Lenses

Snell’s law

n1 sin 1 = n2 sin 2

Descartes’ law

ComputerVision

Paraxial (or first-order) optics

Snell’s law:

n1 sin 1 = n2 sin 2

Small angles:

n1 1 n22R

nn

d

n

d

n 12

2

2

1

1

R γβα

111

h

d

h

222 R

βγαd

hh

22

11 RR d

hhn

h

d

hn

ComputerVision

Thin Lenses

)1(2 and

11

'

1

n

Rf

fzz

R

n

Z

n

Z

11*

R

n

ZZ

n

1

'

1*

ZR

n

Z

n 11*

'

1111

ZZR

n

R

n

spherical lens surfaces; incoming light parallel to axis; thickness << radii; same refractive index on both sides

'

11* ZR

n

Z

n

R

nn

d

n

d

n 12

2

2

1

1

ComputerVision

Thin Lenses

)1(2 and

11

'

1 e wher

''

''

n

Rf

fzzz

yzy

z

xzx

http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html

ComputerVision Thick Lens

ComputerVision The depth-of-field

ComputerVision The depth-of-field

fZo

1

Z

1

1

i

iii ZZZ

fZ

Zf

i

i

Zo

yields

d

ZZ

b

Z iii

ii Zbd

bZ

fbdfZ

fZZ ooo /

) ( Z Z Z

0oo

Similar formula for Z Z Z oo o

)( / Z bddZ ii

fZ

ZfZ

o

oi

)(

Z

0o bdfZb

Zdf o

ComputerVision The depth-of-field

fbdfZ

fZZZZZ

/

)(

0

00000

decreases with d, increases with Z0

strike a balance between incoming light and sharp depth range

ComputerVision Deviations from the lens model

3 assumptions :

1. all rays from a point are focused onto 1 image point

2. all image points in a single plane

3. magnification is constant

deviations from this ideal are aberrations

ComputerVision Aberrations

chromatic : refractive index function of wavelength

2 types :

1. geometrical

2. chromatic

geometrical : small for paraxial rays

study through 3rd order optics

ComputerVision Geometrical aberrations

spherical aberration

astigmatism

distortion

coma

aberrations are reduced by combining lenses

ComputerVision Spherical aberration

rays parallel to the axis do not converge

outer portions of the lens yield smaller focal lenghts

ComputerVision Astigmatism

Different focal length for inclined rays

ComputerVision Distortion

magnification/focal length different for different angles of inclination

Can be corrected! (if parameters are know)

pincushion(tele-photo)

barrel(wide-angle)

ComputerVision Coma

point off the axis depicted as comet shaped blob

ComputerVision Chromatic aberration

rays of different wavelengths focused in different planes

cannot be removed completely

sometimes achromatization is achieved formore than 2 wavelengths

ComputerVision

Lens materialsreference wavelengths :

F = 486.13nm

d = 587.56nm

C = 656.28nm

lens characteristics :

1. refractive index nd

2. Abbe number Vd= (nd - 1) / (nF - nC)

typically, both should be highallows small components with sufficient refraction

notation : e.g. glass BK7(517642)nd = 1.517 and Vd= 64.2

ComputerVision

Lens materials

additional considerations :humidity and temperature resistance, weight, price,...

Crown Glass

Fused Quartz & Fused Silica

Plastic (PMMA)

Calcium Fluoride

Saphire

Zinc Selenide

6000

18000

Germanium 14000

9000

100 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400

WAVELENGTH (nm)

ComputerVision Vignetting

Figure from http://www.vanwalree.com/optics/vignetting.html

ComputerVision Photographs

(Niepce, “La Table Servie,” 1822)

Milestones: Daguerreotypes (1839)Photographic Film (Eastman,1889)Cinema (Lumière Brothers,1895)Color Photography (Lumière Brothers, 1908)Television (Baird, Farnsworth, Zworykin, 1920s)

CCD Devices (1970)more recently CMOS

Collection Harlingue-Viollet.

ComputerVision

Cameras

we consider 2 types :

1. CCD

2. CMOS

ComputerVision CCD

separate photo sensor at regular positionsno scanning

charge-coupled devices (CCDs)

area CCDs and linear CCDs2 area architectures : interline transfer and frame transfer

photosensitive

storage

ComputerVision The CCD camera

ComputerVision CMOS

Same sensor elements as CCDEach photo sensor has its own amplifier

More noise (reduced by subtracting ‘black’ image)Lower sensitivity (lower fill rate)

Uses standard CMOS technologyAllows to put other components on chip‘Smart’ pixels

Foveon4k x 4k sensor0.18 process70M transistors

ComputerVision CCD vs. CMOS

• Mature technology• Specific technology• High production cost• High power

consumption• Higher fill rate• Blooming• Sequential readout

• Recent technology• Standard IC technology• Cheap• Low power• Less sensitive• Per pixel amplification• Random pixel access• Smart pixels• On chip integration

with other components

ComputerVision Color cameras

We consider 3 concepts:

1. Prism (with 3 sensors)2. Filter mosaic3. Filter wheel

… and X3

ComputerVision Prism color camera

Separate light in 3 beams using dichroic prismRequires 3 sensors & precise alignmentGood color separation

ComputerVision Prism color camera

ComputerVision Filter mosaic

Coat filter directly on sensor

Demosaicing (obtain full colour & full resolution image)

ComputerVision Filter wheel

Rotate multiple filters in front of lensAllows more than 3 colour bands

Only suitable for static scenes

ComputerVision Prism vs. mosaic vs. wheel

Wheel 1GoodAverageLowMotion3 or more

approach# sensorsSeparationCostFramerateArtefactsBands

Prism 3HighHighHighLow 3

High-endcameras

Mosaic 1AverageLowHighAliasing 3

Low-endcameras

Scientific applications

ComputerVision

new color CMOS sensorFoveon’s X3

better image qualitysmarter pixels

ComputerVision The Human Eye

Helmoltz’s SchematicEye

Reproduced by permission, the American Society of Photogrammetry andRemote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,Thompson, Radlinski, and Speert (eds.), third edition, 1966.

ComputerVision The distribution of

rods and cones across the retina

Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (1995). 1995 Sinauer Associates, Inc.

Cones in the fovea

Rods and cones in the periphery

Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (1995). 1995 Sinauer Associates, Inc.

ComputerVision

Next class Radiometry: lights and surfaces