Computer Vision Multiple View Geometry & Stereo Marc Pollefeys COMP 256.
Computer Vision - University of California,...
Transcript of Computer Vision - University of California,...
ComputerVision
Cameras, lenses and sensors
Marc PollefeysCOMP 256
ComputerVision
• Camera Models– Pinhole Perspective Projection– Affine Projection
• Camera with Lenses• Sensing• The Human Eye
Reading: Chapter 1.
Cameras, lenses and sensors
ComputerVision
Images are two-dimensional patterns of brightness values.
They are formed by the projection of 3D objects.
Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.
ComputerVision
Animal eye:a looonnng
time ago.
Pinhole perspective projection: Brunelleschi, XVth
Century.Camera obscura: XVIth
Century.
Photographic camera:Niepce, 1816.
ComputerVision Distant objects appear smaller
ComputerVision Parallel lines meet
• vanishing point
ComputerVision Vanishing points
VPL VPRH
VP1VP2
VP3
To different directions correspond different vanishing points
ComputerVision Geometric properties of projection
• Points go to points• Lines go to lines• Planes go to whole image
or half-plane• Polygons go to polygons
• Degenerate cases:– line through focal point yields point– plane through focal point yields line
ComputerVision
Pinhole Perspective Equation
⎪⎪⎩
⎪⎪⎨
⎧
=
=
zyfy
zxfx
''
''
ComputerVision
Affine projection models: Weak perspective projection
0
'where''
zfmmyy
mxx −=⎩⎨⎧
−=−= is the magnification.
When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.
ComputerVision
Affine projection models: Orthographic projection
⎩⎨⎧
==
yyxx
'' When the camera is at a
(roughly constant) distancefrom the scene, take m=1.
ComputerVision
Planar pinhole perspective
Orthographicprojection
Spherical pinholeperspective
ComputerVision Limits for pinhole cameras
ComputerVision Camera obscura + lens
ComputerVision Lenses
Snell’s law
n1
sin
α1 = n2
sin α2
Descartes’
law
ComputerVision
Paraxial (or first-order) optics
Snell’s law:
n1
sin
α1 = n2
sin α2
Small angles:
n1
α1
≈
n2
α2R
nndn
dn 12
2
2
1
1 −=+
R γβα
111
hdh+=+=
222 R
βγαdhh
−=−=
⎟⎟⎠
⎞⎜⎜⎝
⎛−=⎟⎟
⎠
⎞⎜⎜⎝
⎛+
22
11 RR d
hhnhdhn
ComputerVision
Thin Lenses
)1(2 and11
'1
−==−
nRf
fzz
Rn
Zn
Z11
*
−=+
Rn
ZZn −
=+1
'1
*
ZRn
Zn 11
* −−
=
'1111ZZR
nR
n−=
−−
−
spherical lens surfaces; incoming light ±
parallel to axis; thickness << radii; same refractive index on both sides
'11
* ZRn
Zn
−−
=
Rnn
dn
dn 12
2
2
1
1 −=+
ComputerVision
Thin Lenses
)1(2 and11
'1 e wher
''
''
−==−
⎪⎩
⎪⎨
⎧
=
=
nRf
fzzzyzy
zxzx
http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html
ComputerVision Thick Lens
ComputerVision The depth-of-field
ComputerVision The depth-of-field
fZo
1 Z1 1
i
=+−−
−− Δ+= iii ZZZ
fZ
Zf
i
i
Zo −=
−
−−yields
dZZ
bZ iii
−− Δ+=
Δ ii Z
bdbZ−
=Δ −
fbdfZfZZ oo
o / ) ( Z Z Z
0oo −+
−=−=Δ −−
Similar formula for Z Z Z oo o−=Δ ++
)( / Z bddZ ii −=−
fZZfZ
o
oi
−
=
)( Z
0o bdfZb
Zdf o
−+=−
ComputerVision The depth-of-field
fbdfZfZZZZZ−+
−=−=Δ −−
/ )(
0
00000
decreases with d, increases with Z0strike a balance between incoming light and sharp depth range
ComputerVision Deviations from the lens model
3 assumptions :
1. all rays from a point are focused onto 1 image point
2. all image points in a single plane
3. magnification is constant
deviations from this ideal are aberrations
ComputerVision Aberrations
chromatic : refractive index function of wavelength
2 types :
1. geometrical
2. chromatic
geometrical : small for paraxial rays
study through 3rd
order optics
ComputerVision Geometrical aberrations
spherical aberration
astigmatism
distortion
coma
aberrations are reduced by combining lenses
ComputerVision Spherical aberration
rays parallel to the axis do not converge
outer portions of the lens yield smaller focal lenghts
ComputerVision Astigmatism
Different focal length for inclined rays
ComputerVision Distortion
magnification/focal length different for different angles of inclination
Can be corrected! (if parameters are know)
pincushion(tele-photo)
barrel(wide-angle)
ComputerVision Coma
point off the axis depicted as comet shaped blob
ComputerVision Chromatic aberration
rays of different wavelengths focused in different planes
cannot be removed completely
sometimes achromatization is achieved formore than 2 wavelengths
ComputerVision
Lens materialsreference wavelengths :
λF = 486.13nmλd = 587.56nmλC = 656.28nm
lens characteristics :
1. refractive index nd2. Abbe number Vd = (nd - 1) / (nF - nC )
typically, both should be highallows small components with sufficient refraction
notation : e.g. glass BK7(517642)nd = 1.517 and Vd = 64.2
ComputerVision
Lens materials
additional considerations :humidity and temperature resistance, weight, price,...
Crown Glass
Fused Quartz & Fused Silica
Plastic (PMMA)
Calcium Fluoride
Saphire
Zinc Selenide
6000
18000
Germanium 14000
9000
100 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400
WAVELENGTH (nm)
ComputerVision
Vignetting
Figure from http://www.vanwalree.com/optics/vignetting.html
The white openings in the top illustrations denote the entrance pupil, which is the image of the aperture stop seen through all lens elements in front of it and from a position on the optical axis. The bottom illustrations show the lens from the semifield
angle. Here, the white openings correspond to the clear aperture for light that is heading for the image corner.
Darkening of images near the corner
ComputerVision Photographs
(Niepce, “La Table Servie,”
1822)
Milestones: Daguerreotypes (1839)Photographic Film (Eastman,1889)Cinema (Lumière
Brothers,1895)Color Photography (Lumière
Brothers, 1908)Television (Baird, Farnsworth, Zworykin, 1920s)
CCD Devices (1970)more recently CMOS
Collection Harlingue-Viollet.
ComputerVision
Cameras
we consider 2 types :
1. CCD
2. CMOS
ComputerVision CCD
separate photo sensor at regular positionsno scanningcharge-coupled devices (CCDs)area CCDs and linear CCDs2 area architectures :
interline transfer and frame transfer
photosensitive
storage
ComputerVision The CCD camera
ComputerVision CMOS
Same sensor elements as CCDEach photo sensor has its own amplifier
More noise (reduced by subtracting ‘black’ image)Lower sensitivity (lower fill rate)
Uses standard CMOS technologyAllows to put other components on chip‘Smart’ pixels
Foveon4k x 4k sensor0.18μ
process70M transistors
ComputerVision CCD vs. CMOS
• Mature technology• Specific technology• High production cost• High power
consumption• Higher fill rate• Blooming• Sequential readout
• Recent technology• Standard IC technology• Cheap• Low power• Less sensitive• Per pixel amplification• Random pixel access• Smart pixels• On chip integration
with other components
ComputerVision Color cameras
We consider 3 concepts:
1. Prism (with 3 sensors)2. Filter mosaic3. Filter wheel
… and X3
ComputerVision Prism color camera
Separate light in 3 beams using dichroic prismRequires 3 sensors & precise alignmentGood color separation
ComputerVision Prism color camera
ComputerVision Filter mosaic
Coat filter directly on sensor
Demosaicing (obtain full colour & full resolution image)
ComputerVision Filter wheel
Rotate multiple filters in front of lensAllows more than 3 colour bands
Only suitable for static scenes
ComputerVision Prism vs. mosaic vs. wheel
Wheel1
GoodAverageLowMotion3 or more
approach# sensorsSeparationCostFramerateArtefactsBands
Prism3
HighHighHighLow3
High-endcameras
Mosaic1
AverageLowHighAliasing3
Low-endcameras
Scientific applications
ComputerVision new color CMOS sensor
Foveon’s X3
better image qualitysmarter pixels
three layers of pixels. The layers of pixels are embedded in silicon to take advantage of the fact that red, green, and blue light penetrate silicon to different depths
ComputerVision The Human Eye
Helmoltz’sSchematicEye
Reproduced by permission, the American Society of Photogrammetry
andRemote Sensing. A.L. Nowicki, “Stereoscopy.”
Manual of Photogrammetry,Thompson, Radlinski, and Speert
(eds.), third edition, 1966.
ComputerVision The distribution of
rods and cones across the retina
Reprinted from Foundations of Vision, by B. Wandell, SinauerAssociates, Inc., (1995). ©
1995 Sinauer
Associates, Inc.
Cones in the fovea
Rods and cones in the periphery
Reprinted from Foundations of Vision, by B. Wandell, SinauerAssociates, Inc., (1995). ©
1995 Sinauer
Associates, Inc.
ComputerVision Next class
Radiometry: lights and surfaces