Photometric Image Formation
Transcript of Photometric Image Formation
![Page 1: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/1.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Photometric Image Formation
Introduction to Computer Vision ICSE 152ALecture 3
![Page 2: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/2.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Announcements• Assignment 0 is due Jan 13, 11:59 PM• Assignment 1 will be released Jan 13
– Due Jan 27, 11:59 PM• Reading:
– Szeliski• Section 2.2
![Page 3: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/3.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Geometric image formation
![Page 4: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/4.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Photometric image formation
![Page 5: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/5.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Beyond the pinhole CameraGetting more light – Bigger Aperture
![Page 6: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/6.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Pinhole Camera Images with Variable Aperture
1mm
.35 mm
.07 mm
.6 mm
2 mm
.15 mm
![Page 7: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/7.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
The reason for lensesWe need light, but big pinholes cause blur.
![Page 8: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/8.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens
O
• Rotationally symmetric about optical axis• Spherical interfaces
Optical axis
![Page 9: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/9.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens: Center
O
• All rays that enter lens along line pointing at O emerge in same direction
F
![Page 10: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/10.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens: Focus
O
Parallel lines pass through the focus F
F
![Page 11: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/11.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens: Image of Point
O
All rays passing through lens and starting at Pconverge upon P’
So light gather capability of lens is given the area of the lens and all the rays focus on P’ instead of become blurred like a pinhole
F
P
P’
![Page 12: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/12.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens: Image of Point
OF
P
P’ Z’
f
-Z
fzz11
'1
=− Relation between depth of Point (-Z) and the depth where it focuses (Z’)
![Page 13: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/13.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens: Image Plane
OF
P
P’
Image Plane
Q’
Q
A price: Whereas the image of P is in focus,the image of Q is not
![Page 14: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/14.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Thin Lens: Aperture
O
P
P’
Image Plane • Smaller Aperture-> Less Blur
• Pinhole -> No Blur
![Page 15: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/15.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Photometric image formation• Light incident on a given pixel
![Page 16: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/16.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Measuring Angle
• The solid angle subtended by an object from a point P is the area of the projection of the object onto the unit sphere centered at P
• Definition is analogous to projected angle in 2D• Measured in steradians, sr• If I am at P and I look out, the solid angle tells me how much
of my view is filled with an object
![Page 17: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/17.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Radiance• Power traveling at some point
in a specified direction, per unit area perpendicular to the direction of travel, per unit solid angle– Units: watts per square meter per
steradian, W/m2/sr = W m-2 sr-1
Irradiance• Total power arriving at the
surface (from all incoming angles)– Units: power per unit area,
W/m2 = W m-2
x
x
dA
θ
dω
(θ, φ)
radiance in direction different from
surface normal, use spherical coordinates
![Page 18: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/18.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Visible Light Spectrum
![Page 19: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/19.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Camera sensor• Measured pixel intensity is a function of irradiance E
integrated over – Pixel’s area (x,y)– range of wavelengths λ– some period of time t
• Ideally, the camera response function R is linear to the radiance, but it may not be
spatialresponseof pixel
spectralresponseof pixel
![Page 20: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/20.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
LFor a camera with a thin lens, it can be shown that
E(x) = kLLwhere• E(x) is the image irradiance
at point x • L is the radiance coming
from a scene point projecting to image point x
• kL is a proportionality constant that may depend on the lens is may be a function of x
E(x)
Image irradiance is proportional scene radiance
Combined with linear sensor model, we have
I = kckLLIn other words, the measured pixel intensity is proportional to the radiance
![Page 21: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/21.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Color Cameras
Eye: Three types of Cones
Cameras:1. Filter wheel2. Prism (with 3 sensors)3. Filter mosaic
… and X3
![Page 22: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/22.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Filter wheelRotate multiple filters in front of lensAllows more than 3 color bands
Only suitable for static scenes
![Page 23: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/23.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Prism color cameraSeparate light in 3 beams using dichroic prismRequires 3 sensors & precise alignmentGood color separation
![Page 24: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/24.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Filter mosaic Coat filter directly on sensor
Demosaicing (obtain full colour & full resolution image)
![Page 25: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/25.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Color CMOS sensorFoveon’s X3
better image quality smarter pixels
![Page 26: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/26.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Light• Special light sources
– Point sources– Distant point sources– Strip sources– Area sources
• Note, if light is very far away, then view light as coming from a direction in 3D– Directions in 3D can be represented as a point on a sphere– Distant lighting can be viewed as a function giving
brightness over a sphere
![Page 27: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/27.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Light at surfacesMany effects when light strikes a
surface -- could be:• Reflected
– Mirror• Transmitted
– Skin, glass• Scattered
– Milk• Travel along the surface and
leave at some other point• Absorbed
We will assume:• All the light leaving a
point is due to that arriving at that point
• Surfaces don’t fluoresce– e.g. scorpions, detergents
• Surfaces don’t emit light (i.e. are cool)
![Page 28: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/28.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Light at surfaces
![Page 29: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/29.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
BRDF• Bi-directional Reflectance
Distribution Function ρ(θin, φin ; θout, φout)
• Function of– Incoming light direction:
θin , φin– Outgoing light direction:
θout , φout
• Ratio of incident irradiance to emitted radiance
n(θin,φin)
(θout,φout)
![Page 30: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/30.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Specular reflection• Ideal specular reflection is mirror reflection
– Perfectly smooth surface– Incoming light ray is bounced in single
direction– Angle of incidence equals angle of reflection
![Page 31: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/31.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Specular Reflection: Smooth Surface
NN
θi θo
ωoωi
• N, ωi, ωo are coplanar• θi = θo
Speculum – Latin for “Mirror”
![Page 32: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/32.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Diffuse surface• Ideal diffuse material reflects light equally
in all directions• View-independent• Matte, not shiny materials
– Paper– Unfinished wood– Unpolished stone
![Page 33: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/33.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Diffuse reflection• Beam of parallel rays shining on a surface
– Area covered by beam varies with the angle between the beam and the normal– The larger the area, the less incident light per area– Incident light per unit area is proportional to the cosine of the angle between the
normal and the light rays• Object darkens as normal turns away from light• Lambert’s cosine law (Johann Heinrich Lambert, 1760)• Diffuse surfaces are also called Lambertian surfaces
nnn
![Page 34: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/34.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Lambertian (Diffuse) Reflection
The intensity (irradiance) I(u,v) of a pixel at (u,v) is:
• a(u,v) is the albedo of the surface projecting to (u,v)
• n(u,v) is the direction of the surface normal
• s0 is the light source intensity• s is the direction to the light source
ns
a
I(u,v)
^
Do not allow angles less than 0 (light is behind surface)^
![Page 35: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/35.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Glossy surface• Assume surface composed of small mirrors with random
orientation (micro-facets)• Smooth surfaces
– Micro-facet normals close to surface normal– Sharp highlights
• Rough surfaces– Micro-facet normals vary strongly– Blurry highlight
Polished
Smooth
Rough
Very rough
![Page 36: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/36.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Glossy reflection• Expect most light to be reflected in mirror
direction• Because of micro-facets, some light is
reflected slightly off ideal reflection direction
• Reflection– Brightest when view vector is aligned with
reflection– Decreases as angle between view vector and
reflection direction increases
![Page 37: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/37.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Phong reflectance model
Phong Lobe(Lobe illustrates brightness in a
direction)
![Page 38: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/38.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
![Page 39: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/39.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
General BRDF
Example: velvet
Portrait of Sir Thomas Morre, Hans Holbein the Younger, 1527
![Page 40: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/40.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Shadows• Give additional cues on scene lighting
![Page 41: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/41.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Shadows• Contact points• Depth cues
![Page 42: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/42.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Shadows cast by a point source• A point that cannot see the source is in shadow• For point sources, two types of shadows: cast
shadows & attached shadows
Cast Shadow
Attached Shadow
![Page 43: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/43.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
TerminologyUmbra: fully shadowed regionPenumbra: partially shadowed region
(area) light source
receiver shadow
occluder
umbra
penumbra
![Page 44: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/44.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Penumbra and Umbra
![Page 45: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/45.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Hard and soft shadows
• Point and directional lights lead to hard shadows, no penumbra
• Area light sources lead to soft shadows, with penumbra
point directional area
umbra penumbra
![Page 46: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/46.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Hard and soft shadows
Hard shadow from point light source
Soft shadow fromarea light source
![Page 47: Photometric Image Formation](https://reader035.fdocuments.in/reader035/viewer/2022062405/62aba78d94f1640dd9796a14/html5/thumbnails/47.jpg)
CSE 152A, Winter 2021 Introduction to Computer Vision I
Next Lecture• Photometric Stereo• Reading:
– Szeliski• Section 13.1.1