CS4501: Introduction to Computer Vision Projective ...
Transcript of CS4501: Introduction to Computer Vision Projective ...
CS4501: Introduction to Computer VisionProjective Geometry and Light
Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA).
Announcing Office Hours
Masum(mb2vj at virginia.edu)
Office Hours: Tuesdays 3 to 4pm (ET)
Anshuman(as9rw at virginia.edu)
Office Hours: Wednesdays 2 to 3pm (ET)
Fridays 1 to 2pm (ET)
Vicente(vicente at virginia.edu)
Office Hours: Mondays 3 to 5pm (ET)
Last Class
What is a camera? Who invented cameras?Photography – Life of a Photograph - from Light to PixelsShutter Speed / Aperture Size / ISO sensitivity
• Recap from Last Class on Shutter Speed / Aperture / ISO sensitivity• Camera Parameters• Brief Introduction to Projective Geometry (Computer Graphics)• Light Models (Diffuse Light vs Specular Light)
Today’s Class
Life of a Photograph
Slide by Steve Seitz
How to Shoot Photos in Manual?
•Shutter speed•Aperture size•Focus / Auto-focus (Yes, you can shoot in
manual and also probably should focus in manual)
•ISO sensitivity
Fast Shutter Speed
http://www.photographymad.com/pages/view/shutter-speed-a-beginners-guide
Doesn’t allow much light Allows a lot of light
Slow Shutter Speed
Large Aperture SizeSmall Aperture Size
ISO sensitivity – Should be small ideally
https://www.exposureguide.com/iso-sensitivity/
Projection: world coordinatesàimage coordinates
Camera Center (0, 0, 0)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z
Y
úû
ùêë
é=VU
p
.V
U
If X = 2, Y = 3, Z = 5, and f = 2What are U and V?
Projection: world coordinatesàimage coordinates
Camera Center (0, 0, 0)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z
Y
úû
ùêë
é=VU
p
.V
U
ZfXU *-=
ZfYV *-=
52*2-=U
52*3-=V
Projection: world coordinatesàimage coordinates
Camera Center
(tx, ty, tz)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z Y
úû
ùêë
é=vu
p
.Optical Center (u0, v0)
v
u
Homogeneous coordinates vs Cartesian coordinatesConversion
Converting to homogeneous coordinates
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
Invariant to scaling
Point in Cartesian is ray in Homogeneous
úû
ùêë
é=ú
û
ùêë
éÞ
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
wywx
kwkykwkx
kwkykx
wyx
k
Homogeneous Coordinates
Cartesian Coordinates
Homogeneous coordinates vs Cartesian coordinates
Projection: world coordinatesàimage coordinates
Camera Center (0, 0, 0)
úúú
û
ù
êêê
ë
é=ZYX
P.
.. f Z
Y
úû
ùêë
é=VU
p
.V
U
ZfXU *-=
ZfYV *-=
52*2-=U
52*3-=V
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
10100000000
1zyx
ff
vu
w
K
Slide Credit: Savarese
Projection matrix: from World to Image Coordinates
Intrinsic Assumptions• Unit aspect ratio• Optical center at (0,0)• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
X
x
Remove assumption: known optical center
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
101000000
10
0
zyx
vfuf
vu
w
Intrinsic Assumptions• Unit aspect ratio• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Remove assumption: square pixels
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
101000000
10
0
zyx
vu
vu
w ba
Intrinsic Assumptions• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Remove assumption: non-skewed pixels
[ ]X0IKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
10100000
10
0
zyx
vus
vu
w ba
Intrinsic Assumptions Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Note: different books use different notation for parameters
Oriented and Translated Camera
Ow
iw
kw
jw
t
R
X
x
Allow camera translation
[ ]XtIKx =úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
1100010001
1000
0
10
0
zyx
ttt
vu
vu
w
z
y
x
ba
Intrinsic Assumptions Extrinsic Assumptions• No rotation
3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:
úúú
û
ù
êêê
ë
é -=
úúú
û
ù
êêê
ë
é
-=
úúú
û
ù
êêê
ë
é-=
1000cossin0sincos
)(
cos0sin010
sin0cos)(
cossin0sincos0001
)(
gggg
g
bb
bbb
aaaaa
z
y
x
R
R
R
p
p’ g
y
z
Slide Credit: Savarese
Allow camera rotation
[ ]XtRKx =
úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
1100
01 333231
232221
131211
0
0
zyx
trrrtrrrtrrr
vus
vu
w
z
y
x
ba
Slide Credit: Savarese
Projection matrix (Word Coordinates to Image Coordinates)
[ ]XtRKx = x: Image Coordinates: (u,v,1)K: Intrinsic Matrix (3x3)R: Rotation (3x3) t: Translation (3x1)X: World Coordinates: (X,Y,Z,1)
Ow
iw
kw
jwR,t
X
x
Intrinsic Camera Properties: K
Extrinsic Camera Properties: [R t]
Degrees of freedom
[ ]XtRKx =
úúúú
û
ù
êêêê
ë
é
úúú
û
ù
êêê
ë
é
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é
1100
01 333231
232221
131211
0
0
zyx
trrrtrrrtrrr
vus
vu
w
z
y
x
ba
5 6
Assignment – World Coordinates to Image Coordinates
What the camera can “see”assuming this is a wireframe
object
(0,0,0)
(0,0,3)
(1,1,4)
Things to Remember for Quiz
• Pinhole camera model• Focal length in the pinhole camera model• Shutter Time / Aperture / ISO• Homogeneous Coordinates• Extrinsic Camera Properties and Intrinsic Camera Properties• Describe mathematically (and intuitively) the conversion process
from World Coordinates to Image Coordinates
Light
• What determines the color of a pixel?
Figure from Szeliski
BRDF (Bidirectional reflectance distribution function)
Slide by Aaron Bobick
BRDF (Bidirectional reflectance distribution function)
Slide by Aaron Bobick
Reflection
Slide by Aaron Bobick
Phong Reflection Model
Slide by Aaron Bobick
Phong Reflection Model
https://en.wikipedia.org/wiki/Phong_reflection_model
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
Phong Reflection Model - Recap
https://en.wikipedia.org/wiki/Phong_reflection_model
Phong’s Shading / Illumination Model
• Originally from Vietnam / PhD from Utah, Professor at Utah, and later Stanford.
• Died at age 32 from leukemia
• Phong’s professor Ivan Sutherland went on to win the Turing Award (Nobel Prize in CS) for lifelong contributions to Computer Graphics
Same ideas used in Computer Graphics
• Ray Tracing• Radiosity• Photon Mapping
Reflection
Slide by Aaron Bobick
Diffuse Reflection – Lambertian Surface / BRDF
Slide by Aaron Bobick
• Light intensity does not depend on the outgoing direction. Only incoming.
• It is independent of where the viewer stands.
• Smooth surface, not glossy. Can think of any examples?
Slide by Aaron Bobick
The other extreme – Only Specular Reflection
Slide by Aaron Bobick
Problem in Computer Vision: Intrinsic Image Decomposition
Given this Extract this
Images by Marc Serra
Given this Extract this
Images by Aaron Bobick
Problem in Computer Vision: Shape from Shading
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
?
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
A photon’s life choices
• Absorption• Diffuse Reflection• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Specular Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ1
light source
λ2
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
t=1
light source
t=n
Slide by James Hays
A photon’s life choices
• Absorption• Diffusion• Reflection• Transparency• Refraction• Fluorescence• Subsurface scattering• Phosphorescence• Interreflection
λ
light source
(Specular Interreflection)
Slide by James Hays
The Eye
• The human eye is a camera! (not quite)• Iris - colored annulus with radial muscles• Pupil - the hole (aperture) whose size is controlled by the iris• What’s the “film”?
– photoreceptor cells (rods and cones) in the retina
Slide by Steve Seitz
Next Class: Image Processing and Image Filters
Questions?
57