Geometric Model of Camera - people.scs.carleton.ca
Transcript of Geometric Model of Camera - people.scs.carleton.ca
![Page 1: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/1.jpg)
Geometric Model of Camera
Dr. Gerhard Roth
COMP 4900D
Winter 2012
![Page 2: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/2.jpg)
Similar Triangles
![Page 3: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/3.jpg)
Geometric Model of Camera
Perspective projection
Z
Xfx
Z
Yfy
P(X,Y,Z) p(x,y) P
p
principal
point
principal
axis
image
plane
optical
center
x y
![Page 4: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/4.jpg)
Parallel lines aren’t…
Figure by David Forsyth
![Page 5: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/5.jpg)
Lengths can’t be trusted...
![Page 6: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/6.jpg)
Coordinate Transformation – 2D
y
x
y
x
p
p
p
pp
cossin
sincos
'
''
P
'x
x
''xy'y
''y
T
y
x
y
x
y
x
y
x
T
T
p
p
T
T
p
pp
cossin
sincos
'
'''
Rotation and Translation
![Page 7: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/7.jpg)
Homogeneous Coordinates
w
wy
wx
y
x
Go one dimensional higher:
w
wz
wy
wx
z
y
x
is an arbitrary non-zero scalar, usually we choose 1. w
32
31
3
2
1
/
/
xx
xx
x
x
x
43
42
41
4
3
2
1
/
/
/
xx
xx
xx
x
x
x
x
From homogeneous coordinates to Cartesian coordinates:
![Page 8: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/8.jpg)
2D Transformation with Homogeneous Coordinates
2D coordinate transformation using homogeneous coordinates:
y
x
y
x
T
T
p
pp
cossin
sincos''
1100
cossin
sincos
1
''
''
y
x
y
x
y
x
p
p
T
T
p
p
2D coordinate transformation:
![Page 9: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/9.jpg)
3D Rotation Matrix
Rotate around each coordinate axis:
cossin0
sincos0
001
)(1R
cos0sin
010
sin0cos
)(2R
100
0cossin
0sincos
)(3
R
Combine the three rotations:
3D rotation matrix has three parameters,
no matter how it is specified.
321 RRRR
![Page 10: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/10.jpg)
Rotation Matrices
• Both 2d and 3d rotation matrices have two
characteristics
• They are orthogonal (also called orthonormal)
• Their determinant is 1
• Matrix below is orthogonal but not a rotation
matrix because the determinate is not 1
IRRT 1 RRT
100
010
001
this is a reflection matrix
![Page 11: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/11.jpg)
Four Coordinate Frames
wim Pmatrix
tiontransformap
Camera model:
Zc
Xc
Yc
x
y
Xw
Yw Zw
Pw
xim
yim
camera
frame image
plane
frame
world
frame
pixel
frame
optical
center principal
point
imp
![Page 12: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/12.jpg)
Perspective Projection
Z
Xfx
Z
Yfy
These are nonlinear.
10100
000
000
Z
Y
X
f
f
w
v
u
Using homogenous coordinate, we have a linear relation:
P
p
principal
point
principal
axis
image
plane
optical
center
x y
wux / wvy /
![Page 13: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/13.jpg)
World to Camera Coordinate
TRXX wc
1
1
1
w
w
w
c
c
c
Z
Y
X
Z
Y
X
0
TR
Transformation between the camera and world coordinates.
Here we rotate, then translate, to go from world to camera
co-ordinates which is opposite of book, but is simpler and is
the way in which OpenCV routines do it:
R,T
![Page 14: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/14.jpg)
Camera Coordinates to Pixel Coordinates
ximx sxox )(
1100
/10
0/1
1
y
x
os
os
y
x
yy
xx
im
im
pixel sizes in millimeters per pixel :, yx ss
x
y
(ox,oy)
xim
yim
yimy syoy )(
Camera co-ordinates x, and y are in millimetres
Image co-ordinates xim, yim, are in pixels
Center of projection ox, oy is in pixels
![Page 15: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/15.jpg)
Put All Together – World to Pixel
1
100100
0010
0001
100
/0
0/
1
100100
000
000
100
/10
0/1
10100
000
000
100
/10
0/1
100
/10
0/1
3
2
1
w
w
w
yy
xx
w
w
w
yy
xx
c
c
c
yy
xx
yy
xx
Z
Y
X
TRosf
osf
Z
Y
X
TRf
f
os
os
Z
Y
X
f
f
os
os
w
v
u
os
os
x
x
x
1
w
w
w
Z
Y
X
TRK
31 / xxxim 32 / xxyim
![Page 16: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/16.jpg)
Camera Parameters
• Extrinsic parameters define the location and
orientation of the camera reference frame
with respect to a world reference frame • Depend on the external world, so they are extrinsic
• Intrinsic parameters link pixel co-ordinates in
the image with the corresponding co-
ordinates in the camera reference frame • An intrinsic characteristic of the camera
• Image co-ordinates are in pixels
• Camera co-ordinates are in millimetres
![Page 17: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/17.jpg)
Intrinsic Camera Parameters
100
/0
0/
yy
xx
osf
osf
K
K is a 3x3 upper triangular matrix, called the
Camera Calibration Matrix.
There are five intrinsic parameters:
(a) The pixel sizes in x and y directions in millimeters/pixel
(b) The focal length in millimeters
(c) The principal point (ox,oy) in pixels, which is the point
where the optic axis intersects the image plane.
yx ss ,
f
![Page 18: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/18.jpg)
Camera intrinsic parameters
• Can write three of these parameters
differently by letting f/sx = fx and f/sy = fy • Then intrinsic parameters are ox,oy,fx,fy
• The units of these parameters are pixels!
• In practice pixels are square (sx = sy) so that
means fx should equal fy for most cameras • However, every explicit camera calibration process (using
calibration objects) introduces some small errors
• These calibration errors make fx not exactly equal to fy
• So in OpenCV the intrinsic camera
parameters are the four following ox,oy,fx,fy • However fx is usually very close to fy and if this is not the
case then there is a problem
![Page 19: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/19.jpg)
Extrinsic Parameters and Proj. Matrix
[R|T] defines the extrinsic parameters.
The 3x4 matrix M = K[R|T] is called the projection matrix.
It takes 3d points in the world co-ordinate system and maps
them to the appropriate image co-ordinates in pixels
113
2
1
w
w
w
w
w
w
imZ
Y
X
MZ
Y
X
TRK
x
x
x
p
![Page 20: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/20.jpg)
• Where would the point (20,50,200) project to in the image?
= P
11 P
12 P
13 P
14
P21
P22
P23
P24
P31
P32
P33
P34
1000 0 320 -19600
0 1000 240 -27200
0 0 1 -30
Projection matrix
u’
v’
w’
= P
11 P
12 P
13 P
14
P21
P22
P23
P24
P31
P32
P33
P34
Xw
Yw
Zw
1
= 1000 0 320 -19600
0 1000 240 -27200
0 0 1 -30
20
50
200
1
u’
v’
w’
= 64400
70800
170
u=u’/w’ = 64400/170 = 378.8 v=v’/w’ = 70800/170 = 416.5 • World point (20,50,200) project to (379,414)
Using the projection matrix - example
![Page 21: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/21.jpg)
Effect of change in focal length
Small f is wide angle, large f is telescopic
![Page 22: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/22.jpg)
Orthographic Projection
Orthographic Projection
Z y
C c
Y
X
I x
X
Y
m
X
Y
Z
M
![Page 23: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/23.jpg)
Orthographic Projection
Orthographic Projection
Z
C
Y
X
I
1O 2O
3O
![Page 24: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/24.jpg)
Weak Perspective Model
Assume the relative distance between any two points along the
principal axis is much smaller (1/20th at most) than the
average distance
Then the camera projection can be approximated as:
XZ
f
Z
Xfx
YZ
f
Z
Yfy
This is the weak-perspective camera model.
Sometimes called scaled orthography.
Z
![Page 25: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/25.jpg)
2/2/2012 CS-236607 Visual Recognition 25
Two step projection:
image plane average depth
plane
Weak Perspective Projection
C
Y
X
I
Z Zc
![Page 26: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/26.jpg)
Weak Perspective
![Page 27: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/27.jpg)
Impact of different projections
• Perspective projections • Parallel lines in world are not parallel in the image
• Object projection gets smaller with distance from camera
• Weak perspective projection • Parallel lines in the world are parallel in the image
• Object projection gets smaller with distance from camera
• Orthographic projection • Parallel lines in the world are parallel in the image
• Object projection is unchanged with distance from camera
![Page 28: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/28.jpg)
Image distortion due to optics
• Radial distortion which depends on radius r,
distance of each point from center of image
• r2 = (x- ox)2 +(y- oy)2
Correction uses three
parameters, k1,k2,k3
![Page 29: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/29.jpg)
Correcting Radial Distortions
![Page 30: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/30.jpg)
Tangential Distortion
• Lens not exactly parallel to the image plane
• Correction uses two parameter p1, p2
• Both types of distortion are removed (image
is un-distorted) and only then does standard
calibration matrix K apply to the image
• Camera calibration computes both K and
these five distortion parameters
![Page 31: Geometric Model of Camera - people.scs.carleton.ca](https://reader031.fdocuments.in/reader031/viewer/2022020912/620307f9853358130c47efe7/html5/thumbnails/31.jpg)
How to find the camera parameters
• Can use the EXIF tag for any digital image • Has focal length f in millimeters but not the pixel size
• But you can get the pixel size from the camera manual
• There are only a finite number of different pixels sizes
because number of sensing element sizes is limited
• If there is not a lot of image distortion due to optics then this
approach is sufficient (only linear calibration)
• Can perform explicit camera calibration • Put a calibration pattern in front of the camera
• Take a number of different pictures of this pattern
• Now run the calibration algorithm (different types)
• Result is intrinsic camera parameters (linear and non-linear)
and the extrinsic camera parameters of all the images