Colorization of 3D Geometric Model utilizing Laser Reflectivity
Geometric Model Acquisition
description
Transcript of Geometric Model Acquisition
29 April 2008 Birkbeck College, U. London 1
Geometric Model Acquisition
Steve MaybankSchool of Computer Science and Information
SystemsBirkbeck College
London, WC1E 7HX
Edited version of the slides for the VVG Summer School, held at the University of Bath
21 September 2007
29 April 2008 Birkbeck College, U. London 2
Geometric Model Acquisition
Aim: make a 3D model of a scene from two or more images taken from different viewpoints.
Why is it possible: the image differences depend in part on the shapes of the objects in the scene.
29 April 2008 Birkbeck College, U. London 3
Two Images of the Same Scene
http://vasc.ri.cmu.edu/idb/images/stereo/fruitSOURCE "University of Illinois, Bill Hoff“DESCRIPTION "Fruit on table, digitized from 35mm."
29 April 2008 Birkbeck College, U. London 4
Two Images of a Point in R3
• p
c1 •
c2•
q1 •
q2 •
Epipolar plane: <c1,c2,x>
Image 1
Image 2
objectpoint
opticalcentre
opticalcentre
29 April 2008 Birkbeck College, U. London 5
Corresponding Points Points in different images
correspond,qq ~,
qq ~ if they are projections of the same
scene point p. In projective coordinates, projection is a matrix application,
qpM
qMp~~
29 April 2008 Birkbeck College, U. London 6
Method for Finding Corresponding Points
oods.neighbourh levelgrey similar have ,~ i.e. large, is
~
~.~.~.
~.~,
ncorrelatio theif ~ thatassumed isIt .~, of oodsneighbourh levelgrey be ~,Let
2/12/1
uu
uu
uuuu
uuuu
qquu
29 April 2008 Birkbeck College, U. London 7
Example 1 of Correlation Based Matching
Points in lh image =(150,100), (250,150), (350,250), (450,350), (250,450)Correlations (ρ) = 0.750, 0.685, 0.912, 0.644, 0.691Search area = (2d+1)x(2d+1) box, d=20.
29 April 2008 Birkbeck College, U. London 8
What Do We Need for GAM?
Description of image formation in the camera.
Description of the relative positions of the cameras.
Equations involving the measurements, the scene points and the relative positions of the cameras.
Statistical description of the errors in the measurements.
29 April 2008 Birkbeck College, U. London 9
Pinhole Camera
Light tight box
Small hole (optical centre)
Viewingscreen(image)
Object
Light rays
Central perspective projection model for imageformation (Brunelleschi, 15th C.).
29 April 2008 Birkbeck College, U. London 10
Camera Coordinate Frame
(X,Y,Z)(0,0,0)(0,0,-f)
Origin (0,0,0) at the pin hole.Focal length of the camera = f.Axes of image coordinate frame are parallel to X, Y axes
of the CCF.Image point = (-Xf/Z, -Yf/Z)
•
•
• Z
YXx
y
29 April 2008 Birkbeck College, U. London 11
Mathematical Version of the Camera Coordinate Frame
(X,Y,Z)
(0,0,0)
(0,0,f)
Origin (0,0,0) at the pin hole.Focal length of the camera = f.The image is in front of the pin hole!Image point = (Xf/Z, Yf/Z). The minus signs have gone.
•
•
• Z
yxX
Y Image plane
•
29 April 2008 Birkbeck College, U. London 12
Relative Position of the Cameras
Y X~
ZX Z~
Y~
The relative position of the cameras is describedby an orthogonal matrix R and a translation vector t.
R, t
29 April 2008 Birkbeck College, U. London 13
Transformation of Coordinates
YX~
ZX
Z~
Y~
If a point p has coordinates (X,Y,Z)T in the first CCF,then in the second CCF the same point p has coordinates
R, t
● p
t
Z
Y
X
R
29 April 2008 Birkbeck College, U. London 14
Properties of Orthogonal Matrices
angle.rotation the
for one and axis for the two:freedom of degrees threehave rotations The
100
0cossin
0sincos
:matrixrotation a of Example
.1det ifonly and ifrotation a represents matrix orthogonal The
.1det that followsIt
.
100
010
001
ifonly and if orthogonal is matrix 3x3 The
R
RR
R
IRRR T
29 April 2008 Birkbeck College, U. London 15
Projection Ray
Tfyx ,,point Image
Y
X
Z
●
0,,, :ray Projection Tfyx
Any scene point projecting to (x, y, f)T is on the projection ray.
CCF
29 April 2008 Birkbeck College, U. London 16
Projection Rays of Corresponding Points 1
●
The projection rays of corresponding points intersect ata scene point. Geometric model acquisition is based onthis single constraint.
For an extreme example, see http://www.wisdom.weizmann.ac.il/~vision/VideoAnalysis/Demos/Traj2Traj/hall.htm
29 April 2008 Birkbeck College, U. London 17
Projection Rays of Corresponding Points 2
Tfyx ,, Tfyx ~
,~,~~ ●
The equations of the projectionrays are known, but they hold indifferent coordinate systems.
f
y
x
f
y
x
~~
~
29 April 2008 Birkbeck College, U. London 18
Transformation of Coordinates
by CCF second in thegiven is ,,ray The Tfyx
t
f
y
x
R
such that ~, numbers realexist therethus
.~~
~
~
f
y
x
t
f
y
x
R
,~
,~,~ith equation w theofproduct scalar theby taking eliminated are ~, numbersunknown TheT
fyxt
.0.~~
~
f
y
x
R
f
y
x
t
29 April 2008 Birkbeck College, U. London 19
The Essential Matrix
.0~ thatfollowsIt .by matrix essential theDefine
.~~0
thus
0.~ then~ If
.~~such that matrix 33 theDefine
Eqq
RTEE
RqTqRqTq
RqqT
tqqT
T
T
t
tTT
tT
t
t
t
29 April 2008 Birkbeck College, U. London 20
Model Acquisition
matrix essentialan find ,1,~ points matchingGiven Niiqiq
.1 ,0~such that NiiEqiqE T
such that points find and from ,Recover ixEtR
,1 ,~~NiiqitiRqiix
camera. second theof CCF in the
29 April 2008 Birkbeck College, U. London 21
Naïve Estimates of E
9. parameters ofNumber .1 subject to
,~
minimised, is expression following thefor which an Find
1
2
E
iEqiq
EN
i
T
6. parameters ofNumber .1 and subject to
,~
minimised, is expression following thefor which , Find
1
2
tIRR
iRqTiq
tR
T
N
it
T
29 April 2008 Birkbeck College, U. London 22
Better Way of Estimating E
minimiseThen small. are
~ ,
and tsmeasuremen ueunknown tr theare ~ , where
,1 ,~~~
,1 ,
Write
ii
iaia
Niiiaiq
Niiiaiq
.1 , and ,1 ,0~ subject to
,~
1
22
tIRRNiiRaTia
ii
Tt
T
N
i
29 April 2008 Birkbeck College, U. London 23
Geometric Picture
i i~ iq iq~● ●
First image Second image lEll ,
~~
l l~
29 April 2008 Birkbeck College, U. London 24
Camera Calibration
Ideal pixelcoordinates
Measured pixelcoordinates
Ideal CCF
Camera calibration is a transformation from measured pixelcoordinates to ideal pixel coordinates.
29 April 2008 Birkbeck College, U. London 25
Calibration Matrix
plane. image theofation transformaffinean defines matrix The
1,, scoordinate pixel ideal
1,, scoordinate pixel measured
100
0
21
21
K
Krq
qqq
rrr
v
us
K
T
v
u
29 April 2008 Birkbeck College, U. London 26
Fundamental Matrix
.0~therefore
~~~ ,
0~
~ :points Matching
EKrKr
rKqKrq
Eqq
TT
T
The fundamental matrix F is defined by
EKKF T~
29 April 2008 Birkbeck College, U. London 27
Properties of E and F det(E)=det(Tt)det(R)=0 The matrix E is essential iff
SingularValues(E) = (σ,σ,0)
det(F)=det(K~)det(E)det(K)=0 The matrix F is fundamental iff
det(F)=0.
29 April 2008 Birkbeck College, U. London 28
Minimal Data
.)identified are 0 with , (Solutions
solutions. 10 have ,51 ,0~ equations thegeneral,In
.532 is of freedom of degrees ofNumber
EE
iiEqiq
ET
solutions. 3 have ,70~ equations thegeneral,In
7.2-9 is of freedom of degrees ofNumber
iiFqiq
FT