Self-Calibration and Metric Reconstruction from Single Images
-
Upload
laith-grant -
Category
Documents
-
view
28 -
download
0
description
Transcript of Self-Calibration and Metric Reconstruction from Single Images
Self-Calibration and Metric Reconstruction from Single
Images
Self-Calibration and Metric Reconstruction from Single
Images
Ruisheng Wang
Frank P. Ferrie
Centre for Intelligent Machines, McGill University
OutlineOutline
• Contributions
• Existing Methods
• The Idea
• Our Method
• Comparison
• Conclusion
ContributionsContributions
• We developed a direct solution to 3D reconstruction from single images which is model based
– No model-to-image projection and readjustment procedure
• We made it possible to perform accurate 3D measurement using an uncalibrated* camera based on single images only
• We quantitatively evaluated our model-based approach with vanishing point based method, and the results indicate our approach is better than vanishing point based method
* Intrinsic parameters known/estimated
Three vanishing points
Vanishing Points (VPs) Based MethodVanishing Points (VPs) Based Method
• Determine three orthogonal vanishing points
– Manual detection
– Search over Gaussian sphere
– Hough transform
– Projective geometry
• Determine camera focal length and rotation
• Determine camera translation and model dimensions
)/tan( 22 YY yxfa
)/tan( 222222 XXZZ yxfyxfa
)/tan( YY yxak
)( YXYX yyxxf
Problems in VPs Based MethodProblems in VPs Based Method
• Three orthogonal VPs may not be always available
– One or two vanishing point only
• Hard to accurately determine VPs
– Need many lines
• The accuracy of VPs affects the accuracy of the 3D reconstruction
1 Point Perspective 2 Points Perspective
Methods with Ground Control Points/LinesMethods with Ground Control Points/Lines• Point-based methods
– Collinearity Equations
• Line-based methods
– Model-to-Image Fitting
)()()(
)()()(*
)()()(
)()()(*
033032031
023022021
033032031
013012011
ZZmYYmXXm
ZZmYYmXXmfy
ZZmYYmXXm
ZZmYYmXXmfx
2 2 21 1 2 20
( ) ( ) ( )3
l T Ti
lErr h s ds h h h h m A BA m
From Debevec et al.1996
The IdeaThe Idea
• Use model to estimate camera exterior orientation
– Need 6 parameters X3, Y3, L, W, H,
• If an object-centered coordinate system selected– Need three parameters L, W, H
• Divide camera parameters into two groups: rotation and translation
• It’s possible to estimate relative camera exterior orientation without using GCPs and Vanishing Points
L
H
3
6
WX
Y
Z
5
87
4
21
a
H
W
Z
3Y
X
O
L
X3, Y3X3, Y3
a
Overview of Our ApproachOverview of Our Approach
• Self-Calibration
– Recover camera rotation • Initial estimate of camera rotation
• Refinement of camera rotation
– Determine camera translation and building dimensions
• Simultaneous estimates of camera translation and the first building dimensions
• Metric Reconstruction
– Roughly estimate the second building orientation
– Refine the second building orientation
– Determine the second building dimensions and location
• Initial estimate of camera rotation
• Refinement of camera rotation
Recover Camera RotationRecover Camera Rotation
Imaging Geometry Relationship
n
ii
Ti RvmO 2
1 )(
0RvmT
nTnn
Tnn
Tn
TTT
n vRmvRmvRm
vRmvRmvRm
J'''
1'
11'
11'
1
3
• Form objective function
• Solve a constrained
quadratic form minimization
problem
Determine Camera Translation and the First Building DimensionsDetermine Camera Translation and the First Building Dimensions
Imaging Geometry Relationship
0)( tuRmT
n
ii
Ti tuRmO 2
2 ))((
87
6
43
8
7 1
2
Building 2Model edge 56 (v, u)
Camera Coordinate System
Image edge 56{(x1, y1, -f), (x2, y2, -f)}
Object Coordinate System
Building 1Model edge 67(v, u)
5
m (mx, my, mz )C
3( X3, Y3)
4
5
6
12
z
Y
t(X0,Y0,Z0)R
11
• Assuming both buildings lie on the same ground plane
• Initial estimate of the second
building orientation
• Refinement of the second
building orientation
Recover the Second Building OrientationRecover the Second Building Orientation
Imaging Geometry Relationship
n
ii
Ti RvmO 2
1 )(
0cossin 21 WaWa Ti
Ti
Rma Ti
Ti
),,(
x
87
6
43
8
7 1
2
Building 2Model edge 56 (v, u)
Camera Coordinate System
Image edge 56{(x1, y1, -f), (x2, y2, -f)}
Object Coordinate System
Building 1Model edge 67(v, u)
5
m (mx, my, mz )C
3( X3, Y3)
4
5
6
12
z
Y
t(X0,Y0,Z0)R
11
Determine the Second Building Dimensions and LocationsDetermine the Second Building Dimensions and Locations• Unknown parameters
– Building dimensions L, W, H
– Building location X3, Y3
• Known parameters
– Camera pose
– Building orientation
• Solution
– Solve a set of linear equations
Imaging Geometry Relationship
),,(
x
n
ii
Ti tuRmO 2
2 ))((
Comparison with VP based Methods Using Identical Simulation DataComparison with VP based Methods Using Identical Simulation Data• Using same error for two methods
– Additive random noise in endpoints of image segments
– Principle points offsets
• Impact on the outputs from two methods
– Camera pose
– Geometry of the reconstructed buildings
– Topology of the reconstructed buildings
Random Errors in Image SegmentsRandom Errors in Image Segments
Sensi ti vi ty Anal ysi s of Camera Rotati on for Two Methods
05
101520253035
0 5 10 15
Random errors i n endpoi nts of i mage segmentsi n pi xel
Abso
lute
err
ors
inde
gree
Omega-MPhi -MKap-MOmega-VPhi -VKap-V
Sensi t i vi ty Anal ysi s of the Fi rst bui l di ng Di mensi ons f orTwo Methods
0
10
20
30
40
0 5 10 15
Random errors i n endpoi nts of i mage segmentsi n pi xel
Abso
lute
err
ors
inme
ter
Wi dth- MHei ght - MWi dth- VHei ght - V
Sensi t i vi ty Anal ysi s of the Second Bui l di ng Di mensi ons f orTwo Methods
0
10
20
30
40
50
0 5 10 15
Random errors i n endpoi nts of i mage segmentsi n pi xel
Abso
lute
err
ors
inme
ter
Length- MWi dth- MHei ght - MLength- VWi dth- VHei ght - V
Sensi t i vi ty Anal ysi s of Camera Transl at i on f or Two Methods
0
50
100
150
200
250
0 5 10 15
Random errors i n endpoi nts of i mage segments i npi xel
Abso
lute
err
ors
inme
ter
X- MY- MZ- MX- VY- VZ- V
Principle Point OffsetsPrinciple Point Offsets
Sensi t i vi ty Anal ysi s of Camera Transl at i on f or Two Methods
0
1
2
3
4
5
6
0 5 10 15
Pri nci pl e poi nts off sets i n pi xel
Abso
lute
err
ors
inme
ter
X- MY- MZ- MX- VY- VZ- V
Sensi ti vi ty Anal ysi s of Camera Rotati on for Two Methods
0
0. 05
0. 1
0. 15
0. 2
0. 25
0 5 10 15
Pri nci pl e poi nts off sets i n pi xel
Abso
lute
err
ors
inde
gree
Omega-MPhi -MKap-MOmega-VPhi -VKap-V
Sensi t i vi ty Anal ysi s of the Fi rst Bui l di ng Di mensi ons f orTwo Methods
00. 050. 1
0. 150. 2
0. 250. 3
0. 350. 4
0 5 10 15
Pri nci pl e poi nt off sets i n pi xel
Abso
lute
err
ors
inme
ter
Wi dth- MHei ght - MWi dth- VLength- V
Sensi t i vi ty Anal ysi s of the Second Bui l di ng Di mensi ons f orTwo Methods
0
0. 05
0. 1
0. 15
0. 2
0. 25
0 5 10 15
Pri nci pl e poi nt off sets i n pi xel
Abso
lute
err
ors
inme
ter
Length- MWi dth- MHei ght - MLength- VWi dth- VHei ght - V
Comparison Using Identical Real Data Comparison Using Identical Real Data
• Digital Camera: Canon PowerShot SD750
• Image size (3072x 2304 pixels)
Image 2: Burnside Hall, McGill universityImage 1: Two boxes
Results from the Image 1Results from the Image 1
Unit: mm Dimensions measured using a
ruler
Dimensions computed from the image
Absolute errors
Left BoxWidth
V71.1
73.1 2
M 70.9 0.2
HeightV
123.197.9 25.2
M 122.6 0.5
RightBox
LengthV
72.168.8 3.3
M 71.7 0.4
WidthV
50.647.6 3
M 49.6 1
HeightV
15.36.3 9
M 14.8 0.5
Results from the Image 2Results from the Image 2
Unit: meter Dimensions from DWG file Dimensions computed from the image
Length 35.44 35.44
Width 32.42 34.92
Height 50.00 53.33
3D model of Burnside Hall in Google Earth
Visualization of the recovered camera pose and building
ConclusionsConclusions
• Interactive solution to metric reconstruction from single images
– Model-based approach but without model-to-image projection and readjustment procedure
– better than vanishing point based methods
• Using an off-the-shelf camera, taking a picture, one can get object dimensions
Thank You!
Experimental DesignExperimental Design
• Camera Parameters
• Building Parameters
Focal Length
(m)
X0 (m) Y0(m) Z0(m) Omg (degree)
Phi ( degree)
Kap ( degree)
0.0798 -500.672 100.317 650.783 50.346 3.582 2.787
Building 1 Building 2
Length (m) 40 26.413
Width (m) 20 20.927
Height (m) 30 22.315
Orientation along X axis (degree)
0 30.856
Location of a building model vertex (m) X3 100.512
Y3 -200.217