Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre...
-
Upload
tracey-stevens -
Category
Documents
-
view
230 -
download
0
Transcript of Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre...
![Page 1: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/1.jpg)
Self-Calibration and Metric Reconstruction from Single
Images
Self-Calibration and Metric Reconstruction from Single
Images
Ruisheng Wang
Frank P. Ferrie
Centre for Intelligent Machines, McGill University
![Page 2: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/2.jpg)
OutlineOutline
• Contributions
• Existing Methods
• The Idea
• Our Method
• Comparison
• Conclusion
![Page 3: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/3.jpg)
ContributionsContributions
• We developed a direct solution to 3D reconstruction from single images which is model based
– No model-to-image projection and readjustment procedure
• We made it possible to perform accurate 3D measurement using an uncalibrated* camera based on single images only
• We quantitatively evaluated our model-based approach with vanishing point based method, and the results indicate our approach is better than vanishing point based method
* Intrinsic parameters known/estimated
![Page 4: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/4.jpg)
Three vanishing points
Vanishing Points (VPs) Based MethodVanishing Points (VPs) Based Method
• Determine three orthogonal vanishing points
– Manual detection
– Search over Gaussian sphere
– Hough transform
– Projective geometry
• Determine camera focal length and rotation
• Determine camera translation and model dimensions
)/tan( 22 YY yxfa
)/tan( 222222 XXZZ yxfyxfa
)/tan( YY yxak
)( YXYX yyxxf
![Page 5: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/5.jpg)
Problems in VPs Based MethodProblems in VPs Based Method
• Three orthogonal VPs may not be always available
– One or two vanishing point only
• Hard to accurately determine VPs
– Need many lines
• The accuracy of VPs affects the accuracy of the 3D reconstruction
1 Point Perspective 2 Points Perspective
![Page 6: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/6.jpg)
Methods with Ground Control Points/LinesMethods with Ground Control Points/Lines• Point-based methods
– Collinearity Equations
• Line-based methods
– Model-to-Image Fitting
)()()(
)()()(*
)()()(
)()()(*
033032031
023022021
033032031
013012011
ZZmYYmXXm
ZZmYYmXXmfy
ZZmYYmXXm
ZZmYYmXXmfx
2 2 21 1 2 20
( ) ( ) ( )3
l T Ti
lErr h s ds h h h h m A BA m
From Debevec et al.1996
![Page 7: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/7.jpg)
The IdeaThe Idea
• Use model to estimate camera exterior orientation
– Need 6 parameters X3, Y3, L, W, H,
• If an object-centered coordinate system selected– Need three parameters L, W, H
• Divide camera parameters into two groups: rotation and translation
• It’s possible to estimate relative camera exterior orientation without using GCPs and Vanishing Points
L
H
3
6
WX
Y
Z
5
87
4
21
a
H
W
Z
3Y
X
O
L
X3, Y3X3, Y3
a
![Page 8: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/8.jpg)
Overview of Our ApproachOverview of Our Approach
• Self-Calibration
– Recover camera rotation • Initial estimate of camera rotation
• Refinement of camera rotation
– Determine camera translation and building dimensions
• Simultaneous estimates of camera translation and the first building dimensions
• Metric Reconstruction
– Roughly estimate the second building orientation
– Refine the second building orientation
– Determine the second building dimensions and location
![Page 9: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/9.jpg)
• Initial estimate of camera rotation
• Refinement of camera rotation
Recover Camera RotationRecover Camera Rotation
Imaging Geometry Relationship
n
ii
Ti RvmO 2
1 )(
0RvmT
nTnn
Tnn
Tn
TTT
n vRmvRmvRm
vRmvRmvRm
J'''
1'
11'
11'
1
3
![Page 10: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/10.jpg)
• Form objective function
• Solve a constrained
quadratic form minimization
problem
Determine Camera Translation and the First Building DimensionsDetermine Camera Translation and the First Building Dimensions
Imaging Geometry Relationship
0)( tuRmT
n
ii
Ti tuRmO 2
2 ))((
![Page 11: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/11.jpg)
87
6
43
8
7 1
2
Building 2Model edge 56 (v, u)
Camera Coordinate System
Image edge 56{(x1, y1, -f), (x2, y2, -f)}
Object Coordinate System
Building 1Model edge 67(v, u)
5
m (mx, my, mz )C
3( X3, Y3)
4
5
6
12
z
Y
t(X0,Y0,Z0)R
11
• Assuming both buildings lie on the same ground plane
• Initial estimate of the second
building orientation
• Refinement of the second
building orientation
Recover the Second Building OrientationRecover the Second Building Orientation
Imaging Geometry Relationship
n
ii
Ti RvmO 2
1 )(
0cossin 21 WaWa Ti
Ti
Rma Ti
Ti
),,(
x
![Page 12: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/12.jpg)
87
6
43
8
7 1
2
Building 2Model edge 56 (v, u)
Camera Coordinate System
Image edge 56{(x1, y1, -f), (x2, y2, -f)}
Object Coordinate System
Building 1Model edge 67(v, u)
5
m (mx, my, mz )C
3( X3, Y3)
4
5
6
12
z
Y
t(X0,Y0,Z0)R
11
Determine the Second Building Dimensions and LocationsDetermine the Second Building Dimensions and Locations• Unknown parameters
– Building dimensions L, W, H
– Building location X3, Y3
• Known parameters
– Camera pose
– Building orientation
• Solution
– Solve a set of linear equations
Imaging Geometry Relationship
),,(
x
n
ii
Ti tuRmO 2
2 ))((
![Page 13: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/13.jpg)
Comparison with VP based Methods Using Identical Simulation DataComparison with VP based Methods Using Identical Simulation Data• Using same error for two methods
– Additive random noise in endpoints of image segments
– Principle points offsets
• Impact on the outputs from two methods
– Camera pose
– Geometry of the reconstructed buildings
– Topology of the reconstructed buildings
![Page 14: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/14.jpg)
Random Errors in Image SegmentsRandom Errors in Image Segments
Sensi ti vi ty Anal ysi s of Camera Rotati on for Two Methods
05
101520253035
0 5 10 15
Random errors i n endpoi nts of i mage segmentsi n pi xel
Abso
lute
err
ors
inde
gree
Omega-MPhi -MKap-MOmega-VPhi -VKap-V
Sensi t i vi ty Anal ysi s of the Fi rst bui l di ng Di mensi ons f orTwo Methods
0
10
20
30
40
0 5 10 15
Random errors i n endpoi nts of i mage segmentsi n pi xel
Abso
lute
err
ors
inme
ter
Wi dth- MHei ght - MWi dth- VHei ght - V
Sensi t i vi ty Anal ysi s of the Second Bui l di ng Di mensi ons f orTwo Methods
0
10
20
30
40
50
0 5 10 15
Random errors i n endpoi nts of i mage segmentsi n pi xel
Abso
lute
err
ors
inme
ter
Length- MWi dth- MHei ght - MLength- VWi dth- VHei ght - V
Sensi t i vi ty Anal ysi s of Camera Transl at i on f or Two Methods
0
50
100
150
200
250
0 5 10 15
Random errors i n endpoi nts of i mage segments i npi xel
Abso
lute
err
ors
inme
ter
X- MY- MZ- MX- VY- VZ- V
![Page 15: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/15.jpg)
Principle Point OffsetsPrinciple Point Offsets
Sensi t i vi ty Anal ysi s of Camera Transl at i on f or Two Methods
0
1
2
3
4
5
6
0 5 10 15
Pri nci pl e poi nts off sets i n pi xel
Abso
lute
err
ors
inme
ter
X- MY- MZ- MX- VY- VZ- V
Sensi ti vi ty Anal ysi s of Camera Rotati on for Two Methods
0
0. 05
0. 1
0. 15
0. 2
0. 25
0 5 10 15
Pri nci pl e poi nts off sets i n pi xel
Abso
lute
err
ors
inde
gree
Omega-MPhi -MKap-MOmega-VPhi -VKap-V
Sensi t i vi ty Anal ysi s of the Fi rst Bui l di ng Di mensi ons f orTwo Methods
00. 050. 1
0. 150. 2
0. 250. 3
0. 350. 4
0 5 10 15
Pri nci pl e poi nt off sets i n pi xel
Abso
lute
err
ors
inme
ter
Wi dth- MHei ght - MWi dth- VLength- V
Sensi t i vi ty Anal ysi s of the Second Bui l di ng Di mensi ons f orTwo Methods
0
0. 05
0. 1
0. 15
0. 2
0. 25
0 5 10 15
Pri nci pl e poi nt off sets i n pi xel
Abso
lute
err
ors
inme
ter
Length- MWi dth- MHei ght - MLength- VWi dth- VHei ght - V
![Page 16: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/16.jpg)
Comparison Using Identical Real Data Comparison Using Identical Real Data
• Digital Camera: Canon PowerShot SD750
• Image size (3072x 2304 pixels)
Image 2: Burnside Hall, McGill universityImage 1: Two boxes
![Page 17: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/17.jpg)
Results from the Image 1Results from the Image 1
Unit: mm Dimensions measured using a
ruler
Dimensions computed from the image
Absolute errors
Left BoxWidth
V71.1
73.1 2
M 70.9 0.2
HeightV
123.197.9 25.2
M 122.6 0.5
RightBox
LengthV
72.168.8 3.3
M 71.7 0.4
WidthV
50.647.6 3
M 49.6 1
HeightV
15.36.3 9
M 14.8 0.5
![Page 18: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/18.jpg)
Results from the Image 2Results from the Image 2
Unit: meter Dimensions from DWG file Dimensions computed from the image
Length 35.44 35.44
Width 32.42 34.92
Height 50.00 53.33
3D model of Burnside Hall in Google Earth
Visualization of the recovered camera pose and building
![Page 19: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/19.jpg)
ConclusionsConclusions
• Interactive solution to metric reconstruction from single images
– Model-based approach but without model-to-image projection and readjustment procedure
– better than vanishing point based methods
• Using an off-the-shelf camera, taking a picture, one can get object dimensions
![Page 20: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/20.jpg)
Thank You!
![Page 21: Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University.](https://reader036.fdocuments.in/reader036/viewer/2022062500/56649ed95503460f94be77e0/html5/thumbnails/21.jpg)
Experimental DesignExperimental Design
• Camera Parameters
• Building Parameters
Focal Length
(m)
X0 (m) Y0(m) Z0(m) Omg (degree)
Phi ( degree)
Kap ( degree)
0.0798 -500.672 100.317 650.783 50.346 3.582 2.787
Building 1 Building 2
Length (m) 40 26.413
Width (m) 20 20.927
Height (m) 30 22.315
Orientation along X axis (degree)
0 30.856
Location of a building model vertex (m) X3 100.512
Y3 -200.217