Camera calibration technique
-
Upload
krzysztof-wegner -
Category
Education
-
view
39 -
download
3
Transcript of Camera calibration technique
Camera calibration techniquewprowadzenie teoretyczne
Krzysztof WegnerChair of Multimedia Telecommunications and Microelectronics
Poznań University of Technology, Poland
1
Goal of the calibration Knowledge about
Intrinsic camera parameters Focal length Optical center
Extrinsic camera parameters -Position of the camera in 3D world
Orientation of the camera Translation
2
Goal of the calibration Knowledge about
Intrinsic camera parameters Focal length Optical center
Extrinsic camera parameters -Position of the camera in 3D world
Orientation of the camera Translation Common word coordinate system
3
Camera parameters Intrinsic camera parameters
Focal length Optical center
Extrinsic camera parameters -Position of the camera in 3D world
Orientation of the camera Translation
4
𝑨=[ 𝑓 𝑢 𝛾 𝑜𝑢
0 𝑓 𝑣 𝑜𝑣
0 0 1 ]
𝑹=[𝒓𝟏 𝒓𝟐 𝒓𝟑 ]𝑻=[𝑡𝑥𝑡𝑦𝑡 𝑧 ]
Camera model Projection of a 3D point Onto a point at image plane
s is a scale – distance to the point
5
𝑠 ∙[𝑢𝑣1 ]=𝐴 ∙ [𝑅 −𝑅 ∙𝑇 ] ∙ [𝑋𝑌𝑍1 ]
Zhang’s Algorithm Allows estimation of
intrinsic parameters - matrix extrinsic parameters – rotation matrix and
translation vector Planar template
6Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝑠 ∙[𝑢𝑣1 ]=𝐴 ∙ [𝑟 1 𝑟2 𝑟 3~𝑡 ] ∙ [ 𝑋𝑌𝑍1 ]
Zhang’s algorithm Planar template
7Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
Zhang’s algorithm Planar template
8Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
Estimating Homography H We know position of the pattern’s feature points From registrated image we know
So
9Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
Estimating Homography H We know position of the pattern’s feature points From registrated image we know
10Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
Estimating Homography H
Let’s assign
We have 2 equations and 9 variables so we need at least 5 points to solve uniquely for
11Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
Estimating Homography H Defined up to a scale factor We know position of the pattern’s feature points From registrated image we know
Multiplication of both side by don’t change known
So we don’t know whether we obtain or So we don’t know scale of the scene
12Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=𝝀 ∙𝑯 ∙[ 𝑋𝑌1 ] 𝜆 ∙𝑠 ∙[𝑢𝑣1 ]=~𝑯 ∙[ 𝑋𝑌1 ]
Homography H Defined up to a scale factor We know position of the pattern’s feature points From registrated image we know
13Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
Homography H Defined up to a scale factor We know position of the pattern’s feature points From registrated image we know
Multiplication of both side by don’t change known
14Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=𝝀 ∙𝑯 ∙[ 𝑋𝑌1 ]𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=~𝑯 ∙[ 𝑋𝑌1 ]
Homography H Defined up to a scale factor
Multiplication of both side by don’t change known
So we don’t know whether we obtain or So we don’t know Z scale of the scene
15Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=𝝀 ∙𝑯 ∙[ 𝑋𝑌1 ]𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=~𝑯 ∙[ 𝑋𝑌1 ]
Constraints on the intrinsic Vectors , are orthonormal so taking dot product
gives
and length of , should be the same
For
we have
and
16Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝒓𝟏𝑻 ∙𝒓𝟐=𝟎
‖𝒓 𝟏‖𝟐=‖𝒓𝟐‖
𝟐⟹𝒓𝟏𝑻 ∙𝒓 𝟏=𝒓𝟐
𝑻 ∙𝒓𝟐
Constraints on the intrinsic Puting all together
17Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝒓𝟏𝑻 ∙𝒓𝟐=𝟎
𝒓𝟏𝑻 ∙𝒓𝟏=𝒓 𝟐
𝑻 ∙𝒓𝟐
(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑨−𝑻 ∙𝜆−1∙ 𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎
(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙ 𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟏=(𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐 )𝑻 ∙𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟐
𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝜆−1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐
𝑻 ∙𝑨−𝑻 ∙𝜆− 1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐
𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐
𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐
Constraints on the intrinsic Puting all together
18Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝒓𝟏𝑻 ∙𝒓𝟐=𝟎
𝒓𝟏𝑻 ∙𝒓𝟏=𝒓 𝟐
𝑻 ∙𝒓𝟐
(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑨−𝑻 ∙𝜆−1∙ 𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎
(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙ 𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟏=(𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐 )𝑻 ∙𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟐
𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝜆−1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐
𝑻 ∙𝑨−𝑻 ∙𝜆− 1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐
𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐
𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐
Closed form solution Try to solve
Intrinsic matrix
Let’s assign
19Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝑨=[ 𝑓 𝑢 𝛾 𝑜𝑢
0 𝑓 𝑣 𝑜𝑣
0 0 1 ]𝒉𝟏
𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐
𝑩=𝑨−𝑻 ∙ 𝑨−𝟏
𝑩=[1𝑓 𝑢2
−𝛾𝑓 𝑢2 𝑓 𝑣
𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢
𝑓 𝑢2 𝑓 𝑣
−𝛾𝑓 𝑢2 𝑓 𝑣
1𝑓 𝑣
2+𝛾2
𝑓 𝑢2 𝑓 𝑣
2
−𝑜𝑣❑
𝑓 𝑣2 −
𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢)𝑓 𝑢2 𝑓 𝑣
2
𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢
𝑓 𝑢2 𝑓 𝑣
−𝑜𝑣❑
𝑓 𝑣2 −
𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )𝑓 𝑢2 𝑓 𝑣
2 1+𝑜𝑣
2
𝑓 𝑣2 +
(𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )2
𝑓 𝑢2 𝑓 𝑣
2]=[𝐵11 𝐵12 𝐵13
𝐵12 𝐵22 𝐵23
𝐵13 𝐵23 𝐵33]
Closed form solution Let’s assign
is symetrical Let’s assign
20Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝑩=𝑨−𝑻 ∙ 𝑨−𝟏
𝑩=[1𝑓 𝑢2
−𝛾𝑓 𝑢2 𝑓 𝑣
𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢
𝑓 𝑢2 𝑓 𝑣
−𝛾𝑓 𝑢2 𝑓 𝑣
1𝑓 𝑣
2+𝛾2
𝑓 𝑢2 𝑓 𝑣
2
−𝑜𝑣❑
𝑓 𝑣2 −
𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢)𝑓 𝑢2 𝑓 𝑣
2
𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢
𝑓 𝑢2 𝑓 𝑣
−𝑜𝑣❑
𝑓 𝑣2 −
𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )𝑓 𝑢2 𝑓 𝑣
2 1+𝑜𝑣
2
𝑓 𝑣2 +
(𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )2
𝑓 𝑢2 𝑓 𝑣
2]=[𝐵11 𝐵12 𝐵13
𝐵12 𝐵22 𝐵23
𝐵13 𝐵23 𝐵33]
𝒃=[𝐵11 𝐵12 𝐵13 𝐵22 𝐵23 𝐵33 ]
Closed form solution We try to solve
Let’s see pattern in the equations
21Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎
𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐
𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐
𝑩=𝑨−𝑻 ∙ 𝑨−𝟏
𝒉𝟏𝑻 ∙𝑩 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑩 ∙𝒉𝟏=𝒉𝟐𝑻 ∙𝑩 ∙𝒉𝟐
𝒉𝒊𝑻 ∙𝑩 ∙𝒉 𝒋=𝒗𝒊𝒋
𝑻 ∙𝒃=
¿𝐵11 ∙ h𝑖1 ∙ h 𝑗 1+𝐵12 ∙ h𝑖1 ∙ h 𝑗2+𝐵13 ∙ h𝑖1 ∙ h 𝑗3+¿
¿ 𝒗𝒊𝒋𝑻 ∙𝒃
+ ++
𝒗 𝒊𝒋𝑻=[h𝑖 1 ∙ h 𝑗 1 h𝑖1 ∙ h 𝑗 2+h𝑖2 ∙ h 𝑗1 h𝑖1 ∙h 𝑗3+h𝑖3 ∙ h 𝑗 1 h𝑖2 ∙ h 𝑗 2 h𝑖 2 ∙ h 𝑗 3+h𝑖 3 ∙ h 𝑗2 h𝑖3 ∙ h 𝑗3 ]
Closed form solution We try to solve
so
22Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝒉𝟏𝑻 ∙𝑩 ∙𝒉𝟐=𝟎𝒉𝟏
𝑻 ∙𝑩 ∙𝒉𝟏−𝒉𝟐𝑻 ∙𝑩 ∙𝒉𝟐=𝟎
𝒉𝒊𝑻 ∙𝑩 ∙𝒉 𝒋=𝒗𝒊𝒋
𝑻 ∙𝒃
𝒗𝟏𝟐𝑻 ∙𝒃=𝟎𝒗𝟏𝟏𝑻 ∙𝒃−𝒗𝟐𝟐
𝑻 ∙𝒃=𝟎𝒗𝟏𝟐𝑻 ∙𝒃=𝟎
(𝒗𝟏𝟏𝑻 −𝒗𝟐𝟐
𝑻 ) ∙𝒃=𝟎
[ 𝒗𝟏𝟐𝑻
(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻 ] ∙𝒃=𝟎
Solving for b
Two equation are defined but have 6 unknowns. So at least 3 images are required to uniquly solve for Because all images are captured with the same
camera we can stack equation for together
23Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
[ 𝒗𝟏𝟐𝑻
(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻 ] ∙𝒃=𝟎
[ 𝒗𝟏𝟐′𝑻
(𝒗 ′𝟏𝟏−𝒗 ′𝟐𝟐)𝑻 ] ∙𝒃=𝟎
[ 𝒗 ′ ′𝟏𝟐𝑻
(𝒗 ′ ′𝟏𝟏−𝒗 ′ ′𝟐𝟐 )𝑻 ]∙𝒃=𝟎
[ 𝒗𝟏𝟐𝑻
(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻 ] ∙𝒃=𝟎
[𝒗𝟏𝟐𝑻
(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻
𝒗𝟏𝟐′𝑻
(𝒗 ′𝟏𝟏−𝒗 ′𝟐𝟐 )𝑻
𝒗 ′ ′𝟏𝟐𝑻
(𝒗 ′ ′𝟏𝟏−𝒗 ′ ′𝟐𝟐)𝑻] ∙𝒃=𝟎𝑽=[
𝒗𝟏𝟐𝑻
(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻
𝒗𝟏𝟐′𝑻
(𝒗 ′𝟏𝟏−𝒗 ′𝟐𝟐)𝑻
𝒗 ′ ′𝟏𝟐𝑻
(𝒗 ′ ′𝟏𝟏−𝒗 ′ ′𝟐𝟐 )𝑻]
Solving for b
There is trivia solution
But we look for non trivial solution so Such solution is given by eigenvector of
asociated with the smallest eigenvalue (right singulart vector of
24Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝑽 ∙𝒃=𝟎
𝑽 𝑻 ∙𝑽
Retriving intrinsic parameters Once we have We can calculate intrinsic parameters from
25Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝑩=[1𝑓 𝑢2
−𝛾𝑓 𝑢2 𝑓 𝑣
𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢
𝑓 𝑢2 𝑓 𝑣
−𝛾𝑓 𝑢2 𝑓 𝑣
1𝑓 𝑣
2+𝛾2
𝑓 𝑢2 𝑓 𝑣
2
−𝑜𝑣❑
𝑓 𝑣2 −
𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢)𝑓 𝑢2 𝑓 𝑣
2
𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢
𝑓 𝑢2 𝑓 𝑣
−𝑜𝑣❑
𝑓 𝑣2 −
𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )𝑓 𝑢2 𝑓 𝑣
2 1+𝑜𝑣
2
𝑓 𝑣2 +
(𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )2
𝑓 𝑢2 𝑓 𝑣
2]=[𝐵11 𝐵12 𝐵13
𝐵12 𝐵22 𝐵23
𝐵13 𝐵23 𝐵33]
𝒃=[𝐵11 𝐵12 𝐵13 𝐵22 𝐵23 𝐵33 ]
𝑜𝑣=𝐵12 ∙𝐵13−𝐵11 ∙𝐵23
𝐵11 ∙𝐵22−𝐵122𝑓 𝑢=√ 𝜆
𝐵11
𝜆=𝐵33−𝐵132 +𝑜𝑣 (𝐵12 ∙𝐵13−𝐵11 ∙𝐵23 )
𝐵11
𝑓 𝑣=√ 𝜆 ∙𝐵11
𝐵11 ∙𝐵22−𝐵122
𝛾=−𝐵12 ∙ 𝑓 𝑢2 ∙ 𝑓 𝑣
𝜆𝑜𝑢=
𝛾 ∙𝑜𝑣
𝑓 𝑣−𝐵13
𝑓 𝑢2
𝜆
Retriving position We know constraints
To complete rotation matrix we calculater third column
Finally we ortonormalize rotation matrix
26Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000
𝒓𝟑=𝒓𝟏×𝒓 𝟐
‖𝒓 𝟏‖=‖𝒓𝟐‖=‖𝒓𝟑‖=𝟏
Summary Camera parameters requires at least 5
point pattern Intrinsic camera parameters estimation
requires at least 3 images at different orientation
All parameters are defined up to a unknown scale
M37232, October 2015, Geneve 27