Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin...
Transcript of Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin...
![Page 1: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/1.jpg)
Lin ZHANG, SSE, 2016
Lecture 7Measurement Using a Single Camera
Lin ZHANG, PhDSchool of Software Engineering
Tongji UniversityFall 2016
![Page 2: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/2.jpg)
Lin ZHANG, SSE, 2016
If I have an image containing a coin, can you tell me the diameter of that coin?
![Page 3: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/3.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• What is Camera Calibration• Single Camera Calibration• Bird’s‐eye‐view Generation
![Page 4: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/4.jpg)
Lin ZHANG, SSE, 2016
Vector representation
{ , , }a xi y j zk x y z
Length (or norm) of a vector
2 2 2a x y z
Normalized vector (unit vector)
{ , , }a x y za a a a
We say if and only if,a 0 0, 0, 0x y z
Vector operations
![Page 5: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/5.jpg)
Lin ZHANG, SSE, 2016
if 1 1 1( , , ),a x y z
2 2 2( , , ),b x y z
1 2 1 2 1 2( , , ),a b x x y y z z
then
1 2 1 2 1 2cosa b a b x x y y z z
Dot product (inner product)
Laws of dot product:
, ( )= +a b b a a b c a b a c
Theorem
0a b a b (why?)
a
b
Vector operations
![Page 6: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/6.jpg)
Lin ZHANG, SSE, 2016
Cross product
1 1 1 1 1 1
1 1 1
2 2 2 2 2 2
2 2 2
i j ky z z x x y
a b x y z i j ky z z x x y
x y z
Vector operations
![Page 7: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/7.jpg)
Lin ZHANG, SSE, 2016
c a b
Cross product
is also a vector, whose direction is determined by the right‐hand law and
a
bc
,c a c b
sinc a b
represents the oriented area of the parallelogram taking and as two sidesc
a b(easy to prove)
1 2 2 1r r r r
(why?)
Vector operations
![Page 8: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/8.jpg)
Lin ZHANG, SSE, 2016
they are not equal to zero at the same time, and
Cross product
Theorem|| 0a b a b (why?)
Theorem
|| , ,a b
a b 0
1 2 3 1 2 1 3( )r r r r r r r
Property
(easy to understand)
Vector operations
![Page 9: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/9.jpg)
Lin ZHANG, SSE, 2016
Mixed product (scalar triple product or box product)
Geometric Interpretation: it is the (signed) volume of the parallelepiped defined by the three vectors given
1 2 3
1 2 3
1 2 3
( , , ) ( )a a ab b bc c c
a b c a b c
Vector operations
![Page 10: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/10.jpg)
Lin ZHANG, SSE, 2016
Mixed product (scalar triple product or box product)
1 2 3
1 2 3
1 2 3
( , , ) ( )a a ab b bc c c
a b c a b c
( ) cos
sin cos
a b c a b c
a b c
a
cBase h
Vector operations
![Page 11: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/11.jpg)
Lin ZHANG, SSE, 2016
Mixed product (scalar triple product or box product)
1 2 3
1 2 3
1 2 3
( , , ) ( )a a ab b bc c c
a b c a b c
Property:
( , , ) ( , , ) ( , , ) a b c b c a c a b( , , ) ( , , ) ( , , ) a b c b a c a c b
Vector operations
why?
![Page 12: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/12.jpg)
Lin ZHANG, SSE, 2016
Mixed product (scalar triple product or box product)Theorem
, ,a b c are coplanar ( , , ) 0 a b c
, ,a b c are coplanar , , ,v they are not equal to zero at the same time, and v a b c 0
Vector operations
why?
![Page 13: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/13.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?
For a normal point on a plane , ,Tx y 0Its homogenous coordinate is where k can be anynon‐zero real number
, ,1 ,Tk x y
For a homogenous coordinate
Homogenous coordinate for a point is not only one
' ' ', ,T
x y z
we usually rewrite it as ' ' ' '/ , / ,1T
x z y z
![Page 14: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/14.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?
Converting from homogenous coordinate to inhomogeneous coordinate
''
''
''
'
xxzyy
zz
For a normal point on a plane 0Its homogenous coordinate is where k can be anynon‐zero real number
, ,1 ,Tk x y , ,Tx y
![Page 15: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/15.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?
Geometric interpretation
1e
2e
o
1o
1
00 0( , )M x y
3e
1e
2e
In plane , in the 2D frame01 1 2( : , )o e e 0 0: ( , )M x y
Coordinate of any point on line OMin the frame is the homogeneous coordinate of M
1 2 3( : , , )o e e e
These points can be represented as
0 0, ,1 Tk x y
, one point
![Page 16: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/16.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?
Geometric interpretation
1e
2e
o
1o
1
00 0( , )M x y
3e
1e
2e
How about a line passing through O and parallel to ?0
In plane , in the 2D frame01 1 2( : , )o e e 0 0: ( , )M x y
Coordinate of any point on line OMin the frame is the homogeneous coordinate of M
, one point
These points can be represented as
0 0, ,1 Tk x y
1 2 3( : , , )o e e e
![Page 17: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/17.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?
Geometric interpretation
1e
2e
o
1o
1
03e
1e
2e
How about a line passing through O and parallel to ?0
Consider a line passing through Oand M(x0, y0, 0)T
0 0( , )M x y We define: it meets at an infinity point, and also the homogeneous coordinate of such a point can be represented as points on OM
0
So, infinity point has the form (kx0, ky0, 0)T
![Page 18: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/18.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?Normal case:
line k (x0, y0, 1)
a normal point (x0, y0) on the plane 0
The homogeneous coordinate of this normal point is k(x0, y0 ,1)
abnormal case:
line (kx0, ky0, 0)
Define: it meets at an infinity point
0
The homogeneous coordinate of this infinity point is k(x0, y0 ,0)
make an analogy
![Page 19: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/19.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• What is homogeneous coordinate?
Geometric interpretation
1e
2e
o
1o
1
03e
1e
2e
How about a line passing through O and parallel to ?0
0 0( , )M x y
One infinity point determines an orientation
We define: all infinity points on comprise an infinity line
0
In fact, plane meets at the infinity line
01 2oe e
Homogeneous equation of the infinity line is 0x + 0y + 1z = 0
![Page 20: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/20.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry
Properties of a projective plane• Two points determine a line; two lines determine a point (the
second claim is not correct in the normal Euclidean plane)• Two parallel lines intersect at an infinity point; that means one
infinity point corresponds to a specific orientation• Two parallel planes intersect at the infinity line
0 + infinity line = Projective plane
![Page 21: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/21.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry
![Page 22: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/22.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry
JingHu High‐speed railway: rails will “meet” at the vanishing point
![Page 23: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/23.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
On a projective plane, please determine the line passing two points
1e
2e
o1
0M
',o ox x determine two linesxx’ actually is the intersection between oxx’ and 0
'1 1 1 2 2 2( , , ) , ( , , )T Tx y z x y z x x
x'x
![Page 24: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/24.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
1e
2e
o1
0x
'xM Thus, locates on xx’( , , )M x y z oM resides on the plane oxx’ ', ,o oM ox x are coplanar
1 1 1
2 2 2
0x y zx y zx y z
On a projective plane, please determine the line passing two points '
1 1 1 2 2 2( , , ) , ( , , )T Tx y z x y z x x
![Page 25: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/25.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
1 1 1
2 2 2
0x y zx y zx y z
1 1 1 1 1 1
2 2 2 2 2 2
0y z z x x y
x y zy z z x x y
1 1 1 1 1 1
2 2 2 2 2 2
, ,T
y z z x x yy z z x x y
Homogeneous coordinate of the lineHomogeneous coordinate of the infinity line is (0,0,1)T
On a projective plane, please determine the line passing two points '
1 1 1 2 2 2( , , ) , ( , , )T Tx y z x y z x x
![Page 26: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/26.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
1 1 1
2 2 2
0x y zx y zx y z
1 1 1 1 1 1
2 2 2 2 2 2
0y z z x x y
x y zy z z x x y
TheoremOn the projective plane, the line passing two points is',x x
' l x x
1 1 1 1 1 1
2 2 2 2 2 2
, ,T
y z z x x yy z z x x y
On a projective plane, please determine the line passing two points '
1 1 1 2 2 2( , , ) , ( , , )T Tx y z x y z x x
![Page 27: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/27.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
A point is on the line0 0 0( , , )Tx y zx
0T x l
( , , )Ta b cl
0 x l(It is )
![Page 28: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/28.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinateTheorem: On the projective plane, the intersection of two lines is the point ' x l l',l lProof: Two lines 1 1 1 0,a x b y c z 2 2 2 0a x b y c z
'1 1 1 2 2 2( , , ) , ( , , )T Ta b c a b c l l
Inhomogeneous form ,x yX Yz z
1 1 1
2 2 2
00
a X b Y ca X b Y c
1 1 1 1
2 2 2 2
1 1 1 1
2 2 2 2
,
c b a cc b a c
X Ya b a ba b a b
![Page 29: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/29.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
1 1 1 1
2 2 2 2
1 1 1 1
2 2 2 2
, ,1
c b a cc b a c
ka b a ba b a b
x
Homogenous form of the cross point is
1 1 1 1 1 1
2 2 2 2 2 2
, ,c b a c a bc b a c a b
x
let 1 1
2 2
a bk
a b
1 1 1 1 1 1
2 2 2 2 2 2
, ,b c c a a bb c c a a b
x ' x l l
Theorem: On the projective plane, the intersection of two lines is the point ' x l l',l l
![Page 30: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/30.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• Lines in the homogeneous coordinate
Example: find the cross point of the lines 1, 1x y
o x
y
1x
1y 1 2 3
1 2 3
1 0 ( 1) 00 1 ( 1) 0x x xx x x
Homogeneous coordinates of the two lines are (1,0, 1) , (0,1, 1)T T Cross point is
(1,0, 1) (0,1, 1) 1,1,1T T
Homogeneous form
Theorem: On the projective plane, the intersection of two lines is the point ' x l l',l l
1 2
3 3
,x xx yx x
![Page 31: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/31.jpg)
Lin ZHANG, SSE, 2016
(1,0, 1) (1,0, 2) 1 0 1 0,1,01 0 2
T T
i j k
Foundations of Projective Geometry• Lines in the homogeneous coordinate
Example: find the cross point of the lines 1, 2x x
o x
y
1x
Homogeneous coordinates of the two lines are (1,0, 1) , (1,0, 2)T T Cross point is
2x 1 2 3
1 2 3
1 0 ( 1) 01 0 ( 2) 0x x xx x x
Homogeneous form
Theorem: On the projective plane, the intersection of two lines is the point ' x l l',l l
1 2
3 3
,x xx yx x
![Page 32: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/32.jpg)
Lin ZHANG, SSE, 2016
Foundations of Projective Geometry• DualityIn projective geometry, lines and points can swap their positions
0T x lIf x is a variable, it represents the points lying on the line l;If l is a variable, it represents the lines passing a fixed point x
How to interpret?
The line passing two points is',x x ' l x xThe cross point of two lines is',l l ' x l l
Duality Principle: To any theorem of projective geometry, there corresponds a dual theorem, which may be derived by interchanging the roles of points and lines in the original theorem
![Page 33: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/33.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• What is Camera Calibration• Single Camera Calibration• Bird’s‐eye‐view Generation
![Page 34: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/34.jpg)
Lin ZHANG, SSE, 2016
Matrix differentiation
• Function is a vector and the variable is a scalar
1 2( ) ( ), ( ),..., ( ) Tnf t f t f t f t
Definition
1 2 ( )( ) ( ), ,...,T
ndf tdf t df tdfdt dt dt dt
![Page 35: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/35.jpg)
Lin ZHANG, SSE, 2016
• Function is a matrix and the variable is a scalar11 12 1
21 22 2
1 2
( ) ( ),..., ( )( ) ( ),..., ( )
( ) ( )
( ) ( ),..., ( )
m
mij n m
n n nm
f t f t f tf t f t f t
f t f t
f t f t f t
Definition111 12
221 22
1 2
( )( ) ( ) ,...,
( )( ) ( ) ( ),...,
( ) ( ) ( ),...,
m
mij
n m
n n nm
df tdf t df tdt dt dt
df tdf t df t df tdfdt dt dt
dt dt
df t df t df tdt dt dt
Matrix differentiation
![Page 36: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/36.jpg)
Lin ZHANG, SSE, 2016
• Function is a scalar and the variable is a vector
1 2( ), ( , ,..., )Tnf x x xx x
Definition
1 2
, ,...,T
n
df f f fd x x x
x
In a similar way,
1 2( ), ( , ,..., )nf x x xx x
1 2
, ,...,n
df f f fd x x x
x
Matrix differentiation
![Page 37: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/37.jpg)
Lin ZHANG, SSE, 2016
• Function is a vector and the variable is a vector 1 2 1 2, ,..., , ( ), ( ),..., ( )T T
n mx x x y y y x y x x xDefinition
1 1 1
1 2
2 2 2
1 2
1 2
( ) ( ) ( ), ,...,
( ) ( ) ( ), ,...,
( ) ( ) ( ), ,...,
n
nT
m m m
n m n
y y yx x x
y y yd x x xd
y y yx x x
x x x
x x xyx
x x x
Matrix differentiation
![Page 38: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/38.jpg)
Lin ZHANG, SSE, 2016
• Function is a vector and the variable is a vector 1 2 1 2, ,..., , ( ), ( ),..., ( )T T
n mx x x y y y x y x x xIn a similar way,
1 2
1 1 1
1 2
2 2 2
1 2
( ) ( ) ( ), ,...,
( ) ( ) ( ), ,...,
( ) ( ) ( ), ,...,
m
mT
m
n n n n m
y y yx x x
y y yd x x xd
y y yx x x
x x x
x x xyx
x x x
Matrix differentiation
![Page 39: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/39.jpg)
Lin ZHANG, SSE, 2016
• Function is a vector and the variable is a vectorExample:
11 2 2
2 1 1 2 2 3 22
3
( ), , ( ) , ( ) 3
( )
xy
x y x x y x xy
x
xy x x x
x
1 2
1 11
1 2
2 23
1 2
3 3
( ) ( )
2 0( ) ( ) 1 3
0 2( ) ( )
T
y yx x x
d y yd x x
xy y
x x
x x
y x xx
x x
Matrix differentiation
![Page 40: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/40.jpg)
Lin ZHANG, SSE, 2016
• Function is a scalar and the variable is a matrix
11 12 1
1 2
( ) n
m m mn
f f fx x x
dfd
f f fx x x
XX
( ), m nf X X
Definition
Matrix differentiation
![Page 41: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/41.jpg)
Lin ZHANG, SSE, 2016
• Useful results1, nx a
,T Td d
d d
a x x aa ax x
Then,
How to prove?
(1)
Matrix differentiation
![Page 42: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/42.jpg)
Lin ZHANG, SSE, 2016
• Useful results1,m n nA x (2) Then, T
dA Ad
xx
1,m n nA x (3) Then,T T
Td A Ad
xx
1,n n nA x (4) Then, ( )T
Td A A Ad
x x xx
1 1, ,m n m n X a b (5) Then,T
Tdd
a Xb abX
1 1, ,n m m n X a b (6) Then,T T
Tdd
a X b baX
1nx (7) Then, 2Td
d
x x xx
Matrix differentiation
![Page 43: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/43.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• What is Camera Calibration• Single Camera Calibration• Bird‐view Generation
![Page 44: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/44.jpg)
Lin ZHANG, SSE, 2016
Lagrange multiplier
• Single‐variable function
( )f x is differentiable in (a, b). At , f(x) achieves an extremum
0 ( , )x a b
0| 0x
dfdx
• Two‐variables function
( , )f x y is differentiable in its domain. At , f(x, y) achieves an extremum
0 0( , )x y
0 0 0 0( , ) ( , )| 0, | 0x y x yf fx y
![Page 45: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/45.jpg)
Lin ZHANG, SSE, 2016
Lagrange multiplier
• In general case1( ), nf x x If is a stationary point of 0x
0 0 01 2
| 0, | 0,..., | 0n
f f fx x x
x x x
![Page 46: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/46.jpg)
Lin ZHANG, SSE, 2016
Lagrange multiplier
• Lagrange multiplier is a strategy for finding the local extremum of a function subject to equality constraints
Problem: find stationary points for 1( ),x x ny f under m constraints ( ) 0, 1, 2,...,kg k m x
is a stationary point of with constraints
Solution:
11
( ; ,..., ) ( ) ( )m
m k kk
F f g
x x xIf is a stationary point of F, then,
0 10 20 0( , , ..., )m x
0x ( )f xJoseph‐Louis LagrangeJan. 25, 1736~Apr.10, 1813
![Page 47: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/47.jpg)
Lin ZHANG, SSE, 2016
Lagrange multiplier
• Lagrange multiplier is a strategy for finding the local extremum of a function subject to equality constraints
Solution:1
1( ; ,..., ) ( ) ( )
m
m k kk
F f g
x x x
is a stationary point of F0 10 0( , ,..., )m x
1 2 1 2
0, 0,..., 0, 0, 0,..., 0n m
F F F F F Fx x x
n + m equations!at that point
Problem: find stationary points for 1( ),x x ny f under m constraints ( ) 0, 1, 2,...,kg k m x
![Page 48: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/48.jpg)
Lin ZHANG, SSE, 2016
Lagrange multiplier
• ExampleProblem: for a given point p0 = (1, 0), among all the points lying on the line y=x, identify the one having the least distance to p0.
y=xp0
?
The distance is 2 2( , ) ( 1) ( 0)f x y x y
Now we want to find the stationary point of f(x, y) under the constraint
( , ) 0g x y y x According to Lagrange multiplier method, construct another function
2 2( , , ) ( ) ( ) ( 1) ( )F x y f x g x x y y x Find the stationary point for ( , , )F x y
![Page 49: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/49.jpg)
Lin ZHANG, SSE, 2016
Lagrange multiplier
• ExampleProblem: for a given point p0 = (1, 0), among all the points lying on the line y=x, identify the one having the least distance to p0.
y=xp0
?
0
0
0
FxFyF
2( 1) 02 0
0
xy
x y
0.50.51
xy
(0.5,0.5,1) is a stationary point of ( , , )F x y (0.5,0.5) is a stationary point of f(x,y) under constraints
![Page 50: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/50.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• Homograpy Estimation• What is Camera Calibration• Single Camera Calibration• Bird‐view Generation
![Page 51: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/51.jpg)
Lin ZHANG, SSE, 2016
LS for Inhomogeneous Linear SystemConsider the following linear equations system
1 2 1
1 2 2
3 1 1 32 4 2 1 4x x x
x x x
Matrix form: A x bA x b
It can be easily solved 1
2
12
xx
![Page 52: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/52.jpg)
Lin ZHANG, SSE, 2016
LS for Inhomogeneous Linear SystemHow about the following one?
1 21
1 22
1 2
3 1 1 32 4 2 1 4
1 2 62 6
x xx
x xx
x x
It does not have a solution!
What is the condition for a linear equation system can be solved?
A x b
Can we solve it in an approximate way?A: we can use least squares technique!
Carl Friedrich Gauss
![Page 53: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/53.jpg)
Lin ZHANG, SSE, 2016
LS for Inhomogeneous Linear SystemLet’s consider a system of p linear equations with q unknowns
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
...
...
......
q q
q q
p p pq q p
a x a x a x
a x a x a xA
a x a x a x
b
bx b
b
We consider the case: p>q, and rank(A)=qIn general case, there is no solution!
Instead, we want to find a vector x that minimizes the error:22
1 11
( ) ( ... )p
i iq q ii
E a x a x A
x b x b
unknowns
![Page 54: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/54.jpg)
Lin ZHANG, SSE, 2016
LS for Inhomogeneous Linear System2*2
arg min ( ) arg minE A x x
x x x b
1* T TA A A
x bPseudoinverse of A
How about the pseudoinverse of A when A is square and non-singular?
![Page 55: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/55.jpg)
Lin ZHANG, SSE, 2016
LS for Homogeneous Linear SystemLet’s consider a system of p linear equations with q unknowns
11 1 12 2 1
21 1 22 2 2
1 1 2 2
... 0
... 0
...... 0
q q
q q
p p pq q
a x a x a x
a x a x a xA
a x a x a x
x 0
We consider the case: p>q, and rank(A)=q
unknowns
Theoretically, there is only a trivial solution: x = 0
So, we add a constraint to avoid the trivial solution 2
1x
![Page 56: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/56.jpg)
Lin ZHANG, SSE, 2016
LS for Homogeneous Linear SystemWe want to minimize , subject to 2
1x 2
2( )x xE A
*2
arg min ( ), . ., 1x
x x xE s t
Use the Lagrange multiplier to solve it,
2 2*2 2
arg min 1x
x x xA
(1)
Solving the stationary point of the Lagrange function,
2 2
2 2
2 2
2 2
1
10
x x0
x
x x
A
A
(3)
(2)
![Page 57: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/57.jpg)
Lin ZHANG, SSE, 2016
LS for Homogeneous Linear System
Then, we have
2 2
2 21x x
0x
A
(3)
x xTA A
x is the eigen‐vector of ATA associated with the eigenvalue
2
2x x x x x xT T TE A A A
The unit vector x is the eigenvector associated with the minimum eigenvalue of TA A
![Page 58: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/58.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• Homograpy Estimation• What is Camera Calibration• Single Camera Calibration• Bird’s‐eye‐view Generation
![Page 59: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/59.jpg)
Lin ZHANG, SSE, 2016
Homography Estimation
Problem definition: Given a set of points and a corresponding set of points in a projective plane, compute the projective transformation that takes to
ix 'ix
ix'ix
' , 1, 2,...,i iH i n x xWe know there existing an H satisfying Coordinates of and are known, ix 'ix we need to find Hwhere H is a homography matrix
11 12 13
21 22 23
31 32 33
a a aH a a a
a a a
It has 8 degrees of freedom
![Page 60: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/60.jpg)
Lin ZHANG, SSE, 2016
Homography Estimation
11 12 13
21 22 23
31 32 33 1
a a acu xcv a a a yc a a a
11 12 13
21 22 23
31 32 33
a x a y a cua x a y a cva x a y a c
11 12 13
31 32 33
21 22 23
31 32 33
a x a y a ua x a y aa x a y a va x a y a
4 point‐correspondence pairs can uniquely determine a homography matrix since each correspondence pair solves two degrees of freedom
![Page 61: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/61.jpg)
Lin ZHANG, SSE, 2016
Homography Estimation4 point‐correspondence pairs can uniquely determine a homography matrix since each correspondence pair solves two degrees of freedom 11
12
13
21
22
23
31
32
33
1 0 0 00
0 0 0 1
aaaa
x y ux uy ua
x y vx vy vaaaa
Thus, four correspondence pairs generate 8 equations
![Page 62: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/62.jpg)
Lin ZHANG, SSE, 2016
Homography Estimation4 point‐correspondence pairs can uniquely determine a homography matrix since each correspondence pair solves two degrees of freedom
0 (1)A x
8 9 9 1
Normally, ; thus (1) has 1 (9‐8) solution vector in its solution space
( ) 8Rank A
![Page 63: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/63.jpg)
Lin ZHANG, SSE, 2016
Homography Estimation• How about the case when there are more than 4 correspondence pairs?• Use the LS method (for homogeneous case) to solve the model
![Page 64: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/64.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• Homograpy Estimation• What is Camera Calibration• Single Camera Calibration• Bird’s‐eye‐view Generation
![Page 65: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/65.jpg)
Lin ZHANG, SSE, 2016
• Camera calibration is a necessary step in 3D computer vision in order to extract metric information from 2D images
• It estimates the parameters of a lens and image sensor of the camera; you can use these parameters to correct for lens distortion, measure the size of an object in world units, or determine the location of the camera in the scene
• These tasks are used in applications such as machine vision to detect and measure objects. They are also used in robotics, for navigation systems, and 3‐D scene reconstruction
What is camera calibration?
![Page 66: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/66.jpg)
Lin ZHANG, SSE, 2016
What is camera calibration?
![Page 67: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/67.jpg)
Lin ZHANG, SSE, 2016
• Camera parameters include• Intrinsics• Extrinsics• Distortion coefficients
What is camera calibration?
![Page 68: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/68.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• Homograpy Estimation• What is Camera Calibration• Single Camera Calibration• Bird’s‐eye‐view Generation
![Page 69: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/69.jpg)
Lin ZHANG, SSE, 2016
• For simplicity, usually we use a pinhole camera model
Single Camera Calibration
![Page 70: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/70.jpg)
Lin ZHANG, SSE, 2016
• To model the image formation process, 4 coordinate systems are required• World coordinate system (3D space)• Camera coordinate system (3D space)• Retinal coordinate system (2D space)• Pixel coordinate system (2D space)
Single Camera Calibration
![Page 71: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/71.jpg)
Lin ZHANG, SSE, 2016
• To model the image formation process, 4 coordinate systems are required
Single Camera Calibration
![Page 72: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/72.jpg)
Lin ZHANG, SSE, 2016
• From the world CS to the camera CS
Single Camera Calibration
, , Tw w wX Y Z is a 3D point represented in the WCS
In the camera CS, it is represented as,
R tc w
c w
c w
X XY YZ Z
a rotation matrix (orthogonal)
3 3
a translation vector3 1
![Page 73: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/73.jpg)
Lin ZHANG, SSE, 2016
• From the world CS to the camera CS
Single Camera Calibration
, , Tw w wX Y Z is a 3D point represented in the WCS
In the camera CS, it is represented as,
R tc w
c w
c w
X XY YZ Z
1
1 1
R t0
c w
c wT
c w
X XY YZ Z
Homogeneous form
(1)
![Page 74: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/74.jpg)
Lin ZHANG, SSE, 2016
• From the camera CS to the retinal CS
Single Camera Calibration
We can use a pin‐hole model to represent the mapping from the camera CS to the retinal CS
![Page 75: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/75.jpg)
Lin ZHANG, SSE, 2016
• From the camera CS to the retinal CS
Single Camera Calibration
We can use a pin‐hole model to represent the mapping from the camera CS to the retinal CS
x
y
z
f
O is the optical center, f is the focal length, P = [Xc, Yc, Zc]T is a scene point while P’=(x, y) is its image on the retinal plane
![Page 76: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/76.jpg)
Lin ZHANG, SSE, 2016
• From the camera CS to the retinal CS
Single Camera Calibration
We can use a pin‐hole model to represent the mapping from the camera CS to the retinal CS
cc
cc
cc
c
XfX
ZxY
y YfZ
Z
0 0 00 0 0
1 0 0 1 01
c
cc
c
Xx f
YZ y f
Z
Homogeneous form
(2)
![Page 77: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/77.jpg)
Lin ZHANG, SSE, 2016
• From the retinal CS to the pixel CS
Single Camera Calibration
The unit for retinal CS (x-y) is physical unit (e.g., mm, cm) while the unit for pixel CS (u-v) is pixel
One pixel represents dx physical units along the x‐axis and represents dy physical units along the y‐axis; the image of the optical center is (u0, v0)
0
0
1 0
101 1
0 0 1
udxu x
v v ydy
![Page 78: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/78.jpg)
Lin ZHANG, SSE, 2016
• From the retinal CS to the pixel CS
Single Camera Calibration
If the two axis of the pixel plane are not perpendicular to each other, another parameter s is introduced to represent the skewness of the two axis
0
0
1 0
101 1
0 0 1
udxu x
v v ydy
0
0
1
101 1
0 0 1
s udxu x
v v ydy
(3)
![Page 79: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/79.jpg)
Lin ZHANG, SSE, 2016
Single Camera CalibrationFrom Eqs.1~3, we can have
0 0
0 0
0 0
0
1 00 0 0
10 . 0 0 0 0 0 .1 0 0 1 0
0 0 1 0 0 1 01 1
0 00 0 .0 0 1 0
1
c c
c cc
c c
c
c
c
fs u sf uX Xdx dxu fY YfZ v v f vZ Zdy dy
Xu u
Yv
Z
0
0 00 0 . 0 .1
0 0 1 0 0 0 11 1
R tR t
0
w w
w wT
w w
X Xu
Y Yv v
Z Z
Intrinsic Extrinsic
![Page 80: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/80.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
0
0[ ] 0 [ ]1 0 0 1
1 1
K R t R t
w w
w wc
w w
X Xuu
Y YZ v v
Z Z
Intrinsic Extrinsic
, the coordinates of the principal point in the image plane 0 0,u v, the scale factors in image u and v axes and
, describing the skewness of the two image axesR and t determines the rigid transformation from the world coordinate system to the camera coordinate system Altogether, there are 11 parameters to be determined
![Page 81: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/81.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• To accurately represent an ideal camera, the camera model can include the radial and tangential lens distortion• Radial distortion occurs when light rays bend more near the edges of a lens than they do at its optical center; the smaller the lens, the greater the distortion
![Page 82: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/82.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• To accurately represent an ideal camera, the camera model can include the radial and tangential lens distortion• Radial distortion occurs when light rays bend more near the edges of a lens than they do at its optical center; the smaller the lens, the greater the distortion
2 4 61 2 3
2 4 61 2 3
1
1
distorted
distorted
u u k r k r k r
v v k r k r k r
where 2 2 2r u v are the radial distortion coefficients of the lens 1 2 3, ,k k k
![Page 83: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/83.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• To accurately represent an ideal camera, the camera model can include the radial and tangential lens distortion
• Tangential distortion occurs when the lens and the image plane are not parallel
![Page 84: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/84.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• To accurately represent an ideal camera, the camera model can include the radial and tangential lens distortion
• Tangential distortion occurs when the lens and the image plane are not parallel
2 21 2
2 22 1
+ 2 2
2 2
distorted
distorted
u u uv r u
v v uv r v
are the tangential distortion coefficients of the lens 1 2,
![Page 85: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/85.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• The purpose of the camera calibration is to determine the values for the extrinsics, intrinsics, and distortion coefficients
• How to do?• Zhengyou Zhang’s method[1] is a commonly used modern approach
• A calibration board is needed; several images of the board need to be captured; based on the correspondence pairs (pixel coordinate and world coordinate of a feature point), equation systems can be obtained; by solving the equation systems, parameters can be determined
[1] Z. Zhang, A flexible new technique for camera calibration, IEEE Trans. Pattern Analysis and Machine Intelligence, 2000
![Page 86: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/86.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
Calibration board
![Page 87: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/87.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
A set of Calibration board images (50~60)
![Page 88: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/88.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• Matlab provides a “Camera Calibrator”• Straightforward to use• However, based on my experience, it is not as accurate as the routine provided in openCV3.0, especially for large FOV cameras (such as fisheye camera); thus, for some accuracy critical applications, I recommend to use the openCV function, though a little more complicated
• It exports “cameraParams” as the calibration result
![Page 89: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/89.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
![Page 90: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/90.jpg)
Lin ZHANG, SSE, 2016
Single Camera Calibration
• For our purpose (measuring geometric metrics of a planar object), we use the camera parameters to undistort the image
• The essence of this step is to make sure the transformation from a physical plane to the image plane can be represented by a linear projective matrix; or in other words, a straight line should be mapped to a straight line
![Page 91: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/91.jpg)
Lin ZHANG, SSE, 2016
Single Camera CalibrationOriginal image
Undistorted image
![Page 92: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/92.jpg)
Lin ZHANG, SSE, 2016
Contents
• Foundations of Projective Geometry• Matrix Differentiation• Lagrange Multiplier• Least‐squares for Linear Systems• Homograpy Estimation• What is Camera Calibration• Single Camera Calibration• Bird’s‐eye‐view Generation
![Page 93: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/93.jpg)
Lin ZHANG, SSE, 2016
• Our task is to measure the geometric properties of objects on a plane (e.g., conveyor belt)
• Such a problem can be solved if we have its bird‐view image; bird’s‐eye‐view is easy for object detection and measurement
Bird’s‐eye‐view Generation
![Page 94: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/94.jpg)
Lin ZHANG, SSE, 2016
• Three coordinate systems are required• Bird’s‐eye‐view image coordinate system• World coordinate system• Undistorted image coordinate system
X
Y
X
Y
X
Y
Bird’s‐eye‐view image WCS Undistorted image
Similarity Projective
Bird’s‐eye‐view Generation
![Page 95: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/95.jpg)
Lin ZHANG, SSE, 2016
Suppose that the transformation matrix from bird’s‐eye‐view to WCS is and the transformation matrix from WCS to the undistorted image is
B WP
W IP
Then, given a position on bird’s‐eye‐view, we can get its corresponding position in the undistorted image as
, ,1 TB Bx y
1x
B
I W I B W B
xP P y
• Basic idea for bird’s‐eye‐view generation
Then, the intensity of the pixel can be determined using some interpolation technique based on the neighborhood around on the undistorted image
, ,1 TB Bx y
xI
Bird’s‐eye‐view Generation
![Page 96: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/96.jpg)
Lin ZHANG, SSE, 2016
Suppose that the transformation matrix from bird’s‐eye‐view to WCS is and the transformation matrix from WCS to the undistorted image is
B WP
W IP
• Basic idea for bird’s‐eye‐view generation
The key problem is how to obtain and ?B WP W IP
Bird’s‐eye‐view Generation
![Page 97: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/97.jpg)
Lin ZHANG, SSE, 2016
• Determine B WP
X
Y
X
Y
H(mm)
M (pixels)
N (pixels)
Bird’s‐eye‐view Generation
![Page 98: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/98.jpg)
Lin ZHANG, SSE, 2016
• Determine B WP
02
02
1 11 0 0 1
W B B
W B B W B
H HNM Mx x x
H Hy y P yM
For a point on bird’s‐eye‐view, the corresponding point on the world coordinate system is,
, ,1 TB Bx y
Please verify!!
Bird’s‐eye‐view Generation
![Page 99: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/99.jpg)
Lin ZHANG, SSE, 2016
• Determine W IP
The physical plane (in WCS) and the undistorted image plane can be linked via a homography matrix W IP
x xI W I WP
If we know a set of correspondence pairs , 1,x x N
Ii Wi i
W IP can be estimated using the least‐square method
Bird’s‐eye‐view Generation
![Page 100: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/100.jpg)
Lin ZHANG, SSE, 2016
• Determine W IP
A set of point correspondence pairs; for each pair, we know its coordinate on the undistorted image plane and its coordinate in the WCS
Bird’s‐eye‐view Generation
![Page 101: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/101.jpg)
Lin ZHANG, SSE, 2016
When and are known, the bird’s‐eye‐view can be generated via,
W IP B WP
1 1x
B B
I W I B W B B I B
x xP P y P y
Bird’s‐eye‐view Generation
![Page 102: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/102.jpg)
Lin ZHANG, SSE, 2016
Bird’s‐eye‐view Generation
Original image Bird’s‐eye‐view
![Page 103: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/103.jpg)
Lin ZHANG, SSE, 2016
Bird‐view GenerationAnother example
Original fish‐eye image Undistorted image
![Page 104: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/104.jpg)
Lin ZHANG, SSE, 2016
Bird‐view GenerationAnother example
Bird’s‐eye‐viewOriginal fish‐eye image
![Page 105: Lecture 7 Measurement Using a Single Camerasse.tongji.edu.cn/linzhang/DIP/slides/Lecture 07...Lin ZHANG, SSE, 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School](https://reader033.fdocuments.in/reader033/viewer/2022041621/5e3ed938a60d191b36445375/html5/thumbnails/105.jpg)
Lin ZHANG, SSE, 2016
Thanks for your attention