Calibration. Camera Calibration GeometricGeometric – Intrinsics: Focal length, principal point,...
-
Upload
karin-harvey -
Category
Documents
-
view
215 -
download
0
Transcript of Calibration. Camera Calibration GeometricGeometric – Intrinsics: Focal length, principal point,...
CalibrationCalibration
Camera CalibrationCamera Calibration
• GeometricGeometric– Intrinsics: Focal length, principal point, distortionIntrinsics: Focal length, principal point, distortion
– Extrinsics: Position, orientationExtrinsics: Position, orientation
• RadiometricRadiometric– Mapping between pixel value and scene radianceMapping between pixel value and scene radiance
– Can be nonlinear at a pixel (gamma, etc.)Can be nonlinear at a pixel (gamma, etc.)
– Can vary between pixels (vignetting, cosCan vary between pixels (vignetting, cos44, etc.), etc.)
– Dynamic range (calibrate shutter speed, etc.)Dynamic range (calibrate shutter speed, etc.)
Geometric Calibration IssuesGeometric Calibration Issues
• Camera ModelCamera Model– Orthogonal axes?Orthogonal axes?
– Square pixels?Square pixels?
– Distortion?Distortion?
• Calibration TargetCalibration Target– Known 3D points, noncoplanarKnown 3D points, noncoplanar
– Known 3D points, coplanarKnown 3D points, coplanar
– Unknown 3D points (structure from motion)Unknown 3D points (structure from motion)
– Other features (e.g., known straight lines)Other features (e.g., known straight lines)
Geometric Calibration IssuesGeometric Calibration Issues
• Optimization methodOptimization method– Depends on camera model, available dataDepends on camera model, available data
– Linear vs. nonlinear modelLinear vs. nonlinear model
– Closed form vs. iterativeClosed form vs. iterative
– Intrinsics only vs. extrinsics only vs. bothIntrinsics only vs. extrinsics only vs. both
– Need initial guess?Need initial guess?
Caveat - 2D Coordinate SystemsCaveat - 2D Coordinate Systems
• yy axis up axis up vs. vs. yy axis down axis down
• Origin at center vs. Origin at center vs. cornercorner
• Will often write (Will often write (uu,, vv) for image ) for image coordinatescoordinates
uu
vv
uu
uu
vvvv
Camera Calibration – Example 1Camera Calibration – Example 1
• Given:Given:– 3D 3D 2D correspondences 2D correspondences
– General perspective camera model (no distortion)General perspective camera model (no distortion)
• Don’t care about “Don’t care about “zz” after transformation” after transformation
• Homogeneous scale ambiguity Homogeneous scale ambiguity 11 free 11 free parametersparameters
lkzjyix
hgzfyexlkzjyix
dczbyax
z
y
x
lkji
hgfe
dcbashomogeneou
divide
1
lkzjyix
hgzfyexlkzjyix
dczbyax
z
y
x
lkji
hgfe
dcbashomogeneou
divide
1
Camera Calibration – Example 1Camera Calibration – Example 1
• Write equations:Write equations:
1
111
111
1111
111
vlkzjyix
hgzfyex
ulkzjyix
dczbyax
1
111
111
1111
111
vlkzjyix
hgzfyex
ulkzjyix
dczbyax
Camera Calibration – Example 1Camera Calibration – Example 1
• Linear equationLinear equation
• Overconstrained (more equations than unknowns)Overconstrained (more equations than unknowns)
• Underconstrained (rank deficient matrix – any Underconstrained (rank deficient matrix – any multiple of a solution, including 0, is also a multiple of a solution, including 0, is also a solution)solution)
0
l
c
b
a
uzuyuxuzyx
uzuyuxuzyx
uzuyuxvzyx
uzuyuxuzyx
2222222222
2222222222
1111111111
1111111111
10000
00001
10000
00001
0
l
c
b
a
uzuyuxuzyx
uzuyuxuzyx
uzuyuxvzyx
uzuyuxuzyx
2222222222
2222222222
1111111111
1111111111
10000
00001
10000
00001
Camera Calibration – Example 1Camera Calibration – Example 1
• Standard linear least squares methods Standard linear least squares methods forforAx=0 will give the solution x=0Ax=0 will give the solution x=0
• Instead, look for a solution with |x|= 1Instead, look for a solution with |x|= 1
• That is, minimize |Ax|That is, minimize |Ax|22 subject to |x| subject to |x|22=1=1
Camera Calibration – Example 1Camera Calibration – Example 1
• Minimize |Ax|Minimize |Ax|22 subject to |x| subject to |x|22=1=1
• |Ax||Ax|22 = (Ax) = (Ax)TT(Ax) = (x(Ax) = (xTTAATT)(Ax) = x)(Ax) = xTT(A(ATTA)xA)x
• Expand x in terms of eigenvectors of Expand x in terms of eigenvectors of AATTA:A:
x = x = 11ee11+ + 22ee22+…+…
x xTT(A(ATTA)x = A)x = 111122++2222
22+…+…
|x||x|22 = = 1122++22
22+…+…
Camera Calibration – Example 1Camera Calibration – Example 1
• To minimizeTo minimize1111
22++222222+…+…
subject tosubject to 11
22++2222+… = 1+… = 1
set set minmin= 1 and all other = 1 and all other ii=0=0
• Thus, least squares solution is Thus, least squares solution is eigenvector corresponding to minimum eigenvector corresponding to minimum (non-zero) eigenvalue of A(non-zero) eigenvalue of ATTAA
Camera Calibration – Example 2Camera Calibration – Example 2
• Incorporating additional constraints intoIncorporating additional constraints intocamera modelcamera model– No shear (No shear (uu, , vv axes orthogonal) axes orthogonal)
– Square pixelsSquare pixels
– etc.etc.
• Doing minimization in image spaceDoing minimization in image space
• All of these impose All of these impose nonlinearnonlinear constraints constraints ononcamera parameterscamera parameters
Camera Calibration – Example 2Camera Calibration – Example 2
• Option 1: nonlinear least squaresOption 1: nonlinear least squares– Usually “gradient descent” techniquesUsually “gradient descent” techniques
– e.g. Levenberg-Marquardte.g. Levenberg-Marquardt
• Option 2: solve for general perspective Option 2: solve for general perspective model, find closest solution that model, find closest solution that satisfies constraintssatisfies constraints– Use closed-form solution as initial guess forUse closed-form solution as initial guess for
iterative minimizationiterative minimization
Radial DistortionRadial Distortion
• Radial distortion can not be representedRadial distortion can not be representedby matrixby matrix
• ((ccuu, , ccvv) is image center,) is image center,
uu**imgimg= = uuimgimg– – ccuu, , vv**
imgimg= = vvimgimg– – ccvv,,
is first-order radial distortion coefficient is first-order radial distortion coefficient
)(1
)(12*2**
2*2**
imgimgimgvimg
imgimgimguimg
vuvcv
vuucu
)(1
)(12*2**
2*2**
imgimgimgvimg
imgimgimguimg
vuvcv
vuucu
Camera Calibration – Example 3Camera Calibration – Example 3
• Incorporating radial distortionIncorporating radial distortion
• Option 1:Option 1:– Find distortion first (e.g., straight lines inFind distortion first (e.g., straight lines in
calibration target)calibration target)
– Warp image to eliminate distortionWarp image to eliminate distortion
– Run (simpler) perspective calibrationRun (simpler) perspective calibration
• Option 2: nonlinear least squaresOption 2: nonlinear least squares
Calibration TargetsCalibration Targets
• Full 3D (nonplanar)Full 3D (nonplanar)– Can calibrate with one imageCan calibrate with one image
– Difficult to constructDifficult to construct
• 2D (planar)2D (planar)– Can be made more accuracteCan be made more accuracte
– Need multiple viewsNeed multiple views
– Better constrained than full SFM problemBetter constrained than full SFM problem
Calibration TargetsCalibration Targets
• Identification of featuresIdentification of features– ManualManual
– Regular array, manually seededRegular array, manually seeded
– Regular array, automatically seededRegular array, automatically seeded
– Color coding, patterns, etc.Color coding, patterns, etc.
• Subpixel estimation of locationsSubpixel estimation of locations– Circle centersCircle centers
– Checkerboard cornersCheckerboard corners
Calibration Target w. CirclesCalibration Target w. Circles
3D Target w. Circles3D Target w. Circles
Planar Checkerboard TargetPlanar Checkerboard Target
[Bouguet][Bouguet]
Coded CirclesCoded Circles
[Marschner et al.][Marschner et al.]
Concentric Coded CirclesConcentric Coded Circles
[Gortler et al.][Gortler et al.]
Color Coded CirclesColor Coded Circles
[Culbertson][Culbertson]
Calibrating ProjectorCalibrating Projector
• Calibrate cameraCalibrate camera
• Project pattern onto a known objectProject pattern onto a known object(usually plane)(usually plane)– Can use time-coded structured lightCan use time-coded structured light
• Form (Form (uuprojproj, , vvprojproj, , xx, , yy, , zz) tuples) tuples
• Use regular camera calibration codeUse regular camera calibration code
• Typically lots of keystoning relative to Typically lots of keystoning relative to camerascameras
Multi-Camera GeometryMulti-Camera Geometry
• Epipolar geometry – relationship Epipolar geometry – relationship between observed positions of points in between observed positions of points in multiple camerasmultiple cameras
• Assume:Assume:– 2 cameras2 cameras
– Known intrinsics and extrinsicsKnown intrinsics and extrinsics
Epipolar GeometryEpipolar Geometry
PP
CC11 CC22
pp22pp11
Epipolar GeometryEpipolar Geometry
PP
CC11 CC22
pp22pp11
ll22
Epipolar GeometryEpipolar Geometry
PP
CC11 CC22
pp22pp11
ll22
Epipolar lineEpipolar line
EpipolesEpipoles
Epipolar GeometryEpipolar Geometry
• Goal: derive equation for lGoal: derive equation for l22
• Observation: P, CObservation: P, C11, C, C22 determine a plane determine a plane
PP
CC11 CC22
pp22pp11
ll22
Epipolar GeometryEpipolar Geometry
• Work in coordinate frame of CWork in coordinate frame of C11
• Normal of plane is T Normal of plane is T Rp Rp22, where T is , where T is
relative translation, R is relative rotationrelative translation, R is relative rotation
PP
CC11 CC22
pp22pp11
ll22
Epipolar GeometryEpipolar Geometry
• pp11 is perpendicular to this normal: is perpendicular to this normal:
p p11 (T (T Rp Rp22) = 0) = 0
PP
CC11 CC22
pp22pp11
ll22
Epipolar GeometryEpipolar Geometry
• Write cross product as matrix Write cross product as matrix multiplicationmultiplication
PP
CC11 CC22
pp22pp11
ll22
0
0
0
, **
xy
xz
yz
TT
TT
TT
xxT TT
0
0
0
, **
xy
xz
yz
TT
TT
TT
xxT TT
Epipolar GeometryEpipolar Geometry
• pp11 T T* * R pR p22 = 0 = 0 p p11TT EE pp22 = 0 = 0
• E is the E is the essential matrixessential matrix
PP
CC11 CC22
pp22pp11
ll22
Essential MatrixEssential Matrix
• E depends only on camera geometryE depends only on camera geometry
• Given E, can derive equation for line lGiven E, can derive equation for line l22
PP
CC11 CC22
pp22pp11
ll22
Fundamental MatrixFundamental Matrix
• Can define Can define fundamental matrixfundamental matrix F F analogously, operating on pixel analogously, operating on pixel coordinates instead of camera coordinates instead of camera coordinatescoordinates
uu11TT FF uu22 = 0 = 0
• Advantage: can sometimes estimate F Advantage: can sometimes estimate F without knowing camera calibrationwithout knowing camera calibration