Camera Calibration

11Introduction Problem: Estimate cameras extrinsic & intrinsic parameters. Method: Use image(s) of known scene. Tools: Geometric camera models. SVD and constrained least-squares.

Algorithm for camera calibration proposed by Zhengyou Zhang in A Flexible New Technique for Camera Calibration22Camera CalibrationFeatureExtractionPerspectiveEquationsFeatureExtraction33Perspective Projection, Remember?

Image formation geometry fora thin convex lensThe central-projection model.

From Sebastian Thrun and Jana Kosecka44Intrinsic ParametersZXO

Image center(ox, oy)55Intrinsic Camera ParametersIntrinsic Parameters:Focal Length fPixel size sx , syImage center ox , oy(Nonlinear radial distortion coefficients k1 , k2)Calibration = Determine the intrinsic parameters of a camera6Extrinsic Parameters

Pc=R(Pw-T)Translation followed by rotation7Coordinate Frames

XwYwZwWorld Coordinate FrameXcYcZcCamera Coordinate Frame

Image Coordinate FramePixel CoordinatesIntrinsicParametersExtrinsicParameters8Why Calibrate?In general the principal point is not at the centre of the photosite array.

The focal length of a lens correct only if the lens is focused at infinity.

It is also common experience that the intrinsic parameters change if a lenses is detached and reattached, or adjusted for focus or aperture.

The only intrinsic parameters that it may be possible to obtain are the photosite dimensions w and h from the sensor manufacturers data sheet.9Perspective Camera ModelStep 1: Transform into camera coordinates

Step 2: Transform into image coordinates

From Sebastian Thrun and Jana Kosecka1010Image and Camera FramesXimageYimageXcameraYcamera(xim, yim)(ox,oy)Zcamera1111Geometric Model

Transformation fromWorld to Camera Frame.Perspective projection (f, R, T)Point in Camera FrameTransformation from Imageto Camera Frame. (ox,oy,sx,sy)No distortion!3D Point inCameraCoordinateFramex=y=YcTZcTXcTZcT1212Camera Calibration: IssuesWhich parameters need to be estimated.Focal length, image center, aspect ratioRadial distortionsWhat kind of accuracy is needed.Application dependentWhat kind of calibration object is used.One plane, many planesComplicated three dimensional object1313Basic Equation - NotationAssumption : Camera is pinhole

2D point 3D point

S - Arbitrary scale factor[R t] - Extrinsic parametersA - Camera intrinsic matrix

- coordinate of the principle points

- scale factors in image u and v axes

- skewness of the two image axes

1414Basic Equation - Homography between the model plane and its imageAssumption : the model plane is on Z = 0 of the world coordinate system

From 1 ->

Therefore, a model point M and its image m is related by a homography H:15Basic Equation - Constraints on the intrinsic parametersGiven an image of the model plane, H =From 2 ->Using the knowledge that r1 and r2 are orthonormal, we have

16These are the two basic constraints on the intrinsic parameters, given one homography. Because ahomography has 8 degrees of freedom and there are 6 extrinsic parameters (3 for rotation and 3 fortranslation), we can only obtain 2 constraints on the intrinsic parameters.

16Solving Camera CalibrationClosed form solution

LetNote that B is symmetric, defined by a 6D vector

1717Solving Camera CalibrationClosed form solutionLet the ith column vector of H be hi = [hi1; hi2; hi3]T . Then, we havewith

1818Solving Camera CalibrationClosed form solutionTherefore, the two fundamental constraints (3) and (4), from a given homography, can be rewritten as 2 homogeneous equations in b:

If n images of the model plane are observed, by stacking n such equations as (8) we have,

19where V is a 2n6 matrix. If n 3, we will have in general a unique solution b defined up to a scalefactor. If n = 2, we can impose the skewless constraint = 0, i.e., [0; 1; 0; 0; 0; 0]b = 0, which isadded as an additional equation to (9). (If n = 1, we can only solve two camera intrinsic parameters,e.g., and , assuming u0 and v0 are known (e.g., at the image center) and = 0, and that is indeedwhat we did in [19] for head pose determination based on the fact that eyes and mouth are reasonablycoplanar.) The solution to (9) is well known as the eigenvector of VTV associated with the smallesteigenvalue (equivalently, the right singular vector of V associated with the smallest singular value).19Solving Camera CalibrationClosed form solutionThe solution to (9) is well known as the eigenvector of VTV associated with the smallest eigenvalue.Once b is estimated, we can compute all camera intrinsic matrix A.Once A is known, the extrinsic parameters for each image is readily computed. From (2), we have

2020Maximum likelihood estimationWe are given n images of a model plane and there are m points on the model plane.

Assume that the image points are corrupted by independent and identically distributed noise. The maximum likelihood estimate can be obtained by minimizing the following functional:

21Dealing with radial distortionLet (u; v) be the ideal (non observable distortion-free) pixel image coordinates, and (u; v) the corresponding real observed image coordinates.

=d =

22Up to now, we have not considered lens distortion of a camera.

However, a desktop camera usually exhibits significant lens distortion, especially radial distortion.

As the radial distortion is expected to be small, onewould expect to estimate the other five intrinsic parameters, using the technique described in Sect. 3.2,reasonable well by simply ignoring distortion. One strategy is then to estimate k1 and k2 after havingestimated the other parameters, which will give us the ideal pixel coordinates (u; v). Then, from (11)and (12), we have two equations for each point in each image:22Complete Maximum Likelihood EstimationA natural extension to (10) is then to estimate the complete set ofparameters by minimizing the following functional,

23The recommended calibration procedure by

By Zhengyou Zhang in A Flexible New Technique for Camera CalibrationPrint a pattern and attach it to a planar surface;

2. Take a few images of the model plane under different orientations by moving either the plane or the camera;

3. Detect the feature points in the images;

4. Estimate the five intrinsic parameters and all the extrinsic parameters using the closed-form solution as described;

5. Estimate the coefficients of the radial distortion by solving the linear least-squares (13);

6. Refine all parameters by minimizing (14).24Known planar surfaces

25DemoZHANGS CAMERA CALIBRATION(zhang2D_with_demo)

26ReferenceZhang, Zhengyou. "A flexible new technique for camera calibration."Pattern Analysis and Machine Intelligence, IEEE Transactions on22, no. 11 (2000): 1330-1334.

P.I. Corke, Robotics, Vision & Control,Springer 2011, ISBN 978-3-642-20143-1.

http://www.vision.caltech.edu/bouguetj/calib_doc/ : Camera Calibration Toolbox for Matlab ,Sebastian Thrun and Jana Kosecka

2727