Practical Camera Auto-Calibration Based on Object Appearance and Motion for Traffic

Scene Visual Surveillance

Zhaoxiang Zhang, Min Li, Kaiqi Huang and Tieniu Tan

National Laboratory of Pattern Recognition,Institute of Automation, Chinese Academy of Sciences

Reporter: Chia-Hao HsiehDate: 2009/11/3

CVPR 2008

Outline

• Introduction• Methods– Extract foreground– Estimate vanishing points– Auto-calibration

• Experimental results

Introduction

• Recover intrinsic and extrinsic parameters of cameras– Based on appearance and motion of objects– Measure the camera height only

Practical Camera Auto-Calibration Based on Object Appearance and Motion for Traffic Scene Visual Surveillance

Motion Detection

• Disadvantages of Gaussian Mixture Model – Fast illumination changes– Shadows

• Methods deal with the disadvantages– Model each pixel as the product of irradiance

component and reflectance component– Model each reflectance component as a mixture

of Gaussian

Outline

• Introduction• Methods– Extract foreground– Estimate vanishing points– Auto-calibration

• Experimental results

Vanishing Points Estimation

• Helpful general properties– Moving objects move on the ground plane– Vehicles run along the straight roadway– Vehicles are rich in line segments along two

orientations– Pedestrians walk with their trunks perpendicular

to the ground plane

Vanishing Points Estimation

• Coarse Moving Object Classification– Two directions• Velocity direction • Main axis direction

– Difference of direction• K-Mean clustering• Thresholding

θ < 5°

θ > 20°

from moment analysis of silhouette

Vanishing Points Estimation

• Line Equations EstimationVehicles are rich in line segments along two orientations

– Histogram of Orientated Gradient

Vanishing Points Estimation

• Intersection Estimation– Least square– Levenberg-Marquardt– RANSAC

– But… more and more frames• Voting strategy

– Each point lying on every line generatesa Gaussian impulse in the voting space

Vehicles: 2 vanishing pointsPedestrians: 1 vanishing point

Outline

• Introduction• Methods– Extract foreground– Estimate vanishing points– Auto-calibration

• Experimental results

Camera Calibration

• Recovery of K and R– 3 vanishing points 3 orthogonal directions of

world coordinate system

3 DOFAssume αu = αv = fs = 0

Assume (u0, v0) is on the middle of image plane

1 DO

F

3 DO

F

3 DO

F

Camera Calibration

K and λi solvedSolve R

Camera Calibration• Recovery of T– Choose one arbitrary reference point (u4,v4) from image

plane to correspond to the origin of the world coordinate system

Camera height H is measured

The optical center of the camera lies on the z = H plane

Propose a method of complete calibration of surveillance scenes with three estimated orthogonal vanishing points and the measured camera height H

Experimental Results

720 × 576 (u1, v1) = (−217, 70)(u2, v2) = (1806, 31)(u3, v3) = (427, 4906)

αu = αv = 884,(u0, v0) = (336, 226)

vary in a small range less than 2%

Conclusions

• Practical camera auto-calibration method for traffic scene surveillance

• Completely recover both intrinsic and extrinsic parameters of cameras– Only the camera height H measured– Based on appearance and motion of moving

objects in videos• Accuracy and practicability

Thank you!