Slide 1

Slide 2

Simple Calibration of Non-overlapping Cameras with a Mirror Ram Krishan Kumar 1, Adrian Ilie 1, Jan-Michael Frahm 1, Marc Pollefeys 1,2 Department of Computer Science 1 UNC Chapel Hill 2 ETH Zurich USA Switzerland & CVPR, Alaska, June 2008

Slide 3

Motivation 2 Courtesy: Microsoft Research

Slide 4

Motivation 3 Surveillance: Camera 1 Camera 2 Non-overlapping cameras

Slide 5

Motivation 3D reconstruction: 4 UrbanScape cameras: cameras with minimal overlap

Slide 6

Motivation 5 Panorama stitching Courtesy: www.ptgrey.com

Slide 7

Motivation 6 (Only 4 of 6 images shown here) Courtesy: Microsoft Research

Slide 8

Previous Work Single camera calibration Fixed 3D Geometry Tsai (1987) Plane based approach Zhang (2000) 7 Multiple images of the checker board pattern assumed at Z=0 are observed

Slide 9

Previous Work Single camera calibration Fixed 3D Geometry Tsai (1987) Plane based approach Zhang (2000) 8 Yields both internal and external camera parameters

Slide 10

Previous Work Multi-camera environment Calibration board with 3D laser pointer Kitahara et al. (2001) 9

Slide 11

Previous Work Multi-camera environment Calibration board with 3D laser pointer Kitahara et al. (2001) All cameras observe a common dominant plane and track objects moving in this plane (e.g. ground) Lee et al.(2000) 10

Slide 12

Previous Work Multi-camera environment Calibration board with 3D laser pointer Kitahara et al. (2001) All cameras observe a common dominant plane and track objects moving in this plane (e.g. ground) Lee et al.(2000) Automatic calibration yielding complete camera projections using only a laser pointer Svoboda et al. (2005) 11

Slide 13

Previous Work Multi-camera environment Calibration board with 3D laser pointer Kitahara et al. (2001) All cameras observe a common dominant plane and track objects moving in this plane (e.g. ground) Lee et al.(2000) Automatic calibration yielding complete camera projections using only a laser pointer Svoboda et al. (2005) Camera network calibration from dynamic silhouettes Sinha et al (2004) 12

Slide 14

Previous Work Multi-camera environment Calibration board with 3D laser pointer Kitahara et al. (2001) All cameras observe a common dominant plane and track objects moving in this plane (e.g. ground) Lee et al.(2000) Automatic calibration yielding complete camera projections using only a laser pointer Svoboda et al. (2005) Camera network calibration from dynamic silhouettes Sinha et al.(2004) All of these methods require an overlap in field of views (FOVs) of the cameras 13

Slide 15

Previous Work 14 Pose computation of object without direct view Sturm et al. (2006) Rely on computing the mirror plane

Slide 16

Proposed Approach mirror Calibration Pattern 15

Slide 17

Using a Planar Mirror A real camera observing point X is equivalent to a mirrored camera observing the real point X itself 16 X mirror x x C C X.. RHS to LHS Real camera pose Point on calibration pattern Mirrored camera pose

Slide 18

Proposed Approach 17. X mirror x x C C Mirrored camera pose Real camera pose

Slide 19

Proposed Approach 18 X mirror x x C C Move the mirror to a different position.

Slide 20

Proposed Approach 19. X x C C

Slide 21

Proposed Approach 20. Family of mirrored camera pose mirror X x x x x x

Slide 22

Proposed Approach 21. Family of mirrored camera pose mirror X x x x x x Reduces to Standard calibration method: Use any standard technique that give extrinsic camera parameters in addition to internal camera parameters.

Slide 23

Recovering Internal Parameters A two stage process STAGE 1: Internal calibration Image pixel x= x =>intrinsic parameters & radial distortion are the same 22 X mirror x x C C X..

Slide 24

Proposed Approach A two stage process : STAGE 2 : External camera calibration 23 Mirrored camera pose Real camera pose 23 X mirror x x C C.. X r1r1 r2r2 r3r3 r3r3 r2r2 r1r1 C-C

Slide 25

Recovery of External Parameters Mirrored camera pose Real camera pose mirror C C r1r1 r2r2 r3r3 r3r3 r2r2 r1r1 r 1 + r 1 r1r1 C-C = 0 r2r2 (C-C) T (r k + r k ) = 0 for k = 1, 2, 3 24 3 Non-linear constraints

Slide 26

Recovery of External Parameters Mirrored camera pose Real camera pose mirror C C r1r1 r2r2 r3r3 r3r3 r2r2 r1r1 r 1 + r 1 r1r1 C-C = 0 r2r2 25 3 Non-linear constraints C T r k + C T r k - C T r k - C T r k = 0 for k = 1, 2, 3 Non-linear

Slide 27

Recovery of External Parameters 26 mirror C C r1r1 r2r2 r3r3 r3r3 r2r2 r1r1 r 1 + r 1 r1r1 Each mirror position generates 3 non-linear constraints Unknowns : r 1, r 2, r 3, C (12) Equations : 3 constraints for each mirror position + 6 constraints of rotation matrix

Slide 28

Recovery of External Parameters C T r k + C T r k - C T r k - C T r k = 0 for k = 1, 2, 3 C T r k = s k (Introduced variables) linearize 27 Number of unknowns: 12 + 3 (s 1, s 2, s 3 ) ; At least 5 images are needed to solve for the camera center and rotation matrix linearly

Slide 29

Recovery of External Parameters Once we have obtained the external camera parameters, we apply bundle adjustment to minimize the reprojection error Enforce r 1, r 2, r 3 to constitute a valid rotation matrix R = [r 1 r 2 r 3 ] 28

Slide 30

Experiments Five randomly generated mirror positions which enable the camera to view the calibration pattern Error in recovered camera center vs noise level in pixel 29

Slide 31

Experiments Five randomly generated mirror positions which enable the camera to view the calibration pattern 30 Error in rotation matrix vs noise level in pixel

Slide 32

Evaluation on Real Data Experimental Setup with checkerboard pattern kept on the ground Ladybug Cameras 31

Slide 33

Evaluation on Real Data 32 Camera 1

Slide 34

Evaluation on Real Data 33 Camera 2

Slide 35

Evaluation on Real Data 34 Camera 3

Slide 36

Evaluation on Real Data 35 Camera 4

Slide 37

Evaluation on Real Data 36 Camera 5

Slide 38

Evaluation on Real Data 37 Camera 6

Slide 39

Evaluation on Real Data Top View: Initial estimate of the recovered camera poses 38

Slide 40

Evaluation on Real Data Top View : Recovered camera poses after Bundle adjustment 39

Slide 41

Evaluation on Real Data 40 35.1 cm 34.7 cm 36.2 cm 37.6 cm 37.3 cm Actual radius: 37.5 cm Result:

Slide 42

Summary Using a plane mirror to calibrate a network of camera Cameras need not see the calibration object directly Knowledge about mirror parameters is not required ! 41

Slide 43

Practical Considerations Need a sufficiently big calibration object so that they occupy a significant portion in the image Use any other calibration object and any other calibration technique which gives both intrinsic and extrinsic parameters 42

Slide 44

Acknowledgements We gratefully acknowledge the partial support of the IARPA VACE program, an NSF Career IIS 0237533 and a Packard Fellowship for Science and Technology Software at: http://www.cs.unc.edu/~ramkris/MirrorCameraCalib.html 43

Slide 45

Questions 44

Slide 46

Take Away Ideas. 45