Stereoscopic LightStripe Scanning:Interference Rejection,Error Minimizationand Calibration

By: Geoffrey Taylor

Lindsay Kleeman

Presented by: Ali Agha

April 13th, 2009

Motivation

Measuring arbitrary scenes in ambient indoor light (Purpose: Visual Servoing for a Humanoid Robot)

Addresses the problem of rejecting interference due to secondary specular reflections, cross-talk and other mechanisms in an active light stripe scanner.

Motivation

Basic Operation

Color cameras capture stereo images of the stripe at 384 × 288 pixel

Frame rate (25 Hz) on the 2.2 GHz dual Xeon host PC.

System Model Encoder measurement

Problem StatementGiven the laser plane position and the measurements Lx, Rx, Rx�, one of the left/right candidate pairs, (Lx,Rx) or (Lx, Rx�), must be chosen as representing stereo measurementsof the primary reflection.

The measurements shouldthen be combined to estimate the position of the ideal projection

Previous work In Trucco, et al. (1994) and

Nakano, et al. (1988), laser stripe measurements are validated by applying a fixed threshold to the difference between corresponding single-camera reconstructions

Such a comparison requires a uniform reconstruction error over all depths, which this figure illustrates is clearly not the case.

General SolutionGiven is the optimal reconstruction

The Plücker matrix L describing the back-projection line is

The intersection X of the light plane and L is

Minimize

S.t.

Unconstrained Version

General SolutionFinally, the ideal projection corresponding to is obtainedby projecting onto the left image plane:

And, the error function becomes

By some simplifications:

Special Case: Rectilinear Stereo and Pin-Hole Cameras With and

the image plane error E can be

expressed as a function of a single

unknown

Validation

determining which pair of measurements correspond to the primary reflection

1) Light plane parameters α, β, and γ are calculated from e and the system parameters

2)

3)

4) the optimal reconstruction is finally calculated

Laser Plane Error

The above solution assumes that the parameters of the laser plane are known exactly.

In practice, the encoder measurements are noisy Let and , i = 1 . . . n, represent valid corresponding

measurements of the laser stripe on the n scanlines in a frame.

Levenberg–Marquardt (LM) algorithm for minimization The optimal correspondences and encoder count are

calculated recursively.

Additional Constraints

1) stripe candidates must be moving features It has little effect on cross-talk or reflections.

2) that valid measurements only occur within a subregion of the left and right image planes, depending on the angle of the light plane.

Active CalibrationUnknown parameters in the model of the light stripe scanner

The validation problem is approximated by recording only the brightest pair of features per scanline.

Let and , represent the brightest corresponding features on nj scanlines of t captured frames, and let ej represent the measured encoder value for each frame.

where

Active Calibration

where

implemented numerically using LM minimization

The system parameters and encoder values are then sequentially refined in an iterative process.

Initial estimate

The calibration technique presented here is practical, fast, and accurate. The method does not require accurate knowledge of camera parameters b and f.

Implementation

Output of the scanner is a 384 × 288 element range map

The shaft encoder and stereo images are recorded at regular 40 ms intervals (25 Hz PAL frame rate).

A complete scan requires approximately 384 processed frames (15 s).

Implementation: Light Stripe Measurement

Laser stripe extraction is performed using: intensity data only (average of the color channels) motion of the stripe (by subtracting the intensity values in

consecutive frames) predicted sub-region of the image.

The intensity profile on each scanline is then examined to locate candidate stripe measurements.

Implementation: Range Data Post-Processing 1) Despite robust scanning, the raw range map

may still contain outliers Thresholding: the minimum distance between each 3D point

and its eight neighbors should be less than 10 mm

2) Holes fills these gaps with interpolated depth data. The distance between the bracketing points must be less

than 30 mm

3) Finally, a color image is registered with the range map.

Implementation: Range Data Post-Processing

Experimental Results

A mirror behind the objects simulates the effect of cross-talk and reflections.

Experimental Results

output of the single-camera scanner

phantom surfaces appear (Erroneous associations between the phantom stripe and laser plane)

Experimental Results

output of the double-camera scanner

Based on Nakano et al. (1988) and Trucco et al. (1994)

The single-camera reconstructions XL and XR are calculated independently

Discarded when |XL−XR| exceeds a fixed distance

The final reconstruction is calculated as (1/2)(XL + XR)

Experimental Results

robust scanner result

Discussion

Main limitation unsuitable for dynamic scenes Robot must remain stationary during a scan

The experimental prototype uses a red laser diode 1) Only can sense surfaces which contain a high component of red 2) laser diode could be replaced by a white light source 3) Advantages of LDs: physical compactness, low power

consumption and heat generation. 4) The light plane could be generated using a triplet of red, green,

and blue laser diodes. 5) High cost of green and blue laser diodes

Discussion surfaces with high specular and low

Lambertian reflection may appear invisible

Summary and Conclusions

Measuring arbitrary scenes in ambient indoor light

Robustly identify the light stripe in the presence of secondary reflections, cross-talk and other sources of interference.

Optimization-based formulation An image-based procedure for calibrating the

light plane parameters

Future Research Development of a multistripe scanner. Multistripe scanners have the potential to solve a

number of issues associated with single-stripe scanners:

Illuminating a target with two stripes could double the acquisition rate

Projecting the stripes from different positions reveals points that would otherwise be hidden in shadow.

single-camera multistripe systems mostly rely on color, sequences of illumination or epipolar constraints to disambiguate the stripes. However, the method proposed in this paper could allow the stripes to be uniquely identified using the same principles that provide validation for a single stripe.