A family of Domain-Specific Languages for specifying Civilian Missions of Multi-Robot Systems
Statistical Inference and Synthesis in the Image Domain for Mobile Robot Environment Modeling
description
Transcript of Statistical Inference and Synthesis in the Image Domain for Mobile Robot Environment Modeling
Statistical Inference and Synthesis in the Image Domain for Mobile Robot
Environment Modeling
L. Abril Torres-Méndez and Gregory Dudek
Centre for Intelligent Machines
School of Computer Science
McGill University
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Our Application
• Automatic generation of 3D maps.• Robot navigation, localization
- Ex. For rescue and inspection tasks.
• Robots are commonly equipped with camera(s) and laser rangefinder.
Would like a full range map of the
the environment. Simple acquisition of data
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Problem Context
• Pure vision-based methods – Shape-from-X remains challenging, especially in
unconstrained environments.
• Laser line scanners are commonplace, but– Volume scanners remain exotic, costly, slow.– Incomplete range maps are far easier to obtain that complete
ones.
Proposed solution: Combine visual and partial depth
Shape-from-(partial) Shape
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Problem Statement
From incomplete range data combined with intensity, perform scene recovery.
From range scans like thisinfer the rest of the map
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Overview of the Method
• Approximate the composite of intensity and range data at each point as a Markov process.
• Infer complete range maps by estimating joint statistics of observed range and intensity.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
What knowledge does Intensity provide about Surfaces?
• Two examples of kind of inferences:
Intensity image Range image
surface smoothness
variations in depth
surface smoothness
far
close
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
What about Edges?
• Edges often detect depth discontinuities• Very useful in the reconstruction process!
Intensity Rangeedges edges
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Range synthesis basis
Range and intensity images are correlated, in complicated ways, exhibiting useful structure.
- Basis of shape from shading & shape from darkness, but they are based on strong assumptions.
The variations of pixels in the intensity and range images are related to the values elsewhere in the image(s).
Markov Random Fields
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Related Work
• Probabilistic updating has been used for – image restoration [e.g. Geman & Geman, TPAMI
1984] as well as
– texture synthesis [e.g. Efros & Leung, ICCV 1999].
• Problems: Pure extrapolation/interpolation:– is suitable only for textures with a stationary
distribution
– can converge to inappropriate dynamic equilibria
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
MRFs for Range Synthesis
States are described as augmented voxels V=(I,R,E).
ZZmm=(x,y):1≤x,y≤m=(x,y):1≤x,y≤m: mxm lattice over which the image are described.
I = {II = {Ix,yx,y}, (x,y)}, (x,y) Z Zmm: intensity (gray or color) of the input image
E is a binary matrix (1 if an edge exists and 0 otherwise).
R={RR={Rx,yx,y}, (x,y)}, (x,y) Z Zmm: incomplete depth values
We model V as an MRF. I and R are random variables.
RI vx,y
AugmentedRange Map
IR
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Markov Random Field Model
Definition: A stochastic process for which a
voxel value is predicted by its neighborhood
in range and intensity.
€
P(Vx,y = vx,y |Vk,l = vk,l ,(k, l) ≠ (x,y)) =
€
P(Vx,y = vx,y |Vk,l = vk,l ,(k, l)∈ Nx,y )
Nx,y is a square neighborhood of size nxn centered at voxel Vx,y.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Computing the Markov Model
• From observed data, we can explicitly compute
€
P(Vx,y = vx,y |Vk,l = vk,l ,(k, l)∈ Nx,y )
intensity
intensity & range
Vx,y
Nx,y
• This can be represented parametrically or via a table.–To make it efficient, we use the sample data itself as a table.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Further, we can do this even with partial
neighborhood information.
Estimation using the Markov Model
• From
what should an unknown range value be?
For an unknown range value with a known
neighborhood, we can select the maximum
likelihood estimate for Vx,y.
€
P(Vx,y = vx,y |Vk,l = vk,l ,(k, l)∈ Nx,y )
Even further, if both intensity and range are
missing we can marginalize out the unknown
neighbors.
intensity
intensity & range
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Interpolate PDF• In general, we cannot uniquely solve the desired neighborhood
configuration, instead assume
€
P(Rx,y = rx,y | Ix,y = ix,y , Vk,l = vk,l , (k, l)∈ N x,y ) ≈
P(Ru,v = ru,v | Iu,v = iu,v , Vp,q = v p,q , ( p, q)∈ N u,v )
The values in Nu,v are similar to the values in Nx,y, (x,y) ≠ (u,v).
Similarity measureSimilarity measure:: Gaussian-weighted SSD (sum of squared differences).
Update schedule is purely causal and deterministic.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Order of Reconstruction
• Dramatically reflects the quality of the result.• Our reconstruction sequence is based on the amount
of reliable information surrounding each of the voxels to be synthesized.
• Edge information used to defer reconstruction of voxels with edges as much as possible.
Info-edge-driven ordering Correct result With the spiral-ordering
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Experimental Evaluation
• Obtain full range and intensity maps of the same environment.
• Remove most of the range data, then try and
estimate what it is.
• Use the original ground truth data to
estimate accuracy of the reconstruction.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Arbitrary shape of unknown range data
Input Intensity Synthesized result
Scharstein & Szeliski’s Data Set Middlebury College
Compact case
Ground truth rangeInput range imageSynthesized resultGround truth range
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Input IntensityGround truth range Input range image
Arbitrary shape of unknown range data
Scharstein & Szeliski’s Data Set Middlebury College
Less compact case
Synthesized result
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Arbitrary shape of unknown range data
Compact DistributedSynthesized results
Ground truth range
• Expected quality of reconstruction degrades with distance from known boundaries• Need broader distribution of range-intensity combinations in the sampling
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Ground truth rangeApproximated scene size: 550cms.
Type I: Stripes along the x- and y-axis
Input Intensity
Data courtesy of Oak Ridge National Labs & Univ. of Florida
xw rw
Case 1: rw=10, xw=20.39% of missing range
Synthesized ResultMean Absolute Residual (MAR) Error: 5.76 cms.
Ground truth rangeApproximated scene size: 550cms.
2004 IEEE/RSJ International Conference on Intelligent Robots and SystemsInitial range (rw=5, xw=25). 61% of missing range
Case 2 Result
Input intensity
Mean Absolute Residual (MAR) Error: 8.86 cms.
Initial range61% missing range
Ground truth range (approx. scene size: 550 cms.)
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Case 3 Result
Initial range76% missing range
Input intensity
Initial range data (rw=3, xw=28). 76% missing rangeMean Absolute Residual (MAR) Error: 9.99 cms.Ground truth range (approx. scene size: 550 cms.)
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Input intensity and range data Synthesized results Ground truth range
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Input range data. 66% of missing range
Achromatic vs. Color Results
Using achromatic image Using color image
Synthesized range images
Greyscale image Color imageGround truth range
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Type II: Stripes along the x-axis (like those obtained by our robot)
Case 1: rw=10, xw=20.62.5% of missing range
xw rw
xw rw
Input Intensity Ground truth rangeApprox. scene size: 600cms.
Synthesized ResultMAR Error: 20.72 cms.
Ground truth rangeApproximated scene size: 600cms.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Case 2 Result
Mean Absolute Residual (MAR) Error: 18.98 cms.
75% missing rangeInput intensity
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Case 3 Result
Mean Absolute Residual (MAR) Error: 20.23 cms.
78% missing rangeInput intensity
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Mike, just in case somebody ask
The picture on the top appearson the paper and the bottom one on this presentation. The only difference is in the region indicated by the red ellipse in the ground truth range image (right). This region corresponds to the glass window of the intensity image (left). Because it is a reflective surface, our laser range finder cannot measure itcorrectly. We decided to inter-polate all the incorrect measure-ments to get a new ground truth range image (bottom). Thus, the synthesized images also differ.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
More Experimental Results
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Conclusions
• Works very well -- is this consistent?• Can be more robust than standard methods (e.g.
shape from shading) due to limited dependence on a priori reflectance assumptions.
• Depends on adequate amount of reliable range as input.
• Depends on statistical consistency of region to be constructed and region that has been measured.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Discussion & Ongoing Work
• Surface normals are needed when the input range data do not capture the underlying structure
• Data from real robot – Issues: non-uniform scale, registration, correlation
on different type of data
– Integration of data from different viewpoints
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Questions ?
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Adding Surface NormalsNot in the paper
• We compute the normals by fitting a plane (smooth surface) in windows of nxn pixels.
• Normal vector: Eigenvector with the smallest eigenvalue of the covariance matrix.
• Similarity is now computed between surface normals instead of range values.
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Initial range scans
Preliminary Results Not in the paper
Synthesized range image Ground truth range
Edge map Real intensity image Initial range dataReal intensity image Edge map
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
The Neighborhood
• The neighborhood N{x,y} is a square mask of size nxn centered at the augmented voxel.
• Observations: The neighborhood is causal The size is important to capture relevant characteristics
v(x,y)
IR
Neighborhood of v(x,y)
N{x,y}
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
2004 IEEE/RSJ International Conference on Intelligent Robots and SystemsInitial range (none)
Novel View Synthesis (ongoing work)
Intensity image Range image
Known intensity image
• Infer range map from intensity of current view and from range and intensity of other views.
Other view
Current view to estimaterange map
Synthesized range Ground truth range
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Previous results Results using EDGES Ground truth range MAR Errors (cms.)
10.40 8.58
16.58 13.48
12.16 11.39
19.17 7.12
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
More examplesSynthesized results Ground truth rangeThe input intensity and range data
2004 IEEE/RSJ International Conference on Intelligent Robots and Systems
Adding Surface NormalsNot in the paper
• We compute the normals by fitting a plane (smooth surface) in windows of nxn pixels.
• The eigenvector with the smallest eigenvalue of the covariance matrix is the normal vector.
– The smallest eigenvalue measures the quality of the fit.
• In the range synthesis, the similarity is now computed between surface normals instead of range values.