Statistics of Range Images

Jinggang Huang Ann B. Lee David Mumford

Division of Applied Math Dept. of Physics Division of Applied Math

Box F, Brown University Box 1843, Brown University Box F, Brown University

Providence, RI02912 Providence, RI02912 Providence, RI02912

jhuang@cfm.bro wn.edu besu@cns.brown.edu David Mumford@bro wn.edu

Abstract

The statistics of range images from natural envi-

ronments is a largely unexplored �eld of research. It

closely relates to the statistical modeling of the scene

geometry in natural environments, and the modeling

of optical natural images. We have use d a 3D laser

range-�nder to collect range images from mixed forest

scenes. The images are here analyzed with respect to

di�erent statistics.

1 IntroductionMore and more people realize the importance of the

statistics of optical images. Good statistical models ofoptical images are important to applications, such asimage compression, noise remo val and segmentation.The models may also further our understanding of theencoding of visual information in biological visual sys-tems. Not much research has, however, been done onthe statistics of range images. There are two mainreasons why range images are interesting. First, theylead to a direct understanding of the stochastic natureof the geometry of the world, and give priors for stereoalgorithms ([1],[2]). Secondly, they make it possible toconstruct more realistic models of optical images. Forexample, authors in [11], [3] and [8] have modeled op-tical images as a perspective view of the 3D world,with objects of random geometry (size, shape, posi-tion) and intensity. The object geometries in thesemodels are usually based on assumptions, which havenot been directly veri�ed in real scenes. There is nodoubt, that with a fairly large data base of range im-ages, we will better understand the scene geometryof the 3D world, and thus be able to develop morerealistic models for optical images.

We have collected 205 panoramic range imagesfrom varying environments (both outdoor and indoorscenes) | 54 of these images were taken in di�er-ent forests in Rhode Island and Massachusetts dur-ing August-September. In this paper, we will focuson the forest scenes, because the statistics of these

Figure 1: A sample image from our range image database

images appear more stable than those of other cat-egories (for example, residential and interior scenes).Figure 1 shows a sample.

W e used a laser range-�nder with a rotating mir-ror 1 to collect the range images. Each image contains444� 1440 measurements with an angular separationof 0.18 deg. The �eld of view is thus 800 verticallyand 2590 horizontally. Each measurement is calcu-lated from the time of ight of the laser beam. The op-erational range of the sensor is typically 2-200m. Thelaser wavelength of the range-�nder is 0:9�m, whichis in the near infra-red region.

Throughout this paper, we will work withlog(range) instead of range directly, because the for-mer statistics is closer to being shape invariant. Figure2 shows a top view of a laser range-�nder (see circle)centered at O, and two homothetic triangles 4ABCand 4A0B0C0 (PA, PB and PC correspond to threepixels in the range image). Assume that the distancesbetween O and the vertices of 4ABC are rA, rB andrC, respectively, and the distances between O and thevertices of4A0B0C0 are RA, RB and RC , respectively.Let

D = range(PA)� range(PB)

be the di�erence in range for pixels PA and PB, and

13D imaging sensor LMS-Z210 by Riegl

1063-6919/00 $10.00 � 2000 IEEE

O

C’

B’

A’

P

CP

BP

A

C

B

A

Figure 2: A laser range �nder centered at O and twohomothetic triangles, 4ABC and 4A0B0C 0.

0 20 40 60 80 100−11

−10

−9

−8

−7

−6

−5

−4

−3

−2

−1

0

range

log(

pdf)

0 1 2 3 4−11

−10

−9

−8

−7

−6

−5

−4

−3

−2

−1

0

log(range)

log(

pdf)

Figure 3: Semi-log (left) and log-log (right) plots ofthe single pixel statistics (i.e. range statistics). Solid:distribution for the whole image. Dotted: distributionfor the bottom half of a range image. Dashed: distri-bution for the top half of a range image.

let

bD = log(range(PA))� log(range(PB))

= log

�range(PA)

range(PB)

�

be the di�erence in log range for the same two pixels.Then, a scene with 4ABC and a scene with triangle4A0B0C0 will lead to di�erent values for D (rA � rBvs. RA � RB) but the same value for bD (log( rA

rB) =

log(RARB

)). Hence, log(range) is appropriate if we wantthe di�erence statistics (or any mean-0-�lter reaction)to be \shape invariant".

2 Single-Pixel Statistics

The solid line in Figure 3 shows the single-pixelstatistics of log(range) images. W e observe a sud-den change in slope at a range of about 20 meters(or log(range) � 3; see vertical line in �gure) | thismay be related to the accumulation of occlusion ef-fects. In Figure 3, we have also plotted the log(range)histograms for the top half (dashed line) and bottomhalf (dotted line) of a range image separately. Thetwo halves correspond to di�erent distributions of ob-jects | mainly ground for the top half and mainlytrees for the bottom half | and displa y quite di�er-ent statistics. The distribution from the top half hasan approximately linear tail in a semi-log plot (in-dicating an exponential decay � e�0:12r), while thebottom half shows an approximately linear tail in alog-log plot (indicating a power law � r�2:6). We canqualitatively explain the two di�erent behaviors withthe following simpli�ed models:

For the top half, we assume tree trunks (cylinders)uniformly distributed on a plane, according to a Pois-son process with density �. Figure 4 shows a top viewof a randomly generated \forest" scene. Each diskrepresents a cross section of the trunk of a tree. If weassume all disks are of diameter L, a simple calcula-tion will show that the probability that a horizontalbeam from the laser range-�nder �rst hits a tree atdistance r is given by an exponential distribution

f(r) � �Le��Lr :

For the bottom half, we assume at ground only.Let the height of the sensor from the ground be H,as shown in Figure 5. Then at angle �, the distancebetween the sensor and the ground is r = H

sin�. The

laser range-�nder samples evenly with respect to thepolar coordinate(s) � (and �), i.e.

f(�) = constant

for �=2 < � < �. With respect to r, we then get apower-law distribution

f(r) =H

r2p(1� (H=r)2)

�1

r2:

3 Derivative StatisticsWe now look at the marginal distribution of the

horizontal derivative D, which in the discrete case, issimply the di�erence between two adjacent pixels inthe same row. The solid line in Figure 6 shows the logprobability density function of D. As in the studiesof optical images, this distribution has a high kurto-sis with large tails, and a peak at 0. It is closest to

1063-6919/00 $10.00 � 2000 IEEE

α

r

top view

Figure 4: Top view of a randomly generated forestscene. See text.

��������������������������������������������������������������������������������������������������������������������������������������������������������

H r

θ

Figure 5: Ground model. See text.

the statistics for optical images of man-made environ-ments [8], but has an even higher peak at 0. Thisindicates that the structure of range images is simplerthan for optical images | an issue that w e will explorefurther in the next section. The derivative statistichas also been used to test the scale invariance prop-erty (see Sinai [13], Field [5], Ruderman [12], and forscale invariance) in natural images. Given an image I,we de�ne a scaled-down image I(N) by computing theaverage of N �N disjoint blocks. In [12], [14], [3] and[8], it has been shown that, for optical natural images,the distribution of D is the same for I and I(N). Inthis paper, we scale down the images by taking theminimum, instead of the average ofN � N blocks.This is the appropriate renormalization for range im-ages, because laser range �nders measure the distanceto the nearest object in each solid angle. Figure 6shows the distribution of D at di�erent scales. Theresults indicate that range images scale well.

4 Bivariate StatisticsBelow, we study the co-occurrence statistics of two

pixels with di�erent separation distances. Let

K(a; bjx) = PrfI(x1) = a; I(x2) = b j kx1 � x2k = xg

where I(x1) and I(x2) represent the log(range) at pix-els x1 and x2. The left column of Figure 7 shows

−4 −3 −2 −1 0 1 2 3 4−15

−10

−5

0

5

derivative of log(range)

log

(pro

babi

lity)

.

Figure 6: Derivative statistics at di�erent scales.

the contour plots of K(a; bjx) for separation distancesx = 1; 16; 256.

The authors in [8] propose the following model forthe bivariate statistics of optical images:

K(a; bjx) = [1� �(x)]q(a)q(b) +

2�(x)hx(a+ b)gx(b� a)

where q is the marginal distribution for a single pixel,hx are distributions similar to q, and gx are distribu-tions highly concentrated at 0. The �rst term mod-els the case where the two pixels belong to di�erentobjects (we assume that di�erent object are statis-tically independent), the second term represents thecase where they are on the same object (assume thatthe sum and the di�erence of the pixel values are in-dependent), and �(x) is the probability of their beingon the same object.

The right column of Figure 7 shows a �t of themodel above to the empirical bivariate statistics (leftcolumn) of range images; Figure 8 shows the functionsused in the �t. The mixture nature | \same" v ersus\di�erent" objects | of both of data and the model,is seen in the changing shape of the contour levels inthe center versus the tails. W e also �nd that the ran-dom collage model �ts better to the bivariate statisticsof range images than to that for optical images (see[8]). Again, this indicates that range images presenta simpler, cleaner problem than optical images. F orexample, the concept of objects is better de�ned forrange images where we do not need to take lighting,color variations, texture etc. into account.

1063-6919/00 $10.00 � 2000 IEEE

5 Joint Statistics of Haar W avelet Co-e�cien ts

It has been shown that, the statistics in the waveletdomain can be very informative ([4], [7], [8]). We herechoose the Haar wavelet for its simplicity: Any struc-ture in the statistics can be directly related to pixelvalues. Note also that the central limit theorem caneasily make the statistics of larger support �lters ap-pear misleadingly Gaussian.

Figure 9 shows the di�erent Haar �lters. To de-scribe how the wavelet coe�cients in the subbandsare related to each other, we use the same de�nitionsas in [4]: Coe�cients at adjacent spatial locations inthe same subband are brothers, and coe�cients in thesame level and the same position, but di�erent orien-tations are cousins.

Figure 10 shows contour plots of the joint densityfunctions for di�erent wavelet coe�cient pairs. W eobserve strong cusps in all contour level curves; mostof which lie on the lines x = 0, y = 0, x = �y and x =�2y. As a comparison, look at Figure 11 where thecorresponding contour plots are calculated for opticalimages in van Haterens's image database [6]. We seethat many cusps occur along the same lines, but arenot as peaked.

Furthermore, we did the following experiments onthe range images:

1. W e scaled the images down by taking the blockminimum and calculated the joint histograms.Figure 12 shows the result.

2. W e scaled the images down by taking the blockaverage and calculated the joint histograms, Fig-ure 13 shows the result.

When w e scale a range image down by block min-imum, the resulting image is approximately the sameas the image taken in the same environment at half res-olution (as mentioned before, range images are almostscale invariant under block minimum). The secondmethod (block averaging) is, however, closer to how adigital camera for optical images works: The intensityat each pixel is the average (or some weighted mean)of the intensity at points covered by that pixel. Thisexplains why the statistics shown in Figure 12 is sim-ilar to that in Figure 10, and the statistics in Figure13 is similar to that in Figure 11.

Note that the observed cusps in Figure 10 are notcaused by noise, but correspond to real structures inthe images. It is important to see what these struc-tures are:

For the cousin pairs (horizontal and vertical; hori-zontal and diagonal) this is relatively straight-forward| because the join t distribution of horizontal, verticaland diagonal wavelet coe�cients is a su�cient statis-tics for 2�2 blocks modulo mean (3 variables). Figure14 shows an equiprobable surface of the 3D joint distri-bution (horizontal, vertical, diagonal). W e see verticesalong the lines y = z = 0, x = z = 0, x = y = 0 andx = �y = �z. Simple calculations show that thesevertices correspond to the following 2 � 2 blocks andtheir rotations:

�a ab b

�;

�a bb a

�;

�a bb b

�

For the horizontal �lter and left brother pair, we seecusps along y = 0, x = 0, y = x, y = 2x and y = 1

2x.To �nd the 2 � 4 patches that correspond to thesevertices, we sample randomly from our database forpatches with strong �lter reactions along y = 0, y = xand y = 1

2x. The 8 range values in the typical patchesfall clearly into 2 and occasionally 3 tight clusters.Calling the clusters a,b, and c, we get for example:

direction typical patches

y = 0

�a b b cb b b c

�;

�a a b cb b b c

�;�

a b a aa a a a

�

y = x

�a b b ba a a b

�;

�a b b aa a a a

�;�

a a a ab b b b

�;

�a a a ab b b b

�;

y = 12x

�a a a bb b b b

�

For the horizontal �lter and upper brother pair, wesee cusps along y = 0, x = 0, y = �x, y = �2x andy = �1

2x. The typical patches in the database, that

1063-6919/00 $10.00 � 2000 IEEE

lead to high �lter reactions along these lines are:

direction typical patches

y = 0

0BB@

a ab bb bb b

1CCA

y = �x

0BB@

a ab ab aa a

1CCA ;

0BB@

a ab ab aa a

1CCA ;

0BB@

a ab bb ba a

1CCA ;

0BB@

a bb bb bb a

1CCA

y = �2x

0BB@

a bb bb ba a

1CCA ;

0BB@

a ab ab ba a

1CCA

y = x

0BB@

a ab ba ab b

1CCA ;

0BB@

a aa ba bb b

1CCA ;

0BB@

a bb ba aa b

1CCA

We see that all the striking cusps in the contour plotsin the Haar wavelet domain relate to the piecewiseconstant geometric structure in range images.

6 Conclusions

We have investigated several simple statistics ofrange images of forest scenes. The bivariate statis-tics of two pixels verify the basic assumption of therandom collage model | that the w orld can be bro-ken down into piecewise smooth regions (in, for ex-ample, range or intensity) that depend little on eachother. The 2D and 3D joint distributions of Haar �l-ter reactions show some striking features, which alsoindicate the presence of piecewise constant geometricstructures and sharp discontinuities in range images.The above results all point to the fact that range im-ages are much simpler to analyze than optical images:The concept of \objects" is often better de�ned interms of changes in range, than in terms of changes inintensity, color, texture, lighting etc. W e believe thatsegmenting range images from natural scenes and athorough analysis of the results | for example, thestructure within objects, and the sizes, positions, anddependencies between di�erent objects | will lead toa better understanding of the scene geometry of the

3D world, as well as more realistic statistical modelsfor optical images.

References[1] P. N. Belhumeur and D. Mumford. \A ba yesian

treatment of the stereo correspondence problemusing half-occluded regions". Proc. IEEE Confer-

ence Computer Vision and Pattern Recognition,

1992 506-512

[2] P. N. Belhumeur. \A Binocular Stereo Algorithmfor Reconstructing Sloping, Creased, and BrokenSurfaces in the Presence of Half-Occlusion". Proc.Fourth International Conference on Computer Vi-

sion 431-438

[3] Z. Chi. \Probability Models for Complex Sys-tems". Ph.D. Thesis, Div. of Applied Math.,Brown Univ. 1998

[4] R. W. Buccigrossi and E. P . Simoncelli. \ImageCompression via Joint Statistical Characterizationin the Wavelet Domain".GRASP Laboratory Tech-

nical Report #414, Univ. of Pennsylvania, avail-able at:http://www.cns.nyu.edu/�eero/publications.html

[5] D. J. Field. \Scale-invariance and self-similar`wavelet' transforms: An analysis of natural scenesand mammalian visual systems". In M. Farge, J.C. R. Hunt, and J. C. Vassilicos, editors,Wavelets,

fractals, and Fourier transforms 151-193. Claren-don Press, Oxford, 1993

[6] J. H. van Hateren and A. van der Schaaf. \In-dependent Component Filters of Natural ImagesCompared with Simple Cells in Primary VisualCortex" Proc.R.Soc.Lond. B 265:359-366,1998

[7] J. Huang and D. Mumford. \Statistics of Natu-ral Images and Models". Proc. IEEE Conference

Computer Vision and Pattern Recognition, 1999

541-547

[8] A. B. Lee, J. Huang and D. Mumford. \Random-Collage Model for Natural Images". Submitted toInternational Journal of Computer Vision

[9] A. B. Lee and D. Mumford. \Scale-In variantRandom-Collage Model for Natural Images". InProc. IEEE Workshop on Statistic al and Compu-

tational Theories of Vision, Fort Collins, CO, 1999

[10] D. Mumford and B. Gidas. \Stoc hastic Models forGeneric Images". Submitted to Quarterly Journal

of Mathematics.

1063-6919/00 $10.00 � 2000 IEEE

[11] D. L. Ruderman. \Origins of Scaling in NaturalImages".Vision Research vol 37, No23, 3385-3395,1997

[12] D. L. Ruderman. \The Statistics of Natural Im-ages". Network, 5(4):517-548, 1994

[13] Y. G. Sinai. \Self Similar Probability Distribu-tions". Theory of Probability and Its Applications

Volume XXI, 64-80, 1976

[14] S. C. Zhu, and D. Mumford. \Prior Learning andGibbs Reaction-Di�usion". IEEE Transactions on

PAMI, vol 19, no 11, pp 1236-1250, 1997.

−2 0 2 4 6 8 10−6

−4

−2

0

2

4

6

u

v

x = 2

−2 0 2 4 6 8 10−6

−4

−2

0

2

4

6

u

v

Best fitting model, with λ = 0.99

−2 0 2 4 6 8 10−6

−4

−2

0

2

4

6

u

v

x = 16

−2 0 2 4 6 8 10−6

−4

−2

0

2

4

6

u

v

Best fitting model, with λ = 0.91

−2 0 2 4 6 8 10−6

−4

−2

0

2

4

6

u

v

x = 256

−2 0 2 4 6 8 10−6

−4

−2

0

2

4

6

u

v

Best fitting model, with λ = 0.73

Figure 7: Contour plots of the log histograms of pixelpairs for range images (left column) and the best bi-variate �t to the random collage model (righ t column).x: distance between the two pixels, u: sum of the pixelvalues, v: di�erence of the two pixel values.

1063-6919/00 $10.00 � 2000 IEEE

0 1 2 3 4 5 6 70.7

0.75

0.8

0.85

0.9

0.95

1

Psa

me

log2(distances)

−4 −3 −2 −1 0 1 2 3 4−18

−16

−14

−12

−10

−8

−6

−4

−2

0

2

4

v

log

prob

function gx

−2 0 2 4 6 8 10−14

−12

−10

−8

−6

−4

−2

0

u

log

prob

function hx

−2 0 2 4 6 8 10−14

−12

−10

−8

−6

−4

−2

0

log

prob

function qx

Figure 8: The values for � and the 1D functions gx,hx and qx in the best bivariate �t at pixel separationdistances x = 2 (solid), 16 (dashed), and 256 (dotted).

+1/2 +1/2

−1/2 −1/2horizontal filter

+1/2 +1/2

−1/2 −1/2

+1 −1

+1 −1vertical filter

+1/2 −1/2

+1/2 −1/2

+1 −1

−1 +1diagonal filter

+1/2 −1/2

−1/2 +1/2

+1/2 +1/2

+1/2 +1/2low pass filter

Figure 9: Haar �lters

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

vertical component

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

diagonal component

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

upper brotherho

rizon

tal c

ompo

nent

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

left brother

horiz

onta

l com

pone

nt

Figure 10: Contour plots of the log histograms ofwavelet coe�cient pairs for range images.

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

vertical component

horiz

onta

l com

pone

nt

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

diagonal component

horiz

onta

l com

pone

nt

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

upper brother

horiz

onta

l com

pone

nt

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

left brother

horiz

onta

l com

pone

nt

Figure 11: Contour plot of the log histogram ofwavelet coe�cient pairs for natural optical images.

1063-6919/00 $10.00 � 2000 IEEE

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

vertical component

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

diagonal component

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

upper brother

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

left brother

horiz

onta

l com

pone

nt

Figure 12: Contour plots of the log histogram ofwavelet coe�cient pairs, calculated from range imagesscaled down by taking the minimum of 2X2 blocks.

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

vertical component

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

diagonal component

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

upper brother

horiz

onta

l com

pone

nt

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

left brother

horiz

onta

l com

pone

nt

Figure 13: Contour plots of the log histogram ofwavelet coe�cient pairs, calculated from range imagesscaled down by taking the average of 2� 2 blocks.

−1.5−1

−0.50

0.51

1.5−1

−0.5

0

0.5

1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

VerticalHorizontal

Dia

gona

l

−1.6 −0.8 0 0.8 1.6

−1.6

−0.8

0

0.8

1.6

Vertical

Dia

gona

l

−1.6 −0.8 0 0.8 1.6

−1.6

−0.8

0

0.8

1.6

Horizontal

Ver

tical

Figure 14: An equi-surface of a 3D joint histogram ofhorizontal, vertical and diagonal wavelet coe�cientsin range images, viewed from three di�erent angles.

1063-6919/00 $10.00 � 2000 IEEE