MESH-FREE SPARSE REPRESENTATION OF MULTIDIMENSIONAL LIDAR DATA

Kristian L. Damkjer, Hassan Foroosh

University of Central FloridaDepartment of Electrical Engineering and Computer Science

Orlando, Florida

ABSTRACT

Modern LiDAR collection systems generate very large data sets ap-proaching several million to billions of point samples per product.Compression techniques have been developed to help manage thelarge data sets. However, sparsifying LiDAR survey data by meansother than random decimation remains largely unexplored. In con-trast, surface model simplification algorithms are well-established,especially with respect to the complementary problem of surfacereconstruction. Unfortunately, surface model simplification algo-rithms are often not directly applicable to LiDAR survey data dueto the true 3D nature of the data sets. Further, LiDAR data is oftenattributed with additional user data that should be considered as po-tentially salient information. This paper makes the following maincontributions in this area: (i) We generalize some features definedon spatial coordinates to arbitrary dimensions and extend these fea-tures to provide local multidimensional statistics. (ii) We proposean approach for sparsifying point clouds similar to mesh-free sur-face simplification that preserves saliency with respect to the multi-dimensional information content. (iii) We show direct application toLiDAR data and evaluate the benefits in terms of level of sparsityversus entropy.

Index Terms— LiDAR, multidimensional systems, point cloud,mesh-free simplification, principal component analysis

1. INTRODUCTION

Mapping and surveying Light Detection and Ranging (LiDAR) sys-tems produce large amounts of true three-dimensional (3D) data.Modern systems sample several thousand to over a million points persecond resulting in several million to billions of point samples perproduct to be stored, processed, analyzed and distributed [1, 2, 3].

Managing such large data sets presents a host of challenges tocontent providers. Production strategies have been developed tomitigate data management issues inherent in processing large-scaleprojects [4]. However, user demands for simultaneous wide-areacoverage, high-fidelity scene content, and low-latency access keepdata sizing considerations at the forefront of content provider con-cerns.

The LAS file format was developed to facilitate the exchange ofLiDAR data [5]. Extensions to the LAS format, e.g. LASzip, andgeneric exchange formats, e.g. HDF5, further address data sizingconcerns by offering support for lossless compression with typicalperformance yielding files between 10 and 20 percent of the originalfile size [6, 7]. However, even with an effective compression strat-egy, explicit data reduction is often necessary to support users inbandwidth-limited and mobile device environments. It is thereforenecessary to establish approaches to intelligently reduce point datain a manner that preserves information content. Current approaches

focus primarily on preserving the surface structures represented bythe spatial coordinates [8]. We describe an approach that also allowsfor the preservation of non-surface structures and includes point at-tribution in the salience criterion.

2. NOVELTY AND RELATIONSHIP TO PRIOR WORK

Simplification of LiDAR survey data remains largely unexplored,however point-based surface model simplification algorithms arewell-established, especially with respect to the complementaryproblem of surface reconstruction. We refer to the survey conductedby Pauly et al. for an overview of point-based surface simplification[8]. In this problem domain, there is an underlying assumption thatpoints in the cloud all belong to surfaces embedded in the spatialdimensions. This assumption is frequently violated in LiDAR datawhere points often belong to non-surface features. Further, surveydata is often attributed with additional information that should beconsidered in the simplification process lest salient information belost [4]. Regardless of these limitations, we draw inspiration for ourapproach from mesh-free surface simplification approaches.

Dyn et al. [9] present an iterative sub-sampling approach sup-ported by local surface approximation. Their approach operates ina fine-to-coarse manner terminated by a desired point set size, τ .Their point selection is solely based on the input point cloud geom-etry, P ⊂ R3, and a salience criterion, s : T ⊆ P \ {∅} → R.An important aspect of s is that it updates with respect to the currentsubset T ⊆ P throughout the point removal process.

Yu et al. [10] present a similar approach that enforces a post-condition of a terminal point set size and operates in an adaptivemanner driven by point clustering and a user-specified simplificationcriteria and optimization process.

While these approaches operate without generating an explicitmesh surface, they carry forward the legacy of mesh-based ap-proaches by limiting their analysis to spatial coordinates and operat-ing under the assumption that points locally approximate a surface.In contrast, natural scenes are complex and contain significant pointsbelonging to linear, planar, and isotropic structures. LiDAR surveydata is also frequently attributed with intensity or color data, classi-fication, or other user-defined features. These additional dimensionsmay contain content that is salient to end-user applications whichsuggests the need for a multidimensional approach to point removal.

The primary goal of this paper, therefore, is to create a data spar-sifying algorithm by developing a multidimensional salience mea-sure, s : P → R, and therefore demonstrate that such multidimen-sional approach produces sparse point representations that preservesalience. Several approaches have been developed to identify salientpoints based solely on 3D spatial coordinates. West et al. introducefeatures based on structure-tensor–eigenvalue analysis of local point

(a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 1: Visualization of Neighborhood Features on section of Arm-strong/Enderby data set from Applied Imagery. (a) Density, (b) Om-nivariance, (c) Isotropy, (d) Anisotropy, (e) Dimensionality, (f) Di-mension Label, (g), Component Entropy, (h) Dimensional Entropy

neighborhoods [11]. These feature descriptors have been enhancedto extract strong spatially linear features to support scene modelingapplications [12]. Methods have also been developed to direct opti-mal neighborhood scale selection for feature attribution [13]. Next,we generalize these attribute definitions to arbitrary dimensions toserve as the basis for measuring salience.

3. LOCAL STATISTIC ATTRIBUTION

Our salience measure is based on attributes defined by neighbor-hoods in arbitrary dimensions. In this section, we establish our def-inition for locality in arbitrary dimensions and generalize the defi-nitions for previously-established features in the spatial domain toarbitrary dimensions. Figure 1 illustrates the features we considerbased on evaluation of 3D spatial point data.

3.1. Data Conditioning

Our attributes are based on principal components analysis which issensitive to differences in scale within the feature space. The sourcedata should therefore be conditioned prior to analysis so that dif-ferent classes of attributes have approximately the same precisionscale or measurement resolution. Without this adjustment, insignifi-cant variations within one dimension can easily dominate significantvariations in another. We perform this conditioning by first decen-tering the data then normalizing each class by an estimate of themeasurement resolution for the class. We estimate the measurementresolution by computing the standard deviation within a flat responseregion for each attribute in the class. We then take the minimumclass attribute standard deviation as the measurement resolution forthe class.

3.2. Locality

We consider the analysis of multidimensional points, x ∈ Rn, whereN is the set of native attributes for the point and |N | = n is the di-mension of the native feature space. All attributes are assumed to bereal-valued. While boolean and finite-class attributes may be sim-ply represented by an appropriate integer enumeration, our approachis unlikely to yield meaningful results with such classes due to theconditioning issues mentioned previously. Our definition of a pointcloud, D ⊂ Rn, then is simply a database of real-valued multidi-mensional points with consistent feature space definition.

In most cases, it is desirable to restrict neighborhood definitionto a subset of the available native feature space. To support thiscapability, we establish a database of query points, Q ⊂ Rm, whereM ⊆ N is the search space of attributes for the determination oflocality and |M| = m is the dimension of the search space.

We proceed by analyzing the neighborhoods of points about thequery points, Vq ⊆ D. The neighborhoods are defined by an m-dimensional distance metric, δ, between the query points, q ∈ Q,and the data points, x ∈ D. For point cloud simplification, we treateach x ∈ D as a query location (i.e., Q = D). This approach re-quires a reasonable all nearest-neighbor search algorithm to be prac-tical, that is one with complexity no worse than O(p log p) wherep = |D|.

We investigated two neighborhood definitions that each presentmerits. The k-nearest neighborhood, Vkq , consists of the k closestpoints to q inD whereas the fixed-radius neighborhood, Vrq , consistsof all points in D within the ball of radius r centered at q. Similarto Dyn et al., we enforce the condition that q 6∈ Vq [9]. This con-dition is imposed so that Vq can be used to estimate the effects ofeliminating q during the simplification process.

3.3. Structure Features

West et al. and Demantke et al. define several features for describing3D point neighborhoods. In this section, we generalize, and in somecases modify, their proposed features to support multidimensionalanalysis and interpretability. The generalized features are summa-rized in table 1.

Name Equation

Omnivariance Omni :

(n∏d=1

λd

) 1n

(1)

Isotropy Iso :σnσ1

(2)

Anisotropy Ani :σ1 − σnσ1

(3)

Dimensionality αd :

{σd−σd+1

σ1, d < n

Iso , d = n(4)

Dimension Label d∗ : argmaxd∈{1,...,n}

αd (5)

Component Entropy Hσ : −n∑d=1

σd logn σd (6)

Dimensional Entropy Hα : −n∑d=1

αd logn αd (7)

Table 1: Features defined on Vq

West et al. present six features that proved to be most applicableto their work in segmentation and object recognition: omnivariance,anisotropy, linearity, planarity, sphericity, and eigenentropy [11].Each of the features they describe are derived from the eigenvaluesresulting from the principal components analysis of the query neigh-borhoods, Vq . However, while West et al. define the features withrespect to the eigenvalues, λ1 ≥ λ2 ≥ · · · ≥ λn, we generally pre-fer to use the singular values, σ1 ≥ σ2 ≥ · · · ≥ σn, as demonstratedby Demantke et al. [13]. The sole exception to this recommendationis the omnivariance feature which is used to meaningfully comparethe total variance of the neighborhoods to each other. Re-defining

the feature with respect to the singular values, while still meaning-ful, would be more directly related to the standard deviation.

Linearity, planarity, and sphericity are closely related featuresthat each represent the concept of the neigborhood’s participation insubsequently higher dimensions. That is, the values attempt to cap-ture the degree to which the local neighborhood spreads into eachof the respective dimensions [11]. We generalize this concept as di-mensionality and define the family of features by equation (4). Wefeel that it is worth considering the highest order dimensionality ofthe data set as a unique feature as well and generalize the concept toisotropy as defined by equation (2). The complement of this value,anisotropy, is thus easily understood and maintains a definition con-sistent with West et al. as expressed by equation (3)

Eigenentropy is a feature based on the Shannon entropy of theprincipal component eigenvalues. It describes the dimensional par-ticipation of the neighborhood. That is, higher values imply greaterparticipation across more of the available dimensions [11]. We gen-eralize this feature by modifying the logarithmic base to the numberof dimensions, n, and operating on normalized singular values, σd,instead of raw eigenvalues. We normalize the singular values bythe sum over all singular values for the neighborhood so that eachvalue can be treated as a probability that a point in the neighborhoodhas the respective eigenvector as its dominant local coordinate axis.The resulting feature, which we call component entropy, describesthe unpredictability of the neighborhood in the n-dimensional spaceand is expressed by equation (6).

Demantke et al. introduce two additional features to support au-tomated neighborhood scale selection: dimensionality labeling anddimensional entropy [13]. The dimension label is simply the dimen-sion that maximizes equation (4). We use this feature to establish anequivalence relation on D × D,x ∼ y ⇐⇒ d∗(Vx) = d∗(Vy).This equivalence relation creates a partition on D that we leverageas part of our simplification algorithm as described in section 4. Di-mensional entropy is very similar in concept to the component en-tropy, with the exception that it describes the Shannon entropy ofthe dimensionality feature. This feature describes the unpredictabil-ity of the dimension label feature and acts as a figure of merit for theselected label.

4. APPROACH

In this section, we describe a general point cloud sparsifying algo-rithm, derive the multidimensional salience measure, and describethe update operations that must take place per iteration to enforcethe correct dynamic behavior of the salience measure. Algorithm 1describes our solution that supports sparsifying points in arbitrarydimensions. Our objective is to remove least salient points, whilepreserving the proportional distribution of dimension labels in thefinal point set. We also wish to maintain the behavior that the algo-rithm computes a unique nested sequence of subsets that can be usedto define a multiresolution model.

The dimensional partitioning at line 3 of algorithm 1 is simplyachieved by segregating points according to equivalence relation es-tablished by equation (5). This partitioning only happens once toestablish the apparent local dimension of the point neighborhoods.Points are not moved out of their initial partition, regardless of howtheir descriptive features evolve through the sparsifying process.

We simultaneously enforce the proportional sparsifying con-straint and the nested subset constraint by removing points from thepartitions in an interleaved manner. We order the partitions so that|M1| ≥ · · · ≥ |Mn|. The pre-computed priorities for each parti-tion are given by equation (8) where M = maxd∈{1,...,n}|Md|.

Algorithm 1 Multidimensional Point Cloud Simplification

Require: D ⊂ Rn \ {∅}, |D| = N, τ ∈ ZNEnsure: T ⊂ D, |T | = τ

1: function MULTIDIMREMOVEPOINTS(D, τ )2: T ← D3: M←D/∼4: while |T | > τ do5: d∗ ← argmin

d∈{1,...,n}minPd

6: Pd∗ ← Pd∗ \ {minPd∗}7: x∗ ← argmin

x∈Md∗s(x)

8: Md∗ ←Md∗ \ {x∗}9: T ← T \ {x∗}

10: return T

Pd =

{mM

|Md|+d− 1

n: ∀m ∈ 1, . . . , |Md|

}(8)

In each iteration, we seek to select the point that minimizes thechange of information content in the point cloud. Dyn et al. use asalience measure that increases in value as points in the neighbor-hood diverge from the local fit of a smoothed surface [9]. Obviously,we are unable to use a similar model for salience since our measuremust be defined for arbitrary dimension. However, recall from sec-tion 3.3 that equation 6 describes the unpredictability of the neigh-borhood and acts as a measure of information content in the localneighborhood. We therefore select this feature, which is defined forarbitrary dimension, as the basis for our salience measure.

To estimate the change of information content caused by the re-moval of a point, we first establish a baseline estimate. The baseline,Hσ,0, is based on the component entropy of the initial point neigh-borhoods as described by equation (9).

Hσ,0(x) = Hσ(Vx ∪ {x}) (9)

We estimate the change of information content caused by the re-moval of a point as the maximum absolute deviation of the neighbor-hood component entropy from the component baselines as describedby equation (10). This measure acts as the salience function for oursparsifying process.

s(x) = maxy∈Cx

|Hσ,0(y)−Hσ(Vx)| (10)

In each iteration, the point, x∗, that minimizes equation (10)is selected for removal. To ensure that removed points continue toinfluence the sparsifying process, we maintain a constituency, Cx,for each x ∈ D. The constituency sets serve an identical functionto the test sets described by Dyn et al. and are updated in a similarmanner [9].

The constituency contains the set of points represented by x.Initially, each point represents only itself, i.e. Cx = {x}. When apoint, x∗, is selected for removal, its constituency, Cx∗ , is distributedamong its neighbors’ constituencies, {Cy : y ∈ Vx∗}, by selectingthe closest y ∈ Vx∗ as a representative for each z ∈ Cx∗ .

In addition to updating the constituencies, we must also updatethe neighborhoods containing the removed point to make sure thatit does not continue to influence estimates of the current point cloudstate. The set of back-references to the neighborhoods containingeach point, Bx = {Vy : x ∈ Vy}, are maintained to keep this up-date operation efficient. The neighborhoods containing the removed

(a) 75%, Hσ,0 ≈ 0.7369 (b) 50%, Hσ,0 ≈ 0.7367

(c) 25%, Hσ,0 ≈ 0.7384 (d) 10%, Hσ,0 ≈ 0.7393

Fig. 2: Data output by our approach on Dragon from the Stanford3D Scanning Repository

20% 40% 60% 80% 99%0

0.05

0.1

0.15

0.2

0.25

Percentage of Points Removed

Maxim

um

Rem

oved S

ignific

ance

(a) Least Salience

20% 40% 60% 80% 99%0.700

0.705

0.710

0.715

0.720

0.725

0.730

0.735

0.740

0.745

0.750

Percentage of Points Removed

Mean P

oin

t E

ntr

opy

(b) Model Entropy

Fig. 3: Least salience and mean baseline entropy trends during sim-plification of Dragon to 1% of the original point cloud size

point replace it with a new closest point from their neighbor’s neigh-borhoods. That is, from the set⋃

z∈Vy

Vz \ ({x∗} ∪ Vy) (11)

If the set described by equation (11) is empty, a closest pointfrom the current set of remaining points is selected instead.

Finally, the salience measures for each x ∈ Vx∗ ∪ Bx∗ are up-dated according to equation (10).

5. RESULTS AND DISCUSSION

We have implemented our approach using vantage point tree [14]for the spatial indexing structure in support of all nearest neighborsearching and splay tree [15] for managing the salience heap. Theselection of these data structures maintains aymptotic complexityequivalent to the approach proposed by Dyn et al. while compen-sating for higher dimensional data.

To illustrate the effectiveness of our approach, we first appliedour algorithm to the standard Dragon data set from the Stanford 3Dscanning repository which contains only spatial coordinates with

(a) 100% (b) 10% (c) 1%

Fig. 4: Mesh reconstruction from data output by our approach onVellum Manuscript from the Stanford 3D Scanning Repository

no additional attribution. Figure 2 shows results for data sparsi-fied to 75%, 50%, 25% and 10% of the original point cloud size,|D| = 435545. This test case demonstrates that our approach pro-duces a sparse representation of the original data that preserves fea-tures that are salient with respect to representing the original surface.Figure 3 illustrates the behavior of the algorithm during the sparsify-ing process. The salience measure does not increase monotonicallythroughout the sparsifying process since the point removal and up-date process does not enforce any guarantees on the entropies ofthe affected neighborhoods. However, figure 3a illustrates that theleast salience trend increases monotonically throughout the sparsi-fying process. Figure 3b illustrates the effect of our salience mea-sure on the mean baseline entropy for the model. Since we definesalience to minimize change in entropy, the mean entropy remainsvery flat through most of the sparsifying process and in fact in-creases slightly as redundant points are removed. However, thereis a point beyond which significant points are removed and mean en-tropy drops sharply as a result. For the Dragon test case, this occursonce approximately 90% of the original points have been removed.

Next, to illustrate the effectiveness of our approach on multidi-mensional data, we applied our algorithm to the Vellum Manuscriptdata set from the Stanford 3D scanning repository which containsspatial coordinates with color attribution per point. Figure 4 showsmesh reconstructions of data sparsified to 100%, 10%, and 1% ofthe original point cloud size, |D| = 2155617. This test case demon-strates that our salience measure generalizes to multidimensionaldata. The example illustrates preservation of fine features in the nDdata set up to high levels of sparsity. The thin red margin lines arevisible and paper edges are preserved even when data is sparsified tojust 1% of the original data size. Our approach is lossy, though, andsignificant degradation is noticeable at the 1% level. However, weare able to create a very faithful reconstruction of the data set withjust 10% of the original data.

6. CONCLUSION AND FUTURE WORK

In this paper, we develop extensions of established 3D features toarbitrary dimensions and present an application to sparse represen-tation of point clouds. We believe that this approach may be furtherenhanced by better selecting the initial neighborhood sizes using anapproach such as the one proposed by Demantke et al. [13] Wealso believe that there are other potentially interesting applicationsof these features that warrant investigation, for example as featuresthat support correlation and registration algorithms.

7. REFERENCES

[1] Christopher E. Parrish, “Full-waveform lidar,” In Renslow[16], chapter 2.4, pp. 54–61.

[2] Philip W. Smith, “Geiger mode lidar,” In Renslow [16], chapter2.6, pp. 91–97.

[3] Jamie Young, “Key elements of ALS technology,” In Renslow[16], chapter 2.2, pp. 17–37.

[4] Nicolas David, Clement Mallet, and Frederic Bretar, “Libraryconcept and design for LiDAR data processing,” in GEO-BIA 2008 - Pixels, Objects, Intelligence. GEOgraphic ObjectBased Image Analysis for the 21st Century, Geoffrey J. Hay,Thomas Blaschke, and Danielle Marceau, Eds., Calgary, Al-berta, Canada, Aug. 2008, ISPRS, vol. XXXVIII-4/C1 of IS-PRS - International Archives of the Photogrammetry, RemoteSensing and Spatial Information Sciences, University of Cal-gary, Calgary, Alberta, Canada.

[5] American Society for Photogrammetry and Remote Sensing,Bethesda, MD, USA, LAS Specification Version 1.4 R12, 2012.

[6] Martin Isenburg, “LASzip: lossless compression of LiDARdata,” in Proceedings of the 2011 European LiDAR MappingForum, Salzburg, Austria, 2011, Intelligent Exhibitions Lim-ited, pp. 1–9.

[7] The HDF Group, “Hierarchical data format version 5,”http://www.hdfgroup.org/HDF5/, 2000-2014, [February 12,2014].

[8] Mark Pauly, Markus Gross, and Leif P. Kobbelt, “Efficientsimplification of point-sampled surfaces,” in Proceedings ofthe Conference on Visualization ’02, Washington, DC, USA,2002, VIS ’02, pp. 163–170, IEEE Computer Society.

[9] Nira Dyn, Armin Iske, and Holger Wendland, “Meshfree thin-ning of 3D point clouds,” Foundations of Computational Math-ematics, vol. 8, no. 4, pp. 409–425, 2008.

[10] Zhiwen Yu, Hau-San Wong, Hong Peng, and Qianli Ma,“ASM: An adaptive simplification method for 3d point-basedmodels,” Computer-Aided Design, vol. 42, no. 7, pp. 598–612,2010.

[11] Karen F. West, Brian N. Webb, James R. Lersch, Steven Poth-ier, Joseph M. Triscari, and A. Evan Iverson, “Context-drivenautomated target detection in 3D data,” in Proceeding of SPIE,Firooz A. Sadjadi, Ed., Bellingham, WA, 2004, SPIE, vol.5426, pp. 133–143, SPIE.

[12] Hermann Gross and Ulrich Thoennessen, “Extraction of linesfrom laser point clouds,” in Symposium of ISPRS Commis-sion III Photogrammetric Computer Vision PCV ’06, WolfgangForstner and Richard Steffen, Eds., Gottingen, Germany, 2006,ISPRS, vol. XXXVI-3 of ISPRS - International Archives of thePhotogrammetry, Remote Sensing and Spatial Information Sci-ences, pp. 86–91, Copernicus Publications.

[13] Jerome Demantke, Clement Mallet, Nicolas David, and BrunoVallet, “Dimensionality based scale selection in 3D LIDARpoint clouds,” in WG V/3, I/3, I/2, III/2, III/4, VII/7, V/1 IS-PRS Workshop Laser Scanning 2011, Derek D. Lichti and Ay-man F. Habib, Eds., Gottingen, Germany, Aug. 2011, ISPRS,vol. XXXVIII-5/W12 of ISPRS - International Archives of thePhotogrammetry, Remote Sensing and Spatial Information Sci-ences, pp. 97–102, Copernicus Publications.

[14] Peter N. Yianilos, “Data structures and algorithms for near-est neighbor search in general metric spaces,” in Proceedingsof the Fourth Annual ACM-SIAM Symposium on Discrete Algo-rithms, Philadelphia, PA, USA, 1993, Association for Comput-ing Machinery and Society for Industrial and Applied Math-ematics, SODA ’93, pp. 311–321, Society for Industrial andApplied Mathematics.

[15] Daniel Dominic Sleator and Robert Endre Tarjan, “Self-adjusting binary search trees,” J. ACM, vol. 32, no. 3, pp.652–686, July 1985.

[16] Michael S. Renslow, Ed., Manual of Airborne Topographic Li-dar, American Society for Photogrammetry and Remote Sens-ing, Bethesda, MD, USA, 2012.