High Performance Computer Graphics, Multimedia and VisualisationVolume 1, (2000), Number 1 pp. 1-12

A Fast and Efficient Projection-BasedApproach for Surface Reconstruction

M. GopiDepartment of Computer Science

University of North CarolinaChapel Hill, NC 27599-3175

e-mail: gopics.unc.edu

S. KrishnanAT&T Shannon Laboratory

180 Park Avenue, Room E-201Florham Park, NJ 07932-0971

e-mail: krishnasresearch.att.com

Abstract

We present a fast and memory efficient algorithm that generates a manifold triangular mesh S passing through aset of unorganized points P� R3. Nothing is assumed about the geometry, topology or presence of boundariesin the data set except that P is sampled from a real manifold surface. The speed of our algorithm is derived froma projection-based approach we use to determine the incident faces on a point. Our algorithm has successfullyreconstructed the surfaces of unorganized point clouds of sizes varying from 10,000 to 100,000 in about 3–30seconds on a 250 MHz, R10000 SGI Onyx2. Our technique is especially suitable for height fields like terrainand range scan data even in the presence of noise. We have successfully generated meshes for scan data of size900,000 points in less than 40 seconds.

1. Introduction

The problem of surface reconstruction from unorganizedpoint clouds has been, and continues to be, an importanttopic of research. The problem can be loosely stated as fol-lows: Given a set of points P which are sampled from a sur-face inR3, construct a surface S so that the points of P lieon S. A variation of thisinterpolatorydefinition is whenSapproximatesthe set of pointsP.

There are a wide range of applications for which surfacereconstruction is important. For example, scanning complex3D shapes like objects, rooms and landscapes with tactile,optical or ultrasonic sensors are a rich source of data for anumber of analysis and exploratory problems. Surface rep-resentations are a natural choice because of their applica-bility in rendering applications and surface-based visual-izations (like information-coded textures on surfaces). Thechallenge for surface reconstruction algorithms is to findmethods which cover a wide variety of shapes. We brieflydiscuss some of the issues involved in surface reconstruc-tion.

We assume in this paper that the inputs to the surface re-construction algorithm are sampled from an actual surface(or groups of surfaces). A proper reconstruction of thesesurfaces is possible only if they are “sufficiently” sampled.However, sufficiency conditions like sampling theorems arefairly difficult to formulate and as a result, most of the exist-

ing reconstruction algorithms ignore this aspect of the prob-lem. Exceptions include the works of[3, 7, 2].

If the surface is improperly sampled, the reconstructionalgorithm can produce artifacts. A common artifact is thepresence of spurious surface boundaries in the model. Man-ual intervention or additional information about the sampledsurface (for instance, that the surface is manifold withoutboundaries) are possible ways to eliminate these artifacts.The other extreme in the sampling problem is that the sur-face is sampled unnecessarily dense. This case occurs whena uniformly sampled object with a few fine details can causetoo many data points in areas of low curvature variation.

Sometimes the data might contain some additional in-formation on the structure of the underlying surface. Laserscanners that generate samples uniformly on a sphere (or acylinder, depending on its degrees of freedom) are typicalexamples. In this case, it is known that adjacent data pointshave a very high probability of being adjacent to each otherin the final mesh. We refer to these data sets asorganizedpoint clouds. This information can be exploited by some al-gorithms, including ours, to give quick results.

Another issue in surface reconstruction is the presence ofnoise and outliers in the original data. The mode of dataacquisition has a direct impact on this. For example, rangescan data can be very noisy when the surface is not orientedtransverse to the scanning beam. Noisy data introduce high

c High Performance Computer Graphics, Multimedia and Visualisation and ReasearchPress International 2000. Published by Research Press International, PO Box 144 BristolBS9 1YA, UK.

Gopi and Krishnan / Projection Based Surface Reconstruction

frequency artifacts in the reconstructed surface (like micro-facets) and this is a cause of concern for many algorithms.

The choice of underlying mathematical and data structuralrepresentation of the derived surface is also important. Themost common choice are triangular or polygonal mesh rep-resentations. Triangular meshes also allow us to express thetopological properties of the surface, and it is the most popu-lar model representation for visualization and rendering ap-plications.

Finally, the recent thrust in research to build augmentedreality and telepresence applications has introduced an in-teresting variation of the surface reconstruction problem.Consider an application where multiple cameras or camera-projector pairs are used to extract the geometry of dynamicscenes at interactive rates[25]. In this scenario, the surfacereconstruction algorithm should be able to handle extremelylarge data sets (order of many millions of points) and providea suitable surface representation without significant latency.One of the main motivations for this work is to develop anapproach that handles bothorganizedandunorganizedpointclouds very efficiently in time and memory requirements.

1.1. Main Contributions

In this paper, we present a fast and efficient projection-basedalgorithm for surface reconstruction from unorganized pointclouds. Our algorithm incrementally develops an interpola-tory surface using the local estimated surface orientation ofthe given data points. The main contributions of this paperinclude:

� Linear time performance: Each iteration of our algo-rithm advances the reconstructed surface boundary bychoosing one point on it and completes all the faces inci-dent on it in constant time. The constant of proportionalityis very small.

� Speed:We have tested our algorithm on a number of datasets ranging from 10,000 to 100,000 unorganized points.It takes about 3–30 seconds to reconstruct the mesh. Wehave also tested our algorithm on an organized point cloudof size 6.5 million. After simplifying this data to around900,000 points, it took us about 40 seconds to generate themesh on a 250 MHz, R10000 SGI Onyx2 with 16 GB ofmain memory.

� Memory efficiency: Our algorithm has minimal memoryoverhead because it goes through a single pass of all datapoints to generate the mesh. We do not maintain the com-puted triangles in our data structure because our methoddoes not revisit them. Only the input data has to be stored.

� Robustness:In the special case of terrain data or datafrom common center-of-projection scanning devices, ouralgorithm can tolerate high noise levels. The error intro-duced by noise has to be bounded, however.

Organization of the paper

The rest of the paper is organized as follows. Section 2 dis-cusses the previous work in surface reconstruction. We pro-vide an overview of our algorithm in Section 3 and complete

details of our surface reconstruction approach in Section 4.The robustness of our algorithm in the presence of noise anddegenerate conditions in the input data are discussed in Sec-tion 5. We describe the application of our algorithm to telep-resence applications in Section 7. Sections 8 and 9, respec-tively, talk about the implementation issues and performanceresults of our algorithm. We conclude in Section 10.

2. Previous WorkThe problem of surface reconstruction has received signif-icant attention from researchers in computational geometryand computer graphics. In this section, we give a brief surveyof existing reconstruction algorithms. We use a classificationscheme by Mencl et. al.[23] to categorize the various meth-ods. The main classes of reconstruction algorithms are basedon spatial subdivision, distance functions, surface warpingandincremental surface growing.

The common theme in spatial subdivision techniques isthat a bounding volume around the input data set is subdi-vided into disjoint cells. The goal of these algorithms is tofind cells related to the shape of the point set. The cell selec-tion scheme can be surface-based or volume-based.

The surface-based scheme proceeds by decomposing thespace into cells, finding the cells that are traversed by thesurface and finding the surface from the selected cells. Theapproaches of[19, 12, 4, 3]fall under this category. The dif-ferences in their methods lie in the cell selection strategy.Hoppe et. al.[19, 20, 18] use a signed distance function ofthe surface from any point to determine the selected cells.Bajaj and Bernardini[4, 5, 6] construct an approximate sur-face usingα-solids to determine the signed distance func-tion. Edelsbrunner and Mucke[24, 12] introduce the notionof α-shapes, a parameterized construction that associates apolyhedral shape with a set of points. The choice ofα has tobe determined experimentally. More recently, Guo et. al.[15]use visibility algorithms and Teichmann et. al.[27] use den-sity scaling and anisotropic shaping to improve the resultsof reconstruction usingα-shapes. For the two-dimensionalcase, Attali[3] introducesnormalized meshesto give boundson the sampling density within which the topology of theoriginal curve is preserved.

The volume-based scheme decomposes the space intocells, removes those cells that are not in the volume boundedby the sampled surface and creates the surface from the se-lected cells. Most algorithms in this category are based onDelaunay triangulation of the input points. The earliest ofthese approaches is Boissonat’s[9] “Delaunay sculpting”algorithm that successively removes tetrahedra based ontheir circumspheres. Veltkamp[30] uses a parameter calledγ-indicator to determine the sequence of tetrahedra to beremoved. The advantage of this algorithm is that theγ-indicator value adapts to variable point density. However,both the approaches of Boissonat and Veltkamp cannot han-dle objects with holes and surface boundaries. Amenta et.al. [2, 1] use a Voronoi filtering approach based on three-dimensional Voronoi diagram and Delaunay triangulation toconstruct thecrust of the sample points. They provide the-

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

oretical guarantees on the topology of their reconstructedmesh given “good” sampling.

The distance function of a surface gives the shortest dis-tance from any point to the surface. The surface passesthrough the zeroes of this distance function. This approachleads to approximating instead of interpolatory surfaces[19, 18, 8, 10]. Hoppe et. al.[19, 18] use a Reimannian graphto compute consistent normal throughout the surface to de-termine the signed distance function. The approach of Cur-less and Levoy[10] is fine-tuned for laser range data. Theiralgorithm is well suited for handling very large data sets.

Warping-based reconstruction methods deform an initialsurface to give a good approximation of the input point set.This method is particularly suited if a rough approximationof the desired shape is already known. Terzopoulos et. al.[28] usedeformable superquadricsto fit the input data points.A different approach to warping was suggested by Szeliskiet. al.[26] with oriented particles. By modeling the interac-tion between the particles, they construct the surface usingforces and repulsion.

The basic idea behind incremental surface construction isto build-up the surface using surface-oriented properties ofthe input data points. The approach of Mencl and Muller[21, 22] is to start with a global wireframe of the surface gen-erated using Euclidean minimum spanning tree construction,and to fill it iteratively to complete the surface. Boissonnat’ssurface contouring algorithm[9] starts with an edge and iter-atively attaches further triangles at boundary edges of theemerging surface using a projection-based approach. Thisalgorithm is similar in vein to our approach. A crucial dif-ference between our methods is that Boissonnat’s algorithmis edge-based, while ours is vertex-based. Further, his algo-rithm can only generate manifolds without boundaries.

3. Algorithm OverviewThe input to our algorithm is a set of unorganized points withno additional information (like normals). The output is a tri-angulated mesh which interpolates the input point set. Ouralgorithm starts at a data point, and finds all its incident tri-angles. Then each of its adjacent vertices in the boundaryof the triangulation is processed in a breadth-first fashionto find their other incident triangles. Thus the boundary ofthe completed triangulation propagates on the surface of thepoint cloud till it processes all the data points. In the rest ofthe paper, we refer to the point being processed as therefer-ence point, R.

There are three assumptions we make about the data set.The sampling of the data islocally uniform, which meansthat the distance ratio of the farthest and closest neighborof a sample in the given sampling of the object is less thana constant value. The second assumption is to distinguishpoints from two close layers of the object. The closest dis-tance between a pointP in one layer and another layer is atleastµm, whereµ is a constant andm is the shortest distancebetweenP and another point in its layer. The third assump-tion is about the smoothness of the underlying object. Thenormal deviation between the any two triangles incident on

a vertex should be less than 90�. This assumption is used injustifying our choice of tangent plane in section 4.1.

Our algorithm can be broadly divided into three stages:bucketing, point pruning, and finally thetriangulation step.

Bucketing: In this stage, the data structure is initializedwith the input data. Our data structure is a depth pixelarray similar to thedexelstructure[17]. We maintain a 2Dpixel array into which all data points are orthographicallyprojected. The points mapped on to the same pixel are sortedby their depth (z) values.

Point Pruning: This step is similar to clustering algorithmsused by other triangulation schemes[19, 18, 16]. We first ap-ply a distance criterionto prune down our search for candi-date adjacent points in the spatial proximity ofR. It is exe-cuted in two stages. In the first stage, the simplerL1 metricis used to define the proximity aroundR. Our algorithm takesan axis-aligned box of appropriate dimensions centered atR and returns all the data points inside it. The major differ-ence in our approach compared to other approaches is the useof dexellike data structure for this stage. By using our datastructure, this search is limited to the pixels around the pixelwhereR is projected. Another advantage of this data struc-ture is explained in Section 7 where the information aboutthe characteristics of the scanning device, used for collectingthe data points, is used to improve the robustness of the algo-rithm. The second stage of pruning uses a Euclidean metric,which further rejects the points that lie outside asphere ofinfluencecentered atR. The choice of the box dimensionsand the radius of the sphere are described in the next sec-tion. The points chosen after the pruning using the Euclideanmetric are called thecandidate pointsof R, CR.

Visibility Criterion: Next, we estimate the tangent plane atR, and projectR,CR, and the mesh boundary in their vicinityon this tangent plane. The projected points ofCR are thenordered by anglearoundR. Points inCR that are occludedfrom R by the mesh boundary in the projection plane areremoved.

Angle Criterion: This is an optional step, which tries toremove “skinny” triangles atR, to improve the quality oftriangulation.

Triangulation : Finally, the remaining points inCR are thenconnected in order aroundR to complete the triangulation.

4. Surface ReconstructionIn this section, we describe our approach to surface recon-struction in detail. The output of our algorithm is an inter-polatory, non-self-intersecting triangular mesh of the givenpoint cloud.

The implicit function theorem[13] of smooth surfacesforms the basis of our approach. Without loss of general-ity, it states:“Given an implicit surface S� f (x;y;z) = 0,and a point P on it, such that the tangent plane to S at P isparallel to the(x;y) plane, then S in the neighborhood of P

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

R R

(b)(a)

V

SR

Figure 1: (a) Visibility test around R. The black points are behind R’s boundary edges, the white points are occluded by otheredges, and the point V is eliminated as R is behind its boundary edges. (b) Completed mesh at R

can be considered as a height function f(x;y;h(x;y)) = 0, alocal parameterization on its tangent plane”. By a suitablerigid transformation of the coordinate frame, any other pointonScan be made to satisfy the above theorem.

Our algorithm is a greedy method and works with twoparameters:µ, which quantifies our definition oflocally uni-form sampling, and α, which gives a lower bound on theangle between consecutive neighbors of a point on a bound-ary of the surface. Typically,α is a large obtuse angle. Inour implementation, we have setα to be 120�. All other pa-rameters, which are required for the implementation of thealgorithm are derived fromµ. In order to improve the qualityof triangulation, we can optionally specify a minimum angleparameter,β. It is not necessary for the completion of ouralgorithm, though.

Terminology: We categorize the data points at any givenstage of our algorithm asfree, fringe, boundaryand com-pletedpoints. Thefreepoints are those which have no inci-dent triangles. Thecompletedpoints have all their incidenttriangles determined. Points that lie along the current sur-face boundary are eitherfringe or boundarypoints.Bound-ary points are those points which have been chosen as a ref-erence point but have some missing triangles due to the max-imum allowable angle parameterα. Fringe points have notyet been chosen as a reference point.

We maintain two invariants during our algorithm’s execu-tion:Invariant 1: No free, fringe or boundarypoint can be in theinterior of a triangle (because of our distance criterion).Invariant 2: At the end of each iteration, the point cho-sen as the reference point becomes acompletedor abound-ary point. This is used later to prove claims about occludedpoints (for visibility criterion).

Our algorithm starts with the bucketing step by ortho-graphically projecting the data points onto thedexel datastructure. The following steps are used to choose the rightset of points to be connected to the reference pointR.

4.1. Point Pruning

Pruning by Distance Criterion: Points far away from thereference pointRare not likely to be adjacent to it. We elim-inate them by applying the distance criterion in two stages.Initially, we employ the cheaperL1 metric to narrow downour search. It is performed by constructing an axis-alignedbox of suitable dimension aroundR and choosing all thefree, fringeandboundarypoints inside the box. By using ourdexelarray, this is a logarithmic time operation with smallconstant.

The dimension of the box is derived fromµ as follows.In a general case,R (a fringe point) already has a few inci-dent triangles. Letm be the minimum distance fromR to itsexisting adjacent vertices. From our definition oflocally uni-form sampling, the farthest neighbor ofR can be at mostµmaway. This gives an estimate on the dimension of the box.WhenR has no incident triangles (for example, at the verybeginning), we find the closest point toR using thedexelarray representation and findm. The minimum distance be-tween the points in the above box andR, refines the previousestimate ofm. Using this newm, the next step further prunesthe chosen set of points in the proximity ofR.

We call a sphere of radiusµmcentered atR as thesphereof influence(SR) aroundR. The second stage of pruning usesa stricterL2 metric and returns all points insideSR. Thesepoints are thecandidate points(CR) of R. We would like tomake an observation about the candidate point set. The ra-dius ofSR is dependent onm, which changes from one ver-

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

Q

SR

g

f

V

W

Q

SR

U

R

r

(a)

V

e

b

a

(b)

U

Y

d

R

r

c

X

Figure 2: (a) Determining occluding edges (b) Angle XQY is obtuse, so a< c. Further, f< d+a< d+c< d+c+e. Therefore,jQUj < jUVj.

tex to another. Therefore, it is possible that a vertexp mightbe in thesphere of influenceof R, but not vice-versa. But thisasymmetry does not affect the topology of the reconstructedmesh.

Choice of Projection Plane: The triangulation aroundRimplicitly defines an ordering of its adjacent vertices aroundR on a projection plane. We find this ordering directly byprojectingCR on a plane. The choice of the projection planeis an important issue, and dictates the robustness of ouralgorithm. According to the implicit function theorem, thebest projection plane would be the tangent plane atR. Onecan adopt more robust algorithms like the one described inHoppe et.al[19] or Amenta et.al.[2, 1]. An alternate cheaperapproach to compute the projection plane normal is byaveraging normals of existing triangles incident onR. Sincewe are interested only in the relative ordering of pointsaroundR, we use this latter approach in our implementation.The ordering of thecandidatepoints (CR) aroundR in thisplane will be incorrect only if there is a triangle incidenton R with its normal deviating by more than 90� from theprojection plane normal. Our choice of projection plane isjustified by our assumption about the smoothness of theunderlying object. We assume that the object from whichthe input is sampled is smooth enough so that the vari-ation in the tangent planes for proximate points is very small.

Angle Ordering: A main step in our algorithm is to orderpoints inCR projected onPR by angle aroundR. We nowdescribe a fast and inexpensive method to perform this or-dering.

We define a new local coordinate system where the ref-erence pointR is the origin andPR is thexy-plane. Initially,each candidate point ofR is projected onPR in this coor-dinate system. Let this set of projected candidate points beCp

R. The ordering aroundR is based on the angle (θ) betweenthex-axis of the local coordinate system and the vector fromorigin to the projected candidate point.

The setCpR is partitioned by the quadrants in which they

lie. In each of these quadrants we order the points based onsin2(θ). We usesin2(θ) because it is almost linear within aquadrant and is inexpensive to compute. The actual angleθin the projection plane is computed using a look-up tableand a simple linear interpolation. We now order the pointswithin each quadrant and finally merge these four orderedsets. We use the actual angle to identify holes and skinnytriangles in the model.

Pruning by Visibility: We use the angle ordering ofCpR to

efficiently perform the next stage of pruning based on visi-bility in the planePR. It eliminates the points which poten-tially form a self-intersecting mesh. We define theboundaryedgesof a point as the set of edges incident on that pointthat lie on the current surface boundary. Any edge with notriangle formed on one of its side, is aboundary edge. Allboundary edgesconnectfringe and/orboundary points. Onthe other hand,internal edgesare the edges which connectcompletedpoints with any other point. We projectR,CR, andtheirboundary edgeson the planePR. If the line of sight fromR to a projected candidate vertex is obstructed by any edge,then that point is an occluded point. The existence of visibil-ity between these points in the plane is a sufficient but nota necessary condition for the visibility between them in theobject space. In the limit, when the local surface approachesthe tangent plane in a densely sampled point cloud, it be-comes a necessary condition as well. We take a conservativeapproach and prune all the points inCR which are occludedfrom R onPR.

Points occluded fromRare determined as follows.

1. All the points between consecutiveboundary edgesof R(shown by the dotted-line wedge atR in Figure 1(a)) areremoved as they cannot be visible fromR. They are saidto be in theinvisible region ofR. The black points in thefigure are examples.

2. Similarly, points are removed which haveR in their invis-ible region (for example, pointV in the same figure). We

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

R

N

P

R

N

P

(a) (b)

NN

N

N N

NN N

12

3

4

5 5

4

3

21

Figure 3: Pruning by Angle Criterion: (a) Ordering around R and P; angles between N1N2, N3N4, and N4N5 are less thanβ.(b) N3 trapped in4RPN4, and N4 trapped inside4RN3N5

denote the set of points fromCR remaining after this stepasCv

R.3. Finally, we eliminate points that are occluded fromRbe-

cause of an existing edge in the mesh (for example, thewhite point in Figure 1(a)).

A straightforward approach of checking all possible oc-cluding edges is very expensive. We state the following the-orem which limits our search to very few edges. This the-orem is true under the assumption about the smoothness ofthe underlying object.

Theorem: Only theboundary edgesof the points in the setCR can be possible occluding edges betweenRandCv

R.

Proof: From Invariant 1, it is easy to show that if an inter-nal edge is occluding, there must be at least one boundaryedge which is also occluding. This eliminates all the internaledges from our consideration. Figure 2(a) shows an exam-ple where boundary edges (likeUV or/andVW) occlude thepointQ from R, but its endpoints are not inCR. Given that atleast one ofUV orVW is aboundaryedge, it must be part ofsome existing triangle (likeUVW in the figure). This impliesthat one ofU , V or W (let us assumeU) must have alreadybeen chosen as a reference point. We have to prove that ifany one of these points was an earlier reference point, then itshould have chosen eitherQ or R (or both) as its neighbor(s).If it had chosenQ (resp.R) as its neighbor, thenR (resp.Q)will lie on Q’s (resp.R’s) invisible region, andQ would beeliminated fromCv

R.

In the Figure 2(b), let us choose one of the edges, sayUV, to complete our proof. LetX and Y be the intersec-tion points of this edge with the projection ofSR. 6 XQYis obtuse, because angle subtended by the diameter on thecircumference is a right angle, and6 XQY is clearly greaterthan that. Hence the distancec = jXYj is the longest edgeof the4XQY, which means thata < c and b < c. Hence(d+a) < (d+ c) < (d+ c)+e, and by triangle inequality,jUQj = f < (d+a) < (d+c)+e= jUVj. Similarly we canprove thatjVQj < jVUj. This argument extends to any edgethat is placed similar toUV.

From our distance criterion, we claim that vertexQ must

be adjacent toU . With Q as its neighbor,U completes itstriangulation by adding edgesQV andQW. This implies thatR lies in the invisible region ofQ, and hence cannot belong toCv

R as it will be eliminated by condition 2 above. Therefore,UV cannot be an occluding edge.

The rest of the points which are ordered by angle aroundR can be triangulated as shown in Figure 1(b).

Pruning by Angle Criterion: The triangulation we get fromthe previous step is a valid one. However, to improve thequality of triangulation, this pruning step removes pointsthat could potentially form triangles with very small an-gles (“skinny” triangles). This is not a necessary componentfor the working of our algorithm. Since our algorithm doesnot introduce additional (Steiner) points, it cannot alwaysachieve the desired quality. It is a greedy approach, whichwould eliminate sliver triangles whenever possible.

We explain the working of this step using an example. InFigure 3, consider the pointsN1 andN2. Let us assume thatthe angle atRof4RN1N2 is less thanβ (the minimum angleparameter). One of these points can be removed to improvethe triangulation. The choice of the removable vertex is notarbitrary. For example, ifN2 is rejected, it gets trapped insidethe triangle (in the projection plane) formed byR, N1, andany one ofN3, N4, or N5. This violates Invariant 1.

The following algorithm describes a way to avoid suchscenarios and to form a good triangulation whenever possi-ble. Assume that we have to complete the triangulation fromP to N5 aroundR in Figure 3, whereRP andRN5 are con-secutiveboundary edgesof R. We start our processing byordering the points aroundP. In our example, this orderingwould bePs = (N1;N2;N5;N4;N3), and the ordering aroundR is Rs = (N1;N2;N3;N4;N5). Let Ps[i] (Rs[i], respectively)be theith element inPs (Rs, respectively). Without loss ofgenerality, we assumeRs[i] =Ni . The following pseudo-codefinds all possible adjacent points toP aroundR, without trap-ping any other point inside the triangle.

for 1� i � jPsjLet Nj be the vertex inPs[i]Mark Nj asconsidered

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

T(Nj ) = {Nk j k< j , Nk is not markedconsidered}if (T(Nj ) = φ)

thenNj can be an adjacent pointto P aroundRelseNj cannot be an adjacent pointto P aroundR

In our example, we first selectPs[1] = N1. Since there isno Nk such thatk < 1 andNk is not markedconsidered,N1 is a possible adjacent point toP aroundR. This is alsotrue forPs[2] = N2. Now considerPs[3] = N5. Here, we havetwo pointsN3 andN4 which are not markedconsideredandhence belong to the setT(N5). We can see thatN3 andN4 areinside the4RPN5. ThereforeN5 cannot be an adjacent pointto P aroundR. In the general case, the setT(Nj ) consists ofprecisely those vertices that will be trapped ifNj were cho-sen as the adjacent point toP aroundR. If we complete theabove algorithmN3 will also be chosen as a possible adja-cent point.

From the set of possible adjacent pointsfN1;N2;N3g, wecan choose any vertex. For the sake of argument, let uschooseN3 as the adjacent point toP and form the triangleRPN3. Now the same algorithm is applied atN3, and thepointsN4 andN5 are ordered around it. It can be seen thatthe pointN4 cannot be removed, as it will get trapped insidethe triangleRN3N5. Hence, we cannot eliminate the skinnytriangleRN3N4. It is important to note that even if we hadchosenN1 or N2 from the original possible adjacent pointset, we would have ended up in the same situation.

4.2. Triangulation

The remaining points fromCR after the various pruningsteps are the final adjacent points and are connected in orderaroundR to complete the triangulation in the object space. Ifconsecutive adjacent points subtend more thanα (maximumallowable angle parameter) atR in the object space, then theyare not connected to form a triangle. This maximum angledescribes the characteristics of the holes in the model, andR is considered as aboundarypoint. All the free points inthe adjacent point list are labeled asfringe points and areappended in order at the end of the queue. The algorithmchooses the next point from the queue as the new referencepoint R, and continues with the triangulation aroundR.

5. Robustness of our algorithmWe avoid most of the robustness problems faced by purelygeometric methods (like noise and degenerate situations) byour partially combinatorial approach. In our algorithm, weface robustness problems in the projection plane evaluation.For example, sharp curvature variations in the object mightlead to incorrect estimates of the projection plane.

To test the robustness of our approach to perturbations inthe estimated tangent plane atR, we used one of just threeprojection planes –(X;Y), (Y;Z), and(Z;X), whichever wasclose to the actual estimate. We were able to triangulatemany models including the bunny model satisfactorily. Theexecution time with this approach is much less than the timeslisted in the Table 1 because we do not need to explicitly

transform the vicinity ofR to its tangent plane. But the dis-advantage of this approach is that it has a few favorable ori-entations of the model in the coordinate frame, and differentorientations gave different results.

6. Triangulating Terrain DataSimple solutions are available to triangulate smooth terraindata. In this section, we show that these algorithms can bemade as a special case to our 3D triangulation algorithm byfixing the projection plane. We describe the nature of ournoisy terrain data with respect to its source, namely the laserscanner, and way to interpret the data so that our triangu-latioa method can be applied to it. Similar methods can beadopted to specialize our generic triangulation scheme forother depth extraction devices.

Ability to handle noise in the input is an important con-sideration for any surface reconstruction algorithm that isapplied to real-world data. Statistical methods like medianfiltering have been tried before to remove outliers and highfrequency noise and fit non-interpolatory surfaces. But thesemethods can sometimes be slow, and might not even gener-ate surfaces that are intuitively correct for some data sets.

Typically, devices used for data acquisition generate sam-ple data in some order. In our algorithm, we can make useof this order and the characteristics of the device to handlenoise. We have applied our method on a massive data set ofa room (Color Plate 2 - top row) acquired by a laser rangescanner. This is a common Center-Of-Projection (COP) de-vice which returns a spherical depth map ((θ, φ) map) ofthe environment around itself. The output of such a deviceis called afirst-seen surface, as it samples only the surfacethat it first sees. It has no information about occluded regionsfrom its COP. The sampling density of this device is also ex-tremely high (one sample per tenth of a degree in bothθ andφ). When adjacent samples are less than an inch apart, thenoise in the samples is nearly two inches. If we apply ouroriginal algorithm to this noisy data set, the selected pro-jection plane would be completely altered by innumerablemicro-facets formed in the vicinity of a point.

We make use of the fact that the data point is on thefirst-seen surfacein the spherical coordinate system to solve thisproblem. In such a coordinate system, this surface can beconsidered as a monotonic surface with respect to a uniqueprojection plane similar to height field or terrain data. In thiscase, the(θ;φ) plane can be considered as the projectionplane for all the points. Since the perturbations in the data setdue to noise are usually orthogonal to the projection plane,our algorithm is not affected by it.

By just fixing the projection plane, we can reconstruct thesurface without any changes to our algorithm. Traditional3D reconstruction algorithms are not well-suited to handleterrain or range data. As a result, specialized algorithms[14]have been developed to exploit the simplicity of the input.However, in our algorithm, terrain models are a special casewhere we fix the projection plane to remain constant. Fur-ther, since estimating the projection plane at each point isavoided, our algorithm runs significantly faster as well.

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

No. of No. of Init. Time Recons. TimeModel points Triangles (in secs) (in secs)

Club 16864 33660 0.2758 3.9644Bunny 34834 69497 0.5961 9.1809Foot 20021 39919 0.3802 5.2725

Skidoo 37974 75461 0.6680 8.536Mannequin 12772 25349 0.2405 3.9289

Phone 83034 165730 1.5634 26.597

Table 1: Performance of our algorithm: See Color Plate 1

6.1. Specializing our Algorithm

The underlying two dimensionaldexel arrayis considered asthe (θ;φ) projection plane with only one data point at eachdexel. The neighbors of a point in the final triangulation canonly be from its adjacentdexels. Hence the first step of prun-ing (by L1 metric), can be made to choose only the pointsfrom the adjacentdexelsof R. The radius ofSR is set to aslightly higher value than the noise in the system. As all thepoints inCR are visible fromR, visibility and angle checkscan also be skipped. By setting the parameters and removingthese tests, it takes less than seven seconds to reconstruct adata set of size around 900,000 points. This performance iscomparable to the performance of previously designed ter-rain triangulation algorithms[14]. Essentially, we can thinkof our approach as a parameterized algorithm where fixingcertain parameters results in highly specialized and efficientalgorithms for different classes of inputs.

In practice, we fix the dimensions of thedexelarray. Weretain one representative point if multiple points get mappedonto the samedexel. This thins the high sampling density,and forms a level of simplification. The dimensions of thedexelarray controls the amount of simplification and the runtime of the algorithm. Since all the processing time in ouralgorithm is dependent on the number ofcandidate points,bounding this number is a major source of speed-up in han-dling terrain data.

The image in Color Plate 2 (top row) shows the texturedreconstructed room model. The texture is created from theintensity values returned by the laser device. The image onthe right shows the micro-facets in the floor of the room, inspite of point simplification.

7. Future Work for TeleconferenceApplication

The strength of our algorithm lies in its speed and memoryrequirements. In telepresence applications[25], where multi-ple cameras are used to extract the geometry of a dynamicscene at interactive rates, the surface reconstruction algo-rithm should cope up with the multi-million point data in-put, simplify it and provide a suitable surface representation.Our algorithm is well-suited for these kinds of applications,and can be used to simplify, reconstruct the surface, mergethe surfaces extracted from different cameras, and stitch thegeometry to create one single geometric representation of

the scene. The following approach can be used to solve thisproblem.

Multiple cameras are located at known locations in the en-vironment to be sampled. The depth extraction can be doneusing any vision algorithm such as a stereo-based approach.Each pixel in the image captured by every camera has adepth value associated with it. As the image with depth fromeach camera can be considered as a height field, our algo-rithm can be used to triangulate the point cloud from eachcamera independently. The merging of geometry from dif-ferent cameras is done as follows. The reconstructed meshof a camera, sayC1, can be transformed to another camera’s(sayC2) viewpoint, as we know the exact location and ori-entation of each camera. The transformed mesh is projectedas adepth maskon thedexel arrayof C2. Thefirst seen sur-faceproperty ensures that there can be no point seen fromC2with a higher depth than its corresponding depth mask value.Points seen by bothC1 andC2 (equal depth values) are safelyeliminated. The triangulation of the rest of the points wouldgive the mesh representation of the geometry visible fromC2 only. The stitching of these meshes is done by consider-ing only theirboundarypoints. The resulting surface can betexture mapped with the color values returned by the cameraimages, to generate a realistic model with fewer data points.Similar algorithms for building complex models from rangeimages have been developed[29, 11]. Our algorithm is notintended to replace these techniques, but to illustrate the ap-plicability of our approach to different classes of surface re-construction problems by setting a few parameters.

8. ImplementationThis section describes the issues we faced while implement-ing our algorithm. We store the all the data points in ourdexel data structure. Each point in thedexelarray has azvalue and a pointer to other vertex attributes.

Generating a unique surface representationirrespective ofthe orientationof a model is an important requirement ofany surface reconstruction algorithm. The result of our algo-rithm is based on the order in which vertices are processed.This, in turn, is dependent on the order of insertion of allthe newfringepoints in the breadth-first-process queue. Anymetric which is independent of the orientation of the modelcan be used for this ordering. In our implementation, we in-sert points into the queue in the same order as we form newtriangles.

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

No.of No. of Init. Time� Recons. Time Recons. Timepoints Triangles (in secs) 1 (in secs) 2 (in secs)

143858 267131 82.508 5.998 1.020883577 1707468 88.554 38.782 6.913

Table 2: Performance of the system for the Room range data set: See Color Plate 2. Reconstruction Time 2: without visibilityand angle criteria check. (�: Includes the reading time of the original data set – 6479713 points)

Our algorithm derives its speed and efficiency from thesimplicity of each of its stages. Further, as most of the com-putation is on the projection plane, a lot of geometric trickscan be used to speed up the process. We have taken care toavoid costly operations like square roots and trigonometricfunctions. The ordering of vertices on a plane around a pointis done by partitioning the points in different quadrants inthe local coordinate system and ordering them independentlyby computing thesin(θ) function using first principles. Weavoid repeated computations as much as possible by strik-ing a trade-off between memory and speed. Once a point ismarked as acompletedpoint, memory used by it is imme-diately freed. As we have optimized our function to orderpoints aroundR, we use the same function to perform someof the visibility pruning steps.

9. Performance and ResultsThe complexity of our algorithm is input sensitive,i.e. timespent is proportional to the model complexity. This can beseen from the results shown in Table 1. The bunny model,which has fewer points than the skidoo model, takes moretime for reconstruction, due to its complexity. Similarly, themannequin model takes almost the same time as that of theclub model, because of high curvature variations and non-uniform sampling in the regions near the nose, eyes and ears.For the same reason, we are able to handle massive data setsof size in the order of millions of points in a few seconds, aswe are making use of height field data properties.

Our algorithm is a single pass algorithm, and does notneed to revisit the triangles once they are formed. We do notproduce any higher dimensional simplices (like tetrahedra[9, 16]) that require their removal to make the model a validmanifold. We also do not change the triangulation once theyare completed.

Reduced memory requirement is another feature of our al-gorithm. Our algorithm does not store the triangles formedduring reconstruction in the main memory. Only those tri-angles which are incident onfringe andboundarypoints areretained, as they are used for visibility pruning. Hence we areable to handle massive models with millions of data points.

Table 1 shows the time taken by our algorithm on variouspoint clouds. The initialization time in the table includes thetime taken to read in the model and initializing the data struc-ture. All the timing measurements in this paper were madeon a 250 MHz, R10000 SGI Onyx2 with 16 GB of mainmemory. Table 2 shows the timing of our algorithm on thelaser data. The initialization time includes the time to read

in the original model of around 6.5 million points, filling upthe data structure and eliminating the points. The two entriesin the table show the timings for two different sizes of thedexel array: 400� 600 and 1000� 1500.

9.1. Limitations of Our Approach

Any projection-based approach gives different triangulationfor different starting points. Our approach also suffers fromthe same limitation. But once the seed point is fixed, thetriangulation is same for any transformation of the model.The second limitation is also common to most surface re-construction algorithms – sharp curvature variations. If thefaces incident on a vertex do not satisfy our criterion of sur-face smoothness, then our algorithm might produce incor-rect triangulations. For under-sampled and extremely non-uniformed models, our algorithm produces spurious modelboundaries, as shown in the Color Plate 2 (bottom row).

10. ConclusionWe have presented a new projection-based surface recon-struction algorithm from unorganized point clouds. The keyfeatures of our method are speed and memory efficiency.Further, it is a single pass algorithm and can make use ofthe characteristics of the data acquisition phase to handlenoisy data. We have demonstrated the application of ouralgorithm on various data sets, including a massive, noisyrange scan model of a room. We have successfully gener-ated valid, non-self-intersecting, orientable manifold surfacemeshes for point clouds of size a few hundred thousand in amatter of tens of seconds. By fixing certain parameters in ouralgorithm, we obtain highly specialized and efficient meth-ods for various input classes. We believe that such a versa-tility and performance without any manual intervention is abig win for our algorithm.

References

1. N. Amenta and M. Bern. Surface reconstruction byvoronoi filtering. InACM Symposium on ComputationlGeometry, 1998.

2. N. Amenta, M. Bern, and M. Kamvysselis. A newvoronoi-based surface reconstruction algorithm. InProceedings of ACM Siggraph, pages 415–421, 1998.

3. D. Attali. r-regular shape reconstruction from unorga-nized points. InACM Symposium on Computationl Ge-ometry, pages 248–253, 1997.

c HPCG and Reasearch Press International 2000.

Gopi and Krishnan / Projection Based Surface Reconstruction

4. C. Bajaj, F. Bernardini, and G. Xu. Automatic recon-struction of surfaces and scalar fields from 3d scans. InProceedings of ACM Siggraph, pages 109–118, 1995.

5. C. Bajaj, F. Bernardini, and G. Xu. Reconstructing sur-faces and functions on surfaces from unorganized 3ddata.Algorithmica, 19:243–261, 1997.

6. F. Bernardini.Automatic Reconstruction of CAD Mod-els and Properties from Digital Scans. PhD thesis,Purdue University, Department of Computer Science,1996.

7. F. Bernardini and C. Bajaj. Sampling and recon-structing manifolds using alpha-shapes. InProc. ofNinth Canadian Conference on Computational Geom-etry, pages 193–198, 1997.

8. E. Bittar, N. Tsingos, and M. P. Gascuel. Automatic re-construction from unstructured data: Combining a me-dial axis and implicit surfaces.Computer GraphicsForum, Proceedings of Eurographics, 14(3):457–468,1995.

9. J. D. Boissonnat. Geometric structures for three-dimensional shape representation.ACM Transactionson Graphics, 3(4):266–286, 1984.

10. B. Curless and M. Levoy. A volumetric method forbuilding complex models from range images. InPro-ceedings of ACM Siggraph, pages 303–312, 1996.

11. B. Curless and M. Levoy. A volumetric methodfor building complex models from range images. InH. Rushmeier, editor,SIGGRAPH 96 Conference Pro-ceedings, Annual Conference Series, pages 303–312.ACM SIGGRAPH, Addison Wesley, Aug. 1996. heldin New Orleans, Louisiana, 04-09 August 1996.

12. H. Edelsbrunner and E. Mucke. Three dimensional al-pha shapes.ACM Transactions on Graphics, 13(1):43–72, 1994.

13. W. Fulks.Advanced Calculus: An introduction to anal-ysis. John Wiley & sons, 1978.

14. M. Garland and P. S. Heckbert. Fast polygonal approx-imation of terrains and height fields. Technical report,CS Dept., Carnegie Mellon U., Sept. 1995.

15. B. Guo, J. Menon, and B. Willette. Surface reconstruc-tion from alpha shapes.Computer Graphics Forum,16(4):177–190, 1997.

16. B. Heckel, A. C. Uva, and B. Hamann. Clustering-based generation of hierarchical surface models. InC. M. Wittenbrink and A. Varshney, editors,IEEE Vi-sualization ’98: Late Breaking Hot Topics Proceedings,pages 41–44, 1998.

17. T. V. Hook. Real-time shaded NC milling display. InProceedings of ACM Siggraph, pages 15–20, 1986.

18. H. Hoppe. Surface Reconstruction from UnorganizedPoint Clouds. PhD thesis, University of Washington,Department of Computer Science, 1994.

19. H. Hoppe, T. Derose, T. Duchamp, J. McDonald, andW. Stuetzle. Surface reconstruction from unorganizedpoint clouds. InProceedings of ACM Siggraph, pages71–78, 1992.

20. H. Hoppe, T. Derose, T. Duchamp, J. McDonald, andW. Stuetzle. Mesh optimization. InProceedings ofACM Siggraph, pages 21–26, 1993.

21. R. Mencl. A graph-based approach to surface recon-struction. Computer Graphics Forum, Proceedings ofEurographics, 14(3):445–456, 1995.

22. R. Mencl and H. Muller. Graph-based surface recon-struction using structures in scattered point sets.Pro-ceedings of CGI ’98, 1998.

23. R. Mencl and H. Muller. Interpolation and approxima-tion of surfaces from three-dimensional scattered datapoints. State of the Art Reports, Eurographics ’98,pages 51–67, 1998.

24. E. P. Mucke. Shapes and Implementations in Three-Dimensional Geometry. PhD thesis, Department ofComputer Science, University of Illinois at Urbana-Champaign, 1993.

25. R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, andH. Fuchs. The office of the future: A unified approachto image-based modeling and spatially immersive dis-plays. InProceedings of ACM Siggraph, pages 179–188, 1998.

26. R. Szeliski and D. Tonnesen. Surface modeling withoriented particle systems. InProceedings of ACM Sig-graph, pages 185–194, 1992.

27. M. Teichmann and M. Capps. Surface reconstructionwith anisotropic density-scaled alpha shapes. InPro-ceedings of IEEE Visualization, pages 67–72, 1998.

28. D. Terzopoulos, A. Witkin, and M. Kass. Constraints ondeformable models: Recovering 3d shape and nonrigidmotion. Artificial Intelligence, 36:91–123, 1988.

29. G. Turk and M. Levoy. Zippered polygon meshes fromrange images. In A. Glassner, editor,Proceedings ofSIGGRAPH ’94 (Orlando, Florida, July 24–29, 1994),Computer Graphics Proceedings, Annual ConferenceSeries, pages 311–318. ACM SIGGRAPH, ACM Press,July 1994. ISBN 0-89791-667-0.

30. R. C. Veltkamp. Boundaries through scattered pointsof unknown density.Graphical Models and Image Pro-cessing, 57(6):441–452, 1995.

c HPCG and Reasearch Press International 2000.