HAL Id: inria-00327027https://hal.inria.fr/inria-00327027

Submitted on 7 Oct 2008

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Principal Component Analysis in CGALAnkit Gupta, Pierre Alliez, Sylvain Pion

To cite this version:Ankit Gupta, Pierre Alliez, Sylvain Pion. Principal Component Analysis in CGAL. [Research Report]RR-6642, INRIA. 2008, pp.13. �inria-00327027�

appor t de r ech er ch e

ISS

N02

49-6

399

ISR

NIN

RIA

/RR

--66

42--

FR+E

NG

Thème SYM

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE

Principal Component Analysis in CGAL

Ankit Gupta — Pierre Alliez — Sylvain Pion

N° 6642

Septembre 2008

Unité de recherche INRIA Sophia Antipolis2004, route des Lucioles, BP 93, 06902 Sophia Antipolis Cedex (France)

Téléphone : +33 4 92 38 77 77 — Télécopie : +33 4 92 38 77 65

Principal Component Analysis in CGAL

Ankit Gupta∗ , Pierre Alliez† , Sylvain Pion‡

Thème SYM � Systèmes symboliquesProjet Geometrica

Rapport de recherche n° 6642 � Septembre 2008 � 13 pages

Abstract: Principal component analysis is a basic component of many geometric computingand processing algorithms. It is most commonly used on point sets, although applicable aswell to sets of arbitrary primitives through the computation of covariance matrices. In thispaper we provide closed form formulas of covariance matrices for sets of 2D and 3D geometricprimitives such as segments, circles, triangles, iso rectangles, spheres, tetrahedra and isocuboids. We also describe the method of deriving covariance matrices for their dimensionalvariants such as disks, balls etc. We �nally discuss the �exibility and added value of thepresent approach by discussing its potential use in applications. Our implementation willbe available through the next release of the CGAL library.

Key-words: Principal Component Analysis, geometric primitives.

∗ IT Bombay† INRIA‡ INRIA

Analyse en composante principale dans CGAL

Résumé : L'analyse en composantes principales est un outil de base pour le calcul géométriqueet le traitement numérique de la géométrie. On l'utilise pour estimer des normales àpartir de nuages de points, pour calculer un tenseur d'inertie d'un solide, ou encore pourl'approximation de surfaces. Bien que l'analyse en composantes principales soit utilisée leplus souvent à partir de nuages de points, on peut l'appliquer à tout ensemble de primitivesgéométriques (sans les échantillonner au préalable) en calculant la matrice de covarianceassociée en forme close. On décrit dans cet article les formules des matrices de covarianced'ensembles de primitives géométriques 2D et 3D telles que des segments, cercles, disques,triangles, rectangles alignés avec les axes, sphères, boules, tétraèdres et parallélépipèdesalignés avec les axes. On discute la valeur ajoutée apportée par une telle approche ainsique leur utilisation potentielle dans les applications. L'implantation associée sera disponibledans la prochaine version de la bibliothèque CGAL.

Mots-clés : Analyse en composantes principales.

PCA in CGAL 3

1 Introduction

Principal Component Analysis (PCA) is a data analysis tool for determining a linear sub-space of input data such that the variance of the projection of the data onto the subspaceis maximized [Jol02]. More speci�cally, PCA amounts to compute an orthogonal coordinatesystem such that the greatest variance of the orthogonal projection of the data lies on the�rst coordinate (so-called principal component), the second greatest variance lies on thesecond coordinate, and so on. Equivalently, the linear subspaces spanned by the principalcomponents minimize the sum of squared distances from the data to their projection ontothese subspaces. At the intuitive level, PCA provides a way to reduce the dimensionality ofcomplex data so as to reveal the simpli�ed structure that underlies them.

PCA is one of the most popular tool for data analysis, used in applications ranging fromstatistics to geometric computing through mechanical engineering. For geometric applica-tions, PCA is commonly used either in data space or in a transformed space for dimen-sionality reduction, normal estimation [MNG04], surface reconstruction [Hop94], surfaceapproximation [CSAD04], shape alignment and matching [WBZ05], inertia tensors [BB04],to cite a few. PCA is mostly used on point set data in statistics and geometric applications,or on polyhedra in mechanics. Although one way to generalize it to arbitrary objects canbe achieved through uniform sampling, it is more accurate and often faster to derive closedforms when they exist. In this paper we specialize these derivations to sets of various 2Dand 3D geometric primitives, and list potential geometry processing applications.

2 Point Sets

In principal component analysis, least squares analysis is used to determine the requiredlinear subspace. The main idea is to compute the covariance matrix of the points in thepoint set, perform its eigen decomposition and then determine an n-dimensional subspacealong the eigenvectors corresponding to the top n eigenvalues. Eigenvectors correspondingto large eigenvalues are the directions in which the data has a strong component, or equiv-alently large variance. We now brie�y describe this method.

Let P = {pi}ni=1 be a set of points in 3D and c their center of mass. Consider the samepoint set P but now centered at the center of mass {qi = pi − c}ni=1. The covariance matrixof P is de�ned as:

CP =1n

n∑i=1

x2i xiyi xizi

xiyi y2i yizi

xizi yizi z2i

=1n

n∑i=1

qiqTi

After an eigendecomposition of C, choosing the eigenvector corresponding to the top n

eigenvalues gives us n principal components.

Now consider P = {si}ni=1 where each si is a continuous set of points in the spaceresulting in an object such as a segment, triangle, tetrahedron etc. If x represents the

RR n° 6642

4 Gupta, Alliez, Pion

coordinate vector of points in space relative to the center of mass of the set of objects P ,the covariance matrix Ci of each si is,

Ci =∫

si

xxT dx

For the set P , the covariance matrix is de�ned as C =∑n

i=1 Ci. We can then similarly do aneigen-decomposition of C and choose the eigenvector corresponding to the top n eigenvaluesto give us n principal components. Thus, performing Principal Components Analysis on aset of objects requires calculating the covariance matrix corresponding to each object andusing it to calculate the covariance matrix of the object set as a whole. We now describe amethod to achieve this goal for a set of geometric objects.

3 Other Geometric Primitives

In this section, we describe the algorithm used for constructing the covariance matrix ofany object set. We use a set of 2D triangles as an example to illustrate the algorithm. Thenotation used is as follows: if u refers to any object or set of objects, Mu is the second ordermoment of u with respect to the origin; mu is the mass of u (area in 2D, volume in 3D), cu

is the center of mass of u and Cu is the covariance matrix of u.

For each type of object s, we �rst construct a canonical geometry of the same classwhose covariance matrix can be easily calculated and which can be mapped onto an ar-bitrary object of this class. We call such a geometry a standard object. Suppose o isa standard object corresponding to the class of objects like s. For the class of trianglesst = {(x1, y1), (x2, y2), (x3, y3)}, the standard object is ot = {(0, 0), (1, 0), (0, 1)}. The sec-

ond order moment of ot with respect to the origin is Mot =[

1/12 1/241/24 1/12

]. Next, we �nd

an a�ne transformation that maps the standard object onto an arbitrary object. Let xo bethe point coordinates on o and xs, the coordinates of corresponding points on s; we need to�nd the a�ne transformation matrix As and the o�set vector Vs such that xs = Asxo + Vs.

Thus, for the class of triangles, At =[

x2 − x1 x3 − x1

y2 − y1 y3 − y1

]and Vt = [x1, y1]T . Using this

transformation and the second order moment Mo of the standard geometry, we can �nd thesecond order moment Ms with respect to the origin of the arbitrary object.

Ms =∫

s

xsxTs dxs.

=∫

s

(Asxo + Vs)(Asxo + Vs)T dxs

=∫

s

(Asxo + Vs)(xTo AT

s + V Ts )dxs

INRIA

PCA in CGAL 5

Ms =∫

s

(AsxoxTo AT

s + AsxoVTs + Vsx

To AT

s + VsVTs )dxs

= As

(∫o

xoxTo

ms

modxo

)AT

s +(∫

s

Asxo dxs

)V T

s

+Vs

(∫s

(Asxo)T dxs

)+ VsV

Ts

(∫s

dxs

)=

(ms

mo

)(AsMoA

Ts ) +

(∫s

(xs − Vs) dxs

)V T

s

+Vs

(∫s

(xs − Vs)T dxs

)+ VsV

Ts ms

=(

ms

mo

)(AsMoA

Ts ) + ms(xs − Vs)V T

s + Vsms(xTs − V T

s )

+VsVTs ms where xs is the centroid

Ms =(

ms

mo

)(AsMoA

Ts ) + ms

(csV

Ts + Vsc

Ts − VsV

Ts

)For the set of triangles, mo = 1/2 and ms = 4s = 1/2 |At|; where 4s is its area. Thus,Ms = |At|AtMotA

Tt + 1

2 |At|(csV

Tt + Vtc

Ts − VtV

Tt

).

Now consider a set P = {si}ni=1 of objects.

MP =n∑

i=1

Msi

mP =n∑

i=1

msi

cP =1

mP

(n∑

i=1

msicsi

)

RR n° 6642

6 Gupta, Alliez, Pion

To calculate the combined covariance matrix CP ,

CP =∫

P

(xP − xP )(xP − xP )T dxP .

=∫

P

(xP − xP )(xTP − xP

T )dxP .

=∫

P

xP xTP − xP xT

P − xP xPT + xP xP

T dxP .

= MP − xP

∫P

xTP dxP −

∫P

xP dxP xPT

+xP xPT

∫P

dxP .

= MP −mP xP xPT .

CP = MP −mP cP cTP .

Thus, the closed form formula for the covariance matrix of a set of objects is given by,

CP =n∑

i=1

[(msi

mo

)(AsiMoA

Tsi) + msi

(csiV

Tsi + Vsic

Tsi − VsiV

Tsi

)]

− 1(∑n

i=1 msi)

(n∑

i=1

msicsi

)(n∑

i=1

msicTsi

)

The covariance formula above is general and applies to arbitrary sets of primitives. Ap-pendix A and B show several standard objects and their associated order-2 moment w.r.t.origin, transformation matrix and o�set vectors. It is interesting to note that when theunion of all primitives can be decomposed into a set of primitives sharing the same basepoint as center of mass, the formula simpli�es to the sole a�ne transform part. Let cP bethe center of mass of the union of all primitive objects in the set such that all of them havea vertex at cP . Translate the axes such that the origin is at cP . Thus, the standard objecto′ with respect to the new origin is given by xo′ = xo + cp and its second order momentw.r.t cP is Mo′ = Mo. Now, we just need to scale o′ to �t all objects in the set. Thus, wecan �nd a matrix A such that xs = Asxo′ . So, the covariance of the set P is given by

CP =n∑

i=1

(msi

mo

)AsiMoA

Tsi

A concrete example use of this formula has been used in [ACSTD07] to calculate thecovariance matrix of unions of Voronoi cells for surface normal estimation from noisy pointsets. Consider a bounded Voronoi cell V . The covariance matrix of a Voronoi Cell can becalculated by �rst computing its center of mass o and decomposing it into tetrahedra. The

INRIA

PCA in CGAL 7

decomposition is performed in a star fashion such that all tetrahedra ti share the same basepoint which coincides with o. This decomposition is always possible as the Voronoi cell isconvex.Since all tetrahedra consist of a common vertex at the center of mass of the voronoi

Figure 1: A�ne Transform of a standard tetrahedron to an arbitrary one

cell, we can transform the origin to the center of mass. For a single arbitrary tetrahedrons = (a, b′, c′, d′), o = (a, b, c, d) is the standard geometry. Note that s and o have a commonvertex a which is the center of mass of the voronoi cell. For an a�ne transformation to mapo onto s, the transformation matrix As is de�ned as:

As =

(b′ − a′).x (c′ − a′).x (d′ − a′).x(b′ − a′).y (c′ − a′).y (d′ − a′).y(b′ − a′).z (c′ − a′).z (d′ − a′).z

The order-2 moment matrix Mo of the canonical standard tetrahedron o with respect to

the origin is:

Mo =∫ 1

x=0

∫ 1−x

y=0

∫ 1−x−y

z=0

x2 xy xzyx y2 yzzx zy z2

dzdydx

=1

120

2 1 11 2 11 1 2

The mass of s is ms = |As|/6 and the mass of o is mo = 1/6. For the voronoi cell Vdecomposed into a set of tetrahedra V = {si}ni=1, its covariance matrix CP thus is,

CP =n∑

i=1

|Asi |AsiMoATsi

RR n° 6642

8 Gupta, Alliez, Pion

4 Implementation

The algorithm described in this paper is generic and can be applied to various types ofgeometries. This power will be available in the next release of the CGAL library [c] for most�nite primitives of the 2D and 3D kernel: 2D and 3D Points, 2D and 3D segments, 2D and3D triangles, 2D rectangles, 3D cuboids, 3D tetrahedra, 2D circles and 3D spheres.

The API follows the STL paradigm. The user must at least provide an iterator rangeof primitives, and the type of linear subspace to be �tted (2D line, 3D line, or 3D plane).The function returns a scalar ∈ [0; 1] indicating the quality of the �t related to ratios of thecovariance eigenvalues (0 for isotropic data sets, 1 for zero error �t). For example one can�t a line to a 2D point set by calling the function CGAL::linear_least_squares_�tting_2(-points.begin(),points.end(),line) or a 3D line to a set of tetrahedra by callingCGAL::linear_least_squares_�tting_3(tetrahedra.begin(),tetrahedra.end(),line). or a 3D planeto a set of spheres by calling CGAL::linear_least_squares_�tting_3(spheres.begin(),spheres.end(),-plane). The user can �t a point as well by calling CGAL::centroid(begin,end).

Furthermore, the user can provide an additional optional argument specifying the di-mension of the primitive to be considered. By default, this argument is the dimension ofthe primitives in the container (0 for points, 1 for segments, 2 for triangles, rectangles andspheres, 3 for tetrahedra and cuboids). The user can, for e.g., �t a linear subspace to a setof balls (instead of spheres) by specifying 3 as dimension, or can �t the edges of a set oftetrahedra by specifying 1 as dimension, or the boundary triangles of a set of tetrahedra byspecifying 2, without having to �ll another container of primitives. For cases where the userwants to �t more than just one primitive we also propose the functions std::pair<Line,Plane>CGAL::least_squares_line_plane_�tting_3(begin,end) and CGAL::Triple<Point,Line,Plane>CGAL::least_squares_point_line_plane_�tting_3(begin,end)

5 Applications

From the general application point of view, the main added value of the present geomet-ric computations becomes relevant when the discretization of a shape is provided by morecomplex primitives than points, such as segments, triangles, tetrahedra, etc. The proposedclosed forms are more accurate and often more e�cient than the common uniform pointsampling followed by point-based PCA.

Example potential applications are shape alignment (i.e., �nding canonical frames) andcomputation of moment of inertia for surface triangle meshes, without having to uniformlypoint sample the mesh. The same would be valid for embedded graphs where the primitiveswould be 2D or 3D segments for the graph edges, or for solids once properly discretized intotetrahedra. For various data analysis applications, a common approach consists of carryingon PCA computations on point sets living in a so-called feature space. Instead of weighting

INRIA

PCA in CGAL 9

the points as commonly done, one could replace these weighted points by balls.

One recent trend consists of elaborating upon integral approaches for geometric dataanalysis and processing (see e.g., [PWY∗07]), in order to reduce the dependence to theinput discretization. This work can be seen as one step further in the same direction.

6 Conclusion

This work proposes closed-form computations of covariance matrices of sets of various 2Dand 3D geometric primitives, together with a generic implementation for the next releaseof the CGAL library. The main added value is to remove the need for the user to pointsample a shape in order to perform approximate PCA computations, and to perform directcomputations over more elaborate discretizations.

As future work we plan to extend these closed forms to a richer set of primitives, suchas sets of anisotropic versions of the current handled primitives or primitives in higherdimension. We plan to enrich the API so as to handle containers of mixed primitives withhomogeneous dimensions, so as to �t linear subspaces e.g. to sets of tetrahedra, cuboids andballs. Another extension is to handle weighting functions, such as radial Gaussian kernelsused, e.g., for normal estimation or MLS surface �tting. Finally, we want to extend this workto unions of primitives, as well as to arbitrary Boolean operations over sets of primitives.

References

[ACSTD07] Alliez P., Cohen-Steiner D., Tong Y., Desbrun M.: Voronoi-basedvariational reconstruction of unoriented point sets. In Proceedings of the Fifth

EUROGRAPHICS Symposium on Geometry Processing (2007), Belyaev A. G.,Garland M., (Eds.), vol. 257, pp. 39�48.

[BB04] Blow J., Binstock A. J.: How to �nd the inertia tensor (or other massproperties) of a 3d solid body represented by a triangle mesh, 2004.

[c] CGAL, Computational Geometry Algorithms Library. http://www.cgal.org.

[CSAD04] Cohen-Steiner D., Alliez P., Desbrun M.: Variational shape approxima-tion. ACM Trans. Graph 23, 3 (2004), 905�914.

[Hop94] Hoppe H.: Surface Reconstruction from Unorganized Points. PhD thesis,Dept. of Computer Science and Engineering, U. of Washington, 1994.

[Jol02] Jolliffe I. T.: Principal Component Analysis. Springer-Verlag, New York,2002.

RR n° 6642

10 Gupta, Alliez, Pion

[MNG04] Mitra N. J., Nguyen A., Guibas L.: Estimating surface normals in noisypoint cloud data. In special issue of International Journal of Computational

Geometry and Applications (2004), vol. 14 (4�5), pp. 261�276.

[PWY∗07] Pottmann H., Wallner J., Yang Y.-L., Lai Y.-K., Hu S.-M.: Principalcurvatures from the integral invariant viewpoint. Comput. Aided Geom. Design

24 (2007), 428�442.

[WBZ05] Wang B., Bangham J. A., Zhu Y.: Shape retrieval by principal componentsdescriptor. In Pattern Recognition and Image Analysis, Third International

Conference on Advances in Pattern Recognition 2005 (2005), Singh S., SinghM., Apté C., Perner P., (Eds.), vol. 3687, Springer, pp. 626�634.

A Standard Geometries of Various Shapes

(0,1)

(0, 0) (1,0) x axis

y axis

Std. 2D Segment: {(1, 0), (0, 1)}.

(0,1)

(0,0) (1,0) x axis

y axis

Std. 2D Triangle: {(0, 0), (1, 0), (0, 1)}.

INRIA

PCA in CGAL 11

(0, 1)

(0,0)

(1, 0)

x axis

y axis

1

Std. 2D Circle/Disk:{center = (0, 0), r = 1}

(0,1)

(0,0) (1,0) x axis

y axis

(0,0)

Std. 2D Axis-Aligned Rectangle: {(0, 0), (1, 1)}.

(0, 0, 1)

(0, 0, 0)

(1,0,0) x axis

z axis

(0,1,0)

y axis

Std. 3D Segment: {(1, 0, 0), (0, 1, 0)}.

(0,0,1)

(0, 0, 0)

(1,0,0) x axis

z axis

(0,1,0)

y axis

Std. 3D Triangle: {(1, 0, 0), (0, 1, 0), (0, 0, 1)}.

RR n° 6642

12 Gupta, Alliez, Pion

(0,0,0) (1,0,0)

x axis

(0,0,1)

(0,1,0)

y axisz axis

Std. 3D Sphere: {center = (0, 0, 0), r = 1}.

(0,0,1)

(0,0,0)

(1,0,0) x axis

z axis

(0,1,0)

y axis

Std. 3D Tet:{(0, 0, 0), (1, 0, 0), (0, 1, 0), (0, 0, 1)}.

(1,1,1)

(0,0,0)

(1,0,0) x axis

z axis

y axis

(0,0,1)

(1,1,0)

(0,1,1)

Std. 3D Cuboid: {(0, 0, 0), (1, 1, 1)}.

INRIA

PCA in CGAL 13B

Data

Sheet

Sr.No.

Shape

Order-2MomentofStandard

Transformation

O�set

Objectaboutorigin

Matrix

Vector

12D

Segments

M=[√

2/3√

2/6

√2/6√

2/3

]A

=[ x

1x2

y1

y2

]V

=[0,0]T

si

={(x

1,y1),

(x2,y2)}

22D

Triangle

M=[ 1

/12

1/24

1/24

1/12

]A

=[ x

2−x1

x3−x1

y2−y1

y3−y1

]V

=[x

1,y1]T

ti

={(x

1,y1),

(x2,y2),

(x3,y3)}

32D

Disk

M=[ π

/4

00

π/4

]A

=[ r

00

r

]V

=[xc,yc]T

ci

={center

=(xc,yc),radius

=r}

42D

Circle

M=[ π

00

π

]A

=[ r

00

r

]V

=[xc,yc]T

ci

={center

=(xc,yc),radius

=r}

52D

axis-alignedRectangle

M=[ 1

/3

1/4

1/4

1/3

]A

=[ x

2−x1

00

y2−y1

]V

=[x

1,y1]T

ri

={(x

1,y1),

(x2,y2)}

[x1,y1]T

=left-bottom

corner

[x2,y2]T

=right-topcorner

63D

Segments

M=

√2/3√

2/6

0√

2/6√

2/3

00

01

A

=

x1

x2

0y1

y2

0z1

z2

1

V

=[0,0,0]T

si

={(x

1,y1,z1),

(x2,y2,z2)}

73D

Triangle

M=

1/12

1/24

1/24

1/24

1/24

1/12

1/24

1/24

1/12

A

=

x1

x2

x3

y1

y2

y3

z1

z2

z3

V

=[0,0,0]T

ti

={(x

1,y1,z1),

(x2,y2,z2),

(x3,y3,z3)}

83D

SolidSphere

M=

4π/15

00

04π/15

00

04π/15

A

=

r0

00

r0

00

r

V

=[xc,yc,zc]T

ci

={center

=(xc,yc,zc),radius

=r}

93D

Tetrahedron

M=

1/60

1/120

1/120

1/120

1/60

1/120

1/120

1/120

1/60

A

=

x2−x1

x3−x1

x4−x1

y2−y1

y3−y1

y4−y1

z2−z1

z3−z1

z4−z1

V

=[x

1,y1,z1]T

ti

={(x

1,y1,z1),...,

(x4,y4,z4)}

10

3D

axis-alignedSolidCuboid

M=

1/3

1/4

1/4

1/4

1/3

1/4

1/4

1/4

1/3

A

=

x2−x1

00

0y2−y1

00

0z2−z1

V

=[x

1,y1,z1]T

ri

={(x

1,y1,z1),

(x2,y2,z2)}

[x1,y1,z1]T

,[x2,y2,z2]T

=closest,farthestdiagonalcorners

11

3D

axis-alignedHollow

Cuboid

M=

7/3

3/2

3/2

3/2

7/3

3/2

3/2

3/2

7/3

A

=

x2−x1

00

0y2−y1

00

0z2−z1

V

+[x

1,y1,z1]T

ri

={(x

1,y1,z1),

(x2,y2,z2)}

[x1,y1,z1]T

,[x2,y2,z2]T

=closest,farthestdiagonalcorners

RR n° 6642

Unité de recherche INRIA Sophia Antipolis2004, route des Lucioles - BP 93 - 06902 Sophia Antipolis Cedex (France)

Unité de recherche INRIA Futurs : Parc Club Orsay Université - ZAC des Vignes4, rue Jacques Monod - 91893 ORSAY Cedex (France)

Unité de recherche INRIA Lorraine : LORIA, Technopôle de Nancy-Brabois - Campus scientifique615, rue du Jardin Botanique - BP 101 - 54602 Villers-lès-Nancy Cedex (France)

Unité de recherche INRIA Rennes : IRISA, Campus universitaire de Beaulieu - 35042 Rennes Cedex (France)Unité de recherche INRIA Rhône-Alpes : 655, avenue de l’Europe - 38334 Montbonnot Saint-Ismier (France)

Unité de recherche INRIA Rocquencourt : Domaine de Voluceau - Rocquencourt - BP 105 - 78153 Le Chesnay Cedex (France)

ÉditeurINRIA - Domaine de Voluceau - Rocquencourt, BP 105 - 78153 Le Chesnay Cedex (France)

http://www.inria.fr

ISSN 0249-6399