Functional correspondence by matrix completion

Artiom Kovnatsky1, Michael M. Bronstein1,Xavier Bresson2, and Pierre Vandergheynst2

1Faculty of Informatics, Università della Svizzera italiana (USI)2Signal Processing Lab, École Polytechnique Fédérale de Lausanne (EPFL)

Abstract

In this paper, we consider the problem of finding denseintrinsic correspondence between manifolds using the re-cently introduced functional framework. We pose the func-tional correspondence problem as matrix completion withmanifold geometric structure and inducing functional lo-calization with the L1 norm. We discuss efficient nu-merical procedures for the solution of our problem. Ourmethod compares favorably to the accuracy of state-of-the-art correspondence algorithms on non-rigid shape match-ing benchmarks, and is especially advantageous in settingswhen only scarce data is available.

1. Introduction

Correspondence problems are, without doubt, among themost important problems that have to be dealt when an-alyzing highly-complex geometric data. Establishing cor-respondence between images is a cornerstone of many al-gorithms in computer vision and image analysis [50]. Ingeometry processing and graphics, finding intrinsic corre-spondence between deformable 3D shapes is one of thefundamental problems with a plethora of applications rang-ing from texture mapping to animation [46]. In machinelearning applications, correspondence problems in high-dimensional spaces arise in multi-modal data analysis prob-lems such as manifold alignment and multi-view clustering[49, 13].

Related work A wide class of correspondence methodsaims at minimizing some structure distortion, which can in-clude similarity of local features [30, 9, 2, 52, 22], geodesic[25, 7] or diffusion distances [11, 8], or a combinationthereof [44]. Zeng et al. [54] used higher-order structures.Typically, the computational complexity of such methods ishigh, and there have been several attempts to alleviate thecomputational complexity using hierarchical [37] or smartsampling [43] methods. Several approaches formulate the

correspondence problem as an NP-hard quadratic assign-ment and propose different relaxations thereof [45, 19, 34].A recent trend is to use machine learning techniques, suchas random forests, to learn correspondences [38, 36].

Another important class of methods strives to embed theintrinsic structure of the manifold into some other space,where the correspondence can be parametrized with a smallnumber of degrees of freedom. Elad and Kimmel [12] usedmulti-dimensional scaling to embed the geodesic metricof the matched manifolds into the Euclidean space, wherealignment of the resulting “canonical forms” is then per-formed by simple rigid matching (ICP) [10, 3, 27]. Ma-teus et al. [23] used the first eigenfunctions of the Laplace-Beltrami operator as embedding coordinates. A more re-cent method of spectral kernel map [39] follows this path.Lipman et al. [21, 15, 16] used conformal embeddings toparametrize correspondence as a Möbius transformation.

More recently, there is an emerging interest in soft cor-respondence approaches. Several methods formulated softcorrespondence as a mass-transportation problem [24, 41].Ovsjanikov et al. [29] introduced the functional correspon-dence framework, modeling the correspondence as a lin-ear operator between spaces of functions on two manifolds,which has an efficient representation in the Laplacian eigen-bases. In such a formulation, the problem of finding corre-spondence is reduced to solving a linear system of equa-tions given some known set of corresponding functions onthe two manifolds. Pokrass et al. [33] extended this ap-proach to the case when the ordering of the correspondingfunctions is unknown, and introduced a regularization onthe diagonal structure of the correspondence matrix repre-sentation in the Laplacian eigenbases. Kovnatsky et al. [18]proposed constructing coupled bases by simultaneous diag-onalization of Laplacians. In such bases the correspondenceis represented by an approximately diagonal matrix, whichallows to solve the linear system of equations for diagonalelements only.

Main contribution Our paper deals with the problem offinding dense intrinsic correspondence between manifolds

in the functional formulation. We propose treating func-tional correspondence as geometric matrix completion. Ourapproach is inspired by the recent work on recovery of ma-trices on graphs [14], which introduced geometric struc-ture into the matrix completion problem. Treating the rowsand columns of the functional correspondence matrix asvector-valued functions on the respective manifolds, we in-troduce geometric structure using the Dirichlet energy. Weshow that our method includes the previous approachesof [29, 18] as particular settings, and compares favorablyto state-of-the-art methods for non-rigid shape correspon-dence on the challenging Princeton benchmark [16]. Theadvantage of our method is especially pronounced in thescarce data settings. The proposed method is completelygeneric and can be applied to any manifolds and high-dimensional geometric data.

2. BackgroundNotation In this paper, we use bold capital letters to de-note matrices, bold lower-case letters for vectors, and italiclower-case letters for scalars. Given a matrix A, we de-

note by ‖A‖F =(∑

ij a2ij

)1/2its Frobenius norm, by

‖A‖1 =∑ij |aij | its L1-norm, and by ‖A‖∗ =

∑i σi

its nuclear (or trace) norm, where σ1, σ2, . . . denote the sin-gular values of A. Alternatively, provided a decompositionof the form A = UV>, the nuclear norm can be writtenas ‖A‖∗ = 12 (‖U‖

2F + ‖V‖2F) [42]. We denote by δi the

Kronecker delta, i.e. a unit vector with one at position i andall the rest zeros.

Laplacians Let us be given a Riemannian manifold sam-pled at n points X = {x1, . . . ,xn} ⊆ Rd. The localstructure of the manifold is modeled as an undirected graph(X,E) in case of abstract data, or as a simplicial complex(triangular mesh) (X,E, F ) in case of 3D shapes, whereE ⊆ X × X and F ⊂ X3 denote the edges and faces,respectively. We denote by L2(X) ∼= Rn the space of realfunctions f = (f(x1), . . . , f(xn)) on the discrete manifold.

We define the Laplacian of X as L = A−1(D −W), where D = diag(

∑k 6=i wki), and W and A =

diag(a1, . . . , an) are some edge- and vertex weights, re-spectively. The definition of the weights depends on theparticular discretization of the manifold. In this paper, weconsider two types of objects (see Figure 1):

Graphs, for which we use the Gaussian edge weights

wij =

{e−‖xi−xj‖

2/2σ2 ij ∈ E;0 else,

(1)

and construct the edge set E using K-nearest neighbors.The vertex weight is defined as A = D (random walkLaplacian) or A = I (unnormalized Laplacian) [48]. 3Dshapes represented as point clouds are treated in this way.

xi

xj

wij

xi

xj

wij

αij

βij

ai

Figure 1. Laplacian construction for a graph (left) and a triangularmesh (right).

Triangular meshes, for which we use the discretizationof the Laplace-Beltrami operator [20] based on the classicalcotangent weights [32, 26]

wij =

{(cotαij + cotβij)/2 ij ∈ E;0 else,

(2)

where αij , βij are the angles in front of the edge ij. Thevertex weights are defined as local area elements ai =13

∑jk:ijk∈F area(xixjxk), equal to one third of the sum of

the one-ring triangles areas (marked in green in Figure 1).

Harmonic analysis on manifolds The eigenvectors Φ =(φ1, . . . ,φn) of the Laplacian satisfying LΦ = ΦΛ (hereλ1 ≤ λ2 ≤ . . . ≤ λn are the corresponding eigenval-ues and Λ = diag(λ1, . . . , λn)) form an orthonormal basisthat generalizes the Fourier basis to non-Euclidean spaces.Given a function f ∈ L2(X), its Fourier coefficients arecomputed as α = Φ>f . Taking the first k coefficients, oneobtains a smooth (“low-pass”) approximation of the func-tion f ≈

∑kl=1 αlφl = ΦkΦ

>k f , where Φk = (φ1, . . .φk).

Compressed manifold modes The Laplacian eigenbasiscan be found by minimizing the Dirichlet energy,

minΦ

tr(Φ>LΦ) s.t. Φ>Φ = I. (3)

Neumann et al. [28], following Ozoliņš et al. [31], proposedconstructing localized approximate eigenbases on mani-folds (referred to as compressed manifold modes or CMM)by imposing the L1-penalty on ‖Φ‖1, which, in combina-tion with the Dirichlet energy makes the basis functions lo-calized,

minΦ

tr(Φ>LXΦ) + µ‖Φ‖1 s.t. Φ>Φ = I, (4)

for µ > 0.

Functional maps Let X and Y denote two manifoldssampled at n and m points, respectively. Ovsjanikov et al.[29] propose to model the functional correspondence be-tween the spacesL2(X) andL2(Y ) as them×nmatrix T0,which maps a function f ∈ L2(X) into T0f = g ∈ L2(Y ).

Traditional point-wise correspondence is a particular caseof this model wherein T0 maps delta functions into deltafunctions.

Let LX and LY be the Laplacians of X and Y , andlet Φk = (φ1, . . .φk) and Ψk = (ψ1, . . .ψk) be the re-spective truncated Laplacian eigenbases. Let us be givenq corresponding functions F = (f1, . . . , fq) and G =(g1, . . . ,gq) satisfying T0F = G, where T0 is the un-known groundtruth correspondence. The functional corre-spondence can be approximated in these bases as a rank-kmatrix T = ΨkCΦ>k , where C is a k×k matrix translatingFourier coefficients from the basis Φk to the basis Ψk.

The matrix C can be found by solving the system ofequations G = ΨkCΦ>k F in the least-squares sense,

minC‖CΦ>k F−Ψ>k G‖2F. (5)

Note that since the groundtruth correspondence T0 is un-known, in practice the matrices F,G must be computedindependently on X and Y such that T0F ≈ G. In thesimplest case, F = (δi1 , . . . , δiq ),G = (δj1 , . . . , δjq ) rep-resent some known point-wise correspondences (“seeds”)between points xi1 , . . . ,xiq and yj1 , . . . ,yjq . In shape cor-respondence applications, F,G are typically computed us-ing some intrinsic shape descriptor such as HKS [30], WKS[2], MeshHOG [52], ShapeMSER [22], which is invariantto shape deformations.

For applications requiring point-wise maps, Ovsjanikovet al. [29] devise an iterative conversion scheme similar toiterative closest point (ICP) methods [10, 3, 27]. Think-ing of Φk and Ψk as k-dimensional point clouds (withn and m points, respectively), the matrix C can be in-terpreted of as an alignment between them. Starting withsome initial matrix C, first find the closest row ji in ΨkCfor each ith row of Φk (this step is equivalent to closestpoint correspondence in ICP). Second, find an orthonor-mal C minimizing

∑i ‖(Φk)i − (ΨkC)ji‖ (this is equiv-

alent to the alignment step in ICP). The process is repeateduntil convergence, producing point-wise correspondences(x1,yj1), . . . , (xn,yjn).

Finally, note that in order for system (5) to be (over-)determined, we must have k ≤ q, where q is the numberof given corresponding functions (data). At the same time,the quality of the correspondence depends on k: the morebasis elements are taken, the better, since due to truncationof the Fourier expansion, small k results in poor spatial lo-calization of the correspondence. Furthermore, truncatedFourier series of a discontinuous function manifest oscilla-tions, a behavior known in harmonic analysis as the “Gibbsphenomenon” (see examples in Figure 5).

Permuted sparse coding Pokrass et al. [33] addressedthe case when the ordering of the columns of F,G is un-known and is modeled by a k × k permutation matrix Π.

n

m

ti′′ ti′ ti

xi

xi′

xi′′

yj

yj′

yj′′xi′′

Figure 2. Illustration of functional correspondence as matrix com-pletion. Column ti = Tδi of matrix T is the functional map of adelta at xi. Geometric structure is imposed on the columns to en-sure smoothness, i.e. ti, ti′ corresponding to spatially close pointsxi, xi′ on X are similar.

Furthermore, they noted that for near-isometric manifolds,the eigenbases satisfy φ>i T

>0 ψj ≈ ±δij , i.e., the matrix C

is approximately diagonal. The diagonal structure of C wasinduced using an L1 penalty,

minΠ,C‖CΦ>k FΠ−Ψ>k G‖2F + µ‖M ◦C‖1, (6)

where µ > 0 and M is a weight matrix with zero diagonaland ◦ denotes the Hadamard (element-wise) matrix multi-plication. The problem was solved by means of alternatingminimization w.r.t. Π (which turns out a linear assignmentproblem) and C (which is equivalent to sparse coding).

Coupled diagonalization Kovnatsky et al. [18] notedthat the matrix C encoding the correspondence dependson the choice of bases, and proposed finding new approx-imate eigenbases Φ̂k = ΦkP and Ψ̂k = ΨkQ by meansof k × k orthonormal matrices P,Q in which the Fouriercoefficients of F and G are as similar as possible. Thenew bases behave as eigenbases if they approximately di-agonalize the respective Laplacians, i.e., Φ̂>k LXΦ̂k =P>Φ>k LXΦkP = P

>ΛX,kP (respectively, Ψ̂>k LY Ψ̂k =Q>ΛY,kQ), where ΛX,k = diag(λX1 , . . . , λ

Xk ) and

ΛY,k = diag(λY1 , . . . , λ

Yk ). This property is enforced in

the optimization problem

minP,Q

‖F>ΦkP−G>ΨkQ‖2F + µ1off(P>ΛX,kP)

+µ2off(QΛY,kQ>) s.t. P>P = Q>Q = I, (7)

(with µ1, µ2 > 0) using the off-diagonal penalty off(A) =∑i 6=j a

2ij .

After P, Q are found, in the new bases ΦkP, ΨkQ thematrix C is approximately diagonal. Then, the system ofequations CP>Φ>k F = Q

>Ψ>k G is solved for diagonalelements of C only.

3. Functional maps as matrix completion

Kalofolias et al. [14] studied the problem of matrix com-pletion on graphs, where the rows and columns of the ma-trix representing the correspondence have underlying geo-metric structure. They show that adding geometric structureto the standard matrix completion problem improves recov-ery results.

We use the same philosophy to formulate the problem offinding a functional map as matrix completion, whose rowsand columns are interpreted as functions on the respectivemanifolds X and Y . For this purpose, we consider the ma-trix T as a collection of columns T = (t1, . . . , tn) or rowsT = (t1>, . . . , tm>)>, where ti and tj denote the ith col-umn and jth row of T, respectively. The column ti = Tδiis the function on Y corresponding to a delta located at pointxi on X . Similarly, the row tj = δ>j T is the function on Xcorresponding to a delta located at point yj on Y .

As in the classical matrix completion problem, we aim atrecovering the unknown correspondence matrix T0 from afew observations of the form T0F = G, looking for a ma-trix T ≈ T0 that explains the data in a “simple” way, in thesense discussed in the following. Our problem comprisesthe following terms:

Data term The correspondence should respect the data,which is achieved by minimizing ‖TF−G‖F.

Smoothness The correspondence must be smooth, in thesense that if xi, xi′ are two close points on X , then the re-spective corresponding functions are similar, i.e., ti ≈ ti′(see Figure 2). Similarly, for close points yj , yj′ on Y ,the rows tj ≈ tj′ . Smoothness is achieved by minimiz-ing the row and column Dirichlet energy tr(T>LYT) +tr(TLXT

>).

Localization The correspondence is localized using theL1-penalty ‖T‖1, which, in combination with smoothness,results in a few non-zero elements that are close in space.Note that differently from CMM [31, 28] where the penaltyis applied to the basis functions, we apply it to the cor-respondence matrix. Furthermore, we do not impose anystructure but sparseness.

“Simplicity” By simplicity, we mean here that the corre-spondence matrix is ‘explained’ using a small number ofdegrees of freedom. The following models are commonlyused in matrix completion and recommendation systems lit-erature.

Fixed rank. The most straightforward model is to as-sume that rank(T) ≤ k � m,n. In this setting, we candecompose T = UV> into the factors U,V of size m× kand n×k, respectively. Combining all the above terms leads

to the optimization problem

minU,V

‖UV>F−G‖2F + µ1tr(UV>LXVU>)

+µ2tr(VU>LY UV

>) + µ3‖UV>‖1, (8)

where µ1, µ2, µ3 > 0 are parameters determining the trade-off between smoothness and localization.

Low rank. Another popular model is to find the ma-trix with smallest rank(T). However, this minimization isknown to be NP-hard, and the nuclear norm ‖T‖∗ is typi-cally used as a convex proxy, leading to

minT

‖TF−G‖2F + µ1tr(TLXT>) + µ2tr(T>LY T)

+µ3‖T‖1 + µ4‖T‖∗. (9)

This problem is convex and is typically solved using aug-mented Lagrangian methods [5].

Low norm. Srebro et al. [42] rewrite problem (9) usingthe decomposition T = UV> with U and V of size m× kand n×k, respectively. Note that unlike the fixed rank case,here k can be arbitrarily large. The nuclear norm is writtenas ‖T‖∗ = 12 (‖U‖

2F + ‖V‖2F). We thus have the problem

minU,V

‖UV>F−G‖2F + +µ1tr(UV>LXVU>)

+µ2tr(VU>LY UV

>) + µ3‖UV>‖1+µ42

(‖U‖2F + ‖V‖2F). (10)

Unlike (9), this problem is non-convex w.r.t. both U and V,however behaves well for large k [42].

3.1. Subspace parametrization

The main disadvantage of problems (8)–(10) is thatthe number of variables depends on the number of sam-ples m,n, which may result in scalability issues for large(m,n ∼ 106) manifolds. To overcome this issue, we ap-proximate the n×k andm×k factors U,V in the truncatedLaplacian eigenbases of X and Y using k′ ≥ k first expan-sion terms, U ≈ Ψk′A and V ≈ Φk′B, where matricesA,B of the expansion coefficients are of size k′ × k. Thisleads to a subspace version of problem (10) ,

minA,B

‖AB>Φ>k′F−Ψ>k′G‖2F (11)

+µ1tr(AB>ΛX,k′BA

>) + µ2tr(BA>ΛY,k′AB

>)

+µ3‖Ψk′AB>Φ>k′‖1 +µ42

(‖Ψk′A‖2F + ‖Φk′B‖2F)

where we used the invariance of the Frobenius norm to or-thogonal transformations and the fact that Φ>k′LXΦk′ =ΛX,k′ , Ψ>k′LY Ψk′ = ΛY,k′ to simplify the expressions.

Note that now the number of variables 2kk′ is indepen-dent of the number of samples. We emphasize that k′, k can

φ2 φ7 φ12 φ17 φ27

ψ2 ψ7 ψ12 ψ17 ψ27

u2 u7 u12 u17 u27

v2 v7 v12 v17 v27

Figure 3. A few vectors from the standard Laplacian eigenbasesΦ,Ψ (first and second rows) and ad hoc bases U,V (third andfourth rows) constructed as a result of our optimization over thefunctional correspondence between the human and gorilla shapes.Note that our bases are coupled, i.e. T0U ≈ V. Here and in thefollowing, hot colors represent positive values; cold colors repre-sent negative values.

be arbitrarily large and are dictated only by complexity con-siderations and not by the amount of data. This is one of themajor advantages of our approach compared to [29, 33, 18],which we demonstrate experimentally in the following sec-tion (see examples in Figure 5). Ideally, k′ (an upper boundon the rank of T) should be chosen as large as possible,which comes at the expense of computational complexity.Minimization of the Dirichlet energy selects the smoothestsolution in the subspace of k′ first eigenvectors. Finally,we observe that the subspace version of the fixed rank prob-lem (8) is a particular setting of (11) with µ4 = 0.

3.2. Relation to other approaches

Ovsjanikov et al. [29] solve a fixed-rank approxima-tion problem for T expressed in the truncated Laplacianeigenbases as T = Ψk′CΦ>k′ . It boils down to a partic-ular setting of our formulation (11) with C = AB> andµ1 = µ2 = µ3 = µ4 = 0. Note that in (5), rank(C) = kis bounded by the size of the data q: if k > q, the systemof equations in (5) becomes underdetermined. As a conse-quence, in scarce data settings (q � k), the methods per-forms very poorly (see Figure 5). Pokrass et al. [33] solvethe problem of [29] with additional prior on the diagonalstructure of C, which holds only for approximately isomet-ric manifolds. Our method is generic and can handle highlynon-isometric manifolds.

Kovnatsky et al. [18] look for a pair of bases in whichone tries to achieve a diagonal C. The bases are constructed

as a rotation of the Laplacian eigenbases (7) by means oforthonormal matrices A,B. This problem is equivalent tothe setting of our problem (11) with µ1 = µ4 = 0, with theonly difference that the Dirichlet energy terms are replacedwith off-diagonality terms.

Finally, we emphasize that our method is “basis-free”:we work directly with the correspondence matrix, and ratherthan imposing some structure in the correspondence rep-resentation in some basis (as done in prior works of [33,18, 28]), impose structure on the correspondence matrix di-rectly. This also allows us to work simultaneously withvery different manifold representations (such as meshes andgraphs, see Figure 8). The factors U,V in our problemcan be interpreted as ad hoc bases in which the correspon-dence is represented as the identity matrix, T = UIV>.These bases are coupled (see Figure 3, bottom), and thusour problem also acts similarly to the simultaneous approx-imate Laplacian diagonalization procedure [18].

3.3. Numerical optimization

Manifold optimization We used manifold optimizationtoolbox manopt [4] to solve our problem, in which thenon-differentiable L1-norm term was approximated using|t| ≈

√t2 + ξ. The main idea of manifold optimization is

to model the space of fixed-rank matrices as a C∞ Rieman-nian manifold Mk = {X ∈ Rm×n : rank(X) = k} andperform descent (e.g. by the method of conjugate gradients)on this manifold. The intrinsic gradient of the cost functiononMk is a vector in the tangent space; in order to constructthe conjugate gradient search direction, gradients from pre-vious iterations must be brought to the same tangent spaceby means of parallel transport onMk. After performing adescent step on the tangent space, the iterate is retracted tothe manifold by means of the exponential map. For addi-tional details on the use of manifold optimization in matrixcompletion problems, we refer the reader to [47]. In ourMATLAB implementation, for a typical experiment it takes∼ 4 sec/iteration, where the cost function stops changingafter ∼ 90 iterations.

Initialization Assume we are given some T = Ψk′CΦ>k′as initial correspondence matrix. In our experiments, to re-duce the convergence time, we used the correspondence ofOvsjanikov et al. [29] as the initial T. Hence, A = Cand B = I (we pad zeros to be consistent with dimen-sions of matrices). The second option is based on SVDof T = UΣV> and then setting A = Ψ>k′UΣ

1/2 andB = Φ>k′VΣ

1/2.

4. Results4.1. 3D Shapes

Data We used the TOSCA [6] dataset containing CADmodels of human and animal with 5K-50K vertices, and theSCAPE [1] dataset, containing meshes of scanned humansin different poses with about 10K vertices. For meshes,we used the cotangent formula (2) to construct the Lapla-cian. Point clouds were created by removing the mesh struc-ture from triangular meshes; 10 nearest neighbors were thenused to construct unnormalized graph Laplacians accordingto (1) with σ found using the self-tuning method [53]. In ourexperiments, as the data term we used one of the followingthree types of descriptors:

Segments produced by the persistence-based segmenta-tion of [40], represented as binary indicator functions. Atypical number of segments per shape was 3− 4.

Localized WKS+HKM. 100 Wave kernel signatures(WKS) [2] and 100 Heat kernel maps (HKM) [30], local-ized on the aforementioned segments, resulting in a totalof 200 functions per segment, or 600 − 800 functions pershape.

Shape MSER. Stable regions detected using the methodof [22], around 8 − 15 regions per shape. The MSER re-gions were represented as binary indicator functions. Alter-natively, one can use the regions computed as in [35, 51].

The Segments, WKS, and HKM descriptors were pro-vided by the authors of [29]. ShapeMSER descriptors werecomputed using the code and settings of [22].

Wave kernel signature (WKS) Heat kernel map (HKM)

Segments ShapeMSER

Figure 4. Examples of descriptors used as data (columns of F,G)in our experiments.

Performance criteria For the evaluation of the corre-spondence quality, we used the Princeton benchmark pro-tocol [16] with two error criteria:

Hard error is the criterion used in [16] for point-wisemaps. Assume that a correspondence algorithm producesa pair of points (xi1 ,yj′1), whereas the groundtruth corre-spondence is (xi1 ,yj1). Then, the inaccuracy of the corre-spondence is measured as

�hard(xi1) =dY (yj1 ,yj′1)(∑m

j=1 aYj

)1/2 , (12)and has units of normalized length on Y (ideally, zero).Here aY is the local area element on Y .

Soft error is the generalization of (12) to soft maps.Under functional correspondence, Tδi1 = ti1 ∈ L2(Y ),which ideally should be δj1 . The inaccuracy of functionalcorrespondence is measured as the average geodesic dis-tance from the j1th point on Y , weighted by the absolutevalue of the corresponding function,

�soft(xi1) =

∑mj=1 dY (yj1 ,yj)|tj,i1 |(∑mj=1 a

Yj

)1/2∑mj=1 |tj,i1 |

. (13)

Note that (12) is a particular setting of (13) where tj,i1 =δj−j′1 .

For a set of values ρ ≥ 0, we plot the percentage ofpoints 1n |{x ∈ X : �(x) ≤ ρ}| producing correspondencebetter than ρ (the higher the curve, the better).

Influence of parameters To study the behavior of ourproblem, we conducted a set of experiment in ‘sterile’ con-ditions, using a pair of near-isometric human shapes withideally corresponding F,G. The number of correspondingfunctions in the data term was q = 50. The correspon-dence quality was evaluated using the soft criterion (13), torule out any artifacts introduced by conversion to point-wisemap.

Rank. We set problem (11) in the fixed rank regimewith parameters k′ = 350, µ1 = µ2 = 10−8, µ3 = 10−1,µ4 = 0, ξ = 10−3 and varied the rank k. For comparison,we ran the methods of Ovsjanikov et al. [29] and Pokrasset al. [33] with the same k. Note that the performance of[29, 33] degrades when k > q, as opposed to increase in theperformance of our approach (Figure 5), meaning that ourprior is better than the other ones. Practically, this meansthat the correspondence quality they can obtain is boundedby the amount of given data, while our method can workwell with very scarce data.

Localization We repeated the same experiments withµ1 = µ2 = 10

−4, k = 45 fixed, and varying µ3. Increas-ing the weight of the L1 term results in better localizationof the correspondence (Figure 6), and, as a consequence,better performance.

Princeton benchmark We used the Princeton evalua-tion protocol (allowing for symmetric matches, since ourmethod is fully intrinsic and thus produces correspondencedefined up to intrinsic symmetry) on the SCAPE dataset andfollowed verbatim the settings of [29, 33].

0 0.1 0.2 0.3 0.4 0.5 0.60

10

20

30

40

50

60

70

80

90

100

100

Geodesic error

% C

orre

ct c

orre

spon

denc

es

25

25

25

50

50

100

100

350

350

350

Ovsjanikov et al.Pokrass et al.Proposed method

50

k = 25 50 100 350

Source Ovsjanikov et al.

Pokrass et al.

Proposed method

Figure 5. Influence of the rank. Top: quality (soft error) of corre-spondence between near isometric poses of the human shape ob-tained using different rank k. Note that increasing the rank abovethe number of corresponding functions q results in correspondencequality degradation in the methods of Ovsjanikov et al. [29] andPokrass et al. [33], while with our method the performance in-creases. Bottom: functional correspondence of a delta function(red dot on the leftmost shape) using different methods. Note theGibbs phenomenon manifested as negative values.

We compared the methods of [29, 33] (using code pro-vided by the authors) and our method, in two settings:scarce data, using Segments (one binary indicator functionper segment), and rich data, using localized WKS+HKM(200 functions per segment). Our method was used withthe parameters k′ = 350, k = 250 for the first case, andk′ = 50, k = 30 for the second. In both cases, we usedµ1 = µ2 = 10

−8, µ3 = 10−5, µ4 = 10−8, and ξ = 10−3.Resulting functional maps were converted into point-wisemaps using the ICP-like method of [29]. Correspondencequality was evaluated using the hard error criterion. Forcomparison, we also reproduce the performance of some re-cent state-of-the-art point-wise correspondence algorithms:blended maps [16], best conformal (least-distorting mapwithout blending), Möbius voting [21], HKM [30], andGMDS [7]. The results are shown in Figure 7. Our methodachieves state-of-the-art performance; its advantage is espe-cially prominent in the scarce data setting.

0 0.1 0.2 0.3 0.4 0.5 0.60

10

20

30

40

50

60

70

80

90

100

10

0.1

Geodesic error

% C

orre

ct c

orre

spon

denc

es

−5

10−3

0 0.1 0.2 0.3 0.4 0.5 0.60

10

20

30

40

50

60

70

80

90

100

µ3 = 10−5 10−3 0.1Source Groundtruth

Figure 6. Influence of L1 on the localization of the correspon-dence. Top: quality (soft error) of correspondence between nearisometric poses of the human shape obtained using different val-ues of µ3. Bottom: functional correspondence of a delta function(red dot on the leftmost shape) for different µ3.

0 0.05 0.1 0.15 0.2 0.250

10

20

30

40

50

60

70

80

90

100

Geodesic error

% C

orre

ct c

orre

spon

denc

es

Proposed methodPokrass et al.Ovsjanikov et al.Blended mapsMoebius votingBest conformalGMDSHKM

1

1

1

200200

200

Figure 7. Comparison of different algorithms using the Princetonbenchmark (symmetric correspondences, hard error criterion) onthe SCAPE dataset. Solid line corresponds to the rich data setting(200 functions per segment); dashed line corresponds to the scarcedata setting (one function per segment).

Robustness Figure 8 depicts examples of shape corre-spondence obtained with the proposed technique. The toprow shows examples of SCAPE shape matching using Seg-ments data. Given that this data is very scarce (3 − 4segments per shape), we find it quite remarkable that ourmethod is able to obtain very high quality correspondence.Second row shows examples of correspondence in the pres-ence of geometric and topological noise, using ShapeMSERdata. Our method handles well even large missing parts.Third row shows examples of non-isometric shape match-ing, and fourth row shows a particularly challenging setting

Figure 8. Examples of correspondence between near-isometric SCAPE human shapes (first row); TOSCA shapes contaminated withgeometric and topological noise (second row); non-isometric shapes (third row); and mesh to point cloud (fourth row). Leftmost shape isused as reference. Similar colors encode corresponding points. (Compare to Figures 3, 4, and 6 in [33])

of mesh to point cloud matching.

5. ConclusionsWe presented a novel method for finding functional cor-

respondence between manifolds based on the geometric ma-trix completion framework. We model the correspondencebetween the spaces of functions on two manifolds as a ma-trix, whose rows and columns have the geometric structureof the underlying manifolds, captured through their Lapla-cian operators. We discuss several flavors of our problem,and present an efficient subspace version thereof.

Experimental results on standard shape correspondencebenchmarks show that our method beats some of the recentstate-of-the-art methods, and compares particularly favor-ably to previous functional correspondence methods in sit-uations when the given correspondence data is very scarce.This allows tackling challenging cases such as matchingnon-isometric shapes, shapes with missing parts, differentrepresentation (meshes vs point clouds) and shapes cor-

rupted by geometric and topological noise. The proposedapproach may cope with any other multimodal data - theonly change would be the construction of the Laplacian thatcaptures its geometric structure 1.

In terms of optimization, while manopt is well-suitedfor smooth optimization, it would be advantageous to devisea version of manifold optimization for non-smooth func-tions such as the L1-norm. An alternative path is to useAugmented Lagrangian methods.

On a more philosophical note, we believe that the impor-tance of our work is in bringing a well-known signal pro-cessing technique into the domain of geometric processing,continuing the trend of [33] who introduced sparse cod-ing to the problem of shape correspondence. Conversely,our work follows the very recent trend of bringing geomet-ric structure into signal processing problems such as com-pressed sensing and matrix completion. We believe that across-fertilization of these two fields could be very fruiful.

1See the report [17] for additional applications.

AcknowledgementA.K. and M.B. are supported by the ERC Starting grant

No. 307047.

References[1] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers,

and J. Davis. Scape: shape completion and animation ofpeople. TOG, 24(3):408–416, 2005.

[2] M. Aubry, U. Schlickewei, and D. Cremers. The wave kernelsignature: A quantum mechanical approach to shape analy-sis. In Proc. 4DMOD, 2011.

[3] P. J. Besl and N. D. McKay. A method for registration of 3Dshapes. PAMI, 14(2):239–256, 1992.

[4] N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre.Manopt, a Matlab toolbox for optimization on manifolds.JMLR, 15:1455–1459, 2014.

[5] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Dis-tributed optimization and statistical learning via the alternat-ing direction method of multipliers. Foundations and Trendsin Machine Learning, 3(1):1–122, 2011.

[6] A. Bronstein, M. Bronstein, and R. Kimmel. Numerical Ge-ometry of Non-Rigid Shapes. Springer, 2008.

[7] A. M. Bronstein, M. M. Bronstein, and R. Kimmel. General-ized multidimensional scaling: a framework for isometry-invariant partial surface matching. PNAS, 103(5):1168–1172, 2006.

[8] A. M. Bronstein, M. M. Bronstein, R. Kimmel, M. Mah-moudi, and G. Sapiro. A Gromov-Hausdorff framework withdiffusion geometry for topologically-robust non-rigid shapematching. IJCV, 89(2-3):266–286, 2010.

[9] M. M. Bronstein and I. Kokkinos. Scale-invariant heat kernelsignatures for non-rigid shape recognition. In Proc. CVPR,2010.

[10] Y. Chen and G. Medioni. Object modeling by registration ofmultiple range images. In Proc. Conf. Robotics and Automa-tion, 1991.

[11] R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler,F. Warner, and S. W. Zucker. Geometric diffusions as a toolfor harmonic analysis and structure definition of data: Diffu-sion maps. PNAS, 102(21):7426–7431, 2005.

[12] A. Elad and R. Kimmel. Bending invariant representationsfor surfaces. In Proc. CVPR, 2001.

[13] D. Eynard, K. Glashoff, M. M. Bronstein, and A. M. Bron-stein. Multimodal diffusion geometry by joint diagonaliza-tion of laplacians. arXiv:1209.2295, 2012.

[14] V. Kalofolias, X. Bresson, M. Bronstein, and P. Van-dergheynst. Matrix completion on graphs. arXiv:1408.1717,2014.

[15] V. G. Kim, Y. Lipman, X. Chen, and T. A. Funkhouser.Möbius transformations for global intrinsic symmetry anal-ysis. Computer Graphics Forum, 29(5):1689–1700, 2010.

[16] V. G. Kim, Y. Lipman, and T. Funkhouser. Blended intrinsicmaps. TOG, 30(4):79, 2011.

[17] A. Kovnatsky, M. M. Bronstein, X. Bresson, and P. Van-dergheynst. Functional correspondence by matrix comple-tion. CoRR, 2014.

[18] A. Kovnatsky, M. M. Bronstein, A. M. Bronstein,K. Glashoff, and R. Kimmel. Coupled quasi-harmonic bases.Computer Graphics Forum, 32:439–448, 2013.

[19] M. Leordeanu and M. Hebert. A spectral technique for cor-respondence problems using pairwise constraints. In Proc.ICCV, 2005.

[20] B. Lévy. Laplace-Beltrami eigenfunctions towards an algo-rithm that understands geometry. In Proc. SMI, 2006.

[21] Y. Lipman and I. Daubechies. Conformal Wasserstein dis-tances: Comparing surfaces in polynomial time. Advancesin Mathematics, 227(3):1047 – 1077, 2011.

[22] R. Litman, A. M. Bronstein, and M. M. Bronstein. Diffusion-geometric maximally stable component detection in de-formable shapes. Computers & Graphics, 35(3):549–560,2011.

[23] D. Mateus, R. P. Horaud, D. Knossow, F. Cuzzolin, andE. Boyer. Articulated shape matching using laplacian eigen-functions and unsupervised point registration. In Proc.CVPR, 2008.

[24] F. Mémoli. Gromov-Wasserstein Distances and the MetricApproach to Object Matching. Foundations of Computa-tional Mathematics, pages 1–71, 2011.

[25] F. Mémoli and G. Sapiro. A theoretical and computationalframework for isometry invariant recognition of point clouddata. Foundations of Computational Mathematics, 5(3):313–347, 2005.

[26] M. Meyer, M. Desbrun, P. Schröder, and A. H. Barr. Discretedifferential-geometry operators for triangulated 2-manifolds.Visualization and Mathematics III, pages 35–57, 2003.

[27] N. J. Mitra, N. Gelfand, H. Pottmann, and L. J. Guibas. Reg-istration of point cloud data from a geometric optimizationperspective. In Proc. Eurographics Symposium on GeometryProcessing, pages 23–32, 2004.

[28] T. Neumann, K. Varanasi, C. Theobalt, M. Magnor, andM. Wacker. Compressed manifold modes for mesh process-ing. Computer Graphics Forum (Proc. SGP), 33(5):1–10,2014.

[29] M. Ovsjanikov, M. Ben-Chen, J. Solomon, A. Butscher, andL. Guibas. Functional maps: A flexible representation ofmaps between shapes. TOG, 31(4), 2012.

[30] M. Ovsjanikov, Q. Mérigot, F. Mémoli, and L. Guibas. Onepoint isometric matching with the heat kernel. ComputerGraphics Forum, 29(5):1555–1564, 2010.

[31] V. Ozoliņš, R. Lai, R. Caflisch, and S. Osher. Compressedmodes for variational problems in mathematics and physics.PNAS, 110(46):18368–18373, 2013.

[32] U. Pinkall and K. Polthier. Computing discrete minimal sur-faces and their conjugates. Experimental Mathematics, 2:15–36, 1993.

[33] J. Pokrass, A. M. Bronstein, M. M. Bronstein, P. Sprech-mann, and G. Sapiro. Sparse modeling of intrinsic corre-spondences. Computer Graphics Forum, 32:459–468, 2013.

[34] E. Rodolà, A. M. Bronstein, A. Albarelli, F. Bergamasco,and A. Torsello. A game-theoretic approach to deformableshape matching. In Proc. CVPR, 2012.

[35] E. Rodolà, S. Rota Bulo, and D. Cremers. Robust regiondetection via consensus segmentation of deformable shapes.Computer Graphics Forum, 33, 2014.

[36] E. Rodolà, S. Rota Bulo, T. Windheuser, M. Vestner, andD. Cremers. Dense non-rigid shape correspondence usingrandom forests. In Proc. CVPR, 2014.

[37] Y. Sahillioğlu and Y. Yemez. Coarse-to-fine combinatorialmatching for dense isometric shape correspondence. Com-puter Graphics Forum, 30(5):1461–1470, 2011.

[38] J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finoc-chio, A. Blake, M. Cook, and R. Moore. Real-time humanpose recognition in parts from single depth images. Commu-nications of the ACM, 56(1):116–124, 2013.

[39] A. Shtern and R. Kimmel. Matching LBO eigenspace ofnon-rigid shapes via high order statistics. arXiv:1310.4459,2013.

[40] P. Skraba, M. Ovsjanikov, F. Chazal, and L. Guibas.Persistence-based segmentation of deformable shapes. InProc. NORDIA, 2010.

[41] J. Solomon, A. Nguyen, A. Butscher, M. Ben-Chen, andL. Guibas. Soft maps between surfaces. Computer GraphicsForum, 31:1617–1626, 2012.

[42] N. Srebro, J. Rennie, and T. S. Jaakkola. Maximum-marginmatrix factorization. In Proc. NIPS, 2004.

[43] A. Tevs, A. Berner, M. Wand, I. Ihrke, and H.-P. Seidel. In-trinsic shape matching by planned landmark sampling. Com-puter Graphics Forum, 30(2):543–552, 2011.

[44] L. Torresani, V. Kolmogorov, and C. Rother. Feature corre-spondence via graph matching: Models and global optimiza-tion. In Proc. ECCV, 2008.

[45] S. Umeyama. An eigendecomposition approach to weightedgraph matching problems. PAMI, 10(5):695–703, 1988.

[46] O. van Kaick, H. Zhang, G. Hamarneh, and D. Cohen-Or.A survey on shape correspondence. Computer Graphics Fo-rum, 20:1–23, 2010.

[47] B. Vandereycken. Low-rank matrix completion by Rieman-nian optimization. SIAM J. Optimization, 23(2):1214–1236,2013.

[48] U. Von Luxburg. A tutorial on spectral clustering. Statisticsand computing, 17(4):395–416, 2007.

[49] C. Wang and S. Mahadevan. Manifold alignment withoutcorrespondence. In IJCAI, 2009.

[50] F. Wang, Q. Huang, M. Ovsjanikov, and L. Guibas. Unsuper-vised multi-class joint image segmentation. In Proc. CVPR,2014.

[51] T. Windheuser, U. Schlickewei, F. R. Schmidt, and D. Cre-mers. Geometrically consistent elastic matching of 3dshapes: A linear programming solution. In Proc. ICCV,2011.

[52] A. Zaharescu, E. Boyer, K. Varanasi, and R. Horaud. Surfacefeature detection and description with applications to meshmatching. In Proc. CVPR, 2009.

[53] L. Zelnik-Manor and P. Perona. Self-tuning spectral cluster-ing. In Proc. NIPS, 2004.

[54] Y. Zeng, C. Wang, Y. Wang, X. Gu, D. Samaras, and N. Para-gios. Dense non-rigid surface registration using high-ordergraph matching. In Proc. CVPR, 2010.