Spectral geometry of shapes

363
Spectral geometry of shapes Michael Bronstein Institute of Computational Science, Faculty of Informatics University of Lugano, Switzerland July 26, 2013 1 / 363

Transcript of Spectral geometry of shapes

  • Spectral geometry of shapes

    Michael Bronstein

    Institute of Computational Science, Faculty of InformaticsUniversity of Lugano, Switzerland

    July 26, 2013

    1 / 363

  • Dimensions of media

    2 / 363

  • Dimensions of media

    3 / 363

  • Dimensions of media

    4 / 363

  • Dimensions of media

    5 / 363

  • Ubiquitous 3D data

    Google 3D Warehouse

    6 / 363

  • Ubiquitous 3D data

    Microsoft Kinect based on Primesense structured light range sensor

    7 / 363

  • Ubiquitous 3D data

    Intel presenting laptop-integratable range scanner prototype

    8 / 363

  • Ubiquitous 3D data

    3D printer

    9 / 363

  • Ubiquitous 3D data

    Point cloud from Kinect Mesh from Google 3DWarehouse

    10 / 363

  • 3D shapes vs. images

    Array of pixels

    11 / 363

  • 3D shapes vs. images

    Array of pixels

    Implicit surface, Mesh, Point cloud

    12 / 363

  • 3D shapes vs. images

    Affine, projective

    13 / 363

  • 3D shapes vs. images

    Affine, projectiveWealth of nonrigid

    deformations

    14 / 363

  • 3D shapes vs. images

    12 +

    12 =

    12 +

    12

    = ?

    15 / 363

  • 3D shapes vs. images

    12 +

    12 =

    12 +

    12

    = ?

    16 / 363

  • Applications

    Filtering Distances Correspondence

    Symmetry Editing Retrieval

    17 / 363

  • Spectral geometry

    =

    18 / 363

  • Why spectral geometry?

    Representation independent

    Intrinsic, hence deformation-invariant

    Computationally efficient

    Can be interpreted in terms of classical signal processing

    19 / 363

  • Why spectral geometry?

    Representation independent

    Intrinsic, hence deformation-invariant

    Computationally efficient

    Can be interpreted in terms of classical signal processing

    20 / 363

  • Why spectral geometry?

    Representation independent

    Intrinsic, hence deformation-invariant

    Computationally efficient

    Can be interpreted in terms of classical signal processing

    21 / 363

  • Why spectral geometry?

    Representation independent

    Intrinsic, hence deformation-invariant

    Computationally efficient

    Can be interpreted in terms of classical signal processing

    22 / 363

  • Agenda

    Euclidean heat equation

    Non-Euclidean heat equation

    Laplace-Beltrami operator, its eigenvalues and eigenfunctions

    Global stuctures: Diffusion distances

    Local stuctures: Heat kernel and other spectral descriptors

    Glocal stuctures: Stable region detectors

    Applications: Shape retrieval, Correspondence, Symmetry

    23 / 363

  • Heat equation

    Heat equation on a circle S1 2

    u(, t) +tu(, t) = 0

    Initial condition u(, 0) = v()

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    24 / 363

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time t

    Solution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    25 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t)

    and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    26 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t)

    ()T (t) + ()T (t) = 0

    and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    27 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t)

    ()

    ()=T (t)

    T (t)=

    and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    28 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and t

    () = ()

    tT (t) = T (t)

    Basic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    29 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and t

    ein = n2ein

    ten2t = n2en

    2t

    Basic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    30 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    31 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Heat equation on a circle S1{u(, t) + tu(, t) = 0

    Initial condition u(, 0) = v()

    Laplacian = 22

    Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form

    un(, t) = en2tein

    Sum of solutions is also a solution

    u(, t) =n0

    anen2tein

    32 / 363

    heat_circ.mp4Media File (video/mp4)

  • Heat equation

    Solution must satisfy the initial condition

    u(, 0) = v()

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    33 / 363

  • Heat equation

    Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0

    n0ane

    ine0 =n0v, ein

    =an

    ein

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    34 / 363

  • Heat equation

    Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0

    u(, t) =n0v, ein

    an

    einen2t

    =n

    1

    2

    20

    v()eind einen2t

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    35 / 363

  • Heat equation

    Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0

    u(, t) =n0v, eineinen2t

    =1

    2

    20

    v()n

    en2tein()d

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    36 / 363

  • Heat equation

    Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0

    u(, t) =n0v, eineinen2t

    =1

    2

    20

    v()n

    en2tein()

    ht()

    d

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    37 / 363

  • Heat equation

    Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0

    u(, t) =n0v, eineinen2t

    =1

    2

    20

    v()n

    en2tein()

    ht()

    d

    = (v ? ht)()

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    38 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)()

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    39 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)

    ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    40 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).

    Heat operator Ht = et is compact and self-adjoint

    41 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t

    Heat operator Ht = et is compact and self-adjoint

    42 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t

    Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as

    and eigenvalues en2t

    Heat operator Ht = et is compact and self-adjoint

    43 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t

    Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as

    Hteim = eim ? ht() =1

    2

    20

    eimn0

    en2tein()d

    and eigenvalues en2t

    Heat operator Ht = et is compact and self-adjoint

    44 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t

    Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as

    Hteim = eim ? ht() =n0

    en2tein

    1

    2

    20

    ei(nm)d

    and eigenvalues en2t

    Heat operator Ht = et is compact and self-adjoint

    45 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t

    Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as

    Hteim = eim ? ht() =n0

    en2tein

    1

    2

    20

    ei(nm)d mn

    and eigenvalues en2t

    Heat operator Ht = et is compact and self-adjoint

    46 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time tHeat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as

    Hteim = eim ? ht() =n0

    en2tein

    1

    2

    20

    ei(nm)d mn

    = en2teim

    and eigenvalues en2t

    Heat operator Ht = et is compact and self-adjoint

    47 / 363

  • Heat equation

    Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t

    Heat operator Ht = et is compact and self-adjoint

    48 / 363

  • Signal processing intuition

    LSI

    h

    49 / 363

  • Signal processing intuition

    LSI

    v u = v ? h

    50 / 363

  • Euclidean vs Non-Euclidean heat equation

    Euclidean

    Heat equation on S1{u(, t) + tu(, t) = 0

    u(, 0) = v()

    Non-Euclidean

    Heat equation on manifold X{Xu(x, t) +

    tu(x, t) = 0

    u(x, 0) = v(x)51 / 363

    heat_man.mp4Media File (video/mp4)

  • Riemannian geometry

    Manifold X is locallyhomeomorphic to the tangentplane TxX

    Riemannian metric

    , TxX : TxX TxX Rdepending smoothly on x

    Isometry is a shape deformationnot changing the metric

    Exponential map

    expx : TxX X

    x1

    x2

    xTxX

    ?We assume manifolds without boundary for simplicity52 / 363

  • Riemannian geometry

    Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)

    Riemannian metric

    , TxX : TxX TxX Rdepending smoothly on x

    Isometry is a shape deformationnot changing the metric

    Exponential map

    expx : TxX X

    x1

    x2

    xTxX

    ?We assume manifolds without boundary for simplicity53 / 363

  • Riemannian geometry

    Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)

    Riemannian metric

    , TxX : TxX TxX Rdepending smoothly on x

    Isometry is a shape deformationnot changing the metric

    Exponential map

    expx : TxX X

    x1

    x2

    xTxX

    ?We assume manifolds without boundary for simplicity54 / 363

  • Riemannian geometry

    Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)

    Riemannian metric

    , TxX : TxX TxX Rdepending smoothly on x

    Isometry is a shape deformationnot changing the metric

    Exponential map

    expx : TxX X

    x1

    x2

    xTxX

    ?We assume manifolds without boundary for simplicity55 / 363

  • Riemannian geometry

    Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)

    Riemannian metric

    , TxX : TxX TxX Rdepending smoothly on x

    Isometry is a shape deformationnot changing the metric

    Exponential map

    expx : TxX X

    x1

    x2

    xTxX v

    expx(v)

    ?We assume manifolds without boundary for simplicity56 / 363

  • Laplace-Beltrami operator

    Given a smooth function f : X R, define f expx : TxX R

    Gradient

    Xf(v) = (f expx)(0) =((f expx)(0)

    x1,(f expx)(0)

    x2

    )

    Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator

    Xf(x) = (f expx)(0) = i

    2(f expx)(0)x2i

    Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant

    Stokes theorem:

    57 / 363

  • Laplace-Beltrami operator

    Given a smooth function f : X R, define f expx : TxX R

    Gradient

    Xf(v) = (f expx)(0) =((f expx)(0)

    x1,(f expx)(0)

    x2

    )Taylor expansion (f expx)(v) f(x) + Xf(x), vTxX

    Laplace-Beltrami operator

    Xf(x) = (f expx)(0) = i

    2(f expx)(0)x2i

    Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant

    Stokes theorem:

    58 / 363

  • Laplace-Beltrami operator

    Given a smooth function f : X R, define f expx : TxX R

    Gradient

    Xf(v) = (f expx)(0) =((f expx)(0)

    x1,(f expx)(0)

    x2

    )Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator

    Xf(x) = (f expx)(0) = i

    2(f expx)(0)x2i

    Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant

    Stokes theorem:

    59 / 363

  • Laplace-Beltrami operator

    Given a smooth function f : X R, define f expx : TxX R

    Gradient

    Xf(v) = (f expx)(0) =((f expx)(0)

    x1,(f expx)(0)

    x2

    )Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator

    Xf(x) = (f expx)(0) = i

    2(f expx)(0)x2i

    Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant

    Stokes theorem:

    60 / 363

  • Laplace-Beltrami operator

    Given a smooth function f : X R, define f expx : TxX RGradient

    Xf(v) = (f expx)(0) =((f expx)(0)

    x1,(f expx)(0)

    x2

    )Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator

    Xf(x) = (f expx)(0) = i

    2(f expx)(0)x2i

    Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant

    Stokes theorem:X

    Xf(x)g(x)dx =

    XXf(x),Xg(x)TxXdx

    61 / 363

  • Laplace-Beltrami operator

    Given a smooth function f : X R, define f expx : TxX RGradient

    Xf(v) = (f expx)(0) =((f expx)(0)

    x1,(f expx)(0)

    x2

    )Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator

    Xf(x) = (f expx)(0) = i

    2(f expx)(0)x2i

    Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariantStokes theorem:

    XXf(x)g(x)dx Xf,gL2(X)

    =

    XXf(x),Xg(x)TxXdx

    f,gH10(X)

    62 / 363

  • Properties of the Laplace-Beltrami operator

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x

    ) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X

    63 / 363

  • Properties of the Laplace-Beltrami operator

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry: Xf, gL2(X) = Xg, fL2(X)

    Locality: (Xf)(x) does not depend on f(x) at any x 6= x

    Linear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X

    64 / 363

  • Properties of the Laplace-Beltrami operator

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x

    ) at any x 6= x

    Linear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X

    65 / 363

  • Properties of the Laplace-Beltrami operator

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x

    ) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X

    66 / 363

  • Properties of the Laplace-Beltrami operator

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x

    ) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness: Xf, fL2(X) 0

    Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X

    67 / 363

  • Properties of the Laplace-Beltrami operator

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x

    ) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X

    68 / 363

  • Discretization of the Laplace-Beltrami operator

    Manifold X is sampled at n points {x1, . . . , xn}.

    Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))

    Discrete version of the Laplacian

    (Xf)(xi) nj=1

    wij(fi fj)

    In matrix notation

    Lf = (DW)f

    where D is a diagonal matrix with elements dii =

    k 6=iwik

    69 / 363

  • Discretization of the Laplace-Beltrami operator

    Manifold X is sampled at n points {x1, . . . , xn}.Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))

    Discrete version of the Laplacian

    (Xf)(xi) nj=1

    wij(fi fj)

    In matrix notation

    Lf = (DW)f

    where D is a diagonal matrix with elements dii =

    k 6=iwik

    70 / 363

  • Discretization of the Laplace-Beltrami operator

    Manifold X is sampled at n points {x1, . . . , xn}.Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))

    Discrete version of the Laplacian

    (Xf)(xi) nj=1

    wij(fi fj)

    In matrix notation

    Lf = (DW)f

    where D is a diagonal matrix with elements dii =

    k 6=iwik

    71 / 363

  • Discretization of the Laplace-Beltrami operator

    Manifold X is sampled at n points {x1, . . . , xn}.Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))

    Discrete version of the Laplacian

    (Xf)(xi) nj=1

    wij(fi fj)

    In matrix notation

    Lf = (DW)f

    where D is a diagonal matrix with elements dii =

    k 6=iwik

    72 / 363

  • Discretization of the Laplace-Beltrami operator

    xj

    xi

    xj

    xi

    ij

    ij

    Point cloud

    Combinatorial Laplacian

    wij =

    {1 i, j E0 else

    Triangular mesh

    Cotangent weight scheme

    wij =

    { cotij+cotij2 i, j E0 else

    Tutte 1963 Pinkall, Polthier 199373 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Xf = 0 if f = const

    Symmetry:

    Locality:

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    74 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry:

    Locality:

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    75 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: Xf, g = Xg, f

    Locality:

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    76 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT

    Locality:

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    77 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality:

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    78 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: (Xf)(x) does not depend on f(x) at any x 6= x

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    79 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / E

    Linear precision:

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    80 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: if X is a plane and f = ax+ by + c, thenXf = 0

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    81 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    82 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness:

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    83 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness: Xf, f 0

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    84 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness: fTLf 0, i.e., L 0

    Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    85 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    86 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0, sufficient to satisfy discrete maximum principle

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    87 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0, sufficient to satisfy discrete maximum principlePositive weights + Symmetry PSD

    Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    88 / 363

  • Desired properties of discrete Laplacians

    Constant eigenfunction: Satisfied by construction of L

    Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors

    Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then

    (Lx)i =j

    wij(xi xj) = 0

    (a flat plate must have zero bending energy)

    Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0, sufficient to satisfy discrete maximum principlePositive weights + Symmetry PSDConvergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n

    89 / 363

  • No free lunch

    SYM LOC LIN POS PSD CONCombinatorial1 Cotangent2 Mean value3 Intrinsic Delaunay4

    No free lunch theorem: all these properties cannot be satisfiedsimultaneously

    1Tutte 63; 2Pinkal, Polthier 1993; 3Floater, Hormann 2006; 4Bobenko,

    Springborn 200790 / 363

  • No free lunch

    SYM LOC LIN POS PSD CONCombinatorial1 Cotangent2 Mean value3 Intrinsic Delaunay4

    No free lunch theorem: all these properties cannot be satisfiedsimultaneously

    1Tutte 63; 2Pinkal, Polthier 1993; 3Floater, Hormann 2006; 4Bobenko,

    Springborn 2007; Wardetzky et al. 200791 / 363

  • Agenda

    Euclidean heat equation

    Non-Euclidean heat equation

    Laplace-Beltrami operator, its eigenvalues and eigenfunctions

    Global stuctures: Diffusion distances

    Local stuctures: Heat kernel and other spectral descriptors

    Glocal stuctures: Stable region detectors

    Applications: Shape retrieval, Correspondence, Symmetry

    92 / 363

  • Laplace-Beltrami eigenvalues

    For compact manifold X the Laplace-Beltrami operator iscompact

    Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)Eigenvalues are real 0 0 1 , and is the onlyaccumulating point

    Weyls law for 2-manifold: n 4narea(X) for n

    93 / 363

  • Laplace-Beltrami eigenvalues

    For compact manifold X the Laplace-Beltrami operator iscompact

    Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)

    Eigenvalues are real 0 0 1 , and is the onlyaccumulating point

    Weyls law for 2-manifold: n 4narea(X) for n

    94 / 363

  • Laplace-Beltrami eigenvalues

    For compact manifold X the Laplace-Beltrami operator iscompact

    Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)Eigenvalues are real 0 0 1 , and is the onlyaccumulating point

    Weyls law for 2-manifold: n 4narea(X) for n

    95 / 363

  • Laplace-Beltrami eigenvalues

    For compact manifold X the Laplace-Beltrami operator iscompact

    Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)Eigenvalues are real 0 0 1 , and is the onlyaccumulating point

    Weyls law for 2-manifold: n 4narea(X) for n

    Weyl 191196 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as

    ni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i

    97 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= i

    Represent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as

    ni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i

    98 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)

    Rewrite the weak form eigenproblem asni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i

    99 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as

    ni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i

    100 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as

    ni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i

    101 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as

    ni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i

    102 / 363

  • Discrete Laplace-Beltrami eigenvalues

    Weak form eigendecomposition

    X, L2(X) = , L2(X)

    Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as

    ni=1

    iXi, jL2(X) sij stifness

    =

    ni=1

    i i, jL2(X) mij mass

    In matrix notation:S = M

    Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i , S is cotangent Laplacian L

    103 / 363

  • Diagonalization of the Laplacian

    Eigendecomposition can be posed as the minimization problem

    min

    off(TL) s.t. TA = I,

    with off-diagonality penalty off(X) =

    i6=j x2ij .

    Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms

    Analytic expression for optimal rotation for given pivot

    Rotation applied in place no matrix multiplication

    Guaranteed decrease of the off-diagonal terms

    Orthonormality guaranteed by construction

    Jacobi 1846

    104 / 363

  • Diagonalization of the Laplacian

    Eigendecomposition can be posed as the minimization problem

    min

    off(TA1/2LA1/2) s.t. T = I,

    with off-diagonality penalty off(X) =

    i6=j x2ij .

    Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms

    Analytic expression for optimal rotation for given pivot

    Rotation applied in place no matrix multiplication

    Guaranteed decrease of the off-diagonal terms

    Orthonormality guaranteed by construction

    Jacobi 1846105 / 363

  • Diagonalization of the Laplacian

    Eigendecomposition can be posed as the minimization problem

    min

    off(TA1/2LA1/2) s.t. T = I,

    with off-diagonality penalty off(X) =

    i6=j x2ij .

    Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms

    Analytic expression for optimal rotation for given pivot

    Rotation applied in place no matrix multiplication

    Guaranteed decrease of the off-diagonal terms

    Orthonormality guaranteed by construction

    Jacobi 1846106 / 363

  • Diagonalization of the Laplacian

    Eigendecomposition can be posed as the minimization problem

    min

    off(TA1/2LA1/2) s.t. T = I,

    with off-diagonality penalty off(X) =

    i6=j x2ij .

    Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms

    Analytic expression for optimal rotation for given pivot

    Rotation applied in place no matrix multiplication

    Guaranteed decrease of the off-diagonal terms

    Orthonormality guaranteed by construction

    Jacobi 1846107 / 363

  • Diagonalization of the Laplacian

    Eigendecomposition can be posed as the minimization problem

    min

    off(TA1/2LA1/2) s.t. T = I,

    with off-diagonality penalty off(X) =

    i6=j x2ij .

    Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms

    Analytic expression for optimal rotation for given pivot

    Rotation applied in place no matrix multiplication

    Guaranteed decrease of the off-diagonal terms

    Orthonormality guaranteed by construction

    Jacobi 1846108 / 363

  • Diagonalization of the Laplacian

    Eigendecomposition can be posed as the minimization problem

    min

    off(TA1/2LA1/2) s.t. T = I,

    with off-diagonality penalty off(X) =

    i6=j x2ij .

    Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms

    Analytic expression for optimal rotation for given pivot

    Rotation applied in place no matrix multiplication

    Guaranteed decrease of the off-diagonal terms

    Orthonormality guaranteed by construction

    Jacobi 1846109 / 363

  • To see the sound

    Chladni 1787 110 / 363

    chladni.movMedia File (video/quicktime)

  • Chladni plates

    Vibration modes (Laplace-Beltrami eigenfunctions) aresolutions of the Helmholtz equation

    X =

    Eigenvalues are vibration resonance frequencies

    111 / 363

  • Chladni plates

    Vibration modes (Laplace-Beltrami eigenfunctions) aresolutions of the Helmholtz equation

    X =

    Eigenvalues are vibration resonance frequencies

    112 / 363

  • ShapeDNA

    Laplace-Beltrami eigenvalues can be used as anisometry-invariant shape descriptor (shape DNA)

    10 20 30 40 50 60 70 80 90 1000

    0.005

    0.01

    0.015

    0.02

    0.025

    0.03

    0.035

    0.04

    Reuter, Wolter, Peinecke 2005113 / 363

  • Can we hear the shape of the drum?

    isometric = isospectral

    Can one hear the shape of the drum?

    Kac 1966

    114 / 363

  • Can we hear the shape of the drum?

    isometric?== isospectral

    Can one hear the shape of the drum?

    Kac 1966

    115 / 363

  • Can we hear the shape of the drum?

    isometric?== isospectral

    Can one hear the shape of the drum?

    Kac 1966116 / 363

  • Can we hear the shape of the drum?

    By Minakshisundaram-Pleijel expansion of the heat trace

    tr(Ht) =n0

    etn =i0

    antn

    (for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:

    Area: a0 = area(X)

    Total Gaussian curvature: a1 =13

    X K(x)dx

    Can we hear the metric?

    Minakshisundaram, Pleijel 1949117 / 363

  • Can we hear the shape of the drum?

    By Minakshisundaram-Pleijel expansion of the heat trace

    tr(Ht) =n0

    etn =i0

    antn

    (for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:

    Area: a0 = area(X)

    Total Gaussian curvature: a1 =13

    X K(x)dx

    Can we hear the metric?

    Minakshisundaram, Pleijel 1949118 / 363

  • Can we hear the shape of the drum?

    By Minakshisundaram-Pleijel expansion of the heat trace

    tr(Ht) =n0

    etn =i0

    antn

    (for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:

    Area: a0 = area(X)

    Total Gaussian curvature: a1 =13

    X K(x)dx

    Can we hear the metric?

    Minakshisundaram, Pleijel 1949119 / 363

  • Can we hear the shape of the drum?

    By Minakshisundaram-Pleijel expansion of the heat trace

    tr(Ht) =n0

    etn =i0

    antn

    (for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:

    Area: a0 = area(X)

    Total Gaussian curvature: a1 =13

    X K(x)dx, and by

    Gauss-Bonnet theorem, the Euler characteristic: a1 =23 (X)

    Can we hear the metric?

    Minakshisundaram, Pleijel 1949120 / 363

  • Can we hear the shape of the drum?

    By Minakshisundaram-Pleijel expansion of the heat trace

    tr(Ht) =n0

    etn =i0

    antn

    (for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:

    Area: a0 = area(X)

    Total Gaussian curvature: a1 =13

    X K(x)dx, and by

    Gauss-Bonnet theorem, the Euler characteristic: a1 =23 (X)

    Can we hear the metric?

    Minakshisundaram, Pleijel 1949121 / 363

  • One cannot hear the shape of the drum!

    Counter example of isospectral non-isometric shapes

    Gordon, Web, Wolpert 1992122 / 363

  • Laplace-Beltrami eigenfunctions

    0 1 2 3

    Eigenfunctions are invariant to isometric deformations?

    0 1 2 3

    ?Up to some ambiguities, as we will see later

    123 / 363

  • Laplace-Beltrami eigenfunctions

    0 1 2 3

    Eigenfunctions are invariant to isometric deformations?

    0 1 2 3

    ?Up to some ambiguities, as we will see later124 / 363

  • Fourier analysis on manifolds

    A function f L2(X) can be represented as the Fourier series inthe orthonormal basis {i}i0

    f(x) =i0f, iL2(X)

    ai

    i(x)

    = a0 + a1 + a2 + . . .

    125 / 363

  • Fourier analysis on manifolds

    A function f L2(X) can be represented as the Fourier series inthe orthonormal basis {i}i0

    f(x) =i0f, iL2(X)

    ai

    i(x)

    = a0 + a1 + a2 + . . .

    126 / 363

  • Shape filtering

    Filtering by applying a transfer function F () to the Fouriercoefficients

    f(x) =i0

    F (i)f, iL2(X)i(x)

    Filter shape embedding coordinates

    Fourier basis {i}i0 depends on the shape - must beupdated iteratively (fractional filtering)

    0.5

    1

    0.5

    1

    Original Lowpass Bandpass

    Levy 2006

    ; Kovnatsky, Bronstein2 2012

    127 / 363

  • Shape filtering

    Filtering by applying a transfer function F () to the Fouriercoefficients

    f(x) =i0

    F (i)f, iL2(X)i(x)

    Filter shape embedding coordinates

    Fourier basis {i}i0 depends on the shape - must beupdated iteratively (fractional filtering)

    0.5

    1

    0.5

    1

    Original Lowpass Bandpass

    Levy 2006

    ; Kovnatsky, Bronstein2 2012

    128 / 363

  • Shape filtering

    Filtering by applying a transfer function F () to the Fouriercoefficients

    f(x) =i0

    F (i)f, iL2(X)i(x)

    Filter shape embedding coordinates

    Fourier basis {i}i0 depends on the shape - must beupdated iteratively (fractional filtering)

    0.5

    1

    0.5

    1

    Original Lowpass Bandpass

    Levy 2006; Kovnatsky, Bronstein2 2012129 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x)

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    130 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x) =

    i0(x, x), iL2(X)Hti(x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x)

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    131 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x) =

    i0

    eiti(x)i(x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x)

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    132 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x) =

    i0

    eiti(x)i(x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x)

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    133 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x) =

    i0

    eiti(x)i(x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x) =

    i0

    Xv(x)i(x

    )dxeiti(x)

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    134 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x) =

    i0

    eiti(x)i(x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x) =

    Xv(x)

    i0

    i(x)eiti(x)dx

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    135 / 363

  • Heat kernel

    Heat kernel of the heat equation on X

    ht(x, x) = Ht(x, x) =

    i0

    eiti(x)i(x)

    Not shift-invariant anymore!

    For initial condition u(x, t) = v(x), the solution is given by

    Htv(x) =i0v, iL2(X)Hti(x) =

    Xv(x)

    i0

    i(x)eiti(x)dx

    =

    Xv(x)ht(x, x

    )dx

    Non shift-invariant convolution

    136 / 363

  • Heat kernel

    Heat kernel h100(x0, x) for different positions x0

    137 / 363

  • Euclidean vs Non-Euclidean heat equation

    Euclidean{u(, t) + tu(, t) = 0

    u(, 0) = v()

    Laplacian Laplacian eigenfunction

    ein = n2ein

    Heat kernel (shift-invariant)

    ht( ) =n0

    en2tein()

    Solution

    u(, t) = (v ? ht)()

    Heat operator Ht = et

    Non-Euclidean{Xu(x, t) +

    tu(x, t) = 0

    u(x, 0) = v(x)

    Laplace-Beltrami operator XLaplace-Beltrami eigenfunctions

    Xi(x) = ii(x)

    Heat kernel (not shift-invariant)

    ht(x, x) =

    n0

    eiti(x)i(x)

    Solution

    u(x, t) =

    Xv(x)ht(x, x

    )dx

    Heat operator Ht = etX138 / 363

  • Agenda

    Euclidean heat equation

    Non-Euclidean heat equation

    Laplace-Beltrami operator, its eigenvalues and eigenfunctions

    Global stuctures: Diffusion distances

    Local stuctures: Heat kernel and other spectral descriptors

    Glocal stuctures: Stable region detectors

    Applications: Shape retrieval, Correspondence, Symmetry

    139 / 363

  • Diffusion maps

    Diffusion kernel is a kernel of the form

    k(x, x) =i0

    K(i)i(x)i(x)

    satisfying the following properties

    i0K

    2(i)

  • Diffusion maps

    Diffusion kernel is a kernel of the form

    k(x, x) =i0

    K(i)i(x)i(x)

    satisfying the following propertiesi0K

    2(i)

  • Diffusion maps

    Diffusion kernel is a kernel of the form

    k(x, x) =i0

    K(i)i(x)i(x)

    satisfying the following propertiesi0K

    2(i)

  • Diffusion maps

    Diffusion kernel is a kernel of the form

    k(x, x) =i0

    K(i)i(x)i(x)

    satisfying the following propertiesi0K

    2(i)

  • Diffusion maps

    Diffusion kernel is a kernel of the form

    k(x, x) =i0

    K(i)i(x)i(x)

    satisfying the following propertiesi0K

    2(i)

  • Diffusion maps

    Diffusion map is a map from point x to a sequence

    : x 7 {K(i)i(x)}i0 `2

    Depending on the choice of the kernel K(), different maps areobtained

    Laplacian eigenmap2: K() = 1

    Diffusion map1,3: heat kernel K() = et

    Global point signature4: K() = 1

    1Berard, Besson, Gallot 1994; 2Belkin, Niyogi 2001; 3Coifman, Lafon 2006;4Rustamov 2007

    145 / 363

  • Diffusion maps

    Diffusion map is a map from point x to a sequence

    : x 7 {K(i)i(x)}i0 `2

    Depending on the choice of the kernel K(), different maps areobtained

    Laplacian eigenmap2: K() = 1

    Diffusion map1,3: heat kernel K() = et

    Global point signature4: K() = 1

    1Berard, Besson, Gallot 1994; 2Belkin, Niyogi 2001; 3Coifman, Lafon 2006;4Rustamov 2007

    146 / 363

  • Diffusion maps

    Diffusion map is a map from point x to a sequence

    : x 7 {K(i)i(x)}i0 `2

    Depending on the choice of the kernel K(), different maps areobtained

    Laplacian eigenmap2: K() = 1

    Diffusion map1,3: heat kernel K() = et

    Global point signature4: K() = 1

    Defined up to isometry in the diffusion map space (for simpleeigenvalues, sign flip of the corresponding eigenfunction)

    1Berard, Besson, Gallot 1994; 2Belkin, Niyogi 2001; 3Coifman, Lafon 2006;4Rustamov 2007

    147 / 363

  • Diffusion maps

    Diffusion embedding of different poses of the human shape

    Belkin, Niyogi 2001; Coifman, Lafon 2006148 / 363

  • Diffusion maps and distances

    By Parsevals theorem,i0

    K2(i)(i(x) i(x))2 =X

    (k(x, z) k(x, z))2dz

    Diffusion map is an isometric embedding of the diffusionmetric in the (infinite dimensional) Euclidean space

    Since the kernel K() decays, diffusion distance can beapproximated using the first n terms

    d(x, x)

    (ni=1

    K2(i)(i(x) i(x))2)1/2

    149 / 363

  • Diffusion maps and distances

    By Parsevals theorem,i0

    K2(i)(i(x) i(x))2 (x)(x)2

    `2

    =

    X

    (k(x, z) k(x, z))2dz d2(x,x)

    Diffusion map is an isometric embedding of the diffusionmetric in the (infinite dimensional) Euclidean space

    Since the kernel K() decays, diffusion distance can beapproximated using the first n terms

    d(x, x)

    (ni=1

    K2(i)(i(x) i(x))2)1/2

    150 / 363

  • Diffusion maps and distances

    By Parsevals theorem,i0

    K2(i)(i(x) i(x))2 (x)(x)2

    `2

    =

    X

    (k(x, z) k(x, z))2dz d2(x,x)

    Diffusion map is an isometric embedding of the diffusionmetric in the (infinite dimensional) Euclidean space

    Since the kernel K() decays, diffusion distance can beapproximated using the first n terms

    d(x, x)

    (ni=1

    K2(i)(i(x) i(x))2)1/2

    151 / 363

  • Diffusion distances

    K() = et

    t = 100 t = 500 t = 1000

    Coifman, Lafon 2006152 / 363

  • Diffusion distances

    K() = 1

    K() = 1

    Commute time Biharmonic

    Lipman, Rustamov, Funkhouser 2010153 / 363

  • Agenda

    Euclidean heat equation

    Non-Euclidean heat equation

    Laplace-Beltrami operator, its eigenvalues and eigenfunctions

    Global stuctures: Diffusion distances

    Local stuctures: Heat kernel and other spectral descriptors

    Glocal stuctures: Stable region detectors

    Applications: Shape retrieval, Correspondence, Symmetry

    154 / 363

  • Diffusion kernel descriptors

    Associate each point x with a heat kernel signature (HKS)

    p(x) : x 7 (kt1(x, x), . . . , ktn(x, x))ht(x, x) =

    i0

    eit2i (x)

    Multi-scale, isometry-invariant, dense point-wise descriptor

    Sun, Ovsjanikov, Guibas 2009; Gebal et al. 2009155 / 363

  • Diffusion kernel descriptors

    Associate each point x with a heat kernel signature (HKS)

    p(x) : x 7 (kt1(x, x), . . . , ktn(x, x))ht(x, x) =

    i0

    eit2i (x)

    Multi-scale, isometry-invariant, dense point-wise descriptor

    Sun, Ovsjanikov, Guibas 2009; Gebal et al. 2009156 / 363

  • Heat kernel signature

    For small t, heat kernel diagonal can be expanded as

    ht(x, x) 1

    4t

    (1 +

    1

    3K(x)t+O(t2)

    )

    HKS can be interpreted as a multi-scale Gaussian curvature

    Heat kernel diagonal (auto diffusion) ht(x, x) = how muchheat remans at point x after time t

    Sun, Ovsjanikov, Guibas 2009157 / 363

  • Heat kernel signature

    For small t, heat kernel diagonal can be expanded as

    ht(x, x) 1

    4t

    (1 +

    1

    3K(x)t+O(t2)

    )HKS can be interpreted as a multi-scale Gaussian curvature

    Heat kernel diagonal (auto diffusion) ht(x, x) = how muchheat remans at point x after time t

    Sun, Ovsjanikov, Guibas 2009158 / 363

  • Heat kernel signature

    For small t, heat kernel diagonal can be expanded as

    ht(x, x) 1

    4t

    (1 +

    1

    3K(x)t+O(t2)

    )HKS can be interpreted as a multi-scale Gaussian curvature

    Heat kernel diagonal (auto diffusion) ht(x, x) = how muchheat remans at point x after time t

    Sun, Ovsjanikov, Guibas 2009159 / 363

  • Scale invariance

    Heat kernel signature is not scale invariant!

    0 10 20 30 40 50 60 70 80 90 1000

    0.005

    0.01

    0.015

    0.02

    0.025

    0.03

    0.035

    0.04

    Original shape: i, i, ht(x, x)

    Shape scaled by :12i,

    1i,

    12h2t(x, x)

    Bronstein, Kokkinos 2010160 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKS

    HKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )

    =dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)

    Bronstein, Kokkinos 2010

    161 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )

    Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )

    =dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)

    Bronstein, Kokkinos 2010

    162 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )

    =dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)

    Bronstein, Kokkinos 2010

    163 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )=

    dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)

    Bronstein, Kokkinos 2010

    164 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )=

    dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)

    Bronstein, Kokkinos 2010

    165 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )=

    dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)

    Bronstein, Kokkinos 2010

    166 / 363

  • Scale invariant heat kernel signature

    Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative

    d

    dlog h() =

    d

    d

    (2 log + log h( 2 log )

    )=

    dd h( 2 log )h( 2 log ) g(2 log )

    a scale-covariant HKS is obtained

    g() =dd h()

    h()= log

    i0 ie

    i2i (x)i0 e

    i2i (x)

    In the Fourier domain

    g()F G() g() F G()ei2 log

    Take |G()| to obtain scale-invariant HKS (SI-HKS)Bronstein, Kokkinos 2010

    167 / 363

  • Scale invariant heat kernel signature

    HKS h()

    Bronstein, Kokkinos 2010168 / 363

  • Scale invariant heat kernel signature

    HKS h() Scale-covariant HKS h()

    Bronstein, Kokkinos 2010169 / 363

  • Scale invariant heat kernel signature

    HKS h() Scale-covariant HKS h() Scale-invariant HKS |G()|

    Bronstein, Kokkinos 2010170 / 363

  • Scale invariant heat kernel signature

    Three components of descriptors shown as RGB colors

    HKS SI-HKS

    Bronstein, Kokkinos 2010171 / 363

  • Wave kernel signature

    Different physical model: a quantum particle with initialenergy distribution f().

    Described by the Schrodinger equation(iX +

    t

    )(x, t) = 0

    Solution (x, t) complex wave function oscillatory behavior

    |(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain

    (x, t) =k0

    eiktf(k)k(x)

    Aubry, Schlickewei, Cremers 2011172 / 363

  • Wave kernel signature

    Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)

    Described by the Schrodinger equation(iX +

    t

    )(x, t) = 0

    Solution (x, t) complex wave function oscillatory behavior

    |(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain

    (x, t) =k0

    eiktf(k)k(x)

    Aubry, Schlickewei, Cremers 2011173 / 363

  • Wave kernel signature

    Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)

    Described by the Schrodinger equation(iX +

    t

    )(x, t) = 0

    Solution (x, t) complex wave function oscillatory behavior

    |(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain

    (x, t) =k0

    eiktf(k)k(x)

    Aubry, Schlickewei, Cremers 2011174 / 363

  • Wave kernel signature

    Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)

    Described by the Schrodinger equation(iX +

    t

    )(x, t) = 0

    Solution (x, t) complex wave function oscillatory behavior

    |(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain

    (x, t) =k0

    eiktf(k)k(x)

    Aubry, Schlickewei, Cremers 2011175 / 363

  • Wave kernel signature

    Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)

    Described by the Schrodinger equation(iX +

    t

    )(x, t) = 0

    Solution (x, t) complex wave function oscillatory behavior

    |(x, t)|2 = probability to find particle at point x at time t.

    Solution in spectral domain

    (x, t) =k0

    eiktf(k)k(x)

    Aubry, Schlickewei, Cremers 2011176 / 363

  • Wave kernel signature

    Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)

    Described by the Schrodinger equation(iX +

    t

    )(x, t) = 0

    Solution (x, t) complex wave function oscillatory behavior

    |(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain

    (x, t) =k0

    eiktf(k)k(x)

    Aubry, Schlickewei, Cremers 2011177 / 363

  • Wave kernel signature

    Family of log-normal initial energy distributions

    fe() exp((log e log )

    2

    22

    )

    Probability to find particle at point x

    pe(x) = limT

    1

    T

    T0|(x, t)|2dt =

    k1

    f2e (k)2k(x)

    Each point is associated with the wave kernel signature(WKS)

    p(x) : x 7 (pe1(x), . . . , pen(x))

    Aubry, Schlickewei, Cremers 2011178 / 363

  • Wave kernel signature

    Family of log-normal initial energy distributions

    fe() exp((log e log )

    2

    22

    )Probability to find particle at point x

    pe(x) = limT

    1

    T

    T0|(x, t)|2dt =

    k1

    f2e (k)2k(x)

    Each point is associated with the wave kernel signature(WKS)

    p(x) : x 7 (pe1(x), . . . , pen(x))

    Aubry, Schlickewei, Cremers 2011179 / 363

  • Wave kernel signature

    Family of log-normal initial energy distributions

    fe() exp((log e log )

    2

    22

    )Probability to find particle at point x

    pe(x) = limT

    1

    T

    T0|(x, t)|2dt =

    k1

    f2e (k)2k(x)

    Each point is associated with the wave kernel signature(WKS)

    p(x) : x 7 (pe1(x), . . . , pen(x))

    Aubry, Schlickewei, Cremers 2011180 / 363

  • A signal processing view of HKS

    0

    0.2

    0.4

    0.6

    0.8

    1

    0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05

    Collection of low pass filters

    p(x) =k0

    p1(k)...pn(k)

    2k(x)pi() = exp(ti)

    Sun, Ovsjanikov, Guibas 2009181 / 363

  • A signal processing view of WKS

    0

    0.2

    0.4

    0.6

    0.8

    1

    0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05

    Collection of band-pass filters

    p(x) =k0

    p1(k)...pn(k)

    2k(x)pi() = exp

    ((log ei log )

    2

    2

    )Aubry, Schlickewei, Cremers 2011

    182 / 363

  • Waves vs Heat

    Distance in HKS descriptor space

    Poor spatial feature localization!

    Aubry, Schlickewei, Cremers 2011183 / 363

  • Waves vs Heat

    Distance in WKS descriptor space

    Better spatial feature localization

    Lower discriminativity

    Aubry, Schlickewei, Cremers 2011184 / 363

  • Waves vs Heat

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False positive rate

    True

    pos

    itive

    rate

    HKSWKS

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False negative rate

    True

    neg

    ativ

    e ra

    te

    WKS has higher sensitivity than HKS (better at low FPR)

    Yet has lower specificity (worse at low FNR)

    WKS is better for correspondence problems

    HKS is better for shape retrieval problems

    Bronstein 2011185 / 363

  • Waves vs Heat

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False positive rate

    True

    pos

    itive

    rate

    HKSWKS

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False negative rate

    True

    neg

    ativ

    e ra

    te

    WKS has higher sensitivity than HKS (better at low FPR)

    Yet has lower specificity (worse at low FNR)

    WKS is better for correspondence problems

    HKS is better for shape retrieval problems

    Bronstein 2011186 / 363

  • Waves vs Heat

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False positive rate

    True

    pos

    itive

    rate

    HKSWKS

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False negative rate

    True

    neg

    ativ

    e ra

    te

    WKS has higher sensitivity than HKS (better at low FPR)

    Yet has lower specificity (worse at low FNR)

    WKS is better for correspondence problems

    HKS is better for shape retrieval problems

    Bronstein 2011187 / 363

  • Waves vs Heat

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False positive rate

    True

    pos

    itive

    rate

    HKSWKS

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False negative rate

    True

    neg

    ativ

    e ra

    te

    WKS has higher sensitivity than HKS (better at low FPR)

    Yet has lower specificity (worse at low FNR)

    WKS is better for correspondence problems

    HKS is better for shape retrieval problems

    Bronstein 2011188 / 363

  • Optimal spectral descriptors

    Build optimal spectral descriptor of the form

    pf (x) =k0

    f1(k)...fn(k)

    2k(x)

    Parametrized by family of frequency responsesf = (f1(), . . . , fn()).

    Similar in spirit to Wiener filter:

    attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)

    Hard to model axiomatically

    ...yet easy to learn from examples!

    Bronstein 2011189 / 363

  • Optimal spectral descriptors

    Build optimal spectral descriptor of the form

    pf (x) =k0

    f1(k)...fn(k)

    2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).

    Similar in spirit to Wiener filter:

    attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)

    Hard to model axiomatically

    ...yet easy to learn from examples!

    Bronstein 2011190 / 363

  • Optimal spectral descriptors

    Build optimal spectral descriptor of the form

    pf (x) =k0

    f1(k)...fn(k)

    2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).

    Similar in spirit to Wiener filter:

    attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)

    Hard to model axiomatically

    ...yet easy to learn from examples!

    Bronstein 2011191 / 363

  • Optimal spectral descriptors

    Build optimal spectral descriptor of the form

    pf (x) =k0

    f1(k)...fn(k)

    2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).

    Similar in spirit to Wiener filter:

    attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)

    Hard to model axiomatically

    ...yet easy to learn from examples!

    Bronstein 2011192 / 363

  • Optimal spectral descriptors

    Build optimal spectral descriptor of the form

    pf (x) =k0

    f1(k)...fn(k)

    2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).

    Similar in spirit to Wiener filter:

    attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)

    Hard to model axiomatically

    ...yet easy to learn from examples!

    Bronstein 2011193 / 363

  • Optimal spectral descriptor

    Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) x

    Find frequency responses f by solving

    minfpf (x) pf (x+) pf (x) pf (x)

    Boils down to Mahalanobis metric learning problem

    0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05

    0.5

    0

    0.5

    1

    Optimal filters learned from positive and negative examples

    Bronstein 2011194 / 363

  • Optimal spectral descriptor

    Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) xFind frequency responses f by solving

    minfpf (x) pf (x+) pf (x) pf (x)

    Boils down to Mahalanobis metric learning problem

    0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05

    0.5

    0

    0.5

    1

    Optimal filters learned from positive and negative examples

    Bronstein 2011195 / 363

  • Optimal spectral descriptor

    Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) xFind frequency responses f by solving

    minfpf (x) pf (x+) pf (x) pf (x)

    Boils down to Mahalanobis metric learning problem

    0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05

    0.5

    0

    0.5

    1

    Optimal filters learned from positive and negative examples

    Bronstein 2011196 / 363

  • Optimal spectral descriptor

    Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) xFind frequency responses f by solving

    minfpf (x) pf (x+) pf (x) pf (x)

    Boils down to Mahalanobis metric learning problem

    0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05

    0.5

    0

    0.5

    1

    Optimal filters learned from positive and negative examples

    Bronstein 2011197 / 363

  • Optimal spectral descriptor

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False positive rate

    True

    pos

    itive

    rate

    HKSWKSOpt (0.03)Opt (0.09)

    103

    102

    101

    100

    0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1

    False negative rate

    True

    neg

    ativ

    e ra

    te

    HKSWKSOpt (0.03)Opt (0.09)

    Better specificity and sensitivity than HKS and WKS

    Bronstein 2011198 / 363

  • Optimal spectral descriptor

    Better spatial feature localization than WKS

    Better discriminativity than HKS

    Bronstein 2011199 / 363

  • Optimal spectral descriptor

    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 120

    30

    40

    50

    60

    70

    80

    90

    100

    Percentage of best matches

    Hit

    rate

    in %

    HKSWKSOpt

    Better performance in correspondence problems

    Bronstein 2011200 / 363

  • ShapeGoogle

    HKS descriptors Geometric words Bag of words

    Given point-wise descriptor p(x)

    Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}

    Ovsjanikov, Bronstein2, Guibas 2009, 2011201 / 363

  • ShapeGoogle

    HKS descriptors Geometric words Bag of words

    Given point-wise descriptor p(x)

    Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}

    Ovsjanikov, Bronstein2, Guibas 2009, 2011202 / 363

  • ShapeGoogle

    HKS descriptors Geometric words Bag of words

    Given point-wise descriptor p(x)

    Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}Bag-of-words shape descriptor

    P =

    X

    v(x)dx

    Ovsjanikov, Bronstein2, Guibas 2009, 2011203 / 363

  • ShapeGoogle

    HKS descriptors Geometricexpressions

    Spatially-sensitivebag of words

    Given point-wise descriptor p(x)

    Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}Spatially-sensitive bags of pairs of words

    P =

    XX

    v(x)v(x)ht(x, x)dxdx

    Ovsjanikov, Bronstein2, Guibas 2009, 2011204 / 363

  • ShapeGoogle

    Example of bags of words

    Ovsjanikov, Bronstein2, Guibas 2009, 2011205 / 363

  • ShapeGoogle

    Shape retrieval using HKS bag-of-feature on SHREC benchmark

    Ovsjanikov, Bronstein2, Guibas 2009, 2011206 / 363

  • Partial matching

    Given a bag-of-features descriptor P of a part Y Y

    Find part X X with equal area such that the localized bagof features

    P(X ) =

    X

    vda

    best matches P

    Regularize part boundary length a la Mumford-Shah

    minX

    P(X )P2 + L(X ) s.t. area(X ) = area(Y )Ambrosio-Tortorelli approximation

    minu,

    P(u)P2 + 1 X2u2da+ 2

    X2da

    +24

    X

    (1 )2da s.t.Xuda = area(Y )

    Pokrass, Bronstein2 2011, 2013

    ; Mumford, Shah 1989

    207 / 363

  • Partial matching

    Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features

    P(X ) =

    X

    vda

    best matches P

    Regularize part boundary length a la Mumford-Shah

    minX

    P(X )P2 + L(X ) s.t. area(X ) = area(Y )Ambrosio-Tortorelli approximation

    minu,

    P(u)P2 + 1 X2u2da+ 2

    X2da

    +24

    X

    (1 )2da s.t.Xuda = area(Y )

    Pokrass, Bronstein2 2011, 2013

    ; Mumford, Shah 1989

    208 / 363

  • Partial matching

    Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features

    P(X ) =

    X

    vda

    best matches P

    Regularize part boundary length a la Mumford-Shah

    minX

    P(X )P2 + L(X ) s.t. area(X ) = area(Y )

    Ambrosio-Tortorelli approximation

    minu,

    P(u)P2 + 1 X2u2da+ 2

    X2da

    +24

    X

    (1 )2da s.t.Xuda = area(Y )

    Pokrass, Bronstein2 2011, 2013; Mumford, Shah 1989209 / 363

  • Partial matching

    Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features

    P(X ) =

    X

    vda

    best matches P

    Regularize part boundary length a la Mumford-Shah

    minX

    P(X )P2 + L(X ) s.t. area(X ) = area(Y )Relaxation: represent X by an indicator functionu : X [0, 1]

    P(u) =

    X

    vuda

    Ambrosio-Tortorelli approximation

    minu,

    P(u)P2 + 1 X2u2da+ 2

    X2da

    +24

    X

    (1 )2da s.t.Xuda = area(Y )

    Pokrass, Bronstein2 2011, 2013; Mumford, Shah 1989

    210 / 363

  • Partial matching

    Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features

    P(X ) =

    X

    vda

    best matches P

    Regularize part boundary length a la Mumford-Shah

    minX

    P(X )P2 + L(X ) s.t. area(X ) = area(Y )Ambrosio-Tortorelli approximation

    minu,

    P(u)P2 + 1 X2u2da+ 2

    X2da

    +24

    X

    (1 )2da s.t.Xuda = area(Y )

    Pokrass, Bronstein2 2011, 2013; Mumford, Shah 1989211 / 363

  • Partial matching

    Pokrass, Bronstein2 2011, 2013212 / 363

  • Agenda

    Euclidean heat equation

    Non-Euclidean heat equation

    Laplace-Beltrami operator, its eigenvalues and eigenfunctions

    Global stuctures: Diffusion distances

    Local stuctures: Heat kernel and other spectral descriptors

    Glocal stuctures: Stable region detectors

    Applications: Shape retrieval, Correspondence, Symmetry

    213 / 363

  • Component trees

    Create a hierarchy of shape points dependent on time parameter t

    t = 0: start with single points forming disjoint clustersCi = {xi}

    For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance

    d(Ci, Cj) = minxCixCj

    d(x, x) t

    Single-linkage agglomerative clustering

    Hierarchy described as component tree

    Litman, Bronstein2 2011214 / 363

  • Component trees

    Create a hierarchy of shape points dependent on time parameter t

    t = 0: start with single points forming disjoint clustersCi = {xi}For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance

    d(Ci, Cj) = minxCixCj

    d(x, x) t

    Single-linkage agglomerative clustering

    Hierarchy described as component tree

    Litman, Bronstein2 2011215 / 363

  • Component trees

    Create a hierarchy of shape points dependent on time parameter t

    t = 0: start with single points forming disjoint clustersCi = {xi}For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance

    d(Ci, Cj) = minxCixCj

    d(x, x) t

    Single-linkage agglomerative clustering

    Hierarchy described as component tree

    Litman, Bronstein2 2011216 / 363

  • Component trees

    Create a hierarchy of shape points dependent on time parameter t

    t = 0: start with single points forming disjoint clustersCi = {xi}For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance

    d(Ci, Cj) = minxCixCj

    d(x, x) t

    Single-linkage agglomerative clustering

    Hierarchy described as component tree

    Litman, Bronstein2 2011217 / 363

  • Component trees

    Litman, Bronstein2 2011218 / 363

  • Maximally stable components

    Observation: different components grow at different speed

    Stability of a component

    (Ci(t)) =A(Ci(t))ddtA(Ci(t))

    Measures relative change of area as functionof change of threshold

    (Better stability functions are available)

    Maximally stable components: localmaximizers of

    Ci(t 1)

    Ci(t)

    Ci(t+ 1)

    Litman, Bronstein2 2011219 / 363

  • Maximally stable components

    Observation: different components grow at different speed

    Stability of a component

    (Ci(t)) =A(Ci(t))ddtA(Ci(t))

    Measures relative change of area as functionof change of threshold

    (Better stability functions are available)

    Maximally stable components: localmaximizers of

    Ci(t 1)

    Ci(t)

    Ci(t+ 1)

    Litman, Bronstein2 2011220 / 363

  • Maximally stable components

    Litman, Bronstein2 2011221 / 363

  • Agenda

    Euclidean heat equation

    Non-Euclidean heat equation

    Laplace-Beltrami operator, its eigenvalues and eigenfunctions

    Global stuctures: Diffusion distances

    Local stuctures: Heat kernel and other spectral descriptors

    Glocal stuctures: Stable region detectors

    Applications: Shape retrieval, Correspondence, Symmetry

    222 / 363

  • Point-wise correspondence

    x yt

    X Y

    Correspondence is a bijection t : X Y

    Isometry dX(x, x) = dY (t(x), t(x

    )) w.r.t to intrinsic metrics(e.g. diffusion distances) dX on X and dY on Y

    Minimum-distortion correspondence

    NP-hard problem!

    Memoli, Sapiro 2005; Bronstein2, Kimmel 2006

    ; Wang et al. 2011

    223 / 363

  • Point-wise correspondence

    x y

    x

    dX dY

    y

    t

    X Y

    Correspondence is a bijection t : X YIsometry dX(x, x

    ) = dY (t(x), t(x)) w.r.t to intrinsic metrics

    (e.g. diffusion distances) dX on X and dY on Y

    Minimum-distortion correspondence

    NP-hard problem!

    Memoli, Sapiro 2005; Bronstein2, Kimmel 2006

    ; Wang et al. 2011

    224 / 363

  • Point-wise correspondence

    x y

    x

    dX dY

    y

    t

    X Y

    Correspondence is a bijection t : X YIsometry dX(x, x

    ) = dY (t(x), t(x)) w.r.t to intrinsic metrics

    (e.g. diffusion distances) dX on X and dY on YMinimum-distortion correspondence

    mintdX(x, x)dY (t(x), t(x))

    Gromov-Hausdorff distance

    NP-hard problem!

    Memoli, Sapiro 2005; Bronstein2, Kimmel 2006

    ; Wang et al. 2011

    225 / 363

  • Point-wise correspondence

    x y

    x

    dX dY

    y

    t

    X Y

    Correspondence is a bijection t : X YIsometry dX(x, x

    ) = dY (t(x), t(x)) w.r.t to intrinsic metrics

    (e.g. diffusion distances) dX on X and dY on YMinimum-distortion correspondence

    mintdX(x, x)dY (t(x), t(x)) Gromov-Hausdorff distance

    NP-hard problem!

    Memoli, Sapiro 2005; Bronstein2, Kimmel 2006

    ; Wang et al. 2011

    226 / 363

  • Point-wise correspondence

    x y

    x

    dX dY

    y

    t

    X Y

    Correspondence is a bijection t : X YIsometry dX(x, x

    ) = dY (t(x), t(x)) w.r.t to intrinsic metrics

    (e.g. diffusion distances) dX on X and dY on Y

    Minimum-distortion correspondence

    mintdX(x, x) dY (t(x), t(x))+ pX(x) pY (t(x))

    NP-hard problem!

    Memoli, Sapiro 2005; Bronstein2, Kimmel 2006; Wang et al. 2011227 / 363

  • Point-wise correspondence

    x y

    x

    dX dY

    y

    t

    X Y

    Correspondence is a bijection t : X YIsometry dX(x, x

    ) = dY (t(x), t(x)) w.r.t to intrinsic metrics

    (e.g. diffusion distances) dX on X and dY on Y

    Minimum-distortion correspondence

    mintdX(x, x) dY (t(x), t(x))+ pX(x) pY (t(x))

    NP-hard problem!

    Memoli, Sapiro 2005; Bronstein2, Kimmel 2006; Wang et al. 2011228 / 363

  • Functional correspondence

    F(X) F(Y )

    Given f : X R, corresponding function on Y is g = f t1

    Correspondence T : F(X) F(Y ) between spaces offunctions on X and Y is a linear map

    t can be recovered from T

    Ovsjanikov et al. 2012229 / 363

  • Functional correspondence

    F(X) F(Y )

    Given f : X R, corresponding function on Y is g = f t1

    Correspondence T : F(X) F(Y ) between spaces offunctions on X and Y is a linear map

    t can be recovered from T

    Ovsjanikov et al. 2012230 / 363

  • Functional correspondence

    F(X) F(Y )

    Given f : X R, corresponding function on Y is g = f t1

    Correspondence T : F(X) F(Y ) between spaces offunctions on X and Y is a linear


Geometry Learning the Names and Characteristics of Shapes.

Geometry Learning the Names and Characteristics of Shapes.

FRACTAL GEOMETRY AND SUPERFORMULA TO MODEL NATURAL SHAPES

FRACTAL GEOMETRY AND SUPERFORMULA TO MODEL NATURAL SHAPES

Scale Invariant Geometry for Nonrigid Shapes

Scale Invariant Geometry for Nonrigid Shapes

Spectral modeling of channel band shapes in wavelength selective ...

Spectral modeling of channel band shapes in wavelength selective ...

Name: Geometry 2D Shapes Classworkcontent.njctl.org/courses/math/1st-grade/geometry/...Geometry – Review of 3D Shapes Classwork Circle the shapes that answer the riddles. Write the

Name: Geometry 2D Shapes Classworkcontent.njctl.org/courses/math/1st-grade/geometry/...Geometry – Review of 3D Shapes Classwork Circle the shapes that answer the riddles. Write the

Geometry / Names and Shapes Mathematics and Millennials – 6th.

Geometry / Names and Shapes Mathematics and Millennials – 6th.

1 Numerical geometry of non-rigid shapes Introduction Numerical geometry of non-rigid shapes Introduction Alexander Bronstein, Michael Bronstein, Ron Kimmel.

1 Numerical geometry of non-rigid shapes Introduction Numerical geometry of non-rigid shapes Introduction Alexander Bronstein, Michael Bronstein, Ron Kimmel.

Geometry: Lines, angles and shapes

Geometry: Lines, angles and shapes

Maths Assessment Year 6: Geometry - Properties of Shapes

Maths Assessment Year 6: Geometry - Properties of Shapes

Geometry and Spectral Optimization (GeoSpec) - imag.frljk.imag.fr/GeoSpec/doc/GeoSpec.pdf · Geometry and Spectral Optimization (GeoSpec) Heads of the project and participants Research

Geometry and Spectral Optimization (GeoSpec) - imag.frljk.imag.fr/GeoSpec/doc/GeoSpec.pdf · Geometry and Spectral Optimization (GeoSpec) Heads of the project and participants Research

1 Numerical geometry of non-rigid shapes Spectral Methods Tutorial. Spectral Methods Tutorial 6 © Maks Ovsjanikov tosca.cs.technion.ac.il/book Numerical.

1 Numerical geometry of non-rigid shapes Spectral Methods Tutorial. Spectral Methods Tutorial 6 © Maks Ovsjanikov tosca.cs.technion.ac.il/book Numerical.

Geometry 1: Primitive Shapes - x3dgraphics.comx3dgraphics.com/slidesets/X3dForWebAuthors/Chapter02GeometryPrimitives.pdf · 4 Overview: Geometry 1, Primitive Shapes Common pattern

Geometry 1: Primitive Shapes - x3dgraphics.comx3dgraphics.com/slidesets/X3dForWebAuthors/Chapter02GeometryPrimitives.pdf · 4 Overview: Geometry 1, Primitive Shapes Common pattern

Shapes of polyhedra, mixed volumes, and hyperbolic geometry

Shapes of polyhedra, mixed volumes, and hyperbolic geometry

1 Numerical geometry of non-rigid shapes Numerical Geometry Numerical geometry of non-rigid shapes Numerical geometry Alexander Bronstein, Michael Bronstein,

1 Numerical geometry of non-rigid shapes Numerical Geometry Numerical geometry of non-rigid shapes Numerical geometry Alexander Bronstein, Michael Bronstein,

1 Numerical Geometry of Non-Rigid Shapes Spectral methods Spectral methods © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.

1 Numerical Geometry of Non-Rigid Shapes Spectral methods Spectral methods © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book.

Van Hiele Levels of understanding shapes in geometry.

Van Hiele Levels of understanding shapes in geometry.

Molecular Geometry (Shapes of Molecules)

Molecular Geometry (Shapes of Molecules)

Spectral Processing of Point-Sampled Geometry

Spectral Processing of Point-Sampled Geometry

1 Numerical geometry of non-rigid shapes Non-rigid similarity Numerical geometry of non-rigid shapes Non-rigid similarity Alexander Bronstein, Michael.

1 Numerical geometry of non-rigid shapes Non-rigid similarity Numerical geometry of non-rigid shapes Non-rigid similarity Alexander Bronstein, Michael.

Steklov Spectral Geometry - Université de Montréal

Steklov Spectral Geometry - Université de Montréal