1Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Numerical geometryof shapes

Lecture IV – Invariant Correspondenceand Calculus of Shapes

non-rigid

Alex Bronstein

2Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

“Natural” correspondence?

3Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Correspondence

accurate

‘‘

‘‘ makes sense

‘‘

‘‘ beautiful

‘‘

‘‘Geometric Semantic Aesthetic

4Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Correspondence

Correspondence is not a well-defined problem!

Chances to solve it with geometric tools are slim.

If objects are sufficiently similar, we have better chances.

Correspondence between deformations of the same object.

5Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Invariant correspondence

Ingredients:

Class of shapes

Class of deformations

Correspondence procedure

which given two shapes returns a map

Correspondence procedure is -invariant if it commutes with

i.e., for every and every ,

6Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

7Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Closest point correspondence between , parametrized by

Its distortion

Minimize distortion over all possible congruences

Rigid similarity

Class of deformations: congruences

Congruence-invariant (rigid) similarity:

8Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Rigid correspondence

Class of deformations: congruences

Congruence-invariant similarity:

Congruence-invariant correspondence:

RIGID SIMILARITY RIGID CORRESPONDENCEINVARIANT SIMILARITY INVARIANT CORRESPONDENCE

9Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Representation procedure is -invariant if it translates into

an isometry in , i.e., for every and , there exists

such that

Invariant representation (canonical forms)

Ingredients:

Class of shapes

Class of deformations

Embedding space and its isometry group

Representation procedure

which given a shape returns an embedding

10Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

INVARIANT SIMILARITY

= INVARIANT REPRESENTATION + RIGID SIMILARITY

11Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Invariant parametrization

Ingredients:

Class of shapes

Class of deformations

Parametrization space and its isometry group

Parametrization procedure

which given a shape returns a chart

Parametrization procedure is -invariant if it commutes with

up to an isometry in , i.e., for every and ,

there exists such that

12Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

13Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

INVARIANT CORRESPONDENCE

= INVARIANT PARAMETRIZATION + RIGID CORRESPONDENCE

14Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Representation errors

Invariant similarity / correspondence is reduced to finding isometry

in embedding / parametrization space.

Such isometry does not exist and invariance holds approximately

Given parametrization domains and , instead of isometry

find a least distorting mapping .

Correspondence is

15Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Dirichlet energy

Minimize Dirchlet energy functional

Equivalent to solving the Laplace equation

Boundary conditions

Solution (minimizer of Dirichlet energy) is a harmonic function.

16Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Dirichlet energy

Caveat: Dirichlet functional is not invariant

Not parametrization-independent

Solution: use intrinsic quantities

Frobenius norm becomes

Hilbert-Schmidt norm

Intrinsic area element

Intrinsic Dirichlet energy functional

17Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

The harmony of harmonic maps

Intrinsic Dirichlet energy functional

is the Cauchy-Green deformation tensor

Describes square of local change in distances

Minimizer is a harmonic map.

18Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Physical interpretation

METAL MOULD

RUBBER SURFACE

= ELASTIC ENERGY CONTAINED IN THE RUBBER

19Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

Ingredients:

Class of shapes

Class (groupoid) of deformations

Distortion function which given a

correspondence between two shapes

assigns to it a non-negative number

Minimum-distortion correspondence procedure

20Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

Correspondence procedure is -invariant if distortion is

-invariant, i.e., for every , and ,

Proof:

21Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

CONGRUENCES CONFORMAL ISOMETRIES

Dirichlet energy Quadratic stressEuclidean norm

22Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Minimum distortion correspondence

23Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Uniqueness

IS MINIMUM-DISTORTION CORRESPONDENCE UNIQUE?

MINIMUM-DISTORTION CORRESPONDENCE IS NOT UNIQUE

24Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Shape is symmetric, if there

exists a congruence

such that

Am I symmetric?Yes, I am symmetric.

Symmetry

25Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

What about us?

Symmetry

I am symmetric.

26Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Symmetry

Shape is symmetric, if there

exists a congruence

such that

Symmetry group = self-isometry group

Shape is symmetric, if there exists

a non-trivial automorphism

which is metric-preserving, i.e.,

27Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Symmetry: extrinsic vs. intrinsic

Extrinsic symmetry Intrinsic symmetry

28Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Symmetry: extrinsic vs. intrinsic

I am extrinsically symmetric. We are extrinsically asymmetric.We are all intrinsically symmetric.

29Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Intrinsic symmetries create distinct isometry-invariant minimum-

distortion correspondences, i.e., for every

Uniqueness & symmetry

The converse in not true, i.e. there might exist two distinct

minimum-distortion correspondences such that

for every

30Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Partial correspondence

31Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

TIMEReference Transferred texture

Texture transfer

32Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Virtual body painting

33Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Texture substitution

I’m Alice. I’m Bob.I’m Alice’s texture

on Bob’s geometry

34Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

=

How to add two dogs?

+1

2

1

2

CALCULUS OF SHAPES

35Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Addition

creates displacement

Affine calculus in a linear space

Subtraction

creates direction

Affine combination

spans subspace

Convex combination ( )

spans polytopes

36Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Affine calculus of functions

Affine space of functions

Subtraction

Addition

Affine combination

Possible because functions share a common domain

37Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Affine calculus of shapes

?

38Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Affine calculus of shapes

Ingredients:

Space of shapes embedded in

Class of correspondences

Space of deformation fields in

Since all shapes are corresponding, they can be jointly parametrized

in some by

Shape = vector field

Correspondences = joint parametrizations

Deformation field = vector field

39Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Addition:

Subtration:

Combination:

Affine calculus of shapes

CALCULUS OF SHAPES = CALCULUS OF VECTOR FIELDS

40Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Temporal super-resolution (frame rate up-conversion)

TIME

Image processing: motion-compensated interpolation

Geometry processing: deformation-compensated interpolation

41Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Metamorphing

100%

Alice

100%

Bob

75% Alice

25% Bob

50% Alice

50% Bob

75% Alice

50% Bob

42Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Face caricaturization

0 1 1.5

EXAGGERATED

EXPRESSION

43Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Calculus of shapes

44Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

What happened?

SHAPE SPACE IS NON-EUCLIDEAN!

45Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Shape space

Shape space is an abstract manifold

Deformation fields of a shape are vectors in tangent space

Our affine calculus is valid only locally

Global affine calculus can be constructed by defining trajectories

confined to the manifold

Addition

Combination

46Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Choice of trajectory

Equip tangent space with an inner product

Riemannian metric on

Select to be a minimal geodesic

Addition: initial value problem

Combination: boundary value problem

47Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Choice of metric

Deformation field of is called

Killing field if for every

Infinitesimal displacement by

Killing field is metric preserving

and are isometric

Congruence is always a Killing field

Non-trivial Killing field may not exist

48Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Choice of metric

Inner product on

Induces norm

measures deviation of from Killing field

– defined modulo congruence

Add stiffening term

49Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Minimum-distortion trajectory

Geodesic trajectory

Shapes along are as isometric as possible to

Guaranteeing no self-intersections is an open problem

50Numerical geometry of non-rigid shapes Lecture IV - Invariant Correspondence & Calculus of Shapes

Summary

Invariant correspondence = invariant similarity

Invariant parametrization

Minimum-distortion correspondence

Symmetry – self similarity

Extrinsic – self-congruence

Intrinsic – self-isometry

Affine calculus of shapes

Naïve linear model

Manifold of shapes

As isometric as possible