CSE 554 Lecture 9: Laplacian Deformation

CSE554 Laplacian Deformation Slide 1

CSE 554

Lecture 9: Laplacian Deformation

CSE 554

Lecture 9: Laplacian Deformation

Fall 2015

http://www.seas.wustl.edu/


ReviewReview

• Alignment

– Registering source to target by rotation and translation

• Rigid-body transformations

• Methods

– Aligning principle directions (PCA)

– Aligning corresponding points (SVD)

– Iterative improvement (ICP)

• Initialize with PCA

• Alternate between finding correspondences and SVD

Input

After PCA

After ICP

Source

Target



Non-rigid RegistrationNon-rigid Registration

• Rigid alignment cannot account for shape variance

• Non-rigid deformation is needed for improved fitting

After non-rigid deformationRigid alignment

Source

Target



Non-rigid RegistrationNon-rigid Registration

• A minimization problem

– Minimizing the distance between the deformed source and the target

• “Fitting term”

– Minimizing the distortion to the source shape

• “Distortion term”



Intrinsic vs. ExtrinsicIntrinsic vs. Extrinsic

• Intrinsic methods

– Deforms points on the source curve/surface

– App: boundary curve or surface matching

• Extrinsic methods

– Deforms all points in the space around the source curve/surface

– App: image or volume matching



Intrinsic RegistrationIntrinsic Registration

• Given a source and target shape

– Plus correspondences between some source points (called handles) and target points

• Relocate source points such that:

– The handles move to their corresponding targets (fitting term)

– The rest of the shape deforms as naturally as possible (distortion term)

• ICP-style registration

– Alternate between finding correspondences and deformation

Handle

Target



Laplacian-based DeformationLaplacian-based Deformation

• An intrinsic method used in graphics/animation

– Simple and efficient (fitting/distortion terms are quadratic)

– Preserving local shape features

Reference: “Laplacian surface editing”, by Sorkine et al., 2004



• Input

– Source with n points: p1,…,pn

• For notation purpose, assume that the first m points are handles

– Target location of handles: q1,…,qm

• Output

– Deformed locations of source points: p1’,…,pn’

SetupSetup

p1=q1

p3=q3

q2

p2

An example with 3 handles, two of which are stationary (red)

Source

Deformed



OverviewOverview

• Finding deformed locations pi’ that minimize:

– Ef: fitting term

• Measures how close are the deformed handles to the target

– Ed: distortion term

• Measures how much the source shape is changed

E Ef Ed



Fitting TermFitting Term

• Sum of squared distances to target handle locations

Ef i1

m pi' qi2q2

p2



Distortion TermDistortion Term

• Q: How to measure shape locally?

• A: By “bumpiness” at each vertex

– Laplacian: vector from the centroid of neighbors to the vertex

• A linear operator over point locations

• Recall that in fairing, we minimizes this vector to “smooth out” bumps

Lpi pi 1Ni

jNi

pj

pi

pi1pi2

pi1 pi2

2

where Ni are indices of neighboring vertices of pi



Distortion TermDistortion Term

• Minimizing changes in Laplacians during deformation

– Over all source points

Ed ni1

Lpi' i2pi

Lpi'pi'

i

i: Laplacian at pi before deformation



Putting TogetherPutting Together

• Finding deformed locations pi’ that minimize:

– A quadratic equation in terms of variables (pix’, piy’, piz’)

• qi, i are constants

• L[] is a linear operator

E Ef Ed

i1

m pi' qi2 ni1

Lpi' i2



Quadratic MinimizationQuadratic Minimization

• A general form of quadratic minimization:

– There are s variables: x=(x1,…,xs)T

– Each a1,…, ak is a length-s column vector (linear coefficients)

– Each b1,…, bk is a scalar (constant coefficients)

– k should be greater than s (so that the problem is over-constrained)

min ki1

aiT x bi2




• Re-writing our minimization in the general form

– In 2D, there are s=2n variables: x = (p1x’,…, pnx’, p1y’,…, pny’ )T

• In 3D, there are s=3n variables

– We will next re-write each quadratic term in 2D as (aix-bi)2

• Can be extended easily to 3D

E Ef Ed

i1

m pi' qi2 ni1

Lpi' i2




• The ai and bi in the fitting term

– There are 2m quadratic terms

– In the first set of m terms:

• For i=1,…,m, bi=qix, ai contains all zero, except its (i)th entry is 1.

– In the second set of m terms:

• For i=1,…,m, bi+m=qiy, ai+m contains all zero, except its (i+n)th entry is 1

Ef i1

m pi' qi2 i1

m pix ' qix2 i1

m piy ' qiy22 mi1

aiT x bi2 x p1x ', …, pnx ', p2y ', …, pny '




• The ai and bi in the fitting term

– There are 2m quadratic terms

– Example with 3 vertices and 2 fitting constraints (n=3; m=2):

Ef i1

m pi' qi2 i1

m pix ' qix2 i1

m piy ' qiy22 mi1

aiT x bi2x

p1x 'p2x 'p3x 'p1y '

p2y '

p3y '

b1 q1xb2 q2xb3 q1yb4 q2y

a1T 1 0 0 0 0 0

a2T 0 1 0 0 0 0

a3T 0 0 0 1 0 0

a4T 0 0 0 0 1 0

𝑞1 𝑞2

𝑝1 𝑝2

𝑝3




• The ai and bi in the distortion term:

– There are 2n quadratic terms

– The first set of n terms:

• For i=1,…,n, ai is all zero except the (i)th entry is 1, the (j)th entries are -1/|Ni|

for all jNi, and bi=ix

– The second set of n terms:

• For i=1,…,n, ai+n is all zero except the (i+n)th entry is 1, the (j+n)th entries are

-1/|Ni| for all jNi, and bi+n=iy

Ed ni1

Lpi' i2 ni1

Lpix ' ix2 ni1

Lpiy ' iy2Lpi pi

1Ni jNi

pj

2 ni1

aiT x bi2 x p1x ', …, pnx ', p2y ', …, pny '




• The ai and bi in the distortion term:

– There are 2n quadratic terms

– Example with 3 vertices (n=3):

Ed ni1

Lpi' i2 ni1

Lpix ' ix2 ni1

Lpiy ' iy2Lpi pi

1Ni jNi

pj

x

p1x 'p2x 'p3x 'p1y '

p2y '

p3y '

b1 1xb2 2xb3 3xb4 1yb5 2yb6 3y

2 ni1

aiT x bi2

p1 p2

p3a1T 1 1

2 12

0 0 0 a2

T 12

1 12

0 0 0 a3

T 12

12

1 0 0 0 a4

T 0 0 0 1 12

12

a5T 0 0 0 1

21 1

2

a6T 0 0 0 1

2 12

1




• To solve:

• Re-write in matrix form:

min A x B2where A

a1T

akTis a k by s matrix

min ki1

aiT x bi2

B b1bk

is a length-k vector




• The minimizer is where the partial derivatives are all zero

– To solve for x in this equation:

• Taking matrix inverse (good for small s, but numerically unstable for large s)

• Using specialized linear system solver (LinearSolve in Mathematica, Eigen/TNT/LAPACK in C)

0 Ax B2

x

2 AT A x 2 AT B AT A x AT B

x AT A 1 AT B



SummarySummary

• Compute Laplacians (i)

• Construct coefficients (ai, bi)

– Put them into matrices (A,B)

• Solve (x)

– AT A x AT B x p1x ', …, pnx ', p2y ', …, pny '



ResultsResults

Deformed

A small deformation



ResultsResults

Deformed

A larger deformation



ResultsResults

Stretching

Deformed



ResultsResults

Deformed

Shrinking



ResultsResults

Rotation

Deformed



DiscussionDiscussion

• Limitations

– Local features don’t rotate or scale with the model

• Reason: Laplacian is not invariant under rotation or scale

– Two bumps that differ by rotation or scale result in non-zero distortion

pipi'

Lpi Lpi'Lpi Lpi'



A Better Distortion TermA Better Distortion Term

• Not penalizing rotation and scaling of local features

– Estimating how the local neighborhood is transformed

– Transforming the original Laplacian vectors before comparing to the deformed Laplacians

Ed ni1

Lpi' Ti i2

pipi'Ti



Catch 22Catch 22

• Q: How do we find Ti before we know the deformed shape?

• A: Represent Ti as a function of the unknown variables

– A linear function of pi’, just like L

• We will focus in the derivations of the 2D case

– 3D results will be briefly presented at the end

Ed ni1

Lpi' Ti i2



Transformation Matrices (2D)Transformation Matrices (2D)

• Homogeneous coordinates

– A 2D point: (x,y,1)

• A 2D vector: (x,y,0)

– A 3D point: (x,y,z,1)

• A 3D vector: (x,y,z,0)




• Translation

– Cartesian coordinates: vector addition

– Homogeneous coordinates: matrix product

px'

py' vx

vy px

py

px'

py'

1

1 0 vx0 1 vy0 0 1

pxpy1




• Isotropic scaling

– Cartesian coordinates: vector scaling


px'

py' s px

py

px

py

1

s 0 00 s 00 0 1

pxpy1




• Rotation

– Cartesian coordinates: matrix product


px'

py' Cos Sin

Sin Cos pxpy

px'

py'

1

Cos Sin 0Sin Cos 0

0 0 1

pxpy1




• Summary of elementary similarity transformations

– To perform a sequence of transformations: multiple the corresponding matrices in order

px'

py'

1

M pxpy1

Trsv 1 0 vx0 1 vy0 0 1

Rot Cos Sin 0Sin Cos 0

0 0 1

Translation by vector v

Scaling by scalar s

Rotation by angle

Scls s 0 00 s 00 0 1



Similarity Transforms (2D)Similarity Transforms (2D)

• General similarity transformations

– The product of any set of similarity matrices can be written this way

– Any choice of (a, w, tx, ty) can be written as a sequence of rotation,

isotropic scaling and translation

• Note that a and w can’t be both zero

T a w txw a ty0 0 1

a w txw a ty0 0 1

Trstx, ty.Scl a2 w2 .RotArcTan a

w



Computing Ti (2D)Computing Ti (2D)

• Suppose we know the deformed locations p i’

• Compute Ti as the similarity transform that best fits the

neighborhood of pi to that of pi’

min Ti pi pi'2 jNi

Ti pj pj'2pi

pi'Ti




• Suppose we know the deformed locations p i’

• Compute Ti as the similarity transform that best fits the

neighborhood of pi to that of pi’

• This is a quadratic minimization problem for entries of T i

– E.g., a, w, tx, ty

min Ti pi pi'2 jNi

Ti pj pj'2




• The matrix form of the minimization is:

where is a 2|Ni|+2 by 4 matrix, and Ni={i1, i2,…} are indices of neighboring vertices of pi

C

pix piy 1 0

piy pix 0 1

pi1x pi1y 1 0

pi1y pi1x 0 1

min C

awtxty

pix 'piy '

pi1x 'pi1y '

2




• By quadratic minimization:

– Linear expressions of variables (pix’ , piy’)

awtxty

CT C 1 CT

pix 'piy '

pi1x 'pi1y '



Distortion Term (2D)Distortion Term (2D)

• Two parts of each distortion term:

– Transformed Laplacian:

– Laplacian of the deformed locations:

Ti i D

awtxty

D CT C 1 CT

pix 'piy '

pi1x 'pi1y '

Lpi' L

pix 'piy '

pi1x 'pi1y '

L 1 0 1

Ni0 ...

0 1 0 1Ni

...

where

where is a 2 by 2|Ni|+2 matrix

Lpi' Ti i2D

ix iy 0 0

iy ix 0 0




• Putting together:

– They form 2n quadratic terms (aix-bi)2 for x = (p1x’,…, pnx’, p1y’,…, pny’ )T

• All bi are zero

• Each ai can be extracted from H

H L D CT C 1 CT

H1, H2where

are its rows

Ed ni1

Lpi' Ti i2 n

i1

H1 pix 'piy '

pi1x 'pi1y '

2

ni1

H2 pix 'piy '

pi1x 'pi1y '

2

and



Results (2D)Results (2D)

New distortion term

Old distortion term




New distortion termOld distortion term




New distortion term

Old distortion term



RegistrationRegistration

• Use nearest neighbors as corresponding target locations

– Assuming the source is already close to the target

• Iterative closest point (ICP)

– 1. For each point on the source pi, treat it as a handle, and assign its

closest point on the target as its target location qi.

– 2. Compute Laplacian-based deformation.

– 3. Repeat step (1) until a termination criteria is met.



ResultResult

After rigid alignment 1 iteration of Laplacian

7 iterations of Laplacian Overlaying all curves



ResultResult

• Weighting the distortion term

E Ef w Ed

small w

medium w

large w




• Elementary transformation matrices

– To perform a sequence of transformations: take the product of these matrices

Translation by vector v

Scaling by scalar s

Rotation by angle around X axis

px'

py'

pz'

1

M

pxpypz1

Trsv

1 0 0 vx0 1 0 vy0 0 1 vz0 0 0 1

RotX,

1 0 0 00 Cos Sin 00 Sin Cos 00 0 0 1

Scls

s 0 0 00 s 0 00 0 s 00 0 0 1




• General similarity transformations in 3D

– Approximates the product of a set of elementary matrices

• Up to a small rotation angle

• May introduce skewing for large rotations

T

s h3 h2 txh3 s h1 tyh2 h1 s tz0 0 0 1




• Assuming known deformation, by quadratic minimization:

– Linear expressions of the deformed points pi’

• C is a 3|Ni|+3 by 7 matrix

sh1h2h3txtytz

CT C 1 CT

pix 'piy '

piz 'pi1x 'pi1y '

pi1z '

C

pix 0 piz piy 1 0 0

piy piz 0 pix 0 1 0

piz piy pix 0 0 0 1

pi1x 0 pi1z pi1y 1 0 0

pi1y pi1z 0 pi1x 0 1 0

pi1z pi1y pi1x 0 0 0 1

where




• Constructing transformed Laplacian:

whereTi i D

sh1h2h3txtytz

D CT C 1 CT

pix 'piy '

piz 'pi1x 'pi1y '

pi1z '

D

ix 0 iz iy 1 0 0

iy iz 0 ix 0 1 0

iz iy ix 0 0 0 1where


CSE 554 Lecture 9: Laplacian Deformation

Documents

Transcript of CSE 554 Lecture 9: Laplacian Deformation