CSE 554 Lecture 9: Laplacian Deformation

CSE554 Laplacian Deformation Slide 1

CSE 554

Lecture 9: Laplacian Deformation

CSE 554

Lecture 9: Laplacian Deformation

Fall 2015

ReviewReview

• Alignment

– Registering source to target by rotation and translation

• Rigid-body transformations

• Methods

– Aligning principle directions (PCA)

– Aligning corresponding points (SVD)

– Iterative improvement (ICP)

• Initialize with PCA

• Alternate between finding correspondences and SVD

After PCA

After ICP

Source

Target

Non-rigid RegistrationNon-rigid Registration

• Rigid alignment cannot account for shape variance

• Non-rigid deformation is needed for improved fitting

After non-rigid deformationRigid alignment

Source

Target

Non-rigid RegistrationNon-rigid Registration

• A minimization problem

– Minimizing the distance between the deformed source and the target

• “Fitting term”

– Minimizing the distortion to the source shape

• “Distortion term”

Intrinsic vs. ExtrinsicIntrinsic vs. Extrinsic

• Intrinsic methods

– Deforms points on the source curve/surface

– App: boundary curve or surface matching

• Extrinsic methods

– Deforms all points in the space around the source curve/surface

– App: image or volume matching

Intrinsic RegistrationIntrinsic Registration

• Given a source and target shape

– Plus correspondences between some source points (called handles) and target points

• Relocate source points such that:

– The handles move to their corresponding targets (fitting term)

– The rest of the shape deforms as naturally as possible (distortion term)

• ICP-style registration

– Alternate between finding correspondences and deformation

Handle

Target

Laplacian-based DeformationLaplacian-based Deformation

• An intrinsic method used in graphics/animation

– Simple and efficient (fitting/distortion terms are quadratic)

– Preserving local shape features

Reference: “Laplacian surface editing”, by Sorkine et al., 2004

• Input

– Source with n points: p1,…,pn

• For notation purpose, assume that the first m points are handles

– Target location of handles: q1,…,qm

• Output

– Deformed locations of source points: p1’,…,pn’

SetupSetup

An example with 3 handles, two of which are stationary (red)

Source

Deformed

OverviewOverview

• Finding deformed locations pi’ that minimize:

– Ef: fitting term

• Measures how close are the deformed handles to the target

– Ed: distortion term

• Measures how much the source shape is changed

E Ef Ed

Fitting TermFitting Term

• Sum of squared distances to target handle locations

m pi' qi2q2

Distortion TermDistortion Term

• Q: How to measure shape locally?

• A: By “bumpiness” at each vertex

– Laplacian: vector from the centroid of neighbors to the vertex

• A linear operator over point locations

• Recall that in fairing, we minimizes this vector to “smooth out” bumps

Lpi pi 1Ni

pi1pi2

pi1 pi2

where Ni are indices of neighboring vertices of pi

Distortion TermDistortion Term

• Minimizing changes in Laplacians during deformation

– Over all source points

Ed ni1

Lpi' i2pi

Lpi'pi'

i: Laplacian at pi before deformation

Putting TogetherPutting Together

• Finding deformed locations pi’ that minimize:

– A quadratic equation in terms of variables (pix’, piy’, piz’)

• qi, i are constants

• L[] is a linear operator

E Ef Ed

m pi' qi2 ni1

Lpi' i2

Quadratic MinimizationQuadratic Minimization

• A general form of quadratic minimization:

– There are s variables: x=(x1,…,xs)T

– Each a1,…, ak is a length-s column vector (linear coefficients)

– Each b1,…, bk is a scalar (constant coefficients)

– k should be greater than s (so that the problem is over-constrained)

min ki1

aiT x bi2

• Re-writing our minimization in the general form

– In 2D, there are s=2n variables: x = (p1x’,…, pnx’, p1y’,…, pny’ )T

• In 3D, there are s=3n variables

– We will next re-write each quadratic term in 2D as (aix-bi)2

• Can be extended easily to 3D

E Ef Ed

m pi' qi2 ni1

Lpi' i2

• The ai and bi in the fitting term

– There are 2m quadratic terms

– In the first set of m terms:

• For i=1,…,m, bi=qix, ai contains all zero, except its (i)th entry is 1.

– In the second set of m terms:

• For i=1,…,m, bi+m=qiy, ai+m contains all zero, except its (i+n)th entry is 1

m pi' qi2 i1

m pix ' qix2 i1

m piy ' qiy22 mi1

aiT x bi2 x p1x ', …, pnx ', p2y ', …, pny '

• The ai and bi in the fitting term

– There are 2m quadratic terms

– Example with 3 vertices and 2 fitting constraints (n=3; m=2):

m pi' qi2 i1

m pix ' qix2 i1

m piy ' qiy22 mi1

aiT x bi2x

p1x 'p2x 'p3x 'p1y '

b1 q1xb2 q2xb3 q1yb4 q2y

a1T 1 0 0 0 0 0

a2T 0 1 0 0 0 0

a3T 0 0 0 1 0 0

a4T 0 0 0 0 1 0

𝑞1 𝑞2

𝑝1 𝑝2

• The ai and bi in the distortion term:

– There are 2n quadratic terms

– The first set of n terms:

• For i=1,…,n, ai is all zero except the (i)th entry is 1, the (j)th entries are -1/|Ni|

for all jNi, and bi=ix

– The second set of n terms:

• For i=1,…,n, ai+n is all zero except the (i+n)th entry is 1, the (j+n)th entries are

-1/|Ni| for all jNi, and bi+n=iy

Ed ni1

Lpi' i2 ni1

Lpix ' ix2 ni1

Lpiy ' iy2Lpi pi

1Ni jNi

aiT x bi2 x p1x ', …, pnx ', p2y ', …, pny '

• The ai and bi in the distortion term:

– There are 2n quadratic terms

– Example with 3 vertices (n=3):

Ed ni1

Lpi' i2 ni1

Lpix ' ix2 ni1

Lpiy ' iy2Lpi pi

1Ni jNi

p1x 'p2x 'p3x 'p1y '

b1 1xb2 2xb3 3xb4 1yb5 2yb6 3y

aiT x bi2

p3a1T 1 1

0 0 0 a2

0 0 0 a3

1 0 0 0 a4

T 0 0 0 1 12

a5T 0 0 0 1

a6T 0 0 0 1

• To solve:

• Re-write in matrix form:

min A x B2where A

akTis a k by s matrix

min ki1

aiT x bi2

B b1bk

is a length-k vector

• The minimizer is where the partial derivatives are all zero

– To solve for x in this equation:

• Taking matrix inverse (good for small s, but numerically unstable for large s)

• Using specialized linear system solver (LinearSolve in Mathematica, Eigen/TNT/LAPACK in C)

0 Ax B2

2 AT A x 2 AT B AT A x AT B

x AT A 1 AT B

SummarySummary

• Compute Laplacians (i)

• Construct coefficients (ai, bi)

– Put them into matrices (A,B)

• Solve (x)

– AT A x AT B x p1x ', …, pnx ', p2y ', …, pny '

ResultsResults

Deformed

A small deformation

ResultsResults

Deformed

A larger deformation

ResultsResults

Stretching

Deformed

ResultsResults

Deformed

Shrinking

ResultsResults

Rotation

Deformed

DiscussionDiscussion

• Limitations

– Local features don’t rotate or scale with the model

• Reason: Laplacian is not invariant under rotation or scale

– Two bumps that differ by rotation or scale result in non-zero distortion

Lpi Lpi'Lpi Lpi'

A Better Distortion TermA Better Distortion Term

• Not penalizing rotation and scaling of local features

– Estimating how the local neighborhood is transformed

– Transforming the original Laplacian vectors before comparing to the deformed Laplacians

Ed ni1

Lpi' Ti i2

pipi'Ti

Catch 22Catch 22

• Q: How do we find Ti before we know the deformed shape?

• A: Represent Ti as a function of the unknown variables

– A linear function of pi’, just like L

• We will focus in the derivations of the 2D case

– 3D results will be briefly presented at the end

Ed ni1

Lpi' Ti i2

Transformation Matrices (2D)Transformation Matrices (2D)

• Homogeneous coordinates

– A 2D point: (x,y,1)

• A 2D vector: (x,y,0)

– A 3D point: (x,y,z,1)

• A 3D vector: (x,y,z,0)

• Translation

– Cartesian coordinates: vector addition

– Homogeneous coordinates: matrix product

py' vx

1 0 vx0 1 vy0 0 1

• Isotropic scaling

– Cartesian coordinates: vector scaling

py' s px

s 0 00 s 00 0 1

• Rotation

– Cartesian coordinates: matrix product

py' Cos Sin

Sin Cos pxpy

Cos Sin 0Sin Cos 0

• Summary of elementary similarity transformations

– To perform a sequence of transformations: multiple the corresponding matrices in order

M pxpy1

Trsv 1 0 vx0 1 vy0 0 1

Rot Cos Sin 0Sin Cos 0

Translation by vector v

Scaling by scalar s

Rotation by angle

Scls s 0 00 s 00 0 1

Similarity Transforms (2D)Similarity Transforms (2D)

• General similarity transformations

– The product of any set of similarity matrices can be written this way

– Any choice of (a, w, tx, ty) can be written as a sequence of rotation,

isotropic scaling and translation

• Note that a and w can’t be both zero

T a w txw a ty0 0 1

a w txw a ty0 0 1

Trstx, ty.Scl a2 w2 .RotArcTan a

Computing Ti (2D)Computing Ti (2D)

• Suppose we know the deformed locations p i’

• Compute Ti as the similarity transform that best fits the

neighborhood of pi to that of pi’

min Ti pi pi'2 jNi

Ti pj pj'2pi

• Suppose we know the deformed locations p i’

• Compute Ti as the similarity transform that best fits the

neighborhood of pi to that of pi’

• This is a quadratic minimization problem for entries of T i

– E.g., a, w, tx, ty

min Ti pi pi'2 jNi

Ti pj pj'2

• The matrix form of the minimization is:

where is a 2|Ni|+2 by 4 matrix, and Ni={i1, i2,…} are indices of neighboring vertices of pi

pix piy 1 0

piy pix 0 1

pi1x pi1y 1 0

pi1y pi1x 0 1

awtxty

pix 'piy '

pi1x 'pi1y '

• By quadratic minimization:

– Linear expressions of variables (pix’ , piy’)

awtxty

CT C 1 CT

pix 'piy '

pi1x 'pi1y '

Distortion Term (2D)Distortion Term (2D)

• Two parts of each distortion term:

– Transformed Laplacian:

– Laplacian of the deformed locations:

Ti i D

awtxty

D CT C 1 CT

pix 'piy '

pi1x 'pi1y '

Lpi' L

pix 'piy '

pi1x 'pi1y '

L 1 0 1

Ni0 ...

0 1 0 1Ni

where is a 2 by 2|Ni|+2 matrix

Lpi' Ti i2D

ix iy 0 0

iy ix 0 0

• Putting together:

– They form 2n quadratic terms (aix-bi)2 for x = (p1x’,…, pnx’, p1y’,…, pny’ )T

• All bi are zero

• Each ai can be extracted from H

H L D CT C 1 CT

H1, H2where

are its rows

Ed ni1

Lpi' Ti i2 n

H1 pix 'piy '

pi1x 'pi1y '

H2 pix 'piy '

pi1x 'pi1y '

Results (2D)Results (2D)

New distortion term

Old distortion term

New distortion termOld distortion term

New distortion term

Old distortion term

New distortion term

Old distortion term

RegistrationRegistration

• Use nearest neighbors as corresponding target locations

– Assuming the source is already close to the target

• Iterative closest point (ICP)

– 1. For each point on the source pi, treat it as a handle, and assign its

closest point on the target as its target location qi.

– 2. Compute Laplacian-based deformation.

– 3. Repeat step (1) until a termination criteria is met.

ResultResult

After rigid alignment 1 iteration of Laplacian

7 iterations of Laplacian Overlaying all curves

ResultResult

• Weighting the distortion term

E Ef w Ed

small w

medium w

large w

• Elementary transformation matrices

– To perform a sequence of transformations: take the product of these matrices

Translation by vector v

Scaling by scalar s

Rotation by angle around X axis

pxpypz1

1 0 0 vx0 1 0 vy0 0 1 vz0 0 0 1

1 0 0 00 Cos Sin 00 Sin Cos 00 0 0 1

s 0 0 00 s 0 00 0 s 00 0 0 1

• General similarity transformations in 3D

– Approximates the product of a set of elementary matrices

• Up to a small rotation angle

• May introduce skewing for large rotations

s h3 h2 txh3 s h1 tyh2 h1 s tz0 0 0 1

• Assuming known deformation, by quadratic minimization:

– Linear expressions of the deformed points pi’

• C is a 3|Ni|+3 by 7 matrix

sh1h2h3txtytz

CT C 1 CT

pix 'piy '

piz 'pi1x 'pi1y '

pi1z '

pix 0 piz piy 1 0 0

piy piz 0 pix 0 1 0

piz piy pix 0 0 0 1

pi1x 0 pi1z pi1y 1 0 0

pi1y pi1z 0 pi1x 0 1 0

pi1z pi1y pi1x 0 0 0 1

• Constructing transformed Laplacian:

whereTi i D

sh1h2h3txtytz

D CT C 1 CT

pix 'piy '

piz 'pi1x 'pi1y '

pi1z '

ix 0 iz iy 1 0 0

iy iz 0 ix 0 1 0

iz iy ix 0 0 0 1where

CSE 554 Lecture 9: Laplacian Deformation

Documents

Transcript of CSE 554 Lecture 9: Laplacian Deformation

Laplacian Mesh Optimization

The Laplacian Spectrum Graphs

Laplacian - Semantic Scholar...Laplacian algorithm. One approac h to surface smo othing is the Laplacian algorithm. V ersions of Laplacian smo othing are e.g. kno wn in image pro cessing

CSE554Laplacian DeformationSlide 1 CSE 554 Lecture 8: Laplacian Deformation Fall 2012.

An improved least squares Laplacian pyramid for image ...big · An improved least squares Laplacian pyramid for image compression ... M. Unser / An improved Laplacian pyramid for

Laplacian Matrices of Graph

Laplacian on Riemannian manifolds

On the Laplacian Coefficients and Laplacian-Like Energy of ...downloads.hindawi.com/journals/jam/2012/404067.pdf · On the Laplacian Coefﬁcients and Laplacian-Like Energy of Unicyclic

Laplacian Framework for Interactive Mesh Editingylipman/ijsm_editing.pdf · plying Laplacian smoothing which requires only the solution of a sparse linear system. The Laplacian-based

PUREFLEX 616. 554-1100 FAX 616. 554-3633

Laplacian - ISBEM · electrocardiogram and recent developments of body surface Laplacian mapping, ... negative surface Laplacian of the body surface potential [3,9].

Lec15 graph laplacian embedding

Guia Thesis Complex Laplacian

Laplacian Patch-Based Image Synthesis - KAISTvclab.kaist.ac.kr/cvpr2016p2/CVPR2016_LaplacianInpainting_supp.pdf · Laplacian Patch-Based Image Synthesis ... with Other Laplacian-Based

Review - Washington University in St. Louis - Computer ...taoju/cse554/lectures/lect07...1 CSE554 Deformation II Slide 1 CSE 554 Lecture 7: Deformation II Fall 2011 CSE554 Deformation

Subtle Facial Animation Transfer from 2D Videos to 3D ...taicl/projects/ExpSpeech.pdf · Subtle Facial Animation Transfer from 2D Videos to 3D Faces with Laplacian Deformation Hui

Laplacian Dynamics on General Graphs - Harvard University · Laplacian Dynamics on General Graphs Laplacian matrices were ﬁrst introduced by Gustav Kirchhoff in his pioneering study

CSE 554 Lecture 8: Laplacian Deformationtaoju/cse554/lectures/lect08_Deformation.pdfCSE554 Laplacian Deformation Slide 2 Review • Alignment – Registering source to target by rotation

Discrete Laplacian

Boletin 554