CSE554Extrinsic DeformationsSlide 1 CSE 554 Lecture 9: Extrinsic Deformations Fall 2012.

CSE554 Extrinsic Deformations Slide 1

CSE 554

Lecture 9: Extrinsic Deformations

CSE 554

Lecture 9: Extrinsic Deformations

Fall 2012

http://www.seas.wustl.edu/


ReviewReview

• Non-rigid deformation

– Intrinsic methods: deforming the boundary points

– An optimization problem

• Minimize shape distortion

• Maximize fit

– Example: Laplacian-based deformation

Source

Target

Before

After



Extrinsic DeformationExtrinsic Deformation

• Computing deformation of each point in the plane or volume

– Not just points on the boundary curve or surface

Credits: Adams and Nistri, BMC Evolutionary Biology (2010)



Extrinsic DeformationExtrinsic Deformation

• Applications

– Registering contents between images and volumes

– Interactive spatial deformation



TechniquesTechniques

• Thin-plate spline deformation

• Free form deformation

• Cage-based deformation



Thin-Plate SplineThin-Plate Spline

• Given corresponding source and target points

• Computes a spatial deformation function for every point in the 2D plane or 3D volume

Credits: Sprengel et al, EMBS (1996)




• A minimization problem

– Minimizing distances between source and target points

– Minimizing distortion of the space (as if bending a thin sheet of metal)

• There is a closed-form solution

– Solving a linear system of equations




• Input

– Source points: p1,…,pn

– Target points: q1,…,qn

• Output

– A deformation function f[p] for any point p

pi

qi

p

f[p]




• Minimization formulation

– Ef: fitting term

• Measures how close is the deformed source to the target

– Ed: distortion term

• Measures how much the space is warped

– : weight

• Controls how much non-rigid warping is allowed

E Ef Ed




• Fitting term

– Minimizing sum of squared distances between deformed source points and target points

Ef i1

n fpi qi2




• Distortion term

– Minimizing a physical bending energy on a metal sheet (2D):

– The energy is zero when the deformation is affine

• Translation, rotation, scaling, shearing

Ed 2f

x2

2

22f

x y

2

2f

y2

2 2

x y




• Finding the minimizer for

– Uniquely exists, and has a closed form:

• M: an affine transformation matrix

• vi: translation vectors (one per source point)

• Both M and vi are determined by pi,qi,

E Ef Ed

fp M p i1

n

p pi vir r2 Logr

2 1 1 2

0 .5

1 .0

1 .5

2 .0

2 .5

where

r




• Result

– At higher , the deformation is closer to an affine transformation

0

0.001 0.1Credits: Sprengel et al, EMBS (1996)




• Application: image registration

– Manual or automatic feature pair detection

Source Target Deformed source

Credits: Rohr et al, TMI (2001)




• Advantages

– Smooth deformations, with physical analogy

– Closed-form solution

– Few free parameters (no tuning is required)

• Disadvantages

– Solving the equations still takes time (hence cannot perform “Interactive” deformation)



Free Form DeformationFree Form Deformation

• Uses a control lattice that embeds the shape

• Deforming the lattice points warps the embedded shape

Credits: Sederberg and Parry, SIGGRAPH (1986)




• Warping the space by “blending” the deformation at the control points

– Each deformed point is a weighted sum of deformed lattice points




• Input

– Source lattice points: p1,…,pn

– Target lattice points: q1,…,qn

• Output

– A deformation function f[p] for any point p in the lattice grid:

• wi[p]: determined by relative

position of p with respect to pi

piqi

p f[p]

fp i1

n

wip qi




• Desirable properties of the weights w i[p]

– Greater when p is closer to pi

• So that the influence of each control point is local

– Smoothly varies with location of p

• So that the deformation is smooth

–

• So that f[p] = wi[p] qi is an affine combination of qi

–

• So that f[p]=p if the lattice stays unchanged

p i1

nwip pi

1 i1

nwip

piqi

p f[p]




• Finding weights (2D)

– Let the lattice points be pi,j for i=0,…,k and j=0,…,l

– Compute p’s relative location in the grid (s,t)

• Let (xmin,xmax), (ymin,ymax) be the range of grid

p0,0

p0,1

p0,2

p1,0

p1,1

p1,2

p2,0

p2,1

p2,2

p3,0

p3,1

p3,2

p

s

t

s px xmin

xmax xmin

t py ymin

ymax ymin





– Let the lattice points be pi,j for i=0,…,k and j=0,…,l

– Compute p’s relative location in the grid (s,t)

– The weight wi,j for lattice point pi,j is:

i,j: importance of pi,j

• B: Bernstein basis function:Biks k

i si 1 ski

p0,0

p0,1

p0,2

p1,0

p1,1

p1,2

p2,0

p2,1

p2,2

p3,0

p3,1

p3,2

p

s

t

ui,jp i,j Biks Bjlt

wi,jp ui,jp i0

k j0l ui,jp





– Weight distribution for one control point (max at that control point):

p0,0

p0,1

p0,2

p1,0

p1,1

p2,0

p3,0

p3,1

p3,2

w1,1p




• A deformation example




• Registration

– Embed the source in a lattice

– Compute new lattice positions over the target

– Fitting term

• Distance between deformed source shape and the target shape

• Agreement of the deformed image content and the target image

– Distortion term

• Thin-plate spline energy

• Non-folding constraint

• Local rigidity constraint (to prevent stretching)




• Registration example

Credits: Loeckx et al, MICCAI (2004)

Source

Target

Deformed w/o rigidity

Deformed with rigidity




• Advantages

– Smooth deformations

– Easy to implement (no equation solving)

– Efficient and localized controls for interactive editing

– Can be coupled with different fitting or energy objectives

• Disadvantages

– Too many lattice points in 3D



Cage-based DeformationCage-based Deformation

• Use a control mesh (“cage”) to embed the shape

• Deforming the cage vertices warps the embedded shape

Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005)




• Warping the space by “blending” the deformation at the cage vertices

– wi[p]: determined by relative

position of p with respect to pi

fp i1

n

wip qip

f[p]

pi

qi





– Problem: given a closed polygon (cage) with vertices pi and an interior

point p, find weights wi[p] such that:

• 1)

• 2)

1 i1

nwip

p i1

nwip pi pi

p





– A simple case: the cage is a triangle

– The weights are unique, and are the barycentric coordinates of p

p1

p2

p3

pw1p

Areap,p2,p3

Areap1,p2,p3




– The harder case: the cage is an arbitrary (possibly concave) polygon

– The weights are not unique

• A good choice: Mean Value Coordinates (MVC)

• Can be extended to 3D


pi

p

αi

αi+1

uip Tani 2 Tani1 2pi p

wip uip i1n uip

pi-1

pi+1





– Weight distribution of one cage vertex in MVC:

pi

wip




• Application: character animation

– Using improved weights: Harmonic Coordinates




• Registration

– Embed source in a cage

– Compute new locations of cage vertices over the target

• Minimizing some fitting and energy objectives

• Not seen in literature yet… future work!




• Advantages over free form deformations

– Much smaller number of control points

– Flexible structure suitable for organic shapes

• Disadvantages

– Setting up the cage can be time-consuming

– The cage needs to be a closed shape

• Not as flexible as point handles



Further ReadingsFurther Readings

• Thin-plate spline deformation

– “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989)

– “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al. (2001)

• Free form deformation

– “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986)

– “Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling”, by Coquillart (1990)

• Cage-based deformation

– “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005)

– “Harmonic coordinates for character animation”, by Joshi et al. (2007)

– “Green coordinates”, by Lipman et al. (2008)


CSE554Extrinsic DeformationsSlide 1 CSE 554 Lecture 9: Extrinsic Deformations Fall 2012.

Documents

Transcript of CSE554Extrinsic DeformationsSlide 1 CSE 554 Lecture 9: Extrinsic Deformations Fall 2012.