Download - Fast Automatic Skinning Transformations

Transcript
Page 1: Fast Automatic Skinning Transformations

Fast Automatic Skinning Transformations Alec Jacobson Ilya Baran Ladislav Kavan Jovan Popović Olga Sorkine

August 8, 2012

ETH Zurich Disney Research Zurich

ETH Zurich Adobe Systems, Inc.

ETH Zurich

Page 2: Fast Automatic Skinning Transformations

#

Real-time performance critical for interactive design and animation

August 8, 2012 Alec Jacobson 2

2D 3D

Page 3: Fast Automatic Skinning Transformations

#

Real-time performance critical for interactive design and animation

August 8, 2012 Alec Jacobson 3

2D 3D

Page 4: Fast Automatic Skinning Transformations

# August 8, 2012 Alec Jacobson 4

We want speeds measured in microseconds

80k triangles 20µs per iteration

Page 5: Fast Automatic Skinning Transformations

# August 8, 2012 Alec Jacobson 5

We want speeds measured in microseconds

80k triangles 20µs per iteration

Page 6: Fast Automatic Skinning Transformations

#

This means speed comparable to rendering

August 8, 2012 Alec Jacobson 6 30fps

Page 7: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 7

place skeleton in shape

Page 8: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 8

place skeleton in shape compute/paint weights

Page 9: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 9

place skeleton in shape compute/paint weights deform bones

Page 10: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 10

place skeleton in shape compute/paint weights deform bones

v0i =

mX

j=1

wj(vi)Tj

✓vi

1

Page 11: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 11

place skeleton in shape compute/paint weights deform bones

Page 12: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 12

place skeleton in shape compute/paint weights deform bones

Page 13: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 13

place skeleton in shape compute/paint weights deform bones

Page 14: Fast Automatic Skinning Transformations

#

Linear Blend Skinning preferred for real-time performance

August 8, 2012 Alec Jacobson 14

place skeleton in shape compute/paint weights deform bones

15 bones * 3x4 matrix =

180 degrees of freedom

Page 15: Fast Automatic Skinning Transformations

#

LBS generalizes to different handle types

August 8, 2012 Alec Jacobson 15

skeletons

v0i =

mX

j=1

wj(vi)Tj

✓vi

1

Page 16: Fast Automatic Skinning Transformations

#

regions

LBS generalizes to different handle types

August 8, 2012 Alec Jacobson 16

skeletons

v0i =

mX

j=1

wj(vi)Tj

✓vi

1

Page 17: Fast Automatic Skinning Transformations

#

regions

LBS generalizes to different handle types

August 8, 2012 Alec Jacobson 17

skeletons

v0i =

mX

j=1

wj(vi)Tj

✓vi

1

Page 18: Fast Automatic Skinning Transformations

#

regions

LBS generalizes to different handle types

August 8, 2012 Alec Jacobson 18

skeletons points

v0i =

mX

j=1

wj(vi)Tj

✓vi

1

Page 19: Fast Automatic Skinning Transformations

#

regions

LBS generalizes to different handle types

August 8, 2012 Alec Jacobson 19

skeletons points

v0i =

mX

j=1

wj(vi)Tj

✓vi

1

Page 20: Fast Automatic Skinning Transformations

#

User specifies subset of parameters, optimize to find remaining ones

August 8, 2012 Alec Jacobson 20

Full optimization

Mesh vertex positions

Page 21: Fast Automatic Skinning Transformations

#

User specifies subset of parameters, optimize to find remaining ones

August 8, 2012 Alec Jacobson 21

Full optimization

Reduced model

Skinning degrees of freedom

Page 22: Fast Automatic Skinning Transformations

#

User specifies subset of parameters, optimize to find remaining ones

August 8, 2012 Alec Jacobson 22

Full optimization

Reduced model

Matrix form

Page 23: Fast Automatic Skinning Transformations

#

User specifies subset of parameters, optimize to find remaining ones

August 8, 2012 Alec Jacobson 23

Full optimization

Reduced model

Matrix form

Reduced optimization

Page 24: Fast Automatic Skinning Transformations

#

Enforce user constraints as linear equalities

August 8, 2012 Alec Jacobson 24

User constraints

Reduced optimization

Full

Position only

Unconstrained

Page 25: Fast Automatic Skinning Transformations

#

Enforce user constraints as linear equalities

August 8, 2012 Alec Jacobson 25

User constraints

Reduced optimization

Full

Position only

Unconstrained

Page 26: Fast Automatic Skinning Transformations

#

Enforce user constraints as linear equalities

August 8, 2012 Alec Jacobson 26

User constraints

Reduced optimization

Full

Position only

Unconstrained

Page 27: Fast Automatic Skinning Transformations

#

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 27

Full energies

Page 28: Fast Automatic Skinning Transformations

#

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 28

Full energies

triangles Liu et al. 08

“spokes” Sorkine & Alexa 07

“spokes and rims” Chao et al. 10

tetrahedra Chao et al. 10

Page 29: Fast Automatic Skinning Transformations

#

Global step: Fix , minimize with respect to

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 29

Full energies

V0R

Local step: Fix , minimize with respect to RV0

Local/Global optimization

Page 30: Fast Automatic Skinning Transformations

#

Global step: large, sparse linear solve

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 30

Full energies

Local step: Fix , minimize with respect to RV0

Local/Global optimization

V0 = A�1b

precompute

Page 31: Fast Automatic Skinning Transformations

#

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 31

Full energies

Local step: 3x3 SVD for each rotation in R

Local/Global optimization

Global step: large, sparse linear solve V0 = A�1b

Page 32: Fast Automatic Skinning Transformations

#

Global step: small, dense linear solve

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 32

Full energies

Local step: 3x3 SVD for each rotation in R

Local/Global optimization

T = A�1b

precompute

Similar to: [Huang et al. 06] [Der et al. 06] [Au et al. 07] [Hildebrandt et al. 12]

Substitute

Page 33: Fast Automatic Skinning Transformations

#

Direct reduction of elastic energies brings speed up and regularization…

August 8, 2012 Alec Jacobson 33

Page 34: Fast Automatic Skinning Transformations

#

Direct reduction of elastic energies brings speed up and regularization…

August 8, 2012 Alec Jacobson 34

Full ARAP solution

Page 35: Fast Automatic Skinning Transformations

#

Direct reduction of elastic energies brings speed up and regularization…

August 8, 2012 Alec Jacobson 35

Full ARAP solution

Our smooth subspace solution

Page 36: Fast Automatic Skinning Transformations

#

Local step: 3x3 SVD for each rotation in

Global step: small, dense linear solve

We reduce any as-rigid-as-possible energy

August 8, 2012

But #rotations ~ full mesh discretization

Alec Jacobson 36

Full energies

R

Local/Global optimization

T = A�1b

Substitute

Page 37: Fast Automatic Skinning Transformations

#

Global step: small, dense linear solve

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 37

Full energies

R

Local/Global optimization

Substitute

T = A�1b

Local step: 3x3 SVD for each rotation in Cluster

Ek

Page 38: Fast Automatic Skinning Transformations

#

Rotation evaluations may be reduced by clustering in weight space

August 8, 2012 Alec Jacobson 38

Full energies

triangles Liu et al. 08

“spokes” Sorkine & Alexa 07

“spokes and rims” Chao et al. 10

tetrahedra Chao et al. 10

Page 39: Fast Automatic Skinning Transformations

#

Rotation evaluations may be reduced by k-means clustering in weight space

August 8, 2012 Alec Jacobson 39

Full energies

xj =

2

6664

w1(vj)w2(vj)

...wm(vj)

3

7775

weight space

Page 40: Fast Automatic Skinning Transformations

#

Rotation evaluations may be reduced by clustering in weight space

August 8, 2012 Alec Jacobson 40

Full energies

r = 2 r = 4 r = 64

Page 41: Fast Automatic Skinning Transformations

#

Rotation evaluations may be reduced by clustering in weight space

August 8, 2012 Alec Jacobson 41

Full energies

r = 2 r = 4 r = 64

Page 42: Fast Automatic Skinning Transformations

#

Global step: small, dense linear solve

We reduce any as-rigid-as-possible energy

August 8, 2012 Alec Jacobson 42

Full energies

R

Local/Global optimization

T = A�1b

Local step: 3x3 SVD for each rotation in

#rotations ~ #T, independent of full mesh resolution

Substitute

Cluster

Ek

Page 43: Fast Automatic Skinning Transformations

#

Real-time automatic degrees of freedom

August 8, 2012 Alec Jacobson 43

Page 44: Fast Automatic Skinning Transformations

#

Real-time automatic degrees of freedom

August 8, 2012 Alec Jacobson 44

Page 45: Fast Automatic Skinning Transformations

#

With more and more user constraints we fall back to standard skinning

August 8, 2012 Alec Jacobson 45

Page 46: Fast Automatic Skinning Transformations

#

With more and more user constraints we fall back to standard skinning

August 8, 2012 Alec Jacobson 46

Page 47: Fast Automatic Skinning Transformations

#

With more and more user constraints we fall back to standard skinning

August 8, 2012 Alec Jacobson 47

Page 48: Fast Automatic Skinning Transformations

#

With more and more user constraints we fall back to standard skinning

August 8, 2012 Alec Jacobson 48

Page 49: Fast Automatic Skinning Transformations

#

Extra weights would expand subspace…

August 8, 2012 Alec Jacobson 49

Page 50: Fast Automatic Skinning Transformations

#

Extra weights would expand subspace…

August 8, 2012 Alec Jacobson 50

Page 51: Fast Automatic Skinning Transformations

#

Extra weights would expand subspace…

August 8, 2012 Alec Jacobson 51

Page 52: Fast Automatic Skinning Transformations

#

Overlapping b-spline “bumps” in weight space

August 8, 2012 Alec Jacobson 52

farthest point sampling

xj =

2

6664

w1(vj)w2(vj)

...wm(vj)

3

7775

weight space

Page 53: Fast Automatic Skinning Transformations

#

Overlapping b-spline “bumps” in weight space

August 8, 2012 Alec Jacobson 53

b-spline basis parameterized by distance in weight space

in weight space

xj =

2

6664

w1(vj)w2(vj)

...wm(vj)

3

7775

weight space

Page 54: Fast Automatic Skinning Transformations

#

Overlapping b-spline “bumps” in weight space

August 8, 2012 Alec Jacobson 54

b-spline basis parameterized by distance in weight space

in weight space

xj =

2

6664

w1(vj)w2(vj)

...wm(vj)

3

7775

weight space

Page 55: Fast Automatic Skinning Transformations

#

no extra weights

Extra weights expand deformation subspace

August 8, 2012 Alec Jacobson 55

15 extra weights

Page 56: Fast Automatic Skinning Transformations

#

no extra weights

Extra weights expand deformation subspace

August 8, 2012 Alec Jacobson 56

15 extra weights

Page 57: Fast Automatic Skinning Transformations

#

Subspace now rich enough for fast variational modeling

August 8, 2012 Alec Jacobson 57 Full non-linear optimization

[Botsch et al. 2006]

Our reduced method

Page 58: Fast Automatic Skinning Transformations

#

Subspace now rich enough for fast variational modeling

August 8, 2012 Alec Jacobson 58 Full non-linear optimization

[Botsch et al. 2006]

Our reduced method

Page 59: Fast Automatic Skinning Transformations

#

Subspace now rich enough for fast variational modeling

August 8, 2012 Alec Jacobson 59 Full non-linear optimization

[Botsch et al. 2006]

Our reduced method

Page 60: Fast Automatic Skinning Transformations

#

Subspace now rich enough for fast variational modeling

August 8, 2012 Alec Jacobson 60 Full non-linear optimization

[Botsch et al. 2006]

Our reduced method

Page 61: Fast Automatic Skinning Transformations

#

Subspace now rich enough for fast variational modeling

August 8, 2012 Alec Jacobson 61 Full non-linear optimization

[Botsch et al. 2006]

Our reduced method

Page 62: Fast Automatic Skinning Transformations

#

Subspace now rich enough for fast variational modeling

August 8, 2012 Alec Jacobson 62 Full non-linear optimization

[Botsch et al. 2006]

Our reduced method

Page 63: Fast Automatic Skinning Transformations

#

Final algorithm is simple and FAST

August 8, 2012 Alec Jacobson 63

Precomputation per shape+rig - Compute any additional weights - Construct, prefactor system matrices

For a 50K triangle mesh: 12 seconds

2.7 seconds

Page 64: Fast Automatic Skinning Transformations

#

Final algorithm is simple and FAST

August 8, 2012 Alec Jacobson 64

Precomputation per shape+rig - Compute any additional weights - Construct, prefactor system matrices

Precomputation when switching constraint type - Re-factor global step system

For a 50K triangle mesh: 12 seconds

2.7 seconds

6 milliseconds

Page 65: Fast Automatic Skinning Transformations

#

Final algorithm is simple and FAST

August 8, 2012 Alec Jacobson 65

Precomputation per shape+rig - Compute any additional weights - Construct, prefactor system matrices

Precomputation when switching constraint type - Re-factor global step system

~30 iterations global: #weights by #weights linear solve local: #rotations SVDs

For a 50K triangle mesh: 12 seconds

2.7 seconds

6 milliseconds

22 microseconds

[McAdams et al. 2011]

Page 66: Fast Automatic Skinning Transformations

# August 8, 2012 Alec Jacobson 66

Demo

Lightning FAST automatic skinning transformations

Page 67: Fast Automatic Skinning Transformations

#

Extra weights and disjoint skeletons make flexible control easy

August 8, 2012 Alec Jacobson 67

From Cartoon Animation by Preston Blair

Page 68: Fast Automatic Skinning Transformations

#

Extra weights and disjoint skeletons make flexible control easy

August 8, 2012 Alec Jacobson 68

Page 69: Fast Automatic Skinning Transformations

#

Extra weights and disjoint skeletons make flexible control easy

August 8, 2012 Alec Jacobson 69

Page 70: Fast Automatic Skinning Transformations

#

Extra weights and disjoint skeletons make flexible control easy

August 8, 2012 Alec Jacobson 70

Page 71: Fast Automatic Skinning Transformations

#

Our reduction preserves nature of different energies, at no extra cost

August 8, 2012 Alec Jacobson 71

Surface ARAP Volumetric ARAP

V0surf = MsurfT V0

vol

= Mvol

T

Page 72: Fast Automatic Skinning Transformations

#

Our reduction preserves nature of different energies, at no extra cost

August 8, 2012 Alec Jacobson 72

Surface ARAP Volumetric ARAP

V0surf = MsurfT V0

vol

= Mvol

T

Page 73: Fast Automatic Skinning Transformations

#

Simple drag-only interface for point handles

August 8, 2012 Alec Jacobson 73

Page 74: Fast Automatic Skinning Transformations

#

Simple drag-only interface for point handles

August 8, 2012 Alec Jacobson 74

Page 75: Fast Automatic Skinning Transformations

#

Simple drag-only interface for point handles

August 8, 2012 Alec Jacobson 75

Page 76: Fast Automatic Skinning Transformations

#

Skinning rig enables FAST deformation

●  Substitute to reduce DOFs

August 8, 2012 Alec Jacobson 76

Page 77: Fast Automatic Skinning Transformations

#

Skinning rig enables FAST deformation

●  Substitute to reduce DOFs ●  Cluster rotations to reduce energy eval.

August 8, 2012 Alec Jacobson 77

Page 78: Fast Automatic Skinning Transformations

#

Skinning rig enables FAST deformation

●  Substitute to reduce DOFs ●  Cluster rotations to reduce energy eval. ●  Additional weights to expand subspace

August 8, 2012 Alec Jacobson 78

Page 79: Fast Automatic Skinning Transformations

#

Skinning rig enables FAST deformation

●  Substitute to reduce DOFs ●  Cluster rotations to reduce energy eval. ●  Additional weights to expand subspace

August 8, 2012 Alec Jacobson 79

Each innovation takes advantage of input skinning rig

Page 80: Fast Automatic Skinning Transformations

#

Future work and discussion

●  Alternative additional weights: sparsity? ●  Joint limits, balance, etc.

August 8, 2012 Alec Jacobson 80

Page 81: Fast Automatic Skinning Transformations

#

Acknowledgements

We are grateful to Peter Schröder, Emily Whiting, and Maurizio Nitti. We thank Eftychios Sifakis for his open source fast 3×3 SVD code. This work was supported in part by an SNF award 200021_137879 and by a gift from Adobe Systems.

August 8, 2012 Alec Jacobson 81

Page 82: Fast Automatic Skinning Transformations

Fast Automatic Skinning Transformations http://igl.ethz.ch/projects/fast Alec Jacobson ([email protected]), Ilya Baran, Ladislav Kavan, Jovan Popović, Olga Sorkine