Post on 21-Jul-2020
Real-time Performance-BasedFacial Animation
Mark Pauly
Eurographics 2012, Cagliari, Italy
Introduction
2
Eurographics 2012, Cagliari, Italy
Overview
Facial Tracking using Blendshape Model• rigid transformation + blendshape weights per frame
Personalized Blendshape Model• static face capture• example based facial rigging
Tracking• rigid & nonrigid registration• animation prior
3
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
Animation Pipeline
Eurographics 2012, Cagliari, Italy
Facial Expression Model
Static Face Capture
[Weise et al, 2009]
5
Eurographics 2012, Cagliari, Italy
Facial Expression Model
Face Fitting
[Li et al, 2009]
6
Eurographics 2012, Cagliari, Italy
Facial Expression Model
Texture Reconstruction
Poisson Image Editing[Pérez et al, 2003]
7
Eurographics 2012, Cagliari, Italy
Facial Expression Model
Example-based Facial Rigging
8
[Li et al, 2010]generic template personalized expression model
Eurographics 2012, Cagliari, Italy
Rigid Tracking
9
Estimate blendshape weights• similar to scans & color image• plausible animation
MAP estimation• non-rigid registration based on geometry & texture• animation prior from existing animation sequences
Eurographics 2012, Cagliari, Italy
Non-rigid Tracking
10
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
N-Dim Expression Space
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
N-Dim Expression Space
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
N-Dim Expression Space
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
Animation Manifold
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
Probabilistic Expression Prior
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
Probabilistic Animation Prior
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
Temporal Joint Probabilistic Distribution
Eurographics 2012, Cagliari, ItalyEurographics 2012, Cagliari, Italy
MAP Estimation
Eurographics 2012, Cagliari, Italy
Demo
www.faceshift.com