The powerpoint slides
-
Upload
maxisurgeon -
Category
Documents
-
view
445 -
download
0
description
Transcript of The powerpoint slides
Facial Animation
Wilson Chang
Paul Salmon
April 9, 1999
Computer Animation
University of Wisconsin-Madison
Papers Used
Bregler C.,Covell M.,Slaney M., Video Rewrite: Driving Visual Speech with Audio. In SIGGRAPH 97 Conference Proceedings. ACM SIGGRAPH, August 1997
Guenter B.,Grimm C.,Wood D., Malvar H., Pighin F, Making Faces. In SIGGRAPH 98 Conference Proceedings. ACM SIGGRAPH, July 1998
Pighin F, Hecker J., Lischinski D., Szeliski R., Salesin D., Synthesizing Realistic Facial Expressions from Photographs. In SIGGRAPH 1998.
Waters K., A Muscle Model for Animating Three-Dimensional Facial Expression. In SIGGRAPH 1987.
Motivation
Creation of Virtual CharactersTeleconferencing & Video CompressionSimulated MovementFacial Surgery Planning
Why facial animation is hard.
Humans are very good at reading expressions.
Any slight deviation from a “correct” expression will be immediately noticed.
Deep-rooted instinct.
Three general catagories
2-D Facial Model3-D Facial ModelMuscular Model
2-D Facial Animation
Video Rewrite - modify and sync an actors’ lip motion to a new soundtrack.
Keyframe approach.Uses vision techniques to track mouth
movement.
Video Rewrite registration
Hand annotation of 26 images with 54 eigenpoints each.
Morph pairs to 351 images.Learn eigenpoint model.Warp images to standard reference plane.Eigenpoint analysis.
Audio Analysis
Video Rewrite uses TIMIT speech database.Triphones - emphasize middle.“teapot” = /SIL-T-IY/, /T-IY-P/, /IY-P-AA/,
/P-AA-T/, /AA-T-SIL/
Video Synthesis
Triphone Footage selection
error = Dp + (1- )Ds
Dp phoneme-context distance.
Ds distance between lip shapes.
Overall Lip Width & Height
Inner Lip Height
Height of Visible Teeth
Finish Synthesis
Compress and Stretch video.Align and blend mouth to face.
Results
Good Sync and natural articulation.Missing Triphones result in unnatural
speech
E_noreg.mov S_green.movS_wish.mov jfk-forestgump.mov
Making Faces
Motion capture.3D mesh via Cyberware Laser scanner.Deformed by
Position of 128 Dots• Manual identification - 1st frame
• Tracked by vision techniques
Texture Extraction Dot removal. Cylindrical map.
Synthesizing Realistic Facial Expressions from Photographs
3D facial models derived from photographs.Smooth transitioning between model
expressions.Adaptation from one model to another.
Model Fitting
Generic 3D mesh model.Pose Recovery - using
multiple subject views: Identify feature points. Deduce camera pose. Iteratively refine the generic
face model.
Model Fitting
Scattered Data Interpolation: Interpolate mesh between feature points. Uses radial basis functions.
Correspondence based shape refinement: Use less accurate correspondences. Polylines for eyebrows, eyelids, lips, etc. Not used in pose processing due to error.
Texture Extraction
View independent vs View dependent.Weight maps- bias selection of original
photograph: Self-occlusion. Smoothness. Positional certainty. View similarity.
View Dependent Texture Extraction
Select best photographs.Draw model for each photograph.Blend rendered image.Pros
adds detail.
Cons sensitive to original photo. More memory, slower.
View Independent Texture Extraction
Blend photographs to form single texture. Map onto virtual cylinder.
View Independent Texture Extraction
Blurry
Special Case Textures
Fine Detail - hair.Occlusion - eyes, teeth.Intricate Projection - ears. Shadowing - eyes, teethSolutions
Use photo with highest visibility. Simulate shadowing
Expression Morphing
Simplified by common mesh.Linearly interpolated vertices.Blend result of rendering with each texture.
Synthesize new expressions via: Global blend. Regional blend. Painterly interface.
ResultsSmooth transitioned expressions:
ResultsApplied transitions to different human
subject:
Our conclusions
Good results between models.Relatively inexpensive equipment.Notable manual processing.
Muscular Modeling
Easy generalized across models.22 muscle groupsFacial Action Coding System (Ekman,
Wallace) - Action Unit parameterization
Anatomy
Skin as Mesh
Nodal mobility Tensile Strength of skin Proximity to muscle attachment Depth of tissue & proximity to bone Elasticity & interaction with other muscles
Network of springs p = F/k
Mesh expression examples
Muscle types modeled
Linear/parallel musclesSphincter muscles
Linear/parallel muscles
Sphincter muscles
Animating
Not in paperBuild a libraryAbstract languageKeyframe