March 1, 20021ICT Virtual Human Workshop HUMAN FIGURE ANIMATION Norman I. Badler Center for Human...

March 1, 2002 1ICT Virtual Human Workshop

HUMAN FIGURE ANIMATION

Norman I. BadlerNorman I. Badler

Center for Human Modeling and SimulationCenter for Human Modeling and Simulation

University of PennsylvaniaUniversity of Pennsylvania

Philadelphia, PA 19104-6389 USAPhiladelphia, PA 19104-6389 USA

http://www.cis.upenn.edu/~badlerhttp://www.cis.upenn.edu/~badler

Norman I. BadlerNorman I. Badler

Center for Human Modeling and SimulationCenter for Human Modeling and Simulation

University of PennsylvaniaUniversity of Pennsylvania

Philadelphia, PA 19104-6389 USAPhiladelphia, PA 19104-6389 USA

http://www.cis.upenn.edu/~badlerhttp://www.cis.upenn.edu/~badler


Outline

• Generative Methods and TradeoffsGenerative Methods and Tradeoffs• Re-use and RetargettingRe-use and Retargetting• Parametric Models and ControlsParametric Models and Controls• Real-time or Offline? (i.e., Hard problems)Real-time or Offline? (i.e., Hard problems)• Model AvailabilityModel Availability• Body Requirements and AssumptionsBody Requirements and Assumptions• Object – Virtual Human InteractionsObject – Virtual Human Interactions

• Generative Methods and TradeoffsGenerative Methods and Tradeoffs• Re-use and RetargettingRe-use and Retargetting• Parametric Models and ControlsParametric Models and Controls• Real-time or Offline? (i.e., Hard problems)Real-time or Offline? (i.e., Hard problems)• Model AvailabilityModel Availability• Body Requirements and AssumptionsBody Requirements and Assumptions• Object – Virtual Human InteractionsObject – Virtual Human Interactions


The “Realism Ceiling” for Human Models

Visual RealismVisual Realism

Time toTime tocreatecreate

BehavioralBehavioral RealismRealism

SpecialSpecialeffectseffects

InanimateInanimateobjectsobjects

Life-likeLife-likevirtualvirtual

humanshumans

Real-timeReal-time AgentsAgents


What Approaches Push the Curve Toward More Realism?

Visual RealismVisual RealismInanimateInanimateobjectsobjects

MotionMotionCaptureCapture

ParameterizationParameterization(face, gait, gesture)(face, gait, gesture)


humanshumans



Real-timeReal-time


Why is Realism Still Hard?

Visual RealismVisual RealismInanimateInanimateobjectsobjects

MotionMotionCaptureCapture

ParameterizationParameterization(face, gait, gesture)(face, gait, gesture)

Difficultto

generalize

What are the“right”

parameters?


humanshumans



Real-timeReal-time


ProceduresProcedures Use models;Use models;

Generalize control;Generalize control;

FastFast

Find models;Find models;

Specific code;Specific code;

Integration w/othersIntegration w/others

Motion Motion CaptureCapture

Natural;Natural;

Nuance;Nuance;

PhysicsPhysics

Actors; Acting; Actors; Acting;

Motion specificity;Motion specificity;

RetargettingRetargetting

DynamicsDynamics

(Physics-(Physics-basedbased

simulation)simulation)

Physics;Physics;

Apparent interlimb Apparent interlimb coordination;coordination;

Free fall (rag doll)Free fall (rag doll)

Control laws;Control laws;

Model complexity;Model complexity;

RetargettingRetargetting

Collision responseCollision response

Trade-Offs Advantages Disadvantages


Human Movement Categories

• Voluntary (task-oriented, reach, look-at)Voluntary (task-oriented, reach, look-at)

• Involuntary (breathing, balancing, blinking)Involuntary (breathing, balancing, blinking)

• SubconsciousSubconscious– Low level motor functions (fingers, legs, lips)Low level motor functions (fingers, legs, lips)

– Communicative acts (facial expressions, limb Communicative acts (facial expressions, limb gestures, body posture) gestures, body posture)

What generative techniques work for these?What generative techniques work for these?

• Voluntary (task-oriented, reach, look-at)Voluntary (task-oriented, reach, look-at)

• Involuntary (breathing, balancing, blinking)Involuntary (breathing, balancing, blinking)

• SubconsciousSubconscious– Low level motor functions (fingers, legs, lips)Low level motor functions (fingers, legs, lips)

– Communicative acts (facial expressions, limb Communicative acts (facial expressions, limb gestures, body posture) gestures, body posture)

What generative techniques work for these?What generative techniques work for these?


Human Movement Implementations

• Voluntary Voluntary (procedures [IK], mocap)(procedures [IK], mocap)

• Involuntary Involuntary (procedures [physiological or dynamic (procedures [physiological or dynamic models])models])

• Low level motor functions Low level motor functions (procedures, mocap, (procedures, mocap, dynamics)dynamics)

• Communicative acts Communicative acts (mocap, procedures)(mocap, procedures)

category X technique = {animation paper}category X technique = {animation paper}

• Voluntary Voluntary (procedures [IK], mocap)(procedures [IK], mocap)

• Involuntary Involuntary (procedures [physiological or dynamic (procedures [physiological or dynamic models])models])

• Low level motor functions Low level motor functions (procedures, mocap, (procedures, mocap, dynamics)dynamics)

• Communicative acts Communicative acts (mocap, procedures)(mocap, procedures)

category X technique = {animation paper}category X technique = {animation paper}


MoCap Retargetting

• Joint angles don’t neatly map to different Joint angles don’t neatly map to different size or structured figures.size or structured figures.• Need to maintain constraints (IK) to avoid Need to maintain constraints (IK) to avoid feet slipping and sliding.feet slipping and sliding.• Need to add IK to change reach and look-at Need to add IK to change reach and look-at direction.direction.• Moving interactions may fit but change Moving interactions may fit but change dynamics (e.g., figure skating pairs).dynamics (e.g., figure skating pairs).

• Joint angles don’t neatly map to different Joint angles don’t neatly map to different size or structured figures.size or structured figures.• Need to maintain constraints (IK) to avoid Need to maintain constraints (IK) to avoid feet slipping and sliding.feet slipping and sliding.• Need to add IK to change reach and look-at Need to add IK to change reach and look-at direction.direction.• Moving interactions may fit but change Moving interactions may fit but change dynamics (e.g., figure skating pairs).dynamics (e.g., figure skating pairs).


Parametric Models (Procedures)

• IKIK

• GaitGait

• FacesFaces

• EyesEyes

• Muscles (Deformations)Muscles (Deformations)

• Gestures (BEAT/MagiCster)Gestures (BEAT/MagiCster)

• EMOTE EMOTE

• IKIK

• GaitGait

• FacesFaces

• EyesEyes

• Muscles (Deformations)Muscles (Deformations)

• Gestures (BEAT/MagiCster)Gestures (BEAT/MagiCster)

• EMOTE EMOTE


IK at Work

Eye view;Note attentionControl

Instructions translated to Instructions translated to parameterized actions.parameterized actions.


Walking (Sun and Metaxas)

Gait from parameterized biomechanical Gait from parameterized biomechanical model, original movements from motion model, original movements from motion capture and interpolated based on terrain capture and interpolated based on terrain angle.angle.

moviemovie

Gait from parameterized biomechanical Gait from parameterized biomechanical model, original movements from motion model, original movements from motion capture and interpolated based on terrain capture and interpolated based on terrain angle.angle.

moviemovie


Gesture Control: EMOTE

• EMOTE: A real-time motion modification EMOTE: A real-time motion modification system.system.

• Defines Defines qualitiesqualities of movement with 8 of movement with 8 parameters.parameters.

• Based on Effort and Shape components of Based on Effort and Shape components of Laban Movement Analysis.Laban Movement Analysis.

• EMOTE: A real-time motion modification EMOTE: A real-time motion modification system.system.

• Defines Defines qualitiesqualities of movement with 8 of movement with 8 parameters.parameters.

• Based on Effort and Shape components of Based on Effort and Shape components of Laban Movement Analysis.Laban Movement Analysis.


Inputs EMOTE Output

Key PosesKey Poses

End EffectorEnd EffectorGoalsGoals

Motion CaptureMotion Capture

ProceduresProcedures

Frame Frame RateRatePosesPoses

4 Efforts 4 Efforts

4 Shapes4 Shapes

InverseInverseKinematicsKinematics

InterpolationInterpolation


Motion Qualities are Important

• Movements with EMOTE qualities give Movements with EMOTE qualities give insight into the agent’s cognitive state.insight into the agent’s cognitive state.

• When EMOTE qualities spread from limbs When EMOTE qualities spread from limbs to body, movements appear more sincere.to body, movements appear more sincere.

• Movements with EMOTE qualities give Movements with EMOTE qualities give insight into the agent’s cognitive state.insight into the agent’s cognitive state.

• When EMOTE qualities spread from limbs When EMOTE qualities spread from limbs to body, movements appear more sincere.to body, movements appear more sincere.


Don’t Animators Know This?

Good key pose animators know this and use Good key pose animators know this and use character deformation to great advantage;character deformation to great advantage;

EMOTE allows this concept to be applied EMOTE allows this concept to be applied parametrically (under program or interactive parametrically (under program or interactive control) to articulated (jointed) figures.control) to articulated (jointed) figures.

Good key pose animators know this and use Good key pose animators know this and use character deformation to great advantage;character deformation to great advantage;

EMOTE allows this concept to be applied EMOTE allows this concept to be applied parametrically (under program or interactive parametrically (under program or interactive control) to articulated (jointed) figures.control) to articulated (jointed) figures.


An Actor Trying to Act (Apologies to W. Shakespeare)

We’ll manually match these movements inWe’ll manually match these movements inour virtual human model, then vary theour virtual human model, then vary themotion qualities via EMOTE.motion qualities via EMOTE.


Manually Matching the Motions


The Actor with Neutral Efforts (A Politician?)


Actor with Less Rising Shape (Not quite as excited?)


Moving the Shapes Inward (Woody Allen?)


With Light and Sustained Efforts (More solemn and serious?)


Without Torso Movements (Used Car Salesman?)


Hit the ball

… forcefully.

…softly.

Manner Variants (adverbs): HIT (with real dynamic response)


DI-GuyTM – Being combined with EMOTE (mocap + parameters)


Procedural Eye Movement Model from Human Performance Data

SourceSource

Eyes fixedEyes fixedaheadahead

Eyes movedEyes movedby statisticalby statistical

modelmodel

Full MPEG-4Full MPEG-4faceface


Important Problems to Solve in Real-Time• Constrained reach planning/executionConstrained reach planning/execution

• Coordination and consistencyCoordination and consistency

• Dynamics on demand (integrated with Dynamics on demand (integrated with constraints/goals)constraints/goals)

• Individuation (and its specification)Individuation (and its specification)

• Collision responseCollision response

• Deformable surfaceDeformable surface

• ClothingClothing

• Constrained reach planning/executionConstrained reach planning/execution

• Coordination and consistencyCoordination and consistency

• Dynamics on demand (integrated with Dynamics on demand (integrated with constraints/goals)constraints/goals)

• Individuation (and its specification)Individuation (and its specification)

• Collision responseCollision response

• Deformable surfaceDeformable surface



E.g.: Real-Time Constrained Reach Planning


Model Availability

• ““Standard” H-Anim (seamless) avatars Standard” H-Anim (seamless) avatars (VRML / WEB3D / MPEG-4)(VRML / WEB3D / MPEG-4)• DI-Guy (Joint angle control)DI-Guy (Joint angle control)• Jack (Toolkit API; plug-ins)Jack (Toolkit API; plug-ins)• Application-specific models (Extempo, Application-specific models (Extempo, Ananova, …)Ananova, …)• Custom models in Maya, 3DSMax, …Custom models in Maya, 3DSMax, …

Issues: Support, Platform, API, Body SwapIssues: Support, Platform, API, Body Swap

• ““Standard” H-Anim (seamless) avatars Standard” H-Anim (seamless) avatars (VRML / WEB3D / MPEG-4)(VRML / WEB3D / MPEG-4)• DI-Guy (Joint angle control)DI-Guy (Joint angle control)• Jack (Toolkit API; plug-ins)Jack (Toolkit API; plug-ins)• Application-specific models (Extempo, Application-specific models (Extempo, Ananova, …)Ananova, …)• Custom models in Maya, 3DSMax, …Custom models in Maya, 3DSMax, …

Issues: Support, Platform, API, Body SwapIssues: Support, Platform, API, Body Swap


Body Requirements and Assumptions

• Enough joints, correct placement, DOFs, limitsEnough joints, correct placement, DOFs, limits

• Skin surface shape (deformations)Skin surface shape (deformations)

• Torso shape and flexibilityTorso shape and flexibility

• Physiology: breathing, itching, eating, …Physiology: breathing, itching, eating, …

• Rendering: skin, hair, moisture, vascular changesRendering: skin, hair, moisture, vascular changes

• Human or caricature?Human or caricature?

• Resemblance to someone (or not)Resemblance to someone (or not)


• Enough joints, correct placement, DOFs, limitsEnough joints, correct placement, DOFs, limits

• Skin surface shape (deformations)Skin surface shape (deformations)

• Torso shape and flexibilityTorso shape and flexibility

• Physiology: breathing, itching, eating, …Physiology: breathing, itching, eating, …

• Rendering: skin, hair, moisture, vascular changesRendering: skin, hair, moisture, vascular changes

• Human or caricature?Human or caricature?

• Resemblance to someone (or not)Resemblance to someone (or not)



Object – Human Interactions: Challenges

• Grasping (one / two hands, or with other Grasping (one / two hands, or with other body parts)body parts)

• Multi-person actions (carrying furniture)Multi-person actions (carrying furniture)

• Smart objects; Compliant motion Smart objects; Compliant motion (constraints applied to existing motions)(constraints applied to existing motions)

• Knowledge of how things operate, fit Knowledge of how things operate, fit together, disassembletogether, disassemble

• Grasping (one / two hands, or with other Grasping (one / two hands, or with other body parts)body parts)

• Multi-person actions (carrying furniture)Multi-person actions (carrying furniture)

• Smart objects; Compliant motion Smart objects; Compliant motion (constraints applied to existing motions)(constraints applied to existing motions)

• Knowledge of how things operate, fit Knowledge of how things operate, fit together, disassembletogether, disassemble


The Agent needs to Know …

Action choices causeAction choices causeundesirable effectsundesirable effects

Actions changedActions changedto prevent errorsto prevent errors


Questions!

March 1, 20021ICT Virtual Human Workshop HUMAN FIGURE ANIMATION Norman I. Badler Center for Human...

Documents

Transcript of March 1, 20021ICT Virtual Human Workshop HUMAN FIGURE ANIMATION Norman I. Badler Center for Human...