Nicolas Pronost

Definition and realisation of modeling methods and motion computation algorithms for virtual humans

Nicolas Pronost

Advisor : Bruno Arnaldi

Co-advisor : Georges Dumont

Team : IRISA - SIAMES

Thursday 7 December 2006

Nicolas Pronost – Definition and realisation of modeling methods and motion computation algorithms for virtual humans 2

Where are they found ?

Animation

Motion sciences

Robotics

Anthropology

Simulation

Video games Movies

Biomechanics Medicine, Health Sports Bipedal robot

Modern human Fossilised hominids Motion analysis Motion synthesis


Fondamental aspects

Modeling the human Symbolical and controlable representation Kinematical and physical properties

Modeling the motion Manipulable mathematical representation Dependant of the editing methods

Editing methods Based on motions laws Looking for generiness


Outline

Related works

Overview and motivations

Analysis / synthesis loop Kinematical adaptation

Evaluation of the dynamics

Forward dynamics synthesis

Conclusion and future work


Outline

Related works





Conclusion and future works


Modeling a virtual human

Consensus between the anatomical reality and motion control Manipulation of the rotational degrees of freedom

Simplification through a hierarchical representation of rigid bodies [H-Anim 06]

Mechanical joints are perfect and the number of limbs is limited

l_mid_foot

r_hand_center

vt1

l_hip

l_kneel_anklel_subtala

r

l_toe

r_hip

r_kneer_ankle

r_subtalarr_mid_foot

r_toe

vl2

vl3

vt4

vt5

vt6

head_top

r_clav

l_hand_center


Biomechanical modeling

The physical properties describe the movement capacity of the limbs At least the masses and inertias

These data are avalaible from anthropometrical tables and computable from regression laws [Vaughan et al. 99]

Cadaverical data [Dempster 55, Winter 90]

Gamma radiography [Zatsiorsky 90]

Various definitions of limbs [Chandler et al. 75, De Leva 96]

Additionnal data Articular limits, muscular activations [Liu et al. 05], articular

elasticities


Modeling the motion

Motion is a sequence of postures Marey’s work [Marey 1894] on motion decomposition

Kinematical model Positions, velocities and acceleration Description of the possibility of motions

Static and quasi-static (kinetical) models Usefull for slow motions Balance of internal and external forces

Dynamical model Use of motor forces at joints


Methods of kinematics and inverse kinematics

Representation by splines [Zeltzer 82, Bruderlin and Calvert 93]

Local linearisation and secondary tasks [Boulic and Thalmann 92] [Tolani and Badler 96]

Motion editing methods

θ1

θ2

X

J X I J J z

X f ()

f 1(X)



Kinematics and control of the center of mass Important on quasi-static positions [Phillips 91]

Projection of the center of mass on the sustentation polygon [Boulic et al. 94]

Resolution using inverse kinematics with a priority formulation [Boulic et al. 97]

Conservation of the Zero Moment Point Point of null result of linear momentum of the limbs [Tak et al.

00]

Dynamics filtering of motions [Yamane and Nakamura 03]



Methods of dynamics and inverse dynamics Animation engine using a system of secondary order

differential equations [Hodgins 98]

Newton’s laws of motion and fondamental physical laws

Virtual works and Lagrangian formalism [Rémion 00]

A qÝ Ý q B q, Ý q E

Forcess ms aCGsMomentss I CGS

CGS

d

dt

K

Ý q i

K

qi

Ei


Manipulation of real movements

Real movements intrinsically have all of the information of the motion Correct perception of the realism from few positions of

caracteristical joints [Johansson 73]

Good realism of animations Low generiness

Usable database for generating new motions



Adaptation to new characters: retargeting



Adaptation to new characters: retargeting With different morphologies [Gleicher 98] thanks to

spacetime constraints

Use of intermediate skeletons [Monzani et al. 00, Ménardais 03]

Decomposition of articular trajectories into hierarchical splines [Lee and Shin 99]

Take account of muscular forces [Komura et al. 00]



Modification of the motion Frequential description of the articular trajectories and

deformation function [Bruderlin and Williams 95]

displacement map, conservation of the high frequencies

Description by key postures and interpolation on the parameters [Witkin and Popovic 95]

Interpolation of the deformation function (scale and translation)

m '(t) m(t)(t)

m '(t) a(t)m(t)b(t)



Combining motions The motion graphs [Kovar et al. 02]

node = key posture arc = possible transition

Motion blending Synchronisation by dynamic time warping [Bruderlin and

Williams 95] Blending by linear combinaison or weighted sum [Guo

and Robergé 96, Park et al. 02, Rose et al. 98, Ménardais et al. 04]



Motion database Efficient interpolation on simple motions [Wiley et Hahn

97]

Behavioral or frequential decomposition Fourier space [Unuma et al. 95]

Radial basis function [Rose et al. 96]

Hidden Markov chains [Brand and Hertzmann 00]

Static models using PCA [Bowden 00]

Physical simulations [Zordan et al. 05, Arikan et al. 05, Tang et al. 06]


Bipedal locomotion

The locomotion is a cyclic movement Decomposable into phases [Marrey 1894, Plat and Veil 83]

A step is a half cycle walk = sequence of single and double support phases run = sequence of single support and flying phases

Specific modeling of locomotion With cyclic state machines [Multon 98]

With deformed hypertorus from PCA [Martineau 06]

Large number of biomechanical data Articular trajectories [Alexander 84, Patla 91]

Support phases [Girard 87]


Summary

The analysis and the synthesis of virtual human motions [Gibet 02] answer to very differents constraints according to the application Compromise interactivity / realism / generiness

Kinematical corrections Interactivity of natural looking motions

Dynamical constraints Physical realism on specific motions

Spacetime resolution Offline and kinematical high control

Physics-based simulation Generic offline production of motions

Motions editing Balanced applications


Outline

Related works







Overview

Motivation : to study and to realise a process of modeling tools and motion computation Application to locomotion

Database of motions

Adaptation algorithm

Analysis of the dynamics

Synthesis by forward dynamics

adapted motion

resulting forces

synthetised motion


Outline

Related works







Kinematical adaptation


adapted motion

Database of motions



resulting forces

synthetised motion

Database of motions



resulting forces

synthetised motion

Database of motions

Speed profilesScaleAnatomical data

Footprints

Rest posture

Articular limits

Interpolations

- transversal

- frontal

- sagital

Time adjustement and Synchronisation

Post-treatments

Adapted motion

Kinematical and

Dimensionnal

Interpolation


Modeling the human

Definition of a kinematical chain with 11 dof Spherical joints at pelvis and hips Pin joints at knees

right femur reference frame

world

pelvis reference frame

left femur reference frame

left tibia reference frame right tibia reference frame

3 rotations

3 rotations3 rotations

1 rotation1 rotation


Why use this model ?

Application field in paleoanthropology Study of the bipedalism of fossilised hominids Australopithecus afarensis Lucy (A.L. 288-1)

pictures – courtesy of G. Berillon


Modeling the locomotion

Treatments on the motion Homogeneous reconstruction [Ménardais 03]

Orientation of the locomotion Identification of the cycles

Definition of the movement of the articular centers Computed from real landmarks Accurate positions of articular

centers and virtual points1 12

2

33

44


Modeling the locomotion

A parametrical representation of the locomotion: the poulaine

Modeled by a cubic curve using 4 characteristic points of the cycle

1

23

4

Definition: the Cartesian displacement of the ankle in the root reference frame


Method of computation

The principle of dimensional interpolation in the database Definition of the step size on x Definition of the step shift on y Definition of the rest posture on z


Post-treatments

Adding the temporal dimension Use of an average profile of speed, normalised by the

time cycle and distance on the ground Representation of the profile by a polynomial function Integrating the function, computing the curvilinear x-

coordinate and the parameters of the cubic curves Synchronisation of the left and right poulaines

By minimisation of vertical differences By minimisation of ground sliding


Post-treatments

Computation of the postures by an IK solver

Proposition of secondary tasks [Nicolas et al. 04]

(C1) Maximal distance from joint limits (C2) Minimisation of the kinematical energy of rotation (C3) Search of the closest posture to the rest posture

Evaluation of these tasks by 3 criteria The total Jerk, third derivate of the angles The difference between the final and the initial posture The internal work

J X I J J z


Post-treatments

Preparation of the animation Construction of the foot and the ankle angle

Feet lenghts from anthropometrical tables [De Leva 96]

Trajectories of ankles computed by corrections of the ground penetrations

To go to the global motion Global displacement minimising the sliding

Upper body movement Adapted to the morphology and synchronised with the

real motion


Results of the adaptation

Validation of the interpolation

x axis x axis z axis Average

RMS (cm) 2.74 1.22 1.24 1.73

S.D. (cm) 1.18 1.12 0.76 0.82



Validation of the adaptation Comparison between real angular trajectories and

adapted trajectories of 7 subjects in the database

Pelv. incl. Pelv. obliq. Pelv. rot. int/ext Hip flex/ext

Average (rad) 0.03 0.01 0.01 0.05

S.D. (rad) 0.03 0.01 0.01 0.03

Hip abd/add Hip rot. int/ext Knee flex/ext

Average (rad) 0.04 0.03 0.02

S.D. (rad) 0.04 0.01 0.01



Simulations


Partial summary

The method computes a plausible locomotion from biomechanical knowledge and rules [Pronost et al. 05] Controled by limbs sizes, bones configuration, physical parameters

of the limbs, joints types, footprints, articular limits and the style of motion

Applied to paleoanthropology [Pronost et al. 06]

Future work Combining the method with a real time adaptation to the

environment To drive the extrapolation by physical properties A global resolution of the IK to reduce discontinuity of pelvis angles Increase the size and the diversity of the database


Outline

Related works









resulting forces


adapted motion

Database of motions


synthetised motion

Database of motions


synthetised motion


adapted motion

Mechanical model

Adapted motion

Scaled anthropometrical

table

Biomechanical model

Resulting forces and torques

Analysis of the

dynamics

Mapping

dimensional model

angle-based motion

Support phases recognition


Modeling the human

Creation of a biomechanical model Description of Denavit-Hartenberg

[Hartenberg and Denavit 55] Parameters of rotation:

user Parameters of translation:

auto Gender and nature of limbs

Gender: user Nature: auto Using anthropometrical tables

[deLeva 96] and regression laws [Vaughan et al. 99]


Modeling the motion

The mapping issue An iterative method on kinematical

chains from the root joint to the effectors

Using a sequencing of the articular systems

Treatment according to the number of degrees of freedom

1dof => pin joint, minimisation of the error 3 dof => spherical joint, an infinity of solutions

with constraints => minimisation of the future error without constraints => minimal rotation



In order to solve the inverse dynamics issue, we have to know the external forces applied to the system, for locomotion: the gravity

constant value for any motion the aerodynamical forces

supposed negligible the ground reaction forces

When are they applied ? Support phase recognition


Support phase recognition

Evaluation of 4 methods of ground contacts recognition hand-labeled, method of reference, accurate at the

frequency of the motion capture system speed, evaluation of a speed threshold for the effectors height, evaluation of a height threshold for the effectors configuration, particular configurations of the effectors

By four criteria the number of failures the average error its S.D. the normalised S.D. of the thresholds


Support phase recognition

Results with 12 x 2 (left/right) x 2 (flex/ext) = 48 contacts

Results of the thresholds estimation with heels and toes effectors

Our algorithm chooses the best method according to the configuration of the effectors and the evaluation of the criteria

Criteria/rank 1 2 3

Number of failure (0) speed (0) config (16) Height

Average error (2.3) speed (5.6) height (9.7) config

S.D. (1.6) speed (2.1) config (2.4) height

S.D. norm (-) config (0.41) height (0.5) speed

Heel strike Toe off

Height (%) 13.0 13.37

Speed (m/s) 0.73 0.39


Application of Newton’s law

Application of the FBD principle Free Body Diagram on each segment Study of forces and torques applied to the

limbs

Application of Newton’s second law of motion

on limbs s

SS

s

CGCGs

CGss

ITorques

amForces


Resolution of the equations

The translation form Single support, from the free foot to

the support foot Double support, global resolution

No support, independent resolution

The rotation form Iterative resolution of the equation:

CGO1 FR1CGO2 FR2

I CGÝ CG

s Rs T. I CGs

. Rs . Ý CGs


Validation of the resolution

Forces (in N) at left toe and torques (in N.m) at left knee of 6 adapted locomotions

For

ces

(N)

Tor

ques

(N

.m)

cycle % cycle % cycle %



Comparison between 3 real ground reaction forces (black plots) and analysed forces (blue plot) from characters with similar biomechanical properties

GR

F (

N)

support phase support phase support phase



Ground reaction forces of different styles of real locomotions

run

jump

walk


Influence of the retargeting

The global scale Most used parameter Large influence on the dynamics of the motion Linear relation (c.c. = 0.87) between the scale and the

relative values of the GRF Experimental validation between [0.7 , 1.2] scales

GR

F (

N)

cycle % global scale

rela

tive

GR

F



The femur/tibia ratio To evaluate errors on articular centers (relative length of

limbs) Experimental validation between [0.8 , 1.2] ratios The GRF are not compensated by relative lenghts of the

limbs

RM

S e

rror

cycle % femur/tibia ratio

GR

F (

N)



The structure of the skeleton Models with 33 and 21 dof (pin joints at knees, ankles and

elbows) Kinematical influence: 1.4 cm per limbs Dynamical influence: mostly on fore-aft acceleration Here, corresponds to 2.5 kg reduction (4.5 % of the mass)

GR

F (

N)

cycle % cycle %

RM

S e

rror


Influence of the kinematical interpolation

The step size To study the kinematical correction of support phases Results are in agreement with the interpolation: stage of

valid corrections (database of motions) To improve this database with more motions, with more step

sizes

GR

F (

N)

cycle % step size factor

RM

S e

rror



The motion style Defined by the rest posture, i.e. the erect percentile More bent style: nonsignificant errors More erect style: increase of the error Large steps can not be done with a hight erect posture

GR

F (

N)

cycle % erect percentile factor

RM

S e

rror



The character velocity Used for generating new motions Velocity , error but physics OK Velocity , double hump amplitude Need to change the style of locomotion (ex: run)

GR

F (

N)

cycle % velocity factor

RM

S e

rror


Partial summary

Method of evaluation of the dynamics influences of a locomotion editing method [Pronost and Dumont 06 a] The analysis is automatic, generic and independent [Pronost and

Dumont 06 b]

Applied on a retargeting approach and a kinematical interpolation in a database

The editing method is valid on the experimental range of validation

Future work To validate the analysis using more experimental data To overcome standard limitation of inverse dynamics issue, such as

complex articular systems and muscular activation To use the dynamics-based analysis to correct motions


Outline

Related works









resulting forces


adapted motion

Database of motions


synthetised motion

Database of motions


adapted motion


resulting forces

Symbolical equations of

motion

Resolution of the equations

Synthesised motion

Synthesis by

forward

dynamics

Dof variations

Initial state External forces Motor torques

Mechanical system

Velocities and

accelerations of the dof

Ground reaction forces

Bodies: masses, inertias, matrix Joints: type, mechanical parameters

Dof values


Modeling

Of the human Representation of the rigid bodies

masses, inertias and rotation matrix Representation of the mechanical joints

type: free (6 dof), spherical (3 dof), pin (1 dof) mechanical parameters

spring, damper, and limits

Of the motion Values of the degrees of freedom of the mechanical

system


Normalisation of the forces and torques Principle

Preliminary study on 5 motions


MotionInverse

dynamics analysis

MotionMorphology (mass, inertia)Normalised forces & torquesMotion ’ Forward

dynamics synthesis

Morphology ’

{

Subject 1 Subject 2

Normal N1,1,N1,2 N2

Bent F1 F2

N1,1 N1,2 F1 N2 F2

RMS(1) 0 0.234 0.621 0.512 1.001

M.E. 0 0.026 0.068 0.056 0.109

M.STDP (x10-3) 0 8.7 13.9 3.5 23.0

C.C. 1 0.973 0.681 0.980 0.399

C.ACP 1 0.995 0.322 0.332 0.328

Deriv (x10-4) 0 12.0 23.0 6.4 42.0

N1,1 N1,2 F1 N2 F2

RMS(1) 0 0.150 0.426 0.141 0.946

M.E. 0 0.025 0.071 0.026 0.148

M.STDP (x10-2) 0 22.6 55.6 22.9 129.3

C.C. 1 0.938 0.826 0.954 0.702

C.ACP 1 0.945 0.848 0.962 0.802

Deriv (x10-3) 0 36.5 74.1 37.3 162.2

Differences and similarities of root positions Differences and similarities of normalised GRF


Preliminary results

The forward dynamics resolution using the NMECAM library [Arnaldi 89, Dumont 90]

library defining a symbolic solver of motion equations Results on a virtual human without external forces

and with independent limbs


Partial summary

The whole of the principles are not yet implemented Apprehension of the phenomena of the forward dynamics

issue Usefull approach of normalisation in many fields such as

kinematics and kinetics synthesis To make choices according to physical properties of the

synthetised motion Future work

Simulation of a complete chain of a human with external forces

To organise a database including physical properties such as normalised profiles of forces and torques


Outline

Related works







Conclusion

To understand and to simulate human motions Analysis and synthesis approaches Many applicatives fields

animation, biomechanics, mechanics, anthropology Kinematical adaptation of captured motions preserving the

physical credibility of the synthetised motions Proposition of methods and algorithms combining

kinematical and dynamical approaches Adapted to the morphology and the locomotion Normalisation of the data, of the morphological structures

and the forces and torques Use of principles and data from biomechanics


Limitations and future work

On the methods Study of other motions with external forces To increase the size and the diversity of the database

and the experimental data for the interpolation and the physical validation

To use the analysis to drive the kinematical synthesis thanks to physical simulations

To do a unique method with the proposed approaches

On the applications Motion editing, biomechanics, anthropology But also on the modeling and the use of data on virtual

humans


Thank you for your attention

Definition and realisation of modeling methods and motion computation algorithms for virtual humans

Nicolas PronostAdvisor : Bruno Arnaldi

Co-advisor : Georges Dumont

Team : IRISA - SIAMES

Nicolas Pronost

Documents

Transcript of Nicolas Pronost