University of Paris 8University of Paris 8

Animation improvements and face Animation improvements and face creation tool for ECAscreation tool for ECAs

Nicolas Ech ChafaiNicolas Ech ChafaiBenjamin DariouchBenjamin DariouchMaurizio ManciniMaurizio ManciniCatherine PelachaudCatherine Pelachaud

OverviewOverview

aiming at improving agent's facial aiming at improving agent's facial animation quality:animation quality: we are studying some motion captured datawe are studying some motion captured data we apply results to our ECAwe apply results to our ECA

to allow the creation of individualized to allow the creation of individualized ECAs:ECAs: we developed one tool for MPEG4 face we developed one tool for MPEG4 face

creationcreation we propose some refinements to MPEG4 we propose some refinements to MPEG4

specificationspecification

MOCAP data analysisMOCAP data analysis

Three main goals:Three main goals:

displacement of FAPs during emotion displacement of FAPs during emotion presentationpresentation

synchronization between different FAPssynchronization between different FAPs FAP values during transition between FAP values during transition between

consecutive emotionsconsecutive emotions

Collected MOCAP dataCollected MOCAP data

2 actors2 actors 33 markers, 21 of them are the MPEG4 FAPs33 markers, 21 of them are the MPEG4 FAPs 78 sequences:78 sequences:

basic movementsbasic movements rising eyebrowsrising eyebrows smilingsmiling ……

basic emotionsbasic emotions angeranger happinesshappiness surprisesurprise ……

MOCAP problemsMOCAP problems

(we discovered that) obtaining usable (we discovered that) obtaining usable data is not straightforwarddata is not straightforward

right size and shape markers have to be right size and shape markers have to be used used

cameras have to be placed properlycameras have to be placed properly data has to be translated to the needed data has to be translated to the needed

reference systemreference system data has to be filtered from noisedata has to be filtered from noise

Data exampleData example

smilesmile

Video examplesVideo examples

frown clipfrown clip file: clips/coline 36 eyebrowsfile: clips/coline 36 eyebrows

fear clipfear clip file: clips/coline 56 fearfile: clips/coline 56 fear

Facial animation modelFacial animation model

FAPs displacement during basic emotionsFAPs displacement during basic emotions our model was simply based on onset-apex-our model was simply based on onset-apex-

offsetoffset

Data observed modelData observed model

on real data we observed other general on real data we observed other general behaviorsbehaviors

ResultsResults

we started to introduce the ADSR model:we started to introduce the ADSR model:given a sequence of given a sequence of (phase,intensity,duration) (phase,intensity,duration)

where phase is fromwhere phase is from {Attack, Decay, Sustain {Attack, Decay, Sustain Release}, Release}, the FAP curve is built using keyframe the FAP curve is built using keyframe Hermite interpolation:Hermite interpolation:

ADSR vs real dataADSR vs real data

ADSR exampleADSR example

clipclip

MPEG4 face toolMPEG4 face tool

MPEG4 face toolMPEG4 face tool imports models from Poserimports models from Poser allows the selection of the areas allows the selection of the areas

influenced by FDPsinfluenced by FDPs

Tool's featuresTool's features

automatic selection and symmetrizationautomatic selection and symmetrization automatic association automatic association region name region name

available FDPsavailable FDPs

ExampleExample

exports into a data file (containing exports into a data file (containing geometry + regions) readable from the geometry + regions) readable from the Greta's playerGreta's player

ExampleExample

flat.avi (note: female speech)flat.avi (note: female speech)

Added new FAPUsAdded new FAPUs

after adding new faces some refinements after adding new faces some refinements for the MPEG4 player will be neededfor the MPEG4 player will be needed

ExampleExample

clip without new FAPUsclip without new FAPUs clip with new FAPUsclip with new FAPUs

ConclusionsConclusions

more data has to be captured in proper waymore data has to be captured in proper way focus more on interaction between different FAPs focus more on interaction between different FAPs

and transition between sequential expressionsand transition between sequential expressions ADSR has to be fully implementedADSR has to be fully implemented for documentation, papersfor documentation, papershttp://www.iut.univ-paris8.fr/gretahttp://www.iut.univ-paris8.fr/greta

for Greta applications available to the other for Greta applications available to the other HUMAINE members please contact usHUMAINE members please contact usm.mancini@iut.univ-paris8.frm.mancini@iut.univ-paris8.fr