University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements...

20
University of Paris University of Paris 8 8 Animation improvements and face Animation improvements and face creation tool for ECAs creation tool for ECAs Nicolas Ech Chafai Nicolas Ech Chafai Benjamin Dariouch Benjamin Dariouch Maurizio Mancini Maurizio Mancini Catherine Pelachaud Catherine Pelachaud

Transcript of University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements...

Page 1: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

University of Paris 8University of Paris 8

Animation improvements and face Animation improvements and face creation tool for ECAscreation tool for ECAs

Nicolas Ech ChafaiNicolas Ech ChafaiBenjamin DariouchBenjamin DariouchMaurizio ManciniMaurizio ManciniCatherine PelachaudCatherine Pelachaud

Page 2: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

OverviewOverview

aiming at improving agent's facial aiming at improving agent's facial animation quality:animation quality: we are studying some motion captured datawe are studying some motion captured data we apply results to our ECAwe apply results to our ECA

to allow the creation of individualized to allow the creation of individualized ECAs:ECAs: we developed one tool for MPEG4 face we developed one tool for MPEG4 face

creationcreation we propose some refinements to MPEG4 we propose some refinements to MPEG4

specificationspecification

Page 3: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

MOCAP data analysisMOCAP data analysis

Three main goals:Three main goals:

displacement of FAPs during emotion displacement of FAPs during emotion presentationpresentation

synchronization between different FAPssynchronization between different FAPs FAP values during transition between FAP values during transition between

consecutive emotionsconsecutive emotions

Page 4: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Collected MOCAP dataCollected MOCAP data

2 actors2 actors 33 markers, 21 of them are the MPEG4 FAPs33 markers, 21 of them are the MPEG4 FAPs 78 sequences:78 sequences:

basic movementsbasic movements rising eyebrowsrising eyebrows smilingsmiling ……

basic emotionsbasic emotions angeranger happinesshappiness surprisesurprise ……

Page 5: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

MOCAP problemsMOCAP problems

(we discovered that) obtaining usable (we discovered that) obtaining usable data is not straightforwarddata is not straightforward

right size and shape markers have to be right size and shape markers have to be used used

cameras have to be placed properlycameras have to be placed properly data has to be translated to the needed data has to be translated to the needed

reference systemreference system data has to be filtered from noisedata has to be filtered from noise

Page 6: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Data exampleData example

smilesmile

Page 7: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Video examplesVideo examples

frown clipfrown clip file: clips/coline 36 eyebrowsfile: clips/coline 36 eyebrows

fear clipfear clip file: clips/coline 56 fearfile: clips/coline 56 fear

Page 8: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Facial animation modelFacial animation model

FAPs displacement during basic emotionsFAPs displacement during basic emotions our model was simply based on onset-apex-our model was simply based on onset-apex-

offsetoffset

Page 9: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Data observed modelData observed model

on real data we observed other general on real data we observed other general behaviorsbehaviors

Page 10: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ResultsResults

we started to introduce the ADSR model:we started to introduce the ADSR model:given a sequence of given a sequence of (phase,intensity,duration) (phase,intensity,duration)

where phase is fromwhere phase is from {Attack, Decay, Sustain {Attack, Decay, Sustain Release}, Release}, the FAP curve is built using keyframe the FAP curve is built using keyframe Hermite interpolation:Hermite interpolation:

Page 11: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ADSR vs real dataADSR vs real data

Page 12: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ADSR exampleADSR example

clipclip

Page 13: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

MPEG4 face toolMPEG4 face tool

Page 14: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

MPEG4 face toolMPEG4 face tool imports models from Poserimports models from Poser allows the selection of the areas allows the selection of the areas

influenced by FDPsinfluenced by FDPs

Page 15: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Tool's featuresTool's features

automatic selection and symmetrizationautomatic selection and symmetrization automatic association automatic association region name region name

available FDPsavailable FDPs

Page 16: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ExampleExample

exports into a data file (containing exports into a data file (containing geometry + regions) readable from the geometry + regions) readable from the Greta's playerGreta's player

Page 17: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ExampleExample

flat.avi (note: female speech)flat.avi (note: female speech)

Page 18: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

Added new FAPUsAdded new FAPUs

after adding new faces some refinements after adding new faces some refinements for the MPEG4 player will be neededfor the MPEG4 player will be needed

Page 19: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ExampleExample

clip without new FAPUsclip without new FAPUs clip with new FAPUsclip with new FAPUs

Page 20: University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai.

ConclusionsConclusions

more data has to be captured in proper waymore data has to be captured in proper way focus more on interaction between different FAPs focus more on interaction between different FAPs

and transition between sequential expressionsand transition between sequential expressions ADSR has to be fully implementedADSR has to be fully implemented for documentation, papersfor documentation, papershttp://www.iut.univ-paris8.fr/gretahttp://www.iut.univ-paris8.fr/greta

for Greta applications available to the other for Greta applications available to the other HUMAINE members please contact usHUMAINE members please contact [email protected]@iut.univ-paris8.fr