Model Reduction for Facial Model Reduction for Facial Animation and ProceduralAnimation and ProceduralGeneration of CitiesGeneration of CitiesJIRI DANIHELKA

Areas of interestAreas of interest Reduction of computer graphics for mobile phonesReduction of computer graphics for mobile phones

Mobile graphics – rendering, cross-platformMobile graphics – rendering, cross-platform Talking-head interface on mobile phonesTalking-head interface on mobile phones New reduction techniques of models for mobile New reduction techniques of models for mobile

phonesphones Reduction based on semantic informationReduction based on semantic information Reduction of city modelsReduction of city models Application of city models in mobile phonesApplication of city models in mobile phones

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(2)

Framework for creating 3DFramework for creating 3Dtalking-head applicationstalking-head applications Cross-platform designCross-platform design Speech SynthesisSpeech Synthesis Speech RecognitionSpeech Recognition Connection RequirementsConnection Requirements

– GPRS, EDGE, WiFi, ...GPRS, EDGE, WiFi, ...

Performance MeasurementsPerformance Measurements Power ConsumptionPower Consumption

– optimal architectureoptimal architecture– rendering vs. streamingrendering vs. streaming

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(3)

Performance measurement - resultsPerformance measurement - results

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(4)

Viseme reductionViseme reduction How does it workHow does it work

– find similar visemesfind similar visemes(e.g. “f” and “th”)(e.g. “f” and “th”)

– merge them togethermerge them together

ProblemsProblems– How to find similar visemes?How to find similar visemes?– How to merge them optimally?How to merge them optimally?

Reduction of Animated Models for Embedded Devices

(5)

How to find similar visemes?How to find similar visemes? Define a metric for visemesDefine a metric for visemes

– distance between two models A and Bdistance between two models A and B

– used several used several standard standard metrics in papermetrics in paper

Reduction of Animated Models for Embedded Devices

(6)

vA,1

vA,2

vA,3

vB,1

vB,2

vB,3

Problem definitionProblem definition For set of For set of NN visemes find different set of visemes find different set of MM visemes visemes

((MM<<NN) that ) that optimallyoptimally approximates original set approximates original set using selected metricusing selected metric

Input:Input:– visemes, viseme frequenciesvisemes, viseme frequencies– numbers N, Mnumbers N, M

Output:Output:– set of new visemesset of new visemes– it is not a subset of the original visemesit is not a subset of the original visemes– size of the new set is Msize of the new set is M

Reduction of Animated Models for Embedded Devices

(7)

Reduction algoritmReduction algoritm Goal: Reduce set with N visemes to M optimallyGoal: Reduce set with N visemes to M optimally Split original set with N visemes into two sets (each Split original set with N visemes into two sets (each

of them is not empty)of them is not empty) Solve two similar problemsSolve two similar problems

– reduce first subset to C optimally (0<C<M)reduce first subset to C optimally (0<C<M)– reduce second subset M-C optimallyreduce second subset M-C optimally

Do this for all possible ways of splitting of the Do this for all possible ways of splitting of the original set and all values of Coriginal set and all values of C

Use metric to choose the best wayUse metric to choose the best way Use dynamical programming approachUse dynamical programming approach

Reduction of Animated Models for Embedded Devices

(8)

What is in the paperWhat is in the paper Proof that output of our viseme merging algorithm Proof that output of our viseme merging algorithm

is optimal for selected metricsis optimal for selected metrics Proof that output of our viseme reduction algorithm Proof that output of our viseme reduction algorithm

is optimal for selected metricsis optimal for selected metrics

Algorithm is not very fastAlgorithm is not very fast– exponential time and memory complexityexponential time and memory complexity time time O(n2O(n2nn||||VV|||| + 4 + 4nnm)m) memory memory O(nO(n||||VV||||+2+2nnm)m)– runs 3 minutes on desktop computer (runs 3 minutes on desktop computer (2.4GHz2.4GHz processor) processor)– but has to run only once for each modelbut has to run only once for each model

Reduction of Animated Models for Embedded Devices

(9)

ConclusionConclusion reduction algorithm successfully appliedreduction algorithm successfully applied

– 1166 visemes reduced to 10 visemes reduced to 10– speed improvement from 5.4 FPS to 12.2 FPSspeed improvement from 5.4 FPS to 12.2 FPS– faster application startup (from 18 seconds to 8 seconds)faster application startup (from 18 seconds to 8 seconds)

Currently used in our applications on PDACurrently used in our applications on PDA– Virtual Customer Care CenterVirtual Customer Care Center– Virtual Shop AssistantVirtual Shop Assistant

Reduction of Animated Models for Embedded Devices

(10)

Procedural generation of citiesProcedural generation of cities

Generating multiple level of detailGenerating multiple level of detail State of the art – naive algorithmState of the art – naive algorithm

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(11)

Pseudo-infinite citiesPseudo-infinite cities City generating on demantCity generating on demant State of the art – simple regular grid onlyState of the art – simple regular grid only

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(12)

Dissertation ThesisDissertation Thesis

Title: Semantic reduction for facial animation and Title: Semantic reduction for facial animation and procedurally generated buildingsprocedurally generated buildings

Topics:Topics:– Semantic based viseme reduction of facial modelsSemantic based viseme reduction of facial models– Semantic LOD for proceduraly generated buildingsSemantic LOD for proceduraly generated buildings– Generating pseudo-innite cities in real-timeGenerating pseudo-innite cities in real-time

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(13)

Questions and answersQuestions and answers Does any connection exist between the above

mentioned arts of the proposed dissertation work? If so, please, could you explain that connection?

Can the framework created for the 3D facial animation be used in more general graphics applications? If so, how and what effort it would take?

How realistic is it to expect that the real-time city generator would be done by the end of the preparation of the work? What platform do you expect to use?

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(14)

Questions and answersQuestions and answers Does any connection exist between the above

mentioned arts of the proposed dissertation work? If so, please, could you explain that connection?

Semantic reduction of face models Semantic reduction of procedurally generated models Pseudo-infinite cities in real-time Localization in cities

City model in your mobile phone

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(15)

Questions and answersQuestions and answers Can the framework created for the 3D facial

animation be used in more general graphics applications? If so, how and what effort it would take?

Current projects:Current projects:– Sign language for talking heads on mobilesSign language for talking heads on mobiles– Multimodal web browserMultimodal web browser

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(16)

Questions and answersQuestions and answers How realistic is it to expect that the real-time city

generator would be done by the end of the preparation of the work? What platform do you expect to use?

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(17)

Questions and answersQuestions and answers

Infinite citiesInfinite cities Procedural citiesProcedural cities

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(18)

Model Reduction for Facial Animation and Procedural Generation of Cities, Jiri Danihelka

(19)

Thank for your attention!Jiri Danihelka