04.12.2012

1

Stephane GobronISIC labHE‐Arc, School of EngineeringHES‐SO, Switzerland

Introduction

Last presentation“Changing paradigm”… a relatively abstract topic

Current speech“Parameterizing emotion”, an impossible mission ?

Social networks…

3D environments

Sharing information

Thibau

t Weise, S

ofien 

Boua

ziz, Hao Li, Mark Pa

uly , 

LGG, E

PFL

Realtime Performance‐Based Facial Animation

/!\ Not to mix with associating users 

emotion to their avatar

Two short talks

1. October’s HE‐Arc speech

1. Intro to forum 3D VR fundamentals GUI’s limits… avatars’ 

freedom

1.2 An event‐based architecture an overview of major related 

issues

2. Today’s HE‐Arc speech

Parameterizing emotion for virtual worlds

1.1 Fundamentals & limitations

1.2 An event‐based architecture

Two short talks

1. October’s HE‐Arc speech

2. Today’s HE‐Arc speech

2.1 What is emotion and how it has been represented?

2.2 Emotional Expressions:Notions of emoFace and emoMotions

2.3 An emotional model and some results

Discussion & Questions

2.1 Emotion

2.2 “emoMotions”

2.3 Emotional model

04.12.2012

2

Emotion

What is « emotion »?

“An emotion is the complexpsychophysiological experience of an individual's state of mind…”

“Emotions are very complex experiences…”

Solution Psychological  

estimations Linguist statistics

“Lie to me”

Models of Emotion

Strategies

(1) Set of emotion

Try to extract clues for a set of emotion

Joy

Anger

Sadness

Fear

Surprise

Disgust

Emotion Masks used by Arne Za

slov

ecrea

ted by M

elod

y And

erson

Simple set of emotion

Models of Emotion

Strategies

(2) Correlated set of emotions

Main issue: limitation of classification

Plutch

ik’swhe

elof emotions

(198

0)

Correlated set of emotions

Models of Emotion

Strategies

(3) Dimensional model

Offer a seamless emotion from three dimensional emotion space: V. A. D.

Intuitive interface

Axis range: [‐1.0, 1.0]Arousal / Energy

Valence / Pleasure Dominance / Potency

References• Bush, L.E.: Individual differences multidimensional scaling of adjectives denoting feelings. Journal of Personality and Social Psychology 

25(1), 50–57 (1973)• Russell, J.A.: A circumplexmodel of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)• Scherer, K.R.: What are emotions? and how can they be measured? Social Science Information 44(4), 695–729 (2005)

Dimensional model, three dimensions

04.12.2012

3

VAD Model

Conversation Scene

From text and user……to emotion

Data mining Large text database Language classifiers Lexical classifiers

emoFaces VAD to facial expressions

emoMotions VAD to body motion

Text message

Data Mining

V, A, D values

emoMotions

emoFaces

User input

GUI

Formal emotional model

• 27 emoWords and emoFaces• 27 emoMotions• 3 animated examples• From one emotion to another

27 emoWords and emoFaces

Internal emoWords survey11 subjects from 25 to 43 years oldDerived 27 emotionalterms potentiallycorresponding to the 3D emotional space (VAD)

Designed 27 emoFaces for each emoWord

Use interpolation for real‐time use in VAD space

Low Dominance

Average Dominance

High Dominance

27 emoMotions

Captured 27 emoMotions for eachemoWord

Divide 27 emoMotionarea in the VAD space

Excited Surprised

Sleepy Depressed

04.12.2012

4

27 emoMotions

Three examples Sad/Bored

Sleepy Furious

Emotional Expressions in VAD

Change of emotionalaxis

Changing ValenceAxis Changing ArousalAxis Changing DominanceAxis

{‐0.7,0.6,1.0} {0.7,0.6,1.0} {‐0.7,‐0.6,‐0.7} {‐0.7,0.6,‐0.7} {0.6,‐0.6,‐0.8} {0.6,‐0.6,0.8}

Furious {‐,+,+} Excited {+,+,+} Bored/Sad {‐,‐,‐} Afraid {‐,+,‐} Peaceful {+,‐,‐} Vicious {+,‐,+}

emoFaces

emoMotions

• MakingVH More Alive• Types and flow of Emotion• Results and discusion

MakingVH More Alive

Breathing simulation

Lipmovement and speech

Turning head and gaze

Body movement

Emotional ModelEmotional Expression

Breathingsimulation

Lipmovement

Turning headand gaze

Body movements‐‐ especially

hands

04.12.2012

5

Types of Emotion

STE: Short Term Emotions

LTE: Long Term Emotions

IPE: Inter‐Personal Emotions

Let’s the valence, arousal, and dominance flows be Fv, Fa ,and Fd

, 1 sign

, 1

,

Three fields ( )

: self field: discussion field: personal field

Results from the European CYBEREMOTIONS grant 7th

framework program, main contribution from the ETHZ WP

Results

Control discussion and personal fields

Quentin SilvestreJunghyun

Ahn

MireilleClavien

Olivier Renault

Daniel Thalmann

Johann Conus

Ronan Boulic

MarcinSkowron

GeorgiosPaltoglou

ArvidKappas

Mike Thelwall

50 testers

David Garcia

StephaneGobron