1 Affective Computing for Game Design GAMEON-NA 2008 August 13, 2008 McGill University, Montreal,...

1

Affective Computing for Affective Computing for Game DesignGame Design

GAMEON-NA 2008August 13, 2008

McGill University, Montreal, Canada

Eva HudlickaPsychometrix Associates

Blacksburg, [email protected]

psychometrixassociates.com

2

OutlineOutline

• Emotions & Games: Affective Gaming

• Affective Computing

• Background on Emotion Research

• Computational Affective Modeling

• Conclusions

3

Trends in GamingTrends in Gaming

• More engaging, immersive– Multi-modal interaction (haptic controls)– Physical & emotional realism

SMILE math & science game: Villani & Wilbur, Purdue

Drake’s Fortune

Fully-immersive PTSD treatment environment, Fully-immersive PTSD treatment environment, Rizzo - ICTRizzo - ICT

4

Trends in GamingTrends in Gaming

• Adaptive gaming – Games sense user state & adapt gameplay

Emsense headset:

EEG, heart rate,respiration, headmotion, temperature

Emotiv SystemsEmotiv SystemsEEG

5

Trends in GamingTrends in Gaming• Serious games (fastest growing segment)

– Education & Training– Exergaming – Rehabilitation & therapy– Pain reduction in healthcare – Coaching (lifestyle & training games)– Psychotherapy

– … social change, government, healthcare, science, corporate training, military

Neuromatrix: Morphonix

FreeDive: Breakaway GamesVirtual Iraq-PTSD: Rizzo, ICT

6

Trends in GamingTrends in Gaming• Demographics

– Girls– Women– Adults– Elderly

7

We Are About Here…

In Terms of the Full Potential of Gaming..In Terms of the Full Potential of Gaming..

8

Existing Games Limited InExisting Games Limited In

• Interaction bandwidth• Narrative complexity• Player modeling• Adaptation to players• Affective realism of game characters• Affective complexity of game characters

9

We Are About Here…

So How Do We Get Here?

10

Emotion a Key Factor… Emotion a Key Factor… in both “Play” & “Work”in both “Play” & “Work”

• Mediates motivation

• Influences memory & learning

• Key role in decision-making & problem-solving

• Central factor in engagement

• Key factor in serious games

11

Affective Complexity of Games: Affective Complexity of Games:

Current …..Current …..

valencevalence

- +

arousalarousal-

Image fromRussell, 1997

12

…….. & Potential.. & Potential

valencevalence

- +

arousalarousal-

Images adapted fromLim & Aylett, 2005

13

Future Games Need To…Future Games Need To…• Recognize & adapt to players’ emotions

• Understand players’ affective profiles

• Increase affective complexity & autonomy of game characters

• Increase visual affective realism of game characters & player avatars

• Increase affective complexity of the entire game experience

14

Affecting Gaming & Affecting Gaming & Affective ComputingAffective Computing

““Affective Affective gaminggaming is concerned with the application of is concerned with the application of affective computingaffective computing techniques to the domain of digital techniques to the domain of digital games. games.

Affective gamingAffective gaming involves both the ‘evocation’ of emotions, involves both the ‘evocation’ of emotions, as well as the detection of player emotion.” as well as the detection of player emotion.” (Sykes, 2004)(Sykes, 2004)

……which will increasingly require which will increasingly require emotion modelingemotion modeling

15

OutlineOutline


• Affective ComputingAffective Computing



• Conclusions

16

Overview of Affective ComputingOverview of Affective Computing

MAX (Becker, Prendinger et al.)MAX (Becker, Prendinger et al.)BreazealBreazeal

De RosisDe Rosis HudlickaHudlicka

17

Affective ComputingAffective Computing• Broad area of interdisciplinary research and

practice relating computers and affect– “Anything that combines computing and emotions”

• Term coined by Rosalind Picard (MIT Media Lab)– 1997 book “Affective Computing” (MIT Press)– “How can emotions be generatedgenerated in computers,

be recognizedrecognized by computers, and be expressedexpressed by computers?” (Picard, Affective Computing, ‘97)

18

Affective ModelingAffective Modeling

Emotion Sensing & Emotion Sensing & RecognitionRecognition

““Emotion” Emotion” ExpressionExpression

OR?

GRETA, Fiorella de Rosis, U. Bari

19

Affective Computing Includes…Affective Computing Includes…• Emotion sensing & recognition

– via a variety of sensors from multiple modalities

• Generation of ‘affective’ behaviors in machines– Facial expressions in agents and robots– Affective synthetic speech– Affect-induced behavioral variation in robots and agents

• Computational models of emotion & affective phenomena– Emotion generation (via appraisal)– Emotion effects on cognition & behavior– Affective user models

• Cognitive-affective architectures ..for agents & robots– Generic requirements for modeling emotion– Characterizing emotion in computational terms

20

Methods & Techniques Relevant Methods & Techniques Relevant for Affect-Focused Game Designfor Affect-Focused Game Design

• Sensing & recognition of players’ emotions– Adaptive gaming– Game control

• Expression of emotions by game characters– More realistic & believable behavior– Complex social interactions

• Models of emotion in game characters– To support complex, autonomous behavior

– To support adaptive behavior

• Models of players’ emotions – Affective user models to support game adaptation

• Affective game evaluation– Develop games with desired affective profiles

21

OutlineOutline



• Background on Emotion ResearchBackground on Emotion Research


• Conclusions

22

So What ARE Emotions?So What ARE Emotions?

• Evaluative judgments of the: – World– Others– Self

• … in light of agent’s goals & beliefs

• …motivating & coordinating adaptive behavior

23

Roles of EmotionsRoles of Emotions

IntrapsychicIntrapsychicInterpersonalInterpersonal

WHAT? * Social coordination

* Rapid communication of behavioral intent;

HOW? Express emotions via:

-Facial expression-Speech (content & properties)-Gesture, Posture-Specific actions

WHAT? * Motivation

* Homeostasis * Adaptive behavior

HOW? - Global interrupt system- Goal management- Prepare for coordinated actions- Emotion generation (appraisal)- Emotion effects (processing biases)

24

How Do We Recognize an How Do We Recognize an Emotion if We See One? Emotion if We See One?

• Manifested across multiple, interacting modalities:

– Somatic / Physiological (neuroendocrine - e.g., heart rate, GSR)

– Cognitive / Interpretive (“Nothing is good or bad but thinking makes it so…”)

– Behavioral / Motivational (action oriented, expressive, ‘visible’)

– Experiential / Subjective (“that special feeling…”, consciousness)

• Much terminological confusion can be attributed to a lack of consideration of these multiple modalities of emotions– e.g., Is emotion a feeling or a thought? - It’s both

25

Simple Fear “Signature”: Simple Fear “Signature”: Large, Approaching ObjectLarge, Approaching Object

0

10

20

30

40

50

60

70

80

90

Physio BehaviorCognitive Subjective

Increased heart-rate;

Attacked?Crushed?

Flee?Freeze?

Feeling of fear

26

A Taxonomy of Affective FactorsA Taxonomy of Affective Factors

Affective States Emotions Moods

Basic ComplexNegative Positive

Affective Factors

Traits States

“Big 5”“Giant 3” …

AngerJoyFear …ShameGuiltPride

…

Attitudes, Preferences…

27

Alternative Views of EmotionsAlternative Views of Emotions• Categorical theories (discrete theories)

(Tomkins, Izard, Ekman, Panskepp)

– A small number of hardwired basic (fundamental) emotions exist with distinct neuromotor patterns

• Dimensional theories(Russell, & Thayer)– 2D: Valence & arousal – 3D: Valence & arousal + dominance– All other emotions defined in terms of these

• Componential theories (Scherer, Roseman, Smith and Ellsworth, Frijda)– Emotions result from the (parallel) evaluation of a number of appraisal

dimensions (novelty, valence, goal congruence…)– Vast numbers of emotions possible in this space– Individual appraisals represent components of familiar emotions

28

Categorical View: Categorical View: Basic EmotionsBasic Emotions

(Cohn, 2006)

29

Dimensional View: Dimensional View: Arousal & ValenceArousal & Valence

(Breazeal, 2003 adapted from

Russell, 1997)

30

Core Processes of EmotionsCore Processes of EmotionsEffects of Emotions

(on cognition & behavior)

Cognitive-Affective Architecture

Situations

Expectations

Goals

Cognitive Appraisal

Emotions

Generation of Emotions (via cognitive appraisal)

Stimuli

31

Emotion Generation via AppraisalEmotion Generation via Appraisal

StimuliStimuli

Appraisal ProcessAppraisal Process

EmotionsEmotionsAppraisal

Dimensions

Recalled

Perceived

Existing emotions, moods, traits

Imagined

Goals (desires, values, standards) Beliefs, Expectations

Domain-Independent Appraisal Dimensions

NoveltyValenceGoal / Need relevanceGoal congruenceAgency Coping potentialSocial and self norms and values

32

Emotion Effects on BehaviorEmotion Effects on Behavior

EmotionEmotion

Facial Facial expressionexpression

GesturesGestures

Posture

Behavior

Blah blah blahBlah blah blah

33

Emotion Effects on Cognition Emotion Effects on Cognition

• Emotion & cognition function as closely-coupled information processing systems– Complex feedback interactions

• Emotions influence fundamental processes mediating high-level cognition:– Attention & working memory speed & capacity – Long-term memory recall & encoding

• Anxiety– Attentional narrowing / threat bias / self-focus bias

• Anger– Risk tolerance / impulsive action bias / attribution of hostility

34

OutlineOutline




• Computational Affective ModelingComputational Affective Modeling

• Conclusions

35

Why Do We Need Emotion Why Do We Need Emotion Models in Game Design?Models in Game Design?

• Game characters’ emotions– Generate realistic & affectively-complex character behavior &

believable affective expressions– … responsive to dynamically-evolving game scenario & player

behavior– … in real-time– Emotion generation & emotion effects on cognition & behavior

• Players’ emotions– Help recognize player emotions in real-time– Develop gameplay adaptations– Generate more realistic player avatar behavior

36

Affective Modeling OutlineAffective Modeling Outline

• Core processes of emotion– …. Computational requirements– …. Representational alternatives

• Levels of resolution (tutorial)

• Affective architectures (tutorial)

37

Core Processes of EmotionsCore Processes of EmotionsEffects of Emotions

(on cognition & behavior)

Cognitive-Affective Architecture

Situations

Expectations

Goals

Cognitive Appraisal

Emotions

Generation of Emotions (via cognitive appraisal)

Stimuli

38

Models of Emotion Models of Emotion Generation via Generation via

Cognitive AppraisalCognitive Appraisal

39

Computational Tasks for Computational Tasks for Appraisal ModelsAppraisal Models

StimuliStimuli EmotionsEmotions

Emotion attributes:Emotion attributes:- Complexity of emotion construct * type * intensity * cause … * direction * …

Types of stimuli:Types of stimuli:- Internal / External- Real / Imagined - Past / Present / Future- Domain specific / Abstract appraisal dimensions- Complexity of stimulus structure- Mental constructs required (e.g., goals, expectations)

- Stimuli-to-emotion mappings- Intensity calculation- Nature of mapping process: * Stages & functions * Degree of variability-Integrating multiple emotions -Emotion dynamics over time

40

STIMULISTIMULI

Novelty

Valence

Goal relevance

Outcomeprobability

Urgency

Goal congruence

Agency

Coping potential

Norms

high

high

v. high

low

other

lowlow

low

high

FEARFEAR

41

Stimuli --> Appraisal Dimensions --> Emotion(s)Stimuli --> Appraisal Dimensions --> Emotion(s)

WorldWorld

…

N

C

A

GC

GR

V

N

This is thedifficult part!

EmotionsEmotions

??

42

Stimuli --> Appraisal DimensionsStimuli --> Appraisal Dimensions

WorldWorld

…

N

C

A

GC

GR

V

N

• Mental structures required:– Situations, Expectations, Beliefs, Values, Goals, Plans,

Causal structures, Agent history

• Large amounts of domain-specific knowledge required

• Complex reasoning required

• Typically implemented using symbolic representations– Rules, Semantic nets, Belief nets, Goal-Procedure hierarchies

43

Appraisal Dimensions --> Emotion(s)Appraisal Dimensions --> Emotion(s)

…

N

C

A

GC

GR

V

N

• Can be easy– Vector maps directly onto n-dim space of app. dimensions– Emotion identified via some measure of proximity (Euclidian dist.)

• Learning may be involved (connectionist models)

• BUT…– What if the dimension values are ambiguous?– How to combine multiple emotions?– How to calculate intensity?

• Individual variability– Across & within individuals– Across emotions– Across contexts

??

44

Questions Regarding Representational Questions Regarding Representational & Reasoning Requirements& Reasoning Requirements

• What must represented explicitly? – Time (present, past, future)

• Hope needs expectations, regret needs past– Mental constructs

• situations, expectations, goals– Memories

• what type – declarative, episodic, procedural– Explicit representation of the self

• need for complex emotions, social interaction, coping

• What types of reasoning are necessary? – What-if

• …to generate expectations which influence emotions– Causal explanation

• ..important for attribution

45

Representation & Reasoning Representation & Reasoning AlternativesAlternatives

• Vector spaces (Scherer)

• Connectionist (Araujo, Velasquez)

• Symbolic – Rules (Marinier, Jones, Henninger, Hudlicka…)– Belief nets (Hudlicka, de Rosis,…)

• Complex symbolic structures (Elliot, Reilly, Gratch & Marsella)– Appraisal frames, causal plan structures

• Spreading activation over networks of processes (Breazeal)

• Decision-theoretic – Decision trees – Decision theoretic formulations (Lisetti & Gmytrasiewicz, Busemeyer)

• Blackboards and ‘specialists’ • Finite state machines (Kopecek)

• Markov models (El Nasr)

• Theorem proving (Zippora)

• Dynamical systems

46

Modeling Emotion EffectsModeling Emotion Effects

EmotionEmotion

Facial Facial expressionexpression

GesturesGestures

Posture

Behavior

Blah blah blahBlah blah blah

Afternoon Tutorial

47

OutlineOutline





• ConclusionsConclusions

48

Affect-Focused Game DesignAffect-Focused Game Design

• To achieve the next generation of engaging & effective games we need to focus on emotion

• Affective considerations should influence all elements of game design– Visuals: Environment & Character appearance– Music– Narrative structure & plot– Character design– Gameplay design

49

Affect-Focused Game Design Affect-Focused Game Design (cont.)(cont.)

• Emotion research in psychology & neuroscience provides data & theories

• Affective computing provides methods & techniques

for…– Emotion modeling in game characters– Affective user modeling– Emotion recognition in players– Emotion expression in game characters

50

Thank youThank you

Eva HudlickaPsychometrix Associates

Blacksburg, [email protected]

psychometrixassociates.com

1 Affective Computing for Game Design GAMEON-NA 2008 August 13, 2008 McGill University, Montreal,...

Documents

Transcript of 1 Affective Computing for Game Design GAMEON-NA 2008 August 13, 2008 McGill University, Montreal,...