2012 Affect and Emotion-Oriented Systems

Affect and Emotion-Oriented Systems

Human-Computer Interaction

Master on Software Engineering Human-Computer Interaction

“I believe that logic + emotion are a winning combination. When useful and useable meet delight

great things happen. It’s about balance.” David Armano

Stefan Negru – www.blankdots.com

http://www.flickr.com/photos/sskennel/


What are emotions, feelings and the other affect related concepts ?



Definition: “Affect is a feeling or emotion as distinguished from cognition, thought, or action.”



“Emotion encompasses physiological, affective, behavioral, and cognitive components.”

Scott Brave & Clifford Nass, 2008



“Emotion is a reaction to events deemed relevant to the needs, goals, or concerns of an individual.”

Scott Brave & Clifford Nass, 2008



Four systems for emotion activation: Cognitive and noncognitive processes. By Izard, Carroll E. Psychological Review, Vol 100(1), Jan 1993, 68-90.



neurological structure of emotion (LeDoux, 1996)



emotion versus mood

moods are not directed at any object in particular and are

experienced as more diffuse, global, and general state

a person can be sad about something - an emotion or generally depressed - a mood



emotion versus mood

An emotion can lead to or take part in establishing a certain mood

Sadness (& other factors) depression



emotion versus feeling

Feelings relate more to a property associated to and object or being,

than to a person’s state of mind.

Feeling last more (days/weeks/years) than emotions (seconds) or moods (hours / days)



emotion versus feeling

Feelings are often pointed towards a class of objects, beings

distinguishable through a certain property

“I love Adobe CS applications!”

“I hate text-editing applications”



emotion and memory

negative events, which tend to be highly arousing, are

typically remembered better than positive events



emotion and performance

keeping a user happy may, therefore,

not only affect satisfaction, but may also lead to efficiency and creativity



A Circumplex model of affect (Russell, 1980).



Russell’s circumplex remodeled (Desmet, 2002; van Gorp, 2006)



emotion dimensions

Value

pleasant versus unpleasant

Arousal (psychological)

anxiety versus boredom



Anxiety, Boredom and Flow (Csikszentmihalyi, 1990; van Gorp 2006)



Flow

Achieving the desired state and keeping the user in that state is our goal.


Humane interfaces consider emotions



“Products were once designed for the functions they performed. But when all companies can make products that perform their functions equally well, the distinctive advantage goes to those who provide pleasure and enjoyment while maintaining the power. If functions are equated with cognition, pleasure is equated with emotion; today we want products that appeal to both cognition and emotion.”

Don Norman

“The Transmedia Design Challenge: Technology that is Pleasurable and Satisfying”, ACM Interactions Volume 17 Issue 1, January + February 2010




http://getsatisfaction.com/spotify/topics/when_will_we_have_a_blackberry_client

http://satisfaction.mint.com/mint/topics/account_classifications_we_need_ability_to_change_them



http://satisfaction.mint.com/mint/topics/account_classifications_we_need_ability_to_change_them



http://www.nbcchicago.com/news/local-beat/Frank_lloyd-wright-fallingwater-lego.html


It’s all nice, tidy and easy to understand right ?



It’s all nice, tidy and easy to understand right ?


Wishful thinking



Where’s the problem ?


A few Problems

1. Detecting Emotions 2. Labeling Emotions 3. Processing Emotions 4. Obtrusive interface



1. Detecting Emotions

Neurological responses

via electroencephalography (EEG) or magneto‐resonance imaging (MRI)

User input

Facial expression Voice



http://www.emotiv.com/

based on electroencephalography (EEG) technology




“Lie to me”– Tv show http://www.imdb.com/media/rm817007104/tt1235099



METT – Micro Expression Training Tool SETT – Subtle Expression Training Tool



Asking our user how he feels.



Detecting emotions in an Emotion-Oriented system


2. Labeling Emotions

Which are basic emotions ?

Group Emotions Mixed versus pure emotions




http://wefeelfine.org/ 12 million human emotions recorded since 2005 (2009 statement)



Emotion-Oriented Systems The Humaine Handbook, Springer 2011


Basic emotions

anger, disgust, fear, happiness, sadness and surprise



Labeling emotions

Labeling is important but not for a machine



A label for the machine is a x,y coordinate on the circumplex.



Proposal of labeling emotions

FeelTagged an Arduino-based Social-Emotional Interactive System – Stefan Negru 2010



3. Processing emotions

Is it a mood/feeling/emotion ? What category/dimension ?

Multiple and/or complex emotions Emotion intensity, timing and context observed ?

etc.


Annotating emotions

<emotionml xmlns="http://www.w3.org/2009/10/emotionml">

<emotion>

<dimensions set="PleasantAnxiety">

<pleasant value="1.3"/> <!—- somewhat positive pleasant state -->

<anxiety value="1.9"/> 

<modality set="PersonalContext" mode="face voice" medium="visual acoustic"/>

</dimensions>

</emotion>

</emotionml>

EmotionML http://www.w3.org/TR/emotionml/


5. Have an unobtrusive interface

Providing a useful, easy to understand and adaptive interface –

secondary goal

Unobtrusive interface- primary goal



emotion and attention

an interface capable of detecting, or predicting a user’s

emotional or mood state could similarly assume an affect‐regulation role, helping to guide attention away from

negative and toward more positive stimuli



emotion and attention

Disregarding the flow state

“This system failed to understand your command.” – blaming itself “The command was not understood.” – blaming no one “You did not speak clearly enough for your command to be understood.” – blaming the user



Affective based interaction

Affective aspects of interaction (Rogers, 2007)

frustration

how we can detect and reduce it?



Affective based interaction

Affective aspects of interaction (Rogers, 2007)

persuasive technologies

how we can change the user attitude & behavior by using the actual technologies?


http://www.happypills.es/


affective interaction

example: a robot hardware platform called iCAT uses a set of logical rules to convey emotional states as it makes decisions, with the goal of improving human-robot interaction Philips Research Laboratory 2005


http://www.newscientist.com/blog/technology/2007/07/what-if-robots-not-only-seemed.html





http://www.research.philips.com/



http://pleoworld.com/

Pleo is an animatronic dinosaur toy designed to emulate the appearance and (imagined) behavior of a week-old baby Camarasaurus.



http://blog.youeye.com/youeye-announces-webcam-emotion-tracking


Ethical aspects

The goal is not to control people actions but to support them in order to improve their emotional state and to provide a better interaction

based on the user’s emotional state



Dark side of emotions

Humans could have an offensive behavior regarding the digital entities

www.agentabuse.org





example (Tatsuya Nomura, 2005):

impolite behavior of the patients,

when they discovered that the therapist is a virtual agent





experiment (Christoph Bartneck et al., 2005):

humans tend to hurt more easily robots/devices

than people



Unobtrusive versus Informative

Should users be informed that their affective states are being monitored and adapted to ?




Questions

2012 Affect and Emotion-Oriented Systems

Career

Transcript of 2012 Affect and Emotion-Oriented Systems