EMOTIONS IN INTERACTION DESIGNLecture 13

History• During the 1990ies - a wave of new research on the role of emotion in diverse areas such as

psychology, neurology, medicine, and sociology

• Prior to this, emotions - a low-status topic of research

• researchers had mainly focused on how emotion got in the way of our rational thinking (results - focused on issues like when getting really scared, pilots would get tunnel vision and stop being able to notice important changes in the flight’s surroundings)

• emotions were seen the less valued pair in the dualistic pair rational – emotional, and associated with body and female in the “mind – body”, “male – female” pairs

• In 90ies It became clear that emotions were the basis for behaving rationally.

• Without emotional processes we would not have survived (being hunted by a predator (or enemy aircraft) requires focusing all our resources on escaping or attacking - Tunnel vision makes sense in that situation

• Unless we can associate feelings of uneasiness with dangerous situations, as food we should not be eating, or people that aim to hurt us, we would make the same mistakes over and over

The Four Pleasures framework – Tiger (1992)/Jordan(2000)• categorizes the four broad types of pleasure enjoyed by

people

• The Four Pleasure model is a framework that can be used to help evaluate how pleasurable a product will be use and own.

• can also be used to identify and generate opportunities to enhance a product

The Four Pleasures framework• The Four Pleasures

• Physio-pleasure

• Psycho-pleasure

• Socio-pleasure

• Ideo-pleasure

Physio-pleasure• Physio-pleasure is a sensual pleasure that is derived from

touching, smelling, hearing and tasting something.

• It also conveyed by an objects effectiveness in enabling an action to be performed.

• Ex: When we close a car door and it makes a satisfying clunk we experience a certain pleasure

Psycho-pleasure• Psycho-pleasures are pleasures that are derived from

cognition, discovery, knowledge, and other things that satisfy the intellect

• Games are enjoyable because they present challenges that we need to figure out (finishing the Rubik’s cube, or achieving checkmate in a few moves, there is a cognitive-emotional pleasure that is derived from such activities)

Socio-pleasure• Socio-pleasures, as the name suggests, are concerned

with pleasures derived from social signifiers of belonging, social-enablers and other social self-identification factors.

• Facebook is a tool that enables people to have a greater sense of community and involvement with one another.

Ideo-pleasure• Ideo-pleasures then are pleasures that are linked to our

ideals, aesthetically, culturally and otherwise.

• Aesthetic sensibilities are often closely linked to our ideological or cultural identity and determine to a great extent the pleasure a product may bring

Emotional Design• Attractive things works better? (cheap wine – fancy glass,

washed car – it drives better ;))

• Products that make people feel good, work better

• Emotional design framework:• Visceral – how things look, feel, sound – sensory inputs• Behavioral – how things function – effectiveness and usability• Reflective – self-image, personal satisfaction, memories – meaning

of things (influenced by knowledge, learning, culture)

Visceral

Behavioral

Reflective

Emotional Design• Design & trust:

• Trust comes from experience – products that perform accordingly to expectations

• Lack of trust comes from:• Lack of understanding• Lack of control

Design for Emotion• Design for emotion “comprises studying the emotional

experiences of users with products, as well as the emotional meanings assigned by users in relation to experience and interaction with products, assessing how emotions vary with different user characteristics and integrating users’ emotional expectations into the product development. It acknowledges the fact that the emotion is not a feature of the design, but a subjective experience of the user, owner or observer of the product.” ([Engage, 2005])

Design for Emotion• Advances in our understanding of emotion and affect have implications for the science of

design.

• Affect changes the operating parameters of cognition: • positive affect enhances creative, breadth-first thinking• negative affect focuses cognition, enhancing depth-first processing and minimizing distractions.

• it is essential that products designed for use under stress follow good human-centered design, for stress makes people less able to cope with difficulties and less flexible in their approach to problem solving.

• Positive affect makes people more tolerant of minor difficulties and more flexible and creative in finding solutions

• Products designed for more relaxed, pleasant occasions can enhance their usability through pleasant, aesthetic design.

• Aesthetics matter: attractive things work better. (D. A. Norman, 2002)

Design for Emotion• Product design that provides aesthetic appeal, pleasure and satisfaction can

greatly influence the success of a product.

• Traditional cognitive approaches to product usability have tended to underestimate or fragment emotion from an understanding of the user experience.

• Affect, which is inexplicable linked to attitudes, expectations and motivations, plays a significant role in the cognition of product interaction, and therefore can be usefully treated as a design aid.

• Emotion influences and mediates specific aspects of interaction before, during and after the use of a product.

• These affective states regularly impact how a user manipulates and explores a user interface in order to support a desired cognitive state.” (Frank Spillers, 2007)

 Ten Emotion Heuristics• Frowning (...) can be a sign of a necessity to concentrate, displeasure or

of perceived lack of clarity.

• Brow Raising (...) should also be considered a negative expressive reaction (...) is a sign of uncertainty, disbelief, surprise and exasperation

• Gazing Away (...) from the screen may be perceived as a sign of deception.

• Smiling (...) is a sign of satisfaction. The user may have encountered an element of joy during the evaluation process.

• Compressing the Lip (...) should be perceived as a sign of frustration and confusion (...) reflects anxious feelings, nervousness, and emotional concerns.

Ten Emotion Heuristics• Moving the Mouth (...) is associated with a sign of being lost and of uncertainty.

• Expressing Vocally (...) as well as the volume of the expression, the tone or quality of the expression may be signs of frustration or deception.

• Hand Touching the Face (...) is a sign of confusion and uncertainty, generally a sign of the user being lost or tired.

• Drawing Back on the Chair (...) negative or refusing emotions. By drawing back the chair, he / she [the user] may be showing a desire to get away from the present situation.

• (...) Leaning forward and showing a sunken chest may be a sign of depression and frustration with the task at hand (...) the user might be encountering difficulties but instead of showing refusal, leaning forward is a sign of attentiveness, of "getting closer".

Kansei engineering

• Kansei is a Japanese word and implies psychological feeling and needs in mind

• Kansei is the instantaneous feeling and emotion that we experience when we interact with things, such as products and services

• Kansei Engineering is a methodology for ensuring your product or service evokes desirable emotional responses. The process allows you to model customer’s instantaneous feelings and emotions and subsequently translate them into design parameters.

• Kansei is similar to psychology – grasping the image that exists in somebody’s mind

• Kansei engineering – transforming the image into something measurable – multidisciplinary science

Kansei Engineering• Before purchase of for example a passenger car one has images in mind of may be “a

powerful engine”, “easy operation”, “beautiful and premium exterior, “cool and relaxed interior” and so on.

• These words express the kansei, and the consumers really want to have such kind of a vehicle if the manufacturer succeeds in realizing a vehicle fitting to their imaginations.  

• Kansei engineering- a mechanism that technologically translates users Kansei into design elements

• A good product is more appealing to its consumers in terms of price, functions, colors, shape – represents users needs and has Kansei incorporated

• Kansei engineering – mix of feeling, emotion, and engineering

• First the product’s kansei is collected then the relationship to the product is established

Emotions research directions in HCI • In HCI, we understood the importance of considering users’ emotions explicitly in our

design and evaluation processes.

• the HCI research came to go in three different directions with three very different theoretical perspectives on emotion and design.

1. Rosalind Picard and her group at MIT - The cognitivistically inspired design approach she named Affective Computing 

2. counter-reaction to Affective Computing -instead of starting from a more traditional perspective on cognition and biology, the Affective Interaction approach starts from a constructive, culturally-determined perspective on emotion

3. emotion as part of a larger whole of experiences we may design for – we can name the movement Technology as Experience. In a sense, this is what traditional designers and artists have always worked with (see e.g. Dewey 1934) – creating for interesting experiences where some particular emotion is a cementing and congruous force that unites the different parts of the overall system of art piece and viewer/artist

Affective computing• basic idea: human rational thinking depends on emotional

processing

• it should be possible to create machines that relate to, arise from, or deliberately influence emotion or other affective phenomena

• the roots of affective computing came from neurology, medicine, and psychology

• implements a biologistic perspective on emotion processes in the brain, body, and interaction with others and with machines

Affective computing• Emotions, or affects, in users are seen as identifiable states

or at least identifiable processes.

• Based on the identified emotional state of the user, the aim is to achieve an interaction as life-like or human-like as possible, seamlessly adapting to the user’s emotional state and influencing it through the use of various expressions

• Applications: • affective learning - use an emotion model built on James A. Russell’s

model of affect relating phases of learning to emotions• training autistic children to recognize emotional states in others and

in themselves and act accordingly.

Affective computing• The most discussed and widespread approach in the design of

affective computing applications - to construct an individual cognitive model of affect from what is often referred to as “first principles”- the system generates its affective states and corresponding expressions from a set of general principles rather than having a set of hardwired signal-emotion pairs

• The model is combined with a model that attempts to recognize the user’s emotional states through measuring the signs and signals we emit in face, body, voice, skin, or what we say related to the emotional processes going on

• facial expressions - portraying different emotions - can be analyzed and classified in terms of muscular movements.

Affective computing

http://www.interaction-design.org/encyclopedia/affective_computing.html

Affective Computing• Limitations:

• simplification of human emotion in order to model it, • difficult approach into how to infer the end-users emotional states

through interpreting human behaviour through the signs and signals we emit

• Pros• provides for a very interesting way of exploring intelligence, both in

machines and in people.

Affective Computing• Tools for:• affective input

• facial recognition tools• voice recognition• body posture recognition• bio-sensor models

• affective output:• emotion expression for characters in the interface• regulating robot behaviours

Affective Interaction• sees emotions as constructed in interaction, whereas a computer

application supports people in understanding and experiencing their own emotions

• will not aim to detect a singular account of the “right” or “true” emotion of the user and tell them about it as in a prototypical affective computing application, but rather make emotional experiences available for reflection

• creates a representation that incorporates people’s everyday experiences that they can reflect on

• tries to avoid reducing human experience to a set of measurements or inferences made by the system to interpret users’ emotional states.

Affective Interaction• Approach

• recognizes affect as a social and cultural product

• relies on and supports interpretive flexibility

• avoids trying to formalize the unformalizable

• supports an expanded range of communication acts

• focuses on people using systems to experience and understand emotions

• focuses on designing systems that stimulate reflection on and awareness of affect

Affective Interaction• Affector is a distorted video window connecting neighbouring

offices of two friends (and colleagues). A camera located under the video screen captures video as well as 'filter' information such as light levels, colour, and movement. This filter information distorts the captured images of the friends that are then projected in the window of the neighbouring office. The friends determine amongst themselves what information is used as a filter and various kinds of distortion in order to convey a sense of each other's mood.

• eMoto is an extended SMS-service for the mobile phone that lets users send text messages between mobile phones, but in addition to text, the messages also have colorful and animated shapes in the background

Affective Interaction• Affective Diary - as a person starts her day, she puts on a body sensor

armband. • During the day, the system collects time stamped sensor data picking up

movement and arousal. • At the same time, the system logs various activities on the mobile phone: text

messages sent and received, photographs taken, and Bluetooth presence of other devices nearby.

• Once the person is back at home, she can transfer the logged data into her Affective Diary.

• The collected sensor data are presented as somewhat abstract, ambiguously shaped, and coloured characters placed along a timeline.

• To help users reflect on their activities and physical reactions, the user can scribble diary-notes onto the diary or manipulate the photographs and other data

Affective Diary

Technology as experience• a holistic approach to understanding emotion – emotions

should not be separated from other aspects of being in the world

• Emotion processes are part of our social ways of being in the world, they dye our dreams, hopes, and experiences of the world.

• design for emotions - place emotions in the larger picture of experiences, especially if we are going to address aspects of aesthetic experiences in our design processes

Kansei engineering• Kansei – difficult to measure

• Computer Aided Kansei Engineering

Kansei Engineering• The traditional Kansei engineering method includes the

following steps (Nagamachi, 2008):• Decision of product strategy (design domain + customer type)• Collection of kansei expressions that relate to the domain. Usually about

30-40 words are collected• A semantic differential scale is constructed• Samples that represent the domain are collected• Items/categories of the samples are identified, i.e. its "objective" features

are described• Subjects then evaluate each sample item with the constructed scale• Results are analyzed with standard multi-variate methods like factor

analysis, multiple regression or cluster analysis, or quantification theory (a non-parametric method developed by Komazawa and Hayashi)

• Results, i.e. ratings of sampled items as well as kansei structures are then interpreted, explained and mapped to the designer's sketches.

Affective computing• 2 approaches:

• emotion is constructed in interaction – between people and between people and machines > designing for emotion

• emotion is just one of the parameters we have to consider - instead of placing emotion as the central topic in a design process, it is now seen as one component contributing to the overall design goal.