Science matters

[email protected]@mikehurst

Imagine a WorldMichael Hurst

Department of Information Science and Computer Science

Supervisors: Dr Tom W Jackson, Dr Mashhuda Glencross


Motivation

Imagine a world where a computer

can interact with you with the same

quality and fluidity with that as

another human being


The Current Situation is Very Different


Issues

• Can only work on what information they have

• Hand Location (Mouse)

• Written Text (Keyboard)

• Sight

• Smell

• Taste

• Hearing

• Touch


Addressing the Issues

• Historically, design was the answer

• However, one size fits all only goes so far

• Give computers the information people have

• Advancements mean technology is now capable

• Key question:

1. What is the key variable

involved in interaction?


The Research

Using a combination of readily available and

affordable devices that yield sensory data in the

hope to correctly predict the emotional state of a

user at any one time

• Why?

1. How can the Data can be Used (Motivation)


Motivation - Entertainment

Imagine a world:


Motivation – Productivity/Communication

Imagine a world:


Motivation

Imagine a world:


Imagine a World

• Do we really have to Imagine?


Research Questions

1. How do we collect the data needed to

understand emotional state?

2. What do we do with the sensory data once

collected?


How do we Collect the Data?

Give computers the information people have

• Facial Expressions

• Gestures

• Movement

• Vocal Tone

• Vocal Context

• Gaze Velocity

• Gaze Fixations

• Pupil Dilation

• Eye Movement


How do we Collect the Data?

Give computers the information people have

…And possibly further

• Heart Rate

• Heart Rate Variability

• Location of electrical

activity (EEG)


What to do with Sensory Data?

• Compare sensory data with participants self-

assessment of own emotional state to establish

relationships

• For each metric

• Find somatic marker, create a rule

• To combat participant subjectivity, deviation from mean

used

• Rules can be used by computers to assist user/alter

behaviour



• Development of an Emotional Salient Model

• Works in real time

• Determines emotional saliency at time t

• Based on likeliness of emotional state from somatic markers

• Researcher friendly

• Field Standard

• Expandable to incorporate new technologies, task data, other input

methods (twitter), personalisation etc.

• User friendly

• Model is dynamic using only technology that is available to user



Distressed(11%)

Aroused(11%)

Excited(11%)

Displeased(11%)

Neutral(11%)

Pleased(11%)

Depressed(11%)

Sleepy(11%)

Contented(11%)

Aro

usa

l

Valence

Russell’s Circumplex Model of Affect

19

1 9

• Baseline

probabilities

• Presence of

somatic markers

makes

probabilities

fluctuate



MicrosoftKinect

Heart Rate Sensor

EEG Sensor

Eye Tracking

Time t



MicrosoftKinect

Heart Rate Sensor

EEG Sensor

Eye Tracking

Time t

• Increase in Pupil Dilation • Rule Triggered• Emotional Probabilities

recalculated



MicrosoftKinect

Heart Rate Sensor

EEG Sensor

Eye Tracking

Time t

• Pupil Dilation Returns to Mean

• As t Progresses, Likeliness Reverts to Baseline



40% 15% 5%

15% 10% 3%

5% 3% 2%

Time t

Predicted Emotion (Salient Emotion)

Distressed

CPU Alters Behaviour(e.g. … )

• Tell user to take a break• If showing video/image turn off

• Inform friends and/or family• Pause game


To Conclude

• If your research involves people, could emotion

be used to further enhance your findings?

• Study Participation: bit.ly/AffectingHCI

• Questions?

http://www.bit.ly/AffectingHCI

http://www.bit.ly/AffectingHCI

Science matters

Sports

Transcript of Science matters