Emotion recognition - a new design paradigm

38
GRAPHING CUSTOMER EMOTION IN TURBOTAX IN REAL TIME 1

Transcript of Emotion recognition - a new design paradigm

Page 1: Emotion recognition - a new design paradigm

GRAPHING CUSTOMER EMOTION IN TURBOTAX IN REAL TIME

1

Page 2: Emotion recognition - a new design paradigm

Agenda1. background

2. technology

3. experiments, insights and customer benefits

4. opportunities, risks and next steps

2

Page 3: Emotion recognition - a new design paradigm

Part 1. Background

3

Page 4: Emotion recognition - a new design paradigm

4

Page 5: Emotion recognition - a new design paradigm

43 muscles and 23 action units (AUs) – combinations of facial muscle activity that correspond to seven core emotions – are studied on a

second-by-second basis.

Pursuant to US Patent #7,113,916

5

Page 6: Emotion recognition - a new design paradigm

6

Page 7: Emotion recognition - a new design paradigm

Why now?

7

Page 8: Emotion recognition - a new design paradigm

SxSW 2015 + CONVERGENCE

8

Page 9: Emotion recognition - a new design paradigm

9

Gathering emotion response data will allow us to see our customers in a completely new way. It will unlock the unique individual behind the faceless demographic.

Page 10: Emotion recognition - a new design paradigm

This would allow us to…

10

Serve up different versions our tax-prep experience according to emotional responses

Capture a completely new dimension of data in analytics

Change marketing offers based on emotion response

Better facilitate customer care (especially agent interaction)

Page 11: Emotion recognition - a new design paradigm

Baseline requirements

11

real time fast accurate seamless integration

Page 12: Emotion recognition - a new design paradigm

Part 2. Technology

12

Page 13: Emotion recognition - a new design paradigm

13

Page 14: Emotion recognition - a new design paradigm

14

Page 15: Emotion recognition - a new design paradigm

Technology Highlights

Valence, Engagement, and other emotions

SDK runs native/local on mobile + desktop platforms

Real-time, reads every 250 ms

Transient — no images are stored

15

Page 16: Emotion recognition - a new design paradigm

Hands-on demo!

16

Page 17: Emotion recognition - a new design paradigm

Part 3. Experiments, insights and customer benefits

17

Page 18: Emotion recognition - a new design paradigm

Experiment A positive reaction (valence >0)

negative reaction (valence <0)

AFFECTIVA measure

page 1

page 2A

page 2B

Page 19: Emotion recognition - a new design paradigm

19

WHAT THE CUSTOMER SEES

WHAT THE CAMERA SEES

Experiment A

Page 20: Emotion recognition - a new design paradigm

positive reaction negative reaction

Page 21: Emotion recognition - a new design paradigm

Experiment Apositive reaction

(valence >0)

negative reaction (valence <0)

measure

Page 22: Emotion recognition - a new design paradigm

22

negative valence

positive valence

Page 23: Emotion recognition - a new design paradigm

we needed data from real people

23

Experiment B

Page 24: Emotion recognition - a new design paradigm

24

Experiment B

-48

+35

Page 25: Emotion recognition - a new design paradigm

25

Experiment B

Page 26: Emotion recognition - a new design paradigm

we needed a controlled test on a specific event at a specific time

26

Page 27: Emotion recognition - a new design paradigm

We measured emotions between 3 points for a 3 second interval

Experiment C

number starts rolling number stops rolling 2 seconds after number stops rolling

THE EVENT THE TIME INTERVAL

0 1 3

When the EIC credit appears

Page 28: Emotion recognition - a new design paradigm

Stt

28

Experiment C

Page 29: Emotion recognition - a new design paradigm

noisy data - wide range of deviation on a single URL giving low

confidence on what the customer is experiencing

event driven data

With data tied to intervals around events we get an accurate reflection of emotion response.

29

Experiment C

Page 30: Emotion recognition - a new design paradigm

Stt

30

Experiment C

Page 31: Emotion recognition - a new design paradigm

31

Experiment B

Page 32: Emotion recognition - a new design paradigm

Customer benefitswe can understand our customers on a deeper level

we can respond to a negative or positive outcome in a way that builds confidence

we can provide customers with support when they need it most

we can and help in ways we never did before

we can identify areas of concern that are otherwise invisible

32

Page 33: Emotion recognition - a new design paradigm

Conclusions / insights

We can gather a completely new understanding of the end-to-end customer journey AND key moments in the customer experience

We can gather valuable, actionable analytics giving us a radically new data dimension, creating a strategic advantage that we have never had before.

33

Page 34: Emotion recognition - a new design paradigm

Part 4. Opportunities, risks and next steps

34

Page 35: Emotion recognition - a new design paradigm

35

There is no art/ to find the man's

construction in the face William Shakespeare, Macbeth

Page 36: Emotion recognition - a new design paradigm

36

Page 37: Emotion recognition - a new design paradigm

37

-12

Page 38: Emotion recognition - a new design paradigm

38

Thanks for your time!!

Damien Amir

Sharon Clarence