From Body Signals to Brainy Player Insights

Lennart Nacke, Ph.D. Associate Professor

CC by Lennart Nacke/Flickrwww.hcigames.com, @acagamic, #gamesUR, lennart.nacke@acm.org

About me• Associate Professor

• Director of the HCI Games Group at the University of Waterloo

CC by Liz Henry/Flickr

Body Signals

Electrodermal Activity• EDA

• Primary index of arousal

• Increased sweat gland activity

• Sympathetic nervous system

Facial Electromyography• EMG • Muscle contraction result in electrical activity • Primary index of hedonic valence

Brow muscle (negative valence)

Periocular muscle (high arousal, positive valence)

Cheek muscle (positive valence)

C3 CMS DRLCz C4

F3 F4

FC5 FC1 FC2 FC6

Fz

Pz

Oz

P4

AF4

Fp2

PO4

O2O1

CP2

F8

T8T7

P8

CP6

PO3

P3P7

CP1CP5

AF3

F7

Fp1

Chatrian, G. E., Lettich, E., & Nelson, P. L. (1988). Modified nomenclature for the "10%" electrode system. Journal of Clinical Neurophysiology: Official Publication of the American Electroencephalographic Society, 5(2), 183-186.

Frontal (F)

Parietal (P)

Occipital (O)

Temporal (T)Central (C)

Odd Numbers Even Numbers

Electroencephalography

Psychophysiological Inference

• Body and Mind Relations • One-to-one • One-to-many • Many-to-one • Many-to-many • Null relations

• Measure operation and activity of • Muscles • Nerve cells • Glands

Raw Physiological Signal (e.g., EMG)

0 1 2 3 4 5 6 7

x 104

6

6.5

7

7.5

8

8.5

9

9.5

10

10.5

11

Signal Processing

1. Rectify signal (RMS)

2. Filter (bandpass)

3. Smooth (average window)

4. If EEG → Fast-Fourier Transformation

Better EMG Signals

Normalized Skin Conductance 

CC by Nefci/Flickr

Biometric Storyboards• A single User

Experience (UX) graph • Intended UX • Actual UX

• Comparison study (N=24) of different player testing approaches

Pejman Mirza-Babaei, Lennart E. Nacke, John Gregory, Nick Collins, and Geraldine Fitzpatrick. 2013. How does it play better?: exploring user testing and biometric storyboards in games user research. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 1499-1508. DOI=10.1145/2470654.2466200

Repidly• Online tool

• Rapid evaluation

• Annotate videos and physiological signals

Bachelor’s Thesis project from Mike Schaekermann

Repidly• Easy to associate

game events with physiological events

• Rapid annotation

Bachelor’s Thesis project from Mike Schaekermann

Repidly• Level of detail

changes

• Aggregation done smoothly

Bachelor’s Thesis project from Mike Schaekermann

Player Data Visualization

• Mixed visualization of qualitative and quantitative player data overlaid on top of a level

• The overall hue of the paths conveys areas with high (B and C) and low (A) player arousal

Pejman Mirza-Babaei, Günter Wallner, Graham McAllister, and Lennart E. Nacke. 2014. Unified visualization of quantitative and qualitative playtesting data. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 1363-1368. DOI=10.1145/2559206.2581224

Takeaways

• Physiological data is complex

• Explore visual options for rapid insights

• Online tools allow collaboration of GURs

CC by David D/Flickr

Thank youContact me

lennart.nacke@acm.org

@acagamic (Twitter)

www.hcigames.com

uwaterloo.ca/games-institute/

www.immerse-network.com