BUILDING

A Game Design& Brain ScienceCollaboration

Eben Myers Santosh MathanSerious Play ConferenceJuly 22, 2015

2 

Eben MyersVP, DesignSimcoach Games

• BA in Studio Art from Swarthmore College

• Masters in Entertainment Technology from CMU

• EPO for NASA• 10+ years designing games

Introductions

Santosh MathanPrincipal ScientistHoneywell

• Research at the intersection of Neurotechnology and HCI

• Interest in enhancing training, performance, and safety in aviation

• PhD in HCI from CMU• Prior roles at NASA Ames and

Microsoft

FAST: Flexible, Adaptive, Synergistic

TrainingHarvard Site Visit

4 

IARPA’s SHARP Program

5 

• EF moderate complex tasks (Elliott, 2003) :

- Solving novel problems

- Modifying behaviors in light of new information

- Generating strategies

- Sequencing complex actions

• ARP crucially tied to EF- overlap in core computational properties (Diamond, 2013)

- Neural correlates (e.g. DLPFC) (Duncan et al., 2000; Barbey et al, 2012)

Executive Functions and ARP

6 

• Updating: updating and monitoring working memory representations• Switching: shifting between tasks or mental sets • Inhibition: suppression of stimuli and responses that are unrelated to the task

Executive Functions: Component Processes

Diamond, 2013

Miyake, 2000

7 

• Training and practice have robust and predictable impact

- Skills improve – following power law function

- ARP can be strengthened through training (Carlson, 1990; Mackey, 2011)

• Limited generalization:

- Acquired skill highly specific

- Limited evidence for near or far transfer

• By nature ARP must be domain general and widely applicable

Enhancing Executive Functions: Promise, Pitfalls

8 

• Train component processes as cognitive skill

• Combine processes, in novel ways, with range of operations (logical relational), and cognitive domains (symbolic, visual)

• Create rich, but systematically mapped training space

• Promote flexible encoding by changing instructions every few trials

- Rapidly encode instructions (Salthouse & Pink, 2008)

- Flexibly configure cognitive resources (Duncan et al., 2012)

- Strong activation of regions implicated with ARP (Dumontheil et al., 2011)

FAST Training: Implications

9 

• Train component processes as cognitive skill

• Combine processes, in novel ways, with range of operations (logical relational), and cognitive domains (symbolic, visual)

• Create rich, but systematically mapped training space

• Promote flexible encoding by changing instructions every few trials

- Rapidly encode instructions (Salthouse & Pink, 2008)

- Flexibly configure cognitive resources (Duncan et al., 2012)

- Strong activation of regions implicated with ARP (Dumontheil et al., 2011)

Intervention 1 - FAST Training: Implications

10 

Collaborators

11 

FAST - Intervention

12 

FAST: Level 1

13 

FAST: Level 2

14 

FAST: Level 3

15 

FAST: Level 4A

16 

FAST: Level 4B

17 

• Engaging isomorphs of FAST

• Everyday processing demands

• Immediate feedback

• Goal-directed practice

• Self-motivating demands

Gamification

18 

• Augment activity of cortical substrate supporting EF using tES

• Distributed fronto-parietal network: DLPFC, ACC, LPC

- DLPFC: connectivity with sensory motor regions, maintaining rules for action, and guiding response selection and inhibition

- ACC: detect processing conflict, engaging DLPFC

- LPC : Stimuli salience, stimulus-response pairings

• tES to act synergistically with training to boost core ARP processes

tES

19 

• Augment activity of cortical substrate supporting EF using tES

• Distributed fronto-parietal network: DLPFC, ACC, LPC

- DLPFC: connectivity with sensory motor regions, maintaining rules for action, and guiding response selection and inhibition

- ACC: detect processing conflict, engaging DLPFC

- LPC : Stimuli salience, stimulus-response pairings

• tES to act synergistically with training to boost core ARP processes

Intervention 2: tES

20 

• Learning is the development of new neural connections

• Connections are formed through synchronized recruitment

• Anodal stim has depolarizing effect of resting potential of neurons – making it easier for neural cells to fire

• Synergistic interaction with training to boost efficacy of core ARP processes

Causal Mechanism

21 

Pilot Study

1

• Assess game implementation of FAST• Test procedures

Pilot Study 2

• Test expanded FAST game• Assess stimulation types

Pre T&E

Evaluation

• Demonstrate efficacy of intervention (FAST + stimulation)

Empirical Validation – Stages and Objectives

22 

1A Demographics

23 

Pre T&E Fluid Intelligence Results

Gf MeasuretRNS+FAST

b p-value Cohen’s d

BOMAT 1.417 .051 .424

Sandia 1.400 .275 .164

Raven 0.473 .448 .136

24 

• https://www.youtube.com/watch?v=qdhKR0g3bhc

Video Demo

Process:What Worked & What Could Have

Been Better

26 

• Iterative design and development process• Communication• Personnel

What Worked

27 

• Iterative design - from research to 3 concepts to one

Iterative design & development process

28 

• Regular, frequent releases of game

Iterative design & development process

Stage Build # Date

Project Kick-Off 1/29/14

High Level Design Complete 2/18/14

First Playable Proof-of-Concept 1 3/3/14

29 

Proof-of-Concept

30 

• Regular, frequent releases of game

Iterative design & development process

Stage Build # Date

Project Kick-Off 1/29/14

High Level Design Complete 2/18/14

First Playable Proof-of-Concept 1 3/3/14

On-Going Weekly Builds4 Major Releases (for Experimental Data Collection)

1A-1 9 4/18/14

1A-2 20 6/25/14

1A-3 25 9/23/14

1B 34 3/9/15

31 

Mature Game

32 

Communication

33 

• 2 weekly meetings

Communication

34 

• Structuring of the abstract games

Communication

35 

• Structuring of the abstract games

Communication

36 

• Structuring of the abstract games

Communication

37 

• Structuring of the abstract games

Communication

38 

• Structuring of the abstract games

Communication

39 

• GitHub releases and integrated issue tracking

Communication

40 

• Attitude:

- Respectful

- Collaborative

- Honest and critical

Personnel

41 

• Inclusion of research engineer on Simcoach dev team

Personnel

42 

• Building the airplane while flying

What Could Have Been Better

43 

• Building the airplane while flying• Building the experimental rocket while launching into space…

What Could Have Been Better

44 

• More focus on QA

What Could Have Been Better

45 

• More time to respond to feature requests

What Could Have Been Better

46 

• Iterative design and development process• Communication• Personnel• Planning

Summary of Takeaways

47 

Questions?

@santoshmathan@honeywellnow

@ebenmyers@simcoachgames