A COGNITIVE NEUROSCIENCE PERSPECTIVE ON MEMORY FOR PROGRAMMING TASKS

Chris Parnin, Georgia Institute of Technology

In the year 2042, super programmers strapped in SQUIDs (superconducting

quantum interference devices) build programs to save the earth from certain doom.

SQUID SOUND LIKESCIENCE FICTION?

NO!!

These devices exist right now -- just

waiting to be used!

COGNITIVE NEUROSCIENCE

Understanding how the functions of the physical

brain can yield the thoughts and ideas of an intangible

mind.George Miller and Michael Gazzaniga

Benefits?

Image no more keyboards.

Direct measures of function and understanding.

Better predictions and constraints from models.

COGNITIVE NEUROSCIENCE

BackgroundAnatomy,

techniques, findings with focus

on memory

InsightsInsights from

models old and new

ImpactTools, theories, experiments

OVERVIEW

LOBES

Frontal Parietal

OccipitalMedial

Regions divided by deep fissures

LIMBIC SYSTEM

Deep structures within brain.

FUNCTION LOCALIZATION

HUMAN TECHNIQUES

Electro/magno-basedMeasures change in neuron activity

Affects visible within milliseconds.

Location within centimeters.

Examples• EEG• MEG

Blood oxygen-basedMeasures change in blood activity.

Affects visible within seconds.

Location within millimeters.

Examples• fMRI• Optical imaging

ANIMAL TECHNIQUES

Gene TherapyBred mutant rats

DyeDiscover neural pathways

LesionsDamage neural pathways

HabitationThe kitten who only saw vertical lines

Cats, Monkeys, Mice Oh My!

Stimulate neurons with electricity

LONG-TERM POTENTIATION (LTP)

Effect found even after one year!

PLACE CELLS

VIRTUAL NAVIGATION

Participants navigate a virtual town.

Recall paths.

Place cells fire.

NEUROGENESIS

Larger volume by years of experience!

Size of taxi driver hippocampus?

HIPPOCAMPAL ACTIVITY

Forgotten words Remembered words

Forgotten words Familiar words

(Entorhinal cortex)

AUTOMATIC MEMORY

We cannot anticipate important events.We cannot always devote top-down resources to encode and select memory.

Initial memory is largely formed without conscious effort.

HIPPOCAMPUS ANATOMY

Grand Central of the sensesConnections to sensory processing and semantic recognition. Connections to frontal regions.

Brain HashtableHippocampal neurons cannot store directly, index to original source.

Recording factorsNovel, engaging, high-frequency…

SNOWBALL TO BOULDER

Hippocampal LTP

Formed and tagged for storage.

Consolidated memory

Replayed and stored at original regions.

Top-down associations

Properties our bound to memory. “Rehearsal” is activation of speech motor.

INSIGHT

So what?

A review, and new perspectives.

Sensory Short-term Long-termAttention Encoding

Rehearsal

MODAL MODEL OF MEMORY(1968)

Patients with no short-term memory still have long-term memory…

Sensory Short-term Long-term

EpisodicBuffer (2000)

Lifetime of seconds!

“Temporarily infinite”

WORKING MEMORY (1974)Phonologica

l loop

Visio-Spatial Sketch pad

Central Executiv

e

LONG-TERM WORKING MEMORY(1995)

Good insight, theory formed without any neuroscience basis.

MEMORY 2.0

• Distributed• Specialized• Retained and Replayed• Automatic without Rehearsal• Immediate and Long-Lasting

MEMORY TYPES

SpatialProspectiveFamiliarityRecencySource

Semantic…

See paper for more details!

AttentionMaintenanceManipulation

Dynamic filtering

Auto associative

retrieval

Auto associative encoding

Retrieval Encoding

TASK MEMORY MODEL (beta)

Semantic Se

man

tic

Indices

Spatial

Goals

Attention

Bindings

Visual

Frontal

Associative

Perceptual

ProcessRegions

Note, top-down iteration is a “brain design-pattern” found in many neural circuits.

PROGRAMMING ENVIRONMENTS

Often do not associate name with memories of programming.

NAME IS NOT ENOUGH

TOOL DESIGN: MEMORY

• Auto-association– Better distinct cues for learning?

• Spatial memory?– Spatial stability, isolation

• Prospective memory?– Better reminders than a compile error or TODO?

• Forgetting?– Code history and provenance

THEORIES (BETA)

Visual Chunks

Initial formation of memory.“Below the method with

lots of strings”

Iterative Comprehension

Top-down experiences will bind and update partial

representations. Concepts form.

Navigational Jitter

Tabs, scrolling huge files,

little understanding-- need memory aids.

EXPERIMENTS

Gender differences in brain activity during programming?

Neurogenesis in programmer’s brain?

Place cells (spatial) during programming navigation?

Your idea?

MUSCLES OF THE MIND

+ =

Subvocal speech

EMG (electromyograph

y)Cognition

0 100 200 300 400 500 600 700 800 900

Subvocal Debug

Select Click

Search Edit

FormulatingProblem

Found useful code

Formulatingsolution

EditingFound useful code

Evaluating

Moments of interest and difficulty

QUESTIONS?

Indices

Spatial

Goals

Attention

Bindings

Visual

Goal Memoryspace-goal binding.

Lesioned monkeysForget food location.

VariationChange location, keep consistent cue