Lecture 3:Learning and Memory

Prof.dr. Jaap Murre

University of Maastricht

University of Amsterdam

jaap@murre.com

http://neuromod.uva.nl

Overview• We will study Hebbian learning and the formation of

categories• We will do some basic memory experiments• Examine various forms of memory• We will try to locate memory in the brain and relate

brain lesions to amnesia• We will also briefly explore executive functions in

the frontal lobes• We will look at memory improvement

With Hebbian learning, two learning methods are possible

• With unsupervised learning there is no teacher: the network tries to discern regularities in the input patterns

• With supervised learning an input is associated with an output– If the input and output are the same, we speak of

auto-associative learning– If they are different it is called hetero-associative

learning

Supervised learning with Hopfield (1982) network

• Bipolar activations – -1 or 1

• Symmetric weights (no self weights) – wij= wji

• Asynchronous update rule– Select one neuron randomly and update it

• Simple threshold rule for updating

Energy of a Hopfield network

Energy E = - ½ i,jwjiaiaj

E = - ½ i(wjiai + wijai)aj = - iwjiai aj

Net input to node j is iwjiai = netj

Thus, we can write E = - netj aj

The energy minimization question can also be turned around

• Given ai and aj, how should we set the weight wji

= wji so that the energy is minimized?

E = - ½ wjiaiaj, so that

– when aiaj = 1, wji must be positive

– when aiaj = -1, wji must be negative

• For example, wji= aiaj, where is a learning

constant

Hebb and Hopfield

• When used with Hopfield type activation rules, the Hebb learning rule places patterns at attractors

• If a network has n nodes, 0.15n random patterns can be reliably stored by such a system

• For complete retrieval it is typically necessary to present the network with over 90% of the original pattern

We will look at an example of competitive learning

• Competitive learning is a form of unsupervised learning

Example of competitive learning:Stimulus ‘at’ is presented

a t o

1 2

Example of competitive learning:Competition starts at category level

a t o

1 2

Example of competitive learning:Competition resolves

a t o

1 2

Example of competitive learning:Hebbian learning takes place

a t o

1 2

Category node 2 now represents ‘at’

Presenting ‘to’ leads to activation of category node 1

a t o

1 2

Presenting ‘to’ leads to activation of category node 1

a t o

1 2

Presenting ‘to’ leads to activation of category node 1

a t o

1 2

Presenting ‘to’ leads to activation of category node 1

a t o

1 2

Category 1 is established through Hebbian learning as well

a t o

1 2

Category node 1 now represents ‘to’

Before we continue...

• Everybody on the right of the classroom, please, close their eyes until the following words have been presented

• The others, pay attention to the following 10 words. You will be asked to remember them later

• Don’t write them down!

table

car

tree

computer

monkey

paper

scissors

tennis

dessert

bread

Now for the other half...

• Everybody on the left of the classroom, please, close their eyes until the following words have been presented

• The others, pay attention to the following 10 words. You will be asked to remember them later

• Don’t write them down!

table

car

tree

computer

monkey

paper

assassin

tennis

dessert

bread

Memory and attention are strongly intertwined

• Paying attention can be seen as holding in memory

• Attention is required for rehearsal

• The longer an item is attended (held in memory), the higher the chance it will be remembered later

Desimone’s study of V4* neurons

* V4 is visual cortex before inferotemporal cortex (IT)

Brown-Peterson task

• Try to remember three letters, e.g., XJC

• When given a number (e.g., 307), start counting backward in threes (307, 304, 301, 298, …)

• When the Write! text appears, write down the letters you remember

• This has to be done at least several times to obtain the effect

Ready!

RGP

875

Write!

Ready!

ZQN

317

Write!

Ready!

HWB

504

Write!

Typical results of the Brown-Peterson task

• The results typically show very low memory performance

• The reason is that rehearsal of the letters is prevented by the counting task

Before we continue,

• Write down all the words you remember from the presentation

• Make sure you do not verbalize them at this moment

• We will verify the result in a minute, but first we have the following two puzzles

Fragment completion

• Try to complete the following English word fragments

• You have 30 seconds

• Each dot (.) stands for a letter

• Don’t verbalize! (So, we can obtain a better sample)

s..ss..s

.ss.ss..

Stop!

The correct answers were

• scissors

• assassin

The presented words were:

• Left half– table– car– tree– computer– monkey– paper– scissors– tennis– dessert– bread

• Right half– table– car– tree– computer– monkey– paper– assassin– tennis– dessert– bread

Short-term memory and working memory

Typical memory recall curve

Atkinson and Shiffrin model (1968)

This model had some limitations

• Ba, ta, pa, ta, pa, ba is much more difficult to remember than ba, bu, bi, bu, bi, ba

• Hence, there are phonological effects in short-term memory

Working memory model by Alan Baddeley and Graham Hitch (1974)

Executive functions

• What controls the memory retrieval process?

• How does the control process work?

• What determines which areas of brain are ‘allowed’ to active in the first place?

The elusive frontal cortex

Anatomy of prefrontal cortex

• Strong lateral connectivity via stellates in Layer IV

• No direct connections to motor outputs

• Certain cells fire strongly and selectively during the delay period of a task in relation to certain aspects of the taks (e.g., position), especially in area 46 surrounding the principal sulcus

Goldman-Rakic studies of Piaget’s AB Paradigm

• Infants persist in reaching for a target even if they have observed it being hidden in another place and older infants will do this if the delay is large enough (2-5 s at 7.5-9 months)

• Still older infants will not do this

• Monkeys with dorsolateral prefrontal lesions show similar behavior (delays > 2 s)

Similarities with ‘prefrontal’ patients

• Prefrontal patients show perseveration on the Wisconsin card sorting test

• There is evidence that also in adult humans such behavior is mainly caused by lesions to the dorsolateral prefrontal cortex

There is also a working memory aspect to the task

• The subject must keep in mind where the hiding place was, which may involve a kind of ‘working memory’ lasting several seconds

• In other experiments Patricia Goldman-Rakic has implicated area 46 as performing a type of working memory function

• Alan Baddeley is not convinced that this type of working memory is similar to his own concept

Long-term memory and amnesia

Larry Squire’s taxonomy of long-term memory

Forgetting

• There is currently no theory that explains why we forget

• Forgetting seems to follow rather strict rules, but even these have not been fully explored

• It is postulated that very well rehearsed knowledge will never be forgotten (Harry Barrick’s ‘permastore’)

10,000 to 100,000 connections per neuron

Anatomy of a neuron

Memory is stored in the connections (synapses) between

neurons

Main storage sites of memories in brain

• Neocortex

• Hippocampus

Personen, dieren, en voorwerpen in de temporaalschors

Ook met breinscans worden dergelijke locaties aangetroffen

Position of the hippocampus

The neocortex

Whale(5 x human)

Human

1000 PC hard disks

• The neocortex contains about 10 billion

• Every neuron connects to 10,000 others

• That amounts to 100,000 billion connections that each can store about 1 byte

• The neocortex thus has the equivalent capacity of at least a 1000 hard disks of 100 giga bytes

TraceLink model

A connectionist model of memory consolidation and amnesia

TraceLink model: structure

System 1: Trace system

• Function: Substrate for bulk storage of memories, ‘association machine’

• Corresponds roughly to neocortex

System 2: Link system

• Function: Initial ‘scaffold’ for episodes

• Corresponds roughly to hippocampus and certain temporal and perhaps frontal areas

System 3: Modulatory system • Function: Control of plasticity• Involves at least parts of the hippocampus,

amygdala, fornix, and certain nuclei in the basal forebrain and in the brain stem

Stages in episodic learning

Retrograde amnesia

• Primary cause: loss of links

• Ribot gradients

• Shrinkage

Semantic dementia

• The term was adopted recently to describe a new form of dementia, notably by Julie Snowden et al. (1989, 1994) and by John Hodges et al. (1992, 1994)

• Semantic dementia is almost a mirror-image of amnesia

Neuropsychology of semantic dementia

• Progressive loss of semantic knowledge

• Word-finding problems

• Comprehension difficulties

• No problems with new learning

• Lesions mainly located in the infero-lateral temporal cortex but (early in the disease) with sparing of the hippocampus

Severe loss of traceconnections

Stage-2 learning proceedsas normal

Stage 3 learning stronglyimpaired

Non-rehearsed memorieswill be lost

No consolidation in semantic dementia

Nadel and Moscovitch (1997): Trace Replication Theory

• They reject the ‘Standard Theory’ of consolidation• Hippocampus always remains involved• Hippocampal representations increase in strength

with time

• For review and assessment, see Meeter, M., & J. M. J. Murre (2004). Consolidation of long-term memory: Evidence and alternatives. Psychological Bulletin, in press.

Sleep and Dreaming

When memory may be consolidated

“We dream in order to forget”

• Or do we?

• Theory by Francis Crick and Graeme Mitchison (1983)

• Main problem: Overloading of memory

• Solution: Reverse learning leads to removal of ‘obsessions’

Dreaming and memory consolidation

• When should this reverse learning take place?

• During REM sleep– Normal input is deactivated– Semi-random activations from the brain stem– REM sleep may have lively hallucinations

Consolidation may also strengthen memory

• This may occur during deep sleep (as opposed to REM sleep)

• Both hypothetical processes may work together to achieve an increase in the clarity of representations in the cortex

Relevant data by Matt Wilson and Bruce McNaughton (1994)

• 120 neurons in rat hippocampus

• PRE: Slow-wave sleep before being in the experimental environment (cage)

• RUN: During experimental environment

• POST: Slow-wave sleep after having been in the experimental environment

Wilson en McNaughton Data

• PRE: Slow-wave sleep before being in the experimental environment (cage)

• RUN: During experimental environment

• POST: Slow-wave sleep after having been in the experimental environment

Experiment by Robert Stickgold

• Difficult visual discrimination problem

• Several hours of practice

• One group goes home• Other group stays in

the lab and skips a night of sleep

Improvement without further training due to sleep

0

5

10

15

20

25

0 2 4 6 8 10

Days after training

Imp

rove

me

nt (

ms)

Normal sleep

Skipped first night sleep

Connectionist implementationof the TraceLink model

With Martijn Meeter

How the simulations work: One simulated ‘day’

• A new memory is learned

• A period of ‘simulated dreaming’ follows– Artificial neurons are activated randomly– This random activity causes ‘recall’ of a

memory– The recalled memory is strengthened in the

neocortex

Frequency of consolidation of patterns over time

0

0.5

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

A simulation with TraceLink

R2 = 0.932

R2 = 0.922

0

0.25

0.5

0.75

1

0 5 10 15

Control

Lesion

Strongly and weakly encoded patterns

• Mixture of weak, middle and strong patterns

• Strong patterns had a higher learning parameter (cf. longer learning time)

0

0.5

1

0 5 10 15

Weak patterns

Middle patterns

Strong patterns

0

0.5

1

0 5 10 15

Weak patterns

Middle patterns

Strong patterns

A ‘Darwinian’ competition?

• Over time, the consolidation process squeezes out the weak patterns

Memory Improvement

Not automatic but possibly effective

Memory Improvement

• Strengthening of existing memory

• Not suitable for anterograde amnesia– Memory book/-electronic agenda– Errorless learning (Baddeley and Wilson)

The two pillars of effective memory

• Elaboration or Making words more memorable

• Rehearsal or Going back to what you are about to forget

Elaboration: Making words more memorable

• Partition (break it up!)

• Link– use anything that comes into your mind

• Imagine– visualize (bizarre)– hear, feel, smell, etc.– verbalize

Elaboration of names

• Pat Galveston• Pat Galve-ston• Pot Gulf Stone• An enormous pot with

a ‘sea’ (gulf) inside and with a massive rock in it

• Al Kane• (no need to

break up)• Eel Cane• An eel

slithering down a cane

Rehearsal: Go back to what you are about to forget

• The more rehearsal, the better the memory retention

• Rehearse at progressively longer intervals: expanding rehearsal

Expanding rehearsal

• Example schedule– immediately after the lecture (lesson, meeting,

experience, …)– the next day– three days later– a week later– a month later– after half a year

A very useful memory trick

• Uses the journey technique• Best used with lists of objects or

names• In your mind, walk along a

familiar route• Mentally, ‘place’ the objects at

locations along the route• Elaborate upon the

locations

Example of the journey technique

• In front of my building

• The revolving entrance door

• The lobby• Waiting for the

elevator• In the elevator

• Buy cat food

• Call Lucy

• Order chair

• E-mail Ted

• Bill John

Practice

Further hints on the journey technique

• Combine different journeys to remember long lists

• Always use the same locations

• This allows reference by number (e.g., 7-th on the list)

A trick to remember numbers

• One is a bun• Two is a shoe• Three is a knee• Four is a door• Five is a hive

• Six is sticks• Seven is heaven• Eight is a gate• Nine is wine• Ten is a hen

This is a peg technique

• Combine with the journey technique to remember long numbers, e.g., 597928641– bee hive in front of the building

– lots of bottles of wine in the door

– lobby has turned into heaven

– wine is presented while waiting for the elevator

– elevator is full of shoes, etc.

Where to go from here

• More sophisticated memory techniques– Remember numbers with the phonetic

mnemonic– Mental filing

• For a detailed description and trainers: www.murre.com