one of the learned stimuli

one of the learned stimuli

newstimulus

Activity of a single neuron in the cortex

Hebbian plasticity

“When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased”

Donald Hebb, 1949

“Neurons that fire together wire together”

A cortical network

Hebbian plasticity

A cortical network

Network can sustain activity even in the absence of input

Specificity of sustained activity

A model of associative memory

‘Biological’ memories

• Associative: recall is based on content rather than on the address

• A transient cue induces a sustained recall

• Robust to minor failures of the hardware

• Distributed

The mathematical model

I will use a slightly different model than the one presented in the last 10 minutes of Wednesday’s class


1 sgn

12

i

i

h tS t

Neurons are binary:

The activity of neuron i, Si = 0,1

at time t+1 input to neuron i at time t


i ij jj

h J S

1

2

5

3

4

J51

J21

J32

A memory pattern is a vector of desired neural activities

For example:

1,0,0,1,1p


1

2

5

3

4

1 111 0.5 0.5 n n

ij ij i jJ n J n p pN

The Hopfield model

trial n +1

1 111 0.5 0.5 n n


The Hopfield model

“Neurons that fire together wire together”

1 111 0.5 0.5 n n


• local learning rule

• incremental, on-line

The Hopfield model

1

2

5

3

4

J51

J21

J32

0iiJ

The Hopfield model

Questions:

1) What are the fixed points of the dynamics?

2) What is their relation with the memory pattern?

Network connections are symmetrical. It can be shown that with asynchronous updating, the dynamics necessarily converge to a fixed point.

Hopfield.m

The Hopfield model

Memory patterns:If Activities of neurons within and between patterns are independently chosen by tossing an unbiased coin then in the limit of large number o neurons, N the network can store ~N memory patterns

The Perceptron

( )K tD

0 T

Affe

ren

ts

Vthr

V rest

tmax 0 tD

What does a neuron do?

spike

no spike

Affe

ren

ts

0 Ttmax-

VthrNull

We consider a simplified case: input is synchronous

Affe

ren

ts

Alternatively, input is constant

The perceptron

j jj

h W X 11 sgn

2 Y h

1X 2X 3X 4X

Y1W 4W

11 sgn

2

Y W X

Geometrical interpretation

1 1 2 2

W X W X W X1X 2X

Y

1W 2W

W

1W

2WX

1X

2X


W

1W

2WX

1X

2X

1 1 2 2

cos cos sin sin

cos cos sin sin

cos

W X W X

W X W X

W X

W X W X W X

W X W X

W X

W X


11 sgn

21

sgn cos2

Y W X

1X 2X

Y

1W 2W

W

1W

2W

X

1X

2X

The perceptron

The Perceptron categorizes the space of inputs into inputs that should evoke a response and inputs that should not evoke a response

Constraints on possible categorizations

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

1X

2X


1X

2X

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X


1X

2X

22 1 0 X X

one of the learned stimuli

Documents

Transcript of one of the learned stimuli