Learning crossmodal spatial transformations

through STDP

Gerhard Neumann

Seminar B, SS 06

Overview Network Model Hebbian Learning and STDP Properties of STDP Cortical Maps Learning Spatial Transformations Papers:

[Song99] : S. Song, L. Abbott, Competitive Hebbian Learning through Spike-Timing-Dependent Plasticity

[Song00] : S. Song, L. Abbott, Cortical Development and Remapping through Spike Timing-Dependent Plasticity

[Davison06]: A. Davison and Y.Fregnac, „Learning Crossmodal spatial transformations through spike-timing-dependent plasticity.

Leaky Integrate + Fire Model

Membrane Potential Vj of neuron j:

Input consists of: Background noise Excitatory Input (added): Inhibatory Input (substracted):

Neuron Model Excitatory Input:

Incremented by following each spike Inhibatory Input:

incremented by by every spike

Simplified Version: Direct change in synaptic current More Complex Version

Conductance Based IF Neurons used by Abbott Basically the same results

Hebbian Learning Donald Hebb:

When an axon of cell A is near enough to excite cell B or repeatedly or consistently takes part in firing it, some growth or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.

Correlation based learning

Not a stable rule: Weight normalization needed No Competition

Usually we need a „global competition signal“ Not biologically realistic

Only for feed forward networks: No recurrent connections possible

)()()( tOtOtW jiij

Spike-timing-dependent plasticity

Synaptic plasticity is sensitive to the temporal oder of the presynaptic and postsynaptic spike

Long Term Potentiation (LTP) If pre synaptic spike before post synaptic spike Correlated Input

Long Term Depression (LTD) If post synaptic spike before pre synaptic spike Random Input

Experiments with culture of rat Hippocampal cells

STDP: Time Window Time Window: ~ 20 ms

Hard Bounds or Soft Bounds Model

w… either models the conductance for the synaptic input or directly the change in the synaptic current

Area of Depression must be larger

than area of potentiation Typical Values for:

: 20 ms : 20 ms – 100 ms

STDP: Other Models Other Models:

Symmetric STDP:

Short Time Intervalls: LTP Long Time Intervalls: LTD

Inverse STDP Reversed LTP/LTD

Also recurrent loops are possible Surpresses recurrent loops leading to a stable network

Mean Input to a Neuron should only be sufficient to charge the membrane to a point below or only slightly above the treshold Postsynaptic Neuron fires primarily in response to statistical fluctuations

in the input

Basic Experiments with STDP

For a single Post Synaptic Neuron STDP tends to segragate synaptic weights into

strong and weak groups (~ 50 %). Competitive Nature of STDP

Basic Experiments with STDP

Effects of Different Correlation Times - Dots… - Triangles…

also works for larger Correlation times

Network Model: 1-D stimulus

E.g. location of a touch stimulus Encoded in 1000 Input Neurons

Grid over the input stimulus Firing rate: Gaussian Firing Curve

Maximum at the prefered stimulus location of the cell Population Coding

Use periodic boundary conditions 200 network neurons

Sparse connectivity to the input neurons (20 %, random) Learning procedure

Input: Brief presentations of the stimulus at a random input location

Lasts ~ 20 ms (exponential distribution)

Experiments:

Without any recurrent connections: Each Neuron develops a random input selectivity => Nearby input neurons are correlated

Strengthing of synapses in one group of corr. inputs Surpresses other input (competitive)

Experiments

Add all-to-all recurrent excitatory connections to the output neurons Initiliaze weights with zero Selectivity and Column Structure, all neurons are selective in the

same neighborhood Recurrent Connections are quite weak after convergence

Experiments Seeding the network

Network neurons 81 – 120 were given initial input weights for stimulus locations from 401 to 600

Recurrent synapses are strengthened before ff synapses Only one strongly correlated group of network neurons,

many correlated groups of input neurons Seeded network neurons begin to drive unseeded network Synapse from input neurons 401-600 to unseeded network

become strong Recurrent synapses are weakened again

FF synapses compete with recurrent synapses because of shorter latency.

Seeded network units can be seen as sort of teacher signal

Experiments

Seeding the network

Cortical Map

Until now: Competitive nature of STDP leads to a winner

take it all situation Single column structure forms

We want: A Continuous Map Restrict the spread of selectivity from one neuron

to another Limit the recurrent excitatory connections

between network neurons to local neighborhoods Add an initial seed to the FF connections

Experiment: Refinement of Maps

Seeded Cortical Map Initialize FF connections with a coarse map Map is tightened and refined.

Experiment: Unseeded Case

Without initial seeding, a single column structure forms. Map can also arise from random initial conditions if

inhibitatory connections are introduced All to all uniform connections of fixed strength between network

neurons Different neurons in the network tend to develop different location

preferences Local excitatory connections favor similar preferences => formation of a smoothly changing cortical map

Experiment: Unseeded Case

A Map like structure is formed The Map can be arranged in either direction at any point in

the network (random initial conditions)

Learning Spatial Transformations

Learn the transformation of a 1 or 2 DoF simulated arm from proprioceptive input to the visual location of the hand (end effector)

Network structure: Three populations, each consisting of a one (or 2)

dimensional array of cells Input Population: Proprioceptive Stimulus Training Population: Visual Stimulus (Location) Output Population: Prediction

Connectivity: Input -> Output: All to All, learned with STDP Training -> Output: topographical and local, fixed

Learning Procedure A random location is choosen,

the training reference calculated Use for the input population,

for the training population E.g. Again use Gaussian Firing

Curve

Experiment:

1 DoF arm: Evolution of weights

S-Shaped Band becomes visible in the matrix of synaptic weights

Prediction:

Training Layer removed, sweep through the input space Weight pattern from the input to the output layer remains stable

Good approximation found, extreme values are underestimated

Experiment: Other Functions

Linear Functions Non-Linear Functions

Experiment: 2 DoF

Also for the 2-D case, longer learning time

Influence of the population tuning curve shape

Poor accuracy of trigonometric functions near the extremes Multiple input values

giving a single output value

Input values compete with one another

=> Make width of the gaussian population tuning curve smaller

Influence of spatio temporal learning Drawbacks of the learning procedure:

Assumes that all point in space is visited with equal frequency Tested with normal distribution Still works, slower convergence

Requires dwell times (time the input stays in a particular location) Has to be approximately the same as the STDP time

constants Ignores travel time between input locations, input location

changed smoothly Works if motion is fast enough

Inputs must fire together to produce potentiation, but then quickly stop firing to avoid depression

Influence of the correlation time

Increase or use symmetric STDP

Influence of signal latency

Latency difference between the input and the training signal

For negative latencies (training before input), the negative weight pattern can emerge

Influence of Training Pathways

Precise range and strength of the connections to the training population is not critical

Plasticity also for the training connections Pattern is stable for prespecified initial connections Similar to Map refinement from Song

Learning the training pathways Start with random training weights

Recurrent (local) excitatory and inhibitory connections are needed Similar to Map Learning with random initial conditions from Song Connections develop with an arbitrary phase

Tends to wander as training continues

Experiments

Influence of plastic rules: Symmetric: More robust to temporal structure Asymmetric: Less dependent on the population

timing curve Soft weight Boundaries: Modulation of the weight

pattern by correlation is weaker, but still learnt Influence of network size and connectivity

Behavior of the network is the same by adjusting other parameters

Summary

STDP: Biologically Inspired Correlation Based learning Competitive rule which strengthens connections to

correlated input, weakens connections to random input Population coding + STDP:

Can be used to calculate cortical mappings Can also learn (simple) spatial transformations Quite complex model

Learning can also be done by a simple offline-linear regression

Sensitive to a lot of parameters