A supervised learning approach based on STDP and ... -...
Transcript of A supervised learning approach based on STDP and ... -...
A supervised learning approach based on STDPand polychronization in spiking neuron networks
Hélène Paugam-Moisy1, Régis Martinez1 and Samy Bengio2
1LIRIS - CNRS - Université Lumière Lyon 2Lyon, France
http://liris.cnrs.fr
2IDIAP Research InstituteMartigny, Switzerland
http://www.idiap.chSamy is now at Google
ESANN 2007 - April, 27
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 2 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 3 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Motivation
In Spiking Neuron Networks (SNNs), informationprocessing is based on the times of spike emissions.
SNNs are a very powerful new generation of artificial neuralnetworks but efficient learning in SNNs is not straightforward.
A current track is to simulate the synaptic plasticity, as canbe observed by neurobiologists [Bi and Poo,1998] but thismethod lacks supervised control of learning.
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 4 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Theoretical fundations
Theoretically, the use of delays increases the learningcapacity of SNNs...[Maass, 1997] [Schmitt, 1999]
... but delays are rarely used in SNN models
Recent advances in neural networks (ESN [Jaeger, 2001],LSM [Maass et al, 2002]) give interesting results
The concept of polychronization emphasizes theimportance of delays for explaining neural activity[Izhikevich, 2006]
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 5 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 6 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Problematics
A better computational power is a good point, but what about thelearning algorithm ? How to take advantage of the computationalpower of delays ?
We take advantage of polychronous groups activations tomonitor activity in the networkWe define a supervised1 learning mechanism to control thecomputational power of a SNN
Polychronization will help us monitor and understand the networkactivity.
1simplest way for us to show that polychronization can actually be a reliableinformation coding
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 7 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 8 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
The model
Maintains biological plausibility within the internal networkNeuron model : Spike Response Model (SRM0)[Gerstner 1997]Inspired from LSM/ESN architectures :- input layer of spiking neurons- recurrent randomly connected internal network- output layer which supports a supervised learning rule
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 9 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
The model
.
.
.
class 2
class 1
M internal cells
input connections
internal connections
with adaptable delay
K input cellsInternal network
2 output cells
output connections
Input layer (stimulation layer) :10 neuronsInput injection
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 10 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
The model
.
.
.
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
class 2
Internal Network :100 neurons, 80% excitatory, 20% inhibitoryRandom recurrent topologyConnection delays fixed (but randomly chosen) between 1and 20 ms
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 11 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
The model
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
output connectionswith adaptable delay
input connections
internal connections
Output layer :2 neurons : one for each target classrecieves a connection from each internal neuron
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 12 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
The model
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
Tested on a classification task
Two input patterns :Target neuron must fire before non-target neuron
20 msTime [ms]
20 msTime [ms]
Inpu
t neu
rons
Inpu
t neu
rons
Stimulation pattern 1 Stimulation pattern 2
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 13 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 14 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
A two scale learning algorithm
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
1 Unsupervised learning : Spike Time Dependent Plasticity(STDP) within the internal network (ms time scale) [Kempteret al., 1999]
2 Supervised mechanism : delay adaptation on outputconnections (at each input presentation) based on a margincriterion [Vapnik, 95]
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 15 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
1. Unsupervised learning algorithm
Unsupervised learning : Spike Time Dependent Plasticity(STDP) within the internal network (ms time scale)
Temporal hebbian rule, suitable for SNNsAt the synaptic level (local mechanism)Depending on activity going through the synapseCausality based on spike emissions order
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 16 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
2. Supervised learning algorithm
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
After the presentation of a given input pattern p,If target/non-target spikes order is OKANDIf margin between target/non-target spikes > ε
Then : pattern is well classifiedOtherwise,• for target neuron : decrement the delay (−1ms)• for non-target neuron : increment the delay (+1ms)
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 17 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 18 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Simulation protocol
Initial noisy stimulation : noise presented during 300 msLearning phase : alternated presentation of two patternsGeneralization phase : alternated presentation of the twonoisy patterns
NB : One presentation every 100 ms
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 19 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Initialization phase
0
20
40
60
80
100
120
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000
Neu
ron
ID
Time [ms]
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 20 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Learning phase observation
Decreasing internal activity (STDP)Activity pattern different from an input to the otherMargin evolution
0
20
40
60
80
100
120
8500 8600 8700 8800 8900 9000 9100
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 21 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Generalization performance
Error rate with noise 4 : 4%Error rate with noise 8 : 19%Hard to discriminate by human
0
20
40
60
80
100
120
18900 19000 19100 19200 19300 19400 19500 19600
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 22 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 23 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Polychronization [Izhikevich, 2006]
Definition : neuron interactions characterized by spike timesfollowing a precise temporal pattern, depending on delays.
Example :
N2
15ms
8ms
N1
N3Time [ms]
8 ms
15 ms
N3
N2
N1
If N1 emits a spike at t, and N3 at t + 7, then N2 emits a spikeat t + 15.
A set of such interacting neurons is called a polychronousgroup.
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 24 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Scanning for supported polychronous groupsStructurePolychronous groups are supported by the topology.
connections between neuronsdelays of the connections
A given topology = a particular set of supportedpolychronous groupsEach neuron can be involved in several polychronous groups
To find all supported polychronous groups, we use the samealgorithm as [Izhikevich 2006].
Dynamicsset of supported polychronous groups 6= set of activatedpolychronous groups
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 25 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 26 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Polychronous groups activations
0
20
40
60
80
100
60 65 70 75 80 85 90 95 100
Perc
enta
ge o
f ac
tivat
ion
# polychronous groups
activations in response to class 1activations in response to class 2
0
20
40
60
80
100
60 65 70 75 80 85 90 95 100Pe
rcen
tage
of
activ
atio
n
# polychronous groups
activations in response to class 1activations in response to class 2
Figure: Activation ratio from 2000 to 5000 ms, and then from 8000 to11000 ms.
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 27 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Plan
1 Motivations
2 Problematics
3 Network architecture
4 Learning mechanisms
5 Results (1)
6 Polychronization
7 Results (2)
8 Conclusion
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 28 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Conclusion
Algorithm easy to implementThe learning seems to work on a classification taskEasily explained by polychronizationActivity easily monitored with polychronous groupsInternal network is no longer a black-box contrary to ESNand LSM
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 29 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Perspectives
Topology
Dynamics
Polychronousgroups STDP
Complex network analysis :Are polychronous groups the (or a part of the) link betweentopology and dynamicsHow far ?
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 30 / 31
Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion
Thank you for listening.
Questions !
A supervised learning approach based on STDPand polychronization in spiking neuron networks– Hélène Paugam-Moisy, Régis Martinez and Samy Bengio
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 31 / 31
Appendix
Plan9 Appendix
Work in progressReservoir Computing perspectivesGroupes polychrones sur 100 neuronesModèle SRM0Modèle SRM1Forme d’un PPSRéseau expérimentalSensibilité à un motif spécifiqueFenêtre STDP EurichFenêtre STDP classiqueFenêtre STDP MeunierStabilité du classifieurCodage temporelArchitectureActivation des groupes polychronesActivité neuronalePG detectionThe model proposedOriginal problemDifference with synfire chainNetwork activity
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 32 / 31
9 AppendixWork in progressReservoir Computing perspectivesGroupes polychrones sur 100 neuronesModèle SRM0Modèle SRM1Forme d’un PPSRéseau expérimentalSensibilité à un motif spécifiqueFenêtre STDP EurichFenêtre STDP classiqueFenêtre STDP MeunierStabilité du classifieurCodage temporelArchitectureActivation des groupes polychronesActivité neuronalePG detectionThe model proposedOriginal problemDifference with synfire chainNetwork activity
Work in progress
Use larger inputs : encouraging tests with USPS dataset
2 versus 7 : 96% success on train set, 93% on test set3 versus 8 : 89% success on train set, 86% on test set
Switch to more than two classesExtend model with persistant activity
retour
Appendix
Reservoir Computing perspectives : Open questions
Might there be links with reservoir computing. Indeed, sometheoretical properties exists : point-wise separation, universalapproximation, echo state properties...But still difficulties to investigate what’s going on in the reservoir(refering to special session)Polychronous groups can be a reliable way
to analyse dynamics of a spiking neuron reservoirto find optimal topologies (structures)
retour
Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 35 / 31
Groupes polychrones sur 100 neurones
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80
Neu
rone
s
Temps
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80
Neu
rone
s
Temps
[48] 18,24,80 (0,11,11) ==> 37 (16) — [49] 19,31,43 (3,0,5)==> 6 (6)
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80
Neu
rone
s
Temps
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80
Neu
rone
s
Temps
[50] 19,55,76 (0,11,13) ==> 70 (16) — [51] 21,52,76 (7,7,0)==> 11 (12)
retour
Modèle SRM0 (used)
uj(t) = η(t − tfj )︸ ︷︷ ︸
A : refractory periode
+∑
i
wij ε(t − tfi − dij)︸ ︷︷ ︸
B : excitatory potential
retour
Modèle SRM1
uj(t) = η(t − tfj )︸ ︷︷ ︸
A : refractory periode
+∑
i
wij
∑f
ε(t − tfi − dij)︸ ︷︷ ︸
B : excitatory potential
retour
Forme d’un PPS
0
0.2
0.4
0.6
0.8
1
-1 0 1 2 3 4 5
Ups
p [m
V]
Temps [ms]
exp(-x/Tau)
retour
Réseau expérimental
retour
Sensibilité à un motif spécifique
1
2
3
4
5
d = 21
d = 15
d = 8
d = 1
t+6 t+13 t+20 t+21
21
15
8
8
1
temps [ms]
Neuronesdu sac
arrivée au neurone de sortie 1
t
Neurons
1
2
3
4
5
Neurone de sortie 1
d = 8
retour
Fenêtre STDP Eurich
retour
Fenêtre STDP classique
1.0
−1.0
20 mstpost − tpre [ms]
−20 ms
décalage temporel de la synapse (tpost − tpre)
augmentation du poids
Delta W
retour
Fenêtre STDP Meunier
Si ∆W ≤ 0, le poids est augmenté :wij ← wij + α ∗ (wij − wmin) ∗∆W
Si ∆W ≥ 0, le poids est diminué :wij ← wij + α ∗ (wmax − wij) ∗∆W
1.0
POTENTIATION
DEPRESSION −0.5
t
W
10ms 20ms
100ms
1.0
t
W
20ms−20ms
−0.25DEPRESSION DEPRESSION
POTENTIATION
+infini−infini
retour
Stabilité du classifieur
0
5
10
15
20
25
30
0 100000 200000 300000 400000
Stab
ilite
des
repo
nses
Temps
Diagramme de Stabilite des reponses
0
5
10
15
20
25
0 100000 200000 300000 400000St
abili
te d
es re
pons
es
Temps
Diagramme de Stabilite des reponses
retour
Codage temporel
cd
ba a
bc
dtt−1
Neurones
Temps
Codage temporel
Composantes du vecteur
Codage en intensité
Vecteur numérique
Intensité
Vague de spikes dans un intervallede codage temporel
retour
Architecture
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
retour
Activation des groupes polychrones
0
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
100
105
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 15000 16000 17000 18000 19000 20000
Poly
chro
nous
gro
ups
t [ms]
retour
Activité neuronale
retour
PG detection
To find all supported polychronous groups, we use the samealgorithm as [Izhikevitch 2006]. It consists in scanning for spiketime combination of all groups possible of 3 neurons (i.e.combinatorial quiestions), so that the spikes would trigger thefiring of one or more impacted neurons, taking axonal delays intoaccount.
Il est possible de procéder de même en cherchant plus dedéclencheurs, mais la complexité est accrue: O(np), avec pnombre de déclencheurs. retour
The model proposed
.
.
.
class 2
class 1
M internal cells
input connections
internal connections
with adaptable delay
K input cellsInternal network
2 output cells
output connections
Input layer (stimulation layer) :10 neuronsOutgoing connection probability : 0.1Delay to central assembly : 0 ms
retour
The model proposed
.
.
.
class 1
2 output cellsK input cells
Internal network
M internal cells
input connections
internal connections
output connectionswith adaptable delay
class 2
Central assembly :100 neurons, 80% excitators, 20% inhibitorsRandom topologyReccurent connection probability : 0.3Recurrent connections delay from 1 to 20 msSpike Time Dependent Plasticity (STDP)
retour
The model proposed
.
.
.
class 2
class 1
2 output cellsK input cells
Internal network
M internal cells
output connectionswith adaptable delay
input connections
internal connections
Output layer :2 neurons : one for each target classIncoming connection probability : 1 (central assemblycompletely projected)Adaptable delays of input connections (all initialized to 10ms)
retour
Initial work
Originally : problem for learning binary patternsSpike responses : all or nothingSolution : allow diversity in axonal delays
0
100
200
300
400
500
0 20 40
Neu
rone
s
Temps
Diagramme de Pulses
retour
Difference with synfire chainin Synfire Chains and Catastrophic Interference – J. Sougnéand R. French (2001) :
when an initial neuron, A, fired, a second neuron, B,would fire 151ms later, followed by a third neuron, C,that would fire 289ms later with a precision across trialsof 1 ms
in Polychronization : computation with spikes – E. Izhikevich(2006) :
Synfire chains describe pools of neurons firingsynchronously, not polychronously. Synfire activity relieson synaptic connections having equal delays or nodelays at all. Though easy to implement, networkswithout delays are finite-dimensional and do not haverich dynamics to support persistent polychronousspiking.
retour
Initialization phase
0
20
40
60
80
100
120
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000
Neu
ron
ID
Time [ms]
retour
Learning phase observation
Decreasing internal activity (STDP)Activaty pattern different from an input to the otherMargin evolution
0
20
40
60
80
100
120
2300 2400 2500 2600 2700 2800 2900 3000 3100 3200 8500 8600 8700 8800 8900 9000 9100
retour
Generalization performance
Error rate with noise 4 : 4%Error rate with noise 8 : 19%
0
20
40
60
80
100
120
18500 18600 18700 18800 18900 19000 19100 19200 18900 19000 19100 19200 19300 19400 19500 19600
retour