Clustering using Spiking Neural Networks
Biological Neuron:The Elementary Processing Unit of the Brain
Biological Neuron:A Generic Structure
Dendrite SomaSynapse Axon Axon
Terminal
Biological Neuron:Nerve Impulse Transiting
Action Potential(Spike) Postsynaptic
Potential
Membrane Potential
Action Potential(Spike)
Spike-After Potential
Biological Neuron:Soma Firing Behavior
)(tu
)(rest tut
t
t
in1t
in2t
in3t
in4t
outt
a)
b)
c)
Synchrony is the main factor of soma firing
Biological Neuron:Information Coding
Neurons communicate via exact spike timing
Firing rate alone does not carry all the relevant information
Neuroscience Models of Neuron:The Hodgkin-Huxley Model
Alan Lloyd Hodgkin and Andrew Huxley received the Nobel Prize in Physiology and Medicine in 1963
The Hodgkin-Huxley model is too complicated model of neuron to be used in artificial neural networks
Neuroscience Models of Neuron:Leaky Integrate-And-Fire Model
orLeaky Integrate-And-Fire model disregards the refractory capability of neuron
Neuroscience Models of Neuron:Spike-Response Model
Spike-Response model captures the major elements of a biological neuron behavior
Biological Neuron – Computational Intelligence Approach:The First Generation
The first artificial neuron was proposed by W. McCulloch & W. Pitts in 1943
n
iiji
n
iiji
j
xwif
xwify
1
1
,0
,1
Biological Neuron – Computational Intelligence Approach:The Second Generation
Multilayered Perception is a universal approximator
n
iijij xwfy
1
Biological Neuron – Computational Intelligence Approach:Artificial Neurons – Too Artificial?
0
1y
spike occurrence
spike absence
From neurophysiology point of view, y is existence of an output spike
y Number of spikesTime frame
From neurophysiology point of view, y is firing rate
Spike timing is not considered at all!
Biological Neuron – Computational Intelligence Approach:The Third Generation
Spiking neuron model was introduced by J. Hopfield in 1995
Spiking neural networks are - biologically more plausible, - computationally more powerful, - considerably faster than networks of the second generation
Spiking Neural Network:Overall Architecture 1,1RN
1,2RN
1,hRN
2,1RN
2,hRN
2,2RN
nRN ,1
nRN ,2
hnRN
)(1 kx
)(2 kx
)(kxn
s
s
s
1,1,1MS
mhnMS
))((]1[1 kxt
))((]1[2 kxt
))((]1[ kxtm
∫
∫
mSN
∫
2SN
1SN
RN is a receptive neuron
MS is a multiple synapse
SN is a spiking neuron
Spiking neural network is a heterogeneous two-layered feed-forward network with lateral connections in the second hidden layer
Spiking Neural Network:Population Coding
Pool of Receptive Neurons
iliili ckxtkxt ,)(1))(( ]0[]0[max
]0[Input spike:
)(kxi )(sxi
)(sxi
0))((]0[ ,1 kxt i
iRN ,1 iRN ,2 iRN ,3
4))((]0[ ,2 kxt i
1))((]0[ ,1 sxt i
2))((]0[ ,2 sxt i
Spiking Neural Network:Multiple Synapse
))))(((()( ]0[ pli
pjli
pjli dkxttwtu
)(1exp)( tHtt
tpjli
q
p
pjlijli tutu
1
)()(
n
i
h
ljlij tutu
1 1
)()(
Delayed postsynaptic potential:
Spike-response function:
Total postsynaptic potential:
Membrane potential:
1,1RN
hnRN
liRN ∫
s
Σ
qjliw
... ...
1d
2d
1jliw
2jliw
1,1,jMS
jhnMS
jliMS
...
...
...
...jSN
)(1,1, tu j
)( tu jli
))((]0[ kxt nhn
))((]0[ kxt ili
))(( 1]0[
1,1 kxt
)(tu jhn
qd
Spiking Neural Network:Hebbian Learning – WTA and WTM
,~),(
,~),()()()1( w
jjKw
jjtLKKwKw
pjli
pjli
pjlip
jli
))(())(( ]1[]0[ kxtdkxtt jp
ilipjli
0
0
t
)( tL
0
12
)1(2exp1)(
2pjlip
jli
ttL
1ln2
12
Winner-Takes-All:
Winner-Takes-More*:
)()()()()1( ~wpjlijj
pjli
pjli tLtKKwKw
))(())(( ]1[]1[~~ kxtkxtt jjjj
*Proposed for the first time on the 11th International Conference on Science and Technology “System Analysis and Information Technologies” (Kyiv, Ukraine, 2009) by Ye. Bodyanskiy and A. Dolotov
Spiking Neural Network:Image Processing*
Original Image SOM at 50 epoch SNN at 4 epoch
*In Bionics of Intelligence: 2007, 2 (67), pp. 21-26 by Ye. Bodyanskiy and A. Dolotov
Spiking Neuron:The Laplace Transform Basis
Thus, transformation of action potential to postsynaptic potential taken into synapse is nothing other than pulse-position – continuous-time transformation, and soma transformation is just reverse one, continuous-time – pulse-position transformation
From control theory point of view, action potential (spike) is a signal in pulse-position form:
skxtekxttL ))(())((
Spiking Neuron Synapse:A 2nd order critically damped response unit *
*Proposed for the first time on the 6th International Conference “Information Research and Applications” (Varna, Bulgaria, 2009) by Ye. Bodyanskiy, A. Dolotov, and I. Pliss
111
)(21
ss
sG
21
21
1)(~
tt
eet
21 1
21
)(~
t
et
t
)(~)( tet
t
et
t1
)(
21)(
se
sGSynapse
Spiking Neuron:Technically Plausible Description*
Incoming Spike: Time Delay:
Spike-Response Function: Membrane Potential:
Relay: Outgoing Spike:
skxtli
liekxttL ))((]0[ ]0[
))(( sdTimeDelay
p
esG )(
21)(
se
sGSRF
n
i
h
l
q
p
sdkxtpjli
j s
ewsu
pli
1 1 12
))((1
1)(
]0[
2
1))(sign();(
sj
sj
tutu sjj tusLkxttL );())((]1[
*Proposed for the first time on the 6th International Conference “Information Research and Applications” (Varna, Bulgaria, 2009) by Ye. Bodyanskiy, A. Dolotov, and I. Pliss
Spiking Neuron:Analog-Digital Architecture*
1,1RN
1,lRN
1,hRN
iRN ,1
liRN
hiRN
nRN ,1
lnRN
hnRN
sde1 1
jliw
sdpe 2PSP
PSP
1s
e pjliw
sdqe 2PSP
PSP
1s
e qjliw
jliMS
)(1 kx
)(kxi
)(kxn
))((]0[ kxtt ili
1,1,jMS
jhnMS ))((]0[ kxtt nhn
))(( 1]0[1,1 kxtt
s)(suj
)(tuj
t
skxt je ))((]1[
))((]1[ kxtt j
jSN
)(tujli
)(suqjli
)(1 tujli
)(tupjli
)(tuqjli
)(1 sujli
)(supjli )(sujli
t]0[
maxt
skxt ilie ))((]0[
t]0[
maxt
skxte ))(( 1]0[1,1
t]0[
maxt
skxt nhne ))((]0[ )(sujhn
)(1,1, suj
)(tujhn
)(1,1, tuj
s.n. 2PSP
PSP
1s
e
]1[maxt
+
+
+
++
+
sde spike
2SAP 11 sSAPw
-
s.n.
* Proposed for the first time in Image Processing / Ed. Yung-Sheng Chen: In-Teh, Vukovar, Croatia, pp. 357-380 by Ye. Bodyanskiy and A. Dolotov,
Analog-digital spiking neurons corresponds to spike-response model entirely
Fuzzy Receptive Neurons*:
*Proposed for the first time in Information Technologies and Computer Engineering: 2009, 2(15), pp. 51-55 by Ye. Bodyanskiy and A. Dolotov
ix)(kxi
iX ,1 iX ,2 iX ,3 ihX ,1 hiX
r.n.
))((,2 kxii
))((,3 kxii
)( ix ...
Pool of receptive neurons is a linguistic variable, and a receptive neuron within a pool is a linguistic term.
Fuzzy Spiking Neural Network:Fuzzy Probabilistic Clustering*
*Proposed for the first time in Sci. Proc. of Riga Technical University: 2008, 36, P. 27-33 by Ye. Bodyanskiy and A. Dolotov
1,1FRN
1,2FRN
1,hFRN
2,1FRN
2,hFRN
2,2FRN
nFRN ,1
nFRN ,2
hnFRN
)(1 kx
)(2 kx
)(kxn
s
s
s
1,1,1MS
mhnMS
Fuzzy Clustering Layer
))(( kxj
))(( kxm
1SN
2SN
mSN
))((1 kx
mj
j
kxt
kxtkx
1
12
]1[
12
]1[
))((
))(()(
There is no need to calculate cluster centers!
Fuzzy Spiking Neural Network:Fuzzy Possibilistic Clustering*
*Proposed for the first time on the 15th Zittau East-West Fuzzy Colloquium (Zittau, Germany, 2008) by Ye. Bodyanskiy, A. Dolotov, I. Pliss, and Ye. Viktorov
1,1FRN
1,2FRN
1,hFRN
2,1FRN
2,hFRN
2,2FRN
nFRN ,1
nFRN ,2
hnFRN
)(1 kx
)(2 kx
)(kxn
s
s
s
1,1,1MS
mhnMS
Fuzzy Clustering Layer
))(( kxj
))(( kxm
1SN
2SN
mSN
))((1 kx
1
11
2]1[ ))((1)(
j
jj
kxtkx
N
kj
N
kjj
j
kx
kxt
1
1
2]1[
))((
))((
Fuzzy Spiking Neural Network:Image Processing*
*In Proceeding of the 4th International School-Seminar “Theory of Decision Making“ (Uzhhorod, Ukraine, 2008) by Ye. Bodyanskiy, A. Dolotov, and I. Pliss
Original image
Training set
FSNN at 4th epoch
SOM at 40th epoch
Fuzzy Spiking Neural Network:Image Processing*
*In Proceeding of the 11th International Biennial Baltic Electronics Conference "BEC 2008“ (Tallinn/Laulasmaa, Estonia, 2008) by Ye. Bodyanskiy and A. Dolotov
Original image
Training set
FSNN at 3rd epoch
FCM at 29th epoch
Fuzzy Spiking Neural Network:Image Processing*
*In Image Processing / Ed. Yung-Sheng Chen: In-Teh, Vukovar, Croatia, pp. 357-380 by Ye. Bodyanskiy and A. Dolotov
Original image
Training set
FSNN at 1st epoch
FSNN at 3rd epoch
FCM at 3rd epoch
FCM at 30th epoch
Top Related