Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals...
-
Upload
austin-watson -
Category
Documents
-
view
215 -
download
0
Transcript of Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals...
![Page 1: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/1.jpg)
Biological Modeling of Neural Networks
Week 8 – Noisy input models:
Barrage of spike arrivals
Wulfram GerstnerEPFL, Lausanne, Switzerland
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold
8.4 Escape noise - stochastic intensity8.5 Renewal models8.6 Comparison of noise models
Week 8 – part 1 :Variation of membrane potential
![Page 2: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/2.jpg)
Crochet et al., 2011
awake mouse, cortex, freely whisking,
Spontaneous activity in vivo
Neuronal Dynamics – 8.1 Review: Variability in vivo Variability
- of membrane potential? - of spike timing?
![Page 3: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/3.jpg)
Neuronal Dynamics – 8.1 Review. Variability
Fluctuations-of membrane potential-of spike times fluctuations=noise?
model of fluctuations?
relevance for coding?
source of fluctuations?
In vivo data looks ‘noisy’
In vitro data fluctuations
![Page 4: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/4.jpg)
- Intrinsic noise (ion channels)
Na+
K+
-Finite number of channels-Finite temperature
-Network noise (background activity)
-Spike arrival from other neurons-Beyond control of experimentalist
Check intrinisic noise by removing the network
Neuronal Dynamics – 8.1. Review Sources of Variability
![Page 5: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/5.jpg)
- Intrinsic noise (ion channels)
Na+
K+
-Network noise
Neuronal Dynamics – 8.1. Review: Sources of Variability
small contribution!
big contribution!
In vivo data looks ‘noisy’
In vitro data small fluctuations nearly deterministic
![Page 6: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/6.jpg)
Neuronal Dynamics – 8.1 Review: Calculating the mean
)()( k
fk
fk
syn ttwtRI
)'()'(')( 1 k
fk
fkR
syn ttttdtwtI
k
fk
fkR
syn ttttdtwtII )'()'(')( 10
mean: assume Poisson process
k
kkRttdtwI )'('1
0
t
( ) ' ( ') ( ' )fkf
x t dt f t t t t
( ) ' ( ') ( ' )fkf
x t dt f t t t t
rate of inhomogeneousPoisson process
use for exercises
( ) ' ( ') ( ')x t dt f t t t
![Page 7: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/7.jpg)
Neuronal Dynamics – 8.1. Fluctuation of potential
for a passive membrane, predict -mean -varianceof membrane potential fluctuations
Passive membrane=Leaky integrate-and-fire without threshold
)()( tIRuuudt
d synrest
Passive membrane
![Page 8: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/8.jpg)
0RI
)()( k
fk
fk
syn ttwtRI
EPSC
Synaptic current pulses of shape a
)(tIR
)()( tIRuuudt
d synrest
Passive membrane
)()( 0 tIItI fluctsyn
Neuronal Dynamics – 8.1. Fluctuation of current/potential
Blackboard,
Math detour:
White noise
0( ) ( ) ( )synRI t RI t t
2
( ) 0
( ) ( ') ( ')
t
t t a t t
I(t) Fluctuating input current
0I
![Page 9: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/9.jpg)
Neuronal Dynamics – 8.1 Calculating autocorrelations
( ) ' ( ') ( ) '
( ) ( ) ( )
x t dt f t t I t
x t ds f s I t s
( ) ( ) ( )x t ds f s I t s 0( ) ( ) [ ( ) ( ) ]x t ds f s I t s t s
Autocorrelation
( ) ( ')x t x t
ˆ ˆ( ) ( ) ' " ( ') ( ") ( ') ( ")x t x t dt dt f t t f t t I t I t
Mean:Blackboard,
Math detour
I(t)
Fluctuating input current
0 ( )I t
0( ) ( ) ( )I t I t t
0( ) ( ) ( )x t ds f s I t s
![Page 10: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/10.jpg)
White noise: Exercise 1.1-1.2 now
2)()()()()( tututututu
)(tu)(tu
Expected voltage at time t ( ) ?u t
Input starts here
Assumption:far away from theshold
Variance of voltage at time t
Next lecture: 9:56
![Page 11: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/11.jpg)
Diffusive noise (stochastic spike arrival)
2)()()()()( tututututu
)(tu
0( ) ( )u t u t
)()()( ttIRuuudt
drest
Math argument
( ') ( ) ( ) ( ') ( ) ( ')u t u t u t u t u t u t
2 2[ ( )] [1 exp( 2 / )]uu t t
![Page 12: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/12.jpg)
Neuronal Dynamics – 8.1 Calculating autocorrelations
t
( ) ' ( ') ( ' )
' ( ') ( ')
fk
f
x t dt f t t t t
dt f t t S t
( ) ' ( ') ( ')x t dt f t t S t
rate of inhomogeneousPoisson process
( ) ( ) ( )x t ds f s t s
Autocorrelation
( ) ( ')x t x t
ˆ ˆ( ) ( ) ' " ( ') ( ") ( ') ( ")x t x t dt dt f t t f t t S t S t
Mean:
Blackboard,
Math detour
![Page 13: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/13.jpg)
Stochastic spike arrival: excitation, total rate
)()( tSRuuudt
drest
u0u
Synaptic current pulses
Exercise 2.1-2.3 now: Diffusive noise (stochastic spike arrival)
1. Assume that for t>0 spikes arrive stochastically with rate - Calculate mean voltage
2. Assume autocorrelation - Calculate
( ) ( )fef
S t q t t
2( ) ( ') ( ')S t S t t t
?)()( tutu
)(tS
Next lecture: 9h58
![Page 14: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/14.jpg)
Assumption: stochastic spiking rate
Poisson spike arrival: Mean and autocorrelation of filtered signal
( ) ( )f
f
S t t t ( ) ( ) ( )x t F s S t s ds
mean( )t
( )F s
( ) ( ) ( )x t F s S t s ds ( ) ( ) ( )x t F s t s ds
( ) ( ') ( ) ( ) ( ') ( ' ') 'x t x t F s S t s ds F s S t s ds ( ) ( ') ( ) ( ') ( ) ( ' ') 'x t x t F s F s S t s S t s dsds
Autocorrelation of output
Autocorrelation of input
Filter
![Page 15: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/15.jpg)
Biological Modeling of Neural Networks
Week 8 – Variability and Noise:
Autocorrelation
Wulfram GerstnerEPFL, Lausanne, Switzerland
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold8.4 Escape noise -renewal model
Week 8 – part 2 : Autocorrelation of Poisson Process
![Page 16: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/16.jpg)
Stochastic spike arrival:
Justify autocorrelation of spike input:Poisson process in discrete time
In each small time step Prob. Of firing
Firing independent between one time step and the next
tp t
Blackboard
![Page 17: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/17.jpg)
Stochastic spike arrival: excitation, total rate
Exercise 2 now: Poisson process in discrete time
Show that autocorrelation for
2)'()'()( tttStS
TTN )(
In each small time step Prob. Of firing
Firing independent between one time step and the next
Show that in an a long interval of duration T, the expected number of spikes is
tp t
0t
Next lecture: 10:46
![Page 18: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/18.jpg)
0FP v t Probability of spike in time step:
Neuronal Dynamics – 8.2. Autocorrelation of Poisson
t
spike train
math detour now!
Probability of spike in step n AND step k
20 0( ) ( ') ( ') [ ]S t S t v t t v
Autocorrelation (continuous time)
![Page 19: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/19.jpg)
Quiz – 8.1. Autocorrelation of Poisson
t
spike train
( ) ( ')S t S t
The Autocorrelation (continuous time)
Has units
[ ] probability (unit-free)[ ] probability squared (unit-free)[ ] rate (1 over time)[ ] (1 over time)-squred
![Page 20: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/20.jpg)
Biological Modeling of Neural Networks
Week 8 – Variability and Noise:
Autocorrelation
Wulfram GerstnerEPFL, Lausanne, Switzerland
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold8.4 Escape noise -renewal model
Week 8 – part 3 : Noisy Integrate-and-fire
![Page 21: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/21.jpg)
Neuronal Dynamics – 8.3 Noisy Integrate-and-firefor a passive membrane, we can analytically predict the mean of membrane potential fluctuations
Passive membrane=Leaky integrate-and-fire without threshold
)()( tIRuuudt
d synrest
Passive membrane
ADD THRESHOLD Leaky Integrate-and-Fire
![Page 22: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/22.jpg)
I0I
)(tI
)()( tIRuuudt
drest
noiseo IItI )(
effective noise current
u(t)
( ) ( ) rIF u t THEN u t u
noisy input/diffusive noise/stochastic spikearrival
LIF
Neuronal Dynamics – 8.3 Noisy Integrate-and-fire
![Page 23: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/23.jpg)
I(t)
fluctuating input current
fluctuating potential
Random spike arrival
Neuronal Dynamics – 8.3 Noisy Integrate-and-fire
![Page 24: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/24.jpg)
stochastic spike arrival in I&F – interspike intervals
I0I
ISI distribution
Neuronal Dynamics – 8.3 Noisy Integrate-and-fire
0( ) ( ) ( )synRI t RI t t
white noise
![Page 25: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/25.jpg)
Superthreshold vs. Subthreshold regime
Neuronal Dynamics – 8.3 Noisy Integrate-and-fire
![Page 26: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/26.jpg)
u(t)
Neuronal Dynamics – 8.3. Stochastic leaky integrate-and-fire
noisy input/ diffusive noise/stochastic spike arrival
subthreshold regime: - firing driven by fluctuations - broad ISI distribution - in vivo like
ISI distribution
![Page 27: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/27.jpg)
Crochet et al., 2011
awake mouse, freely whisking,
Spontaneous activity in vivo
Neuronal Dynamics – review- Variability in vivo
Variability of membrane potential?
Subthreshold regime
![Page 28: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/28.jpg)
fluctuating potential22( ) ( ) [ ( )] ( )u t u t u t u t
Passive membrane( ) ( ' )
' ( ') ( ')k
fk k
k f
kk
u t w t t
w dt t t S t
for a passive membrane, we can analytically predict the amplitude of membrane potential fluctuations
Leaky integrate-and-firein subthreshold regime In vivo like
Stochastic spike arrival:
Neuronal Dynamics – 8.3 Summary:Noisy Integrate-and-fire
![Page 29: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/29.jpg)
Biological Modeling of Neural Networks
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold
Week 8 – Noisy input models: barrage of spike arrivals
THE END
![Page 30: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/30.jpg)
Biological Modeling of Neural Networks
Week 8 – Noisy input models:
Barrage of spike arrivals
Wulfram GerstnerEPFL, Lausanne, Switzerland
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold
8.4 Escape noise - stochastic intensity8.5 Renewal models8.6 Comparison of noise models
Week 8 – part 4 : Escape noise
![Page 31: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/31.jpg)
- Intrinsic noise (ion channels)
Na+
K+
-Finite number of channels-Finite temperature
-Network noise (background activity)
-Spike arrival from other neurons-Beyond control of experimentalist
Neuronal Dynamics – Review: Sources of Variability
small contribution!
big contribution!Noise models?
![Page 32: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/32.jpg)
escape process,stochastic intensity
stochastic spike arrival (diffusive noise)
Noise models
u(t)
t
)(t
))(()( tuftescape rate
)(tRIudt
dui
i
noisy integration
Relation between the two models: later
Now:Escape noise!
t̂ t̂
![Page 33: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/33.jpg)
escape process
u(t)
t
)(t
))(()( tuftescape rate
u
Neuronal Dynamics – 8.4 Escape noise
1 ( )( ) exp( )
u tt
escape rate
( ) ( )rest
du u u RI t
dt
f frif spike at t u t u
Example: leaky integrate-and-fire model
t̂
![Page 34: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/34.jpg)
escape process
u(t)
t
)(t
))(()( tuftescape rate
u
Neuronal Dynamics – 8.4 stochastic intensity
1 ( )( ) exp( )
( )
u tt
t
examples
Escape rate = stochastic intensity of point process
( ) ( ( ))t f u t
t̂
![Page 35: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/35.jpg)
u(t)
t
)(t
))(()( tuftescape rate
u
Neuronal Dynamics – 8.4 mean waiting time
t
I(t)
( ) ( )rest
du u u RI t
dt
mean waiting time, after switch
1ms
1ms
Blackboard,
Math detour
![Page 36: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/36.jpg)
u(t)
t
)(t
Neuronal Dynamics – 8.4 escape noise/stochastic intensity
Escape rate = stochastic intensity of point process
( ) ( ( ))t f u t
t̂
![Page 37: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/37.jpg)
Neuronal Dynamics – Quiz 8.4Escape rate/stochastic intensity in neuron models[ ] The escape rate of a neuron model has units one over time[ ] The stochastic intensity of a point process has units one over time[ ] For large voltages, the escape rate of a neuron model always saturates at some finite value[ ] After a step in the membrane potential, the mean waiting time until a spike is fired is proportional to the escape rate [ ] After a step in the membrane potential, the mean waiting time until a spike is fired is equal to the inverse of the escape rate [ ] The stochastic intensity of a leaky integrate-and-fire model with reset only depends on the external input current but not on the time of the last reset[ ] The stochastic intensity of a leaky integrate-and-fire model with reset depends on the external input current AND on the time of the last reset
![Page 38: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/38.jpg)
Biological Modeling of Neural Networks
Week 8 – Variability and Noise:
Autocorrelation
Wulfram GerstnerEPFL, Lausanne, Switzerland
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold8.4 Escape noise - stochastic intensity8.5 Renewal models
Week 8 – part 5 : Renewal model
![Page 39: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/39.jpg)
Neuronal Dynamics – 8.5. Interspike Intervals
( ) ( )du F u RI t
dt
f frif spike at t u t u
Example: nonlinear integrate-and-fire model
deterministic part of input( ) ( )I t u t
noisy part of input/intrinsic noise escape rate
( ) ( ( )) exp( ( ) )t f u t u t
Example: exponential stochastic intensity
t
![Page 40: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/40.jpg)
escape process
u(t)Survivor function
t̂ t
)(t))(()( tuft
escape rate
)ˆ()()ˆ( ttStttS IIdtd
Neuronal Dynamics – 8.5. Interspike Interval distribution
t
t
![Page 41: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/41.jpg)
escape processA
u(t)
t
t
dtt^
)')'(exp( )ˆ( ttS I
Survivor function
t
)(t
))(()( tuftescape rate
t
t
dttt^
)')'(exp()( )ˆ( ttPI
Interval distribution
Survivor functionescape rate
)ˆ()()ˆ( ttStttS IIdtd
Examples now
u
Neuronal Dynamics – 8.5. Interspike Intervals
t̂
![Page 42: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/42.jpg)
t̂
))ˆ(exp())ˆ(()ˆ( ttuttuftt
escape rate)(t
Example: I&F with reset, constant input
t̂
Survivor function1
0ˆ( )S t t )')ˆ'(exp()ˆ( ̂
t
t
dtttttS
Interval distribution
t̂
0ˆ( )P t t
)ˆ(
)')ˆ'(exp()ˆ()ˆ(ˆ
ttS
dtttttttP
dtd
t
t
Neuronal Dynamics – 8.5. Renewal theory
![Page 43: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/43.jpg)
t̂
))ˆ(exp())ˆ(()ˆ( ttuttuftt
escape rate)(t
Example: I&F with reset, time-dependent input,
t̂
Survivor function1 ˆ( )S t t )')ˆ'(exp()ˆ( ̂
t
t
dtttttS
Interval distribution
t̂
ˆ( )P t t)ˆ(
)')ˆ'(exp()ˆ()ˆ(ˆ
ttS
dtttttttP
dtd
t
t
Neuronal Dynamics – 8.5. Time-dependent Renewal theory
![Page 44: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/44.jpg)
t̂
escape rate
0( )u
t for u
neuron with relative refractoriness, constant input
t̂
t̂
Survivor function1
)ˆ(0 ttS 0ˆ( )S t t
Interval distribution
t̂
)ˆ(0 ttP 0ˆ( )P t t
0
Neuronal Dynamics – Homework assignement
![Page 45: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/45.jpg)
Neuronal Dynamics – 8.5. Firing probability in discrete time
1
1( ) exp[ ( ') ']k
k
t
k k
t
S t t t dt
1t 2t 3t0 T
Probability to survive 1 time step
1( | ) exp[ ( ) ] 1 Fk k k kS t t t P
Probability to fire in 1 time stepFkP
![Page 46: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/46.jpg)
Neuronal Dynamics – 8.5. Escape noise - experiments
1 ( )( ) exp( )
u tt
escape rate
1 exp[ ( ) ]Fk kP t
Jolivet et al. ,J. Comput. Neurosc.2006
![Page 47: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/47.jpg)
Neuronal Dynamics – 8.5. Renewal process, firing probability
Escape noise = stochastic intensity
-Renewal theory - hazard function - survivor function - interval distribution-time-dependent renewal theory-discrete-time firing probability-Link to experiments
basis for modern methods of neuron model fitting
![Page 48: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/48.jpg)
Biological Modeling of Neural Networks
Week 8 – Noisy input models:
Barrage of spike arrivals
Wulfram GerstnerEPFL, Lausanne, Switzerland
8.1 Variation of membrane potential - white noise approximation8.2 Autocorrelation of Poisson8.3 Noisy integrate-and-fire
- superthreshold and subthreshold
8.4 Escape noise - stochastic intensity8.5 Renewal models8.6 Comparison of noise models
Week 8 – part 6 : Comparison of noise models
![Page 49: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/49.jpg)
escape process (fast noise)
stochastic spike arrival (diffusive noise)
u(t)
^t ^tt
)(t
))(()( tuftescape rate
)(tRIudt
dui
i
noisy integration
Neuronal Dynamics – 8.6. Comparison of Noise Models
![Page 50: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/50.jpg)
Assumption: stochastic spiking rate
Poisson spike arrival: Mean and autocorrelation of filtered signal
( ) ( )f
f
S t t t ( ) ( ) ( )x t F s S t s ds
mean( )t
( )F s
( ) ( ) ( )x t F s S t s ds ( ) ( ) ( )x t F s t s ds
( ) ( ') ( ) ( ) ( ') ( ' ') 'x t x t F s S t s ds F s S t s ds ( ) ( ') ( ) ( ') ( ) ( ' ') 'x t x t F s F s S t s S t s dsds
Autocorrelation of output
Autocorrelation of input
Filter
![Page 51: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/51.jpg)
Stochastic spike arrival: excitation, total rate Re
inhibition, total rate Ri
)()()(','
''
, fk
fk
i
fk
fk
erest tt
C
qtt
C
quuu
dt
d
u0u
EPSC IPSC
Synaptic current pulses
Diffusive noise (stochastic spike arrival)
)()()( ttIRuuudt
drest
Langevin equation,Ornstein Uhlenbeck process
Blackboard
![Page 52: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/52.jpg)
Diffusive noise (stochastic spike arrival)
2)()()()()( tututututu
)(tu)(tu
)()()( ttIRuuudt
drest
( ') ( ) ( ) ( ') ( ) ( ')u t u t u t u t u t u t
Math argument: - no threshold - trajectory starts at known value
![Page 53: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/53.jpg)
Diffusive noise (stochastic spike arrival)
2)()()()()( tututututu
)(tu
0( ) ( )u t u t
)()()( ttIRuuudt
drest
Math argument
( ') ( ) ( ) ( ') ( ) ( ')u t u t u t u t u t u t
2 2[ ( )] [1 exp( 2 / )]uu t t
![Page 54: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/54.jpg)
uu
Membrane potential density: Gaussian
p(u)
constant input ratesno threshold
Neuronal Dynamics – 8.6. Diffusive noise/stoch. arrivalA) No threshold, stationary input
)(tRIudt
dui
i
noisy integration
![Page 55: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/55.jpg)
uu
Membrane potential density: Gaussian at time t
p(u(t))
Neuronal Dynamics – 8.6 Diffusive noise/stoch. arrivalB) No threshold, oscillatory input
)(tRIudt
dui
i
noisy integration
![Page 56: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/56.jpg)
u
Membrane potential density
u
p(u)
Neuronal Dynamics – 6.4. Diffusive noise/stoch. arrivalC) With threshold, reset/ stationary input
![Page 57: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/57.jpg)
Superthreshold vs. Subthreshold regime
up(u) p(u)
Nearly Gaussian
subthreshold distr.
Neuronal Dynamics – 8.6. Diffusive noise/stoch. arrival
Image:Gerstner et al. (2013)Cambridge Univ. Press;See: Konig et al. (1996)
![Page 58: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/58.jpg)
escape process (fast noise)
stochastic spike arrival (diffusive noise)
A C
u(t)
noise
white(fast noise)
synapse(slow noise)
(Brunel et al., 2001)
t
t
dttt^
)')'(exp()( )¦( ^ttPI : first passage
time problem
)¦( ^ttPI Interval distribution
^t ^tt
Survivor functionescape rate
)(t
))(()( tuftescape rate
)(tRIudt
dui
i
noisy integration
Neuronal Dynamics – 8.6. Comparison of Noise Models
Stationary input:-Mean ISI
-Mean firing rateSiegert 1951
![Page 59: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/59.jpg)
Neuronal Dynamics – 8.6 Comparison of Noise Models
Diffusive noise - distribution of potential - mean interspike interval FOR CONSTANT INPUT
- time dependent-case difficult
Escape noise - time-dependent interval distribution
![Page 60: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/60.jpg)
stochastic spike arrival (diffusive noise)
Noise models: from diffusive noise to escape rates
))(()( 0 tuftescape rate
noisy integration
)(0 tu
t
)2
))((exp(
2
20
tu
)/))((( 0 tuerf
)]('1[ 0 tu
tu '0
))('),(()( 00 tutuft
![Page 61: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/61.jpg)
Comparison: diffusive noise vs. escape rates
))(()( 0 tuft
escape rate
)2
))((exp(
2
20
tu )]('1[ 0 tu))('),(()( 00 tutuft
subthresholdpotential
Probability of first spike
diffusiveescape
Plesser and Gerstner (2000)
![Page 62: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/62.jpg)
Neuronal Dynamics – 8.6 Comparison of Noise Models Diffusive noise
- represents stochastic spike arrival - easy to simulate - hard to calculate
Escape noise - represents internal noise - easy to simulate - easy to calculate - approximates diffusive noise - basis of modern model fitting methods
![Page 63: Biological Modeling of Neural Networks Week 8 – Noisy input models: Barrage of spike arrivals Wulfram Gerstner EPFL, Lausanne, Switzerland 8.1 Variation.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649ea15503460f94ba41c7/html5/thumbnails/63.jpg)
Neuronal Dynamics – Quiz 8.4.A. Consider a leaky integrate-and-fire model with diffusive noise:[ ] The membrane potential distribution is always Gaussian.[ ] The membrane potential distribution is Gaussian for any time-dependent input.[ ] The membrane potential distribution is approximately Gaussian for any time-dependent input, as long as the mean trajectory stays ‘far’ away from the firing threshold.[ ] The membrane potential distribution is Gaussian for stationary input in the absence of a threshold.[ ] The membrane potential distribution is always Gaussian for constant input and fixed noise level.
B. Consider a leaky integrate-and-fire model with diffusive noise for time-dependent input. The above figure (taken from an earlier slide) shows that[ ] The interspike interval distribution is maximal where the determinstic reference trajectory is closest to the threshold.[ ] The interspike interval vanishes for very long intervals if the determinstic reference trajectory has stayed close to the threshold before - even if for long intervals it is very close to the threshold[ ] If there are several peaks in the interspike interval distribution, peak n is always of smaller amplitude than peak n-1.[ ] I would have ticked the same boxes (in the list of three options above) for a leaky integrate-and-fire model with escape noise.