Neural noise and neural signals - spontaneous activity and signal transmission

in models of single nerve cells

Benjamin Lindner

Theory of Complex Systems and Neurophysics Institut für Physik Humboldt-Universität Berlin

1

Formal matter

Weekly tutorial on Mondays 13:15-15:45 starting on May 2nd

Problems are handed out two weeks before the tutorial; required for passing: 70% of the points

Oral examination before the new semester starts

ECTS: 6 (both Computational Neuroscience & Physics)

URL of the lecture: http://people.physik.hu-berlin.de/~neurophys/neusig/

2

Motivation

3

Sensory signals are represented by sequences of electrical discharges -

neural spike trains

time [s]

membrane voltage [mV]

4

Neurons fire randomly

Nawrot et al. Neurocomp. (2007)

Spontaneous firing (Neocortex of rat)

Different responses to the same stimulus (auditory neuron of locust)

Karin Fisch & Jan Benda, LMU

Interspike interval (ISI) statistics ?

Aim: reproduce and understand the spike statistics Statistics shapes information transmission and processing! 5

Aim of the course

Analytical calculation of the statistics of single-neuron activity, modeled by dynamical systems in a stochastic framework

Spontaneous activity

Evoked activity

Noise

Spike trainStimulus

Noise

Spike train

neuron

neuron

Statistics: Rate, Cv, ISI density & correlations; spike train spectrum & Fano factor

Statistics: Rate modulation, phase lag; signal-to-noise ratio, coherence, mutual information rate

neuron

6

7

What can be achieved by stochastic models of

single-neuron activity?

p(I) = r0 exp[�r0I]

Spiking statistics of a Poisson process ...

0 50 100 150Frequency [Hz]

10-1

100

101

102

103

Spike trainpower spectrum

Poisson process

2 4 6 8 10 12 14 16 18 20lag

-0.5

0

0.5

1

Intervalcorrelationcoefficient

Poisson process

0 20 40 60 80ISI [ms]

0

0.02

0.04

0.06

0.08

0.1

ISIprobability density

Poisson process

8

Spiking statistics of a Poisson process ...... and of a sensory neuron

0 50 100 150Frequency [Hz]

10-1

100

101

102

103

Spike trainpower spectrum

Poisson processelectro-sensory neuron

2 4 6 8 10 12 14 16 18 20lag

-0.5

0

0.5

1

Intervalcorrelationcoefficient

Poisson processelectro-sensory neuron

0 20 40 60 80ISI [ms]

0

0.02

0.04

0.06

0.08

0.1

ISIprobability density

Poisson processelectro-sensory neuron

data: Neiman & Russell, Ohio University9

Spiking statistics of a Poisson process ...... and of a sensory neuron and the right theory!

data: Neiman & Russell, Ohio University

0 50 100 150Frequency [Hz]

10-1

100

101

102

103

Spike trainpower spectrum

theory

Poisson process

electro-sensory neuron

2 4 6 8 10 12 14 16 18 20lag

-0.5

0

0.5

1

Intervalcorrelationcoefficient

theory

Poisson process

electro-sensory neuron

0 20 40 60 80ISI [ms]

0

0.02

0.04

0.06

0.08

0.1

ISIprobability density

theory

Poisson process

electro-sensory neuron

10

11What can be achieved by stochastic models of

single-neuron activity?

• understand neural spontaneous activity(single neurons are complex non-equilibrium systems)

How much info does the spike train carry about the sensory signal?

time

Information theory of neural spiking

12

Do neurons encode more information on slow or on fast components of a sensory stimulus?

Chacron et al. Nature 2003

encoding of fast stimulus

components

encoding of slow stimulus

components

13

Visual System • Reinagel et al. J Neurophisiol (1999) • Passaglia et al. J Neurophisiol (2004) ... Auditory System • Rieke et al. Proc. R. Soc. Lond. (1995) • Marsat et al. . J Neurophisiol (2004) ... Vestibular System • Sadeghi et al. J Neurosci (2007) • Massot et al. J Neurophisiol (2011) ... Electrosensory System • Oswald et al. J Neurosci (2004) • Middleton et al. PNAS (2006) ...

Information filtering is ubiquitous

14

15What can be achieved by stochastic models of

single-neuron activity?

• understand neural spontaneous activity(single neurons are complex non-equilibrium systems)

• understand the neuron’s capability to transmit and filter information about time-dependent stimuli

(equivalent to populations of uncoupled neurons)

16Methods are useful for problem of recurrent neural nets

17Methods are useful for problem of recurrent neural nets

Network of model neurons:

Equation for the probability density of the membrane voltage:

Brunel J. Comp. Neurosci (2000)

Brunel J. Comp. Neurosci (2000)

Collective behavior of neurons in recurrent networks18

19What can be achieved by stochastic models of

single-neuron activity?

• understand neural spontaneous activity(single neurons are complex non-equilibrium systems)

• understand the neuron’s capability to transmit and filter information about time-dependent stimuli

(equivalent to populations of uncoupled neurons)

• advanced theory of recurrent networks of spiking neurons rely on accurate theory of single-cell activity

(methods are an extension of those for the single cell)

Schedule (approximate)

20.4. Motivation; spike-train statistics 27.4. Spike-train statistics II 4.5. How much information does a spike train contain? 11.5. Mutual information - direct estimates and lower bound 18.5. Beyond statistical models: Sources of neural noise and their incorporation into dynamical neuron models 25.5. Effective time-constant approximation & IF models:Fokker-Planck analysis (intro) 1.6. IF models:Fokker-Planck analysis of spontaneous activity 8.6. IF models:Fokker-Planck analysis of driven activity I15.6. IF models:Fokker-Planck analysis of driven activity II 22.6. Stochastic resonance in driven IF models 29.6. Multi-dimensional IF models: colored noise, spike- frequency adaptation, subthreshold oscillations 6.7. Properties of spontaneous firing & signal transmission 13.7. Signal transmission in neural populations of uncoupled neurons 20.7. Outlook: recurrent networks

20

•Stochastic Processes in general ...

- Risken The Fokker-Planck Equation Springer (1984)

- Gardiner Handbook of Stochastic Methods Springer (1985)

- Van Kampen Stochastic Processes in Physics and Chemistry North-Holland (1992)

- Cox & Isham Point Processes Chapman and Hall (1980)

•... and in neuroscience:

- Holden Models of the Stochastic Activity of Neurones Springer-Verlag (1976)

- Tuckwell Stochastic Processes in the Neuroscience SIAM (1987)

- Gerstner & Kistler Spiking Neuron Models Cambridge University Press (2002)

- Gabbiani & Cox Mathematics for Neuroscientists Elsevier (2010)

- Dayan & Abbott Theoretical Neuroscience MIT Press (2001)

General references21

Spike train statistics

22

x(t) =�

�(t� ti)

From the membrane dynamics to the point process

Spike train:

23

A stationary process - time-independent firing rate

Spike train:

time

.

.

.

Spike trains

r(t) =p(t, t + dt)

dt

firing rate per unit time

Estimate for the firing probability in dt

p(t, t + dt) =# of neurons that fire in(t, t + dt)

total # neurons

Steveninck et al. Science 1997

spontaneous firing of a motion-sensitive neuron

in the blow fly

24

A non-stationary process - time-dependent firing rate

.

.

.

Spike trains

r(t) =p(t, t + dt)

dt

firing rate per unit time

Estimate for the firing probability in dt

p(t, t + dt) =# of neurons that fire in(t, t + dt)

total # neurons

Steveninck et al. Science 1997

evoked firing of a motion-sensitive neuron

in the blow fly

25

Simple example: Poissonian spike train (homogeneous)

x(t) =�

�(t� ti)

1. draw N points from a uniform density along the t -axis

tiDefining property: all are independent!

Three ways to simulate a Poisson process} If

Poisson in

with rate

Only one parameter: the firing rate

26

Simple example: Poissonian spike train (homogeneous)

x(t) =�

�(t� ti)

2. discretize t -axis; draw independent random number for each bin

tiDefining property: all are independent!Only one parameter: the firing rate

Three ways to simulate a Poisson process

27

Simple example: Poissonian spike train (homogeneous)

x(t) =�

�(t� ti)

3. draw time interval to the next spike (Gillespie method)

tiDefining property: all are independent!Only one parameter: the firing rate

Three ways to simulate a Poisson process

0 2 4 6 8 10ISI

0

0.2

0.4

0.6

0.8

1

ISI density

Prob. density

28

C(�) = ⇥x(t)x(t + �)⇤ � ⇥x(t)⇤2

Spike train correlation function

For a stationary process

29

C(⇥) = r0[�(⇥) + m(⇥)] � r20

S(f) = r0

C(⇥) = r0�(⇥)

Spike train correlation function

For a Poisson process

corresponding to a flat spike train power spectrum

Generally, for any stationary stochastic process

Wiener-Khinchin theorem:

30

x̃(f) =T�

0

dt x(t) e2�ift

Spectral measures

Fourier transform of the process

a new random function (of frequency)!

�x̃(f)⇥ = 0 For a stationary process x(t):

S(f) = limT⇥⇤

�x̃x̃�⇥T

.... and the power spectrum indicates possible stochastic

periodicity by finite peaks

hair bundle from a sensory hair cell

of bullfrog

sensory neuron in paddlefish

Neiman & Russell J. Neurophysiol. 200431

Spike train power spectra are not flat in general

Neiman & Russell J. Neurophysiol. 2004

fa

fe

f −a fe

f +a fe

Frequency

100

0

Bair et al. J. Neuroscie. (1994)

Power spectrum

32

N(t) =⇥ t

0dt�x(t�) =

�

0<ti<t

�(t� ti)

spik

e tra

ins

0 1 2 3time

0

10

20

30

40

spik

e co

unts

Spike train -> spike count

33

�N(t)⇥ =� t

0dt��x(t�)⇥ =

� t

0dt�r(t�)

Spike train -> spike count -> diffusion process with drift

spik

e tra

ins

0 1 2 3time

0

10

20

30

40

spik

e co

unts

�N(t)⇥ = r0T

For a stationary process

34

⇥�N2(t)⇤ =� t

0

� t

0dt1dt2⇥x(t1)x(t2)⇤ � ⇥x(t1)⇤⇥x(t2)⇤

⇤�N2(t)⌅ � 2De�t, for t �⇥

F (t) =��N2(t)⇥�N(t)⇥

F (t)� 2De�

r0for t�⇥

Spike train -> spike count -> diffusion process with drift

Spike count variance

Often, spike count shows normal diffusion

Temporal structure of variability is described in the Fano factor

For a Poisson process at all times F (t) = 135

F (t) =��N2(t)⇥�N(t)⇥

Fano factor does not always saturate ...

Lowen & Teich J. Acoust. Soc. Am. 1992

cat auditory nerve unit A

36

Interval statistics

IntervalBrenner et al. Phys. Rev E (2003)

Probability density of the interspike interval (ISI)

��I2⇥Variance of the ISI

Mean ISI �I⇥

CV =�

��I2⇥�I⇥

Coefficient of variation

CV = 1For a Poisson process

p(I)p(I)

37

Examples for ISI densities

Brenner et al. Phys. Rev E (2003)

p(I) = r0 exp[�r0I]Poisson process

0 2 4 6 8 10ISI

0

0.2

0.4

0.6

0.8

1

ISI density

38

The moments do not determine the probability density

Example:

39

interspike intervals (ISI)

with density

and

n-th order intervalswith densities

tti ti+1 ti+2

Ii Ii+1 Ii+2

ti+3ti-1

Ii-1 Ii+3

tti ti+1 ti+2

T1,i

T2,i

T3,i

ti+3ti-1

N-th order intervals

40

ρ1 > 0

ρ1 < 0

time

short short short

long long

time

short short short

long long long

long

Interval correlations (in nonrenewal spike trains)

41

Chacron et al. Phys. Rev. Lett. 2001

Neiman & Russel Phys. Rev. Lett. 2001P-units in weakly

electric fish

Electroreceptors in paddle fish

long-range positive correlations for auditory

neurons

Lowen & Teich J. Acoust. Soc. Am. Rev. 1992

ISI  correla*ons  reviewed  in    Farkhooi  et  al.  Phys.  Rev.  E  2009  Akerberg  &  Chacron  2011

Interval correlations (in nonrenewal spike trains)

42

Renewal spike trains

All intervals are mutually independent!

Consequence for the Fourier transforms of the n-th order interval density

Example: Poisson process

43

Relation between different spike train statistics

Time-averaged firing rate =inverse mean ISI

(stationary processes)

44

Power spectrum - Fourier transforms of n-th order interval densities

Relation between different spike train statistics

45

Lindner Phys. Rev. E (2004)

For a renewal spike train

��T 2n⇥ = n��T 2

1 ⇥

tti ti+1 ti+2

T1,i

T2,i

T3,i

ti+3ti-1⇥�T 2

n⇤ �= n⇥�T 21 ⇤

For a nonrenewal spike train

�k = 0, k > 0

Relation between different spike train statistics

46

Lindner Phys. Rev. E (2004)

Relation between different spike train statistics

47

Last time ... For a stationary process:

1.Firing rate

2.n-th order interval variance

3. Interval correlation coefficient

4. Power spectrum

--> renewal process: 5. Fano factor

renewal:

48

For a stationary process:

1.Firing rate

2.n-th order interval variance

3. Interval correlation coefficient

4. Power spectrum

--> renewal process: 5. Fano factor

nonrenewal: renewal:

49

Negative ISI correlations cause a drop in spectral power at low frequency

Chacron et al. Proc. SPIE 2005

electroreceptor (P-units) in the weakly electric fish

50

Negative ISI correlations cause a drop in spectral power at low frequency

Shuffled spike train

Shuffled spike train

Original spike trainOriginal spike train

Chacron et al. Proc. SPIE 2005

electroreceptor (P-units) in the weakly electric fish

51

Gamma process - ISI density and spike train power spectrum

52

Summary: spike-train statistics

-Point processes can be characterized •by spike train (rate) and count statistics (Fano factor) •by interval statistics (ISI and nth-order intervals, interval correlations) -ISI densities often unimodal; spectra non-flat --- not well described by Poisson statistics!

-Spike train and interval statistics are related in a nontrivial manner

- Cox & Lewis The Statistical Analysis of Series of Events Chapman and Hall (1966)

- Cox & Isham Point Processes Chapman and Hall (1980)

- Holden Models of the Stochastic Activity of Neurones Springer-Verlag (1976)

- Gerstner & Kistler Spiking Neuron Models Cambridge University Press (2002)

References

53