Part II: Population Models

Part II: Population Models

BOOK: Spiking Neuron Models, W. Gerstner and W. Kistler Cambridge University Press, 2002

Chapters 6-9

Laboratory of Computational Neuroscience, LCN, CH 1015 LausanneSwiss Federal Institute of Technology Lausanne, EPFL

Chapter 6: Population Equations


Chapter 6

10 000 neurons3 km wires

1mm

Signal:action potential (spike)

action potential

Spike Response Model

iuij

fjtt

Spike reception: EPSP

fjtt


^itt

^itt

Spike emission: AP

fjtt ^

itt tui j f

ijw

tui Firing: tti ^

linear

threshold

Spike emission

Last spike of i All spikes, all neurons

Integrate-and-fire Model

iui

fjtt


)(tRIuudtd

ii

tui Fire+reset

linear

threshold

Spike emission

resetI

j

escape process (fast noise)

parameter changes (slow noise)

stochastic spike arrival (diffusive noise)

Noise models

A B C

u(t)

noise

white(fast noise)

synapse(slow noise)

(Brunel et al., 2001)

t

t

dttt^

)')'(exp()( )¦( ^ttPI

: first passagetime problem

)¦( ^ttPI Interval distribution

^t ^t ^tt

Survivor functionescape rate

)(t

))(()( tuftescape rate stochastic reset

)¦( ^ttPI )( fttG

Interval distribution

Gaussian about ft

)(tRIudtdu

ii

noisy integration

ft

Homogeneous Population

populations of spiking neurons

I(t)

?

population dynamics? t

t

tNtttntA

);()(population

activity

Homogenous network (SRM)

NJwij

0


Spike emission: AP

fjtt ^

itt tui j f

ijwLast spike of i All spikes, all neurons

fjtt

^itt

Synaptic coupling

potential

fullyconnected N >> 1

dsstIs )( external input

NJwij

0

fjtt ^

itt tui j f

ijwLast spike of i All spikes, all neuronspotential

dsstIs )( external input

dsstIs )( tui ^itt dsstAsJ )(0

potential

^tt ^| ttu )(thinput potential

fullyconnected

refractory potential

Homogenous network

Response to current pulse

Spike emission: AP

s

^itt

potential

^tt ^| ttu )(thinput potential

itt ˆ tui

Last spike of ipotential

dsstIsJ )(0 external input

dsstAs )( Population activity

All neurons receive the same input

Assumption of Stochastic spike arrival: network of exc. neurons, total spike arrival rate A(t)

)()(,

0 fk

fkrest tt

Cq

NJ

uuudtd

u

0u

EPSC

Synaptic current pulses

Homogeneous network (I&F)

)()( tIRuuudtd

rest

)()( 0 tAqJtI

Density equations

Stochastic spike arrival: network of exc. neurons, total spike arrival rate A(t)

)()()(,

0 f

feext

fk

fk

erest tt

Cq

JttCq

NJ

uuudtd

u

0u

EPSC

Synaptic current pulses

Density equation (stochastic spike arrival)

)()()( ttIRuuudtd

rest Langenvin equation,Ornstein Uhlenbeck process

fqJtAqJtI ext )()( 0

u

p(u)

Density equation (stochastic spike arrival)

u

Membrane potential density

)()(),()],()([),( 2

22

21 tAuutup

utupuV

utup

t r

Fokker-Planck

drift diffusionAqJuuV 0)(

kkk w22

spike arrival rate

source term at reset

A(t)=flux across threshold

utupu

tA ),()(

Integral equations

tdtAttPtA I

t

ˆ)ˆ(ˆ|)(

Population Dynamics

tdtAttSI

t

ˆ)ˆ(ˆ|1

Derived from normalization

Escape Noise (noisy threshold)

t̂

)(t

I&F with reset, constant input, exponential escape rate


t̂

)ˆ(0 ttP )')ˆ'(exp()ˆ()ˆ( ̂t

t

dtttttttP

)exp())ˆ(()ˆ()ˆ(u

ttuttuftt

escape rate

tdtAttPtA I

t

ˆ)ˆ(ˆ|)(

Population Dynamics

Wilson-Cowan

population equation


Wilson-Cowan model

h(t)

^t t

)(t

))(()( thftescape rate

(i) noisy firing(ii) absolute refractory time

abs

))(()( thftA

population activity

t

t abs

dttA

]')'(1[

(iii) optional: temporal averaging

))(()()( thgtAtAdtd

abs

abs

ttfor

ttforthftuft

)ˆ(00

)ˆ())(())(()(

escape rate


Wilson-Cowan model

h(t)

^t t

)(t

(i) noisy firing(ii) absolute refractory time

abs

))(()( thftA

population activity

t

t abs

dttA

]')'(1[

abs

abs

ttfor

ttforthftuft

)ˆ(00

)ˆ())(())(()(

escape rate

tdtAttPtA I

t

ˆ)ˆ(ˆ|)(

Population activity in spiking neurons (an incomplete history)

1972 - Wilson&Cowan; Knight Amari

1992/93 - Abbott&vanVreeswijk Gerstner&vanHemmen

Treves et al.; Tsodyks et al. Bauer&Pawelzik

1997/98 - vanVreeswijk&Sompoolinsky Amit&Brunel Pham et al.; Senn et al.

1999/00 - Brunel&Hakim; Fusi&Mattia Nykamp&Tranchina Omurtag et al.

Fast transientsKnight (1972), Treves (1992,1997), Tsodyks&Sejnowski (1995)Gerstner (1998,2000), Brunel et al. (2001), Bethge et al. (2001)

Integral equation

Mean field equationsdensity (voltage, phase)

Heterogeneous netsstochastic connectivity

(Heterogeneous, non-spiking)

Chapter 7: Signal Transmission and Neuronal Coding


Chapter 7

Coding Properties of Spiking Neuron ModelsCourse (Neural Networks and Biological Modeling) session 7 and 8


PSTH(t)

500 trials

I(t)

forward correlationfluctuating input

I(t)reverse correlationProbability of

output spike ?

I(t) A(t)?0t

Theoretical Approach

- population dynamics

- response to single input spike (forward correlation)

- reverse correlations

A(t)

500 neurons

PSTH(t)

500 trials

I(t) I(t)

Population of neurons

h(t)

I(t) ?0t

))(()( thgtA

A(t)

A(t)

A(t)

))(()( tIgtA

))('),(()( tItIgtA

potential

A(t) ))(()()( thgtAtAdtd

t

tNtttntA

);()(population

activity

N neurons,- voltage threshold, (e.g. IF neurons)- same type (e.g., excitatory) ---> population response ?

Coding Properties of Spiking Neurons:


- forward correlations- reverse correlations

1. Transients in Population Dynamics - rapid transmission2. Coding Properties

Example: noise-free

tdtAttPtA I

t

ˆ)ˆ(ˆ|)(

))ˆ(ˆ(ˆ| tTttttPI

)(tA )( TtA

''1

uh

Population Dynamics

I(t) h’>0h(t)

T(t^)

higher activity

noise-free

Theory of transients

)(tA )( TtA

''1

uh

I(t)h(t)

I(t) ?0t

potential dsstIs )( ^tt ^| ttu

)(thinput potential 0)(' ttth

)()( 00 ttAAtA A(t)

External input.No lateral coupling

Theory of transients A(t)

no noise

I(t)h(t)

noise-free

noise model B

slow noise

I(t)h(t)

(reset noise)

u

p(u)

u


Hypothetical experiment: voltage step

u

p(u)

Immediate responseVanishes linearly

Transients with noise




Noise models

A B C

u(t)

noise

white(fast noise)

synapse(slow noise)


t

t

dttt^

)')'(exp()(


^t ^t ^tt


)(t


)¦( ^ttPI )( fttG


Gaussian about ft

)(tRIudtdu

ii

noisy integration

ft

Transients with noise:Escape noise (noisy threshold)

linearize

tdtAttPtA I

t

ˆ)ˆ(ˆ|)(

)()( 0 tAAtA

Theory with noise A(t)

)()( 0 thhth

I(t)h(t)

0A

dsstIsth )()()(

sA 1

0 inverse mean interval

I

Llow noiselow noise: transient prop to h’

high noise: transient prop to h

h: input potential

high noise

Theory of transients A(t)

low noise

I(t)h(t)

noise-free

(escape noise/fast noise) noise model A

low noise

fast

noise model A

I(t)h(t)

(escape noise/fast noise)

high noise

slow

Transients with noise:Diffusive noise (stochastic spike arrival)




Noise models

A B C

u(t)

noise

white(fast noise)

synapse(slow noise)


t

t

dttt^

)')'(exp()(


^t ^t ^tt


)(t


)¦( ^ttPI )( fttG


Gaussian about ft

)(tRIudtdu

ii

noisy integration

ft

u

p(u)

Diffusive noiseu


p(u)


Immediate responsevanishes quadratically

),(

)],()([

),(

2

22

21 tup

u

tupuAu

tupt

Fokker-Planck

u

p(u)

SLOW Diffusive noiseu



Immediate responsevanishes linearly

p(u)

Signal transmission in populations of neurons

Connections4000 external4000 within excitatory1000 within inhibitory

Population- 50 000 neurons- 20 percent inhibitory- randomly connected

-low rate-high rate

input

Population- 50 000 neurons- 20 percent inhibitory- randomly connected

Signal transmission in populations of neurons

100 200time [ms]

Neuron # 32374

50

u [mV]

100

0

10A [Hz]

Neu

ron

#

32340

32440

100 200time [ms]50

-low rate-high rate

input

Signal transmission - theory

- no noise

- slow noise (noise in parameters)

- strong stimulus

- fast noise (escape noise) prop. h(t) (potential)

prop. h’(t) (current)

See also: Knight (1972), Brunel et al. (2001)

fast

slow

Transients with noise: relation to experiments

Experiments to transients A(t)

V1 - transient response

V4 - transient response

Marsalek et al., 1997

delayed by 64 ms

delayed by 90 ms

V1 - single neuron PSTH

stimulus switched on

Experiments

input A(t)

A(t)

A(t)

A(t)

See also: Diesmann et al.

How fast is neuronal signal processing?

animal -- no animalSimon ThorpeNature, 1996

Visual processing Memory/association Output/movement

eye

Reaction time experiment

How fast is neuronal signal processing?

animal -- no animalSimon ThorpeNature, 1996

Reaction time

Reaction time

# ofimages

400 msVisual processing Memory/association Output/movement

Recognition time 150ms

eye

Coding properties of spiking neurons


- response to single input spike

(forward correlations)

A(t)

500 neurons

PSTH(t)

500 trials

I(t)I(t)


- response to single input spike

(forward correlations)

I(t) Spike ?Two simple arguments1)

2)

Experiments: Fetz and Gustafsson, 1983 Poliakov et al. 1997

(Moore et al., 1970)

PSTH=EPSP

(Kirkwood and Sears, 1978)

PSTH=EPSP’

Forward-Correlation Experiments A(t)

Poliakov et al., 1997

I(t) PSTH(t)

1000 repetitionsnoise

high noise low noiseprop. EPSP prop. EPSP

ddt

^^^ )(|)( dttAttPtA I

t

Population Dynamics

)()( 0 thhth h: input potential dsstIsth )()()(

A(t) PSTH(t)I(t)I(t)

full theory

linear theory


Theory: Herrmann and Gerstner, 2001

high noise low noisePoliakov et al., 1997

high noise low noise

blue: full theoryred: linearized theory

blue: full theoryred: linearized theory


Poliakov et al., 1997

I(t) PSTH(t)

1000 repetitionsnoise

high noise low noiseprop. EPSP prop. EPSP

ddt

prop. EPSP

prop. EPSPddt

Reverse Correlations


fluctuating input

I(t)

Reverse-Correlation Experiments

after 1000 spikes

)(tI

)()( 0 thhth h: input potential dsstIsth )()()(

Linear Theory

Fourier Transform

)(~)(~)(~ IGA

0

)()()( dsstIsGtA

Inverse Fourier Transform

)(~1)(~)(~

)(~ 0

PLAiG

Signal transmissionI(t) A(t)

)()()(

fIfAfG T=1/f

(escape noise/fast noise) noise model A

low noise

high noise

noise model B (reset noise/slow noise)

high noiseno cut-off

low noise

Reverse-Correlation Experiments (simulations)

after 1000 spikes

0

)()()( dsstIsGtA

theory:G(-s)

)(tI

after 25000 spikes

Laboratory of Computational Neuroscience, EPFL, CH 1015 Lausanne

Coding Properties of spiking neurons

I(t)

?

- spike dynamics -> population dynamics- noise is important - fast neurons for slow noise - slow neurons for fast noise

- implications for - role of spontaneous activity - rapid signal transmission - neural coding - Hebbian learning

Chapter 8: Oscillations and Synchrony


Chapter 8

Stability of Asynchronous State

Stability of Asynchronous State

Search for bifurcation points

linearize

^^^ )(|)( dttAttPtA I

t

)()( 0 tAAtA )()( 0 thhth dsstAsJth )()()(

h: input potential

A(t)

ttieAtA 1)(0

fully connected coupling J/N

Stability of Asynchronous State A(t)

delayperiod

)()( sess0 for

stable0

03

02

noise

T

)(s

s

)(s

Stability of Asynchronous State s

)(s

ms0.1

ms2.1

4.1

ms0.2

ms0.3

ms4.0

06

05

04

03

02

T 2

0

Chapter 9: Spatially structured networks


Chapter 9

Continuous Networks

)(tAi

Several populations

i k

Continuum

)(),( AtA

Continuum: stationary profile

Part II: Population Models

Documents

Transcript of Part II: Population Models