Synaptic Failure Rao Pärnpuu, Oliver Härmson

Introduction The Aim: -  Understanding the role of synaptic failure (SF) in the

brain Two parts: -  Overview of previous studies, both computational and

biological -  Creating a model and running simulations under

different circumstances

Synaptic failure and literature

•  SF reported to be 0.1 … 0.9 •  Some energy losing

transformations - Channel failure - Quantal failure - Quantal amplitude fluctuation - Dendrosomatic summation

Benefit? (a) Energy expenditure (b) Information processing

tion of the computation. This upper bound occurs when the second termis zero, i.e. in the failure-free, noise-free situation with all quanta thesame size. Appealing to the central limit theorem, this entropy is wellapproximated by the entropy of a normal distribution. Therefore, if wesuppose each of the n inputs is an independent Bernoulli process with thesame parameter p ! p*, we get:

H"# Xi $ !12 log2%2"enp*"1 # p*$& ! 6.5 bits for n ! 104 and p* ! 0.05,

where this value of p* comes from the Levy and Baxter (1996) calcula-tions as well as the actual observed value of average firing rates inneocortex. Although 6.5 bits is a tremendous drop from H(X), whichunder these assumptions is 2860 bits (10,000 inputs each with 0.286 bits),this 6.5 bits is still a very large number of bits to be transmitted percomputational interval compared to the energy-efficient channel capacityof H(p*) ! 0.286 bits.

The reason why 6.5 bits is a tremendous excess arises when we considerShannon’s source/channel theorems. These say that the channel limits themaximum transmittable information to its capacity. As a result, anyenergy that goes toward producing I(X; # Xi ) that exceeds the channelcapacity H(p*) is wasted information. This idea is at the heart of theanalysis that follows. Because the total information of the computation ismany times the energy-efficient channel capacity, much waste is possible.Indeed, even if we dispense with the independence assumption (whilestill supposing some kind of central limit result holds for the summedinputs) and suppose that statistical dependence of the inputs is so badthat every 100 inputs act like 1 input, an approximation that strikes us asmore than extreme, there still are too many bits ('3.2 bits) beinggenerated by the computation compared with what can be transmitted.Thus, the computation is not going to be energy-efficient if it takesenergy (and it does) to develop this excess, nontransmittable computa-tional information.

RESULTSWe now begin the formal analysis that substantiates and quanti-fies the conjecture and that brings to light a set of assumptionsmaking the conjecture true.

AssumptionsA0: A computation by an excitatory neuron is the summation ofits inputs every computational interval. The mutual informationof such information processing is closely approximated as:

IC ! I"X; !i

$"Xi$Qi$.

A1: Axons are binary signaling devices carrying independentspikes and used at their energy optimum; that is, each axon isused at the information rate CE bits per computational interval,which implies firing probability p*.

A2: The number of inputs to a neuron is not too small—sayn ( 2/p*. Clearly this is true in neocortex; see Fig. 3 for evalua-tion of this assumption.

A3: With the proviso that A1 and A2 must be obeyed, a processrequiring less energy is preferred to a process requiring moreenergy.

A4: The spike generator at the initial segment, which incorpo-rates generic nonlinearities operating on the linear dendriticsummation, creates a bitwise code suitable for the axonal channel,and this encoding is nearly perfect in using the informationreceived from the dendrosomatic computation. That is, as an

Figure 1. Partitioning communication andcomputation for a single neuron and its in-puts. A, The presynaptic axonal inputs to thepostsynaptic neuron is a multivariate binaryvector, X ! [X1 , X2 , . . ., Xn]. Each input, Xi ,is subject to quantal failures, the result ofwhich is denoted by $ (Xi ), another binaryvector that is then scaled by quantal ampli-tude, Qi. Thus, each input provides excitation$(Xi )Qi. The dendrosomatic summation,#i $(Xi )Qi is the endpoint of the computa-tional process, and this sum is the input to thespike generator. Without specifying any par-ticular subcellular locale, we absorb genericnonlinearities that precede the spike genera-tor into the spike generator, g (#i $(Xi )Qi ).The spike generator output is a binary vari-able, Z, which is faithfully transmitted downthe axon as Z). This Z) is just another Xielsewhere in the network. In neocortex, ex-perimental evidence indicates that axonalconduction is, essentially, information loss-less, as a result I(Z; Z)) * H(Z). The infor-mation transmitted through synapses and den-drosomatic summation is measured by themutual information I(X; # $(Xi )Qi ) !H(X) + H(X"#i $(Xi )Qi ). Given the assump-tions in the text combined with one of Shan-non’s source-channel theorems implies that,H(X) + H(X"#i $(Xi )Qi ) ! H(p*), whereH(p*) is the energy-efficient maximum valueof H(Z). B, The model of failure prone syn-aptic transmission. An input value of 0, i.e., nospike, always yields an output value of 0, i.e.,no transmitter release. An input value of 1, anaxonal spike, produces an output value of 1,transmitter release, with probability successs ! 1 + f. A failure occurs when an input valueof 1 produces an output value of 0. The prob-ability of failure is denoted by f.

4748 J. Neurosci., June 1, 2002, 22(11):4746–4755 Levy and Baxter • Efficient Neural Computation Via Quantal Failures

of how well the MID filters capture retinogeniculate informationtransmission. For the example cell pair, the combined MIDmodel performed better than the MID filters individually, ac-counting for 91% and 92% of the information carried by singlespikes of the retinal and LGN cells respectively (supplementalTable 1, available at www.jneurosci.org as supplemental mate-rial). There was no significant difference in the percentage ofinformation accounted for by the combined MIDs across thepopulation ( p ! 0.73, Wilcoxon paired test). The MID filtersexplained a mean of "85% of the total information available inthe spike trains (Fig. 5b), and often nearing 100%. Therefore, as a

reduced model of the retinal and LGN neural activity, the com-bined MID filters captured most of the information that could beextracted about the stimulus in the spike trains. Because infor-mation here was computed indirectly by a stimulus reconstruc-tion procedure, the quantities represent a lower bound on theamount of total information carried in single spikes (Strong et al.,1998; Reinagel et al., 1999; Reinagel and Reid, 2000; Adelman etal., 2003; Fairhall et al., 2006). The remaining "15% of the infor-mation in the spike train was presumably provided by higher-order filters (which we could not compute because of data limi-tations), or by inputs from other neurons that were undetectable

Figure 4. Stimulus representation changes from retina to LGN. a, b, Normalized MID filters for the RGC (red) and LGN neuron (black) are temporally altered, as revealed by subtracting the filters(blue). MID1 yields the most information, while MID2 represents an orthogonal stimulus dimension adding maximal information to the first. Error estimates computed over data subsets had averageSEMs #0.01 for each point along all filters (data not shown). c, d, Spike rate gain measures how far above the mean firing rate (at gain ! 1) any stimulus can drive the cell, plotted as a function ofprojection value distributions (see Materials and Methods for details). Filter combinations significantly increase the spike rate gain of the LGN neuron (d) over the RGC (c). Projection values withpositive SDs represent stimuli with increasing resemblance to the filters. e, Population data comparing RGC and LGN peak spike rate gains for all filters (e.g., the peaks in supplemental Fig. 1g–i,available at www.jneurosci.org as supplemental material), normalized to the peak RGC STA rate gain. The combined MID filters exhibited the highest gains. f, Peristimulus time histograms of spikerate gains for 3 cells, in response to the same repeated stimuli. Stimulus segment is the same as shown in Figure 1d. When LGN gains (black) are above the unity gain line (gray), they exceed RGC gains(red); and when below unity, they are lower than RGC gains, suggesting a greater modulation range for LGN neurons. The information transmission ratios for these cell pairs were: magno ON ! 1.0;magno OFF ! 0.74; parvo ON ! 0.75.

6212 • J. Neurosci., May 13, 2009 • 29(19):6207– 6216 Sincich et al. • Preserving Information in Neural Transmission

The Model ●  Integrate-and-fire (2 input

neurons, 1 output neuron) ●  Different input spike trains: a.  Totally regular b.  Poissonian, 100% correlated c.  Poissonian, 80% correlated d.  Poissonian, uncorrelated e.  Poissonian, 100% correlated, independent fail ●  Input rate normalization

Figure Input rate normalization

Figure SVM classifier decision boundrary

Methods ●  Data analysis and implementation: MATLAB ●  Classification used SVM classifier: Training set: millisecond averages of membrane potentials. Training set: 80% and test set:20% of outputs ●  Simulated failure rates of 0.0 to 0.9 , with 0.1 steps ●  Used two combinations of input neuron weights: 5-5 and

7-3

Aims (i) To see whether changing the membrane potential dynamics in response to a spike event in the postsynaptic neuron will change the amount of spikes fired. (ii) To see whether different probabilities of synaptic failure will affect the neuron’s ability to distinguish one input from the other, using machine learning. (iii) To see how well the postsynaptic neuron distinguishes its inputs when input spike train types are altered, using machine learning.

Results

Changing membrane potential parameters What should the membrane potential do after a spike event occurs in the soma? ●  Vreset = always -65 mV -  Vrestchanging = -2mV after every spike, constantly changing

Experiment 1 Regular spiketrains, 100% correlated and independent failure events Simulations with normalised input of 10Hz and 30Hz, over 101 seconds

10 Hz input

Figure 1.

Figure 2.

30 Hz input

Experiment 2 Poissonian fully correlated spike trains •  Probability of failure was simulated by

removing spikes from both spiketrains at simultaneous time moments.

Figure 3 10 Hz Input

Figure 4 30 Hz input

Experiment 3 Poissonian 80% correlated spike trains

Figure 5 10 Hz input

Figure 6 30 Hz input

Experiment 4 Poissonian, totally uncorrelated spiketrains Simulations with normalised input of 10Hz and 30Hz, over 101 seconds

Figure 7.

10 Hz input

Figure 8.

30 Hz input

Experiment 5 Poissonian spiketrains with independent failure events Simulations with normalised input of 10Hz and 30Hz, over 101 seconds

Figure 9.

10 Hz input

Figure 10.

30 Hz input

Conclusions •  In the model used, change in resting membrane potential did not influence

output rates •  In some cases, failure rates influenced the accuracy of classification.

- In some cases higher failure rates increased classification accuracy (the effect of decorrelation)

Future Directions •  Running longer simulations over several runs for

reduction of classification errors

•  Implementing models with higher number of neurons as input (4,10,20,50,...)

•  Creating complex experiments that are more sensitive to different correlation levels between inputs

•  Measuring plasticity in different synapses after application of synaptic failure

•  Trying different classifiers

Thank You!

Discuss...