Neurogrid : A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations

IntroductionNeurogrid’s Design Choices

Neurogrid SystemNeurogrid Neuron

Data TransferComparison

Neurogrid: A Mixed-Analog-Digital Multichip System forLarge-Scale Neural Simulations

Group 7Ashlesha Patil

Nithin NethipudiTitto Thomas

EE 746 : Literature Review 1/29




Neurogrid

Neurogrid : Introduction

⇒ First hardware system with the capability of performing biological real-timesimulations of a million neurons and their synaptic connections.

⇒ It’s a system for simulating large-scale neural models in real time.

⇒ Four neural elements are axonal arbor, synapse, dendritic tree and soma are realizedby emulating their structure.

⇒ 3 Major design choices for such a system

Emulate the four neural elements with dedicated or shared electronic circuits

Implement these electronic circuits in an analog or digital manner

Interconnect arrays of these neurons with mesh or tree network





Neurogrid

Neurogrid : Features

⇒ Emulated all neural elements except soma with share electronic circuits to maximizethe number of synaptic connections.

⇒ All electronic circuits except axonal arbors were realized in an analog to maximizeenergy efficiency.

⇒ Interconnected neural arrays in a tree network to maximize throughput.





Hardware RealizationArchitectureRealizationsSpike routing networks

Analog or Digital

⇒ Axonal arbors, synapses, dendritic trees, and somas may be implemented in analogor digital.

⇒ Fully analog and fully digital implementations will be,






Four Architectures

⇒ Fully Dedicated : A fully connected network of N neurons requires N2 synapseelements

⇒ Shared Axon : Realized using Address event representation (AER), each neuron isassigned a unique address. Also requires N2 synapse elements for N neurons.

⇒ Shared Synapse : N electronic circuits are required to fully connect N neurons. Ituses RAM to realize axonal branching instead of dedicated wire.

⇒ Shared Dendrite : Also requires only N electronic circuits to fully connect Nneurons. Each shared-synapse circuit feeds its neighbouring neurons as well.






Four Architectures (Contd.)

⇒ (a) Fully dedicated, (b) Shared axon, (c) Shared synapse, (d) Shared dendrite






Realization Comparison

⇒ The hybrid realizations are all analog except for their axonal arbors.

⇒ Available choices are,

Fully dedicated analog (FDA)

Shared axon hybrid (SAH)

Shared axon digital (SAD)

Shared synapse hybrid (SSH)

Shared dendrite hybrid (SDH)

⇒ FDA is faster than real time and have arbitrary connectivity, while SDH has lots ofneurons with mostly local connectivity.

⇒ FDA achieves its cost-effectiveness by minimizing T, while SDH does it byminimizing A.






Realization Comparison (Contd.)






RAM Costs






Spike routing networks

⇒ Meshes and trees have been explored for routing spikes between neural-elementarrays.

⇒ Meshes offer high bandwidth, but have long latency.

⇒ Trees offer short latency, but have low bandwidth

⇒ Unlike meshes, however, trees support deadlock-free multicast communication,enabling them to utilize their limited bandwidth efficiently, and thereby maximizethroughput.

⇒ Neurogrid targets can use multicast to realize secondary axon-branching, so theychose tree.





The system

Neurogrid System

⇒ Two main components

Software to perform interactive visualization

Hardware to perform real-time simulation

⇒ Neurocore packet is a sequence of 12-b words that specify a route, an address, anarbitrarily long payload, and a tailword. They are mapped on to Neurocores overUSB using driver programs.

⇒ A Neurocore has a 256 × 256 silicon-neuron array, a transmitter, a receiver, arouter, and two RAMs. A neuron has a soma, a dendrite, four gating-variable andfour synapse-population (i.e.,shared synapse and dendrite) circuits.





The system

Neurogrid System (Contd.)





The system

Neurocore

⇒ RAM0 provides 256 locations for target synapse types (or no connection). RAM1

stores 18 configuration bits and 61 analog biases, common to all the Neurocoressilicon neurons.

⇒ DACs produce the analog biases. RstMB provides five resets and generates DACsreference current. ADCs digitize four analog signals from a selected neuron.

⇒ Ti0−2, To0−2, Li0−2, L00−2, Ri0−2, and Ro0−2 communicate with parent or eitherchild.

⇒ The 12 × 14 mm 2 die, with 23 M transistors and 180 pads, fabricated in a 180nmcomplementary CMOS process.





Dimensionless ModelsSoma and DendriteSynapse and Ion channel populationCircuit RealizationCircuit Realization

Passive membrane model

⇒ Models composed of conductors, capacitors, voltage, and current sources may beconverted to dimensionless form.

⇒ A passive membrane model is given by,

CV = −Gleak(V − Eleak) + Iin

⇒ Change the reference voltage to Eleak and normalize with GleakVn to give

τ v = −v + u






Soma model

⇒ Soma model is given by,

τs vs = −vs + isin +1

2v 2s − gkvs − gresvspres(t) + vd

⇒ τs - membrane time constant, isin - input current, vd - dendritic input.

⇒ Quadratic positive feedback 12v 2s models the spike-generating sodium current

⇒ The reset conductance gres , active for the duration tres of a unit-amplitude pulse pres ,models the refractory period

⇒ High-threshold potassium conductance gK models spike-frequency adaptation.

τK gK = −gK + gK∞pres(t)

τK - decay time constant, gK∞ - saturation value






Dendrite model

⇒ Dendrite model is given by,

τd vd = −vd + idin − ibppres(t) + gch(ech − vd)

⇒ vd - membrane time constant, idin - input current, ibp - back propagating input, andgch - channel populations conductance, ech - reversal potential

⇒ Soma and dendrite could receive synaptic inputs.






Synapse population

⇒ Soma model is given by,

τsyngsyn = −gsyn + gsatprise(t)

⇒ τsyn - synaptic time constant, gsat - saturation conductance for the population

⇒ The unit-amplitude pulse prise(t) is triggered by an input spike; its width trise modelsthe duration for which neurotransmitter is available in the cleft.

⇒ gsyn decays spatially in the shared dendritic tree and provides an input current to thesoma or dendrite.






Ion channel population

⇒ Ion-channel populations conductance gch is obtained by scaling a maximumconductance gmax with a gating variable c ( i.e, gch = cgmax )

⇒ c is modelled as,τgv c = −c + css

css - steady-state activation or inactivation, τgv - its time constant

⇒ css is given by,

css =α

α + βor

β

α + β

α, β - model a channels opening and closing rates






Log domain realization

⇒ Subthreshold PMOS current is given by,

Id = ΛI0eκvgb−vsb

VT (1− evdsVT )

⇒ For vds < −4VT and Vsb = 0

Id = ΛI0eκvgbVT

⇒ Taking natural logarithm on both sides

lnId − lnI0 − lnΛ =κvgbVT

⇒ The final equation will beIdId

= −κvgbVT






Passive membranes circuit

⇒ Im → membrane potential, Ilk → membrane’s leak, Iback → membrane’s input.

CVm = Ilk − Iback

Vleak + Vin = Vback + Vm ⇒IleakΛ1

IinΛ2

=IbackΛ4

ImΛ5






Soma circuit

⇒ Circuit realization operates according to,

⇒ The high-threshold potassium conductances circuit realization operates according to






Circuit realizations

⇒ Dendrite circuit,

⇒ Soma circuit,






Other realizations

⇒ Dendrite,

⇒ Synapse population,

⇒ Ion-channel population,





Transmitter and ReceiverRouter

Transmitter and Receiver

⇒ AER : Realize multiple virtual point-to-point channels with a single physical link byrepresenting a communication on any channel with its input-ports address.

⇒ Encodes the spikes row and column addresses, conveying these addresses to anotherchip, and decoding them to recreate the spike at its destination.

⇒ Two M-way arbiters, built with M-1 two-way arbiters connected in a binary treegenerate a log2 (M)-bit address for each row or column selected.

⇒ Latches allow the next packets addresses to be decoded while the current onesspikes are being delivered to the array.






Transmitter and Receiver (Contd.)






Transmitter and Receiver (Contd.)

⇒ 256 × 256 version of the transmitter and a 2048 × 256 version of the receiver - itseight lines per row select one of four shared-synapses to activate, one of three setsof analog signals to sample, or a neuron to disable

⇒ 86 ns is needed to transfer a rows spikes to the arrays periphery and 23 ns to encodeeach spikes column address.

⇒ Pipelining and parallelism enable the transmitter to sustain a maximum transmissionrate of 43.4 Mspike/s, or 663 spike/s per neuron. receiver decodes an additionalcolumn address every 16 ns, sustaining a maximum rate of 62.5 Mspike/s, or 956spike/s per neuron






Router

⇒ Realizes the primary and secondary axon branching using the multicast capabilityand the Neurocores embedded memory.

⇒ Two distinct phases: a point-to-point phase and a branching phase.

⇒ Point-to-point phase : the router steers the packet up/down or left/right, based ona bit in the packets first word. Branching phase : the router copies the packet toboth left and right ports.

⇒ If flood bit (F) set, the packet recursively branches to all descendants (flood mode),Otherwise, delivered to exclusively (target mode).

⇒ In F mode Depending on how the SRAM is programmed, the packet is either filteredor delivered. If the packet is delivered, current system appends two additional bitsretrieved from the SRAM to its row address.

⇒ The obtained rates were 234 Mspike/s in normal mode and up to 1.17 Gspike/s inburst mode.






Router (Contd.)





Energy and it’s comparison

Energy consumption it’s comparison

⇒ The energy expended to activate a silicon axons synapses is given below. Dividing itwith total connecting neurons ( ncol × np × nsyn )

⇒ Comparing the energy consumption with the other networks.


Neurogrid : A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations

Engineering

Transcript of Neurogrid : A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations