Basics of Neuromorphic Design

Sylvain Saïghi

IMS Lab – University of Bordeaux

2

Outline

Integrated Circuit Design

Models for neuromorphic implementation

Main types of neuromorphic systems

GDR BioComp – Colloque 2015 – St Paul de Vence

3

How to design an Integrated Circuit? IC technology providers for prototyping and low volume

production

Circuits Multi-Projets http://cmp.imag.fr/

Europractice http://www.europractice-ic.com/

Same kind of providers in North America

GDR BioComp – Colloque 2015 – St Paul de Vence

4

How to design an Integrated Circuit? Schedule

GDR BioComp – Colloque 2015 – St Paul de Vence

5

How to design an Integrated Circuit? Schedule and Prices…

GDR BioComp – Colloque 2015 – St Paul de Vence

6

How to design an Integrated Circuit? Second step: the schematic

Design

GDR BioComp – Colloque 2015 – St Paul de Vence

7

How to design an Integrated Circuit? Second step: the schematic

Design

Simulation

Monte Carlo to

check the reliability

against the parameters

mismatch

GDR BioComp – Colloque 2015 – St Paul de Vence

8

How to design an Integrated Circuit? Third step: the layout

Mask drawings are sent

to the IC provider

The provider gathers all

chips from different

designers

GDR BioComp – Colloque 2015 – St Paul de Vence

9

How to design an Integrated Circuit? Third step: the layout

Mask drawings are sent

to the IC provider

The provider gathers all

chips from different

designers

You will receive 25 samples

3 or 4 months later

Total duration about 12-18 months

GDR BioComp – Colloque 2015 – St Paul de Vence

10

Another solution for digital design VHDL: Very High-Level Design Language

Descriptive language Behavioral simulation

Choice of target• IC (Founder library)• FPGA (Field

Programmable

Gate Array) Physical simulations

GDR BioComp – Colloque 2015 – St Paul de Vence

11

S o m a

A x o n

N o d e o f R an v ie r

M y e lin sh ea th

P o st-sy n ap ticce ll

P re -sy n ap ticce ll

S y n ap tic te rm in a lsA neuron for neuromorphic designers

GDR BioComp – Colloque 2015 – St Paul de Vence

A few figures Soma diameter ~ 4 to 100 mm Axonal time propagation ~ 1 to 30ms Membrane voltage from ~ -120mV to 50mV Ionic currents ~ 10 nA

12

A neuron for neuromorphic designers

GDR BioComp – Colloque 2015 – St Paul de Vence

Spiking Neuron → time dependent

In tegrator Thresholddetector

Spikegenerator

SYNAPTICCURRENTS

13

Hebbian rule: Spike Timing Dependent Plasticity (STDP)

Learning in Neuromorphic Systems

Nj

Ni

Presynaptic neuron

Postsynaptic neuron

Wji

Causality

Increasing of Synaptic Weight

(potentiation LTP)

No causality

Decreasing of Synaptic Weight (depression LTD)

14

Threshold = 2

1/3

1/3

1/3

1/3

1/3

1/3

15

2/3

Threshold = 2

2/3

2/3

2/3

2/3

2/3

16

1

Threshold = 2

1

1

1/3

1/3

1/3

17

1

Threshold = 2

1

2/3

0

0

0

18

1

Threshold = 2

1

0

0

0

1/3

19

1

Threshold = 2

1

0

0

0

0

20

First neuromorphic design Carver Mead – Caltech

The term neuromorphic was coined by Carver Mead, in the late 1980s to describe Very-Large-Scale Integration (VLSI) systems containing electronic circuits that emulate the bio-physics of neural systems using the physics of silicon.

GDR BioComp – Colloque 2015 – St Paul de Vence

1989

21

2 main types of neuromorphic design

GDR BioComp – Colloque 2015 – St Paul de Vence

DigitalAnalog

Neuromimetic Rebuild, understand the life

Biological Real Time (99% of case)

Could be connected to life cells

Bio-Inspired Engineering issues

Event based computation

?

?

?

?

22

Neuromimetic designNeuroGrid (Standford)

Simulate the brain

in real-time Specifications:

sixteen Neurocores

each of which has

256 x 256 silicon neurons in an 11.9 mm x 13.9 mm chip Analog core and Digital communication bus

GDR BioComp – Colloque 2015 – St Paul de Vence

“Neurogrid simulates a million neurons connectedby billions of synapses in real-time, rivaling a supercomputer while consuming a 100,000 times less energy—five watts instead of a megawatt!”

23

Neuromimetic designSpiNNaker (Manchester University) – Human Brain Project

Simulate the brain in real time

Specifications a million-core computing

engine massively-parallel fully digital design

GDR BioComp – Colloque 2015 – St Paul de Vence

24

Neuromimetic designHeidelberg’s system (Heidelberg University) – Human Brain Project

Simulate the brain 104 faster than real-time

Specifications Based around wafer-scale VLSI Analog core and Digital

communication bus Each 20-cm-diameter silicon wafer

 contains 384 chips, each of which

implements 128,000 synapses and up to 512 spiking neurons.

Total of around 200,000 neurons and 49 million synapses per wafer.

GDR BioComp – Colloque 2015 – St Paul de Vence

25

Bio-Inspired design TrueNorth – IBM

Video treatment

Specifications Fully digital

GDR BioComp – Colloque 2015 – St Paul de Vence

26

Bio-Inspired design TrueNorth – IBM

TrueNorth video

GDR BioComp – Colloque 2015 – St Paul de Vence

27

Bio-Inspired design Zeroth – Qualcomm

Neural Processing Units

which learn your abs

Specifications Fully digital

GDR BioComp – Colloque 2015 – St Paul de Vence

28GDR BioComp – Colloque 2015 – St Paul de Vence

Qualcomm video

29

Other neuromorphic research groups(non-exhaustive)

Europe• Institute of Neuroinformatics (Zurich)• Instituto Microelectronica (Sevilla)• University of Bordeaux• …

North America• Georgia Tech (Atlanta)• Johns Hopkins University (Baltimore)• University of California (San Diego)• …

Asia• University of New South Wales (Sydney)• …

GDR BioComp – Colloque 2015 – St Paul de Vence

Thanks for your attention