Josephson Junction Based Neuromorphic Computing

Mike Schneider, Stephen Russek, Christine Donnelly, Burm Baek, Matt Pufall, Ian Haygood, Pete Hopkins, Paul Dresselhaus, Sam Benz, Bill Rippard

12/14/2017 2

Artificial Intelligence

“Intelligence” exhibited by machines

Machine Learning

Field of study that gives computers the ability to learn

without being explicitly programmed

Bio – Inspired

Drawing inspiration from the human brain

Neuromorphic Hardware

Deep Neural Networks

eyeriss.mit.edu

12/14/2017 3

Software neural networks are mainstream

• Apple: A11 bionic SoC, Siri

• Google: Search results (RankBrain), translate, street view, …

• Microsoft: Computational Network Toolkit (speech recognition)

• Amazon: from product recommendation to robotic picking routines

• Facebook Ranking, Language translation, …

M. Nielsen, “Neural Networks and Deep Learning” (2015)

12/14/2017 4

Neural networks dominate modern image recognition

2010 2011 2012 2013 2014 2015 2016 20170

5

10

15

20

25

30

Cla

ssific

atio

n E

rro

r (%

)

Year

Deep Convolutional Neural Network (Trained with GPUs)

Human Accuracy ~ 5% error rate

Feature based algorithms

ImageNet Competition

12/14/2017 5

What interaction is being modeled in neural networks?

Wikimedia neuronimage

x0

axon

w0

synapse

w0 x0

x1w1

Cell body

w1 x1

𝑓

𝑖

𝑤𝑖𝑥𝑖 + 𝑏output

…

Neuromorphic Single Flux Quantum

6

Information transfer: quantized pulse trains

Long distance “lossless” pulse transmission

3D architecture

Memory/ plasticity

Neural Single Flux Quantum

axons Josephson/passive transmission lines

Magnetic JJsynapse

• Barrier = insulator, semiconductor, normal metal, ferromagnet, …

7

Josephson junction (JJ)

sincs II

1

101

ie 2202

ie

𝜑0

2𝜋𝐶 ሷ𝜃 +

𝜑0

2𝜋𝑅𝑛

ሶ𝜃 + 𝐼𝑐𝑠𝑖𝑛 𝜃 − 𝐼𝑏 = 0

Ib

Circuit Model of a Josephson Junction

8

𝑈 =𝐼𝑐𝜙02𝜋

(−cos 𝜃 −𝐼𝑏𝐼𝑐𝜃)DampingMass

Vse

h 150 1003.2

2 where

Interaction of order parameters

9

S F

Phase Progression

Exchange Field

Ph

ase

FM Thickness|C

riti

cal C

urr

ent

Den

sity

|

Jc > 0 Jc < 0(-JJ)

Jc > 0

S-F-S

– eiQ∙R – e-iQ∙R

= ( – ) cos(Q∙R)+ i( + ) sin(Q∙R)

Singlet(S = 0)

Singlet + Triplet(S = 0) (S = 1)

Superconducting Spintronics Reviews

Buzdin, Rev Mod Phys 2005Bergeret, Rev Mod Phys 2005Eschrig, Phys Today 2011Blamire, J of Phys Cond Matt 2014Linder and Robinson, Nature Phys 2015

Fulde-Ferrell-Larkin-Ovchinnikov(FFLO) state

S

Magnetic nanoclusters in a Josephson junction

10

Thin Films

Si

Mn

0 100 200 300

0

30

60

90 10 mT

5 mT

0.5 mT

0 mT

Mom

ent (n

Am

2)

Temperature (K)

ordered

disordered

Nb

Nb

Changing magnetic order with fast electrical pulses in devices

12/14/2017 11

H

H

Small applied magnetic field

250 psElectrical pulse

Only the small applied magnetic field

250 psElectrical pulse

+

H=0

The basic neuro-inspired cell

12

Magnetic Josephson junction neural network simulation

12/14/2017 13

Pre-synaptic Neurons

Synaptic Weights

Post-synaptic Neurons

• 9 input pixel (JJs)• 27 weights (MJJs)• 3 outputs (MJJ SQUIDs)

Real time (~ 3ns) recognition

14

100111001

Prezioso et al. Nature 2015

z

v

n

Real time (~ 3ns) recognition

15

100111001

0 20 40 60 80 100

0

20

40

60

80

100

120

Time (ns)

Offse

t V

olta

ge

(

V)

Prezioso et al. Nature 2015

Real time (~ 3ns) recognition

16

000111001

0 20 40 60 80 100

0

20

40

60

80

100

120

Time (ns)

Offse

t V

olta

ge

(

V)

Prezioso et al. Nature 2015

Large scale image recognition

• ARGUS

• 1.8 billion pixels at 12 fps

• resolution of 6 inches over 10 square miles

12/14/2017 17

• Currently pre-processed on board using kilowatts

• Full stream processed on a supercomputer

Scale to a large network

12/14/2017 18

• Recent ImageNet winners use ~ 1010 MA-operations / image

• ~ 1017 MA-operations /sec to process ARGUS real time

• JJ neural network would take ~ 3 watt (including cooling)

( 1 spike / MA-operation)

Benchmarks

12/14/2017 19

• Energy• Operational energy < 10-18

• Training energy < 10 -17

• Cooling overhead ~ 103

• (Human brain ~ 10-15)

• Speed• Operational (device) ~ 100 GHz• Operational (circuit) ~ 10 GHz

• Size • Demonstrated 1.5 x 3 µm• Demonstrated for MJJ 100 nm

• Scalability• Demonstrated ~ 101

• There is a lot of scaling to go…

Conclusions

12/14/2017 20

• Josephson junctions are a very promising neuromorphic computing technology

• Magnetic Josephson junctions are a cryogenic non-volatile memory

• Magnetic Josephson junctions can be used for the synaptic function

• There is a lot of scaling needed for Josephson junction based neuromorphic computing