Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

34
Whole Brain Simulations and the Discrepancy / Similarity between Artificial & Natural Neural Networks 1 st Deep Learning Club Seminar Tuesday , 11th October 2016 Guillaume Dumas , Human Genetics & Cognitive Functions

Transcript of Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Page 1: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Whole Bra in S imulat ions and the Discrepanc y/Similar i ty betweenAr t i f ic ia l & Natural Neural Networks

1st Deep Learning Club SeminarTuesday, 11th October 2016

Guillaume Dumas, Human Genetics & Cognitive Functions

Page 2: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Introduction

2 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 3: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

3 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

GoogLeNet, a 22 layers deep network

Page 4: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

4 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

“It’s not a human move.I’ve never seen a human play this move.

So beautiful.”Fan Hui, Go European champion

Page 5: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

5 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

IBM Neuromorphic Computer TrueNorth

DARPA SyNAPSE Program Plan

Page 6: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

6 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 7: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

7 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

“a nerve cell is more than a single basic active organ (…) Thus, all the complexities

referred to here may be irrelevant, but they may also endow the system with a analog

character, or with a ”mixed” character.”Von Neumann (1958)

The Computer & the Brain

Page 8: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

8 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

2 main differences:

Structure : redundancy

Dynamics : Evolution vs. Design

Page 9: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

9 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Felleman & Van Essen (1991) asimovinstitute.org/neural-network-zoo/

. . .

Page 10: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Part 1

10 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 11: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

11 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Plasticity — Going beyond backpropagation

Connectivity — From weight sharing to recurrent networks

Astrocytes — Managing multiple time scales

Body — Convenient to get its own training set!

Oscillations — Time, attention, & subthreshold computing

. . .

Page 12: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Izhikevich & Edelman, PNAS 200812 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 13: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

13 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 14: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

14 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

“The dirty secret is that we don’t even understand the nematode C. Elegans, which only has 302 neurons”

Christof Koch, Allen Brain Institute Chief Scientific Officer

“There is a lot of benefits for each neuroscientist because we have now a new Atlas, we can use supercomputers, we can

proof our models, a Neurorobotics Platform, have new simulation tools and so on.”

Katrin Amunts, JULICH SP2 Leader

Page 15: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

15 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 16: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

16 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Unsupervised Learning of Visual Features through Spike TimingDependent Plasticity. Masquelier & Thorpe, PLoS Comp Biol 2007

Page 17: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Part 2

17 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 18: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

18 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 19: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

19 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Dumas et al., PLoS ONE 2010

Page 20: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Dumas et al., PLoS ONE 201220 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

+ x 2 =

Large-scale, anatomically detailed models of the brain allow to perform experiments

that are impossible (physically or ethically)

Page 21: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Dumas et al., PLoS ONE 201221 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Normal Shuffle

Page 22: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Dumas et al., PLoS ONE 201222 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

FFT

Brai

n Ar

ea S

igna

ls

Cortical Level Scalp Level

C in

tra

C in

traFF

TEE

G S

igna

ls

Page 23: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

23 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Dumas et al., PLoS ONE 2012

Real connectivity facilitate inter-brain synchronization

Residual synchronization

Information exchanged between the two virtual brains

Inte

r-br

ain

sync

hron

izat

ion

Page 24: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

24 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Kelso, Dumas, & Tognoli, Neural Networks 2013

Expe

rimen

tal

Com

puta

tiona

l

Page 25: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

25 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Dumas et al. « The Human Dynamic Clamp » PNAS 2014

Page 26: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

26 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Dumas et al. « The Human Dynamic Clamp » PNAS 2014

”The Turing test implies only that judges are unable to tell if an agent is a human or a machine, and as such says nothing about the genuineness of

the path toward that decision. Here, the Human Dynamic Clamp is a tool to test hypotheses and gain understanding about how humans interact with each other as well as with machines. In the HDC paradigm, exploration of the machine’s behavior may be viewed as an exploration of us as well.”

Page 27: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Conclusion

27 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 28: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

28 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

19361950

1952

Page 29: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

29 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

”Unless our methods can deal with a simple

processor, how could we expect it to work on our

own brain?”

Jonas & Kording 2016

Lesion method Spike trains recordings

Local field potential recordings

Page 30: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

30 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Top-Dow

n(Sem

ioticalView)

Botto

m-U

p(In

form

atio

nal V

iew

)

Source: lkm.fri.uni-lj.si

Page 31: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

31 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Top-Dow

n(Sem

ioticalView)

Theory

Hypothesis

Experiment

Data

Pattern

Model

Botto

m-U

p(In

form

atio

nal V

iew

)

Page 32: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

32 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Top-Dow

n(Sem

ioticalView)

Botto

m-U

p(In

form

atio

nal V

iew

)Models

Page 33: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

33 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

Page 34: Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks

Thanks for your [email protected] – Extrospection.eu – @introspection