Modélisation en neurosciences - et ailleurs Modelling in neuroscience – and elsewhere

http://www.lps.ens.fr/~nadal/Cours/MVA

Jean-Pierre Nadal CNRS & EHESS

Laboratoire de Physique de l’ENS (LPENS, UMR 8023 CNRS - ENS – SU – Univ. Paris Diderot)

Ecole Normale Supérieure (ENS)

&

Centre d’Analyse et de Mathématique Sociales (CAMS, UMR 8557 CNRS - EHESS)

Ecole des Hautes Etudes en Sciences Sociales (EHESS)

nadal@lps.ens.fr

Modélisation en neurosciences - et ailleurs Modelling in neuroscience – and elsewhere

Course 20 hours + exam – Thursdays morning, ENS Cachan – Cournot C102

First Class: next week, Thursday January 17, 9am-11am (2 hours) Then, from January 24 to March 7 (no class on February 7) from 9 am to 12:20 (3 hours with a 20mn break in the middle) Language: French and/or English depending on the students Validation: + regular attendence + written report [critical reading of an article + micro-project] + oral [presentation + questions on the course]

(date to be chosen with the students, between March 14 and April 18)

http://www.lps.ens.fr/~nadal/Cours/MVA

Modélisation en neurosciences - et ailleurs Modelling in neuroscience – and elsewhere

This Course is an introduction to the modelling of learning and adaptation (mainly) in natual and (secondarily) in artificial systems, making use of tools from statistical physics, (Bayesian) inference and information theory. Most of the course is about computational neuroscience, but with openings to other topics, notably complex systems in social sciences. The course is an opportunity to show links between different domains and between different disciplines.

Different types of learning Associative memory learning by heart – learning associations

Learning from examples generalization – statistical inference supervised learning Coding – building a neural representation Data analysis – clustering cortical maps unupervised learning Behavioral learning motor control reinforcement learning

© R

émi M

unos

W

illia

m o

f Ock

ham

12

80 -

1349

Different types of learning Associative memory learning by heart – learning associations

Learning from examples generalization – statistical inference supervised learning Coding – building a neural representation Data analysis – clustering cortical maps unupervised learning Behavioral learning motor control reinforcement learning

© R

émi M

unos

W

illia

m o

f Ock

ham

12

80 -

1349

Marcel Proust, Du côté de chez Swann, Grasset, 1913

“Involuntary Memory”: In Proust's In Search of Lost Time (Volume One: Swann's Way), the narrator eats a Madeleine cookie soaked in a decoction of lime-blossom, and recalls a childhood memory in which he ate similar cookies.

Associative memory

Charles Baudelaire

Les paradis artificiels 1860

Artificial Paradises “What is the human brain, if not an immense and natural palimpsest? My brain is a palimpsest, and yours too, reader. Innumerable layers of ideas, images, and, sentiments fall upon your brain, as softly as light. It seems that each [new layer] buries the previous one. But no layer has perished.

Palimpsest: In ancient and medieval times, parchments were often scraped down to enable new text to be written over the old. This overlaying technique of palimpsest masked the original texts but never truly effaced them.

Associative memory

Archimedes Palimpsest Thirteenth-century manuscript, overwritten with prayer book

Storage capacity

Long term memory vs. Short-term memory

The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information George A. Miller The Psychological Review, 1956, vol. 63, pp. 81-97

http://cogprints.org/730/1/miller.html

My problem is that I have been persecuted by an integer. (...) This number assumes a variety of disguises, being sometimes a little larger and sometimes a little smaller than usual, but never changing so much as to be unrecognizable. (…) There is, to quote a famous senator, a design behind it, some pattern governing its appearances. Either there really is something unusual about the number or else I am suffering from delusions of persecution.

Short-term memory Working memory

Memory span is around seven for digits, six for letters, and five for words.

Experiments with monkeys • cue: visual stimulus • delay (cue is no longer on the screen during the delay period) • go signal (action specific to the cue)

Delay Match to Sample (DMS) experiments

Y. Miyashita - Nature, 1988

correct

error

stimulus: same or different ? stimulus (sample)

Associative memory Neural correlates: Recurrent neural networks

Tools: Dynamical systems, Markov process Statistical physics Collective phenomena Ergodicity breaking

Related topics: Maths: measure concentration Physics of disorders systems: Spin Glasses Optimization: K-satisfiability, compressed sensing Economics, social sciences: choice under social influence

Synaptic plasticity (Hebbian plasticity) « cells that fire together wire together » Attractor neural networks

Y. Miyashita - Nature, 1988

selectivity in the presence of noise large number of interacting cells

Amit & Mongillo, Cerebral cortex 2003 “Selective Delay Activity in the Cortex: Phenomena and Interpretation” (review)

Different types of learning Associative memory learning by heart – learning associations

Learning from examples generalization – statistical inference supervised learning Coding – building a neural representation Data analysis – clustering cortical maps unupervised learning Behavioral learning motor control reinforcement learning

© R

émi M

unos

W

illia

m o

f Ock

ham

12

80 -

1349

Generalization Learning a rule from examples

• x(n) = x(n-1) + x(n-2) x(5) = 13

• x(n) = x(n-1) + n-1 x(5) = 12 • and why not x(5) = π =3.1415… ?

+ X

●

Generalization

●

+ X

Best guess? Several theoretical approaches:

Statistical (Bayesian) inference ( most probable rule)

Complexity: Minimum description length ( simplest rule that matches the examples)

(Occam’s razor)

x(1) = 2 x(2) =3 x(3) = 5 x(4) = 8 x(5) = ?

From the biography of A.Turing by Andrew Hodges, « Alan Turing: The Enigma », 1983

(« On computable numbers », 1936) Alan Turing

His high-pitched voice already stood out above the general murmur of well-behaved junior executives grooming themselves for promotion within the Bell corporation. Then he was suddenly heard to say: "No,I'm not interested in developing a powerful brain. All I'm after is a mediocre brain, something like the President of American Telephone & Telegraph Company". The room was paralysed while Alan nonchalantly continued to explain how he imagined feeding in facts on prices of commodities and stocks and asking the machine the question "Do I buy or sell"?

1943, visiting the Bell Laboratories of AT&T – while at the cafetaria:

Generalization The main supervised learning paradigm

Frank Rosenblatt (1928-1971)

• « The perceptron: a probabilistic model for information storage and organization in the brain », Psychological Review, Vol. 65:6 (1958)

• « Principles of neurodynamics », New York: Spartan (1962)

The Perceptron

On Rosenblatt see http://csis.pace.edu/~ctappert/srd2011/rosenblatt-contributions.htm

July 08, 1958

Rosenblatt and the Mark-I Perceptron Source: Arvin Calspan Advanced

Technology Center

The supervised learning paradigm The Perceptron

Feedforward processing – simple and multilayer perceptrons Machine learning beyond the simple perceptron, multi-layer perceptrons : SVM, Boltzmann machines, Deep learning Statistical inference, Learning theory(ies) Neuroscience: Willshaw et al, 69: simple Hebbian learning, sparse coding limit Marr 68, Albus 71: The Cerebellum as a Perceptron

Learning associations Neural correlates Feedforward architectures

Parallel Fibers (PF): inputs to the Purkinje Cells (PC) – plasticity of the synapses PF/PC Output: axons of the PCs Climbing fibers: bring an error signal to the PCs Marr (1968) and Albus (1971): Purkinje cells as perceptrons

Figu

re fr

om

http

s://g

rey.

colo

rado

.edu

/Com

pCog

Neu

ro/in

dex.

php/

CC

NBo

ok/M

otor

Multi-layer perceptrons: Deep learning

Figure from Bengio & LeCun, in Large-Scale Kernel Machines, Bottou et al Ed., MIT Press 2007

• Prior knowledge specific architecture

• Supervised gradient descent • Very large training set

Hinton, G. E., Osindero, S. and Teh, Y. (2006) http://www.deeplearning.net/tutorial/ Approach further developped by Hinton, Bengio, LeCun and others

Last ~5 years, applications to: brain image analysis – Course Théo Papadopoulos, Maureen Clerc & Bertrand Thirion language acquisition – Course Emmanuel Dupoux & Benoît Sagot

Different types of learning Associative memory learning by heart – learning associations

Learning from examples generalization – statistical inference supervised learning Coding – building a neural representation Data analysis – clustering cortical maps unupervised learning Behavioral learning motor control reinforcement learning

© R

émi M

unos

W

illia

m o

f Ock

ham

12

80 -

1349

Cortical maps

Development and adaptation of cortical maps unsupervised learning, « self-organization » Epigenetic development Use-dependent adaptation

Related topics: • Principal Component Analysis (PCA) • Independent Component Analysis (ICA) • Clustering • ‘Natural images’ analysis

Neural coding

Unsupervised learning

Homonculus (here: motor cortex) Source: http://physiol.gu.se/maberg/images.html

Ecological approach to sensory coding: efficient adaptation to the natural environment

Horace Barlow , 1961 H. B. Barlow. Possible principles underlying the transformation of sensory messages. Sensory Communication, pp. 217-234, 1961 efficient coding hypothesis sensory processing in the brain should be adapted to natural stimuli e.g.: Neurons in the visual (or auditory) system of a given animal should be optimized for coding images (or sounds) representative of those found in the natural environment of that animal.

Filters optimized for coding natural images lead to filters which resemble the receptive fields of simple-cells in V1. In the auditory domain, optimizing a network for coding natural sounds leads to filters which resemble the impulse response of cochlear filters found in the inner ear.

Formalization: tools from Information Theory, Statistical (Bayesian) inference, parameter estimation

Neural coding

Histogram equalization

Fly visual system Laughlin 1981

Moth (Antheraea polyphemus) olfactory system Kostal et al 2008

Efficient coding

Single cell adaptation

Large populations of cells involved in the representation of a given set of stimuli, objects, motor commands, …

Selectivity: tuning curves

Neural coding

Population coding

Primary visual cortex (V1) orientation selectivity

Head direction cells active only when the animal's head points in a specific direction within an environment.

Hubel and W

iesel, 1968 H

enry et al 1974; Wandell 1995 Head direction (°) Ta

ube,

Mul

ler,

Ran

ck 1

990

Figu

re: J

. Tau

be

Firi

ng ra

te (H

z)

population coding of categories categorical perception perceptual decision making identification, discrimination, reaction times

cat or

dog?

population coding of a continuous stimulus, e.g., an orientation link between Shannon information and Fisher information

Neural coding

Population coding

Perception, coding and decision making

Shadlen & Newsome 1996; Gold Shadlen 2001; Beck et al 2008; Churchland et al 2011;… https://shadlenlab.columbia.edu/

Freedman et al

Formalization & modeling: • Information Theory, Statistical (Bayesian) inference Ideal observer; Accumulation of evidence • Biophysical models: attractor networks

cat? dog?

Different types of learning Associative memory learning by heart – learning associations

Learning from examples generalization – statistical inference supervised learning Coding – building a neural representation Data analysis – clustering cortical maps unupervised learning Behavioral learning motor control reinforcement learning

© R

émi M

unos

W

illia

m o

f Ock

ham

12

80 -

1349

exple: the maze task average reward at this location

P. Dayan, Reinforcement learning, in: Steven's Handbook of Experimental Psychology,

Wiley 2001

temporal credit assignment problem:

entry → A; left → B; right → small reward:

how to decide if the error was to turn left at A, or to turn right at B? exploration-exploitation dilemma Theory: Reinforcement learning

R.S. Sutton & A.G. Barto, Psychological Review 1981; MIT Press 1998

Behavioral learning

MVA Course, 1st semester Alessandro Lazaric “Reinforcement learning”

Categorization Memorization vs. Coding vs. Making a decision

Through the Looking-Glass, and What Alice Found There - Lewis Carroll 1871

cat? dog?

Freedman et al

• Learning a class from examples generalization, supervised learning • Clustering – unsupervised learning • Behavioral Learning

prosopagnosia

’Good-bye, till we meet again!’ she said as cheerfully as she could.

’I shouldn’t know you again if we DID meet,’ Humpty Dumpty replied in a discontented tone, giving her one of his fingers to shake; ’you’re so exactly like other people.’

’The face is what one goes by, generally,’ Alice remarked in a thoughtful tone.

’That’s just what I complain of,’ said Humpty Dumpty. ’Your face is the same as everybody has–the two eyes, so–’ (marking their places in the air with this thumb) ’nose in the middle, mouth under. It’s always the same. Now if you had the two eyes on the same side of the nose, for instance–or the mouth at the top–that would be SOME help.’

Illus

tratio

n by

Joh

n Te

nnie

l, 18

71

Different types of learning Associative memory learning by heart Learning from examples generalization – statistical inference Coding – building a neural representation cortical maps Behavioral learning motor control supervised learning unupervised learning reinforcement learning

Synaptic plasticity

Plasticity

«Cells that fire together, wire together » (Carla Shatz, 1992)

Santiago Ramón y Cajal (1894) ____________________________ « l’exercice mental n’est pas capable d’améliorer l’organisation cérébrale en augmentant le nombre de cellules, mais plutôt en favorisant le développement de l’appareil dendritique et du système de collatérales axonales dans les régions cérébrales les plus utilisées. En ce sens, les associations déjà établies parmi certains groupes de cellules seraient significativement renforcées par la multiplication des petites branches terminales de l’arborisation dendritique et des collatérales axonales ; mais, de plus, des connexions intercellulaires totalement nouvelles pourraient être établies, grâce à la formation de nouvelles collatérales axonales et de nouvelles dendrites. »

Synaptic plasticity LTP LTD STDP…

more here

Plasticity

Hebbian plasticity 𝑊𝑖𝑖 = synaptic weight from j (« pre-synaptic » neuron) to i (« post-synaptic » neuron) « pure » Hebb: ∆ 𝑊𝑖𝑖 = ε 𝑟𝑖 𝑟𝑖 Covariance rule (Sejnowski & Tesauro 1989): ∆ 𝑊𝑖𝑖 = ε (𝑟𝑖 − < 𝑟𝑖>) (𝑟𝑖− < 𝑟𝑖>) BCM rule (Bienenstock, Cooper, Munroe 1982):

pure Hebb rule covariance rule

BCM rule

∆ 𝑊𝑖𝑖

𝑟𝑖 (for a given high activity 𝑟𝑖 of cell j)

𝑣𝜃 = 𝑓(< 𝑟𝑖 >)

Hebbian learning/adaptation

SYNAPTIC LEVEL pre-synaptic activity post-synaptic activity

Unsupervised Hebbian learning

Neuron or network level

Unsupervised scheme Supervised scheme external input (e.g. sensory input) clamped pre- and post- synaptic activities Hebbian learning

+ Hebbian learning network activity sensory coding/adaptation learning associations

Cerebellum: motor control/ supervised learning

Menu

From unsupervised learning to neural coding The Oja model Neural coding & decoding: PCA and infomax (linear case) optimization of the transfer function infomax and ICA Spiking neurons: Poisson process Population coding Categorical perception Perceptual decision making Supervised learning: feedfoward architectures

The Willshaw model The Perceptron

Storage capacity & learning algorithms The Purkinje cell as a Perceptron

Associative memory: recurrent networks The Hopfield model and variants Random Markov Fields Ergodicity Breaking Working memory Reinforcement learning Behavioral learning and the dopaminergic system

Modélisation en neurosciences - et ailleurs Modelling in neuroscience – and elsewhere

Course 20 hours + exam – Thursdays morning, ENS Cachan – Cournot C102

First Class: today, Thursday January 17, 9am-11am (2 hours) Then, from January 24 to March 7 (no class on February 7) from 9 am to 12:20 (3 hours with a 20mn break in the middle) Language: French and/or English depending on the students Validation: + regular attendence + written report [critical reading of an article + micro-project] + oral [presentation + questions on the course]

(date to be chosen with the students, between March 14 and April 18)

http://www.lps.ens.fr/~nadal/Cours/MVA