Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic...
Transcript of Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic...
![Page 1: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/1.jpg)
Learning and Stability
![Page 2: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/2.jpg)
Learning and Memory
Ramón y Cajal, 19th century
![Page 3: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/3.jpg)
Learning and Memory
Ramón y Cajal, 19th centuryneuron
axon dendrite
synapse
![Page 4: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/4.jpg)
Learning and Memory
neuron synapse
Envi
ron
men
tal
Inp
ut
activeneuron
James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; …
![Page 5: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/5.jpg)
Learning and Memory
neuron synapse
Envi
ron
men
tal
Inp
ut Synaptic Plasticity:
If the neurons i and j are active together the synaptic efficiency wij increases.
Thus, the influence of neuron j on the firing of neuron i is enhanced.
i
j
wij
James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; …
![Page 6: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/6.jpg)
Learning and Memory
Synaptic Plasticity:If the neurons i and j are active together the synaptic efficiency wij increases.
Re
call
A later presented (partial) recall signal of the original learned input has to activate the same group of neurons.
Cell Assembly:A group of strongly connected neurons which tend to fire together can represent a memory item.
James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; …
![Page 7: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/7.jpg)
Learning and Memory
Synaptic Plasticity:If the neurons i and j are active together the synaptic efficiency wij increases.
Cell Assembly:A group of strongly connected neurons which tend to fire together can represent a memory item.
Synaptic Plasticity and Memory Hypothesis
James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; …
Synaptic Plasticity = Learning
![Page 8: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/8.jpg)
Synaptic Plasticity: Hebbian
Classical Hebbian plasticity:
u: inputv: outputω: weightμ : learning rate
SynapseNeuron BNeuron A
u vω
ω=μ u v
Long-TermPotentiation (LTP)
What are the long-term dynamics of Hebbian Plasticity?
![Page 9: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/9.jpg)
Learning and Stability
![Page 10: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/10.jpg)
Synaptic Plasticity: Hebbian
Classical Hebbian plasticity:
u: inputv: outputω: weightμ : learning rate
SynapseNeuron BNeuron A
u vω
The weights follow divergent dynamics
v=u ωlinear neuron model:
ω=μ u v
stability analysis
Long-TermPotentiation (LTP)
![Page 11: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/11.jpg)
Mechanisms of stabilization
Sliding threshold: = m vu (v - Q) m << 1dw
dt
= n (v2 - Q) n < 1dQ
dt
Subtractive normalization:
= m (vu – a v2w), a>0dw
dtMultiplicative normalization:
1
μ
𝑑𝒘
𝑑𝑡= 𝑣𝒖 −
𝑣 𝒏 ∙ 𝒖 𝒏
𝑁
![Page 12: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/12.jpg)
BCM- Rule
= m vu (v - Q) m << 1dw
dt
As such this rule is again unstable, but BCM introduces a sliding threshold
= n (v2 - Q) n < 1dQ
dt
Note the rate of threshold change n should be faster than then weight
changes (m), but slower than the presentation of the individual inputpatterns. This way the weight growth will be over-dampened relative to the (weight – induced) activity increase.
Sliding Threshold
![Page 13: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/13.jpg)
τ𝑤 ሶ𝑤 = 𝑢 ∙ 𝑣 (𝑣 − Θ)BCM rule:
τΘ ሶΘ = 𝐹(𝑣, Θ)Sliding Threshold Θ:
The sliding threshold directly changes synaptic plasticity by the self-organized adaptation of the plasticity-parameters.
LTP
LTD𝚯 𝑣
ሶ𝑤Increase activity 𝑣↑: Θ ↑
𝚯𝒗↑
Decrease activity 𝑣↓: Θ ↓𝚯𝒗↓
ሶ𝑤 = 0, if Θ = 𝑣
Abraham and Bear, 1996; Abraham, 2008
Sliding Threshold
![Page 14: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/14.jpg)
Kirkwood et al., 1996
light-deprived (less activity)
after two days control condition (more activity)
The threshold Θ shifts slowly according to neuronal activity (sensory stimulation).
𝜏𝛩 > 𝜏𝑤
Sliding Threshold
![Page 15: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/15.jpg)
τ𝑤 ሶ𝑤 = 𝑢 ∙ 𝑣 (𝑣 − Θ)BCM rule:
τΘ ሶΘ = 𝐹(𝑣, Θ)Sliding Threshold Θ:
𝝉𝜣 = 𝝉𝒘 𝝉𝜣 > 𝝉𝒘
Yger and Gilson, 2015
oscillations
oscillations are stronger than convergence
Sliding Threshold
![Page 16: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/16.jpg)
Mechanisms of stabilization
Sliding threshold: = m vu (v - Q) m << 1dw
dt
= n (v2 - Q) n < 1dQ
dt
Subtractive normalization:
= m (vu – a v2w), a>0dw
dtMultiplicative normalization:
biologicallyunrealistic
1
μ
𝑑𝒘
𝑑𝑡= 𝑣𝒖 −
𝑣 𝒏 ∙ 𝒖 𝒏
𝑁
![Page 17: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/17.jpg)
Subtractive normalization
Subtractive:
With N number of inputs and n a unit vector (all “1”). This yields that
n.u is just the sum over all inputs.
This normalization is rigidly apply at each learning step. It requires global information (info about ALL inputs), which is biologically unrealistic.
v <u>
1
μ
𝑑𝒘
𝑑𝑡= 𝑣𝒖 −
𝑣 𝒏 ∙ 𝒖 𝒏
𝑁
![Page 18: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/18.jpg)
Mechanisms of stabilization
Sliding threshold: = m vu (v - Q) m << 1dw
dt
= n (v2 - Q) n < 1dQ
dt
Subtractive normalization:
= m (vu – a v2w), a>0dw
dtMultiplicative normalization:
biologicallyunrealistic
biologicallyunrealistic
1
μ
𝑑𝒘
𝑑𝑡= 𝑣𝒖 −
𝑣 𝒏 ∙ 𝒖 𝒏
𝑁
![Page 19: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/19.jpg)
Multiplicative normalization:
= m (vu – a v2w), a>0dw
dtMultiplicative:
This normalization leads to an asymptotic convergence of |w|2 to 1/a.
It requires only local information (pre-, post-syn. activity and the local synaptic weight).
(Oja’s rule, 1982)
![Page 20: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/20.jpg)
Mechanisms of stabilization
Sliding threshold: = m vu (v - Q) m << 1dw
dt
= n (v2 - Q) n < 1dQ
dt
Subtractive normalization:
= m (vu – a v2w), a>0dw
dtMultiplicative normalization:
biologicallyunrealistic
biologicallyunrealistic
no biologicalevidence
Synaptic Scaling (Turrigiano et al., 1998) could be a candidate to stabilize plasticity
1
μ
𝑑𝒘
𝑑𝑡= 𝑣𝒖 −
𝑣 𝒏 ∙ 𝒖 𝒏
𝑁
![Page 21: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/21.jpg)
Synaptic Scaling
Long-lasting alterations of neuronal activity induces some changes of the basic properties.
Block neuronal activity
Increase neuronal act.
Firing rate homeostasis in cultured networks
ControlBlock or raise activity
(for 2 days) After washing out
After TTX
After Bicuculline
Response increased
Response decreased
Change of EPSP amplitudes
Turrigiano and Co-workers, 1998, 2004, 2008,…
![Page 22: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/22.jpg)
Synaptic Scaling
After TTX
After Bicuculline
Change of EPSP amplitudes
(aft
er w
ash
ing
ou
t)
~ Change of the synaptic weight
Increased activity
Turrigiano and Co-workers, 1998, 2004, 2008,…
![Page 23: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/23.jpg)
Synaptic Scaling
(aft
er w
ash
ing
ou
t)
Firi
ng
rate
[v]
Synaptic weight [w]
Increased activity
Turrigiano and Co-workers, 1998, 2004, 2008,…
![Page 24: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/24.jpg)
Synaptic Scaling
Firi
ng
rate
[v]
Synaptic weight [w]
ሶ𝑤 = γ (𝑣𝑇 − 𝑣)
𝑣𝑇
Theoretical formulation
Does synaptic scaling balance the divergent dynamics of synaptic plasticity?
Synaptic scaling induces stable weight dynamics
![Page 25: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/25.jpg)
Hebbian Plasticity and Scaling
Synaptic scalingHebbian Plasticity
ω = μ u v + γ [vT – v] ω2
stable
ω=μ u v
Hebbian Plasticity
unstable
+
![Page 26: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/26.jpg)
Synaptic Plasticity: STDP
Makram et al., 1997Bi and Poo, 2001
SynapseNeuron BNeuron A
u vω
LTP
LTD
![Page 27: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/27.jpg)
Synaptic Plasticity: Diversity
Kampa et al., 2007
![Page 28: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/28.jpg)
Synaptic plasticity together with synaptic scaling is globally stable
regardless of learning rule and neuron model.
Synaptic scaling is an appropriate candidate to guarantee stability.
![Page 29: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/29.jpg)
LearningStimulus
Memory Re-presentationof Stimulus
Inp
ut
Me
an A
ctiv
ity
Me
anSy
n.
We
igh
t
Formation of Memory Representations
Time
Long-term plasticity Synaptic Scaling
ሶ𝑤𝑖,𝑗 =1
τ𝑝𝑙𝑎𝑠𝑡𝐹𝑗 ∙ 𝐹𝑖 +
1τ𝑠𝑐𝑎𝑙
(𝐹𝑇 − 𝐹𝑖) ∙ 𝑤𝑖,𝑗2
Da
ta fro
m Tetzla
ff et al., 2
01
3
![Page 30: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/30.jpg)
Memory Re-presentationof Stimulus
James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; …
Hebbian Cell Assembly: A group of strongly interconnected neurons, which tend to fire together, form a memory representation of a stimulus.
Inp
ut
Me
an A
ctiv
ity
Me
anSy
n.
We
igh
t
Time
RecallStimulus
Formation of Memory RepresentationsLong-term plasticity Synaptic Scaling
ሶ𝑤𝑖,𝑗 =1
τ𝑝𝑙𝑎𝑠𝑡𝐹𝑗 ∙ 𝐹𝑖 +
1τ𝑠𝑐𝑎𝑙
(𝐹𝑇 − 𝐹𝑖) ∙ 𝑤𝑖,𝑗2
Da
ta fro
m Tetzla
ff et al., 2
01
3
![Page 31: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/31.jpg)
Structural Plasticity
Another learning mechanism
![Page 32: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/32.jpg)
Structural plasticity of axon terminals
Top: Axonal outgrowth/retraction
Red arrow: Axonal outgrowth
Yellow: Remains
Blue: Retracts
Bottom: Rewiring
Blue: Retracts and looses synapse
Red: Grows and creates new syn.
In-vivo 2 Photon-Laser Imaging from the cortex of living mice reveals a permanent axonal remodelling even in the adult brain leading to synaptic rewiring
![Page 33: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/33.jpg)
Structural plasticity of dendritic spines
Dendritic spines on an appical dendrite on a LIV-neuron in V1 of the rat
Arellano et al., 2007
2μm
Dendritic spines
Spine growth precedes synapse formation
Knott et al., 2006
Spine formation via filopodia-shaped spines (see arrow, top figure) precedes synapse formation. Spines in synapses are rather mushroom-shaped and carry receptor plates (active zones, red, top figure). Spines contact axonal terminals or axonal varicosities in reach and form synapses (left).
![Page 34: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/34.jpg)
Stable and transient spines
In-vivo imaging of dendritic trees within the barrel cortex of living rats
Trachtenberg et al., 2002
Spines are highly flexible structures that are responsible (together with axonal varicosities) for synaptic rewiring. Only one third of all spines are stable for more than a month. Another third is semistable, meaning that it is present for a couple of days. Transient spines appear and disappear within a day.
Structural plasticity of dendritic spines
![Page 35: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/35.jpg)
Structural plasticity creates and deletes synapses
Structural Plasticity: More abstract
![Page 36: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/36.jpg)
Structural Plasticity and Learning
Xu et al., 2009; Yang et al., 2009; Ziv and Ahissar 2009
before aftertraining
structuralchanges
camera
poor mice
![Page 37: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/37.jpg)
Structural Plasticity and Learning
new spine formation old spine elimination
new training task
training duration
Xu et al., 2009; Yang et al., 2009; Ziv and Ahissar 2009
![Page 38: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/38.jpg)
Structural plasticity
Kater et al., 1989; Mattson and Kater, 1989
Hofer et al., 2009
Dendrite of anadult mouse inthe visual cortex
high activitylow activity
Dendritic Outgrowth Dendritic Regression
monoculardeprivation
![Page 39: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/39.jpg)
Activity shapes neuronal form and connectivity
Morphology
.
Connectivity
Neuronal
activity
Neuronal
morphology
Network
connectivity
Neuronal activity changes the intracellular calcium. Via changes in intra-cellular calcium, neurons change their morphology with respect to their axonal and dendritic shape. This leads to changes in neuronal connectivity which, in turn, adapts neuronal activity. The goal is that by these changes neurons achieve a homeostatic equilibrium of their activity.
![Page 40: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/40.jpg)
Circular axonal and dendritic probability spaces
Individual shrinkage and growth following a homeostatic principle.
Overlap determines the synaptic connectivity.
Calcium dependentdendritic and axonal
growth or shrinkage for synapse generation or
deletion
# dendrites:
# axons:
overlap≈
# synapses
![Page 41: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/41.jpg)
Homeostatic Process
HighInputHigh
Calcium
AxonalGrowth
&Dendr.
Shrinkage
Less Synapsesat THIS neuron
LessInput
NormalInput
The model (without the equations)
![Page 42: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/42.jpg)
Model the development of the neuronal topology
Structural plasticity model
How to define in which developmental stage is a neuronal network?
Using self-organized criticality ?
![Page 43: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/43.jpg)
“critical” is a state between two different phases of a system.
e.g. the Ising model of magnetism
ferromagnetic critical paramagnetic
TC Curie temperature
Chialvo, 2007How to distinguish between states in a dynamic system?
![Page 44: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/44.jpg)
Now you have a scientific excuse for going to the beach.
Bak, 1996, How Nature works: The Science of Self-Organized Criticality
An avalanche is an object of spatial-temporal linked events.
Size – number of events within one avalancheDuration – time between first and last event of the avalanche
![Page 45: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/45.jpg)
“self-organized” describes the property of a system to develop towards a
state driven by local rules.
“critical” is a state between two different phases of a system.
• The critical state is very robust against external perturbations and responds
with avalanches
• These avalanches are power law distributed in size and duration
• Avalanches are invariant in size and duration: p(ax) = C(a) p(a)
Properties of SOC:
![Page 46: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/46.jpg)
subcritical critical supercritical
log(size)log(size) log(size)
log(P
(siz
e))
Levina et al., 2007
Critical state –> power law –> linear in loglog-plot
An avalanche is an object of spatial-temporally linked events.
Size – number of events within one avalancheDuration – time between first and last event of the avalanche
Self-Organized Criticality(a long chapter in Physics)
![Page 47: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/47.jpg)
Beggs and Plenz, 2003
Neuronal networks are critical.
bin width
∆t
![Page 48: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/48.jpg)
- 20 cell cultures between day 11 and 95 in vitro have been assessed
Poisson supercritical subcritical critical
![Page 49: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/49.jpg)
Model the development of the neuronal topology
Structural plasticity model
How to define in which developmental stage is a neuronal network?
Using self-organized criticality
![Page 50: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/50.jpg)
Circular axonal and dendritic probability spaces
Individual shrinkage and growth following a homeostatic principle.
Overlap determines the synaptic connectivity.
Calcium dependentdendritic and axonal
growth or shrinkage for synapse generation or
deletion
# denrites:
# axons:
overlap≈
# synapses
![Page 51: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/51.jpg)
Phase 2 – Overshoot/Pruning phase
Phase 3 – Equilibrium phase
Phase 1 – Outgrowth phase
Developmental dynamics of the model
# synapses
aver
age
acti
vity
![Page 52: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/52.jpg)
outgrowth overshoot equilibrating
Eplileptiform
Activity
Remains
supercriticalThis state not
existing
Criticality of model corresponds not to real culturesex
pe
rim
ents
mo
de
l
![Page 53: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/53.jpg)
Inhib
itio
n
Hedner et al., 1984Jiang et al., 2005Sutor and Luhmann, 1995
Inhibition as an additional „Ingredient“
![Page 54: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/54.jpg)
outgrowth
exp
eri
me
nts
mo
de
l
Tetzlaff et al., 2010
overshoot equilibrating
![Page 55: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/55.jpg)
General Conclusion
Homeostatic principles may solve the major problem of how to stabilze complete large networks even when other plasticity mechanisms co-act and interfere.
Synaptic Scaling is a good candidate for synapses
- guarantees stability
Structural Plasticity stabilizes the network topology
- brings networks into the critical state
- allows prediction about inhibition
![Page 56: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/56.jpg)
neuron
synapse
Input
Plasticity+Scaling
Structural Plasticity
tim
e
strong, interconnected clusterrepresenting memory
Synaptic Plasticity/Scaling and StructuralPlasticity Hypothesis
![Page 57: Learning and Stability - uni-goettingen.de · 2019. 2. 11. · Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases.](https://reader035.fdocuments.in/reader035/viewer/2022071402/60ee5a60d1a5e34cc368b443/html5/thumbnails/57.jpg)
Long-term plasticity
Synaptic scaling
Working memory
msec sec min hrs days weeks
Short-term memory
Activity
Short-termplasticity
Long-term memory Long-lasting memory
Structural plasticity
Adaptive Processes on Different Time Scales
Synaptic scaling
Long-term plasticity
Me
mo
ryP
last
icit
y
Tetzlaff et al. (2012). Biol. Cybern.