Fiona Cosgrove Wellness Coaching Australia [email protected] .
Plasticity and Sensitive Periods in Self-Organising Maps Fiona M. Richardson and Michael S.C. Thomas...
1
Plasticity and Sensitive Periods in Self-Organising Maps Fiona M. Richardson and Michael S.C. Thomas [email protected], [email protected] Developmental Neurocognition Lab, Birkbeck College, University of London Plasticity and the Brain • Within the brain there are systems that exhibit different plastic qualities: -The somatosensory cortex adapts quickly and retains its plasticity into adulthood (Braun et al. 2001). - In the language system, the ability of adults to learn phonological distinctions outside their language appears reduced in comparison to children (McCandliss et al. 2002). How is plasticity lost or retained with age in self-organising systems? SOFMs: An Approach • Self-organising feature maps (SOFMs) are an unsupervised learning system, in which the similarity of exemplars is represented topographically. • Changes in plasticity in SOFMs are characterised by changes in learning rate and neighbourhood distance over training. • Typically, these values decrease over training, as the map organises and then fine tunes the representations. In our simulations we explored the ability of SOFMs to modify their category representations in two systems with contrasting plasticity. C ategories tools utensils dairy produce animals hum ans vegetables fruit vehicles oddm ents (i) Development • Fixed plasticity maps develop their representations quicker, but show little subsequent change or expansion. • The granularity of these representations is different for the two systems. Reducing Fixed 5 epochs 50 epochs 100 epochs 250 epochs 550 epochs 850 epochs 1000 epochs 0 20 40 60 80 100 120 5 50 100 250 400 550 700 850 1000 epochs number of active unit reducing fixed A comparison of active units for reducing and fixed maps (ii) Effective plasticity across training • Probed by introducing a new category at a later point in training. •For reducing maps adding a new category in the early stages of learning resulted in better overall representations. • Interference: new categories positioned themselves nearest the most similar existing category, causing disruption to existing representations. 0 2 4 6 8 10 12 14 16 18 20 5 50 100 250 400 550 700 850 epoch category added num of category uni hum ans default hum ans added vehicles default vehicles added oddm ents default oddm ents added 0 2 4 6 8 10 12 14 16 18 20 5 50 100 250 400 550 700 850 epoch added number of active units reducing humans reducing vehicles reducing oddments fixed humans fixed vehicles fixed oddments Deterioration of category learning with decreasing plasticity A comparison of category activity for reducing and fixed maps Reducing: end state Fixed: end state 50 epochs 250 epochs 400 epochs 850 epochs (iii) Thresholds Reducing: end state Fixed: end state no threshold threshold 1 threshold 2 threshold 3 - How do thresholds impact upon plasticity and category learning? •Constant thresholds seem to act as an impediment to learning. • Fixed plasticity maps seem more resistant to the adverse effects of thresholds. Suggested threshold function no threshold threshold 1 threshold 2 threshold 3 epochs t (iv) Atypical categorisation and developmental disorders -Perhaps poor maps produce impaired categorisation? Inputlayer (i.e. sensory) O utputlayer (i.e. motor) = Initialconnectivity = Finalconnectivity Comparing Reducing and Fixed Plasticity Maps Categorisation in normal and atypical systems Input, output and topology This research was funded by UK MRC Career Establishment Grant G0300188 to Michael Thomas 154 dimensional feature space consisting of 9 categories, with 15-30 exemplars per category. (point in training new category added at) (graphs showing the size of the added category upon completion of training) Key: Humans = high similarity Vehicles = lower similarity Oddments = low similarity • For a poor map to result in impaired categorisation behaviour (Gustafsson, 1997): -Downstream output must be topographically organised for poor input topology to matter (Oliver et al. 2000). - Must have initial full connectivity between input and output. - Connectivity must be pruned back during the learning process, and pruning must produce receptive fields. References: • Braun, C., Heinze, U., Schweizer, R., Weich, K., Birbaumer, N., & Topka, H. (2001). Dynamic organization of the somatosensory cortex induced by motor activity. Brain, 124 2259-2267. • Gustafsson, L. (1997). Inadequate cortical feature maps: A neural circuit theory of autism. Biological Psychiatry, 42, 1138-1147. •Oliver, A., Johnson, M.H., Karmiloff-Smith, A., & Pennington, B. (2000). Deviations in the emergence of representations: A neuroconstructivist framework for analysing developmental disorders. Developmental Science, 3, 1-23. O utput= good categorisation O utput= poorcategorisation (a)N orm alsystem (b)A typicalsystem (1)R educing plasticity 0 2 4 6 8 10 12 14 16 18 20 0 100 250 550 700 850 1000 Ep och of train ing N eighbourhood size 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 learning rate neighbourhood learning rate (2)Fixed plasticity 0 0.5 1 1.5 2 2.5 0 100 250 550 700 850 1000 Ep o ch of train in g N eighbourhood size 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 learning rate neighbourhood learning rate Parameter changes over time
-
Upload
preston-woods -
Category
Documents
-
view
215 -
download
0
Transcript of Plasticity and Sensitive Periods in Self-Organising Maps Fiona M. Richardson and Michael S.C. Thomas...
Plasticity and Sensitive Periods in Self-Organising Maps
Fiona M. Richardson and Michael S.C. Thomas [email protected], [email protected]
Developmental Neurocognition Lab, Birkbeck College, University of London
Plasticity and the Brain
• Within the brain there are systems that exhibit different plastic qualities:
-The somatosensory cortex adapts quickly and retains its plasticity into adulthood (Braun et al. 2001).
- In the language system, the ability of adults to learn phonological distinctions outside their language appears reduced in comparison to children (McCandliss et al. 2002).
How is plasticity lost or retained with age in self-organising systems?
SOFMs: An Approach
• Self-organising feature maps (SOFMs) are an unsupervised learning system, in which the similarity of exemplars is represented topographically.
• Changes in plasticity in SOFMs are characterised by changes in learning rate and neighbourhood distance over training.
• Typically, these values decrease over training, as the map organises and then fine tunes the representations.
In our simulations we explored the ability of SOFMs to modify their category representations in two systems with contrasting plasticity.
Categories
tools utensils dairy produce
animals humans
vegetables
fruit vehicles oddments
(i) Development
• Fixed plasticity maps develop their representations quicker, but show little subsequent change or expansion.
• The granularity of these representations is different for the two systems.
Reducing
Fixed
5 epochs 50 epochs 100 epochs 250 epochs 550 epochs 850 epochs 1000 epochs
0
20
40
60
80
100
120
5 50 100 250 400 550 700 850 1000
epochs
nu
mb
er o
f ac
tive
un
its
reducing
fixed
A comparison of active units for reducing and fixed maps
(ii) Effective plasticity across training
• Probed by introducing a new category at a later point in training.
•For reducing maps adding a new category in the early stages of learning resulted in better overall representations.
• Interference: new categories positioned themselves nearest the most similar existing category, causing disruption to existing representations.
0
2
4
6
8
10
12
14
16
18
20
5 50 100 250 400 550 700 850
epoch category added
nu
m o
f c
ate
go
ry u
nit
s humans default
humans added
vehicles default
vehicles added
oddments default
oddments added
0
2
4
6
8
10
12
14
16
18
20
5 50 100 250 400 550 700 850
epoch added
num
ber o
f act
ive
units
reducing humans
reducing vehicles
reducing oddments
fixed humans
fixed vehicles
fixed oddments
Deterioration of category learning with decreasing plasticity
A comparison of category activity for reducing and
fixed maps
Reducing: end state
Fixed: end state
50 epochs 250 epochs 400 epochs 850 epochs
(iii) Thresholds
Reducing: end state Fixed: end state
no threshold threshold 1 threshold 2 threshold 3
- How do thresholds impact upon plasticity and category learning?
•Constant thresholds seem to act as an impediment to learning.
• Fixed plasticity maps seem more resistant to the adverse effects of thresholds.
Suggested threshold function
no threshold threshold 1 threshold 2 threshold 3
epochs
t
(iv) Atypical categorisation and developmental disorders
-Perhaps poor maps produce impaired categorisation?
Input layer (i.e. sensory)
Output layer (i.e. motor)
= Initial connectivity = Final connectivity
Comparing Reducing and Fixed Plasticity Maps
Categorisation in normal and atypical systems
Input, output and topology
This research was funded by UK MRC Career Establishment Grant G0300188 to Michael Thomas
154 dimensional feature space consisting of 9 categories, with 15-30 exemplars per category.
(point in training new category added at)(graphs showing the size of the added category upon completion of training)
Key:
Humans = high similarity
Vehicles = lower similarity
Oddments = low similarity
• For a poor map to result in impaired categorisation behaviour (Gustafsson, 1997):
-Downstream output must be topographically organised for poor input topology to matter (Oliver et al. 2000).
- Must have initial full connectivity between input and output.
- Connectivity must be pruned back during the learning process, and pruning must produce receptive fields.
References:• Braun, C., Heinze, U., Schweizer, R., Weich, K., Birbaumer, N., & Topka, H. (2001). Dynamic organization of the somatosensory cortex induced by motor activity. Brain, 124 2259-2267.
• Gustafsson, L. (1997). Inadequate cortical feature maps: A neural circuit theory of autism. Biological Psychiatry, 42, 1138-1147.
•Oliver, A., Johnson, M.H., Karmiloff-Smith, A., & Pennington, B. (2000). Deviations in the emergence of representations: A neuroconstructivist framework for analysing developmental disorders. Developmental Science, 3, 1-23.
Output = good categorisation Output = poor categorisation
(a) Normal system (b) Atypical system
(1) Reducing plasticity
0
2
4
6
8
10
12
14
16
18
20
0 100 250 550 700 850 1000
Epoch of training
Nei
gh
bo
urh
oo
d s
ize
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
lear
nin
g r
ate
neighbourhood
learning rate
(2) Fixed plasticity
0
0.5
1
1.5
2
2.5
0 100 250 550 700 850 1000
Epoch of training
Nei
gh
bo
urh
oo
d s
ize
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
lear
nin
g r
ate
neighbourhood
learning rate
Parameter changes over time
