Data Visualisation using Topographic Mappings
-
Upload
catherine-mccall -
Category
Documents
-
view
36 -
download
4
description
Transcript of Data Visualisation using Topographic Mappings
![Page 1: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/1.jpg)
Data Visualisation using Topographic Mappings
Colin Fyfe
The University of Paisley,
Scotland
![Page 2: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/2.jpg)
Outline
• Topographic clustering.
• Topographic Product of Experts, ToPoE
• Simulations
• Products and mixtures of experts.
• Harmonic Topographic Mapping, HaToM
• 2 Varieties of HaToM
![Page 3: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/3.jpg)
![Page 4: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/4.jpg)
The somatosensory homunculus
• Larger area of cortex for more sensitive body parts.
Lower lip
Upper lipFace
Eye
Thumb
GenitalsToesFoot
LegTrunkHeadArmHand
LittleRing
MiddleIndex
Thumb
Pharynx
![Page 5: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/5.jpg)
Kohonen’s Self-organizing Map
• camtasia\som.html
![Page 6: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/6.jpg)
Orientation selectivity
![Page 7: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/7.jpg)
Topology preservation
• Data space Feature Space
• Nearby Nearby
• Distant Distant (SOM)
• Nearby Nearby (SOM)
• Distant Distant
![Page 8: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/8.jpg)
The Model
• K latent points in a latent space with some structure.
• Each mapped through M basis functions to feature space.
• Then mapped to data space to K points in data space using W matrix (M by D)
• Aim is to fit model to data to make data as likely as possible by adjusting W
![Page 9: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/9.jpg)
Mental Model
• 1 2 3 4 5 6 ……
![Page 10: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/10.jpg)
Details
• t1, t2, t3, … ,tK (points in latent space)
• f1(), f2(), …, fM() (basis functions creating feature space)
• Matrix Φ (K by M), where φkm = fm(tk), projections of latent points to feature space.
• Matrix W (M by D) so that ΦW maps latent points to data space. tk mk
![Page 11: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/11.jpg)
Products of Gaussian Experts
)||||2
exp()(
))||||(2
exp()(
2
1
2
1
nk
K
kn
nk
K
kn
xmxp
xmxp
![Page 12: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/12.jpg)
Maximise the likelihood of the data under the model
M
mkmmd
kd
kd
K
k
ndkmmdn
wm
mxw
1
)(
)(
1
)( )(
K
kknn mxxp
1
2))(log(
![Page 13: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/13.jpg)
Using Responsibilities
M
mkmmd
kd
knkd
K
k
ndkmmdn
wm
rmxw
1
)(
)(
1
)( )(
||||||||
)exp(
)exp(
1
2
2
jnjm
M
mmnjn
jjn
knkn
mxwxd
d
dr
![Page 14: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/14.jpg)
![Page 15: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/15.jpg)
![Page 16: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/16.jpg)
Comparison with GTM
)||||2
exp()2(
1)|()()( 22
kn
D
kknn mx
Kkxpkpxp
))||||(2
exp()2()( 2
1
2knnk
K
k
D
n rxmxp
t nt
nkknxxk d
dr
n )exp(
)exp(2
2
|
![Page 17: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/17.jpg)
Growing ToPoEs
![Page 18: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/18.jpg)
Demonstration
![Page 19: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/19.jpg)
![Page 20: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/20.jpg)
Advantages
• Growing : need only change Φ which goes from K by M to (K+1) by M.
• W is approximately correct and just refines its learning.
• Pruning uses the responsibility: if a latent point is never the most responsible point for any data point, remove it.
• Keep all other points at their positions in latent space and keep training.
![Page 21: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/21.jpg)
)||(2
exp(1
)( 11
knnk
K
kn rxm
Zxp
![Page 22: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/22.jpg)
Twinned ToPoEs
• Single underlying latent space
• Single Φ
• Two sets of weights
• Single responsibility – distance between current projections and both data points.
![Page 23: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/23.jpg)
Twinned ToPoEs - 2
![Page 24: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/24.jpg)
Using φkm=tanh(mtk)
![Page 25: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/25.jpg)
Products of Experts
![Page 26: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/26.jpg)
![Page 27: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/27.jpg)
![Page 28: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/28.jpg)
Responsibilities with Tanh()
1
19
37
55
S1
S1
1
0
0.2
0.4
0.6
0.8
1
![Page 29: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/29.jpg)
![Page 30: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/30.jpg)
![Page 31: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/31.jpg)
A close up
![Page 32: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/32.jpg)
![Page 33: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/33.jpg)
![Page 34: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/34.jpg)
![Page 35: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/35.jpg)
![Page 36: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/36.jpg)
![Page 37: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/37.jpg)
Harmonic Averages
• Walk d km at 5 km/h, then d km at 10 km/h• Total time = d/5 +d/10• Average Speed = 2d/(d/5+d/10)=
• Harmonic Average =
10
1
5
12
K
k ka
K
1
1
![Page 38: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/38.jpg)
K-Harmonic Means beats K-Means and MoG using EM
• Perf =
N
iK
kki mx
K
1
12
1
N
iK
l ilik
ki
k
dd
mxK
m
Perf
1
1
22
4 )1
(
)(4
N
iK
l ilik
N
iiK
l ilik
k
dd
x
dd
m
1
1
22
4
1
1
22
4
)1
(
1
)1
(
1
![Page 39: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/39.jpg)
Growing Harmony Topology Preservation
• Initialise K to 2. Init W randomly.
1. Init K latent points and M basis functions.
2. Calculate mk=φkW, k=1,…,K.
1. Calculate dik, i=1,…,N, k=1,…,K
2. Re-calculate mk, k=1,…,K. (Harmonic alg.)
3. If more, go back to 1.
3. Re-calculate W=(ΦTΦ+γI)-1ΦTШ
4. K= K + 1. If more, go back to 1.
![Page 40: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/40.jpg)
Disadvantages-Advantages ?
• Don’t have special rules for points for which no latent point takes responsibility.
• But must grow otherwise twists.
• Independent of initialisation ?
• Computational cost ?
![Page 41: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/41.jpg)
Generalised K-Harmonic Meansfor Automatic Boosting
• Perf =
N
iK
kp
ki mx
K
1
1
1
N
iK
lpil
pik
ki
k
dd
mxpK
m
Perf
1
1
21 )1
(
)(2
N
iK
lpil
pik
N
iiK
lpil
pik
k
dd
x
dd
m
1
1
22
1
1
22
)1
(
1
)1
(
1
![Page 42: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/42.jpg)
![Page 43: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/43.jpg)
![Page 44: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/44.jpg)
![Page 45: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/45.jpg)
![Page 46: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/46.jpg)
![Page 47: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/47.jpg)
![Page 48: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/48.jpg)
![Page 49: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/49.jpg)
![Page 50: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/50.jpg)
Two versions of HaToM
• D-HaToM (Data driven HaToM) :– W and m change only when adding a new
latent point– Allows the data to influence more the
clustering
• M-HaToM (Model driven HaToM) :– W and m change in every iteration– The data is continually constrained by the
model
![Page 51: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/51.jpg)
Simulations(1): 1D dataset
K=2 K=4 K=8
K=20
M-HaToMD-HaToM
![Page 52: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/52.jpg)
Simulations(3): algae dataset
D-HaToM
M-HaToM
Wider rij Narrower rij
![Page 53: Data Visualisation using Topographic Mappings](https://reader036.fdocuments.in/reader036/viewer/2022062422/56813323550346895d9a0159/html5/thumbnails/53.jpg)
Conclusion
• New forms of topographic mapping.
• Based on latent space concept but– free from probabilistic constraints.
• Product Mixture of experts.– automatic setting of local variances.
• Two types based on K-harmonic Means
• Very sensitive to data, or not, as required