Self-Organizing Map · presentation by Andreas T oscher Self-Organizing Map. Introduction...

IntroductionMathematical Definitions

ExamplesInverse Kinematics

Netflix User MappingConclusion

Self-Organizing Map

presentation by Andreas Toscher

19. May 2008

presentation by Andreas Toscher Self-Organizing Map




1 Introduction

2 Mathematical Definitions

3 Examples

4 Inverse Kinematics

5 Netflix User Mapping

6 Conclusion





Self-Organizing Map (SOM)

aka Kohonen Network

introduced by Teuvo Kohonen

implements a discrete nonlinear mapping

unsupervised learning





Mathematical Definitions





Structure of a SOM





Learning Rule

wi[t] is the weight vector of neuron i at the time t

x[t] is the input vector at time t

η[t] is the learning rate at time t

φ(i, k) is the neighbourhood function, k is the winner neuron

wi[t+ 1]← wi[t] + η[t]φ(i, k)(x[t]−wi[t]) (1)





Neighborhood Function φ(i, k)

is aware of the mapping

simplest choice:φ(i, k) = 1 if dist(i, k) < τφ(i, k) = 0 else

gaussian:

φ(i, k) = exp

(−dist(i, k)

2

σ2

)





General Tips for the Training

rescale the input (e.g. µ = 0, σ2 = 1)

decrease the learning Rate η during the training

start with a large neighbourhood for φ(i, k) and reduce itduring the training

avoid using too many neurons, in the extreme case the SOMends up in a simple one to one mapping





Examples





2D to 1D map

input: randomly drawn points from a triangular 2D region

1D Self-Organizing map

Peano Curve





2D to 2D map

input: randomly drawn points from a quadratic region100, 1000, 5000, 10000 epochs





2D Mapping with a Knot

the nice unfolding is not guaranteed

a possible solution is to start training with a bunch of randominitializations





Remarks:

in general 2D to 2D mappings are of no interest

examples of interesting maps:

a sphere to a planea plane in a 3D space to a 2D space

in the above examples the inner dimension of the data wasknown

if the target is too low dimensional, the SOM folds like aPeano Curve





Inverse Kinematics





Robot Arm Movement

input space: angle of the joints of the robot armtarget: x and y position of the robot armassumption: the robot arm is constructed in that way, that agiven position (x, y) is only reachable with one exact one setof angels (α, β)modified training: no winner evaluation, the winner is theneuron closest to the given (x, y) position





to move the robot arm from A to B, you only have tointerpolate between the weights of the neurons





Obstacle Avoidance

works only if the obstacle was there during the training phase





Netflix

500000 users

18000 movies

100 Mio ratings

user based Collaborative Filtering (aka KNN)

rui =

∑v∈N(u,i) cvurvi∑

v∈N(u,i) cvu(2)

Problems

2 arbitrary users have few common ratings, which is very badfor the correlation coefficient

the user/user correlation matrix could not be kept in mainmemory





rui =

∑v∈N(u,i) cvurvi∑

v∈N(u,i) cvu(3)

N(u, i) are the best K user neighbours

cvu is a correlation between users, based on the mapping





Conclusion

Kohonen Maps learn a mapping (usually from a high to a lowdimensional space)

the mapping can be used easily in both directions

the learning rule is simple, and can be implemented veryefficiently

the basics of Kohonen Maps are over 30 years old and havebeen used for very interesting projects





Thanks for your attention

References

R. Rojas, Neural Networks, Springer 1996

Scholarpedia, http://www.scholarpedia.org/article/Kohonen_network

Wikipedia,http://en.wikipedia.org/wiki/Self-organizing_map


http://www.scholarpedia.org/article/Kohonen_network

http://www.scholarpedia.org/article/Kohonen_network

http://en.wikipedia.org/wiki/Self-organizing_map

Self-Organizing Map · presentation by Andreas T oscher Self-Organizing Map. Introduction...

Documents

Transcript of Self-Organizing Map · presentation by Andreas T oscher Self-Organizing Map. Introduction...