Download - Self Organizing Maps

Transcript
Page 1: Self Organizing Maps

Self Organizing Maps

Page 2: Self Organizing Maps

Self Organizing Maps• This presentation is based on:

http://www.ai-junkie.com/ann/som/som1.html

• SOM’s are invented by Teuvo Kohonen.• They represent multidimensional data in much lower dimensional spaces - usually two dimensions. • Common example is the mapping of colors from their three dimensional components - red, green

and blue, into two dimensions. • 8 colors on the right have been presented as 3D vectors and the system has learnt to represent

them in the 2D space. • In addition to clustering the colors into distinct regions, regions of similar properties are usually

found adjacent to each other.

Page 3: Self Organizing Maps

Network Architecture• Data consists of vectors, V, of n

dimensions: V1, V2, V3…Vn

• Each node will contain a corresponding weight vector W, of n dimensions: W1, W2, W3...Wn.

Page 4: Self Organizing Maps

Network Example• Each node in the 40-by-40 lattice

has three weights, one for each element of the input vector:

red, green and blue.

• Each node is represented by a rectangular cell when drawn to display.

Page 5: Self Organizing Maps

Overview of the AlgorithmIdea: Any new, previously unseen input vector presented to the network

will stimulate nodes in the zone with similar weight vectors.

1. Each node's weights are initialized.2. A vector is chosen at random from the set of training data and

presented to the lattice.3. Every node is examined to calculate which one's weights are most

like the input vector. The winning node is commonly known as the Best Matching Unit (BMU).

4. The radius of the neighborhood of the BMU is now calculated. This is a value that starts large, typically set to the 'radius' of the lattice,  but diminishes each time-step. Any nodes found within this radius are deemed to be inside the BMU's neighborhood.

5. Each neighboring node's (the nodes found in step 4) weights are adjusted to make them more like the input vector. The closer a node is to the BMU, the more its weights get altered.

6. Repeat step 2 for N iterations.

Page 6: Self Organizing Maps

DetailsInitializing the Weights• Set to small standardized random values 0 < w < 1

Calculating the Best Matching Unit• Use some distance

Determining the Best Matching Unit's Local Neighborhood

Area of the neighborhood shrinks over time by shrinking the radius of the neighborhood over time

For this use the exponential decay function:

0 is the width of the lattice at time t0

is a constantt is the iteration number

Page 7: Self Organizing Maps

Details

Over time the neighborhood will shrink to the size of just one node... the BMU

Page 8: Self Organizing Maps

DetailsAdjusting the Weights• Every node within the BMU's neighborhood (including the BMU) has

its weight vector adjusted according to the following equation:

• where t represents the time-step and L is a small variable called the learning rate, which decreases with time.

• The decay of the learning rate is calculated each iteration using the following equation:

Page 9: Self Organizing Maps

Details• Also, the effect of learning  should be proportional to the

distance a node is from the BMU.

Page 10: Self Organizing Maps

Applications• SOM’s are commonly used as visualization aids.

– They can make it easy to see relationships between vast amounts of data.