Efficient SOM Learning by Data Order

Adjustment

Authors: Miyoshi et al.

Advisor: Dr. Hsu

Graduate :Yu-Wei Su

Outline

Motivation Objective SOM Data order and learning convergence

Data order Ending point of convergence

Experiments Conclusion opinion

Motivation

In SOM, there are many factors to aggravate computational load and competition

Objective

Reducing the competition load and increasing the speed of SOM

SOM

SOM is a algorithm that map dimension from high to low, always two dimension

Step 1: finding BMU of each datum

Step 2: modifying the value of BMU and neighborhood nodes

cc

b mxmx min

)]()()[()()()1( tmtxthttmtm cbccc

))(2

exp()(2

2

t

rrth cb

bc

Data order and learning convergence

The SOM spends lots of time to learn because of large map size, large quantity of input data and many dimensions in data et al.

In the beginning stage of learning process, SOM map is dynamically and widely and that is depended on the distance of each input data

Data order and learning convergence( cont.)

Adjusting data order based on the distance between data classes to reduce the competition load

Data order

To change order of input data, using class distance that is calculated by class center

First select typical data as class center in each class

And calculate Euclidian distance between all class centers as class distance

Data order( cont.)

Order 1: random order

Order 2: the largest distance order based on previous data class

)(: ii

n uclrandomscl

)),((max: 1 ini

n uclsclcdscl

scli : selected data class iucli : still unselected data class icd(A,B) :class distance between A and B

Data order( cont.)

Order 3: the smallest distance order based on previous data class

Order 4: the smallest distance order based on all classes

)),((min: 1 ini

n uclsclcdscl

ijucluclcdsclj

jii

n ],),([min:

scli : selected data class iucli : still unselected data class icd(A,B) :class distance between A and B

Data order( cont.)

Order 5: the largest distance order based on all classes

Order 6: average distance order based on all classes

ijucluclcdsclj

jii

n ],),([max:

ijucluclcdaveragesclj

jii

n ],),([:

scli : selected data class iucli : still unselected data class icd(A,B) :class distance between A and B

Ending point of convergence

Definition of converging point of learning Keep maximum Euclidian distance for all nodes in

each step of learning

Test the difference between (|xdn-xdn-1|) whether (|xdn-xdn-1|) is smaller than Th1

If it is, test how long the distance are continued smaller

If it continues long enough than Th2 it is determined as the ending point of learning

Ending point of convergence (cont.)

1,...,1 21 ThxdxdThxdxd Thnnnn

)0,0,(,)),((minmax maxmax NjDinoondtedxd jiji

n

Th1 : threshold of differenceTh2 : threshold of periodxdn : n-th max distance through all input data and output nodesed(A,B) : Euclidian distance between A and Bdti : input data Ionj: output node jdmax : total of input datanmax : total of output nodes

Experiments

Experiment data Synthetic data, 5 dim, 7 classes each 49 data

Parameters Size of map 8x8, initial neighborhood from 3 to 5,

initial learning rate from 0.2 to 0.8, 300 maps that initialized at random

Experiments ( cont.)

Learning rate function

Neighborhood functiontA

At

)0(

)(

tB

BNtN

)0()(

Experiments ( cont.)

Experiments ( cont.)

Conclusion

The data stream of small distance makes maximum 9% improvement

The data stream of large distance still similar with conventional SOM

All order make no remarkable difference in result map

Opinion

No experiments of comparison with others The terminal condition is a good idea