Kohonen Maps and Counterpropagation

Artificial Neural Networks Toolbox for MATLAB

Davide BallabioMilano Chemometrics and QSAR Research GroupUniversità Milano - Bicocca

Kohonen and CPANN Toolbox for MATLAB

• Kohonen Maps (or Self Organizing Maps) and Counterpropagation Artificial Neural Networks (CPANNs) are two of the most popular Neural Networks (algorithms that simulate the human learning).

• can handle both supervised and unsupervised problems

• Kohonen and CP-ANN toolbox is a collection of MATLAB modules freely available via internet: http://michem.disat.unimib.it/chm

Kohonen maps

• Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems

SOM are made of neurons and need time (iterations) to learn to describe the data.

The time (number of iterations) and the size (number of neurons) must be defined by the user

• Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems

p Kohonen layers(p = number of variables)

Kohonen maps

AAx1

x2

x3

x4

TOP MAP

Kohonen layers

X

Toroidal geometryToroidal geometry

• Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems

Kohonen maps

• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems)

• CPANNs are an evolution of Kohonen Maps

CPANN

• Class unfolding: transform the class vector in a matrix of zeros and ones

CPANN

Multi-y (n x G)Class (n x 1)

112233..GG

1 0 0 … … … 01 0 0 … … … 00 1 0 … … … 00 1 0 … … … 00 0 1 … … … 00 0 1 … … … 0

0 0 0 … … … 10 0 0 … … … 1

unfolding

AAx1

x2

x3

x4

y1

y2

y3

TOP MAP

• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems)

CPANN

Output layers

Kohonen layers

• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems):

CPANN

out x wc si jii

m

min2

1

For each sample, for each epoch:

1) Find the winning neuron

• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems):

CPANN

For each sample, for each epoch:

2) Update the Kohonen weights

oldwxΔw rir

r η

1d

d1

max

learning rate

topological distance

• Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems):

CPANN

For each sample, for each epoch:

3) Update the output weights

oldycΔy rir

r η

1d

d1

max

• In order to better understand how variables can characterize the data, we can do Principal Component Analysis on the Kohonen weights:

Kohonen Maps and CPANN

PCA

Weights matrix

N x p

W

2. Eigenvectors (loadings)analyse the variables

variables

neur

ons

1. Eigenvalues:decide how many component we can retain

3. Scoresanalyse the neurons

relate neurons (and samples placed in neurons) and variables in a global way

Kohonen Maps and CPANN toolbox

• Features of the toolbox:• user defined data scaling + automatic range scaling• modules for fitting and validating models• main available settings:

• size and epochs (required)• boundary condition• learning rates can be modified• missing values are handled

• visualize top map with Graphical User Interface• visualize PCA on weights with Graphical User Interface

The toolbox is based on the algorithm explained in: Zupan J, Novic M, Ruisánchez I. Chemometrics and Intelligent Laboratory Systems (1997) 38 1-23.

Kohonen Maps and CPANN toolbox

• Example of application on the ITAOILS dataset:

• 572 olive oil samples, • each sample is described by the percentage composition of 8 fatty acids (variables)• samples belong to 9 different Italian areas of productions (classes)• the final aim of the classification model is the geographical origin determination of the samples.

Data reference:M. Forina, C. Armanino, S. Lanteri, E. Tiscornia, in: Food Research and Data Analysis, Classification of olive oils from their fatty acid composition, Applied Science Publishers, London, 1983.

Kohonen Maps and CPANN toolbox

• Principal Component Analysis for data structure evaluation

Thanks for your attention !

Davide Ballabio

Milano Chemometrics and QSAR Research GroupDepartment of Environmental SciencesUniversità Milano – Bicocca

You can download the Kohonen and CP-ANN toolbox 1.0 here:http://michem.disat.unimib.it/chm

Version 2.0 will be available soon…

Paper in preparation, to be submitted to Chemometrics and Intelligent Laboratory Systems