# A principal components analysis self-organizing map

of 26/26

date post

07-Jan-2016Category

## Documents

view

37download

0

Embed Size (px)

description

A principal components analysis self-organizing map. Ezequiel Lopez-Rubio, Jose Munoz-Perez, Jose Antonio Gomez-Ruiz. Advisor : Dr. Hsu Student : Sheng-Hsuan Wang Department of Information Management. Neural Network 17 (2004) 261-270. Outline. Motivation Objective - PowerPoint PPT Presentation

### Transcript of A principal components analysis self-organizing map

Research Progress ReportAdvisor : Dr. Hsu

Student : Sheng-Hsuan Wang

Ezequiel Lopez-Rubio, Jose Munoz-Perez, Jose Antonio Gomez-Ruiz

Intelligent Database Systems Lab

Motivation

The adaptive subspace self-organizing map (ASSOM) is an alternative to the standard principal component analysis (PCA) algorithm

Look for the most relevant features of the input data.

However, its training equations are complexes.

Separate ability in the classical PCA and ASSOM.

Intelligent Database Systems Lab

Objective

This paper proposed a new self-organizing neural model that performs principal components analysis

Like the ASSOM, but has a broader capability to represent the input distribution.

Intelligent Database Systems Lab

N.Y.U.S.T. I. M.

The ASSOM network

The ASSOM network uses subspaces in each node rather than just single weights.

The ASSOM network is based on training not just using single samples but sets of slightly translated, rotated and/or scaled signal or image samples, called episodes.

Each neuron of an ASSOM network represents a subset of the input data with a vector basis which is adapted so that the local geometry of the input data is build.

Intelligent Database Systems Lab

orthogonal projection

A vector x on an orthonormal vector basis B={bh|h=1,…,K}

The vector x can be decomposed into two vectors

orthogonal projection and projection error.

Intelligent Database Systems Lab

N.Y.U.S.T. I. M.

The ASSOM network

The input vectors are grouped into episodes in order to be presented to the network.

An episode S(t) has many time instants tp belongs to S(t), each with an input vector x(tp).

episodes: sets of slightly translated, rotated or scaled samples.

Intelligent Database Systems Lab

Intelligent Database Systems Lab

The objective function is the average expected spatially weighted normalized squared projection error over the episodes.

The Robbins-Monro stochastic approximation is used to minimize objective function, which leads to Basis vectors rotation.

Intelligent Database Systems Lab

Use the covariance matrix to store the information.

The covariance matrix of an input vector x is defined as

M input samples

If we obtain N new input samples

Intelligent Database Systems Lab

the new approximations.

N.Y.U.S.T. I. M.

The PCASOM network

Competition among neurons

The neuron c that has the minimum sum of projection errors is the winner:

Orth(x, B) is the orthogonal projection of vector x on basis B.

Intelligent Database Systems Lab

Intelligent Database Systems Lab

For every unit i, obtain the initial covariance matrix R(0).

For every unit i, build the vector ei(0) by using small random value.

At time instant t, select the input vectors x(t). Compute the winning neuron c.

For every unit i; update the vector ei and the matrix Ri

Convergence condition.

Matrix sums

It has a wider capability to represent the input distribution.

Intelligent Database Systems Lab

Intelligent Database Systems Lab

projection error norm for BMU

the norm of the input vector

Intelligent Database Systems Lab

Related to the ASSOM

Its training equations are much simpler

Its input representation capability is broader

Experiments show that the new model has better performance than the ASSOM network.

Intelligent Database Systems Lab

Student : Sheng-Hsuan Wang

Ezequiel Lopez-Rubio, Jose Munoz-Perez, Jose Antonio Gomez-Ruiz

Intelligent Database Systems Lab

Motivation

The adaptive subspace self-organizing map (ASSOM) is an alternative to the standard principal component analysis (PCA) algorithm

Look for the most relevant features of the input data.

However, its training equations are complexes.

Separate ability in the classical PCA and ASSOM.

Intelligent Database Systems Lab

Objective

This paper proposed a new self-organizing neural model that performs principal components analysis

Like the ASSOM, but has a broader capability to represent the input distribution.

Intelligent Database Systems Lab

N.Y.U.S.T. I. M.

The ASSOM network

The ASSOM network uses subspaces in each node rather than just single weights.

The ASSOM network is based on training not just using single samples but sets of slightly translated, rotated and/or scaled signal or image samples, called episodes.

Each neuron of an ASSOM network represents a subset of the input data with a vector basis which is adapted so that the local geometry of the input data is build.

Intelligent Database Systems Lab

orthogonal projection

A vector x on an orthonormal vector basis B={bh|h=1,…,K}

The vector x can be decomposed into two vectors

orthogonal projection and projection error.

Intelligent Database Systems Lab

N.Y.U.S.T. I. M.

The ASSOM network

The input vectors are grouped into episodes in order to be presented to the network.

An episode S(t) has many time instants tp belongs to S(t), each with an input vector x(tp).

episodes: sets of slightly translated, rotated or scaled samples.

Intelligent Database Systems Lab

Intelligent Database Systems Lab

The objective function is the average expected spatially weighted normalized squared projection error over the episodes.

The Robbins-Monro stochastic approximation is used to minimize objective function, which leads to Basis vectors rotation.

Intelligent Database Systems Lab

Use the covariance matrix to store the information.

The covariance matrix of an input vector x is defined as

M input samples

If we obtain N new input samples

Intelligent Database Systems Lab

the new approximations.

N.Y.U.S.T. I. M.

The PCASOM network

Competition among neurons

The neuron c that has the minimum sum of projection errors is the winner:

Orth(x, B) is the orthogonal projection of vector x on basis B.

Intelligent Database Systems Lab

Intelligent Database Systems Lab

For every unit i, obtain the initial covariance matrix R(0).

For every unit i, build the vector ei(0) by using small random value.

At time instant t, select the input vectors x(t). Compute the winning neuron c.

For every unit i; update the vector ei and the matrix Ri

Convergence condition.

Matrix sums

It has a wider capability to represent the input distribution.

Intelligent Database Systems Lab

Intelligent Database Systems Lab

projection error norm for BMU

the norm of the input vector

Intelligent Database Systems Lab

Related to the ASSOM

Its training equations are much simpler

Its input representation capability is broader

Experiments show that the new model has better performance than the ASSOM network.

Intelligent Database Systems Lab