Organizing a spectral image database by using Self-Organizing Maps

Research Seminar 7.10.2005Oili Kohonen

Motivation?

Image retrieval from conventional databases since 1990's ... many efficient techniques have been developed

However, efficient techniques for querying images from spectral image database does not exist.

Due to the high amount of data in the case of spectral images, the efficient techniques will be needed.

Spectral imaging?

Metameric imaging: cheap and practical way to achieve a color match.

Spectral imaging: needed to achieve a color match for all observers across the changes in the illumination.

Principle of SOM:

The Self-Organizing Map (SOM) algorithm:

Is an unsupervised learning algorithm.

Defines mapping from high-dimensional data into lower-dimensional data.

SOM:

Consists of arranged units (or neurons), which are represented by weight vectors.

Units are connected to each other by neighborhood relation.

Principle of SOM:

SOM Algorithm:

beginInitialize the SOMfor i = 1 : number of epochs

take input vector x randomly from the training data;find the BMU for x;update the weight vectors of the map;decrease the learning rate & neighborhood

function;end;

end;

Principle of SOM: finding the BMU

Mathematically the BMU is defined for input data vector, x, as follows:

Euclidean distance is a typically used distance measure.

Principle of SOM: updating the weight vectors

Learning rate: product of learning rate parameter & neighborhood function:

Principle of SOM: neighborhood function

Neighborhood function

h(t) has to fullfill the following two requirements:

It has to be symmetric about the maximum point (BMU).

It's amplitude has to decrease monotonically with an increasing distance from BMU.

Gaussian function is a typical choice for h(t)

Principle of SOM: Lattice structure

Lattice structures: hexagonal & rectangular

Searching Technique: Constructing histogram database

Train SOM

Find BMU for each pixel in an image

Generate BMU-histogram & normalize it by the number of pixels in an image

Repeat steps 2 & 3 for all images in a spectral image database

Save histogram database with the information of SOM-map

Searching Technique: making a search

Choose an image and generate its histogram.

Calculate the distances between the generated histogram and the existing histogram database.

Order images by these distances.

The results of the search are shown to user as RGB-images

Searching techniques:

One-dimensional SOM:

Searching techniques:

Two-dimensional histogram-trained SOM

Distance Calculations:

H1 & H2 are the compared histograms

L1 & L2 are the indices of max. values|

H3=(H1+H2)/2

Experiments:

One-dimensional SOM for unweighted images

One-dimensional SOM for images weighted by HVS-function

Two-dimensional SOM

From histogram data

From spectral data

Human Visual Sensitivity-function

(Unweighted images)

(Unweighted and weighted images)

The Used Database:

106 images: 61 components, spectral range from 400 nm to 700 nm at 5 nm interval.

Training of the SOMs:

10 000 spectra were selected randomly from each image.

2 000 000 & 4 000 000 epochs in ordering & fine tuning phases, respectively.

Unit sizes: 50 – chosen empirically 49 – to have comparable results with 1D-SOM 14*14 map in the case of histogram-trained SOM

Results: 1d-SOM, Unweighted images

Pure data Multiplied data

The distance measure: Euclidean distance

Results: 1D, Unweighted images

Energy

K-L

Peak

DPD

JD

Results: 1D, Weighted images

Energy

K-L

Peak

DPD

JD

Conclusions I:

The “structure” of the database is different for weighted and unweighted images.

The “best” results were got by using euclidean distance and Jeffrey divergence.

Importance of normalization?? * Better results with Euclidean distance & DPD * Worse results with Jeffrey divergence

Results: 2D, Unweighted spectral data

Euclidean

Energy

K-L

Peak

DPD

JD

Results: 2D, Weighted spectral data

Euclidean

Energy

K-L

Peak

DPD

JD

Conclusions II:

In the case of two-dimensional SOM better results are achieved by using non-weighted images.

When the weighted images are used, the use of 1D- SOM seems to be more reasonable.

Results: histogram-trained 2D-SOM

Euclidean

Energy

K-L

Peak

DPD

JD

Connections between images and histograms:

non-weighted

weighted

Past, Present & Future:

Past: What you have seen so far...

Present: Texture features in addition to color features

Future: Testing the effect of different metrics in ordering and fine-tuning phases (during the training of SOM)

Questions:

?Thank you for not asking any... =)