Content-Based Image Indexing Joel Ponianto Supervisor: Dr. Sid Ray.

Content-Based Image Indexing

Joel Ponianto

Supervisor:

Dr. Sid Ray

Outline

Introduction to Content-Based Image Indexing Image’s Features Extraction Tree Structure System Model Retrieval Approach Experiment Results Conclusion

Introduction to Content-Based Indexing

Content-Based Image Indexing (CBII) is an interrelated issue with Content-Based Image Retrieval (CBIR).

CBIR depends on CBII and vice versa. CBIR focus on how to retrieve image

accurately and efficiently. While CBII concern with how to support

retrieval process.

Introduction to Content-Based Image Indexing Cont…

CBiI as pre-process of CBIR sequences. Cannot ignore retrieval process to create good

indexing structure. The idea of indexing is similar with a library Every book has a unique id Every book has properties


Examples: title, author, publisher, etc Those properties are used to search the book. People know it as “keyword” Similar idea with images, however not that

simple. Cannot represent an image with simple text.

(can but not make sense)


How to represent an image? By using its properties such as, colour, shape,

texture and others. Choose which properties need to be extracted

for indexing purpose ( and also retrieval). Also choose which method to extract those

properties / features.

Image’s Features Extraction Cont…

Colour, shape and texture have their own sub-features.

Colour: grey level, RGB/HUE value, grey sigma, local histogram and average colour value.

Shape: area, centroid, circularity and moment invariant.

Texture: contrast, orientation and anisotropy.


The selection of features is also effected by the data set.

what we want to achieve at the retrieval stage is effected by the data set.

If the data set is full of houses’ image and a user want to look for a car image.

Try to select features that can differentiate each class in the data set.


For this project I select the following features:– Colour Sigma (Global)– Edge density (Global)– Colour Average (Global)– Boolean edge Density (Global)– Edge Direction (Global)– Region area (Region)– Moment invariant (Region)– Grey level (Region)


Colour Sigma– Find the standard deviation (σ) of the image, for

each colour layer.


Edge Density– Enhance the pixels that belong to the edges and

boundaries by using a standard edge detector. Pixels far from edges will drop to 0 and those near to an edge will increase to max. calculate the mean pixel value of the resultant image.

Colour Average– Sum all the pixel value for each colour layer and

divide by the number of pixel.


Boolean Edge Density– From above edge density, the image is thresholded

so that what could be called edge pixels are white (1) and non-edge pixels are black (0). Count white pixel in the image.

Edge Direction– With some edge detection (Sobel Operator), allow

us to make a crude estimation of a edge direction for particular region.


Area, Grey Value and moment invariant– These features is calculate on regional basis.– The region is calculated with combination of “k-

mean clustering” and “Connected Component labelling Algorithm”

– Calculate a grey level value of an image and perform the k-mean clustering.

– Use the connectivity algorithm to group similar grey value by its location.


http://www.cis.rit.edu/class/simg782.old/talkMoments/momentEquations.html

I use the first four of seven invariant moment for this project.


Quantisation – To be suitable for computer processing and features

extraction (colour), an image must be digitized in amplitude.

– The idea is to reduce the colour space while gaining the ability to localize colour information spatially.

– this project applies quantisation at HSV colour space.


RGB to HSV– Let RGB values ranged from 0 to 1 and MIN/MAX corresponds with

RGB values.


HSV to RGB– H range from 0 - 360– V and S range from 0 – 1– If S == 0 then RGB = V

Else use next formula

Tree Structure

There are many choices of tree structures that can handle multi-dimensional space. Such as R-Tree, R*-Tree and Vp-Tree

We look at R-Tree tree structure:– This project used R-Tree to simplify the

computation.– Other tree structures can be use on the system.

Tree Structure Cont…

R-Tree (Antonin Guttman)– A R-Tree is a height balance tree and all leaves are on the same

level. – Root node has at least two children unless it is the leaf node. – Every non-leaf node contains between m and M entries unless it is

the root. – For each entries (I, childnode-pointer) in a non-leaf node, I is the

smallest rectangle that spatially contains all rectangles in its child nodes.

– Every leaf node contains between m and M index records unless it is the root.

– For each index record (I, tuple-identifier) in a leaf node, I is the smallest rectangle that spatially contains the n-dimensional data object represented by the indicated tuple.

Tree Structure Cont…

System Model

Put into data base

Original Image

Quantised Image

K-mean clustering

Binary Threshold Apply Global features extraction.

Connected Component labelling

Apply Region features extraction. Insert into tree

structure

System Model Cont…

The System input around 300 images into the data base.

Those images is divided into 10 different classes: animal, car, flower, face, fruit, house, lake, mountain, plane and sunset.

Store into persistence storage.


In the “binary threshold” stage, I attempt to separate the background image with the object.

Although this stage is very weak, but in some images. The result can be helpful (and possible the other way around).


Binary Threshold good result


Binary Threshold bad result

Retrieval Approach

Query sequence

Query Image

Global Extraction

Region Extraction

Pre-process stage

Find similarity with data set

Display the result in ascending order

Retrieval Approach Cont…

For finding similarity, I use Euclidean distance measure formula:

Where:– p is the database image– q is the query image– Pi is the database images ith features

– Qi is the query’s ith features

– n is the number of features– W” is the weight for ith feature

W”i


w’i is the weight of feature i

from relevant images

(σi) is the standard deviation of

feature i from relevant images w’t is the total weight of feature I

w”t is the normalised weight


Gaussian Normalisation (for feature normalization):– d’(fi,fj) is the similarity of image fi and fj,

range in [-1, 1]

– σij and μij are the standard deviation and

mean of each feature respectively.– d”(fi,fj) is to make d’(fi,fj) in range [0, 1]

Experiment Result

Go to Excel file m1-m8 only use global features m3 uses colour avg, colour sigma and edge

density m2 uses colour avg and colour sigma m8 uses colour sigma and edge density m9 use region features + m3

Conclusion

Indexing depend on retrieval and vice versa No universal system / method for indexing or

retrieval. We can try to develop something that robust. Indexing base on regional features give better

result then global features. With more time, more result can be produced.

Reference

• Kompatsiaris, I., Triantafillou, E. and Strintzis, M. G., “Region-Based Color Image Indexing and Retrieval”, 2001

• Parker, J. R., Behm, B., “Use of Multiple Algorithm in Image Content Searches”, International Conference on Information Technology: Coding and Computing (ITCC’04) Volume2 p.246.

• Smith, J. R., Chang, S., “Single Color Extraction and Image Query”, International Conference on Image Processing (ICIP-95), Washington, DC, Oct, 1995.

• Park, J. M., Looney, C. G., Chen, H. C., ”Fast Connected Component Labeling Algorithm Using A Divide and Conquer Technique”, Technical Report, 2000

• Chiueh, T., "Content-Based Image Indexing," in Proceedings of International Very Large DataBase Conference, VLDB '94, Santiago, Chile, September, 1994.

• Gonzalez, R. C. and Woods, R. E., “Digital Image Processing”, 1993, Addison-Wesley Publishing Company, inc, 3rd edition.

Content-Based Image Indexing Joel Ponianto Supervisor: Dr. Sid Ray.

Documents

Transcript of Content-Based Image Indexing Joel Ponianto Supervisor: Dr. Sid Ray.