Performance Analysis of Color Components

in Histogram-Based Image Retrieval

Presented by

Tienwei TsaiDepartment of Information Management

Chihlee Institute of Technology, 2006/6/26

Outline

1. Introduction 1. Introduction

2. Color Space2. Color Space

3. Color Histogram3. Color Histogram

4. Distance Measurement4. Distance Measurement

5. Experimental Results5. Experimental Results

6. Conclusions6. Conclusions

1. Introduction

• Two approaches for image retrieval: – query-by-text (QBT): annotation-based image retrieval

(ABIR)– query-by-example (QBE): content-based image

retrieval (CBIR)

• Drawbacks of ABIR:– time-consuming and therefore costly. – human annotation is subjective. – some images could not be annotated because it is

difficult to describe their content with words.

• Standard CBIR techniques can find the images exactly matching the user query only.

• In QBE, the retrieval of images has been done via the similarity between the query image and all candidates on the image database. – Euclidean distance

• CBIR is a technology to search for similar images to a query based only on the image pixel representation. – It is necessary to devise a means of describing the

location of each pixel and its intensity. • How to choose a suitable color space is a critical

problem in image retrieval. – In this paper, we investigate the appropriateness of the

three well-known color spaces, i.e., RGB, YUV, and HSV, for CBIR.

• In our demo system, – each image is first transformed from the standard RGB

color space into the YUV (or HSV) space. – then, the histogram for each component (e.g., luminance

Y, blue chrominance U, and red chrominance V) of the image is obtained, which can be served as the color feature of the image.

• In the image database establishing phase– the features of each image are stored;

• In the image retrieving phase– the system compares the features of the query image with

those of the images in the database, using the Euclidean distance metric and find out good matches.

2. Color Space

• A color space is a model for representing color in terms of intensity values.

• There exist many models:– RGB (Red, Green, and Blue), – CMYK (Cyan, Magenta, Yellow, and Black

Key), – YUV (Luminance and Chroma channels), and – HSV (Hue, Saturation, and Value ), etc.

2.1 RGB Color Space

• For a gray-level digital image– It can be defined as a function of two variables, f(x,

y), where x and y are spatial coordinates, and the amplitude f at a given pair of coordinates is called the intensity of the image at that point.

• For a color image– each pixel (x, y) consists of three components: R(x,

y), G(x, y), and B(x, y), each of which corresponds to the intensity of the red, green, and blue color in the pixel, respectively.

2.2 YUV Color Space

• Originally used for PAL (European "standard") analog video

– The Y primary was specifically designed to follow the luminous efficiency function of human eyes.

– Chrominance is the difference between a color and a reference white at the same luminance.

• The following equations are used to convert from RGB to YUV color space:

,114.0587.0299.0 BGRY ),(492.0 YBU ).(877.0 YRV

2.3 HSV Color Space

• The HSV stands for the Hue, Saturation, and Value based on the artists (Tint, Shade, and Tone).

– As hue varies from 0 to 1.0, the corresponding colors vary from red, through yellow, green, cyan, blue, and magenta, back to red.

– As saturation varies from 0 to 1.0, the corresponding colors (hues) vary from unsaturated (shades of gray) to fully saturated (no white component).

– As value, or brightness, varies from 0 to 1.0, the corresponding colors become increasingly brighter.

• The conversion formula from RGB to HSV color space is as follows:

))(()(

2/)]()[(cos

2

1

BGBRGR

BRGRH

),/()],,[min(31 BGRBGRS

.3/)( BGRV

3 Color Histogram

• The color histogram for an image is constructed by discretizing (or quantizing) the colors within the image and counting the number of pixels of each color.

• More formally, it is defined as

• For gray-scale images these are 2-D vectors. – One dimension gives the value of the gray-level and the other

the count of pixels at the gray-level. • As for color images

– each color channel can be regarded as gray-scale images. • More generally, we can set the number of bins in the

color histograms to obtain the feature vector of desired size.

),,,(Pr),,(,, zZyYxXobNzyxh ZYX

• To obtain the color feature of an image, some color components can easily be analyzed according their histogram type.

– One can realize the brightness of an image by checking the histogram type of the color components such as G (for RGB), Y (for YUV), and V (for HSV);

• One can realize the dominant color of an image according to the following heuristic rules:

1. if the U component is high key, then the dominant color is blue; 2. if the U component is low key, then the dominant color is yellow; 3. if the V component of YUV space is high key, then the dominant color

is red; 4. if the V component of YUV space is low key, then the dominant color

is green; 5. if the H component is low key, then the dominant color is red; 6. if the H component is midtone, then the dominant color is green; 7. if the H component is high key, then the dominant color is blue, and

so on.

4. Distance Measurement

• To decide which image in the image database is the most similar one with the query image, we have to define a measure to indicate the degree of similarity.

• Therefore, the distance (or dissimilarity) between a feature vector Fm of the query image and that of an image in the database is based on the distance function.

• In our approach, the distance between two vectors is calculated on the basis of the sum of squared differences (SSD).

• Assume that and represent the mth feature of the query image Q and an image X in the database, respectively; each feature comes from the color histograms.

• Then, the distance is

,])[][(),( 21

0

iFiFFFd xmqm

K

ixmqmm

5. Experimental Results

• We evaluated performance on a test image database, which was downloaded from the WBIIS.

• It is a general-purpose database including 1,000 color images. The images are mostly photographic and have various contents, such as natural scenes, animals, insects, building, people, and so on.

• In the experiment, an image retrieval demo system is built to test the three color spaces and their components .

Figure 2. The GUI of our demo system.

• To compare the three color spaces and their components in a quantitative manner, five classes of query images, referring to

– white owl (5 images), pumpkins (4 images), red apples (3 images), green apples (3 images), and deer (9 images), are served as the benchmark queries.

• To assess the ground-truth relevance score to each image for each benchmark query, each target image in the collection is assigned a relevance score as follows:

– 1 if it belonged to the same class as the query image, – 0 otherwise.

• The process was repeated for all the relevant images and an overall average retrieval effectiveness was computed for each of the color component and each of the query example.

• The overall average relevance score in top n was computed by averaging the individual values in each top n. The bin number used for each color component is 5.

• We can conclude that:– The HSV and RGB color spaces perform

well in fine matching; however, the YUV color space performs better in coarse classification.

– Even though the B and U components are less sensitive to human vision, they are quite important in terms of the effectiveness of the histogram-based retrieval.

– The V component performs well in coarse classification.

6. Conclusions

• In this paper, we investigate the appropriateness of the three well-known color spaces, i.e., RGB, YUV, and HSV, for histogram-based color image retrieval.

• An image retrieval demo system is built to make it easy to test the retrieval performance.

• We have performed experiments on a database with 1000 images, using the Euclidean distance metric.

• The results show that the HSV and RGB color spaces perform well in fine matching; however, the YUV color space performs better in coarse classification.

• Moreover, even though the B and U components are less sensitive to human vision, they are quite important for the histogram-based retrieval.

Thank You !!!