Modified Color Texton Histogram for Image Retrieval

Qin-Jun Qiu, Yong Liu, Da-Wei Cai, Jia-Zheng TanThree Gorges UniversityYichang, Hubei province

709225784@163.com

Abstract—In this paper, HSV based texton histogram

(HSV-TH) is proposed for content based image retrieval

(CBIR).The HSV-MT is proposed in contrast to the RGB

based texton histogram (RGB-TH). The proposed HSV-TH

method is based on Julesz’s textons theory, and it works

directly on nature images as shape descriptor and a color

texture descriptor. HSV-TH integrates the advantages of

co-occurrence matrix and histogram by representing the

attribute of co-occurrence matrix using histogram. The

retrieval results of the proposed method are tested on the

different image databases i.e. Corel 1000 (DB1) ,Corel-10,000

(DB2) and MIT VisTex(DB3). Two experiments have been

carried out for proving the worth of our algorithm. The results

after being investigate show that a significant improvement in

terms of their evaluation measures as compared to RGB-TH.

Key words-texton; image retrieval; texton histogram; the loc

al structure detection

I. INTRODUCTION With the development of digital image processing

technology, content-based image retrieval (CBIR) has become an active and fast-advancing research area in image retrieval [1]. In a typical CBIR , features related to visual content such as shape, color, and texture are first extracted from a query image, the similarity between the set of features of the query image and that of each target image ina DB is then computed, and target images are next retrieved which are most similar to the query image. Extraction of good features which compactly represent a query image is one of the important tasks in CBIR. Comprehensive and extensive literature survey on CBIR is presented in [2-4].

Many researchers have put forward various algorithms to extract color, texture, and shape features. Swain et al. proposed the concept of the color histogram in 1991 and also introduced the histogram intersection distance metric to

measure the distance between the histograms of images [5].Huang et al. used a new color feature called color correlogram [6] which characterizes not only the color distributions of pixels, but also spatial correlation of pair of colors.

Texture is another salient and indispensable feature for CBIR. Moghaddam et al. proposed the Gabor wavelet correlogram (GWC) for CBIR [7-8]. Birgale et al. [9] andSubrahmanyam et al. [10] combined the color (color histogram) and texture (wavelet transform) features for CBIR.

Jhanwar et al. [11] have proposed the motif co-occurrence matrix (MCM) for content based image retrieval. The MCM is derived from the motif transformed image which is calculated by dividing the whole image intonon-overlapping 2×2 pixel patterns. Heikkila et al. have used the LBP for interest region description [12]. Li et al. have used the combination of Gabor filter and LBP for texture segmentation [13]. Zhang et al. have proposed the local derivative pattern for face recognition [14]. Dr. Adivel et al have proposed a combination of gray level co-occurrence matrix and color co-occurrence matrix of image retrieval methods [15]; It can be combined with gray scale and color information. Atul gawande elzinis have proposed a new method to mining co-occurrence matrix description ability [16].

This paper presents a new feature extractor and descriptor, namely modified texton histogram (TH), for image retrieval. It integrates the advantages of the co-occurrence matrix and histogram by representing the attribute of co-occurrence matrix using histogram, and can represent the spatial correlation of color and texture orientation.

The organization of the paper as follows: In section 1, a brief review of image retrieval and related work is given.

2013 International Conference on Computer Sciences and Applications

978-0-7695-5125-8/13 $26.00 © 2013 IEEE

DOI 10.1109/CSA.2013.142

585

Section 2, presents the proposed system framework and similarity measure. Experimental results and discussions are given in section 3. Based on above work conclusions are derived in section 4.

II.THE TEXTON HISTOGRAM (TH) A. Color quantization in the HSV color space

It is well known that color information provides powerful information for image retrieval and object recognition, even in the absence of shape information. HSV color space could mimic human color perception well. In order to extract color information and simplify manipulation, in this paper the HSV color space is used and it quantized into 256 colors. In Section 2.3, the experiments demonstrated that HSV color space is well suitable for our framework. Given a color image of size M*N, we uniformly quantize the H, S and V channels into 16 bins, 4 bins, 4 bins, respectively. So that 256 colors are obtained. Denote by f(x,y) the quantized image, where x [0,1,…M-1], y[0,1,…N-1]. The each value of f(x,y) is ranging from 0 to 255.

B. Texton detectionThe concept of “texton” was proposed by Julesz[17]

more than 20 years ago and it is a very useful tool in texture analysis. In general, textons are defined as a set of blobs or emergent patterns sharing a common property all over the image; however, defining textons remains a challenge.

In this paper, four special textons types are defined ona 3×3 grid, as shown in Fig.1. Denote the four pixels as v1,v2, v3, v4, v5, v6, v7, v8 and v9. If four pixel highlighted in gray color have the same values, the 3×3 grid will form atexton. Those four special types of textons are denoted as Q1 Q2 Q3 and Q4 , respectively as shown in Fig.2.

The working mechanism of texton detection is illustrated in Fig. 3. In the color image C(x,y), The 3×3block is moved from left-to-right and top-to-bottomthroughout the image to detect textons with 3 pixels as thestep-length .If a texton is detected, the original pixel valuesin the 3×3 grids are kept unchanged. Otherwise it will havezero value. Finally, we will obtain a texton image, denotedby Q(x, y).

Fig.1 3×3 grid

Fig.2 Four special types of textons: (a)Q1;(b)Q2;(c)Q3;(d)Q4.

Original image (b) texton location and texton types

Fig. 3. Illustration of the texton detection process

C. Feature representationThe values of a texton image fare denoted as

f(P)=w w {0,1,…,W-1}. Denote by P1(x1, y1) and P2(x2,y2) two neighboring pixels, and their values are f (P1) =w1

and f (P2) =w2. In the texture orientation image θ(x,y), the

angles at P1 and P2 are denoted by θ(P1)=v1 and θ(P2)=v2.In texton image f, two different texture orientations may have the same color, while in texture orientation image θ(x,y), two different colors may have the same texture orientation. Denote by N the co-occurring number of two

v1 v2 v3

v4 v5 v6

v7 v8 v9

11 11 15 4 18 19 11 11 15 4 18 19 11 11 15 0 0 0

Q111 11 17 5 18 18 11 11 17 5 18 18 11 11 17 0 0 0

12 13 8 1 2 2 12 13 8 1 2 2 12 13 8 0 0 0

1 7 7 3 14 10 1 7 7 3 14 10 1 7 7 3 14 10

Q3 Q44 4 3 3 5 5 4 4 3 3 5 5 4 4 3 3 5 5

4 4 12 8 5 5 4 4 12 8 5 5 4 4 12 8 5 5

586

values v1 and v2 and by N’ the co-occurring number of two values w1 and w2. With two neighboring pixels whose distance is D, we define the TH [17] as follows:

� �( ), ( )H H f H �� (1)

1 1 2 2 1 21 1

1 2 1 2

{ ( ) ( ) || (| | )( ( , ))

( ) ( )N P v P v P P D

H f x ywhere P P v v� �

� �� � � � ��

� � � �

(2)1 1 1 2 2 1 2

1 11 2 1 2

{ ( ) ( ) || (| | )( ( , ))

( ) ( )N f P w f P w P P D

H x ywheref P f P w w

�� � � � ��

� � � � (3)The distance parameter of tuple histogram based on

D=1, that is, only considering spatial relationships between two adjacent pixels. H (f (P1)) can represent thespatial correlation between neighboring textureorientation by using color information, leading to a 64 dimensional vector. H (θ (P1)) can represent the spatialcorrelation between neighboring colors by using thetexture orientation information, leading to an 18dimensional vector. Thus in total TH uses a 64+18=82dimensional vector as the final image features in imageretrieval.

III.EXPERIMENTAL RESULTS AND DISCUSSIONS In order to analyze the performance of proposed

method for image retrieval, two experiments were conducted on three different databases (Corel 1000, Corel 10,000 and MIT VisTex). Results obtained are discussed in the following subsections.

Corel database contains large amount of images of various contents ranging from animal and outdoor sports to natural images. Ten categories are provided in the database namely Africans, beaches, buildings, buses,dinosaurs, elephants, flowers, horses, mountains and food. The database DB2 used in our experiment consists of 40 different textures. The size of each texture is 512×512.Each 512×512 image is divided into sixteen 128×128 non-overlapping sub-images, thus creating a database of 640 (40×16) images.

For each template image in the dataset, an M-dimensional feature vector T=[T1,T2,T3,…,T64] will be extracted and stored. Let Q=[Q1,Q2,Q3,…,Q64] be the feature vector of a query image, the distance metric

between them is simply calculated as

642

, 1

( , ) ( )i ii j

D i j T Q�

� �� (4)

The performance of the proposed method is measured in terms of average Precision and average Recall.

Table and summarizes the retrieval results of the proposed method (HSV-TH) and RGB-TH in terms of average retrieval precision and recall respectively. From table I, ,and Fig.5(a), (b) &(c),it is clear that the proposed method showing better performance compared to RGB-TH in terms of average retrieval precision and recall.

According to the retrieval of experimental data in Table I, in the HSV color space, the retrieval accuracy of proposed method than the retrieval performance in the RGB color space. Finally, we choose the HSV color space for image retrieval.

Table I Result of the proposed method in terms of precision (%) on

DB1 database

Category HSV-TH(%) RGB-TH (%)

Africans 81.2 47.9

Beaches 35.3 33.3

Buildings 64.6 50

Buses 93.8 91.7

Dinosaurs 100.0 100.0

Elephants 62.5 47.9

Flowers 100 88.9

Horses 70.8 77.8

Mountains 31 41.7

Food 45.8 66.7

Total 78.5 64.6

Table II Result of the proposed method in terms of precision (%) on

DB2 database

Category HSV-TH(%) RGB-TH (%)

Africans 45.12 32.19

Beaches 34.33 31.03

Buildings 30.46 22.42

Buses 68.53 51.41

Dinosaurs 90.63 81.12

587

Elephants 26.10 21.78

Flowers 75.32 66.51

Horses 61.55 60.72

Mountains 27.98 23.64

Food 44.69 26.27

Total 49.17 39.91

10 20 30 40 50 60 70 80 90 10035

40

45

50

55

60

65

70

75

80

Number of top matches

Ave

rage

Ret

rieva

l Pre

cisi

on(%

)

RGB-TH

HSV-TH

(a)

10 20 30 40 50 60 70 80 90 1005

10

15

20

25

30

35

40

45

50

55

Number of top matches

Ave

rage

Ret

rieva

l Rec

all(%

)

HSV-TH

RGB-TH

(b)

Fig.4: Comparison of proposed method HSV-TH with RGB-TH in

terms: (a) average retrieval precision, (b) average retrieval rate.

Figure 5 show two retrieval examples on the Core dataset. In Fig.5 (a) ,(b) and (c), the top 12 retrieval images show good match of texture and color to the query image. The two retrieval examples only use to validate that HSV-TH has the discrimination power of texture and color features, and do not suggest that all queries in the dataset can obtain such high retrievalaccuracy.

Fig 5(a). Query Results of proposed method on Corel database

Fig 5(b). Query Results of proposed method on Corel database

Fig 5(c). Query Results of proposed method on VisTex database

IV .CONCLUSION In this paper, a modified method, namely modified

color texton histogram (TH) is proposed, to describe image features for image retrieval. It is quite difference from the existing histograms, many of the existing histogram techniques merely statistics the pixels number or frequency. TH can represent the local structure about colors and edge orientations, and contain the spatial information between them. The vector dimension of TH is only 82, and the computational burdens of feature extracting in the proposed algorithm are also low, which

588

is very efficient for image retrieval. The results have demonstrated that it is efficient. It has good discrimination power of color, texture, shape features and spatial layout information.

ACKNOWLEDGEMENT This research was supported by Three Gorges

University Postgraduate Innovation Fund.

REFERENCES [1] Ritendra Datta, Dhiraj Joshi, Jia Li, et al. Image retrieval: ideas,

influences, and trends of the new age [ J ]. ACM Computing Surveys,

2008, 40 (2) : 1260.

[2] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain,

Content-based image retrieval at the end of the early years, IEEE

Trans. Pattern Anal. Mach. Intell., 22 (12) 1349–1380, 2000.

[3] M. Kokare, B. N. Chatterji, P. K. Biswas, A survey on current

content based image retrieval methods, IETE J. Res., 48 (3&4)

261–271, 2002.

[4] Ying Liu, Deng sheng Zhang, Guo jun Lu, Wei-Ying Ma, A survey

of content-based image retrieval with high-level semantics, Elsevier

J. Pattern Recognition, 40, 262-282,2007.

[5] M. J. Swain and D. H. Ballar, Indexing via color histograms, Proc.

3rd Int. Conf. Computer Vision, Rochester Univ., NY, (1991) 11–32.

[6] J. Huang, S. R. Kumar, and M. Mitra, Combining supervised

learning with color correlograms for content based image retrieval,

Proc. 5th ACM Multimedia Conf., (1997) 325–334.

[7] H. A. Moghaddam, T. T. Khajoie, A. H Rouhi and M. Saadatmand

T., Wavelet Correlogram: A new approach for image indexing and

retrieval, Elsevier J. Pattern Recognition, 38 (2005) 2506-2518.

[8] H. A. Moghaddam and M. Saadatmand T., Gabor wavelet

Correlogram Algorithm for Image Indexing and Retrieval, 18th Int.

Conf. Pattern Recognition, K.N. Toosi Univ. of Technol., Tehran,

Iran, (2006) 925-928.

[9] L. Birgale, M. Kokare, D. Doye, Color and Texture Features for

Content Based Image Retrieval, International Conf. Computer

Grafics, Image and Visualisation Washington, DC, USA, (2006) 146

– 149.

[10] M. Subrahmanyam, A. B. Gonde and R. P. Maheshwari, Color and

Texture Features for Image Indexing and Retrieval, IEEE Int.

Advance Computing Conf., Patial, India, (2009) 1411-1416.

[11] N. Jhanwara, S. Chaudhuri, G. Seetharamanc, and B.Zavidovique,

Content based image retrieval using motif co-occurrence matrix,

Image and Vision Computing 22,(2004) 1211–1220.

[12] M. Heikkila, M. Pietikainen, C. Schmid, Description of interest

regions with local binary patterns, Elsevie J. Pattern recognition, 42:

425-436, 2009.

[13] M. Li, R. C. Staunton, Optimum Gabor filter design and local

binary patterns for texture segmentation, Elsevier Pattern

recognition, 29: 664-672, 2008.

[14] B. Zhang, Y. Gao, S. Zhao, J. Liu, Local derivative pattern versus

local binary pattern: Face recognition with higher-order local pattern

descriptor, IEEE Trans. Image Proc., 19 (2): 533-544, 2010.

[15] VADIVELA, SURALS, MAJUMDARAK. An Integrated Color

and Intensity Co-occurrence Matrix [J]. Pattern Recognition

Letters, 2007 (28) : 974 - 983.

[16] GELZINISA, VERIKASA, BACAUSKIENEM. Increasing the

discrimination power of the co-occurrence matrix based features [J].

Pattern Recognition, 2007, 40 ( 12) : 2367- 2372.

[17] Julesz B. Textons, the Elements of Texture Perception and Their

Interactions [J]. Nature, 1981, 290(5802): 91-97.

[18] G.-H. Liu, L. Zhang, et al. Image retrieval based on multi-texton

histogram. Pattern Recognition, 2010, 43(7): 2380-2389.

589