Color Texture Classication by Integrative Co-Occurrence Matrices

7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices

1/12

Pattern Recognition 37 (2004) 965976

www.elsevier.com/locate/patcog

Color texture classication by integrativeCo-occurrence matrices

Christoph Palm1

Institute of Medical Informatics, Aachen University of Technology, D-52057 Aachen, Germany

Received 26 November 2002; received in revised form 4 August 2003; accepted 18 September 2003

Abstract

Integrative Co-occurrence matrices are introduced as novel features for color texture classication. The extended

Co-occurrence notation allows the comparison between integrative and parallel color texture concepts. The information prot

of the new matrices is shown quantitatively using the Kolmogorov distance and by extensive classication experiments on

two datasets. Applying them to the RGB and the LUV color space the combined color and intensity textures are studied

and the existence of intensity independent pure color patterns is demonstrated. The results are compared with two baselines:

gray-scale texture analysis and color histogram analysis. The novel features improve the classication results up to 20% and

32% for the rst and second baseline, respectively.

? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

Keywords: Color texture; Co-occurrence matrix; Integrative features; Kolmogorov distance; Image classication

1. Introduction

Texture analysis is one of the main topics in the eld of

digital image processing. Texture is described as a pattern

with some kind of regularity. The approaches of mathemat-

ical modeling are grouped into structural, statistical and sig-

nal theoretic methods [1]. Structural methods are based on a

more or less deterministic arrangement of textural elements

(texels). They are mainly used in industrial quality control,

where artical patterns are regular and dierences betweenmodel and reality indicate failures [2,3]. Statistical meth-

ods dene textures as stochastic processes and characterize

them by a few statistical features. Most relevant statistical

approaches are Co-occurrence matrices [4], Markov random

elds [5] and autocorrelation methods [6]. Signal theoretic

approaches focus on periodic pattern resulting in peaks in

the spatial frequency domain, e.g. Gabor ltering [7,8] and

wavelet decomposition [9].

1 Christoph Palm is now with the Aixplain AG, Monheimsallee

22, D-52062 Aachen, Germany.

E-mail address: [email protected] (C. Palm).

Beside texture, color is an important issue not only in hu-

man vision but in digital image processing where its impact

is still rising. In contrast to intensity, coded as scalar gray

values, color is a vectorial feature assigned to each pixel in

a color image. The mathematical dierence between scalars

and vectors for gray and color values, respectively, demands

a careful transfer of methods from the gray-scale to the color

domain. Although the use of color for texture analysis is

shown to be advantageous, the integration of color and tex-

ture is still exceptional. We dierentiate between parallel,sequential and integrative methods (Section 2).

Several studies favor the statistical texture modeling and

in particular Co-occurrence matrices in the gray-scale do-

main [10]. Nevertheless, just few approaches were made

to transfer Co-occurrence matrices to color images [35]. In

this paper, we study existing approaches for color texture

classication by Co-occurrence features and propose novel

matrices and features to allow the exploitation of the color

information.

After the introduction of general color texture con-

cepts (Section 2) and details of the most relevant color

spaces (Section 3), we shortly repeat the concept of

0031-3203/$30.00 ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

doi:10.1016/j.patcog.2003.09.010
mailto:[email protected]:[email protected]


2/12

966 C. Palm / Pattern Recognition 37 (2004) 965 976

Co-occurrence matrices for gray-scale images (Section 4).

This concept is extended to the color domain (Section

5) dening single- and multi-channel matrices, respec-

tively. The description of the experimental setup (Section

6) is followed by the results and a detailed discussion

(Section 7).

2. Combining color and texture

We group the approaches combining color and texture

into parallel, sequential and integrative approaches.

The parallel concept for color texture analysis separates

the processing of both phenomena. Color is measured glob-

ally according to the histogram ignoring local neighboring

pixels. Texture is characterized by the relationship of the in-

tensities of neighboring pixels ignoring their color. The re-

sults of both analyses are combined subsequently to a feature

vector (Fig. 1, top). The parallel approach is advantageous,because the known methods of gray-scale texture analysis

as well as pixel based color analysis can be applied directly.

Up to now, parallel algorithms are most commonly used for

image retrieval applications with color and texture as sep-

arated discriminative features [11]. However, the view on

texture as a pure intensity based structure is very simplied

and disregards, for instance, colored texture primitives with

constant intensity.

Sequential approaches use color analysis as a rst step of

a process chain. Clustering the color histogram, a partition-

ing of the image is obtained. These consecutive numbered

segments contain color information, but they are just repre-sented by their indices and, hence, by scalars. Subsequently,

the index pattern is processed as gray-scale texture (Fig. 1,

middle). The sequential approach is related to the structural

texture model, where the pattern is composed by (color)

primitives. Hauta-Kasari et al. used it for industrial qual-

ity control [12], Song et al. especially for defect detection

in granite images [13], and Kukkonen et al. for inspection

of ceramic tiles [14]. These examples show the usefulness

of the sequential approach. Nevertheless, it is based on the

segmentation procedure, which depends on several parame-

ters and, therefore, gives no reproducible results. Addition-

ally, the assumption of colored texture primitives cannot be

generalized.The third way of combining the analysis of color and

texture is called integrative, because the information de-

pendency between both image features is taken into ac-

count. Integrative methods can be further dierentiated into

single- and multi-channel strategies (Fig. 1, bottom). The

single-channel methods apply the gray-scale texture anal-

ysis on each color channel separately. The color informa-

tion is used indirectly restricting the intensity pattern to the

wavelength interval associated with the color channel. The

advantage of the single channel approach is the easy adap-

tion of known methods based on the gray-scale domain.

Recently, some of these were compared with the paral-

grayscale image

color image

analysis

color histogram

feature

feature

analysis

segmen-

analysistation

color image color histogram label image

feature

color image

featureanalysis

feature

analysis

analysis

feature

feature

analysis

feature

analysis

analysis

feature

single channel

multi channel

color channels

Fig. 1. Top: Illustration of the parallel concept for color texture

analysis. Textural features and histogram features are strictly sep-

arated. Middle: Illustration of the sequential concept for color tex-

ture analysis. After the color segmentation of the histogram the

texture features are determined on basis of the segment indices.

Bottom: Illustration of integrative single- and multi-channel color

texture analysis. Both methods take color and texture into account.

lel concept and showed signicantly improved results [15].Integrative multi-channel texture analysis handles two (or

more) channels simultaneously. These approaches have al-

ready been proposed for well known textural features like

Markov random elds [16], wavelet [17], Gabor lters [18

21], and autocorrelation features [22], showing encouraging

results.

Up to now, Co-occurrence matrices are just adapted for

sequential [12,14] and integrative single-channel color tex-

ture analysis [15]. The idea of multi-channel Co-occurrence

matrices was rstly addressed by Rosenfeld et al. in 1982,

but found to be computationally cumbersome [23]. In this

paper, we rstly extend the Co-occurrence approach to


3/12

C. Palm / Pattern Recognition 37 (2004) 965 976 967

integrative multi-channel color texture features. Addition-

ally, we introduce an information measure to describe the

supplementary information content of the multi-channel in

comparison to the single-channel approach. The extensive

experimental evaluation of the features in a classication

task takes two color spaces into consideration. The results

show the advantages of the multi-channel approach and

demonstrate the existence of non-intensity based pure color

textures.

3. Color spaces

To study the relation of intensity textures, pure color tex-

tures and combinations of both, respectively, the represen-

tation of intensity within the color space is important. The

RGB (red, green, blue) color space is a projection of the

cumulated intensity of reected light within a dened range

of wavelength to the axes of a three-dimensional space.Certainly, the RGB values are not only dependent on the

reectance properties of the observed pattern but on the il-

luminant and the recording characteristics of the camera.

The problems of color constancy and device independency

are studied in detail elsewhere [22,24] and are not topic of

this paper. Because of the wavelength-dependent intensity

cumulation each color band is inherently a combination of

color and intensity. Additionally, a strong correlation be-

tween the color channels is reported [25].

De-correlating the RGB color bands and separating color

and intensity yields the LUV color space [26]:L

U

V

= HLUV

R

G

B

=1

6

2

2

2

2 1 10

3

3

R

G

B

: (1)Tan et al. reported improving results for color histogram

analysis using LUV [26]. Independently from the idea of

mathematical de-correlation, a similar color space is denedmodeling human opponent color vision [27]. Additionally,

the LUV space is strongly related to the complex color space

since the color planes spanned by UV and the complex col-

ors are equal [28]. With RGB and LUV we study color

spaces where intensity and color are combined or separated,

respectively.

4. Gray-scale Co-occurrence matrices

In contrast to histograms, Co-occurrence matrices

(CMs) characterize the relationship between the values of

d

Fig. 2. Illustration of the concept of gray-scale Co-occurrence ma-

trices. The Co-occurrence matrices within one octant are combined

to one mean meatix to be more robust for long distances d.

neighboring pixels [4]. Therefore, they represent a secondorder statistical measurement. We denote the values of a

gray-scale image f as w {0; : : : ; W 1} with f(p) = w.The pixel position is given by p = (m; n) with m; nZ. Thedistance vector d is noted in polar coordinates (d; %) with

discrete length and orientation d; %N, respectively, com-puted from linear coordinates applying the polar transform

and an appropriate truncation. The probability Pr of two val-

ues w and w co-occurring with pixel positions related by d,

denes the cell entry (w1; w2) of the CM Cd:

Cd (w; w)

:= Pr(f(p1) = w f(p2)

= w| |p1 p2| = d): (2)Hence, Cd is a symmetric two-dimensional neighborhood

histogram, which depends on d. To study the eect of

changing d to the classication results, we extend Cd to

long-distance CMs. Fig. 2 visualizes the idea to build the

mean CM for each octant of a discrete circle approxima-

tion. The single CMs within the octant are summed up and

normalized keeping d constant. Because of the symmetric

denition of CM only four mean CMs remain. Neverthe-

less, with (W

W) they are high dimensional. To reduce

the large amount of data represented by CMs, two methods

are applicable:

quantize the image into few values as a preprocessing step extract special features to describe CM data as a

post-processing method

In this work we applied the last approach and used the matrix

features introduced by Haralick et al. [4]. Unfortunately,

some of the 14 features show redundancies, but it is not yet

clear, which of them can be ignored. Therefore, we selected

eight features (homogeneity, contrast, correlation, variance,

inverse dierence moment, entropy, correlation I, correlation


4/12


II), that build the set union used in [10,15,29]. They are well

distributed over the four feature groups dened in [10].

The Cartesian product for the selected Haralick features

generates a (8 4)-space for the rotationally variant CMfeatures. To be rotationally invariant, the mean and the vari-

ance of the orientation-dependent features are determined

separately. Overall, the gray-scale CM features (GCFs) are

vectors of size 8 2.

5. Integrative color Co-occurrence matrices

According to our categorization (Section 2), in the follow-

ing two integrative extensions of gray-scale CMs to color

CMs are proposed. The single-channel Co-occurrence ma-

trices (SCMs) consist of gray-scale CMs successively ap-

plied to separated color channels. The correlation between

textures of dierent color channels is captured by novel

multi-channel Co-occurrence matrices (MCMs).

5.1. Single-channel Co-occurrence matrices

5.1.1. Background

For the denition of SCMs, we assume a limited

K-dimensional Cartesian space like RGB and LUV with

K = 3. A SCM SCkd corresponds to Cd applied to the kth

color channel fk of the image f = (f1; : : : ; fK)T:

Pr(wk

; wk

)

:= SCkd(w; w)

:= Pr(fk (p1) = w fk(p2) = w| |p1 p2| = d): (3)

The joint probability Pr(wk; wk) indicates the adjacency

of w and w on the same channel k. The corresponding

rotational invariant single-channel Co-occurrence features

(SCFs) consist of K feature vectors SCFk, where SCFk is

analog to GCF according to k: SCF = SCF1 SCFK.Following Section 3, RGB combines intensity as well

as color information. Therefore, RGB-based SCFs contain

combined color and intensity texture information. Recently,

Drimbarean et al. showed RGB-based SCF-like features tooutperform GCFs [15]. In contrast to RGB, LUV separates

intensity and color strictly. Hence, SCF1 represents pure

intensity texture and equals GCF. In contrast, pure color

textures are modeled by SCF2 and SCF3 according to the

channels U and V, respectively.

5.1.2. Information measurement

The quantication of the information prot of CMs in

comparison to common histograms is based on the Kol-

mogorov distance [30], which measures the dierence be-

tween two probability distributions. The one-dimensional

histogram Pr(wk

) for each channel k is derived from the

Image Number

Distance

1

0 30

0.2

0.8

1

Image Number

Distance

0 30

0.2

0.8

Image Number

Distance

1

0 30

0.2

0.8

Image Number

Distance

1

0 30

0.2

0.8

Fig. 3. Top left: Kolmogorov distance D(SCkd ) for single-channel

Co-occurrence matrices with k = R (white), k = G (black),

and k = B (gray). The image numbers follow the ascending

order of D(SCGd ). Bottom left: D(SCkd ) with k = L (black)

k = U (gray), and k = V (white). The image numbers fol-low the ascending order of D(SCLd ). Top right: D(MC

k1 ;k2d )

for multi-channel Co-occurrence matrices with (k1; k2) = (R; G)

(white), (k1; k2) = (R; B) (black), and (k1; k2) = (G; B) (gray). The

image numbers follow the ascending order of D(MCR;Bd ). Bottom

right: D(MCk1 ;k2d ) with (k1; k2) = (U; V) (black) (k1; k2) = (L; U)

(gray), and (k1; k2) = (L; V) (white). The image numbers follow

the ascending order of D(MCU;Vd ).

corresponding SCkd by summing up the columns:

Pr( wk

) =

Wk1

w=0 SCkd(w; w): (4)

In the case of stochastic independency ofd-adjacent values,

the joint probability Pr(wk; wk) is estimated by the multi-plicative combination of the univariate densities Pr(wk) and

Pr( wk):Pr(wk; wk) = Pr(wk) Pr( wk): (5)In contrast, the actually measured density Pr(wk; wk) is re-

lated to the conditional probability Pr(wk| wk):Pr(w

k; w

k) = Pr(w

k| wk) Pr( wk): (6)The Kolmogorov distance D(SCkd) as dierence betweenPr(wk; wk) and Pr(wk; wk) is equal to the dierence betweenrandom adjacency and the actual adjacency of w and w.Therefore, it is a measure of stochastic dependency and,

consequently, a measure of information gain of the CM in

contrast to the univariate histogram:

D(SCkd) =

1

2

Wk1w=0

Wk1w=0

|Pr(wk; wk) Pr(wk; wk)|: (7)With the help of D(SCkd), the general information prot of

CMs specic to the color channels is determined. Fig. 3

(left) shows the Kolmogorov distance values of 30 images

of the VisTex database (see Section 6) corresponding to

RGB and LUV.


5/12


6/12


Fig. 3 (right) shows D(MCk1 ; k2d ) for the natural color

textures of the VisTex database (see Section 6) for RGB

and LUV, respectively. According to RGB, the values

range from 0.1 to 0.7. Considering the baseline of zero for

stochastic independency of color and texture, this range

indicates clearly the high information prot using the novel

multi-channel Co-occurrence features in contrast to any

parallel method. The similarity of D(MCk1; k2d ) for dierent

color channels regarding to each image separately allows

their interpretation as a global color texture feature rather

than a channel-specic feature.

In contrast to RGB, the curves for dierent channel com-

binations of LUV vary signicantly. Whereas D(MCk1 ;k2d )

for the intensity independent combination (k1; k2) = (U; V)

reaches values in [0:7; 1:0], the distances according to the

combinations of color and intensity channels are mostly less

than 0.3. Therefore, the information prot analyzing inten-

sity independent color textures is very high and the existence

of pure intensity independent color texture obvious.

6. Experimental evaluation

Studying the Kolmogorov distance, we already showed

the existence of intensity independent color textures as well

as the usefulness of the integrative color texture analysis.

These results will be conrmed by experimental evaluation.

6.1. Experimental setup

To allow the generalization of the classication results,

two dierent image sets are used. From the VisTex dataset

[31] of MIT we choose 30 images of size 512512 (Fig. 4),each of which contains one natural colored texture. One tex-

ture class is built splitting one image into 64 non-overlapping

sub-images of size 64 64. Therefore, inter-class variationcorresponds to local variation of the full-size images. This

dataset is here addressed by DS1. The second dataset called

DS2 contains images of natural barks [32], built up in the

context of the PhD thesis of Lakmann [33]. The images con-

sist of one type of bark and background. Each image shows

a dierent tree, which is previously classied into one of six

classes (Fig. 5). Therefore, the inter-class variation reectsthe natural surface aberration of trees of the same kind. Each

class consists of 68 images of size 384 256 yielding acollection of 408 images. Since we apply an image classi-

cation not a texture segmentation, the image border is ex-

cluded dening a Region-of-Interest of xed size 300200located at the image center.

For classication, the leaving-one-out method is used to

guarantee strict separation of test and training set in com-

bination with the maximization of the number of training

images. A 5-Nearest-Neighbor classier using the Euclid-

ian vector distance was employed. To achieve a compara-

ble range of values for dierent features, a normalization

Fig. 4. Images of the VisTex database (leftright, updown): Bark0,

Bark4, Bark6, Bark8, Bark9, Brick1, Brick4, Brick5, Fabric0, Fab-

ric4, Fabric7, Fabric9, Fabric11, Fabric13, Fabric16, Fabric17, Fab-

ric18, Food0, Food2, Food5, Food8, Grass1, Sand0, Stone4, Tile1,

Tile3, Tile7, Water6, Wood1, Wood2 [31].

Fig. 5. Images of the BarkTex database, one example for each

class (leftright, updown): birch, beech, spruce, pine, oak,

robinia [32].


7/12


Table 1

The notation dened in this table will be needed for understanding the following tables of classication results

Space Feature vector Abbreviation E

gray GCF m(L) 16

RGB SCF m(R), m(G), m(B) 16

RGB MCF m(RG), m(RB), m(GB) 16RGB HCF m(RG0), m(RB0), m(GB0) 8

LUV SCF m(L), m(U), m(V) 16

LUV MCF m(LU), m(LV), m(UV) 16

RGB m(R)m(G)m(B) m(RGBSC) 48

RGB m(RG)m(RB) m(GB) m(RGBMC) 48

RGB m(RGBSC) m(L) m(RGBSC; L) 64

RGB m(RGBMC)m(L) m(RGBMC; L) 64

RGB m(RG0)m(RB0)m(GB0) m(RGB0) 24

RGB m(RGB0)m(L) m(RGB0 ; L) 72

RGB m(RGB0)m(RGBSC) m(RGB0; SC) 72

RGB m(RGB0)m(RGBMC) m(RGB0; MC) 96

RGB m(RGBSC

) m(RGBMC

) m(RGBSC; MC

) 96

RGB m(RGB0)m(RGBSC)m(RGBMC) m(RGB0; SC; MC) 120

LUV m(U)m(V) m(UV) m(UVII) 48

LUV m(LU)m(LV)m(L) m(LUVID) 48

LUV m(UVII) m(LUVID) m(LUVII; ID) 96

The gray-scale Co-occurrence features, single-channel Co-occurrence features, multi-channel Co-occurrence

features and histogram Co-occurrence features are here noted as the feature vectors GCF, SCF, MCF and HCF,

respectively. The abbreviations of the feature vectors m make clear, which color channels are involved. These

vectors are combined by Cartesian product . The indices MC, SC and 0 indicate multi-channel Co-occurrence,

single-channel Co-occurrence and histogram features, respectively, in the RGB color space. For LUV, the

feature vectors are combined to intensity independent (II) and intensity dependent vectors (ID). E symbolizes

the dimension of the feature space.

according to variance e and mean e of each dimension eis done separately exploiting the training data TR:

m

e =me e

e(12)

with

e =1

M

mTR

me; e =

1

M 1

mTR

(me e)2: (13)

The eth element of an E-dimensional feature vector m and

the corresponding normalized element is denoted with meand me, respectively.

The optimal parameterd of a CM depends on the resolu-

tion of the texture. Zucker et al. developed a method to de-termine this optimal distance [34]. However, this optimum

refers to a distinct texture and one single color channel. In

best case, it can be generalized to the entire texture class. In

our classication experiments several classes are involved.

Hence, an optimal distance for all color channels and all

classes cannot be determined. Consequently, we studied the

color textures according to several distances. The follow-

ing experiments are performed with d = 0 (histogram) and

d = 1; 5; 10; 15; 20.

We combined the previously mentioned SCFs and MCFs

and compared them to pure gray-scale texture and pure color

histogram classication results. To give a clear reference to

the feature vectors m used in the experiments, the abbrevi-ations of the single- and combined feature vectors are listed

in Table 1.

Note, that the combinations in LUV are grouped into

intensity independent (II) and intensity dependent (ID)

whereas in RGB they are grouped into SCFs (SC) and

MCFs (MC).

6.2. Results

The classication results for DS1 and DS2 are listed in

Tables 2 and 3, respectively. The CM-representation of the

color histogram allows the direct comparison of the paralleland the integrative color texture concept (Section 2). Rows

13 of both tables show the results of gray-scale texture fea-

tures m(L) and the color histogram, described by its Haral-

ick features m(RGB0). Obviously, the importance of color

and intensity texture is quite dierent for the two datasets.

Whereas the HCFs forDS1 performs with 0:938 better than

the GCFs with 0:855, the texture features show better dis-

crimination eciency with 0:787 compared to 0:654 for

m(RGB0) according to DS2. For both datasets the parallel

concept combining the two single features outperform with

0.977 and 0.838 for DS1 and DS2, respectively, the results

of the single features.


8/12


Table 2

Classication results for dataset DS1 according to dierent color texture feature combinations

DS1

0 1 5 10 15 20

1 m(L) 0.855 0.652 0.544 0.531 0.5182 m(RGB0) 0.938

3 m(RGB0; L) 0.977 0.945 0.929 0.910 0.928

4 m(R) 0.868 0.771 0.726 0.702 0.691

5 m(G) 0.869 0.779 0.697 0.691 0.677

6 m(B) 0.896 0.819 0.738 0.720 0.704

7 m(RGBSC) 0.951 0.884 0.826 0.811 0.793

8 m(RGBMC) 0.960 0.870 0.862 0.850 0.864

9 m(RGBSC; MC) 0.971 0.903 0.887 0.864 0.869

10 m(RGB0; SC) 0.974 0.951 0.928 0.906 0.913

11 m(RGB0; MC) 0.972 0.933 0.920 0.925 0.926

12 m(RGB0; SC; MC) 0.977 0.946 0.928 0.917 0.920

13 m(RGBSC; L) 0.946 0.868 0.821 0.807 0.804

14 m(RGBMC; L) 0.966 0.894 0.865 0.864 0.882

15 m(RGBSC; MC; L) 0.968 0.902 0.905 0.876 0.879

16 m(U) 0.886 0.754 0.666 0.609 0.595

17 m(V) 0.856 0.719 0.592 0.536 0.532

18 m(UV) 0.891 0.799 0.795 0.794 0.661

19 m(UVII) 0.969 0.910 0.867 0.840 0.843

20 m(LUVID) 0.948 0.895 0.870 0.852 0.863

21 m(LUVII; ID) 0.986 0.949 0.923 0.916 0.922

The dierent results in one row refer to CM distances from zero (color histogram) up to 20. The feature

combinations are divided into semantically interrelated blocks: parallel color texture concept (Rows 13),

integrative color texture concept in the RGB color space (Rows 49), combination of integrative features with

pure color and pure intensity texture features, respectively (Rows 10 15) integrative color texture concept in

the LUV color space (Rows 1621).

The results for SCF of the RGB color space are presented

in Rows 46. The best results are obtained by m(B) with

d = 1 and correctness 0.896 and by m(B) with d = 5 and

correctness 0.794 for DS1 and DS2, respectively. A general

superior performance of the B channel is not plausible and

depends on the image dataset. All single results for bothdatasets are superior to the corresponding pure gray-scale

texture result. These classication rates for m(R), m(G),

m(B) are improved signicantly (unidirectional Binomial

test statistic, p 0:05) by combining them to m(RGBSC)

(Row 7). According to m(RGBSC) we achieved a correct-

ness of 0.951 and 0.828 for DS1 and DS2, respectively. The

corresponding combination of MCFs to m(RGBMC) yield

with 0.960 better and with 0:795 even worth results forDS1and DS2. Combining m(RGBSC) and m(RGBMC) (Row 8),

the dimension of the feature space rises to 96 and, hence,

the complexity of classication task rises, too. Nevertheless,

the results improved to 0.971 and 0.841, respectively.

The results of Rows 1012 in Tables 2 and 3 allow the

evaluation of possible advantages combining RGB-related

integrative color texture features with the pure color his-

togram features. ForDS1 the combination of SCF, MCF and

both with histogram features showed slightly increasing re-

sults, respectively. The overall combination m(RGB0; SC; MC)performed best with a correctness of 0.977. According to

DS2 the combination of SCF with histogram features im-

prove the performance of SCF with a correctness of 0.841.

In contrast, the performance of the overall combination de-

creased to 0.808. However, the feature space dimension and,

therefore, the complexity of the classication problem, rose

to 120 for this combination (see Table 1). All results of tex-

ture feature combination with the histogram features outper-

form the pure histogram features.

Rows 1315 show the classication results combining

RGB-based SCFs and MCFs with GCFs. For both datasets

all color texture combinations performed better than the pure


9/12


Table 3

Classication results of dataset DS2 according to dierent color texture feature combinations

DS2

0 1 5 10 15 20

1 m(L) 0.721 0.787 0.742 0.694 0.6372 m(RGB0) 0.654

3 m(RGB0; L) 0.831 0.838 0.814 0.765 0.767

4 m(R) 0.723 0.775 0.708 0.664 0.639

5 m(G) 0.745 0.777 0.740 0.689 0.627

6 m(B) 0.740 0.794 0.750 0.703 0.708

7 m(RGBSC) 0.787 0.828 0.784 0.708 0.666

8 m(RGBMC) 0.807 0.795 0.747 0.705 0.738

9 m(RGBSC; MC) 0.809 0.841 0.789 0.767 0.746

10 m(RGB0; SC) 0.826 0.841 0.806 0.794 0.779

11 m(RGB0; MC) 0.812 0.798 0.759 0.764 0.746

12 m(RGB0; SC; MC) 0.824 0.808 0.813 0.803 0.795

13 m(RGBSC; L) 0.777 0.824 0.784 0.703 0.668

14 m(RGBMC; L) 0.809 0.797 0.767 0.794 0.733

15 m(RGBSC; MC; L) 0.804 0.847 0.772 0.762 0.768

16 m(U) 0.658 0.776 0.714 0.646 0.595

17 m(V) 0.653 0.614 0.597 0.585 0.548

18 m(UV) 0.532 0.548 0.484 0.548 0.453

19 m(UVII) 0.756 0.807 0.759 0.778 0.729

20 m(LUVID) 0.786 0.740 0.694 0.729 0.673

21 m(LUVII; ID) 0.863 0.806 0.836 0.856 0.786

The notation corresponds to that of Tables 1 and 2.

gray-scale texture features. The combined features show

similar results as the corresponding single features in Rows

79. Therefore, no tendency can be remarked which indi-

cates the usefulness of these combinations. For both datasets

the best result of this block is achieved by the overall com-

bination m(RGBSC; MC; L). It yields a correctness of 0:968

and 0:847 for DS1 and DS2, respectively.

The results related to the intensity independent single

feature vectors of the LUV color space are listed in Rows

1618. ForDS1 the best performance is achieved with 0:891

by m(UV) and d = 1. On the other hand, for DS2 the bestcorrectness of 0:776 is achieved by m(U) and d = 5. There-

fore, both are below the respective maxima according to the

single RGB features (Rows 46).

For both datasets the results are enhanced by combin-

ing the single features to m(UVII) (Row 19). With 0:969

and 0:807 for DS1 and DS2, respectively, the improvement

is signicant (p 0:05). The intensity independent texture

features are in both cases superior to the intensity depen-

dent features (Row 20). Note, that m(L) is inherently in-

tegrated into m(LUVID) as SCF1 of the LUV color space.

The best results are obtained by the combination of both, in-

tensity independent and intensity dependent features (Row

21). These are superior not only to other combinations of

the RGB color space but to the parallel color texture con-

cept m(RGB0; L) as well. The results 0:986 and 0:863 for

DS1 and DS2, respectively, refer both to d = 1.

Taking d into account, a strong tendency to an opti-

mum distance for each dataset can be stated. All supe-

rior results for each experiment block are found for d = 1

and (except one) for d = 5 according to DS1 and DS2,

respectively.

7. Discussion

In our experiments, gray-scale texture and color histogram

analysis were compared with novel color texture features.

Additionally, two dierent concepts of color texture analy-

sis, parallel and integrative, were evaluated. This was possi-

ble for the rst time, because in the framework of CMs color

histograms and textural features are both well described.

The two image datasets used are rather dierent in content,

color texture appearance as well as discrimination capabil-

ity of color histogram and gray-scale texture. Nevertheless,

the tendencies of the results are quite similar. Therefore,


10/12


it is possible to generalize them to large application elds

dealing with natural color textures.

It has been shown, that color information improve the re-

sults of gray-scale texture features [14]. Our experiments

support these results. The combination of color histograms

and gray-scale texture features shows a high supplementa-

tion ability. The results of this parallel color texture concept

are just slightly below the best results we achieved by inte-

grative color texture features.

Whereas the parallel concept is most commonly applied

in the color image processing community, this paper intro-

duced novel integrative color texture features to learn more

about the aspects of color texture and develop new meth-

ods to model them. According to SCMs, we realize notice-

able dierences between the single-channel results. This in-

dicates dierences in the textures of dierent color channels

and relativizes their correlation. This observation supports

our evaluation of Kolmogorov distances in Section 5. Addi-

tionally, the channel-specic features supplement each otherresulting in a signicant better performance for the combi-

nation features. This enhancement is noticeable in particu-

lar considering the increase of dimensionality of the feature

space. Because the gray-scale texture results are below all

single-channel results, the transform from color to gray val-

ues is lossy with respect to textural features.

The novel method of MCMs improve the results of the

SCMs taking the correlation between color channels into ac-

count. With the help of the Kolmogorov distance, we demon-

strated that the multi-channel modeling is not equivalently

realized by the single-channel features in combination with

the color histogram. Therefore, MCMs measure new colortextural features. The low redundancy of both integrative

methods is also shown by the experimental evaluation. The

combination ofm(RGBSC) and m(RGBMC) leads to increas-

ing correctness although the feature space dimensionality

rises by a factor of two.

The combination of the SCFs with the histogram features

show rather similar results to the parallel color texture con-

cept. This could be expected since the only dierence is the

already stated wavelength dependency of colored texture.

Unfortunately, the information prot of texture modeling

subsequently on the color channels is over-compensated by

the disproportionate extension of the feature space. Never-

theless, color histogram and single-channel texture analysissupplement each other. Therefore, the classication correct-

ness rises in comparison to both single components. On the

other hand, the combination of m(RGBSC) with the GCFs

yields worse results according to both components. Again,

this shows the high redundancy of GCFs and SCFs. Study-

ing the redundancy of MCFs with HCFs and GCFs, re-

spectively, in both cases better results in comparison with

the single components can be stated. Therefore, a new kind

of texture information is measurable by the multi-channel

analysis.

In addition to RGB, the experimental evaluation take

the LUV color space into account. Although the methods

of single- and multi-channel color texture analysis are the

same, the interpretation of the results has to be adapted.

Because of the two intensity independent channels U and

V we achieve three intensity dependent and three intensity

independent matrices. Obviously, the intensity independent

features model a completely dierent kind of texture. Al-

though the RGB related color texture features consider both

color and texture, the texture is again intensity based, even

though this intensity is measured on a small wavelength in-

terval. In contrast, the MCMs related to U and V charac-

terize the Co-occurrence of pure colors. Because the three

single feature vectors m(U), m(V) and m(UV) represent

dierent parts of the color plane, they supplement each other

and, hence, their combination yields to a signicant better

classication result. Additionally, the results to the intensity

independent features are better than that of the intensity de-

pendent. The novel pure color texture features in compari-

son with pure intensity texture features show the overall best

performance. Again, this proves the fact low redundancybetween these two concepts.

8. Conclusion

In this paper, we presented novel integrative single- and

multi-channel Co-occurrence matrices for color texture anal-

ysis and put intensity independent color textures in perspec-

tive.

The multi-channel approach integrates color histograms

into the Co-occurrence notation and allows the comparison

of parallel and integrative approaches on the same levelfor the rst time. Additionally, the Kolmogorov distance

is recommended to measure quantitatively the information

gain of the multi-channel approach.

The advantages of the integrative methods were demon-

strated by the Kolmogorov distance as well as by the re-

sults of the experimental evaluation. Nevertheless, because

of the disproportional increase of feature space dimension-

ality, the best results according to RGB are achieved by the

parallel concept of separate analysis of color by histogram

and texture by intensity pattern analysis. However, the inte-

grative color features based on the LUV color space show

not only the existence of intensity independent color texture

but its discriminative power. In combination with the com-mon intensity dependent pattern, colored textures are more

adequately described.

Summary

Color image analysis and texture classication are two

basic tasks in pattern recognition. The dependencies of

both are still not clear and, hence, several concepts dealing

with colored textures exist. The objective of this paper is

to strengthen the basis for the discussion in the eld of

color texture recognition. To achieve this goal, we treated


11/12


ve main elds within the framework of Co-occurrence

matrices:

Categorization of the concepts for color texture featureextraction.

The identication of these concepts enables an overview

over the relevant literature and allows the comparisonbetween these approaches. Two of them, the parallel and

the integrative method, have been compared within the

experimental section.

Extension of Co-occurrence matrices to multi-channelcolor texture characterization.

Since Co-occurrence matrices are one of the most fre-

quently used methods for gray-scale texture analysis,

multi-channel matrices enlarges the toolbox of color

texture analysis methods signicantly.

Application of Kolmogorov distance to the novel single-and multi-channel matrices.

The Kolmogorov distance serves as a quantitative mea-sure to determine the dependency of intensity textures and

colors. Therefore, the discussion about this dependency

in the scientic community can be adjudged objectively

for each image separately.

Application of integrative Co-occurrence matrices to theRGB as well as the LUV color space.

Since the LUV space separates one intensity channel L

and two color channels U and V, Co-occurrence matri-

ces according to U and V provide intensity independent

and, hence, pure color texture features. This features are

used to prove the existence of intensity independent color

textures.

Classication experiments on two large image databases.The large number of experimental data allows the gen-

eralization of the conclusions drawn on the basis of the

results. In particular, the parallel and the integrative con-

cept are compared in the framework of Co-occurrence

matrices.

Two main conclusions have to be emphasized. Firstly, the

parallel concept of intensity texture analysis and separately

color histogram analysis is suitable for color image classi-

cation. Secondly, the existence of pure color textures as well

as their supplementation ability with respect to intensity de-

pendent textures features is proven. Therefore, the integra-tive color texture features introduced in this paper enable

the characterization of intensity independent color textures

for the rst time. Using these novel features in combina-

tion with gray-scale texture and color histogram features the

classication correctness is increased up to 32%.

References

[1] T.R. Reed, J.M.H. Du Buf, A review of recent texture

segmentation and feature extraction techniques, CVGIP

Image Understanding 57 (3) (1993) 359372.

[2] I. Tus, Automated fabric inspection based on a structural

texture analysis method, in: Recent Issues in Pattern Analysis

and Recognition, Springer, Berlin (1989) 377390.

[3] A. Bodnarova, M. Bennamoun, K.K. Kubik, Suitability

analysis of techniques for aw detection in textiles

using texture analysis, Pattern Anal. Appl. 3 (2000)

254266.[4] R. Haralick, K. Shanmugam, I. Dinstein, Texture features

for image classication, IEEE Trans. Syst. Man Cybernet.

3 (1973) 610621.

[5] Lei Wang, Jun Liu, Texture classication using

multi-resolution markov random eld models, Pattern

Recognition Lett. 20 (2) (1999) 171182.

[6] M. Pietikainen, T. Ojala, Z. Xu, Rotation-invariant texture

classication using feature distributions, Pattern Recognition

33 (2000) 4352.

[7] A.K. Jain, F. Farrokhnia, Un-supervised texture segmentation

using Gabor lters, Pattern Recognition 24 (12) (1991)

11671186.

[8] T. Randen, J.H. HusH

y, Multichannel ltering forimage texture segmentation, Opt. Eng. 33 (8) (1994)

26172625.

[9] Jing-Wein Wang, Chin-Hsing Chen, Wei-Ming Chien,

Chih-Ming Tsai, Texture classication using non-separable

two-dimensional wavelets, Pattern Recognition Lett. 19 (13)

(1998) 12251234.

[10] C. Gotlieb, H. Kreyszig, Texture descriptors based on

Co-occurrence matrices, Comput. Vision Graph. Image

Process. 51 (1990) 4752.

[11] K. Messer, J. Kittler, A region-based image database system

using colour and texture, Pattern Recognition Lett. 20 (1999)

13231330.

[12] M. Hauta-Kasari, J. Parkkinen, T. Jaaskelainen, R. Lenz,

Multi-spectral texture segmentation based on the spectral

Co-occurrence matrix, Pattern Anal. Appl. 2 (4) (1999)

275284.

[13] K.Y. Song, J. Kittler, M. Petrou, Defect detection in

random colour textures, Image Vision Comput. 14 (1996)

667683.

[14] S. Kukkonen, H. Kalviainen, J. Parkkinen, Color features for

quality control in ceramic tile industry, Opt. Eng. 40 (2)

(2001) 170177.

[15] A. Drimbarean, P.F. Whelan, Experiments in colour texture

analysis, Pattern Recognition Lett. 22 (2001) 11611167.

[16] P.-H. Suen, G. Healey, Modeling and classifying color

textures using random elds in a random environment, Pattern

Recognition 32 (1999) 10091017.[17] G. Van de Wouwer, P. Scheunders, S. Livens, D. Van

Dyck, Wavelet correlation signatures for color texture

characterization, Pattern Recognition 32 (1999) 443451.

[18] A.K. Jain, G. Healey, A multi-scale representation including

opponent color features for texture recognition, IEEE Trans.

Image Process. 7 (1) (1998) 124128.

[19] C. Palm, D. Keysers, T.M. Lehmann, K. Spitzer, Gabor

ltering of complex hue/saturation images for color texture

classication, Joint Conference on Information Science 2,

Atlantic City, NJ, USA (2000) 4549.

[20] G. Paschos, Perceptually uniform color spaces for color texture

analysis: an empirical evaluation, IEEE Trans. Image Process.

10 (6) (2001) 932937.


12/12


[21] C. Palm, T.M. Lehmann, Classication of color textures by

Gabor ltering, Mach. Graph. and Vision 11 (2/3) (2002)

195219.

[22] G. Healey, L. Wang, Illumination-invariant recognition of

texture in color images, J. Opt. Soc. Am. A 12 (9) (1995)

18771883.

[23] A. Rosenfeld, C.-Y. Wang, A.Y. Wu, Multispectral texture,IEEE Trans. Syst. Man and Cybernet. SMC-12 (1) (1982)

7984.

[24] T.M. Lehmann, C. Palm, Local line search for illuminant

estimation in real world color scenes, J. Opt. Soc. Am. A

18 (11) (2001) 26792691.

[25] B.A. Wandell, Foundations of Vision Science, Sinauer Ass.,

Sunderland, MA, USA, 1995.

[26] T.S.C. Tan, J. Kittler, Colour texture analysis using colour

histogram, IEE Proc. Vision Image Signal Process. 141 (6)

(1994) 403412.

[27] M. Mirmehdi, M. Petrou, Segmentation of color textures,

IEEETrans.Pattern Anal. Mach. Intell.22 (2)(2000) 142159.

[28] A. McCabe, T. Caelli, G. West, A. Reeves, Theory of

spatiochromatic image encoding and feature extraction, J. Opt.Soc. Am. A 17 (10) (2000) 17441754.

[29] A. Bradley, P. Jackaway, B. Lovell, Classication in

scale-space: applications to texture analysis, 14th International

Conference on Information Processing in Medical Imaging,

Ile de Berder, France (1995) 375376.

[30] A.N. Kolmogorov, Three approaches to the quantitative

denition of information, Int. J. Comput. Math. 2 (1968)

157168.[31] Database VisTex of color textures of MIT, USA:

ftp://whitechapel.media.mit.edu/pub/VisTex.

[32] Database of color textured barks of University of Koblenz,

Germany: ftp://ftphost.uni-koblenz.de/outgoing/vision/

Lakmann/BarkTex.

[33] R. Lakmann, Statistische Modellierung von Farbtexturen,

Ph.D. Thesis, University Koblenz-Landau, Folbach, Koblenz,

1998 (in German).

[34] S.W. Zucker, D. Terzopoulos, Finding structure in

Co-occurrence matrices for texture analysis, Comput. Graph.

Image Process. 12 (1980) 286308.

[35] J.F.R. Ilgner, C. Palm, A.G. Schutz, K. Spitzer, M.

Westhofen, T.M. Lehmann, Colour texture analysis for

quantitative laryngoscopy, Acta Otolaryngol. 123 (2003)730734.

About the AuthorCHRISTOPH PALM studied Computer Science at the Aachen University of Technology (RWTH), Germany, and

received his Diploma degree in 1997. Between 1997 and 2001 he was Ph.D. student at the Institute of Medical Informatics at the University

Hospital of the RWTH. During his Ph.D. research, his research interests covered all aspects of pattern recognition, texture classication

and color image processing. He is co-editor of a proceedings volume and author of several scientic conference and journal papers. He is

peer reviewer of the Optical Society of America. In 2001 Christoph Palm joined the Aixplain AG, Aachen. Beside image processing, his

research focus is now enlarged to information retrieval and statistical machine translation.
ftp://whitechapel.media.mit.edu/pub/VisTexftp://whitechapel.media.mit.edu/pub/VisTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://whitechapel.media.mit.edu/pub/VisTex

Color Texture Classication by Integrative Co-Occurrence Matrices

Documents

Transcript of Color Texture Classication by Integrative Co-Occurrence Matrices