SIViP (2012) 6:231–245DOI 10.1007/s11760-010-0182-8

ORIGINAL PAPER

A simple and effective texture characterization for imagesegmentation

Tao Xu · Iker Gondra

Received: 20 August 2009 / Revised: 25 August 2010 / Accepted: 25 August 2010 / Published online: 8 September 2010© Springer-Verlag London Limited 2010

Abstract Although texture plays a critical role in imagesegmentation, because of the omnipresent irregularitiesamong texture patterns in real images, it is not easy to char-acterize. Approaches that use fixed-sized windows to extractlocal features are popular for texture identification and classi-fication. However, besides being computationally intensive,due to the unawareness of texture scales and boundary loca-tions, those block-based methods have limited success inimage segmentation. In this paper, we present an algorithmthat generates statistical descriptors that are adaptive to thevariation of texture patterns based on a simple rule of prun-ing and concatenating the approximately repetitive patterns.A comparative study of this approach with Haralick features,the most commonly used method in texture classification,with emphasis on image segmentation is conducted. The pro-posed texture characterization is demonstrated to producehigher segmentation accuracies.

Keywords Texture · Image · Segmentation

1 Introduction

Texture can be defined as the repetition of a certain atomicpattern residing in a region. For a low-level image analysis,texture features play a very important role in distinguishingtextured regions from one another based on the measurementof optical homogeneity of surfaces. However, texture repre-

T. Xu · I. Gondra (B)Department of Mathematics, Statistics, and Computer Science,St. Francis Xavier University, Antigonish, NS, Canadae-mail: igondra@stfx.ca

T. Xue-mail: x2006opl@stfx.ca

sentation, which is concerned with the feature extraction ofsuch repetitive patterns from images, has long been a chal-lenge for image analysis (e.g., image segmentation, imageretrieval, object recognition, etc.). In spite of the existence ofa large number of algorithms for texture analysis, up to now,limited success has been made in acquiring effective texturefeatures for general applications.

The most well-known texture feature extraction algo-rithm is the grey-level co-occurrence matrix (GLCM) methoddeveloped by Haralick [14]. As early as in the 1970s, Haralicket al. [14] proposed the first systematic analysis of texture byusing the GLCM, which describes the distribution of grey-level co-occurring pixel values at a given offset. Based onsuch matrix, several texture features (e.g., contrast, entropy),which explore the spatial dependence of pixel values, can beextracted. Other researchers carried on investigating texture.For example, Tamura et al. [26] developed a set of texturefeatures designed to measure the visual properties of coarse-ness, contrast, directionality, line-likeness, regularity, androughness which, based on conducted psychological exper-iments, are thought to dominate human visual perception oftexture. Haralick’s approach, which was initially designedfor exploring the spatial dependence of grayscale texture,was later extended to color texture extraction by using colorco-occurrence matrices [21] and shown to have good perfor-mance [22].

In recent years, frequency-domain tools such as Fouriertransform, discrete cosine transform and wavelet transformhave been used very actively in texture analysis. For instance,Gabor 2D-wavelets [8] that employ a set of filter banks (i.e.,pairs of high-pass and low-pass filters) in different orienta-tions and scales for characterizing image features, have beenadopted in the MPEG7 standard as texture descriptors. Mostof those approaches use a fixed-size window to derive localfeatures at a specific scale. This type of technique has been

123

232 SIViP (2012) 6:231–245

largely employed in content-based image retrieval [13,16,24] as well as in image segmentation [2,4,12].

There are a couple of inherent drawbacks associated withthe above-mentioned block-based approaches. First, in orderto derive texture features, the size of a local window has tobe fixed a priori. As a result, algorithms fail to adapt tothe variation of scales. Second, problems arise when object(region) boundaries fall into a sampling window, which lim-its the ability of distinguishing patterns along boundaries.The weakness of a unique-scale representation of texturefeatures has been well recognized and a few multi-resolu-tion-based approaches have been proposed to address thisproblem [5,15,17,28]. However, those approaches can bevery computationally intensive.

In this paper, we present a relatively simple, yet effective,algorithm that generates statistical descriptors that are adap-tive to the variation of texture patterns based on a simplerule of pruning and concatenating the approximately repeti-tive patterns. A preliminary study of this idea was presentedin [29]. A comparative study of this approach with Haralickfeatures, the most commonly used method in texture classifi-cation, with emphasis on image segmentation is conducted.

The rest of this paper is structured as follows. Section 2presents our proposed methodology. A brief overview ofunsupervised image segmentation with particular emphasison Mean Shift-based clustering [7], which is the approachused in our comparative study, is given in Sect. 3. Section 4presents experimental results based on segmentation perfor-mance on real images. Finally, concluding remarks are givenin Sect. 5.

2 Proposed method

One difficulty concerning texture analysis is the processingof irregularities of real texture patterns. For real images, due

to factors such as scales, spatial orientations, varying per-spectives and amounts of illumination, a perfect regularityof repetitive texture pattern hardly exists. It is thus difficultfor us to accurately describe texture patterns by using fre-quency domain analytical tools such as Fourier coefficientsor wavelet transforms. Thus, to accommodate for this, insteadof using a fixed-sized 2D window, we use a simple scan-line-based means to probe the potential texture patterns andcollect their local characteristics.

2.1 Pruning and concatenating of patterns

Let p(x, y) denote the pixel at position (x, y) in an image.A scanline Sα = {p1, . . . , pn} is a sequence of n consecutivepixels that form a straight line over an image, starting from apixel p1 = p(x, y) and moving along a certain direction α,passes through (See Fig. 1a). Because each color channel willbe analyzed and processed individually, we simply use pi torefer to the pixel intensity value for one of the color channelsof the i th pixel. If we put this sequence of pixels into a Carte-sian coordinate system with x-axis as indices and y-axis asone-channel intensity values, we obtain a one-dimensionalsignal (See Fig. 1b).

We use the following measure as the center of pattern fora given scanline Sα

C =∑n

i=1 pi i∑n

i=1 pi(1)

This is analogous to the definition of the center of mass inphysics. Thus, if Sα is a piece of a strictly repetitive patternsegment, the following will be satisfied with a small balanceerror e

C = n + 1

2+ e (2)

Fig. 1 a Original image with a horizontal scanline (in white color); b the index-intensity chart of this scanline (red color channel)

123

SIViP (2012) 6:231–245 233

Fig. 2 Scanlines that have heterogeneous patterns inside but still with their pattern centers approximately at the center of scanlines: a a piece ofscanline segment with dense areas at two sides; b a piece of scanline segment with dense area concentrated around the center

In the case that e ≈ 0, the pattern center of Sα is close to thepixel with index �(n + 1)/2�.

In most cases, the balance error e indicates the perfect-ness of repetition for a given pattern segment. Intuitively,a small absolute value for e means an approximately bal-anced pattern. In contrast, a large absolute value implies thatthe given pattern is not coherent everywhere, which suggeststhat the incoherence has to be trimmed off in order to retainits homogeneity (balance). Based on this observation, thefollowing pruning rule is used for extracting a homogenouspattern from a given scanline Sα: if C obtained by (1) cannotsatisfy (2) with e < τ (τ is a predefined balance threshold),then we prune the scanline by �e� pixels either on the rightside or left side of Sα . The balance error e can be simplycalculated by

e = n + 1

2− C (3)

If e is positive, then �e� pixels on the most left side ofSα are pruned off because some part on the right side (i.e.,pixels with higher indices) of Sα is denser. For the samereason, a negative e leads to a pruning of �|e|� pixels on themost right side of Sα . The pruned scanline goes through thesame procedure until a balance is finally achieved and thusa homogenous pattern Pα ⊆ Sα is acquired. All pixels trun-cated off are concatenated to form a new scanline, and thesame procedure is repeated until all pixels are associated withpatterns. Eventually, a scanline is divided into a set of con-secutive nonoverlapping patterns. This rule guarantees thatduring each iteration of pruning, only one side of a scanlinewill be trimmed off, which largely protects the integrity oftexture patterns. As an informal proof of this, consider the

situation in which e > 0 pixels are trimmed off from the leftside of a pattern because the right side is denser on average.Let C ′ denote the new center for the rest of the pattern. Theindex of the middle point becomes mid ′ = (e+ 1+ n)/2 =e/2 + (1 + n)/2 = e/2 + mid, where mid = C − e (byEq. 2) is the middle point of the original pattern. We now haveC ′−mid ′ = C ′−e/2−mid−C+e = C ′−C+e/2. It is evi-dent that C ′ −C is always nonnegative because e > 0 pixelswith small indices have been pruned off. Thus C ′−mid ′ > 0,which means over-correction will not occur with this pruningrule. This can be similarly verified for the opposite situation.In other words, with this rule, pruning can only take place onone side of a pattern.

However, even if the balance threshold τ is restricted tozero, this does not guarantee the existence of an approxi-mately repetitive pattern because there are cases (See Fig. 2)of patterns having their pattern centers approximately at thecenters but not presenting any coherent texture patterns at all.Hence, this pruning rule only works for limited cases. A sim-ple way to overcome the problem is to detect homogenouspatterns in a concatenating manner. More precisely, the algo-rithm starts with the first l data points in a scanline, wherethe initial window length l should be selected large enoughto capture texture patterns. If Eq. (2) with e < τ over thisl-point sequence is satisfied, then it considers the next l datapoints in the scanline. If the center of pattern for these 2 l datapoints still satisfies (2) with e < τ , then these two sequencesare concatenated and the same procedure is repeated until thebalance is broken. Whenever the balance is not held, the valueof l is cut in half (i.e., l = 0.5 l) and the same concatenatingrule continues. If l is reduced to zero, which probably impliesa hit of the boundary between two distinct patterns, it means

123

234 SIViP (2012) 6:231–245

Fig. 3 In step 1, the algorithm starts with the first l data points in thescanline. Because Eq. (2) with e < τ over this l-point sequence is satis-fied, it considers the next l data points in the scanline. In step 2, becausethe center of pattern for these 2 l data points still satisfies (2) with e < τ ,these two sequences are concatenated. The same procedure is repeateduntil the balance is broken. In step 5, the balance is not held. Thus, thevalue of l is cut in half (i.e., l = 0.5 l) and the same concatenating rulecontinues. In step 10, l is reduced to zero and the first texture pattern(from the starting position to the breaking point where l becomes zero)has been located. The algorithm then restarts from the last breakingpoint and keeps doing this until every data point is associated with apattern. Finally, eight nonoverlapping patterns are obtained

a texture pattern has been located from the starting positionto the breaking point where l becomes zero. The algorithmthen restarts from the last breaking point and keeps doing thisuntil every data point is associated with a pattern. Finally, aset of consecutive nonoverlapping patterns is obtained. Fig-ure 3 is a visual illustration of this algorithm on a samplescanline.

To make this algorithm more robust in distinguishing pat-tern boundaries, the evaluation of the pattern center in termsof inverse intensities is necessary. The following

C̄ =∑n

i=1 p̄i i∑n

i=1 p̄i(4)

where p̄i = (M − pi ) is the inverse intensity of pi givenM as the cap value of color intensity is calculated. The cal-culation of C̄ is actually the calculation of C on a negativeimage. The purpose of doing this is to avoid making a falsedetection of patterns, which could happen (See Fig. 4). Byintroducing (4), the concatenation of the next l data pointshappens only if both (1) and (4) satisfy (2) with e < τ .

Another useful trick to enhance the discriminability ofpattern boundaries is to raise both the original intensitiesand the inverse intensities by taking a high-order powerof them. Obviously, this operation increases the between-pattern discrepancy while retaining the intra-pattern coher-ence. Results from our experiments show that using thesecond-order power of intensities and inverse intensities

C =∑n

i=1 p2i i

∑ni=1 p2

i

, (5)

Fig. 4 a A scanline with C approximately at the center of it; b when intensity is inverted, C̄ is located far from the center of the scanline. They-axis indicates the pixel intensity values, and the x-axis indicates the pixel location/indices in a scanline

123

SIViP (2012) 6:231–245 235

Fig. 5 A given scanline is divided into several pattern segments (sep-arated by dashed lines)

and

C̄ =∑n

i=1 p̄2i i

∑ni=1 p̄2

i

(6)

produces an excellent result while a higher-order power doesnot improve the performance much. Algorithm 1 summarizesthe entire procedure for extracting patterns from a scanlineSα given the balance threshold τ and initial length l.

Figure 5 shows the results of applying the algorithm toa given scanline. Although the first pattern (from pixel 0to about 400) is divided into four segments, distinct pat-terns are successfully separated from others and, more impor-tantly, all boundaries are preserved. For 2D images, due to thevariety of texture patterns in different orientations, a multi-directional scanline operation surely generates a more infor-mative representation than a single-directional one. In ourexperiments, a four-directional operation (namely 0, 45, 90,and 135 degrees) was employed. The results show that it isfairly good for image segmentation.

2.2 Generation of texture maps

Once approximately repetitive patterns have been extractedfrom scanlines in all directions of interest, the texture map ofthe image can be readily constructed. To generate the finalrepresentation, we first calculate the mean of each extractedpattern Pα and assign the results to all participating pixelsinto another bitmap of the same size and color channels asthe original one. For each scanline orientation α, we obtaina mean texture map (MTM) denoted as Tα that describes theaverage of each detected texture pattern in a given direction.Mathematically, let Pα,(x,y) denote an extracted pattern thatcontains the pixel at position (x, y) in the image, then thevalue at each pixel in an MTM is determined by

T (1)α (x, y) = 1

n

∑

pi∈Pα,(x,y)

pi (7)

where n is the number of pixels in Pα,(x,y). Next, we weight-sum all single-directional MTMs into one MTM by using theformula

T (x, y) =∑

α

wαT (1)α (x, y)

=∑

α

T (1)α (x, y)

∑α T (1)

α (x, y)T (1)

α (x, y)

=∑

α

T (1)α (x, y)2

∑α T (1)

α (x, y)(8)

where, as indicated by the formula, the weight wα is deter-mined by the significance of a texture pattern in a givendirection with respect to the overall summation. Thus, thefinal MTM is actually a blurred image of the original onein the sense that the color values at each pixel are replacedby the mean color of the texture pattern the pixel belongsto. Different from the mean filter used for image process-ing, boundaries are well preserved and only blurred withintexture regions.

Patterns of different variance may have the same mean val-ues. Thus, to promote robustness in terms of pattern discrim-inability, it is sometimes necessary to introduce higher-orderstatistics into the analysis of texture features. We introducethe standard deviation texture map (SDTM), denoted by T (2)

α

and defined as follows

T (2)α (x, y) =

√√√√

1

n

∑

pi∈Pα,(x,y)

(T (1)

α (x, y)− pi

)2(9)

or, for computational efficiency, approximated by

T (2)α (x, y) = 1

n

∑

pi∈Pα,(x,y)

∣∣∣(

T (1)α (x, y)− pi

)∣∣∣ (10)

where | · | denotes the absolute value operator.Applying (8) again (with T (1)

α (x, y) replaced by T (2)α

(x, y)) to all STDMs for all directions of interest, wefinally obtain a single STDM. Similarly to the MTM, whichdescribes the average color of each texture pattern, the STDMdescribes the average variance. The use of standard deviationinstead of variance is justified by the observation that if thechosen color space (such as CIELUV color space) agreesperceptually with human vision, then the standard deviationtightly follows this and thus avoids having to weight featuredimensions for the subsequent analysis.

Higher-order texture maps can be similarly defined andcalculated. For data-driven image segmentation, MTMs pro-vide adequate information for distinguishing patterns fromone another. This is due to the aforementioned concatenat-ing rules. Whenever the algorithm looks into a scanline forthe next l pixels of the same mean value as the previouslydetected pattern, the algorithm would refuse to concatenate

123

236 SIViP (2012) 6:231–245

Algorithm 1 PatternDetection(Sα, τ, l)Output: P � a set of detected patterns from scanline Sα using threshold τ and initial length l1: P, P ← ∅2: lcurrent ← l3: current Pos, start Pos ← 14: Plookahead ← Sα[current Pos : current Pos + lcurrent ] � Sα[i : j] is sequence of pixels in Sα with indices from i to j5: repeat6: mid Pos = 0.5(current Pos − start Pos + lcurrent )

7: Use Eq. (5) to compute C over sequence of pixels P ∪ Plookahead8: Use Eq. (6) to compute C̄ over sequence of pixels P ∪ Plookahead9: if

((|C − mid Pos| ≤ τ) &

(∣∣C̄ − mid Pos

∣∣ ≤ τ

))then

10: P ← P ∪ Plookahead � concatenate a lookahead sequence of pixels11: current Pos ← current Pos + lcurrent12: else13: lcurrent ← �0.5lcurrent� � reduce the lookahead length14: if (lcurrent == 0) then � true indicates a potential boundary15: P ← P ∪ P � save a detected pattern16: P ← ∅17: current Pos, start Pos ← current Pos + 118: lcurrent ← l19: end if20: end if21: Plookahead ← Sα[current Pos : current Pos + lcurrent ]22: until current Pos ≥ |Sα | � Stop the algorithm when the end of Sα is met

the two if they present different variances. However, for gen-eral cases such as image retrieval and pattern recognition,higher-order statistical descriptors would have to be incor-porated in order to enhance the discriminability of patterns.

By taking into account all color channels and all orderstatistics, the final representation of a pixel is a vector con-sisting of number of color channels × number of statisticsdimensions. For instance, for texture maps including first-order and second-order statistics, a pixel in the RG B colorimage space is represented as

p(x, y)=(

T (1)R (x, y), T (2)

R (x, y), T (1)G (x, y), T (2)

G (x, y),

T (1)B (x, y), T (2)

B (x, y))

.

2.3 Reducing sensibility to scales

The only parameter that affects performance is the initialwindow length l (and the balance threshold τ ). In order toreduce the impact of scales, we can run the same algorithmseveral times with various initial window lengths. The exper-imental results indicate that the sensitivity to scales can beeffectively avoided by iteratively running the same algorithmover the previous generated MTM with an incremental ini-tial window length. For example, using a fixed incrementalstep length s, the initial length at iteration i + 1 is li+1 =li+s. It is worthy to mention that when running the algorithmin an iterative manner, the new MTM is calculated basedon the previous one at each iteration whereas the SDTMis calculated only once between the original image and thefinal MTM.

3 Unsupervised image segmentation

Over the last four decades, the development of image seg-mentation algorithms has been an area of considerableresearch activity. The reason for such significant attention tothis problem lies in its practical importance. Image segmen-tation is a key step toward high-level tasks such as imageunderstanding and serves in a wide range of applicationsincluding object recognition, scene analysis or content-basedimage/video retrieval.

Many image segmentation algorithms have been devel-oped and different classification schemes have been proposed(e.g., [3]). In edge-based approaches (e.g., [25]), segmenta-tion is based on spatial discontinuities. That is, by detectingsudden changes in local features, region boundaries can beobtained. In region-based approaches, segmentation is basedon spatial similarity among pixels. Thus, a measure of regionhomogeneity has to be defined in advance. An approach tohomogeneity-based segmentation makes use of clusteringmethods, which classify pixels into one of several groups.Mean shift clustering [7] is widely used in the vision com-munity. It is derived from the Parzen window approach fornonparametric density estimation [10]. It finds the modes(dense areas) of the underlying probability density functionof the image pixel values and associates with them pixelsin their basin of attraction. One of the main advantages ofthis algorithm over most other clustering techniques is thatit does not rely upon a priori knowledge of the number ofclusters (e.g., the value k in the case of k-means clustering[20]). Furthermore, it does not implicitly assume any partic-ular shape (e.g., elliptical) for the clusters. Thus, it allows

123

SIViP (2012) 6:231–245 237

for the analysis of arbitrarily structured feature spaces. Thenormalized cuts framework [23], which is capable of detect-ing clusters of various shapes, is an example of a clus-tering-based approach derived from graph theory. Otherimportant methods, that are difficult to classify under theedge-based or region-based categories, include segmentationusing the Expectation-Maximization algorithm (e.g., [1]),using Markov Chains (e.g., [27]), and hybrid techniques thatuse both edge and homogeneity information (e.g., [11]).

3.1 Mean shift clustering

The Parzen window approach (kernel density estimation) toestimating an unknown probability density function p(x) isthe most popular density estimation method. As a nonpara-metric procedure, it can be used with arbitrary distributionsand without the assumption that the shape of the underly-ing density is known. Although an estimate of p(x) can beobtained, sometimes, we are more interested on the modes(high-density areas) of p(x) rather than on p(x) itself. Themodes of p(x) are located among the zeros of its gradient,which is estimated by the gradient of p̂(x), where p̂(x) is anestimate of the space-averaged density at a point x. It can beshown (see [7]) that the gradient of p̂(x) is the product of twoterms. The first term is proportional to p(x) and the secondterm is the mean shift vector (i.e., the difference between theweighted mean and x, the center of the window). The meanshift at x, mx, is proportional to the density gradient esti-mate. Therefore, mx always points in the gradient-increas-ing direction of p(x). Thus, mx can define a path leading toa stationary point (mode) of the estimated density. The meanshift procedure is obtained by successive computation of mx

and translation of the window by mx. An important propertyof this procedure which makes it unique from other gradi-ent-based algorithms is that, when moving along such path,there is no need to specify the step size explicitly [7].

The fact that the mean shift procedure results in a walkalong the direction of increasing gradient toward the near-est mode makes it an ideal tool for cluster analysis. Forimage segmentation, it was first introduced in [7] and soonbecame one of the most popular image segmentation tech-niques. There are two major steps involved: an applicationof the mean shift procedure on the image pixels to locate allconvergence points, followed by a clustering step to mergeall convergence points (and associated pixels) on the samebasin of attraction into regions. Briefly, each image pixelis represented in the joint domain by a (2+p)-dimensionalvector, which is the concatenation of its 2 spatial coordinates(in the spatial domain) with its p-dimensional color spacerepresentation (in the range domain, usually the 3-dimen-sional L*u*v* color space). In the first step, the window (aproduct of two kernels with window width parameters hs andhr ) is initialized at each individual pixel location and moves

in the direction of the maximum increase in the joint densitygradient, until convergence. Thus, hs and hr are the employedkernel bandwidths. In the second step, convergence pointsthat are closer than hs in the spatial domain and hr in therange domain are assigned to the same region. Finally, eachpixel is assigned the region label of its corresponding con-vergence point.

This procedure has a relatively small number of adjustableparameters (i.e., only the scale (window width) parameters),which largely reduces the search space for optimal parame-ter settings. However, the color and spatial information thatit uses are usually not sufficient for good segmentation per-formance. A method that combines the mean shift procedurewith the minimum description length (MDL) principle is sug-gested in [18]. In that approach, an over-segmented image isobtained through the mean shift procedure, and the MDLprinciple is subsequently employed to eliminate those tinyregions which are most possibly caused by texture. Anotherremarkable method proposed in [9] first converts an imageinto a quantized color map and then a criterion for ‘good’segmentation is applied for each local window, where a largevalue of the criterion indicates region boundaries and a smallvalue implies interiors of texture regions. These two methodsattempt to avoid the direct analysis of texture when segment-ing texture images, which in turn makes them prohibitive inquantitative texture analysis.

4 Experimental results

A set of 100 randomly selected images from the Berkeley seg-mentation dataset [19] was used for the experiments. An intu-itive way of assessing the performance of an image segmen-tation algorithm is to compare the segmentations it producesagainst ground truth (human-generated) segmentations of thesame images. Let S = {R1,R2, . . .}, where R j is the set ofpixels in a region, be the segmentation of an image obtainedwith a segmentation algorithm. Let S∗ = {R∗1,R∗2, . . .

}be a

ground truth segmentation of the same image. The segmen-tation quality measure Qi for region R∗i in the ground truthsegmentation is defined as

Qi = max1≤ j≤|S|

∣∣R∗i ∩R j

∣∣

∣∣R j

∣∣×

∣∣R∗i ∩R j

∣∣

∣∣R∗i

∣∣

The segmentation quality measure over the entire image isthen

Q =|S∗|∑

i=1

wi Qi

where wi = |R∗i ||R∗1 ⋃R∗2⋃

...| , the weight of region R∗i in the

ground truth segmentation, is based on its size (number ofpixels) relative to the size (total number of pixels) in the

123

238 SIViP (2012) 6:231–245

entire image. The measure Q can be simply thought of asthe percentage of pixels in agreement with the ground truthsegmentation. Thus, 0 ≤ Q ≤ 1 is to be maximized.

As previously discussed, the most well-known texturefeature extraction algorithm is the grey-level co-occur-rence matrix (GLCM) method developed by Haralick [14].A comparative study of the performance of the proposedtexture characterization with Haralick features when appliedto image segmentation was conducted. As previously men-tioned, the color and spatial information that is used by themean shift segmentation algorithm [7] are usually not suffi-cient for good segmentation performance. We alleviate thisproblem by first preprocessing the image with the two tex-ture characterizations methods. The preprocessed image thenbecomes the input image representation for the mean shiftsegmentation algorithm [7].

For our proposed texture characterization, texture mapswere created using the proposed four-directional (namely0, 45, 90 and 135 degrees) scanline algorithm with balancethreshold τ = 1 which, over a small set of test images,did not generate too many small pattern segments and kepta good track of boundaries. Various initial lengths l ={12, 18, 24, 30, 36} (line-striking effects do appear in thesegmentation if l is selected too large, e.g., greater than100) for pattern detection were tested with a fixed incre-mental step length s = 4. The MTMs that were createdafter the fifth iteration were used as the final image rep-resentations. A sample of some of the testing images withtheir MTMs is given in Fig. 6. We do not show the SDTMsbecause their visualization does not help much intuition. Aswe can observe, complex texture areas were all blurred intoa few homogenous regions as intended. Also, the objectboundaries were preserved. The images in Fig. 7 are theobtained MTMs for the 100 images in the experimentalset. The images in Fig. 8 are the corresponding segmenta-tions of those images obtained with the proposed texturecharacterization.

In the case of Haralick’s features, a full set of descrip-tors contains highly redundant information because many ofHaralick’s features are correlated with one another. Based onthe correlation analysis of Haralick’s descriptors conductedin [6], only entropy (ENT), correlation (COR), and contrast(CON) are selected for texture characterization in our exper-iments. Although the mean of the GLCM is also consid-ered independent with respect to all the other descriptors,the experimental results indicate that ENT, COR, and CONoutperform any other combination of Haralick’s descriptors.

There are four major parameters concerned with the calcu-lation of a local GLCM. In order to evaluate texture featuresat different locations in a given image, a local window of fixedsize w×w slides from pixel to pixel to acquire pointwise fea-tures. The choice of w is very subtle. A small window mightnot be able to provide adequate information for a statistical

(a) (b)

Fig. 6 Mean texture maps. a Original images; b MTMs

123

SIViP (2012) 6:231–245 239

Fig. 7 Mean texture maps of images in experimental set obtained with proposed texture characterization method

description of texture while a large window is very likely toconfuse the feature space, particularly at texture boundaries.Another parameter is the offset at which occurrence probabil-ities are calculated. Intuitively, a large offset is able to capture

slowly varying texture patterns and a short offset is capableof capturing repetitive sudden changes. In our case, whereno prior knowledge about texture is assumed to be known,occurrences at four distinct offsets are inspected, namely,

123

240 SIViP (2012) 6:231–245

Fig. 8 Segmentations of images in experimental set obtained by using mean shift segmentation on MTMs obtained with proposed texture charac-terization method. For each segmentation, the corresponding Q value appears below it

1, 3, 5, and 7. Offsets beyond this range are excluded becausethey could easily confuse the feature space as the windowsize does. The GLCM is not rotation-invariant according to

its definition. Thus, we have to consider offsets at differentorientations in order to increase robustness. Similarly to theorientations used for texture maps, local GLCMs at directions

123

SIViP (2012) 6:231–245 241

Fig. 9 Texture representations of images in experimental set obtained with Haralick features

0, 45, 90, and 135 degrees in combination with previouslychosen offsets are calculated. The GLCM is usually not cal-culated at the full grayscale range (i.e., [0, 255]) due to high

computational complexity. For example, consider computingthe GLCM based on a local window of size 9× 9. There is atotal of 256× 256 = 65, 536 entries in the GLCM for a full

123

242 SIViP (2012) 6:231–245

Fig. 10 Segmentations of images in experimental set obtained by using mean shift segmentation on texture representations obtained with Haralickfeatures. For each segmentation, the corresponding Q value appears below it

grayscale range but only at most 81 entries have values otherthan zero. The high sparsity causes a huge waste in computa-tion. Quantization addresses this problem by grouping gray

scales into much fewer levels. In our case, a quantization of32 levels generates acceptable results over the experimentalimages.

123

SIViP (2012) 6:231–245 243

Table 1 Overall segmentation performance for different texture char-acterization methods

Method Average Q

Proposed characterization 0.325396

Haralick 0.210383

To retain a relatively low-dimensional yet discrimi-native feature space for image representation, the finalrepresentation for each pixel is chosen to be a three-dimensional vector, each component of which is a weightedsum of the selected Haralick descriptors (i.e., ENT, CORand CON) calculated at different orientations and offsets.More specifically, each image is associated with a set

of separate feature images computed at different orienta-tions and offsets of interest. The final representation isconstructed using the same formula, i.e., Eq. (8), definedfor calculating the final texture maps, i.e., the most sig-nificant features are given the greatest weight. All threefeature dimensions are eventually normalized to [0,255]for the sake of visualization and segmentation parame-ter selection. The images in Fig. 9 are the obtained tex-ture representations for the 100 images in the experimentalset.

All experimental color images are converted into corre-sponding grayscale ones. Then, image features are extractedas stated previously. The mean shift-based segmentation isagain used to partition the featured images into regions. The

Fig. 11 Sample segmentationsof images obtained by usingmean shift segmentation on:a texture representation obtainedwith Haralick features; b MTMsobtained with proposed texturecharacterization. For eachsegmentation, the correspondingQ value appears below it

(a) (b)

123

244 SIViP (2012) 6:231–245

parameter setting for the mean shift algorithm (i.e., a pairof bandwidths) that results in the best performance over asmall set of experimental images is chosen for segmentingthe entire image set. That is, hr = 6 and hs = 2, respec-tively. The calculation of Haralick’s descriptors is very time-consuming. The images in Fig. 10 are the correspondingsegmentations of the images in the experimental set.

The overall performance measures by averaging Q forall 100 images are given Table 1. Figure 11 shows enlargedsample segmentations obtained by using mean shift-basedsegmentation on SDTMs obtained with the proposed texturecharacterization method and on the texture representationobtained with Haralick features.

5 Conclusions

We presented a novel algorithm that tends to generate sta-tistical descriptors that are adaptive to the variation oftexture patterns based on a simple rule of pruning or con-catenating the approximately repetitive patterns. A com-parative study (with emphasis on image segmentation) ofthis approach with Haralick features, the most commonlyused method in texture classification, was conducted. Theproposed texture characterization was demonstrated to pro-duce higher segmentation accuracies when tested on com-plex real images from the Berkeley segmentation dataset.The experimental results indicated that the proposed scan-line-based algorithm is capable of capturing complex tex-ture patterns while keeping the integrity of boundaries. Inthe proposed algorithm, although an initial length has to bespecified manually for pattern detection, by using an incre-mental initial length with multiple iterations, the sensitivity toscales can be significantly reduced. In the current approach,only the mean texture maps are used. As a part of our futurework, more descriptive features that incorporate high-ordertexture maps will be applied so that patterns that are sim-ilar in average but different in high-order statistics can bedistinguished.

References

1. Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld:color and texture-based image segmentation using EM and its appli-cations to image querying and classification. IEEE Trans. PatternAnal. Mach. Intell. 24(8), 1026–1038 (2002)

2. Chan, D.Y., Lin, C.H., Hsieh, W.S.: Image segmentation withfast wavelet-based color segmenting and directional region grow-ing. IEICE Trans. Inf. Syst. E88-D(10), 2249–2259 (2005)

3. Cheng, H.D., Jiang, X.H., Sun, Y., Wang, J.: Color image segmen-tation: advances and prospects. Pattern Recognit. 34, 2259–2281(2001)

4. Choi, H., Baraniuk, R.G.: Multiscale image segmentation usingwavelet- domain hidden markov models. IEEE Trans. Image Pro-cess. 10(9), 1309–1321 (2001)

5. Hyeokho, C., Richard, B.: Image segmentation using Wavelet-domain classification. In: Proceedings of the SPIE Technical Con-ference on Mathematical Modeling, Bayesian Estimation, andInverse Problems, pp. 306–320 (1999)

6. Clausi, D.A.: An analysis of co-occurrence texture statistics as afunction of grey level quantization. Can. J. Remote Sen. 28(1),45–62 (2002)

7. Comaniciu, D., Meer, P.: Mean shift: a robust approach towardfeature space analysis. IEEE Trans. Pattern Anal. Mach. In-tell. 24(5), 603–619 (2002)

8. Daugman, J.G.: Uncertainty relation for resolution in space, spatialfrequency, and orientation optimized by two-dimensional visualcortical filters. J. Opt. Soc. Am. 2, 1160–1169 (1985)

9. Deng, Y., Manjunath, B.S.: Unsupervised segmentation of color-texture regions in images and video. IEEE Trans. Pattern Anal.Mach. Intell. 23(8), 800–810 (2001)

10. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, chapter 4.2nd edn. Wiley, London (2000)

11. Fan, J., Yau, D.K.Y., Elmagarmid, A.K., Aref, W.G.: Auto-matic image segmentation by integrating color-edge extraction andseeded region growing. IEEE Trans. Image Process. 10(10), 1454–1466 (2001)

12. Figueiredo, M.A.T.: Bayesian image segmentation using wave-let-based priors. In: Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition, vol. 1,pp. 437–443 (2005)

13. Flores-Pulido, L., Starostenko, O., Flores-Quéchol, D., Rodrigues-Flores, J.I., Kirschning, I., Chávez-Aragón, J.A.: Content-basedimage retrieval using wavelets. In: Proceedings of the 2nd WSEASInternational Conference on Computer Engineering and Applica-tions, pp. 40–45 (2008)

14. Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features forimage classification. IEEE Trans. Syst. Man. Cybern. 3(6), 610–621 (1973)

15. Hong, B.W., Soatto, S., Ni, K.Y., Chan, T.F.: The scale of a textureand its application to segmentation. In: Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition, pp. 1–8 (2008)

16. Karam, O.H., Hamad, A.M., Ghoniemy, S., Rady, S.: Enhancementof wavelet-based medical image retrieval through feature evalua-tion using an information gain measure. In: Proceedings of theACM Symposium on Applied computing, pp. 220–226 (2003)

17. Liang, K.H., Tjahjadi, T.: Adaptive scale fixing for multiscale tex-ture segmentation. IEEE Trans. Image Process. 15(1), 249–256(2006)

18. Luo, Q., Khoshgoftaar, T.M.: Efficient image segmentation bymean shift clustering and MDL-guided region merging. In: Pro-ceedings of the 16th IEEE International Conference on Tools withArtificial Intelligence, pp. 337–343 (2004)

19. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of humansegmented natural images and its application to evaluating seg-mentation algorithms and measuring ecological statistics. In: Pro-ceedings 8th International Conference Computer Vision, vol. 2,pp. 416–423 (2001)

20. McQueen, J.: Some methods for classification and analysis ofmultivariate observations. In: Proceedings of the 5th BerkeleySymposium on Mathematical Statistics and Probability, vol. 1,pp. 281–297 (1967)

21. Palm, C.: Color texture classification by integrative co-occurrencematrices. Pattern Recognit. 37, 965–976 (2004)

22. Porebski, A., Vandenbroucke, N., Macaire, L.: Iterative featureselection for color texture classification. In: IEEE InternationalConference on Image Processing, vol. 3, pp. 509–512 (2007)

123

SIViP (2012) 6:231–245 245

23. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEETrans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

24. Suematsu, N., Ishida, Y., Hayashi, A., Kanbara, T.: Region-basedimage retrieval using Wavelet transform. In: Proceedings of the10th International Workshop on Database and Expert SystemsApplications, pp. 167–173 (1999)

25. Sumengen, B., Manjunath, B.S.: Multi-scale edge detection andimage segmentation. In: European Signal Processing Conference(2005)

26. Tamura, H.: Textural features corresponding to visual percep-tion. IEEE Trans. Syst. Man Cybern. 8(6), 460–473 (1978)

27. Tu, Z., Zhu, S.C.: Image segmentation by data-driven mar-kov chain and monte carlo. IEEE Trans. Pattern Anal. Mach.Intell. 24(5), 657–673 (2002)

28. Tzotsos, A., Iosifidis, C., Argialas, D.: Integrating texture featuresinto a region-based multi-scale image segmentation algorithm.In: Proceedings of the International Conference on Object-BasedImage Analysis, pp. 772–7776 (2006)

29. Xu, T., Gondra, I.: Texture map: an effective representation forimage segmentation. In: Proceedings of the C3S2E Conference,pp. 197–203. ACM (2009)

123