IEICE TRANS. INF. & SYST., VOL.E89–D, NO.7 JULY 20061

PAPER Special Issue on Machine Vision Applications

Visual Characterization of Paper Using Isomap and Local

Binary Patterns

Markus TURTINEN†a), Student Member, Matti PIETIKAINEN†,

and Olli SILVEN†, Nonmembers

SUMMARY In this paper, we study how a multidimensionallocal binary pattern (LBP) texture feature data can be visuallyexplored and analyzed. The goal is to determine how true pa-per properties can be characterized with local texture featuresfrom visible light images. We utilize isometric feature map-ping (Isomap) for the LBP texture feature data and performnon-linear dimensionality reduction for the data. These 2D pro-jections are then visualized with original images to study dataproperties. Visualization is utilized in the manner of selectingtexture models for unlabeled data and analyzing feature perfor-mance when building a training set for a classifier. The approachis experimented on with simulated image data illustrating differ-ent paper properties and on-line transilluminated paper imagestaken from a running paper web in the paper mill. The sim-ulated image set is used to acquire quantitative figures on theperformance while the analysis of real-world data is an exampleof semi-supervised learning.key words: texture analysis, visualization, paper inspection

1. Introduction

Texture analysis provides a useful set of tools for real-world paper inspection. Even though texture analysisresearch is rather old, there still are several differentdefinitions of texture itself [1] and only a few succesfulexploitations of texture methods have been made in thepaper industry.

The major problem with using texture informationin paper characterization is that the produced paperhas a very homogenous appearance and the featuresshould be highly discriminative to detect relevant tex-ture variations. The extreme conditions of paper millscause variations for image appearance and the featuresdescribing the papers should be invariant with respectto the monotonic gray-scale variations caused by, for ex-ample, illumination changes. Operating environmentsare usually hard to fix completely and the camera po-sition can change between installations. These issuesimpose requirements that the features should be ableto handle different rotations and scales as well. Alsocomputational issues are important in such applicationsto achieve, for example, a reasonable classification timefor acquired samples.

Manuscript received November 1, 2005.Manuscript revised January 18, 2006.Final manuscript received March 13, 2006.

†Machine Vision Group, Department of Electrical andInformation Engineering, University of Oulu, Finland

a) E-mail: dillian@ee.oulu.fi

Paper mainly consists of pulp, chemicals and flocs.Flocs are wood fiber bunches that are visible whenlooking through the paper against the light source. Inpaper characterization, the main goal is to determinethe properties of flocs, like their size and shape. Theycharacterize indirectly important properties of the pa-per such as formation, strength and printability. Papermaking procedure has random processes resulting in theflocs randomly distributing over the paper. This causesthe appearance of transilluminated paper to look likea stochastic texture even though some properties, likethe direction of production line, are visible from thefloc distributions.

Texture has been applied satisfactorily to paperdefect detection [2] and some attempts have also beenproposed for paper formation analysis [3], [4]. In thispaper, we aim to determine the general visual qualityof paper based on its textural appearance. This is ba-sically what paper procuders do subjectively after pro-duction when analyzing light through paper with thehuman eye.

No texture features or descriptors are availabe thatare the best in all kind texture analysis problems. Tra-ditionally, the best texture features for a given ap-plication are found by a comparative study using su-pervision and pre-labeled data, as in [5]. The datais analyzed and tested with different features and al-gorithms to find, for example, the most discriminat-ing features. In real-world applications the labelingis usually very problematic, causing great difficultiesin analysis. In addition, the computational burden ofoptimization and search algorithms in feature analysismight be far too high.

LBP texture features [6] detect microstructres, likeedges, lines, curves and flat areas from textures andhave performed very well in many real-world textureanalysis applications including paper characterization[7]. In that study a self-organizing map [8] was used asa user interface for classifying textured paper samplesproviding a useful view into the data. In this paper,we study how visualization of LBP texture data withIsomap [9] can be utilized in feature analysis, trainingdata selection and labeling. The dimensionality of thefeature data is reduced with Isomap to two dimensionswhich then is visualized utilizing both manifold struc-ture and original texture images. Visualization makes

2IEICE TRANS. INF. & SYST., VOL.E89–D, NO.7 JULY 2006

it possible to analyze and compare the performanceof the features semi-supervisely without using labeleddata. Isomap performs the higher-to-lower dimension-ality mapping differently compared to the SOM andpreserves relational distances between samples morefaithfully revealing the manifold structures of the data.The approach presented in [7] is more suitable for on-line use and classification tasks, but here we constructa tool for the early stage visual analysis of paper’s tex-ture and quality. The tool can be used when training atexture classifier and analyzing features with unlabeleddata.

2. Visual Paper Characterization and Texture

of Paper

Visual paper characterization is basically what paperproducers do subjectively after production by visual-izing transilluminated papers and try to make someassumptions about the quality. This requires a deepknowledge of paper and only experts might see the rel-evant quality differences. Our purpose is to create moreobjective and automated methods for visual paper char-acterization.

In the paper making process, individual fibers formlarger fiber bunches. These fiber flocs can be seen asdarker regions when visualizing transilluminated paper.Flocs are somehow randomly distributed over the pa-per causing the appearance of the paper to appear asstochastic texture as shown in Fig. 1.

Fig. 1 The appearance of the paper is a sum of many differentphases causing the final product to look like stochastic texture.

The properties of flocs, such as their size andshape, for example, indirectly characterize importantproperties of the paper. In this sense, visual paper char-acterization is culminated with paper’s texture analy-sis. Fig. 2 shows an example of simulated paper images,where the average floc size and amount changes, caus-ing the textural appearance to look different. In thisparticular example, the appearance of the paper can beutilized for deriving information about the paper for-mation.

Intensity (beta formation index) increases

Flo

csi

zein

cre

ase

s

Fig. 2 The average floc size (y-axis) and amount of flocs (x-axis, relative to beta formation index) increases causing the tex-tural appearance to look different.

Proper texture analysis methods might provideuseful tools to achieve objective information about dif-ferent paper properties [3], [4], [10], [11]. Flocs play themost important role in visual paper characterization.Depending on application, the characterization methodshould be able to determine different properties of flocs.For example, in an application comparable to humanmade general visual quality analysis or in formationmeasurement, the methods should tolerate small-scaleanisotropy caused by fiber orientation. The methodsshould be easy to adapt into new environments. Itis commonly known that texture appears to be differ-ent depending on the viewing distance. A good papercharacterization system should bear some variation inviewing distance but still be discriminative for detect-ing different floc sizes. In general it can be noted thatpaper characterization sets very high requirements onthe texture analysis methods.

3. Describing the Texture of Paper with Local

Binary Pattern Features

The local binary pattern (LBP) operator offers an effi-cive way of analyzing textures [6]. It has a simple the-ory and it combines properties of structural and statis-tical texture analysis methods. Fig. 3 shows how LBPfeatures are calculated. For each pixel in an image,a binary code is produced by thresholding its neigh-borhood (8 pixels) with the value of the center pixel.A histogram is then constructed to collect up the oc-currences of different binary patterns representing, forexample, different types of curved edges, spots and flatareas. LBP is invariant against monotonic gray-scalevariations and it has been extended to have rotationinvariant and multiscale properties.

The basic 8-bit version of the LBP operator con-siders only eight neighbors of each pixel at distance oneand is rotation variant. The maximum number of differ-ent local patterns that the 8-bit operator can representis 28, so the dimensionality of the feature data is 256.

TURTINEN et al.: VISUAL CHARACTERIZATION OF PAPER USING ISOMAP AND LOCAL BINARY PATTERNS3

7 3 5

6 5 1

2 3 8

1 0 1

1 0

0 0 1

1 2 4

128 8

64 32 16

threshold by

centre value

original 3x3

neighbourhood

codeword

10010101 = 1492 10

LBP operator

summing:

1+0+4+128+0+0+0+16 = 149

1 0 4

128 0

0 0 16

Fig. 3 Computation of LBP feature.

Ojala et al.[6] proposed powerful extensions to the ap-proach to handle larger scales than one and presenteda rotation invariant version of the operator. These ex-tensions make the LBP very attractive for use in papertexture analysis. The neighborhood of the center pixelis considered to be circular and any number of neighborsamples can be selected from the circular perimeter.Neighbor samples are interpolated on the circle withequal space. In the multiresolution model of LBP, sep-arate operators at different scales are first constructedand the final feature vector is a combination of individ-ual feature vectors created simply by joining them oneafter another.

The size of the feature distribution increasesrapidly when the number of neighbor samples grows.For example with 16 neighbors the size of the histogramwould be 216 bins which is impractical to use in any re-alistic applications. The size of the feature distributionis reduced by considering only so called ’uniform’ pat-terns, where the maximum amount of bit-wise changesin the circular neighborhood from one to zero or viceversa is limited. Usually the maximum number of bitchanges is allowed to be two. It is observed that thesekinds of uniform patterns represent a great majority,sometimes over 90 percent of all local patterns in theimage and the loss of information is not that signifi-cant in reduction. With a uniform pattern the size ofthe feature vector with 8-bit LBP is reduced to 59 bins,where 58 are the actual uniform patterns and the lastone contains all the others.

Rotation invariance of LBP is achieved consider-ing only a small rotation invariant subset of the origi-nal binary patterns. They are constructed rotating theneighborhood set clockwise so many times that a max-imal number of the most significant bits is zero. Sorotation invariance is basically derived by normalizingthe original feature: the neighborhood code is shifted toits minimum. Rotation invariance also reduces the sizeof the feature distibution and with and 8-bit LBP op-erator there are totally 36 rotation invariant patterns.The quantization in angular space is quite crude, butusing larger spatial neighborhoods and multiple scales,the quantization can be finer.

4. Visual LBP Feature Analysis with Isomap

Isomap [9] is a non-linear dimensionality reductionmethod built on classical MDS [12]. It tries to preservethe intrinsic geometry of the data utilizing geodesicmanifold distances, instead of Euclidean distances, be-tween all pairs of data points. The Isomap procedureis the following [9] [13]:

1. Determine a neighborhood graph G of the observeddata xi in a suitable way. For example G mightcontain xixj if xj is one of the k nearest neigh-bors of xi (and vice versa). Alternatively, G mightcontain the edge xixj if |xi − xj | < ǫ, for some ǫ.

2. Compute the shortest paths in the graph for allpairs of data points. Each edge xixj in the graphis weighted by its Euclidean length |xi −xj |, or bysome other useful metric.

3. Apply MDS to the resulting shortest-path distancematrix D to find a new embedding of the data inEuclidean space, approximating Y .

The Isomap algorithm builds a neighborhoodgraph using k nearest neighbors or certain distancethreshold ǫ. The distances are calculated along themanifold instead of direct Euclidean distances and fi-nally the classical MDS is run for this distance data.There is a guarantee of asymptotic convergence to thetrue structure of the data when running Isomap with’sufficiently large’ sample sets [14]. Isomap can alsorecover the true intrinsic dimensionality of the non-linear manifolds utilizing residual variance of projec-tions, and when representing the data with global co-ordinates it can provide a very useful way of analyzinghigh-dimensional observations [9]. Fig. 4 demonstratesthe power of Isomap in manifold learning and visual-ization using a synthetic three-dimensional ”Swiss roll”data set. Data points are colored according to their lo-cations on the manifold. It can be seen that Isomapcan recover the manifold structure very well.

-1-0.5

00.5

1 0

2

4

-1

-0.5

0

0.5

1

1.5

2

2.5

3

X

Y

Z-6 -4 -2 0 2 4 6

-3

-2

-1

0

1

2

3

First Isomap component

Se

con

dIs

om

ap

com

po

ne

nt

a) Original 3d data b) 2d Isomap projection

Fig. 4 Isomap on 3-dimensional ”Swiss roll” data with 2000points.

Multi-dimensional LBP features can be visualizedas 1D histograms. This does not provide very clear rep-resentation of the data, especially when the amount of

4IEICE TRANS. INF. & SYST., VOL.E89–D, NO.7 JULY 2006

the samples to be visualized increases, as shown in Fig.5. Thus more intelligent and user friendly methods forvisualizing the feature data are needed. We proposeto use Isomap to reduce the dimensionality of multi-dimensional LBP feature data. Isomap is applied in thesimilar manner than in the synthetic example shown inFig. 4, but now the input data is multi-dimensionalLBP feature data. Isomap projections are then visu-alized to study how the feature data behaves on thelow-dimensional manifolds. Instead of using only datapoints in the visualization, one can attach original tex-ture images to them and see how the texture appear-ance changes on the manifold.

0 2550

200

Pattern #

Num

ber

ofpattern

s

0 2550

200

Pattern #

Num

ber

ofpattern

s

Fig. 5 256 bins LBP histograms of two samples from differentclasses.

Visual analysis of Isomap projections is based onthe fact that similar data in the feature space tend tocluster close to each other also in the low-dimensionalspace. If the features are good enough one can see fromthe projection that there is clear dependence betweenthe texture appearance and spatial location in the low-dimensional projection space. Fig. 6 illustrates thisidea: 256 dimensional LBP features are extracted fromimage samples and these are projected with Isomap.It can be seen from the projection that two textureclasses can be separated by drawing a boundary on the2d Isomap plane (near x=0). The user can visualize theprojection with original texture images and determinehow features can discriminate different classes. Alsoa separate clustering algorithm, like k-means, can berun on projection coordinates to help visual analysis,especially when there is some pre-knowledge about, forexample, the number of classes.

Visual exploration of multidimensional texturedata can be utilized in several texture analysis applica-tions, including the unsupervised learning and trainingof a classifier. In this paper, we exploit it primarilyin feature performance analysis. Different LBP opera-tors are used for modeling paper textures and a visualanalysis is then made. Isomap has a few properties thatmake it attractive to use in this kind of application: dis-tance relations between samples are preserved very well

-200 -100 0 100 200-150

0

150

First Isomap component

Se

con

dIs

om

ap

com

po

ne

nt

Fig. 6 2d Isomap projection of six samples from two textureclasses.

in the dimensionality reduction, and the global struc-ture of the data is represented faithfully. These proper-ties culminate feature performance analysis with visualcluster analysis: if the features used are discriminativeenough we are able to visualize appearance differencesin the projection space, otherwise different textures aremixed badly in the projection space. Of course, some ofthe clustering information is lost in the dimensionalityreduction and the clusters might be more overlappedand divided into several parts in the low-dimensionalspace. The main problem is to determine how badlythe classes are mixed with each other. This can bedone by visualization, also with unlabeled data.

In addition to the visualization of original imagesand projection points, other properties of the Isomapprojection can also be utilized in feature analysis. In[15] it was adviced to use the measure of residual vari-ance to characterize how well the low-dimensional Eu-clidean embedding captures the geodesic distances es-timated from the neighborhood graph to determine theoptimal neighborhood size. Residual variance can alsobe utilized when determining the true dimensionalityof the data [9]. The residual variance is defined asRes = 1 − R2(DG, DY ), where DY is the matrix ofEuclidean distances in the low-dimensional embedding,DG is the graph distance matrix and R is the standardlinear correlation coefficient, taken over all entries ofDG and DY . According to [15] appropriate neighbor-hood size selection can be done based on a trade-offbetween maximizing the number of points captured inthe Euclidean embedding and minimizing the residu-als. When estimating the optimal neighborhood sizeit is calculated over projections obtained by differentvalues of k or ǫ.

In LBP feature analysis, and such applications ingeneral, the maximization of captured points is not socrucial as minimization of residuals. This is because ef-fective features cluster data in the feature space and dif-ferent clusters might be so distant that they will be dis-connected when constructing the neighborhood graphin the Isomap procedure. Especially with small neigh-borhoods there might be found several disconnectedsub-manifolds from the data and this is not necessar-

TURTINEN et al.: VISUAL CHARACTERIZATION OF PAPER USING ISOMAP AND LOCAL BINARY PATTERNS5

ily an unfavorable thing if the sub-manifolds themselvesare not badly confusing several texture apperances. Vi-sualization of projections calculated from sub-manifoldscan be used for examining the power of the features.With smaller neighborhoods the local properties of thedata are preserved better than with large neighbor-hoods. In applications where features are able to createclearly distinct clusters of classes, it might be a verygood idea to use small neighborhood sizes and createone’s own Isomap projections from sub-manifolds. Lo-cal properties of the data are also preserved better whenusing smaller neighborhoods. With more homogenousdata the features can not discrimate classes that clearly.

The true intrinsic dimensionality of the LBP tex-ture data is usually much smaller than the original di-mensionality, typically between three and fifteen. Thissupports the idea of using dimensionality reduction andvisualization in texture data analysis. Kouropteva et al.[16] proposed a data visualization framework in whichthey reduced the dimensionality of the original datawith locally linear embedding (LLE) [17] first into es-timated intrinsic dimensionality. After that they madepairwise plots of each new dimension and calculatedthe joint mutual information (JMI) criteria [18] fromeach plot to find the optimal visualization plane. Thesame kind of approach can also be utilized with Isomap.There are several ways of estimating the intrinsic di-mensionality of the data. One can, for example, applyPCA to the data and find how many dimensions arerequired to represent majority, let say 95 percent, ofthe data. Another approach is to estimate the intrinsicdimensionality from residual variance plots of Isomap.

Typically paper texture data is so homogenousthat there will be no disconnected components in thelow-dimensional embedding even when small neighbor-hoods are used. The residual variance and JMI criteriacan be utilized when finding the optimal mapping fromthe feature space to the 2D. The goal is to select sucha projection that provides useful visualization informa-tion and represents the original data optimally.

5. Experiments

In the experiments, both simulated and real-world pa-per images were used. The appearance of the paperwas simulated with a simple periodic function z(x, y) =r ∗ sin(tx ∗x)+ cos(ty ∗ y), where r ∈ [1,3], tx,ty ∈ [1,6]and x,y ∈ [1,10π]. The dark areas of the synthetizedimage can be interpreted as flocs and the bright areasas voids. The images were categorized into 11 classesbased on the ”floc size” (so that tx+ty ∈ [2,12]). Totally720 images were generated and Gaussian noise with µ

= 0.0 and σ = 0.01 was added to the images to cre-ate more realistic 2d paper textures. In the resultingimages, the form, and in some cases, the orientation ofthe ”flocs” varied, but the mean size of the ”flocs” re-mained relatively constant. Fig. 7 shows two examples

Table 1 Classification results of synthetized data set usingdifferent LBP operators.

LBP operator l-o-o rate [%] DimensionalityLBP8,1 87.9 256LBP

ri8,1 72.4 36

LBPu28,1 85.8 59

LBPriu28,1 74.2 10

LBPu28,1+16,2+24,3 98.1 857

LBPriu28,1+16,2+24,3 99.3 54

LBPu28,1+16,2+24,3 99.6 857

LBPriu28,1+16,3+24,5 99.9 54

of generated samples (tx+ty=7).

Fig. 7 Examples of synthetized texture samples.

The real world data set consisted of paper imagesacquired from the Stora Enso papermill in Oulu, Fin-land. The running paper web was imaged for one andhalf months in December 2003 and January 2004 with15 minute intervals between images using a fast, highquality CCD camera. The imaging system was installedby Honeywell Corporation but in this paper we do notdescribe details of the imaging system. The main in-terest is that the system is able to produce images withreasonable quality. Images taken during the downtimeof the mill were rejected. The total number of sampleswas 2316.

5.1 Visual Feature Analysis

The synthetized data set was used in this experiment.Leave-one-out 3-NN classifications with eight differentLBP operators were first made. Table 1 shows theclassification results obtained. These results representthe ground truth for different features. Our aim isto demonstrate how visualization of the features withIsomap can be used when analysing the performance ofdifferent LBP operators.

For each feature set, its own Isomap projectionwas created using ten different neighborhood sizes(k=[3,5,...,21]). If the features cluster the data into sep-arate clusters, Isomap can detect these efficiently whenusing small neighborhoods. With k=3, the number ofconnected components varied between 3–21 with differ-ent features. With larger neighborhoods (k ≥ 13) onlyone connected component was found with every fea-ture set. The intrinsic dimensionality of the data wasestimated from the residual variance versus Isomap di-mensionality plots by searching for the ”elbow” from

6IEICE TRANS. INF. & SYST., VOL.E89–D, NO.7 JULY 2006

the plot [9]. Then the JMI criteria was used to selectthe most informative dimensionality components whenconstructing the 2D projection.

Analysis of the residual variance plots suggests,for example, that with LBPu2

8,1 features the neighbor-hood size k ≥ 7 should be used, as illustrated in Fig.8. With LBP riu2

8,1+16,3+24,5 features k = 3 providesthe best result. Applying Isomap with k=3 for thesedata sets it found 5 sub-manifolds from LBPu2

8,1 (orig-inal dimensionality = 59) and 19 sub-manifolds fromLBP riu2

8,1+16,3+24,5 (original dimensionality = 54) featuredata. The largest sub-manifolds are shown in Figs. 9and 10. The multiresolution and rotation invariant op-erator can discriminate classes (only 2 involved in thisprojection) very well, but with the LBPu2

8,1 operatorthere are confusions between classes (9 classes in theprojection). In fact, sub-manifolds of the multiresolu-tion operator could discriminate the classes nicely butsome of the classes are fragmented into several sub-manifolds (19 clusters and 11 classes). Visualization ofprojections using the original textured images revealsrapidly that with LBPu2

8,1 features the classes are badlymixed. Figs. 11 and 12 show the projections using alarger neighborhood (k=15). Now the local propertiesof the data are not preserved as well as with smallerneighborhoods and there is not that significant differ-ence between projections. Both projections representthe classes interestingly constucting threads on the pro-jection plane and with LBPu2

8,1 features the projectionlooks better than with the smaller k value.

3 5 7 9 11 13 15 17 19 210

0.05

0.1

0.15

0.2

0.25L B P

8,1

u2texture data

Err

or

frac

tio

n

Neighborhood size (k)3 5 7 9 11 13 15 17 19 21

0

0.2

0.4

0.6

0.8

1L B P

8,1+16,3+24,5

riu2texture data

Err

or

frac

tio

n

Neighborhood size (k)

a) b)

Fig. 8 Error fraction plots of LBPu28,1 (a) and LBP

riu28,1+16,3+24,5

(b) features. The residual variances for each size of neighbor-hoods in the blue solid line and the fraction of points not includedin the largest Euclidean embedding in the dotted red line.

-2 -1 0 1 2

x 104

-2

-1.5

-1

-0.5

0

0.5

1x 10

4

First Isomap component

Se

con

dIs

om

ap

com

po

ne

nt

Fig. 9 Isomap projections of LBPu28,1 features using k=3 near-

est neighbors.

-1 0 1 2

x 104

-1

-0.5

0

0.5

1

1.5

2

2.5x 10

4

First Isomap component

Se

con

dIs

om

ap

com

po

ne

nt

Fig. 10 Isomap projections of LBPriu28,1+16,3+24,5 features using

k=3 nearest neighbors.

-1 -0.5 0 0.5 1

x 104

-4000

-2000

0

2000

4000

6000

8000

First Isomap component

Se

con

dIs

om

ap

com

po

ne

nt

Fig. 11 Isomap projections of LBPu28,1 features using k=15

nearest neighbors.

-4 -2 0 2 4

x 104

-2

-1

0

1

2

3

x 104

Se

con

dIs

om

ap

com

po

ne

nt

First Isomap component

Fig. 12 Isomap projections of LBPriu28,1+16,3+24,5 features using

k=15 nearest neighbors.

The results indicate that a proper visualization isuseful when studying the performance of the texturefeatures. Visualization reveals which features are ableto discriminate the data and the experiments showed agood correlation with ground truth classifications. Themost informative projection for each feature set is foundautomatically, but in practice one should also visual-ize the projection obtained with a small neighborhood.This is because with small neighborhoods the Isomapmay detect different classes automatically and repre-sent the data using disconnected manifolds. This helpsthe visualization task because there is not so much datainvolved.

5.2 Creation of Training Set

In the second experiment, real-world image data wasused. Our main aim was to develop a method used totrain a classifier for classifying on-line paper samples todifferent ’quality classes’ according to their appearance.The same features as in the previous experiment were

TURTINEN et al.: VISUAL CHARACTERIZATION OF PAPER USING ISOMAP AND LOCAL BINARY PATTERNS7

extracted and Isomap projections with the same pa-rameters were constructed. With all features and sizesof neighborhoods the Isomap found only one connectedcomponent from the data. So the data does not haveas clear class structures as in the previous experimentsand the textures are more homogenous and uniformlydistributed.

For such a homogenous and hardly visualizabletexture material as paper, it is difficult to find thebest LBP operator to be used. But when visualizingthe projections carefully we notice that the roughnessof the paper texture changes relatively smoothly whenmoving from one end of the manifold to the other. Theroughness correlates with the formation and printingproperties of paper. Paper producers want their paperto be as uniform and smooth as possible to assure goodquality and press properties of the paper.

By selecting a group of samples from different partsof the manifold for visualization, we can try to locatethe apparent ’class boundaries’. For clarification, theidea is illustrated in Fig. 13 using texture samples thatare easier to visualize than paper images but the pro-cedure is similar also with paper data. Fig. 14 showsthe projection of LBP riu2

8,1+16,2+24,3 data extracted fromon-line paper samples labeled to five roughness classes.Because the visual labeling is quite subjective for such amaterial, we first applied the k-means clustering to theprojection points and then visualized and adjusted theboundaries. The left end of the projection representssmooth paper while the texture is rougher at the rightend. After labeling, we can train a classifier, like k-NN,using recently labeled samples as the training data andclassify on-line paper samples with it.

Fig. 13 The user can select regions from the projection planeand study how well the samples are discriminated.

Visual labeling of the data, and the creation of atraining set can be utilized in various texture analysistasks. The class boundaries can be found by visualizingthe 2D projection (depending on features) and with aneffective user interface a large number of samples canbe easily labeled. For a human, it is very difficult tosee differences between homogenous samples. The visu-alization of Isomap projections is therefore useful with

Se

con

dIs

om

ap

com

po

ne

nt

First Isomap component

Fig. 14 Isomap projection labeled to five roughness classes.

such data because the user can study data points locallyand globally, and determine if there is some change intexture appearances when moving on the manifold.

6. Discussion

Early data processing, including feature performanceanalysis, data labeling and training set creation for clas-sification are laborious, but fundamental, tasks whencharacterizing textures in a real-world environment. Inthis paper, we studied the combined use of LBP tex-ture features and the Isomap dimensionality reductionmethod for analyzing transilluminated paper textures.The 2D Isomap projections of high-dimensional featuredata were visualized and utilized in the above men-tioned tasks. Visualization of the 2D manifolds andthe original texture images reveals interesting informa-tion about the feature data and shows how well featurescan discriminate different texture samples. Visualiza-tion can be used when constructing the training setfor classification and the user can select representativetraining samples and label them.

The approach was experimented on with both syn-thetized and real-world paper texture images. Withsynthetized data it was shown that the visualiza-tion based method works well in feature performanceanalysis of different LBP operators. Real-world datawas used to demonstrate capabilities in semisupervisedlearning: paper texture samples were visually labeledaccording to their appearance.

The analysis tool described in this paper can beutilized in various texture analysis problems and mightprovide very useful information about the inspecteddata. Paper textures are very homogenous, but to char-acterize them is a difficult texture analysis problem.With the tool presented here, papermakers can performvisual grading for the produced paper more objectivelyand analyze what would be the properties of the pro-duced paper. It is also possible to analyze separateoff-line data sets and create a classification training setfor characterizing papers according their textural ap-

8IEICE TRANS. INF. & SYST., VOL.E89–D, NO.7 JULY 2006

pearance on-line.

Acknowledgments: Financial support provided bythe Infotech Oulu Graduate School is gratefully ac-knowledged.

References

[1] M. Tuceryan and A. Jain, “Texture analysis,” C. Chen andL. Pau and P. Wang (eds.) Handbook of Pattern Recog-nition and Computer Vision, pp.207–248, World Scientific,Signapore, 2nd Edition, 1999.

[2] J. Iivarinen and J. Rauhamaa, “Surface inspection of webmaterials using the self-organizing map,” Intelligent Robotsand Computer Vision XVII: Algorithms, Techniques, andActive Vision, pp.96–103, 1998.

[3] T. Cresson and P. Luner, “Characterization of paper for-mation, part 2: The texture analysis of paper formation,”Tappi Journal, vol.73, no.12, pp.175–184, 1990.

[4] M. Bouyndain, J. Colom, R. Navarro, and J. Pladellorens,“Determination of paper formation by fourier analysis oflight transmission images,” Appita Journal, vol.54, no.2,pp.103–105, 115, 2001.

[5] T. Randen and J. Husoy, “Filtering for texture classifica-tion: A comparative study,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol.21, no.4, pp.291–310, February 1999.

[6] T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolu-tion gray-scale and rotation invariant texture classificationwith local binary patterns,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol.24, no.7, pp.971–987, 2002.

[7] M. Turtinen, M. Pietikainen, O. Silven, T. Maenpaa,and M. Niskanen, “Paper characterization by texture us-ing visualization-based training,” International Journal ofAdvanced Manufacturing Technology, vol.22, no.10–11,pp.890–898, 2003.

[8] T. Kohonen, Self-organizing Maps, Springer-Verlag, Berlin,Germany, 1997.

[9] J. Tenenbaum, V. de Silva, and J. Langford, “A global geo-metric framework for nonlinear dimensionality reduction,”Science, vol.290, pp.2319–2323, December 2000.

[10] D.S. Keller, J. Lewalle, and P. Luner, “Wavelet analysis ofsimulated paper formation,” Paperi ja Puu, vol.1999, no.7,pp.499–505, 1999.

[11] B. Jordan and N. Nguyen, “Specific perimeter - a graininessparameter for formation and print-mottle textures,” Paperija Puu, vol.6, no.7, pp.476–482, 1986.

[12] T. Cox and M. Cox, Multidimensional Scaling, Chapmanand Hall, London, 1994.

[13] V. de Silva and J. Tenenbaum, “Global versus local meth-ods in nonlinear dimensionality reduction,” Neural Informa-tion Processing Systems 15 (NIPS’2002), pp.705–712, 2002.

[14] M. Bernstein, V. de Silva, J. Langford, and J. Tenen-baum, “Graph approximations to geodesics on embed-ded manifolds,” tech. rep., Department of Psychol-ogy, Stanford University, USA, 2000. Available athttp://isomap.stanford.edu/BdSLT.pdf.

[15] M. Balasubramanian, E. Schwartz, J. Tenenbaum,V. de Silva, and J. Langford, “The isomap algorithm andtopological stability,” Science, vol.295, p.7a, 2002. Techni-cal Comments.

[16] O. Kouropteva, O. Okun, and M. Pietikainen, “Data visu-alization with multiple machine learning methods,” FourthIASTED International Conference on Visualization, Imag-ing, and Image Processing, pp.190–196, 2004.

[17] S.T. Roweis and L.K. Saul, “Nonlinear dimensionality re-duction by locally linear embedding,” Science, vol.290,pp.2323–2326, December 2000.

[18] R. Moddemeijer, “On estimation of entropy and mutualinformation of continuous distributions,” Signal Processing,vol.16, no.3, pp.233–246, 1989.

Markus Turtinen received the M.Sc.degree from the Department of Electricaland Information Engineering at the Uni-versity of Oulu in 2002. He is working inthe Machine Vision Group at he Univer-sity of Oulu as a postgraduate student ofthe Infotech Oulu Graduate School. Hisresearch interest include texture-basedimage analysis, visual inspection and vi-sualization of high-dimensional data.

Matti Pietikainen received his Doc-tor of Technology degree in electrical engi-neering from the University of Oulu, Fin-land, in 1982. In 1981, he establishedthe Machine Vision Group at the Univer-sity of Oulu. The research results of hisgroup have been widely exploited in in-dustry. Currently, he is Professor of Infor-mation Engineering, Scientific Director ofInfotech Oulu research center, and Leaderof Machine Vision Group at the Univer-

sity of Oulu. From 1980 to 1981 and from 1984 to 1985, he visitedthe Computer Vision Laboratory at the University of Maryland,USA. His research interests are in machine vision and image anal-ysis. His current research focuses on texture analysis, facial imageanalysis, and machine vision for sensing and understanding hu-man actions. He has authored about 180 papers in internationaljournals, books, and conference proceedings, and about 100 otherpublications or reports. He is Associate Editor of Pattern Recog-nition journal and was Associate Editor of IEEE Transactionson Pattern Analysis and Machine Intelligence from 2000 to 2005.He was Chairman of the Pattern Recognition Society of Finlandfrom 1989 to 1992. Since 1989, he has served as a member ofthe governing board of the International Association for PatternRecognition (IAPR) and became one of the founding fellows ofthe IAPR in 1994. He has also served on committees of severalinternational conferences. He is a senior member of the IEEEand Vice-Chair of IEEE Finland Section.

Olli Silven received the M.Sc. andDr.Tech. degrees in Electrical Engineer-ing from the University of Oulu in 1982and 1988, respectively. Since 1996 he hasbeen the professor in signal processing en-gineering. His research interests are inmachine vision and signal processing. Hehas contributed to several system solu-tions ranging from reverse vending ma-chines to vision based lumber sorters.