Io

SDT

a

ARRAA

KIBCIO

1

nqaB2BfeJTscip(tta

s

0d

Computers and Chemical Engineering 33 (2009) 421–428

Contents lists available at ScienceDirect

Computers and Chemical Engineering

journa l homepage: www.e lsev ier .com/ locate /compchemeng

mage quality in image classification: Design and constructionf an image quality database

huo Yan, Saed Sayad, Stephen T. Balke ∗

epartment of Chemical Engineering and Applied Chemistry, University of Toronto, 200 College Street,oronto, ON M5S 3E5, Canada

r t i c l e i n f o

rticle history:eceived 5 July 2007eceived in revised form 28 July 2008ccepted 13 October 2008vailable online 5 November 2008

a b s t r a c t

Image quality affects automated classification of images from process camera monitors. The objective ofthis work was to obtain a database of reference images that could enable automated, customized imagequality modification to improve classification of new images. Here, images from an extruder monitorwere to be classified as either showing or not showing contaminant particles in a polymer melt. A novel

eywords:mage qualityayesianlassification

mage quality databaseptimization

task-based definition of image quality was important to this work: image quality was defined in terms ofthe probabilities of a particle being present and not being present in the image as assigned by a Bayesianclassification model. Image quality was optimized using a Nelder Mead Simplex search. The optimizedimage was classified using another Bayesian model to demonstrate the improved classification perfor-mance. The resulting reference image database consisted of image similarity attributes describing eachraw image and the corresponding quality improvement instructions from the optimization. The next stepis to use the database to improve the quality and classification of new images.

ancaanuvii

attObc

. Introduction

The use of in-line cameras for process monitoring results in theeed to automatically identify image objects relevant to productuality. Image monitoring examples range from chemical unit oper-tions such as polymer extrusion (Farahani et al., 2003; Gilmor,alke, Calidonio, & Rom Roginski, 2003; Torabi, Ing, Sayad, & Balke,002), to industrial process control (Yu, MacGregor, Haarsma, &ourg, 2003), to biological processes such as cell growth and

ermentation (Joeris, Frerichs, Konstantinov, & Scheper, 2002), tolectronic manufacturing of printed circuit boards (Darwish &ain, 1988; Persoon, 1988) and wafer manufacture (Yoda, Ohuchi,aniguchi, & Ejiri, 1988). Previously, Torabi, Sayad, and Balke (2005)howed how a Bayesian classification model could automaticallylassify images from a camera monitoring flowing polymer meltn an extruder into those showing a contaminant particle (“witharticle” (WP) images) and those not showing such a particle

“without particle” (WO) images). The model was made adaptiveo enable it to adjust to changes in image quality and classifica-ion accuracy was approximately 90%. However, for process controlpplications, higher accuracy is very desirable, as is wide adapt-

∗ Corresponding author. Tel.: +1 416 978 7495; fax: +1 416 978 8605.E-mail addresses: balke@chem-eng.utoronto.ca,

tephen.balke@utoronto.ca (S.T. Balke).

jr(autaoe

098-1354/$ – see front matter © 2008 Elsevier Ltd. All rights reserved.oi:10.1016/j.compchemeng.2008.10.013

© 2008 Elsevier Ltd. All rights reserved.

bility to changes in image quality. Torabi et al. (2005) had madeo attempt to customize image quality improvement previous tolassification: they applied the same basic brightness adjustmentnd background flattening to all input images. They achieved somedaptability by incrementally training the classification model withew images when image quality changed. This situation motivateds to examine the possibility of examining each input image pre-ious to classification and applying a customized modification inmage quality to improve classification accuracy and adaptabil-ty.

The first step in accomplishing this was to design and construct“reference image database” that would contain a description of

he diverse raw image types to be encountered along with instruc-ions on how to modify each image to improve its classification.nce such a database is available, then methods such as case-ased reasoning can be used to find the reference image mostlosely resembling a new image. Use of the database is the sub-ect of a subsequent paper. In this paper, defining image qualityelevant to classification, selection of image quality (IQ) operatorsthe image processing methods available to change image quality)nd their order of application, determination of parameter val-

es in IQ operators and finally, construction of the database, arehe focus. The next section summarizes theory relevant to thesespects. Then the experimental extrusion work and software devel-pment is described. Finally, how the tasks were accomplished andxperimentally verified is shown.

4 mical

2

2

w(1e(2t(22H

tcmvbl(u

ineAts

2

ostaaii

o&B2P&1

ttpthmww

2

s

mttiqv(miqadv

2

omnttTtto

snai(tw(moTt(aoc

sr

2

iactT

P

22 S. Yan et al. / Computers and Che

. Theory

.1. Defining image quality: Image quality metrics

Image quality metrics (IQ Metrics) are the quantitative valueshich define image quality. Two main uses are quality control

Baykut, Atalay, Ercil, & Güler, 2000; Darwish & Jain, 1988; Persoon,988; Siew, Hodgson, & Wood, 1988; Torabi et al., 2002; Yodat al., 1988) and the benchmarking of image processing methodsEskicioglu & Fisher, 1995; Wang & Bovik, 2002; Wang, Lu, & Bovik,004; Wang, Sheikh, & Bovik, 2002). Image quality then is often aask-dependent quantity. This is evident in much of the literatureAvcibas, Sankur, & Sayood, 2002; Eskicioglu, 2000; Fiete & Tantalo,001; Jacobson, 1995; Shnayderman, Gusev, & Eskicioglu Ahmet,004; Veldkamp & Karssemeijer, 2000; Wang et al., 2002; Xu &auske, 1994).

Objective IQ Metrics that do not require a reference image arehe emphasis in this work. They appear to be the most practi-al for in-line monitoring. Five IQ Metrics are used here. Theseetrics and their respective methods of calculation have been pre-

iously detailed: noise (Chen et al., 2003; Chuang & Huang, 1992),rightness (Yan, 2007), global contrast by Michelson (Peli, 1997),

ocal contrast by King-smith and Kulikowski (Peli, 1997), blurrinessMarziliano, Dufaux, Winkler, & Ebrahimi, 2002) and illuminationniformity (Yan, 2007).

Changing the values of IQ Metrics requires the application ofmage quality operators (IQ operators). These are the subjects of theext section. Then, the problem becomes determining the param-ter values for the IQ operators that provide the best quality image.s described in Section 2.3, the method used to do that requires

hat values of the five IQ Metrics listed above be summarized as aingle number denoting image quality.

.2. Image quality operators

There are a large number of well-established image qualityperators (IQ operators) available. ImageJ, the image processingoftware used in this work, for example, has 54 different opera-ors. When it is realized that most operators contain their owndjustable parameters, that they can be applied more than once,nd that their order of application affects the results on the image,t can be seen that the dimension of the operator selection problems staggering. A strategy for their application is needed.

Over the years, many systems and methods have been devel-ped for image analysis (e.g., Beksac, Egemen, Izzetoglu, Ergun,Erkmen, 1996; Belien, Somi, de Jong, van Diest, & Baak, 1999;

latt, Clark, Courtney, Tully, & Tucker, 2004; Krooshoop et al.,003; Kuklin, Shams, & Shah, 2000; Mahadevan & Casasent, 2003;etropoulos, Sibbitt, & Brooks, 1999; Tanaka & Kayama, 2001; WitBusscher, 1998; Yeasin & Chaudhuri, 2000; Zalewski & Buchholz,

996).However, from all of the vast amount of literature in this area,

here is no real guidance how best to automatically use IQ opera-ors to improve performance of an image classification model. Therimary difficulty is defining image quality so that it is relevanto classifier performance. An image that appears superior to theuman eye may not be considered superior to the classificationodel! The struggle to define image quality is a significant part ofhat was done to accomplish image quality improvement in thisork and will be detailed in Section 5.

.3. Modifying image quality for improved classification

Once the IQ operators and their order of application have beenelected for a specific raw image the problem is to rapidly deter-

P

Engineering 33 (2009) 421–428

ine the best values of the parameters in the IQ operators so aso obtain the highest quality image. The approach used here waso define image quality with a single number and to use a numer-cal optimization program to adjust the parameters in the imageuality operators in order to maximize image quality for each indi-idual image. In this case the Nelder Mead Simplex OptimizationWalters, Parker, Morgan, & Deming, 1991) method was used. The

ethod systematically guessed the values of the parameters, mod-fied the image, computed image quality and iterated until imageuality was a maximum. The optimization method is well knownnd has the advantage of not requiring calculation of derivatives. Aisadvantage of the method is that it may fail, or at least becomeery inefficient, if large numbers of parameters are selected.

.4. Design and construction of the reference image database

In order to be used to customize the quality of a new imagebtained by the monitoring system the reference image databaseust allow the reference image in the database most similar to the

ew image to be located. Then, it must supply instructions on howhat similar image was modified for improved classification so thathe same application of IQ operators can be done on the new image.hus, for each image, values of “similarity attributes” which allowhe “most similar” image to be identified are required, along withhe identity of IQ operators, values of their parameters and theirrder of application.

In image analysis and processing, attributes sometimes con-ist of non-image information and image information. They can beumerical, categorical or symbolic. The attributes used to describen image case depend on the type of image and the task ofmage interpretation. A notable example is the work of PernerPerner, 1999, 2002). who selected image attributes to enable CBRo find image segmentation parameters for similar images. In thisork the similarity attributes selected were: the five IQ metrics

noise, brightness, contrast, blurriness and illumination unifor-ity) described in Section 2.1 and, in addition, the two attributes

f mean pixel density and percentage area of the feature object.he mean pixel density and percentage area are extracted by firsthresholding the image and then measuring the feature objectTorabi, Sayad, & Balke, 2006; Yan, 2007). Together, these seventtributes are referred to here as image metrics. They describe notnly the image quality characteristics of a raw image but also theontent (i.e., particles) of a raw image.

The effectiveness of this selection of attributes will be demon-trated in the next publication where the application of case-basedeasoning is used to select the most similar image.

.5. Classification method

In this work and in Torabi et al. (2005), Bayesian classifications used. In Torabi’s research, six attributes were extracted fromn image and it was assumed that those attributes were statisti-ally independent by expressing the conditional probabilities onhe right hand side of the equation by the product of probabilities.hat is

(C = WP|X = x) = P(C = WP)P(X = x|C = WP)P(X = x)

= P(C = WP)∏k

1P(Xi = xij|C = WP)

P(X = x)(1)

(C = WO|X = x) = P(C = WO)P(X = x|C = WO)

P(X = x)

= P(C = WO)∏k

1P(Xi = xij|C = WO)

P(X = x)(2)

S. Yan et al. / Computers and Chemical Engineering 33 (2009) 421–428 423

P); s

cicciapbftPtrntaujaTd

tfpi

P

wtiiiae

P

c

o

a

3

teTpots

uwpp

4

dItim(

uI

b

o

d

t

Fig. 1. Sample image with particle (W

where P(C = WP | X = x) is the probability that the image should belassified as WP given that the attribute values are given by x; sim-larly P(C = WO | X = x) is the probability that the image should belassified as WO given that the attribute values are given by x. Thelassification model will classify the image as WP if P(C = WP | X = x)s larger than P(C = WO | X = x) and as WO otherwise. In practice thectual probability values are not calculated or compared; ratherrobability densities are used. This will be explained in more detailelow. Also, the quantity P(X = x) in the denominator is omittedrom Eqs. (1) and (2) because its existence makes no difference tohe classification (i.e., to the relative values of P(C = WP | X = x) and(C = WO | X = x)). P(C = WP) and P(C = WO) are the prior probabili-ies for a new image to be with or without a contaminant particle,espectively. They are calculated based on the frequency of theumber of WO or WP images used to build the model. Xi denoteshe ith image attribute, and xij is the jth value of attribute Xi inny given image. P(Xi = xij | C = WP) is the probability of an individ-al attribute Xi of attribute vector X taking a value xij which is the

th value of Xi given the image belonging in class WP. There are “k”ttributes in each image (i = 1 to k), and in Torabi’s work k was 6.ypical attributes were mean pixel density, pixel density standardeviation, particle percentage area and its standard deviation, etc.

In Torabi’s research (Torabi et al., 2005), as mentioned above,he attributes are independent from each other and their valuesollow a normal distribution. Therefore, the calculation of posteriorrobability for the attribute values obtained given that the image

s in the WP class is obtained from:

(Xi = xij|C = WP) = f (Xi = xij|C = WP)dx

= 1√2��i

exp

{− (xij − �i)

2

2�2i

}dx (3)

here f(Xi = xij | C = WP) is the probability per dx increment and isermed the probability density of attribute Xi with value xij givenmage belonging in class WP; �i is the mean of ith attribute and �is its standard deviation. As mentioned above, in practice dx is notncluded in the calculation and it is probability density rather thanctual probability that is examined by the classification model. Anxactly analogous equation to Eq. (3) is used for the WO images:

(Xi = xij|C = WO) = f (Xi = xij|C = WO)dx{ 2 }

= 1√

2��i

exp − (xij − �i)

2�i2

dx (4)

Then the class of an image being classified, Y (=WP or WO in thisase), is the class for which the image has the maximum products

hcvpi

ample image without particle (WO).

f probability density multiplied by prior probability, i.e.,

rg maxY

k∏1

f (Xi = xij|C = Y)P(C = Y) → class (5)

. Experimental

This research utilized the same in-line image monitor for plas-ics film extrusion as used by Torabi et al. (2005). A single screwxtruder (L/D = 24/1) (currently equipped with a film die) was used.wo sapphire windows were installed opposite each other justrevious to the die. A white light source was mounted outsidene window. At the other window was a specialized CCD camera,ermed the “scanning particle monitor”. The captured images wereent to a Windows computer for processing and analysis.

The digital images used in this work were the same as thosesed in Torabi’s research (Torabi et al., 2005). That is, the imagesere produced with the material feed of commercial polyethyleneellets with the injection of different kinds and concentration ofarticles.

. Computational

The software developed for this work was written in Java 2 Stan-ard Edition 5.0. Image processing software ImageJ (from National

nstitute of Health at http://rsb.info.nih.gov/) was integrated intohis software. A list of the main software modules and their functions as follows. There are many interactions involved among different

odules. Further details are described in the dissertation by Yan2007).

Simplex optimization: Systematically guesses parameter val-es in IQ operators, applies the operators to the input image using

mageJ, and maximizes image quality.Image processing: Utilizes ImageJ whenever an image needs to

e modified.Image measurement: Obtains image quality metrics as well as

ther particle-related attributes.Image thresholding: Determines the threshold grey level that

istinguishes feature objects (i.e., particles) from their background.Image classification: Applies the Bayesian classification model

o classify an image as with or without particle (WP or WO). Seven

undred and forty-five images were used to develop the model. Thelassification results in this manuscript are based on 10-fold cross-alidation. That is, the training image set was separated into tenartitions (subsets). Each subset was successively selected for test-

ng once while the remainder of the partitions was used for training.

424 S. Yan et al. / Computers and Chemical Engineering 33 (2009) 421–428

Ta

5

5

“Ios

oai

mpn

eowiaeatr

Fig. 3. Qualitative pre-selection of image operators for blurriness removal.

F

poItGqEar

ut

TI

I

BCMSEMGGG

U

S

Nb

Fig. 2. Qualitative pre-selection of image operators for noise removal.

he classification errors for each partition used as a testing set wereccumulated and reported in the respective confusion matrix.

Database: Modified as necessary by optimization results.

. Results and discussion

.1. Selection of image quality operators

Typical sample images judged to be “with particle” (WP) andwithout particle” (WO) by human observers are shown in Fig. 1.mage quality operators that could be used to make the presencef a particle in an image more apparent to the image classificationoftware needed to be selected.

The IQ operators available for grey images are shown in Table 1.From the literature search, it was evident that the vast majority

f IQ operators were directed at changing only five image char-cteristics. These characteristics are: noise, blurriness, contrast,llumination uniformity and brightness.

Initial screening of IQ operators consisted of qualitatively deter-ining which of these characteristics was present in the image. The

ublished literature could then provide guidance as to whether orot a particular IQ operator should be selected.

Dimension reduction was achieved by filtering out some unnec-ssary operators based on the examination of the characteristicsf a given image. Image characteristics were used to determinehat image processing tasks were needed to improve image qual-

ty. Image processing tasks can involve using a single IQ operator or

combination of IQ operators. These tasks include noise removal,dge sharpening, blur removal, contrast enhancement, brightnessdjustment, illumination correction and others. For example, ifhere was a very low noise level in the image, the task of noiseemoval was unnecessary. If noise presence was strong an appro-

aeats

able 1mage quality operators.

dentifier Method Purpose

R Brightness linear shift Change the brightnessON Contrast stretch Adjust the contrastN Mean filter Blur the active image

HP Sharpen Remove motion induced or out-oQL Histogram equalization Enhance image contrastD Median filter Remove noiseB Gaussian Blur Smooth the imageDIL Minimum Grayscale dilationER Maximum Grayscale erosion

NSHP Unsharp mask Sharpen and enhance edges

UB Subtract background Correct non-uniform illuminatio

otes: (1) For image operators with lower level of 0 and high level 1, 0 represents the abserightness linear shift, � is the standard deviation of image grey values.

ig. 4. Qualitative pre-selection of image operators for contrast enhancement.

riate operator for noise removal must be chosen. The choice ofperators for noise removal depended on the noise characteristics.f the noise was impulsive (salt-and-pepper), then a median fil-er was an appropriate choice. If the noise was Gaussian, then aaussian filter or mean filter was suitable. An overall strategy forualitative pre-selection of IQ operators is illustrated in Figs. 2–6.ach of these figures shows a “decision tree” for selection of IQ oper-tors to remedy noise, blur, contrast, illumination and brightness,espectively.

As shown in these figures, qualitative image characteristicnderstanding leads to selection of mutually exclusive IQ opera-ors. IQ operators are mutually exclusive when they are directed

t the same image characteristic. The leaves of the decision tree inach figure represent IQ operators. If more than two leaves sharen immediate parent node then the choice of image quality opera-or is not deterministic, the choice of IQ operator is then randomlyelected.

Parameters Low level High level

Bias shift −� +�Contrast gain 0.5 2Filter radius 1 4

f-focus blur Not applicable 0 1Not applicable 0 1Filter radius 1 4Filter radius 1 4Filter radius 1 4Filter radius 1 4

Filter radius 1 4Mask weight 0.2 0.9

n Filter radius 20 50

nce of the operator while 1 means the presence of the operator. (2) For method of

S. Yan et al. / Computers and Chemical

F

aIdFa(autog

(

(

fp

F

viicfw

te

5

icodfoiiom

aeptC(tp

wdbpdem

opIc

ig. 5. Qualitative pre-selection of image operators for illumination correction.

In general, the degree of visual examination of images utilized inpplying the decision trees of Figs. 2–6 to attempt to select a set ofQ operators that can accomplish the needed image modification,epends upon the image characteristic and the images involved.or example, in this work it was found that a single value serveds a threshold for the brightness metric. The threshold value of 180where black is zero and white is 255) was determined as the aver-ge grey value of correctly classified images based upon the methodsed by Torabi (Torabi et al., 2005) However, for noise, blur, con-rast and illumination, a rapid and effective method for the imagesf this study was to visually compare images and use the followinguidelines:

(i) Noise removal operators should precede sharpness operators.This can be justified due to the fact that noise removal oftencauses blurring and sharpness operators have a very strongtendency to magnify noise.

(ii) The sharpening operator is preferred over the unsharp maskfor applications where small features (particles) are of interestbecause the latter, in general, magnifies noise more than theformer.

iii) Contrast operators including contrast stretching and histogramequalization magnify noise. Thus noise removal operatorsshould precede the contrast operators.

iv) Histogram equalization tends to create artifacts which are diffi-cult to remove by other image operators particularly for imageswhere noises are at the same grey level range as features. Thuscontrast stretching is preferred over histogram equalization inthis situation.

It may be possible in future to define quantitative thresholdsor more of the image characteristics. However, current commonractice in image analysis is to base selection of IQ operators on

ig. 6. Qualitative pre-selection of image operators for brightness adjustment.

lnqlctoqsT“rbnitw

pb

Engineering 33 (2009) 421–428 425

isual examination of images: the human eye is difficult to replacen image evaluation, even when single image characteristics arenvolved. As will be seen below, the method used was very suc-essful at readily reducing the number of IQ operators to a selectew that could be examined using task-specific criteria and thatorked very well in practice.

With the conclusion of qualitative screening as described above,he number of IQ operators is reduced sufficiently to allow closerxamination of the remaining operators using task-specific criteria.

.2. Defining image quality

An important part of the reference image database is thenstruction on how to modify the quality of an image so that thelassification model can more accurately decide whether it is a WOr a WP image. Selecting what operators to modify the image wasescribed in the previous section. Thus, the required instructionor an image consists of what parameter values to insert into theseperators in order to achieve the desired image quality. Definingmage quality was the essential requirement. Once image qualitys defined by an equation that provides a single number, then theptimization program can be used to find the parameter values byaximizing the image quality.Finding an acceptable definition of image quality required a trial

nd error procedure. First an image quality function was hypoth-sized, the Nelder Mead Simplex method was used to find thearameter values in the IQ operators so as to optimize this “objec-ive function” and then the optimized image was fed to the Bayesianlassification model to be classified as WO or WP. The true classesas judged by a human observer) were known for all images so thathe overall accuracy of the model could then be evaluated and com-ared to classification of raw (un-optimized) image classification.

The first three image quality definitions tried were least squares,eighted least squares and a “desirability function”. Error rateecreased significantly compared to the classification of raw imagesut remained at about 2% for all three methods. The fundamentalroblem was that there was no guidance as to how to make theseefinitions more relevant to what the classification model consid-red a “good” image. This led to the idea of using the classificationodel itself as a source of an image quality definition.The Bayesian classification model used in this work was previ-

usly developed by Torabi et al. (2005). The input to this model isercentage area of the particles and mean density of the particles.

n contrast, the image quality functions described above use imageharacteristics more commonly found in the image processingiterature (noise, blur, contrast, illumination uniformity and bright-ess). Thus, one way of improving the link between the imageuality definition and the classification accuracy was to replace theiterature image quality characteristics with those actually used forlassification. However, the uncertainties associated with how besto assign the values of weighting factors and even the exact formf the objective function would then still remain. Instead, an imageuality definition that was based upon using the same type of clas-ification model to obtain a measure of image quality was devised.hat meant dealing with two classification models. One (termed thepre-optimization model”) is used to characterize image quality ofaw images that have previously been assigned to a WO or WP classy a human observer (i.e., it is used only to assign an image qualityumber to a training image). This model is created the same way as

n Torabi’s research (Torabi et al., 2005). The second, and formerly

he only model used, is termed the “post-optimization model” andas used to classify “quality modified” testing images as WO or WP.

As described earlier (Eq. (5)), in conventional data miningractice the Bayesian model utilizes the calculation of proba-ility densities for the attribute values. More specifically, the

4 mical

vffiqbwaI2“aoTi&ulat

wacs(vttbE(Epr

oicWfatifi

ia(

diaStnpt“

tfi

wal

tPctvcPatttdIcdttpoab

pmhi

5o

fifrrin Fig. 7. The overall classification error rate is 0.1%. In Fig. 7, eacherror bar shows the 95% confidence interval for an error rate. Thecalculated confidence intervals are asymmetric and termed “scoreconfidence intervals” in the published literature (Agresti & Coull,1998; Kohavi, 1995).

26 S. Yan et al. / Computers and Che

alue of f(X = x | C = WP)P(C = WP) is compared to the value of(X = x | C = WO)P(C = WO) for an image and the image is classi-ed as WP or WO depending upon whether the former or latteruantity is the greater. Probability densities instead of proba-ilities are used because exactly the same classification resultsould be obtained if the actual probabilities (P(X = x | C = WP)

nd P(X = x | C = WO) were used in place of probability densities.t has been well documented (Domingos & Pazzani, 1997; Rish,001) that, although this approach is implemented using theNaïve” Bayesian equation (Eqs. (1) and (2)), that assumes thatttributes are statistically independent, classification results areften very satisfactory even when this assumption is violated.hat is, the classification tolerates inaccuracies in the probabil-ty densities very well. Furthermore, it was found (Domingos

Pazzani, 1997) that the greater the difference between val-es of f(X = x | C = WP)P(C = WP) and f(X = x | C = WO)P(C = WO) the

ess likely that random error associated with sample size woulddversely affect the accuracy of the classification. Thus, consideringhese aspects, image quality (IQ) was defined as follows:

IQ = f (X = x|C = WP)P(WP) − f (X = x|C = WO)P(WO)if image is WP

IQ = f (X = x|C = WO)P(WO) − f (X = x|C = WP)P(WP)if image is WO

(6)

here f(X = x | C = WP)P(WP) and f(X = x | C = WO)P(WO) are thectual quantities compared by a Bayesian model to decide on thelass and are calculated using the product of probability densities ashown in Eq. (5). The term f(X = x | C = WP) is the probability densityas defined by the normal distribution of Eq. (3)) of the attributeector X of the image with value of x given the image belonging inhe WP class; and term f(X = x | C = WO) is the probability density ofhe attribute vector X of the image with value of x given the imageelonging in the WO class (as defined by the normal distribution ofq. (4)). In accord with normal practice when the Bayesian is usedWitten & Frank, 2005), P(X = x) is omitted from the denominator inq. (6) as it was in using Eqs. (1) and (2). P(WP) and P(WO) are therior probabilities of an image belonging to WP class or WO class,espectively.

When a Bayesian classifier is used to classify an image, the signf probability density difference is used to determine whether anmage belongs to one class or another (Witten & Frank, 2005). Thelasses are WP (with particle) or WO (without particle) in this work.

hen the probability density difference is used as an objectiveunction (Eq. (6)), the objective is to find the values of the vari-ble parameters in the image quality operators that will maximizehe probability density difference. This is the needed improvementn image quality because it is relevant to subsequent correct classi-cation.

Since the above approach is being applied only to trainingmages, it is already known whether the image has been assigneds WP or WO by an independent observer and the appropriate Eq.6) can be selected.

Note that f(X = x | C = WP) and f(X = x | C = WO) are the probabilityensities of the attributes given the class and do not add to unity. IQ

s computed from the pre-optimization Bayesian model using thettribute values of the raw image. Then, as mentioned above, theimplex search is used to maximize IQ by systematically varyinghe adjustable parameters of the IQ modifiers (e.g. brightness shift,oise filters, unsharp mask and background flatten, etc.). Since therior probabilities (P(C = WO) and P(C = WP)) are constants for a par-

icular training data set it is really the probability densities that areoptimized” by the search.

As can be seen from the above description, this approach utilizeshe same calculated quantities as are used in the Bayesian classi-cation and obtains the parameter values in the IQ operators that

Ff

Engineering 33 (2009) 421–428

ill transform the raw image into one that has the largest attain-ble difference in the two critical classification quantities (i.e., theargest IQ value).

An alternate approach would be to deviate from normal prac-ice and to calculate the actual Bayesian posterior probabilities (i.e.,(C = WP | X = x) and P(C = WO | X = x)) This could readily be done byalculating the denominator, P(X = x), in Eqs. (1) and (2). As men-ioned earlier, P(X = x) is the probability of the specific attributealues occurring. It is conventionally calculated as a normalizingonstant (i.e., as the value which would cause P(C = WP | X = x) and(C = WO | X = x) to add to unity as they should because they arell possible, mutually exclusive probabilities). Since these quanti-ies are thus normalized (i.e., add to unity), image quality couldhen be defined as one or the other of them. Maximizing one ofhem would automatically minimize the other. Alternatively, theifference (e.g. P(C = WP | X = x) − P(C = WO | X = x)) could be used as

Q. These alternatives were not tested in this work but in bothases and in the definition of IQ used (Eq. (6)), the maximumifference between the quantities used to effect the classifica-ion would be found by the Simplex search. Since we know thathe relative value of these quantities is unaffected by convertingrobability density to probability, then we would expect the sameptimized image to result from application of the Simplex searchnd the same optimized parameter values for the IQ modifiers toe obtained.

The parameters in both the pre-optimization model and theost-optimization model were determined by first training eachodel with images which have been previously classified by a

uman observer and randomly selected from the sample of suchmages available.

.3. Classification results using probability density difference asbjective function

The use of the probability density difference as an objectiveunction provided the required breakthrough in the definition ofmage quality. Classification results are shown in Fig. 7. We see that,or WO images no images are incorrectly classified (a false negativeate of zero). While for WP images, only 1 (0.2%) out of 505 are incor-ectly classified, which corresponds to a false positive rate of 0% as

ig. 7. Classification error rate for training image set using probability density dif-erence as objective function.

S. Yan et al. / Computers and Chemical Engineering 33 (2009) 421–428 427

Fig. 8. Performance comparison of classification error rate of training image set fordifferent objective functions.

5o

adtbi

5f

fe

rtdidtmce

5

fasaoipstav

6

TA

(

123456789

10. .

I

Fig. 9. Comparison of false negative rate for different objective functions.

.4. Relationship between probability density difference asbjection function and classification results

Unlike the previous IQ definitions examined, use of the prob-bility density difference objective function as an image qualityefinition automatically provided an intrinsic correlation betweenhe image quality definition and the classification performanceecause the classifier uses the difference in probability densities

n order to assign the class to an image.

.5. Comparison of classification results for different objective

unctions

Figs. 8–10 show the direct comparison of classification resultsor all definitions of image quality. Fig. 8 shows the classificationrror rate, Fig. 9 the false negative rate and Fig. 10 the false positive

fiqApc

able 2portion of reference image database.

1) id (2)Brightness

(3)Contrast

(4)Blurriness

(5) Noise (6) IU (7)Mean density

(8Pe

0.62 0.04 0.40 0.53 0.25 155.90 6.0.41 0.04 0.54 0.64 0.34 139.68 8.0.42 0.05 0.65 0.68 0.31 159.61 8.0.41 0.04 0.51 0.66 0.26 113.92 3.0.41 0.04 0.66 0.65 0.29 158.33 0.0.41 0.03 0.63 0.38 0.68 168.29 6.0.42 0.04 0.60 0.64 0.18 143.21 4.0.41 0.03 0.52 0.63 0.29 150.10 0.0.41 0.17 0.06 0.90 0.91 83.87 0.0.41 0.07 0.58 0.70 0.36 148.05 0.

. . . . . . . . . . . . . . . . . . . . .

U, illumination uniformity; B/C, brightness contrast shift; BF, background flattening; SHP

Fig. 10. Comparison of false positive rate for different objective functions.

ate. In every case the use of the optimized images is superior tohe use of raw images for classification and the probability densityifference objective function provided by far the best definition of

mage quality for image quality improvement. The best results wereue to the use of higher quality images for the Bayesian classifica-ion and the image quality definition responsible for those images

ade the critical difference. The algorithm for classification in allases is the same as that developed in a previous paper by Torabit al. (2005).

.6. The construction of reference image database

Two types of relevant image information were obtained andormed the reference image database: quantitative similarityttributes of the raw images and IQ operator instructions. Theimilarity attributes consisted of brightness, contrast, blur, noisend illumination uniformity, percentage area and mean densityf potential particles. IQ operator instructions consisted of thedentity of each operator, the value of the respective operatorarameters and the order of application of the operators. Table 2hows a small portion of the database. Column 1 is an image iden-ifier, columns 2 through 8 contain the similarity attribute valuesnd columns 9 through 12 the IQ operators with their parameteralues and in their order of application to the specific image.

. Conclusions

Obtaining a reference image database to enable improved classi-

cation performance of newly acquired image by modifying imageuality depended strongly on a suitable definition of image quality.

definition relevant to classification performance was accom-lished by using probability density difference values from thelassification model (a Bayesian classifier in this case). Optimizing

)rcent area

(9) IQ operator (10) Parameter (11)IQ operator

(12)Parameter

24E−05 MD Radius = 126.0 UNSHP Weight = 0.321E−05 B/C Brightness = 144.0 BF Radius = 44.005E−05 B/C Brightness = 121.0 BF Radius = 46.094E−05 B/C Brightness = 159.0 BF Radius = 40.0000167 B/C Brightness = 124.0 BF Radius = 46.057E−05 B/C Brightness = 146.0 MD Radius = 4.060E−05 B/C Brightness = 142.0 BF Radius = 44.0000122 B/C Brightness = 133.0 BF Radius = 45.0000256 B/C Brightness = 106.0 SHP Radius = 2.0000297 B/C Brightness = 136.0 BF Radius = 45.0. . . . . . . . . . . . .

, sharpening; MD, mean filter; UNSHP, unsharp mask.

4 mical

imdatiaatbuiocfwq

A

Nw

R

A

A

B

B

B

B

C

C

D

D

E

E

F

F

G

J

J

K

K

K

M

M

P

P

P

P

P

R

S

S

T

T

T

T

V

W

W

W

W

W

W

X

Y

Y

Y

Y

28 S. Yan et al. / Computers and Che

mage quality using this definition and then classifying the opti-ized images gave far superior results to the other image quality

efinitions examined (simple least squares, weighted least squaresnd a desirability function). The final database then consisted ofwo types of information for each reference image: image similar-ty attributes and image quality operator instructions. The formerllow a reference image in the database to be compared to a newlycquired image in order to obtain the most similar reference image;he latter proposes how the quality of the newly acquired image cane modified to improve classification. Our next paper will show these of case-based reasoning with this database for in-line adaptive

mage quality modification (Yan, Sayad, & Balke, 2009). The strategyf using a definition of image quality based upon what the classifi-ation software considers important (probability density differenceor the Bayesian classifier used here) is potentially a very generalay of forming a case-based reasoning database to enable image

uality improvement for more accurate classification.

cknowledgements

We are very pleased to acknowledge funding provided by theatural Sciences and Engineering Research Council of Canada. Also,e thank Dr. Keivan Torabi for very helpful discussions.

eferences

gresti, A., & Coull, B. A. (1998). Approximate is better than “exact” for intervalestimation of binomial proportions. The American Statistician, 52(2), 119–126.

vcibas, I., Sankur, B., & Sayood, K. (2002). Statistical evaluation of image qualitymeasures. Journal of Electronic Imaging, 11(2), 206–223.

aykut, A., Atalay, A., Ercil, A., & Güler, M. (2000). Real-time defect inspection oftextured surfaces. Real-Time Imaging, 6(1), 17–27.

eksac, M. S., Egemen, A., Izzetoglu, K., Ergun, G., & Erkmen, A. M. (1996). An auto-mated intelligent diagnostic system for the interpretation of umbilical arteryDoppler velocimetry. European Journal of Radiology, 23(2), 162–167.

elien, J. A. M., Somi, S., de Jong, J. S., van Diest, P. J., & Baak, J. P. A. (1999). Fullyautomated microvessel counting and hot spot selection by image processing ofwhole tumour sections in invasive breast cancer. Journal of Clinical Pathology,52(3), 184–192.

latt, R. J., Clark, A. N., Courtney, J., Tully, C., & Tucker, A. L. (2004). Automated quan-titative analysis of angiogenesis in the rat aorta model using image-pro plus 4.1.Computer Methods and Programs in Biomedicine, 75(1), 75–79.

hen, T.-J., Chuang, K.-S., Wu, J., Chen, S. C., Hwang, I.-M., & Jan, M.-L. (2003). A novelimage quality index using Moran I statistics. Physics in Medicine and Biology,48(8), N131.

huang, K.-S., & Huang, H. K. (1992). Assessment of noise in a digital image usingthe join-count statistic and the Moran test. Physics in Medicine and Biology, 37(2),357.

arwish, A. M., & Jain, A. K. (1988). A rule based approach for visual pattern inspec-tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(1), 56–68.

omingos, P., & Pazzani, M. (1997). On the optimality of the simple Bayesian classifierunder zero-one loss. Machine Learning, 29(2–3), 103–130.

skicioglu, A. M. (2000). Quality measurement for monochrome compressed imagesin the past 25 years. In Proceedings of the 2000 IEEE international conference onacoustics, speech, and signal processing Istanbul, Turkey,

skicioglu, A. M., & Fisher, P. S. (1995). Image quality measures and their perfor-mance. IEEE Transactions on Communications, 43(12), 2959–2965.

arahani, F., Jeong, K., Sayad, S., Balke, S. T., Calidonio, F., & Rom Roginski, A. (2003).In-line color monitoring during polyethylene extrusion: Reflectance spectraand images. In Proceedings of the 61st annual technical conference ANTEC 2003Nashville, TN, United States.

iete, R. D., & Tantalo, T. (2001). Comparison of SNR image quality metrics for remotesensing systems. Optical Engineering, 40(4), 574–585.

ilmor, C., Balke, S. T., Calidonio, F., & Rom Roginski, A. (2003). In-line color monitor-ing of polymers during extrusion using a charge coupled device spectrometer:Color changeovers and residence time distributions. Polymer Engineering andScience, 43(2), 356–368.

acobson, R. E. (1995). An evaluation of image quality metrics. Journal of PhotographicScience, 43(1), 7–16.

oeris, K., Frerichs, J. G., Konstantinov, K., & Scheper, T. (2002). In-situ microscopy:Online process monitoring of mammalian cell cultures. Cytotechnology, 38(1–2),129–134.

ohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimationand model selection. In Proceedings of international joint conference on artificialintelligence Montreal, Que., Canada.

Y

Z

Engineering 33 (2009) 421–428

rooshoop, D., Torensma, R., van den Bosch, G. J. M., Nelissen, J., Figdor, C. G., Ray-makers, R. A. P., et al. (2003). An automated multi well cell track system tostudy leukocyte migration. Journal of Immunological Methods, 280(1–2), 89–102.

uklin, A., Shams, S., & Shah, S. (2000). High throughput screening of gene expressionsignatures. Genetica, 108(1), 41–46.

ahadevan, S., & Casasent, D. (2003). Automated image processing for grain bound-ary analysis. Ultramicroscopy, 96(2), 153–162.

arziliano, P., Dufaux, F., Winkler, S., & Ebrahimi, T. (2002). A no-reference percep-tual blur metric. In Proceedings of ICIP 2002 international conference on imageprocessing Piscataway, NJ, USA.

eli, E. (1997). In search of a contrast metric: Matching the perceived contrastof Gabor patches at different phases and bandwidths. Vision Research, 37(23),3217–3224.

erner, P. (1999). An architecture for a CBR image segmentation system. Lecture Notesin Computer Science, 1650, 525–534.

erner, P. (2002). Are case-based reasoning and dissimilarity-based classificationtwo sides of the same coin? Engineering Applications of Artificial Intelligence,15(2), 193–203.

ersoon, E. (1988). A pipelined image analysis system using custom integratedcircuits. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(1),110–116.

etropoulos, H., Sibbitt, W. L., & Brooks, W. M. (1999). Automated T-2 quantitationin neuropsychiatric lupus erythematosus: A marker of active disease. Journal ofMagnetic Resonance Imaging, 9(1), 39–43.

ish, I. (2001). An empirical study of the naive Bayes classifier. In Proceedings ofthe IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, Seattle,Washington, USA.

hnayderman, A., Gusev, A., & Eskicioglu Ahmet, M. (2004). A multidimensionalimage quality measure using singular value decomposition. In Proceedings of theImage Quality and System Performance San Jose, California, USA.

iew, L. H., Hodgson, R. M., & Wood, E. J. (1988). Texture measures for carpet wearassessment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(1),92–105.

anaka, M., & Kayama, A. (2001). Automated image processing for fractal analysisof fracture surface profiles in high-temperature materials. Journal of MaterialsScience Letters, 20(10), 907–909.

orabi, K., Ing, L. D., Sayad, S., & Balke, S. T. (2002). Data mining for image analy-sis: In-line particle monitoring in polymer extrusion. In Proceedings of the 3rdinternational conference on data mining. Data mining III Bologna, Italy.

orabi, K., Sayad, S., & Balke, S. T. (2005). On-line adaptive Bayesian classification forin-line particle image monitoring in polymer film manufacturing. Computers &Chemical Engineering, 30(1), 18–27.

orabi, K., Sayad, S., & Balke, S. T. (2006). Adaptive image thresholding for real-timeparticle monitoring. International Journal of Imaging Systems and Technology,16(1), 9–14.

eldkamp, W. J. H., & Karssemeijer, N. (2000). Normalization of local contrast inmammograms. IEEE Transactions on Medical Imaging, 19(7), 731–738.

alters, R. H., Parker, L. R., Morgan, S. L., & Deming, S. N. (1991). Sequential SimplexOptimization. Boca Raton, Florida: CRC Press LLC.

ang, Z., & Bovik, A. C. (2002). A universal image quality index. IEEE Signal ProcessingLetters, 9(3), 81.

ang, Z., Lu, L., & Bovik, A. C. (2004). Video quality assessment based on struc-tural distortion measurement. Signal Processing: Image Communication, 19(2),121–132.

ang, Z., Sheikh, H. R., & Bovik, A. C. (2002). No-reference perceptual quality assess-ment of JPEG compressed images. In Proceedings of the ICIP 2002 internationalconference on image processing Piscataway, NJ, USA.

it, P., & Busscher, H. J. (1998). Application of an artificial neural network in theenumeration of yeasts and bacteria adhering to solid substrata. Journal of Micro-biological Methods, 32(3), 281–290.

itten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools andtechniques. Morgan Kaufmann.

u, W., & Hauske, G. (1994). Picture quality evaluation based on error segmenta-tion. Proceedings of the SPIE the International Society for Optical Engineering, 2308,1454–1465.

an, S. (2007). Adaptive image quality improvement with Bayesian classification forin-line monitoring. Toronto, Ontario, Canada: University of Toronto.

an, S., Sayad, S., & Balke, S. T. (2009). Image quality in image classification: Adaptiveimage quality modification with adaptive classification. Computers and ChemicalEngineering, 33, 429–435.

easin, M., & Chaudhuri, S. (2000). Development of an automated image process-ing system for kinematic analysis of human gait. Real-Time Imaging, 6(1), 55–67.

oda, H., Ohuchi, Y., Taniguchi, Y., & Ejiri, M. (1988). An automatic wafer inspec-tion system using pipelined image processing techniques. IEEE Transactions onPattern Analysis and Machine Intelligence, 10(1), 4–16.

u, H. L., MacGregor, J. F., Haarsma, G., & Bourg, W. (2003). Digital imaging foronline monitoring and control of industrial snack food processes. Industrial &Engineering Chemistry Research, 42(13), 3036–3044.

alewski, K., & Buchholz, R. (1996). Morphological analysis of yeast cells usingan automated image processing system. Journal of Biotechnology, 48(1–2), 43–49.