Color Based Segmentation of Cancer Cell

7/28/2019 Color Based Segmentation of Cancer Cell

1/7

158 ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.5, NO.2 August 2007

Color Based Segmentation of Nuclear StainedBreast Cancer Cell Images

Pornchai Phukpattaranont1 and Pleumjit Boonyaphiphat2, Non-members

ABSTRACT

We present an algorithm for segmenting cells ina microscopic image of immunohistologically stainedslides from breast cancer based on color contents.The procedure for the approach consists of color cat-egorization using neural network, noise removal andshape simplification using mathematical morphology,and cell size consideration. In order to obtain the

more accurate segmentation, we further separate theattached cancer cells with marker-controlled water-shed. Results show that cancer cells in the back-ground with heterogeneous intensities and colors areappropriately segmented using the proposed method.In other words, the visualization of segmented cancercells is in agreement with their perception in the orig-inal red-green-blue (RGB) image. In addition, theresistance of the method to intensity and color varia-tions in the background is obtained without compro-mising of computational speed.

Keywords: Quantitative Immunohistopathology,

Image Segmentation, Cancer Cell Images

1. INTRODUCTION

Immunohistochemistry is a technique used for de-tecting in situ a tissue antigen by a specific antibody.An antigen-antibody reaction is visualized by thecolor development of specific dye and can be seen bylight microscope. The tissue antigen is presented atany part of the cell, i.e., cell membrane, cytoplasm ornucleus. Therefore, it is a useful technique to demon-strate the protein markers including cancer cell. Es-trogen receptor (ER) and progesterone receptor (PR)are prognostic markers for breast cancer detected bythis method. Evaluation of ER and PR positive cellsare useful for hormonal therapy. Brown nuclear stainsare recognized as positive cells and blue ones are neg-ative cells. Traditionally, the percentage of positivecells of those markers are semiquantitatively counted.It is time consuming, costly, subjective and tedious.In order to overcome these problems, an automation

Manuscript received on March 1, 2007 ; revised on June 27,2007.1 The author is with the Department of Electrical Engineer-

ing, Prince of Songkla University, Hatyai, Songkhla, Thailand,Emails: [email protected] The author is with the Department of Pathology, Prince

of Songkla University, Hatyai, Songkhla, Thailand, Emails:[email protected]

of image analysis that previously requires manual op-erations is performed on the basis of the developmentsin computer capabilities and image processing algo-rithms [1-3].

There are a number of benefits that result froman automated analysis. These include an acceler-ation of the process, a reduction in cost for imageanalysis, as well as a decrement in a false inspectiondue to fatigue. Additionally, the automated analy-

sis provides a quantitative description. Based on thisquantitative measurement, the analysis result is ob-jective. Furthermore, the correlation of the quantita-tive categorization with patient symptoms may allowfor an automated diagnostic system [4]. However, it isnot expected that automated image analysis will re-place pathologists experience. It is only an aid to thepathologist for the repeated routine work and yieldsquantitative results that complement and enhance in-terpretations by pathologists. Visual examination bythe pathologist is still required where unusual or ab-normal cells that the method is not trained to deal

with are encountered.Cell image segmentation methods for pathologyare largely relied on two image processing techniques:thresholding and region growing. Thresholding iden-tifies each pixel in the image into either object orbackground based on its intensity. Image threshold-ing is a simple yet often effective means for obtaininga segmentation of images in which cells of interest sit-uate in uniform background intensity across the im-age. Consequently, it is often used as a part in a se-quence of image processing operations. For example,thresholding is one of important steps in segmentinglive cell image [5].

For the image with uneven background, local adap-tive thresholding method is an effective techniquebecause the threshold value can be adapted to thebackground intensity variations. In other words, thethreshold value is derived in each subregion, which isconsidered to be composed of both background andobjects. Examples of medical image analysis thatcontain the local adaptive thresholding algorithm areas follows: (1) Segmentation of fluorescence tumorcells from tissue sections [2], (2) Segmentation of cellnuclei from the stroma and fat like regions [3], (3)Segmentation of tissue components in liver tissue [4],(4) Segmentation of dead and live hepatocytes (livercells) in cultures from microscopic images [6], and (5)Segmentation of nuclei from a breast tissue image [7].One of disadvantages of the thresholding technique is


2/7

Color Based Segmentation of Nuclear Stained Breast Cancer Cell Images 159

that it does not take into account the spatial charac-teristics of the image. This causes it to be sensitiveto noise and intensity heterogeneity. Therefore, fur-ther image processing algorithms that consider spa-tial modeling of the image need to be incorporated

[8].Region growing separates objects of interest from

background in the image based on some predeter-mined criteria, i.e. intensity and/or edges [9]. Whileedge-based methods are sensitive to noise and arti-facts, the intensity-based algorithms are usually morecomputationally expensive. Additionally, one of maindisadvantage of region growing is the requirement ofmanual interaction to obtain the seed point [10]. Sim-ilar to thresholding, region growing is partially usedin a set of image processing operations [11,12]. An ex-ample of region growing used in medical image anal-ysis includes the extraction of noisy cell contour asappear in [13]. In addition to thresholding and regiongrowing, other examples of segmenting techniques aresegmentation of white blood cells based on morpho-logical granulometries [14] and the Principle of LeastCommitment [15].

We have recently presented the segmentation al-gorithm for breast cancer cell image based on localadaptive thresholding and mathematical morphology[16]. The method is appropriate for microscopic im-ages with low histological noise, i.e., low variations onbackground color and intensity. However, the degreeof histological noise in breast cancer images varies.

For the image with high histological noise, the localadaptive thresholding approach is sensitive to noiseand intensity heterogeneity. In addition, the com-putational time for the approach is quite lengthy.In order to address these problems, we propose anew strategy for segmenting cancer cells in a micro-scopic image of immunohistological nuclear stainingof breast cancer tissue based on the color of pixel.This is motivated from the way that a pathologistdetermines the positive and negative of tumor cellsby using their color contents manually.

The remainder of this paper is organized as follows:Section 2 describes the acquisition of microscopic im-

ages and features of cancer cells from breast cancertissue; Section 3 provides our proposed segmentationalgorithm; Section 4 gives results and discussion; Sec-tion 5 draws conclusions of the paper.

2. MATERIALS AND METHODS

2.1 Acquisition of the Images

The image used in this paper were breast cancertissue stained with the antibody to ER in the nuclei.The positive cells were marked by brown color andthe negative ones were blue color. In addition, theinflammatory cells such as lymphocytes, plasma cellsand some connective tissue cells were labeled withblue color in the nuclei. However, these non-tumorcells were identified by their contours and pattern

histologic arrangements. The tissue sections were ob-served under a microscope with a magnifying factorof 40x. The contrast and intensity of staining weremanually corrected in such a way that the digitizedimage became visually acceptable for further manual

classification by a specialist. The cell images wereacquired in color using the Eclipse 80i advanced re-search microscope (Nikon Instech Co., Ltd., Japan).The digital image was saved as a color 25603200JPEG files for processing.

2.2 Features of Cancer Cells

Fig.1: An example of stained cancer cell image. Thebrown and blue nuclei of the cells in the right andleft circles are representative samples of positive andnegative staining of estrogen receptor of breast cancercells, respectively.

Fig. 1 shows an example of stained cancer cell im-age from microscope with a magnification of 40x.This staining procedure is utilized to demonstratethe existing of estrogen or progesterone receptors inthe breast cancer cells. In other words, stained can-cer cells are classified into two categories accordingto their nuclear color contents, i.e. brown (P) andblue (N). The brown color indicates a positive stain-ing while the blue one demonstrates a negative result.

The brown and blue cells in the right and left circlesshown in Fig. 1 are representative samples of positiveand negative staining of estrogen receptor of cancercells, respectively. The ratio of the total number ofpositive cancer cells to the total number of cancercells in the whole image is used by a doctor for med-ical planning and treatment. It is also noticeable inFig. 1 that there are color and intensity variationsboth in the cancer cells and in the background (non-cancer cells).

The ultimate goal of our research project is todevelop an automatic algorithm for counting posi-tive and negative cancer cells on immunohistologi-cal stained slides from breast cancer tissue. In or-der to achieve this objective, the method to segmentboth negative and positive cancer cells from the back-


3/7


ground is investigated. In this paper, we proposedthe use of pixel color partitioning based on a neu-ral network classifier and morphological operators forsegmenting cancer cells microscopically.

3. SEGMENTATION ALGORITHMThe originally acquired image is in the red-green-

blue (RGB) color space. That is, the color image isformed by the combination of red, green, and bluemonochrome images. In the first step, we classifycolor pixels in the image into one of three categories,i.e. background, P, or N, based on their RGB com-ponents. There are many classifiers that can be usedfor partitioning color of a pixel. However, a neuralnetwork is chosen in this paper due to its well knownas a successful classifier for many applications [17-19].Subsequently, morphology operations are used for ad-

dressing the spatial characteristics of cells. Finally,in order to obtain accurate cell counting results, themarker-controlled watershed is applied for separatingattached multiple cells into distinct single cells. Analgorithm for segmenting cancer cells based on theircolors and sizes is as follows.

An algorithm for cancer cell segmentation

Read image fileClassify pixels using neural networkDo threshodingDo morphology openingFill holesLabel objects in the image

Classify each object using its sizeIF size < 1500 THEN

Ignore objectELSE IF 1500 size 7500 THEN

Object is a distinct single cellApply morphology closing (1)

ELSE size > 7500 THENObject is attached multiple cellsApply morphology openingApply marker-controlled watershed (2)

ENDIFCombine the results from (1) and (2)

Details of the algorithm are given below.

3.1 Neural Network

We use backpropagation neural network to clas-sify pixels in the microscopic image according to theircolor contents. Backpropagation is created by gen-eralizing the Widrow-Hoff learning rule to multiple-layer networks and nonlinear differentiable transferfunctions. Input vectors and the corresponding tar-get vectors are used to train the network until itcan classify the defined pattern. The training al-gorithms use the gradient of the performance func-tion to determine how to adjust the weights to min-imize performance. The gradient is determined bya technique called backpropagation, which involves

Fig.2: Diagram of backpropagation neural network.

performing computations backwards through the net-work. The backpropagation computation is derivedusing the chain rule of calculus [20].

Based on our experiences, the number of neuralnetwork layer between two and three is appropriatefor classifying color of pixels in cancer cell images.Therefore, a backpropagation neural network of threelayers shown in Fig. 2 is chosen to classify image pix-els whether they are from background, P, or N regionsin this paper. The input vector is composed of 3 ele-ments corresponding to the RGB color vector of pixel.Two hidden layers are determined empirically to be3 and 4 neurons and the output layer consists of oneneuron. In addition, the transfer functions of hiddenand output layers are tan-sigmoid and linear, respec-

tively. For the training of neural network, the targetis assigned to be -1, 0, and 1, which are correspond-ing to RGB components from background, P, and Nregions, respectively. The networks are trained us-ing the Levenberg-Marquardt (LM) algorithm. Thetraining stops when the maximum number of epochsreaches 100 or the mean square error is less than1 1012. The number of samples used for trainingneural network from each reference region is 1600.

3.2 Morphology Operations

Mathematical morphology is a nonlinear operator

based on set theory operating on object shape. It isa powerful tool to numerous image processing prob-lems, for example, image preprocessing, segmentationusing object shape, and quantitative description ob-jects [?, ]. We utilize mathematical morphology as atool for noise filtering and shape simplification in ourwork. Note that the disk-shaped structuring element(SE) with a radius ofR is used for all morphologicaloperations in this paper. As a result, the size of SEmatrix is 2R + 1 by 2R + 1. In addition, all math-ematical morphology operations are applied once foreach stage.

After finishing color partition for all pixels, theoutput image from neural network is transformed to ablack and white image by thresholding. That is, whilethe pixels in background region are transformed to


4/7


be zero, the pixels in P and N regions, i.e., objects ofinterest, are transformed to be one. In order to elim-inate spike noise, the binary image is processed usingmorphological opening. The disk-shaped SE with aradius of 1 is used in this stage. The algorithm based

on morphological reconstruction is subsequently usedto fill holes in the image before performing size con-sideration.

In the next step, we classify each object in theimage into one of three categories according to its size:small, medium, and large. The value of sizes used forcell classification is predetermined from guidance bya specialist. The small object is regarded as noiseand is ignored. The medium object is considered tobe a distinct single cell. For the large object, it isdetermined as attached multiple cells. All distinctsingle cells are processed further with mathematicalclosing to complete their shape. It is used as spatialcompensation for an uneven distribution of color inthe cell. The SE used in this stage has a radius of9. On the other hand, all attached multiple cells areapplied with morphological opening. The radius ofthe SE in this stage is 12. There are two explanationsfor doing this performing. First, the multiple cellswith small degree of attachment can be kept apart.Second, it can be used as a preparation step beforemarker-controlled watershed processing.

3. 3 Marker-Controlled Watershed

In order to separate attached cancer cells into in-dividual objects, we further process the result fromlast step with marker-controlled watershed. The wa-tershed algorithm is shown to be a powerful tool fordividing attached objects [22]. The marker computa-tion is used as an additional processing because thedirect use of watershed transform usually yields theover-segmented result [2]. The computational proce-dures for marker-controlled watershed are as follows.

Step 1. Use the Sobel edge marks to compute thegradient magnitude of all attached cells,

Step 2. Determine the marker, which is connectedblobs of pixels inside each cell, based on the distant

transform.Step 3. Combine the results of Step 1. and Step 2,Step 4. Compute the watershed transform of the

result from Step 3.

Finally, to obtain whole cancer cells, we combinethe image of attached multiple cells with the imageof distinct single cells using a logical operator OR.

4. RESULTS AND DISCUSSION

4.1 Neural Network

Fig. 3 shows an original RGB image of cancer cells.Most of cancer cells are located separately, but someof them attach with their neighborhood. We alsosee an uneven distribution of color and intensity inthe background region surrounding cancer cells. It is

Fig.3: Original RGB image of stained cancer cells.Boxes B1, B2, and B3 indicate reference regionsfrom background, P, and N, respectively. Pixel val-ues in the box regions are used for training neuralnetwork.)

considered as histological noise. The R-, G- and B-components of pixel values from background, P, andN reference regions (Boxes B1, B2, and B3) shownin Fig. 3 are used as the input vectors for trainingneural network. After the backpropagation neuralnetwork is trained, network responses of pixel valuesthroughout the image are calculated.

Fig.4: Output image from neural network. Pix-els from background, P, and N regions are shown inwhite, gray, and black colors, respectively.

Fig. 4 shows the output image from neural net-work. Pixels from background, P, and N regions areshown in white, gray, and black colors, respectively.Results demonstrate that the neural network can clas-sify color pixels very well. That is, pixel values areappropriately categorized into a connected region cor-responding to the cancer cells shown in Fig. 3. Ad-ditionally, the results from neural network show thatcolor contents of pixel values for each cancer cell areunevenly distributed. For example, we can noticethe appearance of an area with background color in-side the biggest brown cancer cell situating at the lefthand side of box B1. This is in agreement with theoriginal RGB image.


5/7


Fig.5: (a) Binary image after thresholding and mor-phological operations of the image from neural net-work. (b) Binary image of distinct single cells beforemorphological closing. (c) Binary image of distinct

single cells after morphological closing. (d) Binaryimage of attached multiple cells before morphologi-cal opening. (e) Binary image of attached multiplecells after morphological opening. (f) Binary imageof attached multiple cells after marker-controlled wa-tershed.

4.2 Morphology Operations

Fig. 5 (a) shows the binary image after thresh-olding and morphological opening for spike noise re-moval. Subsequently, hole filling is performed using

the algorithm based on morphological reconstruction.Each object in the image is classified into three cat-egories according to its size. Fig. 5 (b) shows thebinary image of distinct single cells after size consid-eration. One can notice that some cells do not havethe complete shape due to an uneven distribution ofcolor and intensity. To compensate for this shortcom-ing, we perform mathematical closing and show resultin Fig. 5 (c). It is shown that distinct single cells withperfect round shape are obtained after mathematicalclosing.

Fig. 5 (d) shows the binary image of attached mul-tiple cells after size consideration. The size of cancercells in this image is large in terms of area comparedwith the size of distinct single cell. Fig. 5 (e) showsthe binary image of attached multiple cells after mor-

phological opening. One can see that a slightly at-tached multiple cell is separated into two distinct sin-gle cells as we expect. In addition, the shape of eachcell is smoothed and simplified, which make it appro-priate for marker-controlled watershed processing.

4. 3 Marker-Controlled Watershed

Fig. 5 (f) shows the segmented image after the ap-plication of marker-controlled watershed. A more ac-curate segmented result is achieved. That is, two at-tached multiple cells are appropriately separated intofour distinct single cells.

Fig.6: Segmented image after the application of alogical operator OR.

Fig. 6 shows the segmented image after the appli-cation of a logical operator OR. Note that we discardcells at the boundary of image because of their partialappearance. We superimpose this segmented imageon the original RGB image and demonstrate the re-sult in Fig. 7. The number of segmented cancer cellsis 24. It can be clearly seen that the perception of seg-mented cancer cells is in agreement with their originalvisualization.

Fig.7: Segmented image superimposed on the origi-nal RGB image.


6/7


5. CONCLUSIONS

This paper presents the segmentation of cancercells in a microscopic tissue image from breast can-cer based on color contents. The segmenting resultsof cancer cells from the background are used as a

preliminary step before extracting cell features andclassifying cell types. The excellent segmentation re-sults from the proposed algorithm are demonstratedwith microscopic images under histological noise con-ditions. However, to make the method automatic,the algorithm that can compensate for the selectionof training data by a specialist need to be incorpo-rated. Comparison of automatic cell image classifica-tion based on the proposed segmentation algorithmto a pathologist is ongoing research. Result will bereported in the near future.

6. ACKNOWLEDGEMENT

This research is supported by Department of Elec-trical Engineering, Faculty of Engineering and De-partment of Pathology, Faculty of Medicine, Princeof Songkla University.

References

[1] J. P. Thiran and B. Macq, Morphological fea-ture extraction for the classication of digital im-ages of cancerous tissues, IEEE Trans. Biomed.Eng., vol. 43, no. 10, pp. 1011-1020, Oct. 1996.

[2] B. Fang, W. Hsu, and M. L. Lee, On the ac-curate counting of tumor cells, IEEE Trans.Nanobiosci., vol. 2, no. 2, pp. 94-103, Jun. 2003.

[3] S. Petushi, C. Katsinis, C. Coward, F. Garcia,and A. Tozeren, Automated identification ofmicrostructures on histology slides, in IEEE In-ternational Symposium on Biomedical Imaging:Macro to Nano, vol. 1, pp. 424-427., 2004.

[4] L. OGorman, A. C. Sanderson, and K. PrestonJr., A system for automated liver tissue im-age analysis: Methods and results, IEEE Trans.Biomed. Eng., vol. 32, no. 9, pp. 696-706, Sep.1985.

[5] K. Wu, D. Gauthier, and M. D. Levine, Livecell image segmentation, IEEE Trans. Biomed.Eng., vol. 42, no. 1, pp. 1-12, Jan. 1995.

[6] H. Refai, L. Li, and T. K. Teague, Automaticcount of hepatocytes in microscopic images,inInternational Conference on Image Processing,vol. 2, pp. 1101-1104., 2003.

[7] X. Zhao and S. H. Ong, Adaptive local thresh-olding with fuzzy-validity-guided spatial parti-tioning, in International Conference on PatternRecognition, vol. 2, pp. 988-990., 1998

[8] H. D. Li, M. Kallergi, L. P. Clarke, V. K. Jain,and R. A. Clark, Markov random field for tu-mor detection in digital mammography, IEEETrans. Med. Imag., vol. 14, no. 3, pp. 565-576,Sep. 1995.

[9] R. M. Haralik and L. G. Shapiro, Im-age segmentation techniques, Comput. Vis.Graph. Image Proc., vol. 29, pp. 100-132, 1985.

[10] R. Adams and L. Bischof, Seeded region grow-ing, IEEE Trans. Pattern Anal. Mach. Intell.,

vol. 16, pp. 641-647, Jun. 1994.[11] J. R. Beveridge and et al., Segmenting images

using localized histograms and region merging,International Journal of Computer Vision, vol.2, pp. 311-347, 1989.

[12] M. A. Wani and B. G. Batchelor, Edged-region- based segmentation of range images,IEEE Trans. Pattern Anal. Mach. Intell., vol.16, pp. 314-319-647, Mar. 1994.

[13] H. S. Wu and J. Barba, An algorithm for noisycell contour extraction via area merging, J.Imag. Sci. Technol., vol. 38, pp. 604-607, Nov.

1994.[14] N. Theera-Umpon and P. D. Gader, Count-

ing white blood cells using morphological gran-ulometries, Journal of Electronic Imaging, vol.9, no. 2, pp. 170-177, Apr. 2000.

[15] J. Park and J. Keller, Fuzzy patch label re-laxation in bone marrow cell segmentation, inProc. IEEE Int. Conf. Syst. Man. Cybern., vol.2, pp. 1133-1138., 1997.

[16] P. Phukpattaranont and P. Boonyaphiphat,Automatic classification of cancer cells inmicro- scopic images: Preliminary results, in

The 2006 ITC-CSCC International Conference,vol. 1, pp. 113-116., 2006.

[17] E. Gelenbe, Y. Feng, and K. R. R. Krishnan,Neural network methods for volumetric mag-netic resonance imaging of the human brain,Proc. IEEE, vol. 84, no. 10, pp. 1488-1496, Oct.1996.

[18] W. E. Reddick, G. O. Glass, E. N. Cook, T. D.Elkin, and R. J. Deaton, Automated segmenta-tion and classication of multispectral magneticresonance images of brain using articial neuralnetwork, IEEE Trans. Med. Imag., vol. 16, pp.911-918, 1997.

[19] P. D. Heermann and N. Khazenie, Classifica-tion of multispectral remote sensing data using aback-propagation neural network, IEEE Trans.Geosci. Remote Sensing, vol. 30, no. 1, pp. 81-88,Jan. 1992.

[20] M. T. Hagan, H. B. Demuth, and M. H.Beale,Neural Network Design, PWS PublishingCom- pany, Boston, MA, 1996.

[21] R. C. Gonzalez and R. E. Woods, Digital ImageProcessing, Prentice-Hall, Inc., 2002.

[22] L. Vincent, Morphological grayscalereconstruc- tion in image analysis: applica-tions and ecient algorithms, IEEE Trans.Image Process., vol. 2, no. 2, pp. 176-201, Apr.1993.


7/7


Pornchai Phukpattaranont was bornin Songkla, Thailand. He received the B.Eng. and M. Eng. degrees in electricalengineering from Prince of Songkla Uni-versity in 1993 and 1997, respectively,the Ph.D. degree in electrical engineer-ing from the University of Minnesota, in

2004. He is currently an assistant pro-fessor of electrical engineering at Princeof Songkla University. His research in-terests are ultrasound contrast imaging,

ultrasound signal processing, medical image processing, andbiomedical signal processing. Dr.Phukpattaranont is a mem-ber of the IEEE.

Pleumjit Boonyaphiphat was born inThailand. She received the B.Sc. de-gree from Faculty of Nursing, Prince ofSongkla University, in 1980, the M.Sc.degree from Faculty of Science, MahidolUniversity, in 1986. She has been work-

ing at Department of Pathology, Facultyof Medicine, Prince of Songkla Univer-sity since 1986. Her position is an assis-tant professor in anatomical pathology.Her research field is head and neck can-

cer, breast cancer and image analyzer for histopathology.

Color Based Segmentation of Cancer Cell

Documents

Transcript of Color Based Segmentation of Cancer Cell