Color-based Diagnosis: Clinical Images Research Project Funded In Part by NIH Yue (Iris) Cheng, Dr....

Color-based Diagnosis: Clinical Images

Research Project Funded In Part by NIH

Yue (Iris) Cheng, Dr. Scott E Umbaugh@

Computer Vision and Image Processing Research Lab

Electrical and Computer Engineering Department

Southern Illinois University EdwardsvilleE-mail: [email protected]

https://www.ee.siue.edu/CVIPtools

Yue (Iris) Cheng, Dr. Scott E Umbaugh@

Computer Vision and Image Processing Research Lab

Electrical and Computer Engineering Department



07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE

Overview Skin tumors can be either malignant or benign

Clinically difficult to differentiate the early stage of malignant melanoma and benign tumors due to the similarity in appearance

Proper identification and classification of malignant melanoma is considered as the top priority because of cost function

Classification of skin tumors using computer imaging and pattern recognition

Previous texture feature algorithm successfully differentiate the deadly melanoma and benign tumor seborrhea kurtosis

Relative color feature algorithm is explored in this research for differentiate melanoma and benign tumors, dysplastic nevi and nevus

Successfully classify 86% of malignant melanoma using relative color features, compared to the clinical accuracy by dermatologists in detection of melanoma of approximately 75%


Materials and Tools Image database

Original tumor images 512x512 24-bit color images digitized from 35mm color

photographic slides and photographs 160 melanoma, 42 dysplastic, and 80 nevus skin tumor

images Border images

Binary images drawn manually and reviewed by the dermatologist for accuracy

Software CVIPtools

Computer vision and image processing tools developed at our research lab

Partek Statistical analysis tools


CVIPtools


Method Design

Creation of relative color images

Segmentation and morphological filtering

Relative color feature extraction

Design of tumor feature space and object feature space

Establishing statistical models from relative color features


Create Relative Color Skin Tumor Images

Purpose to equalize any variations caused by

lighting, photography/printing or digitization process

to equalize variations in normal skin color between individuals

the human visual system works on a relative color system

Algorithm Mask out non-skin part in the image to

calculate the normal skin color Separate tumor from the image Remove the skin color from the tumor to

get a relative color skin tumor image CVIPtools functions were used to create

relative color skin tumor images


Calculate Skin Color

Original Noisy Skin

Tumor Image

Non-skin Algorithm

Calculate

Mask out tumor

Skin Tumor Image W/O

Noise

Average R, G, B Value

of Skin

Skin-OnlyImage


Tumor Image

AND

Original Noisy Skin

Tumor Image

Border Image

Tumor Image


Relative Color Tumor Image

SUBTRACT

TumorImage

Average R, G, B Value

of Skin

Relative Color Image of the Tumor


Segmentation and Morphological Filtering

Image segmentation was used to find regions that represent objects or meaningful parts of objects

Morphological filtering was used to reduce the number of objects in the segmented image

Easy to use CVIPtools for experimenting and analysis


Relative Color Feature Extraction Necessary to simplify the raw image data into higher

level, meaningful information Feature vectors are a standard technique for classifying

objects, where each object is defined by a set of attributes in a feature space.

Totally 17 color features and binary features were extracted using CVIPtools

The three largest objects, based on the binary feature ‘area’, were used in feature extraction

Histogram features, that is, color features, were extracted in each color band from relative color image objects


17 Features Binary features

Area

Thinness

Histogram features in R, G, B bands

Mean

Standard deviation

Skewness

Energy

Entropy

r c

crIArea ),(

24

Perimeter

AreaThinness

r c M

crIMean

),(

1

0

2 )()(L

gg gPgg

)()(1 1

0

33

gPggSkewnessL

gg

1

0

2)(L

g

gPEnergy

1

02 )(log)(

L

g

gPgPEntropy


17 Features (Cont.)


Design Two Feature Spaces

Tumor feature space consists of 277 feature vectors correspond to 277

skin tumor images. each feature vector has 51 feature elements, which

are the total of 17 features of each three largest objects within the same tumor.

Object feature space had 842 feature vectors corresponding to 842 image

objects each feature vector has 17 feature elements, which

were the binary features and color features stated as above


Establishing Statistical Models Two feature spaces serve as two data models in order to

maximize the possibility of success

Two classification models, Discriminant Analysis and Multi-layer Perceptron, were developed for both data models

The training and test paradigm is used in statistical analysis to report unbiased results of a particular algorithm

due to small size of data set, 282 images, we used the leave x out method, with both one and ten for x

Partek software was used to analyze the data representing the features to develop a model or rules for classifying the tumors


Quadratic Discriminant Analysis A statistical pattern recognition technique based on

Bayesian theory, which classifies data based on the distribution of measurement data into predefined classes

Normalization the feature data as preprocessing performed to maximize the potential of the features to

separate classes and satisfy the requirement of the modeling tool such as Quadratic discriminant analysis for a Bayesian distribution of the input data

Variable selection was used to choose dominant features.


Multi-Layer Perceptron A feed forward neural network

neural networks modeled after the nervous system in biological systems, based on the processing element the neuron

widely used for pattern classification, since they learn how to transform a given data into a desired output.

Principal Component Analysis (PCA) as preprocessing a popular multivariate technique, is to reduce

dimensionality by extracting the smallest number components that account for most of the variation in the original multivariate data and to summarize the data with little loss of information

the dispersion matrix selected for PCA in this project is correlation


Multi-Layer Perceptron (Cont.) Creation, training and testing of neural networks

Creation a neural network involves selection of hidden and output neuron types and a random number generation.

Four output neuron types – Softmax, Gaussian, Linear and sigmoid Three hidden neuron types – Sigmoid, Gaussian and Linear

Scaled Conjugate Gradient algorithm is used for learning in this project.

Automated and independent of user parameters Avoids time consuming Stopping criteria, sum-squared error, is selected to determine after

how many iterations the training should be stopped The trained data is then tested on itself first to examine how

far the neural network is able to classify the objects correctly. Leave x partition out method is used for testing the algorithm


Experiments and Analysis in Tumor Feature Space

Discriminant Analysis 24 features selected for leave ten out method

HistogramFeatures

Mean STD Skewness Energy Entropy

R G B R G B R G B R G B R G B

Object 1 X X X X X X X X

Object 2 X X X X X X X

Object 3 X X X X X X X X 10 features selected for leave one out method

HistogramFeatures



Object1 X X X

Object 2 X X X X

Object 3 X X X


Experiments and Analysis in Tumor Feature Space (Cont.)

0

10

20

30

40

50

60

70

80

90

100

DA on data with 24features usingleave 10 out

DA on data with 10features usingleave 10 out

DA on data with 24features using

leave 1 out

DA on data with 10features using

leave 1 out

Su

cces

s P

erce

nta

ge

Dys %

Mel %

Nev %

Discriminant Analysis (Cont.)


Experiments and Analysis in Tumor Feature Space (Cont.)

Multi-layer Perceptron

0

10

20

30

40

50

60

70

80

90

1000 iterationsoutput layer

softmax,hidden layer

sigmod

700 iterationsouter layersoftmax,

hidden layersigmod,

700 iterationsouter layer

softmax, 17hidden layers

sigmod,


sigmod,hidden layer

sigmod

800 iterations ,output_layer


gauss

Dys correct%

Mel correct %

Nev correct %

Best features, being in the first three components of the PCA projection data, were used

Success percentages of melanoma as high as 77% and nevus is as high as 68%


Experiments and Analysis in Object Feature Space Discriminant Analysis

8, 9, 11 and 12 significant features were selected respectively for leave one out method

Number of HistogramFeatures

Area



8 X X X X X X X X

9 X X X X X X X X X

11 X X X X X X X X X X X

12 X X X X X X X X X X X X


Experiments and Analysis in Object Feature Space (Cont.)

Discriminant Analysis (Cont.)

0

10

20

30

40

50

60

70

80

90

12 Features 11 Features 9 Features 8 Features

Number of Features

Su

ce

ss

Pe

ce

nta

ge

Dys %

Mel %

Nev %

Yield consistent results in classifying melanoma from other skin tumor with above 80% success rate


Experiments and Analysis inObject Feature Space (Cont.)

Multi-layer Perceptron (MLP)

0

10

20

30

40

50

60

70

80

90

100


sigmoidhidden layer

sigmod

425 iterationsouter layergaussian

hidden layergaussian

255 iterationsouter layer

linear, hiddenlayers

gaussian


softmaxhidden layer

gaussian

130 iterations ,output_layer


sigmoid

Dys correct%

Mel correct %

Nev correct %

5 out of 12 hidden-output layer neuron combinations gave better classification results

Leave one out method Yield success

percentage as high as 86% for classifying melanoma.

MLP is more consistent in classifying melanoma as well as nevus


Conclusion Multi-Layer perceptron (MLP) with feature data

preprocessed by Principal Component Analysis (PCA) gave better classification results for melonoma than Discriminant Analysis (DA)

The best overall successful rate of 78%, of which percentage correct of melanoma is 86%, nevus is 62% and dysplastic is 56%.

The best classification results are achieved with sigmoid used as the hidden and output layer neuron

type for the MLP with PCA on Object Feature Space. The three largest tumor objects are representative

for the whole skin tumor.


Conclusion (Cont.) However the small percentage of melanoma

misclassification as well as the relatively low success rate for nevus and dysplastic nevi suggests that we may not have the complete data set for the experiments.

In order to achieve better classification results, future experiments

Needs more complete skin tumor image database. Should combine texture and color methods to get better

results Will include dermoscopy images


Acknowledgement

Dr. Scott E Umbaugh, SIUE Mr. Ragavendar Swamisai Ms. Subhashini K. Srinivasan Ms. Saritha Teegala Dr. William V. Stoecker,

Dermatologist, UMR


Thank You!

Yue (Iris) ChengGraduate Student

@Computer Vision and Image Processing Research

LabElectrical and Computer Engineering Department



Yue (Iris) ChengGraduate Student

@Computer Vision and Image Processing Research

LabElectrical and Computer Engineering Department



Color-based Diagnosis: Clinical Images Research Project Funded In Part by NIH Yue (Iris) Cheng, Dr....

Documents

Transcript of Color-based Diagnosis: Clinical Images Research Project Funded In Part by NIH Yue (Iris) Cheng, Dr....