CHAPTER 4 MODIFIED ARTIFICIAL NEURAL NETWORKS FOR...
Transcript of CHAPTER 4 MODIFIED ARTIFICIAL NEURAL NETWORKS FOR...
43
CHAPTER 4
MODIFIED ARTIFICIAL NEURAL NETWORKS FOR IMAGE
CLASSIFICATION
4.1 INTRODUCTION
Image classification is one of the pattern recognition applications in which the
images are classified into different categories based on some similarity measures.
Several computational techniques are available in the literature for classification
but Artificial Neural Network (ANN) has outperformed other automated
techniques because of the various advantages such as high accuracy and quick
convergence. But, an interesting fact is that these advantages are not
simultaneously available in the same network. The networks such as Counter
Propagation Networks (CPN) are accurate but less efficient in terms of
computational complexity. On the other hand networks such as Self-Organizing
Map (SOM) are less accurate but quickly converge to the results. These
drawbacks are eliminated in this research work by proposing few modified neural
networks which has the ability of yielding accurate results within quick time. The
ability of these networks is tested in the context of abnormal MR brain image
classification. Experimental results have verified the efficiency of these proposed
networks in terms of the performance measures.
4.2 PROPOSED METHODOLOGY OF IMAGE CLASSIFICATION
SYSTEM
The framework of the proposed automated system for image classification is
shown in Figure 4.1. Real-time MR brain images from four categories such as
Meningioma, Astrocytoma, Metastasis and Glioma are used in this work. The
details of pre-processing technique and the feature extraction techniques have
been already discussed in Chapter 3.
44
Figure 4.1 Framework of the image classification system
In this work, five different types of ANN are used among which two are
conventional networks and three are the proposed modified networks. The focus
is on the modified networks but the conventional networks are also implemented
to aid the comparative analysis. These networks are implemented for abnormal
MR brain image classification and the classifiers are analyzed in terms of
classification accuracy and convergence rate. The rest of the chapter is organized
as follows: Section 4.3 deals with the conventional neural networks, Section 4.4
covers the modified neural networks, Section 4.5 illustrates the experimental
results and Section 4.6 concludes the chapter by providing the highlights of this
work.
4.3 CONVENTIONAL ARTIFICIAL NEURAL NETWORKS
In this research work, the applicability of the conventional neural networks such
as CPN and SOM for MR brain image classification is explored.
Performance analysis
Image Pre-processing
(Skull removal)
Feature Extraction
(Image based)
MR brain images
CPN
classifier
SOM
classifier
MCPN
classifier
MSOM1
classifier
MSOM2
classifier
45
4.3.1 Counter Propagation Neural Network
One of the prime hybrid neural networks is the CPN in which both the
supervised and unsupervised training methodology are used for weight
estimation. It consists of the input Kohonen layer which uses the “winner take-
all” strategy and the output Grossberg layer which uses the error signal for weight
adjustment. The error signal is used to update only the output layer weights unlike
BPN where error is used to update weights of both the layer. Thus, this network is
named as Counter Propagation Neural Network to show that it is contrary to the
conventional BPN.
4.3.1.1 Architecture of CPN
The CPN network consists of three layers: the input layer, the competition layer
and the output layer. The configuration of the network is as follows: number of
neurons in the input layer is ‘n’, number of neurons in the competition layer is
‘N’, number of neurons in the output layer is ‘p’. Let U be the weight matrix
between the input layer and the competition layer and V be the weight vector
between the competition layer and the output layer. In this work, the number of
input layer neurons is 8 which are equal to the number of input features. The
number of output layer neurons is 4 which are equal to the number of output
classes. The number of hidden layer neurons is fixed to be 8 to maintain
uniformity between the CPN and Modified CPN (MCPN) which will be discussed
in section 4.4.1.1. The topology of the conventional CPN is shown in Figure 4.2.
46
Figure 4.2 Topology of CPN
4.3.1.2 Training algorithm of CPN
The training algorithm of the CPN is explained with the following procedural
steps.
Step 1 : The training instance X is presented to the input layer. Let ijU be the
arbitrary initial weights vector assigned to the links connecting input neuron ‘i’
with the competition neuron ‘j’ and jkV be the arbitrary weights vector assigned to
the links connecting competition neuron ‘j’ and output neuron ‘k’. The Euclidean
distance jD between the weight vector U and the input vector X is calculated
using the following formula
jXUXUD iijij .......21212
=1, 2... N (4.1)
Step 2 : Each neuron in the competition layer is allowed to compete with the
other neurons and the node with the shortest Euclidean distance wins. The output
of the winning node is set to 1 and the rest to 0. Thus, the output of the jth
node in
the competition layer is
0.1jZ if jD is minimum
.........0.0jZ Otherwise (4.2)
.
.
.
.
.
.
.
.
.
. 8
1
2
1
2
8
1
2
4
X
Inp
ut
X
U
Ou
tpu
t
Y
V
47
Step 3 : The weight adjustment between the input layer and the competition layer
is given by
j
t
ij
t
ij
t
ij ZUxUU
1
(4.3)
where ‘t’ is the iteration number and ‘ ’ is the learning coefficient.
Step 4 : The weight adjustments for the output layer is given by
j
t
jk
t
jk
t
jk ZVTVV
1
(4.4)
Step 5: This process is repeated for the specified number of iterations and the
stabilized weight matrices are observed.
The number of iterations used for the practical implementation is approximately
1800 and the value of the learning coefficient is 0.7. Though this hybrid network
has been enjoying the merits of both the methodologies, there are some serious
setbacks which limits its practical applications. CPN networks are sufficiently
accurate but the high accuracy is achieved at the cost of high computational
complexity. The computational complexity also increases with the increase in the
number of layers. Also, the accuracy is dependent on the number of iterations
which again raises serious doubts about the robustness of the system. The
accuracy of the results differs with change in the number of iterations which again
has lead to the problem of local minimum. Hence, suitable modifications must be
performed to improve the efficiency of the conventional systems.
4.3.2 Self-Organizing Map
One type of the unsupervised neural networks which posses the self-organizing
property is called Kohonen Self-Organizing Map. Similar to statistical clustering
algorithms, these Kohonen networks are able to find the natural groupings from
the training data set. As the training algorithm follows the “winner take-all”
principle, these networks are also called as competitive learning networks.
4.3.2.1 Architecture of SOM
The topology of the Kohonen self-organizing map is represented as a 2-
Dimensional, one-layered output neural network. Each input neuron is connected
to each output neuron. The number of input layer neurons is based on the
48
dimension of the training pattern. During the process of training, the input
patterns are fed into the network sequentially. The ‘trained’ classes are
represented by the output layer neurons and the center of each class is stored in
the connection weights between input and output layer neurons. The architecture
of SOM is shown in Figure 4.3.
Figure 4.3 Topology of SOM
4.3.2.2 Training algorithm of SOM
The kohonen self-organizing map makes use of the competitive learning rule
for training the network. The “winner-take all” principle is used in this network in
which a winner neuron is selected based on the performance metrics. The weight
adjustment is performed for the winner neurons and also the neighboring neurons
of the winner neuron. The weights of all other neurons remain unchanged. The
neighboring neurons are determined using a radius around the winner neuron. In
this work, unit radius is selected which shows the weights of the winner neuron
alone is adjusted during the process. A detailed training algorithm has been given
below:
Step 1: The weight vectors are randomly initialized.
Step 2: While stopping condition is false, do steps 3 to 6.
Step 3: For each ‘j’(output layer neurons), the Euclidean distance is computed.
jD 2 i
iij xw (4.5)
Step 4: The index j is determined such that jD is a minimum.
Step 5: Update the winner neuron’s weight using the rule
.
.
.
.
.
.
. 8
1
2
1
2
4
X
Inp
ut
X
W
Ou
tpu
t Y
49
oldwxoldwneww ijijiji (4.6)
‘xi’ denotes the intensity values of input data set.
‘α’ denotes the learning rate.
Step 6: Test for stopping condition which is defined by the maximum number of
iterations.
The training process is carried out with the training image set. The entire process
is repeated for the specified number of iterations in the algorithm. The weights
yielded by the network in the last iteration are stored as the stabilized weights.
Further, the testing images are used to estimate the performance of the neural
network. The number of iterations used for the practical application is 1100 and
the value of the learning rate is 0.7. Like CPN, SOM is also purely dependent on
the number of iterations for convergence due to the lack of standard convergence
condition. This has lead to incorrect weight matrix which ultimately affects the
classification accuracy. Though the complexity is less, these are seldom preferred
for practical applications because of the low quality results. Hence, suitable
modifications must be performed to enhance the accuracy of the network without
compromising the convergence rate.
4.4 MODIFIED ARTIFICIAL NEURAL NETWORKS
The analysis of the conventional ANN has clearly shown the necessity for
modifications in the existing ANN to eliminate the demerits associated with them.
In this work, three modified ANN are proposed with an objective to achieve
performance enhancement over the conventional ANN.
4.4.1 Modified CPN
A modified approach of CPN is implemented in this work which guarantees high
accuracy within low convergence time. An iteration-free technique is used in the
proposed approach for weight estimation which ultimately minimizes the
computational time required for the convergence of the system. Since the weights
are estimated without any iteration, the robustness of the system is highly
guaranteed. The problem of local minimum is also eliminated which improves the
efficiency of the modified approach to high extent. Thus, this innovative approach
50
yields high accuracy and superior convergence rate simultaneously which is the
main objective of this work.
Suitable modifications in the architecture and the training algorithm are
performed in the conventional CPN to formulate the MCPN. In terms of
architecture, the number of neurons in the input layer is equal to the number of
neurons in the hidden layer. In terms of training, the algorithm used in the MCPN
does not follow the conventional iterative procedure. Since the number of neurons
in the hidden layer is equal to the number of training sets in MCPN, an
assignment methodology is followed for weight estimation in the network which
overcomes the above mentioned drawbacks of the conventional system. Initially,
the input features are normalized to the range between 0 and 1. The distribution
method of normalization is used in this work which involves the mean and the
standard deviation of the input data. Then, the weights between the input layer
and the hidden layer are automatically assigned to the input and the weights of the
link between the hidden layer and the output layer is assigned to the desired
output. Thus, the weights are estimated without any iterative training procedures.
Since MCPN is devoid of training, the convergence time is highly reduced
besides maintaining substantial accuracy which is evident from the experimental
results. Thus, MCPN function essentially as a partial self-organizing look-up
table and is taught in response to a set of “illustrations” with the help of a
“recording algorithm” rather than through “training examples” with the help of
“learning algorithm”.
4.4.1.1 Architecture of MCPN
The architecture of MCPN is same as Figure 4.2. In this work, an “assignment”
methodology is used for weight estimation and hence the dimensions of the
network must be carefully analyzed. As per the training methodology, the number
of neurons in the hidden layer must be an integral multiple of the number of
classes. The number of hidden layer neurons cannot be lesser than the input layer
neurons and hence an equal number of neurons are used for both the layers. The
51
practical implementation is much easier when equal number of neurons is
selected for both the layers. Thus, architecture of size 8-8-4 is used in this work.
4.4.1.2 Training algorithm of MCPN
The training algorithm is summarized as follows:
Step 1:The weight matrices (U and V ) are randomly initialized. Initially, each
component of the training instance X is presented to the input layer. The
neighborhood function of the hidden layer is defined by the Euclidean distance
jd between the weight vector U and the input vector X is estimated using
Equation (4.1).
Step 2: For each input vector, each neuron in the hidden layer competes with the
other neurons and the neuron with the shortest Euclidean distance wins. The
weight adjustment between the input and the hidden layer selects the weight
vector mU such that
j
m UXUX min (4.7)
Here, mU indicates the associated weights of the winning neurons that are closest
approximations to the input X . The minimum value for the right hand side of
Eqn. (11) is obtained when
mUX = 0 (4.8)
i.e., mUX (4.9)
Step 3: Similarly, for the weight adjustments between the hidden layer ‘j’ and the
output layer ‘k’, the weight vector mV is selected such that
k
m VTVT min (4.10)
Here, mV indicates the associated weights of the winning neurons that are closest
approximations to the targetT . The minimum value for the right hand side of
Equation (4.10) is obtained when
TVm (4.11)
52
Thus the weight vectors ijU and jkV are calculated without any training algorithm.
The winner node changes for every input vector from different categories. This
process is repeated with all the training images from the four categories. The final
set of weight matrices is stored in an array. These weights are further used for the
testing process. In the testing process, a new test vector is given as input and the
output is calculated using the two weight matrices. Each output neuron
corresponds to a class and hence the input image belongs to the class for which
the corresponding neuron yields the maximum value. This procedure is repeated
for all the test images and the classification accuracy is calculated. Thus, this
network is computationally fast since the training methodology follows an
iteration free technique. The estimated weight values yield promising output
accuracy which is evident from experiment results.
In conventional networks such as CPN, the weight adjustment process occurs
across all connection weights for given learning coefficients. Also, a large
number of iterations are required for the connection weights for stability. For each
training instance, a new set of connection weights minimizing the system error
must be calculated. Moreover, the stabilized weights do not guarantee a global
minimum for the system error. These drawbacks are eliminated in MCPN.
4.4.2 Modified SOM1
The proposed MSOM1 is framed by performing suitable modifications in the
conventional SOM network. The same architectural changes and training
procedural changes of MCPN is adopted for MSOM1 also. In terms of
architecture, the number of input layer neurons is equal to the number of output
layer neurons. Unlike MCPN, this modification has limited the size of the input
feature vector. For example, in a 4-level classification system, the number of
input features are only 4 since the number of output neurons is 4. Since less
number of input features yield low accuracy, an alternative arrangement to
increase the size of the input feature vector is used. More than single neuron is
allotted for each class and hence more number of neurons is used in the output
layer which ultimately paves way for a larger input feature set. In terms of
53
training, the same iteration-free methodology adopted for MCPN is followed for
weight estimation which ultimately reduces the computational complexity of the
system. Thus, the proposed system is an iteration-free technique which proves to
be more efficient than the conventional SOM systems.
4.4.2.1 Architecture of MSOM1
The architecture used in this work is a single layered network with an input layer
and an output layer. The number of input features is 8 and the number of classes
is 4 for this application. Since the number of neurons in both the layers is equal,
the number of output layer neurons is increased to 8 with 2 neurons representing
each class. The architecture is same as that of Figure 4.3 except for the number of
neurons in the output layer.
4.4.2.2 Training algorithm of MSOM1
A different methodology is employed for the weight calculation procedure for the
MSOM1. The weights of the winner neurons are estimated using an assignment
methodology rather than the conventional iterative procedures. This method has
also avoided the necessity for other parameters such as the learning rate,
momentum, etc. The number of computational operations are also significantly
reduced which improves the efficiency of the system. The training algorithm used
for MSOM1 is explained through the following procedural steps:
Step 1: The random weights are initialized.
Step 2: For each j (output layer neurons), the difference between the weights and
the input vector is computed using Equation (4.5).
Step 3: Find index j such that jD is a minimum.
Step 4: The weight adjustment between the input and the output layer selects the
weight vector mW such that
j
m WXWX min (4.12)
Here, mU indicates the associated weights of the winning neurons that are closest
approximations to the input X . The minimum value is obtained for the right hand
side of Equation (4.12) when
54
mWX = 0 (4.13)
i.e., mWX (4.14)
Thus, the weight matrix is obtained without any necessity for the iterations. These
weights are further used for testing the MSOM1 with the test images. The number
of misclassified images is observed and the overall classification accuracy is
calculated. The drawbacks of the conventional SOM are eliminated in this
proposed approach. The significant factor is that the proposed approach is devoid
of any iterations which has made the MSOM1 highly efficient than the
conventional SOM.
4.4.3 Modified SOM2
Another significant drawback of the SOM is the lack of standard convergence
condition. In the second modified approach, suitable modifications are performed
to incorporate the standard convergence condition. This objective is achieved by
concatenating the Hopfield neural network to the conventional SOM. The energy
function used in the Hopfield neural network is used as the convergence condition
for the Modified SOM2.
4.4.3.1 Architecture of MSOM2
The framework of the proposed Modified SOM2 is shown in Figure 4.4.
Figure 4.4 Topology of Modified SOM2
The proposed Modified SOM2 is a 2-layer network with the input layer, hidden
layer and an output layer. The number of neurons in the input layer is equal to the
number of input features. The number of neurons in the hidden and output layer is
also eight in order to satisfy the following two conditions. Since the output layer
represents the Hopfield layer, there shall not be any one-to-one connection
1
2
8
1 1
2 2
8 8
W U
55
between the hidden layer and the output layer to ensure stability of the network.
Hence, equal number of neurons is used. Secondly, the feedback from the output
layer neuron is given to the corresponding input layer neuron. Thus, the number
of neurons used in all the layers must be same in the proposed network.
4.4.3.2 Training algorithm of MSOM2
The training procedures of the conventional SOM and the Hopfield networks are
used in this modified network. The training inputs are given to the input layer and
the hidden layer output is calculated using the “winner take-all algorithm” in a
single step. These outputs are given as inputs to the Hopfield layer where the
output values of each neuron is estimated with the help of fixed set of weights
‘U’. These outputs are further fed back to the input layer and the winner neuron’s
weights of the previous iteration are changed using the weight adjustment
equations. With the new set of weights, the output values are once again
calculated and given as input to the Hopfield layer. This procedure continues till
the change in energy value becomes zero. The following mathematical steps
illustrate the training algorithm.
Step 1: The weights are randomly initialized.
Step 2: The input features are supplied to the input layer and output values are
determined using Equation (4.5).
Step 3: Determine j for which jD is minimum. This neuron is selected as the
winner neuron and stored separately.
Step 4: The weight matrix of output layer is calculated using the following
equations.
kjjk DDu (4.15)
kjuu kjjk ; (4.16)
ju jj ;0 (4.17)
Step 5: The output value of Hopfield layer is calculated using following formula
kj
jjkk DuNET (4.18)
56
kNET
k eOUT
11 (4.19)
Step 6: The Liapunov energy function E for the continuous Hopfield network is
further calculated using the output values.
kj
j k
jk OUTDuE 21 (4.20)
This equation is framed from the standard energy function for continuous systems
but without the external inputs. This equation is specified by Hopfield in his
earlier research works.
Step 7: The change in energy is estimated by partially differentiating the above
equation with respect to kOUT . The change in energy is given by
k
ji
jjk OUTDuE
(4.21)
= kk OUTNET (4.22)
Step 8: The values kOUT are fed back to the input layer if rE where ‘r’ is the
error tolerance value. The value of ‘r’ in this work is 0.01.
Step 9: The weights of the winner neuron of the previous iteration is then adjusted
using the following equation
twOUTtwtw ijkijij 1 (4.23)
Step 10: After weight adjustment, the new values of jD is calculated with the new
weights and Steps 3-7 is repeated till the convergence condition ( 01.0E ) is
reached. The number of iterations required for convergence is 1430.
The above training methodology is repeated for training images from all
categories and the final weight matrix ‘W’ represents all the four stored patterns.
The testing process is then carried out with the stabilized weights and the
performance measures are estimated for the testing images. In this method, since
the convergence condition is available, the weights of this method are more
accurate than the weights obtained by conventional SOM.
In this work, accuracy is given more emphasis than the computational
complexity. The accuracy is higher than the conventional SOM since better
57
stabilized weights are obtained through this technique. Even though the
mathematical analysis is slightly complex, the convergence time of this approach
is nominal for practical applications due to the reduction in the number of
iterations.
4.5 EXPERIMENTAL RESULTS AND DISCUSSIONS
The experiments are carried out on the Pentium processor with speed 1.66 GHz
and 1 GB RAM. The software used for the implementation is MATLAB (version
7.0), developed by Math works Laboratory. The five classifiers are trained and
tested individually with the dataset shown in Table 4.1.
Table 4.1 Dataset for brain image classification
Tumor type Training data Testing data No.of
images/class
Meningioma 60 82 142
Glioma 60 76 136
Astrocytoma 60 64 124
Metastasis 60 78 138
Total abnormal
images
540
The efficiency of the classifiers is analyzed in terms of performance measures
such as Classification Accuracy (CA), sensitivity and specificity. A brief analysis
on the computational complexity and the convergence rate is also performed in
this work. These performance measures are estimated for each abnormal category
and then the average measures of all the four categories is determined to judge the
performance of the classifier. The same process is repeated for all the five
classifiers. The formulae for calculating these performance measures are given
by:
FNFPTNTPTNTPCA (4.24)
FNTPTPySensitivit (4.25)
FPTNTNySpecificit (4.26)
In the above equations, TP corresponds to True Positive, TN corresponds to
True Negative, FP corresponds to False Positive and FN corresponds to False
58
Negative. These parameters for a specific category, say, meningioma are as
follows: TP=True Positive (an image of ‘meningioma’ type is categorized
correctly to the same type), TN=True Negative (an image of ‘Non-meningioma’
type is categorized correctly as ‘Non-meningioma’ type), FP= False Positive (an
image of ‘Non-meningioma’ type is categorized wrongly as ‘meningioma’ type)
and FN=False Negative (an image of ‘meningioma’ type is categorized wrongly
as ‘Non-meningioma’ type). ‘Non-meningioma’ actually corresponds to any of
the three categories other than ‘meningioma’. Thus, ‘TP and TN’ corresponds to
the correctly classified images and ‘FP and FN’ corresponds to the misclassified
images The same parameters are determined for all the categories by replacing
‘meningioma’ in the above definitions with other abnormal categories. Thus,
different parameter values are obtained for each class and also for the different
classifiers. These parameters are estimated from the confusion matrix which
provides the details about the false and successful classification of images from
all categories for each classifier.
In the confusion matrix, the row-wise elements correspond to the four
categories and the column-wise elements correspond to the target class associated
with that abnormal category. Hence, the number of images correctly classified
(TP) under each category is determined by the diagonal elements of the matrix.
The row-wise summation of elements for each category other than the diagonal
elements corresponds to the ‘FN’ of that category. The column-wise summation
of elements for each category other than the diagonal elements corresponds to the
‘FP’ of that category. Similarly, ‘TN’ of the specific category is determined by
summing the elements of the matrix other than the elements in the corresponding
row and column of the specific category. The results of the individual classifiers
are discussed in the next section.
The computational complexity of the system is determined by the amount of
mathematical calculations. The computational calculations must be minimal for
an efficient system. In this section, an analysis is performed between the
classifiers based on the number of mathematical operations. The convergence
59
time is directly proportional to the number of operations. The following 2-step
procedure is used to determine the number of mathematical operations for each
classifier.
Step 1: Sub-divide the training algorithm into small segments.
Step 2: Determine the number of mathematical calculations in each segment.
Only the number of calculations is used to determine the computational
complexity. Even though all the mathematical operations are not unique, the
impact of these operations on the modified networks is almost zero except for
Modified SOM2. Hence, it is sufficient to estimate the number of calculations
rather than the effect of these operations.
The convergence time is also dependent on the number of iterations. The
number of iterations required by each network for convergence is different since
the mode of convergence is different. In this work, the convergence time is
calculated based on the number of iterations required for convergence. If the
network converge quickly, then the convergence time is less and vice-versa.
Hence, the analysis is performed with non-uniform iterations but with uniform
values for convergence parameters.
4.5.1 Results of SOM
The performance analysis is performed individually for the SOM classifier.
4.5.1.1 Accuracy measures of SOM
The confusion matrix of the conventional SOM is shown in Table 4.2.
Table 4.2 Confusion Matrix of SOM
Class 1 Class 2 Class 3 Class 4
Meningioma 58 9 8 7
Glioma 7 52 8 9
Astrocytoma 7 7 44 6
Metastasis 8 6 6 58 Class 1=Meningioma; Class 2=Glioma; Class 3 = Astrocytoma; Class 4=Metastasis
The misclassification rate is quite high for the conventional SOM which is
evident from Table 4.2. The performance measures calculated from the confusion
matrix is shown in Table 4.3.
60
Table 4.3 Performance Measures of SOM
TP TN FP FN Sensitivity Specificity CA (%)
Meningioma 58 196 22 24 0.71 0.89 85
Glioma 52 202 22 24 0.68 0.90 85
Astrocytoma 44 214 22 20 0.69 0.91 86
Metastasis 58 200 22 20 0.74 0.90 86
Average value 0.71 0.90 86
The low classification accuracy results of the SOM are clearly shown in Table
4.3. The lack of standard convergence condition is the main drawback of this
network. The stopping condition is quite unclear which results in un-stabilized
weights. The testing process with these weights ultimately fails which accounts
for the inaccurate results. These networks are also forced to depend on iterations
since these networks do not have the assistance of the target vector.
4.5.1.2 Computational complexity and convergence rate of SOM
The mathematical operations involved in the training algorithm are as follows:
(a) Euclidean distance calculation
This technique involves basically 1 subtraction and 1 multiplication
operations. For an input vector of size a, the number of operations required for
calculating the distance of 1 neuron is equal to 2a. Since the Euclidean distance is
calculated for p output layer neurons, the number of mathematical operations
increases to 2ap.
(b) Weight adjustment procedure between the input and the hidden layer
This technique involves 1 subtraction, 2 multiplications and 1 addition
operations. These operations are performed individually for a matrix of size a×p
and hence the total number of operations increases to 4ap.
Thus, the total number of operations involved in SOM is 6ap. This
conventional network is also an iterative network and hence the total number of
operations increases to ‘t(6ap)’. Thus, the complexity of the algorithm is based on
the number of neurons in each layer and the number of iterations used in the
training process. The convergence time requirement is approximately 650 CPU
seconds. Even though the training is performed beyond the specified number of
61
iterations, there is no significant improvement in the accuracy. Thus, the
conventional SOM is quick but the results are less accurate.
4.5.2 Results of CPN
The accuracy measures and the convergence rate analysis are discussed in this
section.
4.5.2.1 Accuracy measures of CPN
The confusion matrix of the conventional CPN is shown in Table 4.4.
Table 4.4 Confusion Matrix of CPN
Class1 Class 2 Class 3 Class 4
Meningioma 71 4 3 4
Glioma 3 65 3 5
Astrocytoma 4 3 55 2
Metastasis 2 5 4 67
The level of misclassification has been slightly reduced in comparison to the
conventional SOM. The performance measure analysis is shown in Table 4.5.
Table 4.5 Performance Measures of CPN
TP TN FP FN Sensitivity Specificity CA (%)
Meningioma 71 209 9 11 0.86 0.95 93
Glioma 65 212 12 11 0.85 0.94 92
Astrocytoma 55 226 10 9 0.86 0.95 93
Metastasis 67 211 11 11 0.86 0.95 92
Average value 0.86 0.95 92.5
The overall performance measures are better than the conventional SOM. The
reason is the presence of the supervised mode of training in the output layer. But,
this accuracy is again dependent on the number of iterations which limits the
robustness of the system. Hence, iteration independent CPN can lead to better
efficiency measures.
4.5.2.2 Computational complexity and convergence rate of CPN
The mathematical calculations involved in the training algorithm are:
62
(a) Euclidean distance calculation
This technique involves basically 1 subtraction and 1 multiplication
operations. For an input vector of size ‘a’, the number of operations required for
calculating the distance of 1 neuron is equal to 2a. Since the Euclidean distance is
calculated for ‘N’ hidden neurons, the number of mathematical operations
increases to 2aN.
(b) Weight adjustment procedure between the input and the hidden layer
This technique involves 1 subtraction, 2 multiplications and 1 addition
operations. These operations are performed individually for a matrix of size a×N
and hence the total number of operations increases to 4aN.
(c) Weight adjustment procedure between the hidden and the output layer
This technique involves 1 subtraction, 2 multiplications and 1 addition
operations. These operations are performed individually for a matrix of size N×p
and hence the total number of operations increases to 4Np. In this calculation, p
denotes the size of the output layer neurons.
The total number of basic mathematical operations per iteration is given by
6aN+4Np. Since CPN is iterative in nature, the system must be trained for
sufficient number of iterations. If ‘t’ is the number of iterations, then the overall
mathematical operations increase to t(6aN+4Np). Usually, the value of ‘t’ is very
high which implicates the high computational complexity of CPN. The time
requirement for CPN is approximately 1280 CPU seconds. In comparison to
SOM, the number of required iterations ‘t’ is high for the conventional CPN.
Thus, CPN is computationally heavy but the results are quite accurate.
4.5.3 Results of MCPN
The MCPN is also tested with the same dataset and the results are tabulated in
this section.
4.5.3.1 Accuracy measures of MCPN
The confusion matrix of the MCPN is shown in Table 4.6.
63
Table 4.6 Confusion Matrix of MCPN
Class1 Class 2 Class 3 Class 4
Meningioma 80 1 0 1
Glioma 1 72 1 2
Astrocytoma 3 1 60 0
Metastasis 0 2 2 74
The number of correctly classified images is significantly high which is evident
from Table 4.6. The performance measures of the MCPN are displayed in Table
4.7.
Table 4.7 Performance Measures of MCPN
TP TN FP FN Sensitivity Specificity CA (%)
Meningioma 80 214 4 2 0.97 0.98 98
Glioma 72 220 4 4 0.95 0.98 97
Astrocytoma 60 233 3 4 0.94 0.99 98
Metastasis 74 219 3 4 0.95 0.98 98
Average value 0.95 0.98 98
The superior classification accuracy of the proposed MCPN is clearly depicted in
Table 4.7. The weight matrices calculated using the proposed approach have
yielded a substantial increase in the classification accuracy. One of the main
reasons is that the modified network is an iteration-free technique. Hence, the
weight matrices are almost error-free unlike conventional CPN in which the
quality of the output weight matrices is dependent on iterations. The requirement
for exact selection of modifiable parameters such as learning rate is also avoided
in this network. Thus, the modified approach is much superior to the CPN in
terms of classification accuracy.
4.5.3.2 Computational complexity and convergence rate of MCPN
The Euclidean distance calculation procedure alone is involved in the training
algorithm of MCPN since there is no necessity for the weight adjustment
equations. Hence, the number of mathematical operations is only 2aN. The
difference between the output layer weights and the target vector is also
calculated in this algorithm which accounts for another 2Np operations. Hence,
64
the total number of operations is given by 2aN+2Np. The impact of the iterations
is also completely minimized since MCPN is an iteration-free technique. The
convergence rate is typically less than 2 seconds. Thus, high accuracy within
quick time is guaranteed in MCPN.
4.5.4 Results of Modified SOM1
The results of the Modified SOM1 are discussed in this section.
4.5.4.1 Accuracy measures of MSOM1
The confusion matrix of the Modified SOM1 is shown in Table 4.8.
Table 4.8 Confusion Matrix of Modified SOM1
Class 1 Class 2 Class 3 Class 4
Meningioma 75 2 2 3
Glioma 2 68 3 3
Astrocytoma 3 2 57 2
Metastasis 2 2 4 70
The level of misclassification is reduced in comparison to the conventional SOM.
The performance measure analysis is shown in Table 4.9.
Table 4.9 Performance Measures of Modified SOM1
TP TN FP FN Sensitivity Specificity CA (%)
Meningioma 75 211 7 7 0.91 0.96 95
Glioma 68 218 6 8 0.89 0.97 95
Astrocytoma 57 227 9 7 0.89 0.96 94
Metastasis 70 214 8 8 0.90 0.96 95
Average value 0.90 0.96 95
The improvement in the correct classification rate of Modified SOM1 over the
conventional SOM is evident from Table 4.9. A more accurate weight matrix is
obtained since the training methodology is different from the conventional
system. The probability of the results being trapped in local minima is low since
the training algorithm does not involve any iterative convergence equations.
4.5.4.2 Computational complexity and convergence rate of MSOM1
The Euclidean distance calculation procedure alone is involved in the training
algorithm of MSOM1 since there is no necessity for the weight adjustment
65
equations. Hence, the total number of mathematical operations is only 2ap. The
number of operations required for the Modified SOM1 is very much lesser than
the other networks proposed in this work. The convergence rate is typically less
than 1 second. Thus, the proposed network has provided quick convergence
without compromising the accuracy.
4.5.5 Results of Modified SOM2
The accuracy measures and the convergence rate analysis are detailed in this
section.
4.5.5.1 Accuracy measures of MSOM2
The confusion matrix of the Modified SOM2 is shown in Table 4.10.
Table 4.10 Confusion Matrix of Modified SOM2
Class1 Class 2 Class 3 Class 4
Meningioma 80 1 0 1
Glioma 0 74 1 1
Astrocytoma 1 1 62 0
Metastasis 0 1 1 74
The number of misclassified images has been highly reduced in comparison to
other classifiers. The performance measures of the Modified SOM2 are displayed
in Table 4.11.
Table 4.11 Performance Measures of Modified SOM2
TP TN FP FN Sensitivity Specificity CA (%)
Meningioma 80 215 1 2 0.98 0.99 99
Glioma 74 219 3 2 0.98 0.98 98
Astrocytoma 62 232 2 2 0.97 0.99 99
Metastasis 74 220 2 2 0.97 0.99 99
Average value 0.98 0.99 99
The classification accuracy of conventional SOM is enhanced due to the inclusion
of the Hopfield network in the architecture and training algorithm. Thus, the basic
drawback (low accuracy) of conventional SOM has been eliminated by this
modified network. Though iteration dependent, the presence of the stable
66
convergence condition is the main reason for the high accuracy of the proposed
approach.
4.5.5.2 Computational complexity and convergence rate of MSOM2
The computational complexity is slightly higher than the other modified networks
due to the inclusion of the Hopfield network. But, the complexity is still lesser
than the conventional multi-layer neural networks. The various operations
involved in this network are:
(a) Euclidean distance calculation
The input layer is similar to the Kohonen layer and hence the number of required
operations is ‘2aN’.
(b) Weight adjustment for the hidden layer neurons.
Basically, the number of mathematical operations involved is 1 subtraction, 1
multiplication and 1 addition. Hence, the total number of operations is ‘3aN’.
(c) Calculation of output value and energy function
But, the output value in the weight adjustment equation is determined by 2
additions, 1 multiplication and 1 division. These operations are performed in the
output layer and hence the total number of operations is ‘4Np’. The calculation of
energy values requires 2 addition operations and 2 multiplication operations
additionally. Hence, the number of mathematical operations is ‘8Np’.
The total number of operations required is‘t(5aN+8Np)’ with ‘t’ being the
number of iterations required for convergence. The complexity is higher than the
modified networks and the conventional single layer networks but it is better than
the conventional multi-layer networks. The first reason is that only one weight
adjustment equation is involved unlike two weight adjustment equations in the
conventional multi-layer networks. Another factor is that the number of iterations
required for convergence is lesser than the number of iterations required for
conventional networks. Even though computational operations required for the
algorithm is high, the convergence rate is better than the conventional multi-layer
networks. The convergence rate of this network is 990 CPU seconds. Thus, high
67
accuracy is guaranteed for the proposed approach within reasonable convergence
time.
4.6 CONCLUSION
In this work, three modified networks are proposed for MR brain image
classification. The performance of these networks is analyzed in terms of
classification accuracy and convergence rate. The results of these networks are
compared with the performance measures of two conventional neural networks.
Experimental results have verified the superior nature of the proposed networks
over the conventional neural networks. The modified networks are found to
possess the capability of accurately classifying the images within nominal
convergence time which is the main objective of this work. Thus, this work has
suggested suitable alternates for the conventional neural networks for practical
medical imaging applications.