EJSR_49_3_15
-
Upload
ganeshbtech -
Category
Documents
-
view
215 -
download
0
Transcript of EJSR_49_3_15
-
8/2/2019 EJSR_49_3_15
1/16
European Journal of Scientific Research
ISSN 1450-216X Vol.49 No.3 (2011), pp.468-483
EuroJournals Publishing, Inc. 2011http://www.eurojournals.com/ejsr.htm
Neural Network Based Respiratory Signal Classification Using
Various Feed-Forward Back Propagation Training Algorithms
A. Bhavani SankarResearch Scholar (Part Time), Anna University of Technology Trichirappalli
Assistant Professor, Department of ECE, Anjalai Ammal- Mahalingam Engineering College
Kovilvenni-614 403, Tamilnadu, IndiaE-mail: [email protected]
Tel: 9443890164
D. KumarDean/Research, Periyar Maniyammai University,Vallam, Tamilnadu, India
E-mail: [email protected]
K. SeethalakshmiSenior Lecturer, Department of ECE
Anjalai Ammal- Mahalingam Engineering College, Kovilvenni-614 403, Tamilnadu, India
E-mail: [email protected]
Abstract
In this work, we examined the classification of respiratory signals using an
Artificial Neural Networks (ANN) system. Respiratory signals contains potentially preciseinformation that could assist clinicians in making appropriate and timely decisions during
sleeping disorder and labor. The extraction and detection of the sleep apnea from composite
abdominal signals with powerful and advance methodologies is becoming a very importantrequirement in apnea patient monitoring. The states were classified into normal, sleep
apnea and respiratory signals with artifacts. Four significant features extracted from
respiratory signal were Energy Index, Respiration Rate, Dominant Frequency and Strength
of Dominant Frequency. In our work, we analyze the performance of five back propagationtraining algorithms, namely, Levenberg-Marquardt, Scaled Conjugate Gradient, Quasi
Newton BFGS Algorithm, One Step Secant and Powell-Beale Restarts algorithm for
classification of the respiratory states. First two significant features, Energy Index and
Respiration Rate were fed as input parameters to the ANN for classification. Then theresult was further improved by taking four features as input for the ANN classifier. The
Levenberg-Marquardt algorithm was observed to be correct in approximately 99% of thetest cases.
Keywords: Sleep Apnea, Motion Artifact, Back propagation training algorithms-Levenberg-Marquardt, Scaled Conjugate Gradient, BFGS algorithm, One Step
Secant, Powell-Beale Restarts
-
8/2/2019 EJSR_49_3_15
2/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 469
1. IntroductionRespiration monitors are of crucial importance in providing timely information regarding pulmonaryfunction in adults and the incidence of Sudden Infant Death Syndrome (SIDS) in neonates. However,
to accurately monitor respiration, the noise inherent in measuring devices, as well as artifacts
introduced by body movements must be removed or discounted. One can imagine a multitude ofintelligent classification algorithms that could help to reach better identification mechanism. For
example an algorithm should be capable of classifying different types of signal with different
characteristics feature. Such an algorithm has the potential to become major classification tool. Therehave been enormous growth in developing efficient algorithm for classification of the respiratory
signals, the reduced computational steps, reduced number of parameters used, increasing the capability
to differentiate the signals and easy to implement in hardware setup to provide clinical support.ANN are biologically inspired networks inspired by the human brain with its organization of
neurons and decision making processes which are useful in application areas such as pattern
recognition and classification. The decision making process of ANN is more holistic, based on the
aggregate of entire input patterns, whereas the conventional computer has to wade through theprocessing of individual data elements to arrive at a conclusion. Neural networks derive their power
due to their massively parallel structure and ability to learn from experience. They can be used for
fairly accurate classification of input data into categories, provided they are previously trained to do so.The accuracy of the classification depends on the efficacy of training, which, in turn, depends upon the
rigor and depth of the training. The knowledge gained by the learning experience is stored in the form
of connection weights, which are used to make decisions on fresh input. The processing elements areorganized into layers, and layers interconnect to form a network. The inputs to the processing unit are
weighted signals derived from similar processing units of the previous layer. Usually, a processing
element is linked to all the neurons of its immediate neighboring layers, which gives rise to massive
parallelism in architecture.
2. Previous ResearchMany researchers have suggested various techniques including unconventional approaches, such as
engineering diagnostic techniques, for determining patient conditions. A review of the literature
includes, in [1] a method for the online classification of sleep/wake states based on cardio respiratorysignals produced by wearable sensors was described. The method was conceived in view of its
applicability to a wearable sleepiness monitoring device. The method uses a fast Fourier transform as
the main feature extraction tool and a feed forward artificial neural network as a classifier. Twoapproaches to classify the ECG biomedical signals are presented in [2]. One is the Artificial Neural
Network (ANN) with multilayer perceptron and the other is the Fuzzy Logic with Fuzzy Knowledge
Base Controller (FKBC). It is focused on eye blink detection using kurtosis and amplitude analysis ofEEG signal in [3]. An Artificial Neural Network (ANN) is trained to detect the eye blink artifact. In
research work [4], a simple scheme has been proposed to identify the type as well as the distance of the
moving vehicles based on the noise emanated by them. Using simple feature extraction techniques, theone-third-octave band frequency spectrum of the noise were computes and used as a feature set. The
feature sets were then used to model a feed forward network trained by back propagation algorithm. In
[5], they proposed a classification method entailing time-series EEG signals with back propagation
neural networks (BPNN). To test the improvement in the EEG classification performance with theproposed method, comparative experiments were conducted using Bayesian Linear Discriminant
Analysis (BLDA). The work in [6] takes a step in that direction by introducing a hybrid evolutionary
neural network classifier (HENC) combining the evolutionary algorithm, which has a powerful globalexploration capability, with gradient-based local search method, which can exploit the optimum
offspring to develop a diagnostic aid that accurately differentiates malignant from benign pattern.
-
8/2/2019 EJSR_49_3_15
3/16
470 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
A spike sorting method using a simplified feature set with a nonparametric clustering algorithm
was presented in [7]. The performances of different off-line methods for two different
Electroencephalograph (EEG) signal classification tasks motor imagery and finger movement, areinvestigated in [8]. The main purpose of this paper is to provide a fair and extensive comparison of
some commonly employed classification methods under the same conditions so that the assessment of
different classifiers will be more convictive. As a result, a guideline for choosing appropriate
algorithms for EEG classification tasks is provided. An alternative evaluation of Obstructive Sleep
Apnea (OSA) based on ECG signal during sleep time was proposed in [9]. K-Nearest Neighbor (KNN)supervised learning classifier was employed for categorizing apnea events from normal ones, on a
minute-by-minute basis for each recording. The authors have focused on the various schemes forextracting the useful features of the ECG signals for use with artificial neural networks in [10]. Once
feature extraction is done, ANNs can be trained to classify the patterns reasonably accurately. The
paper [11] deals with a novel method of analysis of EEG signals using wavelet transform andclassification using artificial neural network (ANN) and logistic regression (LR). A modification on
Levenberg-Marquardt algorithm for MLP neural network learning was proposed in [12]. The proposed
algorithm has good convergence.The paper [13] reports the results of the clinical evaluation for detection and classification of
sleep apnea syndromes. An automatic classification of respiratory signals using a Field Programmable
Gate Array (FPGA) was implemented in [14]. The design of an automatic infant cry recognitionsystem that classifies three different kinds of cries, which come from normal, deaf and asphyxiatinginfants, of ages from one day up to nine months old was presented in [15]. The classification of the
states of patients with certain diseases in the intensive care unit using their ECG and an Artificial
Neural Networks (ANN) classification system was examined in [16]. The paper [17] proposed a noveland simple local neural classifier for the recognition of mental tasks from on-line spontaneous EEG
signals. The proposed neural classifier recognizes three mental tasks from on-line spontaneous EEG
signals. Correct recognition is around 70%. The work in [18] introduces an innovative signalclassification method that is capable of on-line detection of the presence or absence of normal
breathing. A classification method for respiratory sounds (RSs) in patients with asthma and in healthy
subjects is presented in [19]. Grow and Learn (GAL) neural network is used for the classification. The
trade-off between the time consuming training of ANNs and their performances in ECG analysis isexplored in [20], [21].Multilayer perceptrons trained with the back propagation algorithm are tested in
detection and classification tasks and are compared to optimal algorithms resulting from likelihood
ratio tests in [22].The extraction of features derived from the autoregressive modeling and thresholdcrossing schemes that was used to classify respiratory signals was referred in [23]. The Matlab coding
and different back propagation training algorithms for the classification of respiratory signal using
neural network was referred using [24], [25].
3. Proposed WorkOur work focused on the classification of the respiratory signal using Neural Network. The capability
of classifying respiratory signals and detecting apnea episodes are of crucial importance for clinicalpurposes. The features are derived from the autoregressive modeling and modified threshold crossingschemes and it is fed as input to neural network. The network classifies the respiratory signals into the
following categories: (1) normal respiration, (2) respiration with artifacts and (3) sleep apnea. This
classification is capable of detecting fatigue of the human by identifying sleep apnea, early detection ofsleep troubles and disorders in groups at risk, reduces the risks of being affected by serious heart
diseases in future. The main contribution of this paper is the analysis of different back propagation
training algorithms those are necessary for classification of the respiratory signals which yields not
only the classification but also the analysis of various ailments.
-
8/2/2019 EJSR_49_3_15
4/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 471
4. Respiratory Data AnalysisThe traditional methods for assessment of sleep related breathing disorders are sleep studies with therecordings of ECG, EEG, EMG and respiratory effort. Sleep apnea detection with ECG recordings
requires more number of electrodes on the skin and people may wear it continuously for effective
monitoring. EEG measurement can also be used for the detection of sleep apnea but the brain signalsare always random in nature. For the complete detection, we need more number of samples for
analysis. Also, the mathematical modeling of EMG signals is very complex for sleep apnea detection.
From the results in [1], the respiratory signals alone are sufficient and perform even better than ECG,EEG and EMG. In our paper, we consider only the respiratory signal for the detection of sleep apnea
since it is more convenient and do not require more number of electrodes on the skin.
Our previous work deals with the feature extraction using modified threshold based algorithmthat plays a vital role since the classification is completely based on the values of the extracted
features. The fundamental features of respiratory signal provide the numerical value which is compared
with the threshold values and the classification results will be produced. A Synthetic respiratory signal
of 0.5 Hz is used for simulation. The signal was sampled at 16 Hz and the four main features ofrespiratory signal is extracted using modified threshold based algorithm. Total of 300 samples with
four feature set is used for simulation.
The fundamental features of respiratory signals that are extracted and used for simulation are,1. Energy Index (EI)2. Respiration frequency estimated by a modified Zero crossing scheme (FZX)3. Dominant frequency estimated by AR modeling (FAR)4. Strength of the dominant frequency estimated by AR modeling (STR)
5. Neural ClassifierThe use of neural network systems in respiratory signal analysis offers several advantages over
conventional techniques. The neural network can perform the necessary transformation and clusteringoperations automatically and simultaneously. The neural network is also able to recognize complex and
nonlinear groups in the hyperspace. The latter ability is a distinct advantage over many conventional
techniques. The neural networks have been defined as systems composed of many simple processing
elements, that operate in parallel and whose function is determined by the network's structure, thestrength of its connections, and the processing carried out by the processing elements or nodes.
Generally, the neural networks are adjusted or trained so that an input in particular leads to a specified
or desired output. The training of a network is done trough changes on the weights based on a set ofinput vectors. The training adjusts the connection's weights from the nodes, after obtaining an output
from the network and comparing it with a wished output, with previous presentation of the whole set of
input vectors. The neural networks have been trained to make complex functions in many applicationareas including the pattern recognition, identification, classification, speech, vision, and control
systems.
-
8/2/2019 EJSR_49_3_15
5/16
472 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
Figure 1: Overview of proposed approach
In general, the training can be supervised or not supervised. The methods of supervised training
are those that are more commonly used, when labeled samples are available. Among the most popular
models are the feed-forward neural networks, trained under supervision with the back-propagationalgorithm. The proposed approach for the classification of respiratory signal involves preprocessing of
the respiratory signal, extraction of characteristic features and classification using ANN techniques.
The overview of the proposed approach is shown in Figure 1. Using neural network techniques, thepatient states were classified into three classes: normal, sleep apnea and respiration with artifacts.
6. Multilayer Feed forward NetworkMultilayer feed-forward network architecture is made up of multiple layers: an input layer, a number ofhidden layers and an output layer as shown in Figure 2. Neurons are the computing elements in eachlayer. The acceleration or retardation of the input signals is modeled by the weights. The weighted sum
of the inputs to each neuron is passed through an activation function to get the output of a neuron. In
addition to the inputs there are also biases to each neuron.
Figure 2: A Multilayer feed-forward network
-
8/2/2019 EJSR_49_3_15
6/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 473
7. Back Propagation AlgorithmsThe decision making process of the Artificial neural networks ANN is holistic, based on the features ofinput patterns, and is suitable for classification of biomedical data. Typically, multilayer feed forward
neural networks can be trained as non-linear classifiers using the generalized Back propagation
algorithm (BPA). The BPA is a supervised learning algorithm, in which a sum square error function isdefined, and the learning process aims to reduce the overall system error to a minimum. Training a NN
can be viewed as the minimization of an error function. The performance can be improved if suitable
error functions and minimization algorithms are chosen. Regarding the minimization algorithm, weselected the five different back propagation training algorithms, namely, Levenberg-Marquardt, Scaled
Conjugate Gradient, Quasi Newton BFGS Algorithm, One Step Secant and Powell-Beale Restarts and
compared their performance in the classification of respiratory signal.
7.1. Levenberg-Marquardt
The Levenberg-Marquardt algorithm was designed to approach second-order training speed withouthaving to compute the Hessian matrix H. When the performance function has the form of a sum of
squares (as is typical in training feed forward networks), then the Hessian matrix can be approximated
asH = JTJ (1)
and the gradient can be computed asg= JTe (2)
where J is the Jacobian matrix that contains first derivatives of the network errors with respect to the
weights and biases, and e is a vector of network errors. The Jacobian matrix can be computed through a
standard back propagation technique that is much less complex than computing the Hessian matrix.The Levenberg-Marquardt algorithm uses this approximation to the Hessian matrix in the following
update:
eJ]IJJ[XX T1Tk1k
+ += (3)
When the scalar is zero, this is just Newtons method, using the approximate Hessian matrix.
When is large, this becomes gradient descent with a small step size. Thus, is decreased after eachsuccessful step (reduction in performance function) and is increased only when a tentative step would
increase the performance function. In this way, the performance function will always be reduced ateach iteration of the algorithm.
7.2. Scaled Conjugate Gradient
Each of the conjugate gradient algorithms requires a line search at each iteration. This line search iscomputationally expensive, since it requires that the network response to all training inputs be
computed several times for each search. The scaled conjugate gradient algorithm was designed to avoid
the time-consuming line search. This algorithm is too complex to explain in a few lines, but the basic
idea is to combine the model-trust region approach used in the Levenberg-Marquardt algorithm withthe conjugate gradient approach.
7.3. BFGS Algorithm
Newtons method is an alternative to the conjugate gradient methods for fast optimization. The basicstep of Newtons method is
k1
kk1k gAXX
+ = (4)
Where Ak is the Hessian matrix of the performance index at the current values of the weightsand biases. Newtons method often converges faster than conjugate gradient methods. Unfortunately, it
is complex and expensive to compute the Hessian matrix for feed forward neural networks. There is a
-
8/2/2019 EJSR_49_3_15
7/16
474 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
class of algorithms that is based on Newtons method, but which doesnt require calculation of second
derivatives. These are called quasi-Newton (or secant) methods. They update an approximate Hessian
matrix at each iteration of the algorithm. The update is computed as a function of the gradient. Thequasi-Newton method that has been most successful in published studies is the Broyden, Fletcher,
Goldfarb, and Shanno (BFGS) update.
7.4. One Step Secant Algorithm
Since the BFGS algorithm requires more storage and computation in each iteration than the conjugate
gradient algorithms, there is need for a secant approximation with smaller storage and computation
requirements. The one step secant (OSS) method is an attempt to bridge the gap between the conjugate
gradient algorithms and the quasi-Newton (secant) algorithms. This algorithm does not store thecomplete Hessian matrix; it assumes that at each iteration, the previous Hessian was the identity
matrix. This has the additional advantage that the new search direction can be calculated without
computing a matrix inverse.
7.5. Powell-Beale Restarts
For all conjugate gradient algorithms, the search direction will be periodically reset to the negative of
the gradient. The standard reset point occurs when the number of iterations is equal to the number ofnetwork parameters (weights and biases), but there are other reset methods that can improve theefficiency of training. One such reset method was proposed by Powell, based on an earlier version
proposed by Beale. For this technique we will restart if there is very little orthogonality left between
the current gradient and the previous gradient. This is tested with the following inequality.2
kk
T
1k g2.0gg (5)
If this condition is satisfied, the search direction is reset to the negative of the gradient.
8. Performance Measures
The neural networks performance in our classification process is evaluated by means of the followingfour performance indices: Mean Squared Error, Confusion Matrix, Receiver Operating CharacteristicsCurve and Linear Regression Curve.
8.1. Mean Squared Error
The MSE is computed by taking the differences between the target and the actual neural networkoutput, squaring them and averaging over all classes and internal validation samples. Because the
neural network outputs are real numbers between 0 and 1, this result in a Mean Squared Error between
0 and 1. As the neural network is iteratively trained, the MSE should drop to some small, stable value.Each neural network has its MSE plotted independently. Some components may stop if they reach
stability earlier than others, and hence have MSE plots which do not extend over all iterations.
8.2. Confusion Matrix
Given a classifier and an instance, there are four possible outcomes. If the instance is positive and it isclassified as positive, it is counted as a true positive; if it is classified as negative, it is counted as a
false negative. If the instance is negative and it is classified as negative, it is counted as a true negative;
if it is classified as positive, it is counted as a false positive. Given a classifier and a set of instances(the test set), a two-by-two confusion matrix as shown in Table 1 can be constructed representing the
dispositions of the set of instances. The matrix can also be extended for nine or more possible
outcomes.
-
8/2/2019 EJSR_49_3_15
8/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 475
Table 1: Confusion Matrix
True Class
Hypothesized
Cl
ass
p n
P True Positives False Positives
N False Negatives True Negatives
8.3. Receiver Operating Characteristics Curve
Receiver operating characteristics (ROC) graphs are useful for organizing classifiers and visualizing
their performance. ROC graphs are commonly used in medical decision making, and in recent years
have been used increasingly in machine learning and data mining research. A ROC graph is a plot withthe false positive rate on theXaxis and the true positive rate on the Yaxis. The point (0,1) is the perfect
classifier: it classifies all positive cases and negative cases correctly. It is (0,1) because the false
positive rate is 0 (none), and the true positive rate is 1 (all). The point (0,0) represents a classifier thatpredicts all cases to be negative, while the point (1,1) corresponds to a classifier that predicts every
case to be positive. Point (1,0) is the classifier that is incorrect for all classifications. In many cases, a
classifier has a parameter that can be adjusted to increase TP at the cost of an increased FP or decrease
FP at the cost of a decrease in TP. Each parameter setting provides a (FP, TP) pair and a series of suchpairs can be used to plot an ROC curve. A non-parametric classifier is represented by a single ROC
point, corresponding to its (FP, TP) pair.
8.3.1. Features of ROC Graphs
An ROC curve or point is independent of class distribution or error costs. An ROC graph encapsulates all information contained in the confusion matrix, since FNis the
complement ofTP and TNis the complement ofFP. ROC curves provide a visual tool for examining the tradeoff between the ability of a classifier
to correctly identify positive cases and the number of negative cases that are incorrectly
classified.
The more each curve hugs the left and top edges of the plot, the better the classification.8.4. Linear Regression
A data model explicitly describes a relationship between predictor and response variables. Linear
regression fits a data model that is linear in the model coefficients. The most common type of linear
regression is a least-squares fit, which can fit both lines and polynomials, among other linear models. Itis the study of the behavior of one variable in relation to several compartments induced by another
variable. By the use of regression line or equation as shown in Figure 3; we can predict scores on the
dependent variable from those of the independent variable. There are different nomenclatures of
independent and dependent variables. The R value is an indication of the relationship between theoutputs and targets. If R=1, this indicates that there is an exact linear relationship between outputs and
targets. If R is close to zero, then there is no linear relationship between outputs and targets.
-
8/2/2019 EJSR_49_3_15
9/16
476 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
Figure 3: Regression Line
9. Training, Validation and Testing of Neural NetworkA Synthetic respiratory signal of 0.5 Hz is used for simulation. The signal was sampled at 16 Hz and
the four main features of respiratory signal is extracted using modified threshold based algorithmwhich is part of our research work. Total of 300 samples with two and four feature set is used for
simulation. The neural network and the training algorithm are implemented with the Matlab's Neural
Network Toolbox.To train the networks, the data were divided into three sets: 1) training, 2) validation, and 3)
test. The training set (TR) contained the data used to update the synaptic weights. The performance of
the network was evaluated on the validation set (VA) after each iteration and the training was stoppedif the performance of VA did not increase for more than 15 training iterations or the minimal gradient
was reached. The test set (TE) was used to measure the performance of the network after the training.
The Procedure to train the neural network for classification is given as follows:
1. Load the data, consisting of input vectors and target vectors.2. Create a network. We used a feed-forward network with the tan-sigmoid transfer function in the
hidden layer and linear transfer function in the output layer. This structure is useful for data
classification problems. Use 20 neurons (arbitrary) in one hidden layer. The network has threeoutput neurons, because of three classes.
3. Train the network. The network uses five different back propagation algorithms for training.The application divides input vectors and target vectors into three sets as follows:
60% are used for training. 20% are used to validate that the network is generalizing and to stop training before over
fitting.
The last 20% are used as a completely independent test of network generalization.4. The classification accuracy was calculated by taking the number of samples correctly classified,
divided by the total number of samples.
10. Simulation Results and DiscussionThe Specifications for the neural network employed for the classification of respiratory signal are
given in Table.2
-
8/2/2019 EJSR_49_3_15
10/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 477
Table 2: Specifications for Neural Network
S.No Parameters Value
1. Type of network Feed forward
2. No. of neurons in the input layer 2
3. No. of neurons in the hidden layer 20
4. No. of neurons in the output layer 3
5. Performance function MSE6. Activation function in the hidden layer Tan-sigmoid
7. Activation function in the output layer Linear
8. Learning rate 0.05
9. Maximum no. of epochs 1000
10. Minimum MSE Value 0
Class distributions of the samples in the training, validation and testing data sets are given in
Table 3. It includes four main features of respiratory signal.
Table 3: Class distributions of the Samples
Class Training Validation TestingNormal 62 18 20
Sleep apnea 60 20 20
Artifacts 58 20 22
Total 180 58 62
First two significant features extracted from the respiratory signal were fed as input parameters
to the ANN for classification. Then the result was further improved by taking four features as input for
the ANN classifier. The results with four features are demonstrated in the following figures, whichplots the mean square error versus number of epochs for several algorithms. First we train the network
to classify the respiratory signal using Levenberg-Marquardt algorithm. The mean square error plot and
ROC plot for the simulation is shown in Figure.4.
Figure 4: (a) Plot of MSE (b) Plot of ROC using Levenberg-Marquardt algorithm
(a) (b)
The results are reasonable because of the following considerations:
The final mean-square error is small. The test set error and the validation set error has similar characteristics.
-
8/2/2019 EJSR_49_3_15
11/16
478 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
No significant overfitting has occurred by iteration 7, where the best validation performanceoccurs.
Each curve in ROC plot hugs more the left and top edges of the plot, and hence the betterclassification.
Figure 5: Confusion Matrix
The Confusion matrix showing a 98.7 % precision, after 13 training epochs and mean squareerror of 0.0453 is shown in Figure 5. The classification accuracy was calculated by taking the number
of samples correctly classified, divided by the total number of samples.
The regression plot in Figure 6 shows the relationship between the outputs of the network andthe targets. If the training were perfect, the network outputs and the targets would be exactly equal, but
the relationship is rarely perfect in practice.
Figure 6: Regression plot for training, testing and validation
-
8/2/2019 EJSR_49_3_15
12/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 479
The three axes represent the training, validation and testing data. The dashed line in each axis
represents the perfect result outputs = targets. The solid line represents the best fit linear regressionline between outputs and targets. The R value is an indication of the relationship between the outputs
and targets. If R=1, this indicates that there is an exact linear relationship between outputs and targets.
If R is close to zero, then there is no linear relationship between outputs and targets. The training dataindicates a good fit. The validation and test results also show R values that greater than 0.9.Then we
repeat the training using Scaled Conjugate Gradient, Quasi Newton BFGS Algorithm, Powell-BealeRestarts and One Step Secant algorithm and MSE and ROC plots are shown in Figures 7 and 8.
Figure 7: MSE plot for a) Scaled Conjugate Gradient b) BFGS c) Powell-Beale Restarts d) One Step Secant
(a) (b)
(c) (d)
-
8/2/2019 EJSR_49_3_15
13/16
480 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
Figure 8: ROC plot for a) Scaled Conjugate Gradient b) BFGS c) Powell-Beale Restarts d) One Step Secant
(a) (b)
(c) (d)
11. Comparative EvaluationIt is very difficult to know which training algorithm will be the fastest for a given problem. It will
depend on many factors, including the complexity of the problem, the number of data points in thetraining set, the number of weights and biases in the network, the error goal, and whether the network
is being used for pattern recognition or function approximation.
The Comparative evaluation is done for all the algorithms with two and four features as input
parameters and it is tabulated in Tables 4 and 5. In both cases, it is observed that, the training of theneural network with LM algorithm is faster than other algorithms. With LM algorithm, the number of
iterations taken to achieve the performance goal is less (13 iterations) with four features whencompared to performance with two features (17 iterations). But the other algorithms with two featuresachieves same accuracy consumes only less number of iterations when compared to performance with
four features. With other algorithms, the neural network takes more epochs to reach the defined MSE
which is zero. On the other hand, with LM algorithms, the network converges when it reaches thedefined error, that is 0, and after the training has reached only around 7 epochs. Scaled Conjugate
gradient and BFGS achieves 97.3% and 96.7% accuracy but it takes 35 and 39 number of iterations to
reach the performance goal. Powell-Beale Restarts and One Step Secant algorithms provide 97.7%
classification accuracy and very low MSE value of 0.033. But it takes 57 and 88 number of iterations
-
8/2/2019 EJSR_49_3_15
14/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 481
to achieve the performance goal of zero MSE. With the reported experiments we confirm that it is
easier to classify the respiratory signals using LM algorithm.
Table 4: Performance Evaluation with 2 features
S.No Training Algorithm Lowest MSE
Obtained
Epoch at which
Performance goal is met
No. of Iterations Accuracy
(%)1 Levenberg-Marquardt 0.0222 11 17 98.72 Scaled Conjugate Gradient 0.0415 16 22 96.73 BFGS 0.0411 19 25 984 Powell-Beale Restarts 0.0437 13 19 975 One Step Secant 0.0416 38 44 96.3
Table 5: Performance Evaluation with 4 features
S.No Training Algorithm Lowest MSE
Obtained
Epoch at which
Performance goal is met
No. of Iterations Accuracy (%)
1 Levenberg-Marquardt 0.0453 7 13 98.72 Scaled Conjugate Gradient 0.0512 29 35 97.3
3 BFGS 0.0471 33 39 96.74 Powell-Beale Restarts 0.0333 51 57 97.75 One Step Secant 0.0333 82 88 97.7
Since the LM algorithm is designed for least squares problems that are approximately linear,the performance of the LM algorithm is better relative to the other algorithms. From the literature it is
observed that the LM is suitable for small and medium size networks with enough memory available.
If memory is a problem, then we have to opt for scaled conjugate gradient or BFGS algorithm. SCGand other algorithms are suitable for pattern recognition problems rather than classification problems.
12. Conclusion and Future WorkIn the present study, an automatic system for the classification of respiratory states employing ANNtechniques for decision making was developed and implemented. The decision-making was performedusing features extracted from respiratory signals. Emphasis was placed on selection of the
characteristic features and for the accurate extraction of these features. From the results, it can be seen
that the Levenberg-Marquardt back propagation algorithm provided an excellent performance for thestudied application. The performance and tradeoffs between the five training methods was also studied.
The choice of algorithm to be used is a tradeoff between the performance, convergence power and
training input. The proposed Levenberg-Marquardt approach exhibited a superior performance in terms
of classification accuracy and was also easier and simpler to implement and use. The performance ofthe system can be further enhanced by training with a larger number of training inputs, which increase
the network ability to classify unknown signals.
References[1] Walter Karlen, Claudio Mattiussi and Dario Floreano, 2009. Sleep and Wake Classification
with ECG and Respiratory Effort Signals IEEE Transactions on Biomedical circuits and
systems, Vol. 3, No. 2.
[2] Saad Alshaban and Rawaa\ Ali, 2010. Using Neural and Fuzzy Software for the Classificationof ECG SignalsResearch Journal of Applied Sciences, Engineering and Technology.
-
8/2/2019 EJSR_49_3_15
15/16
482 A. Bhavani Sankar, D. Kumar and K. Seethalakshmi
[3] Brijil Chambayil, Rajesh Singla, R. Jha, 2010. EEG Eye Blink Classification Using NeuralNetwork Proceedings of the World Congress on Engineering 2010, London, Vol I.
[4] Norasmadi Abdul Rahim, Paulraj M P, Abdul Hamid Adom, Sathishkumar Sundararaj, 2010.Moving Vehicle Noise Classification using Backpropagation Algorithm 6th InternationalColloquium on Signal Processing & Its Applications (CSPA).
[5] Arjon TURNIP, Keum-Shik HONG, Shuzhi Sam GE, 2010. Backpropagation NeuralNetworks Training for Single Trial EEG Classification Proceedings of the 29th Chinese
Control Conference, China.[6] R. El hamdi, M. Njah, M. Chtourou, 2010. Breast Cancer Diagnosis Using a Hybrid
Evolutionary Neural Network Classifier 18th Mediterranean Conference on Control &Automation Congress, Morocco.
[7] Zhi Yang, Qi Zhao and Wentai Liu, 2009. Neural Signal Classification Using a SimplifiedFeature Set with Nonparametric ClusteringElsevier.
[8] Boyu Wang, Chi Man Wong, Feng Wan, Peng Un Mak, Pui In Mak, and Mang I Vai, 2009.Comparison of Different Classification Methods for EEG-Based Brain Computer Interfaces: A
Case Study Proceedings of the International Conference on Information and Automation,China, IEEE.
[9] Martin O. Mendez, Davide D. Ruini, Omar P. Villantieri, Matteo Matteucci, 2007. Detectionof Sleep Apnea from surface ECG based on features extracted by an Autoregressive ModelProceedings of the 29th Annual International Conference of the IEEE EMBS, France.
[10] Rajesh Ghongade, Dr. A.A. Ghatol, 2007. A Brief Performance Evaluation of ECG FeatureExtraction Techniques for Artificial Neural Network Based ClassificationIEEE.
[11] Abdul Hamit Subasia, Ergun ErcElebi, 2005. Classification of EEG signals using neuralnetwork and logistic regression Computer Methods and Programs in Biomedicine, Elsevier.
[12] Amir Abolfazl Suratgar, Mohammad Bagher Tavakoli, and Abbas Hoseinabadi, 2005.Modified Levenberg-Marquardt Method for Neural Networks Training World Academy of
Science, Engineering and Technology.
[13] Khaled M. Al-Ashmouny, Ahmed A. Morsy, Shahira F. Loza, 2005. Sleep Apnea Detectionand Classification Using Fuzzy Logic: Clinical Evaluation Proceedings of the IEEE
Engineering in Medicine and Biology 27th Annual Conference, China.[14] Bozidar Marinkovic, Matthew Gillette, and Taikang Ning, 2005. FPGA Implementation of
Respiration Signal Classification Using a Soft-Core ProcessorIEEE.
[15] Mohamed Soltane, Mahamood Ismail, Zainol Abinin Abdul Rashid, 2004. Artificial NeuralNetworks(ANN) Approach to PPG Signal Classification International Journal of Computingand Information Sciences, Vol.2, No.1.
[16] Orion F. Reyes-Galaviz, Carlos Alberto Reyes-Garci, 2004. A System for the Processing ofInfant Cry to Recognize Pathologies in Recently Born Babies with Neural Networks 9thConference Speech and Computer, St. Petersburg, Russia.
[17] N.Kannathal, U. Rajendra Acharya, Choo Min Lim , PK. Sadasivan, SM Krishnan, 2003.Classification of cardiac patient states using artificial neural networks Journal in Clinical
Cardiology, Vol 8, No 4.[18] Jose del R. Millan, Josep Mourino, Marco Franze, Febo Cincotti, 2002. A Local Neural
Classifier for the Recognition of EEG Patterns Associated to Mental TasksIEEE Transactionson Neural Networks, Vol. 13, No. 3.
[19] Peter Varady,Tamas Micsik, Sandor Benedek, and Zoltan Benyo, 2002. A Novel Method forthe Detection of Apnea and Hypopnea Events in Respiration Signals IEEE Transactions on
Biomedical Engineering, Vol. 49, No. 9.
[20] Li Gang, Ye Wenyu, Tin ling, Yu Qilian, Y U Xuemin, 2000. An Artificial-IntelligenceApproach to ECG Analysis IEEE Engineering in Medicine and Biology.
-
8/2/2019 EJSR_49_3_15
16/16
Neural Network Based Respiratory Signal Classification Using Various Feed-Forward
Back Propagation Training Algorithms 483
[21] Rosaria Silipo and Carlo Marchesi, 1998. Artificial Neural Networks for Automatic ECGAnalysisIEEE Transactions on Signal Processing, Vol. 46, No. 5.
[22] Z.H.Michalopoulou, L.W.Nolte and D.Alexandrou, 1995. Performance Evaluation ofMultilayer Perceptrons in Signal Detection and Classification IEEE Transactions on Neural
Networks, Vol. 6, No. 2.[23] Taikang Ning and Joseph D. Bronzino, 1989. Automatic Classification of Respiratory Signals
IEEE Engineering in Medicine & Biology Society 11th Annual International Conference, IEEE.[24] S.N.Sivanandam, S.Sumathi, S.N.Deepa, 2006. Introduction to Neural Networks using Matlab
6.0 Tata McGraw-Hill.[25] Mark Hudson Beale, Martin T. Hagan, Howard B. Demuth, 2010. Neural Network Toolbox 7-
Users Guide, The Math Works Inc.