Computational Intelligence in Complex Decision Systems G. Oltean
1
ANN for
pattern recognition applications
Computational Intelligence in Complex Decision Systems G. Oltean
Pattern recognition includes two main
activities:
• Feature extraction: The features are essential for the particular
recognition / classification process. Typically, the decision taken
at this stage is relatively subjective and depends on practical
considerations such as the accessibility of measurements, their
costs, etc. The criteria for the features selection and their ordering
are based either on their importance in characterizing the forms
or on the contributions they bring to the recognition performance
(the correct / erroneous recognition percentages).
• pattern recognition (pattern classification), meaning decision
making about the membership to one class or another class for
each pattern.
Computational Intelligence in Complex Decision Systems G. Oltean
Pattern classification:
Data clustering, uses unsupervised learning, that
is, determining the number of groups, group centers andthe membership of each object (pattern) to each group;
The development and utilization of a patternrecognition systems based on supervised learning.
The once-built and trained recognition system is able torecognize to what class a new pattern belongs;
new pattern - a pattern that did not participate in theprocess of developing (training, testing) the recognitionsystem.
Computational Intelligence in Complex Decision Systems G. Oltean
• Recunoastere de catre om prin
perceptie (similaritate, relatii).
• Se folosesc direct obiectele de
clasificat
• Recunoastere de catre calculator -
utilizare formalism matematic: extragerea
trasaturilor in format numeric. Similaritati
matematice pe date, relatii pe date.
•Nu se utilizeaza direct obiectele (formele) ci
descrierea lor prin trasaturi - obiecte de date
Computational Intelligence in Complex Decision Systems G. Oltean
The typical structure of a pattern recognition system
Reading SegmentationFeature
extraction
Pattern
classification
Post-
processing
Pattern Decision
Reading - Retrieval and transformation of information from the real world into the space of
patterns, in a format compatible with the way of representing data in the computer (sensors /
translators: video camera, microphone, etc.)
Segmentation (if required) - Isolation of objects of interest from the background or from other
objects (isolation of objects from images or isolation of words)
Feature extraction - Measuring the properties of patterns to build a feature vector. Features are
chosen so that they have similar values for patterns in the same class and different values for
patterns in different classes - distinctive features. PCA - (Principal Component Analysis) tries to
reduce the feature vector size, resulting in an easy-to-process, less-redundant vector.
Pattern classification - Uses the feature vector to assign the pattern to a class: pattern
recognition. Because perfect classification is (almost) impossible, a more general task is to
determine a pattern probability for each class.
Post-processing - Used to recommend certain actions to be taken, each action having a
certain cost (example: decision on what to do next, after the pattern has been recognized.
Computational Intelligence in Complex Decision Systems G. Oltean
Start
Collect data
Select features
Select ANN architecture
Train ANN
Evaluate ANN
Accepted
performance?
ANN for pattern
recognition
Workflow for Neural
Network Design(intended for pattern recognition)
Computational Intelligence in Complex Decision Systems G. Oltean
Block diagram of a ANN-based pattern recognition system
Each output stands for one class:
1 – the input pattern is recognized in that class
0 – the input pattern is not recognized in that class
Computational Intelligence in Complex Decision Systems G. Oltean
Pattern Recognition for Iris flowers
Classify iris flowers based on four features (attributes)
The data set consists of 150 samples.
"irisInputs" is an 4x150 matrix, whose rows are:
1. Sepal length in cm2. Sepal width in cm3. Petal length in cm4. Petal width in cm
"irisTargets" is an 3x150 matrix, where each ith column indicates which category the ith iris belongs to with a 1 in one element (and zeros in the other two elements).
1. Iris Setosa2. Iris Versicolour3. Iris Virginica
One category of iris is linearly separable, the other two are not.
Computational Intelligence in Complex Decision Systems G. Oltean
Iris setosa Iris versicolor Iris virginica
Iris dataset is by far the earliest and the most commonly
used dataset in the literature of pattern recognition.
The dataset contains 150 instances of iris flowers
collected in Hawaii.
Computational Intelligence in Complex Decision Systems G. Oltean
5.5000 2.5000 4.0000 1.3000 0 1.0000 05.5000 2.6000 4.4000 1.2000 0 1.0000 06.1000 3.0000 4.6000 1.4000 0 1.0000 05.8000 2.6000 4.0000 1.2000 0 1.0000 05.0000 2.3000 3.3000 1.0000 0 1.0000 05.6000 2.7000 4.2000 1.3000 0 1.0000 05.7000 3.0000 4.2000 1.2000 0 1.0000 05.7000 2.9000 4.2000 1.3000 0 1.0000 06.2000 2.9000 4.3000 1.3000 0 1.0000 05.1000 2.5000 3.0000 1.1000 0 1.0000 05.7000 2.8000 4.1000 1.3000 0 1.0000 06.3000 3.3000 6.0000 2.5000 0 0 1.00005.8000 2.7000 5.1000 1.9000 0 0 1.00007.1000 3.0000 5.9000 2.1000 0 0 1.00006.3000 2.9000 5.6000 1.8000 0 0 1.00006.5000 3.0000 5.8000 2.2000 0 0 1.00007.6000 3.0000 6.6000 2.1000 0 0 1.00004.9000 2.5000 4.5000 1.7000 0 0 1.00007.3000 2.9000 6.3000 1.8000 0 0 1.00006.7000 2.5000 5.8000 1.8000 0 0 1.00007.2000 3.6000 6.1000 2.5000 0 0 1.0000
Inputs [cm] Targets
Dataset illustration setosa versicolor virginica
Computational Intelligence in Complex Decision Systems G. Oltean
Features of iris flowers
40
45
50
55
60
65
70
75
80
Lungime
sepala
20
25
30
35
40
45
Latime
sepala
10
20
30
40
50
60
70
Lungime
petala
0
5
10
15
20
25
Latime
petala
Setosa
Versicolor
Virginica
Sepal length
Sepal width
Petal length
Petal width
Computational Intelligence in Complex Decision Systems G. Oltean
2D representation of iris dataset – only 2 features
45 50 55 60 65 70 75 80
20
25
30
35
40
45
lungime sepala
latim
e s
epala
Setosa
Versicolor
Virginica
Sepal length
Sepal w
idth
Computational Intelligence in Complex Decision Systems G. Oltean
>> nprtool >> (nnstrt - nprtool)
Matlab R2016a
Computational Intelligence in Complex Decision Systems G. Oltean
Computational Intelligence in Complex Decision Systems G. Oltean
18 / 19
Computational Intelligence in Complex Decision Systems G. Oltean
Computational Intelligence in Complex Decision Systems G. Oltean
Computational Intelligence in Complex Decision Systems G. Oltean
Computational Intelligence in Complex Decision Systems G. Oltean
Computational Intelligence in Complex Decision Systems G. Oltean
number of cases correctly classified
misclassified cases
total percent of correctly classified cases
total percent of misclassified cases
Training: 104 samplesValidation: 23 samplesTest: 23 samples
percent of correctly classified cases
percent of misclassified cases
Confusion matrix
(Error matrix)
Computational Intelligence in Complex Decision Systems G. Oltean
Classification results (partial)
3 misclassified out of 150
The ANN produces numerical values on each output:
ideally close to zero - input pattern is not in this class
ideally close to one - input pattern is in this class
The final decision is made considering the maximum between all the outputs
Computational Intelligence in Complex Decision Systems G. Oltean
Classification results 3 misclassified out of 150
Computational Intelligence in Complex Decision Systems G. Oltean
26 / 19
51 0.0000 0.9959 0.004152 0.0001 0.9909 0.009053 0.0001 0.9813 0.018754 0.0000 0.9983 0.001655 0.0000 0.9863 0.013756 0.0001 0.9949 0.005057 0.0002 0.9574 0.042458 0.0008 0.9990 0.000259 0.0000 0.9978 0.002160 0.0002 0.9965 0.003361 0.0002 0.9996 0.000262 0.0001 0.9936 0.006363 0.0000 0.9998 0.000264 0.0001 0.9870 0.012965 0.0001 0.9993 0.000666 0.0000 0.9979 0.002167 0.0002 0.9686 0.0312
68 0.0000 0.9996 0.000369 0.0000 0.9011 0.098970 0.0000 0.9996 0.000371 0.0002 0.3908 0.609072 0.0000 0.9993 0.000773 0.0001 0.8275 0.172474 0.0000 0.9972 0.002775 0.0000 0.9989 0.001176 0.0000 0.9977 0.002377 0.0000 0.9926 0.007478 0.0001 0.6913 0.308679 0.0001 0.9821 0.017880 0.0000 0.9998 0.000281 0.0000 0.9996 0.000382 0.0000 0.9997 0.000283 0.0000 0.9995 0.000584 0.0001 0.2744 0.7255
85 0.0003 0.9559 0.043886 0.0003 0.9673 0.032387 0.0001 0.9879 0.012088 0.0000 0.9979 0.002189 0.0001 0.9981 0.001890 0.0000 0.9986 0.001491 0.0001 0.9977 0.002392 0.0001 0.9912 0.008893 0.0000 0.9995 0.000594 0.0004 0.9994 0.000295 0.0001 0.9978 0.002196 0.0001 0.9987 0.001297 0.0001 0.9980 0.001998 0.0000 0.9986 0.001499 0.0008 0.9990 0.0002
100 0.0000 0.9986 0.0014
Predicted output for data from index 51 t0 100 (versicolor)
Computational Intelligence in Complex Decision Systems G. Oltean
Problem
Implement in Python the ANN pattern trained before to recognize the iris flowers.The parameter of the ANN (weight and bias of hidden layer and for output layer) will be automatically load from text files generated previously from matlab.
Compare the results of your ANN implemented in Python with the results provided by the ANN in matlab, using the same input vectors.
Computational Intelligence in Complex Decision Systems G. Oltean
Retraining …. worse results; why?
Computational Intelligence in Complex Decision Systems G. Oltean
Classification results 53 misclassified out of 150
Computational Intelligence in Complex Decision Systems G. Oltean
1 2 3
1
2
3
13
32.5%
1
2.5%
1
2.5%
86.7%
13.3%
0
0.0%
13
32.5%
2
5.0%
86.7%
13.3%
0
0.0%
0
0.0%
10
25.0%
100%
0.0%
100%
0.0%
92.9%
7.1%
76.9%
23.1%
90.0%
10.0%
Target Class
Ou
tpu
t C
las
s
Confusion Matrix
Exercise The results obtained when testing a
3-class pattern recognition system
are synthesized in the confusion
matrix.
a) How many patterns do we have
in each of the three classes
according to the reference
distribution?
b) Determine how many patterns
have been recognized in each of
the three classes. How many of
these were correctly recognized?
c) Interpret the values in all cells
on 2nd row in the confusion matrix.
d) Interpret the values of all the
cells in the 1st column.
e) Interpret the values in the lower
right cell
Computational Intelligence in Complex Decision Systems G. Oltean
Statistical measures of the performance
of a binary classification
The performance of pattern recognition system can be
very well characterized by the recognition rates
considered for each class considering a binary classification
problem.
Thus, for each class
The instance (example, sample) belonging to the class is
considered positive
the instance (example, sample) not belonging to the
class is considered negative.
Computational Intelligence in Complex Decision Systems G. Oltean
True positive TP (correctly classified as positive; hit) – an
instance belonging to the class is correctly classified as
belonging to the class;
True negative TN (correctly classified as negative, correct
rejection) – an instance that does not belong to the class is
correctly classified as not belonging to the class;
False pozitiv FP (incorrectly classified as positive, false
alarm) – an instance that does not belong to the class is
classified (incorrectly) as belonging to the class;
False negative FN (incorrectly classified as negative, miss) –
an instance belonging to the class is classified (incorrectly) as
not belonging to the class;
Classification results
Computational Intelligence in Complex Decision Systems G. Oltean
True positive rate (sensitivity; recall; hit rate):
TPR = TP / P = TP / (TP + FN)
True negative rate (specificity; selectivity):
TNR = TN / N = TN / (TN + FP)
False positive rate (fall-out):
FPR = FP / N =FP / (TN + FP)
False negative rate (miss rate):
FNR = FN / P = FN / (TP + FN)
TPR + FNR = 1
TNR + FPR = 1
Recognition rates
Computational Intelligence in Complex Decision Systems G. Oltean
36 / 19
Precision (positive predicted values):
PPV = TP / (TP + FP)
success probability of making a correct positive class classification
Accuracy:
ACC = (TP + TN) / (P + N) = (TP + TN) / (TP + FN + TN + FP)
success probability of detecting the positive and negative class
Recall (TPR):
Recall = TP / P = TP / (TP + FN)
how sensitive the model is towards identifying the positive class
The 3 pillars of binary classification
Computational Intelligence in Complex Decision Systems G. Oltean
Rate
ClassInstances in the
target (reference) class
TPR[%]
TNR [%]
FPR[%]
FNR[%]
123
For a pattern recognition
problem the confusion matrix
is presented in the next figure.
Fill in the next table.
Exercise
Top Related