Fuzzy Signature Neural Network

Presented by: U5251881 XuanYing ZHU

Supervisor: Professor Tom GEDEON

Final presentation for COMP8780 IHCC Project

Outline

l  Background l  Neural Network l  Fuzzy Logic, Fuzzy Rule Based System and

Fuzzy Signature l  Fuzzy Signature Neural Network l  Previous work

l  Design & Implementation

U5251881 XuanYing ZHU

l  Construct Fuzzy Signature Neural Network l  Implement testing suite

l  Experiment l  Experiment 1: with no missing data l  Experiment 2: with 20% missing data l  Experiment 3: with less fuzzy neurons

l  Conclusion & Future work

Background

l  Neural Network:

U5251881 XuanYing ZHU

Fig2. Example of a feed-forward back propagate neural network Source: Chandra, P. “Fuzzy Signature Neural Networks for Rule Discovery”

Fig1. Example of a single neuron Source: Kun, H. “Fuzzy Signature Neural Network”

A mathematical model that is inspired by biological neural network.

Background

l  Fuzzy Logic:

U5251881 XuanYing ZHU

Fig 3: Difference between crisp set and fuzzy set Source: Gedeon,  T.D.  2013,  Bio-­‐inspired  Compu8ng  –  COMP8420  Lecture  Notes,  Research  School  of  Computer  Science,  Australian  Na8onal  University.  

l  Fuzzy Rule Based System:

Represent knowledge based on degrees of membership

l  Rule: If A THEN B (A, B: collections of propositions containing linguistic variables) e.g. Rule: IF x is A3 OR y is B1 THEN z is C1

l  Problem: Number of inputs

Number of terms in the input => Rule Explosion

Background U5251881 XuanYing ZHU

Fig4: Two structures of fuzzy signature Source: Gedeon,  T.D.  2013,  Bio-­‐inspired  Compu8ng  –  COMP8420  Lecture  Notes,  Research  School  of  Computer  Science,  Australian  Na8onal  University.  

l  Fuzzy Signature: l  Structure data into vectors of fuzzy values,

each of which can be a further vector l  A solution for rule explosion

Background U5251881 XuanYing ZHU

Fig5: Example of aggregation Source:  Gedeon,  T.D.  2013,  Bio-­‐inspired  Compu8ng  –  COMP8420  Lecture  Notes,  Research  School  of  Computer  Science,  Australian  Na8onal  University.  

l  Fuzzy Signature: l  Aggregate:

l  GPLAB l  a  Gene8c  Programming  toolbox  for  MATLAB    l  Produce  fuzzy  signatures  based  on  their  inner-­‐structures  

and  intra-­‐rela8ons  

U5251881 XuanYing ZHU

Fig6: Example of Fuzzy Signature Neural Network

l  Fuzzy Signature Neural Network

Background

Background U5251881 XuanYing ZHU

l  Previous work l  Similar neural network has been created by

Kun HE. l  Semi-randomly created fuzzy signatures. l  Number of fuzzy signatures is determined

by users. l  Our approach

l  Data-driven way to create fuzzy signatures l  Self-determined fuzzy signatures number l  Improve HE’s fuzzy signature neural

network l  More automatic l  Reduce risks caused by manual selection

of fuzzy signatures number

Design & Implementation U5251881 XuanYing ZHU

l  Construct fuzzy signature neural network

Fig7: Steps of constructing fuzzy signature neural network

l  Implement testing suite

Fig8: Steps of implementing testing suite

Design & Implementation U5251881 XuanYing ZHU

l  Damage input l  Randomly remove some values

l  Cluster input l  Agglomerative hierarchical clustering

l  Advantages:

Fig9: Example of agglomerative hierarchical clustering

l  Do not need users to specify number of clusters

l  More informative l  Deterministic

Design & Implementation U5251881 XuanYing ZHU

l  Obtain fuzzy signatures l  Generate fuzzy signatures l  Obtain membership values

l  Create & Train neural network

Fig6: Example of Fuzzy Signature Neural Network

Receive input

Get membership

value

Generate actual output

Compare with desired

output

Update weights

Initialize weights

Fig10: Steps to create and train neural network

Design & Implementation U5251881 XuanYing ZHU

l  Implement testing suite l  Test and collect results

l  K-fold cross validation -> split dataset into training and testing datasets

l  Map function

l  Extract network information

0

0.5

1

1 2 3 4 5

membership value

Class 0

0.5

1

1 2 3 4 5

membership value

Class

Fig11(a): actual output Fig11(b): desired output Fig11: Example of actual output and desired output

Experiment U5251881 XuanYing ZHU

l  Experiment 1: with no missing data

Table 1: Results of our approach with no missing values and five fuzzy signatures

Table 2: Results of HE’s approach with no missing values and five fuzzy signatures

Experiment U5251881 XuanYing ZHU

l  Experiment 2: with 20% missing data

l  Cancer dataset: missing one attribute

cancer   diabetes   high  salary   medium  salary   low  salary  This  project   34.88170445   4.92186359   -­‐1.840490798   -­‐5.421686747   6.650860993  

Kun's  approach   0.308510638   5.324141977   -­‐1.923076923   12.42937853   19.34968791  

-­‐10  

-­‐5  

0  

5  

10  

15  

20  

25  

30  

35  

40  

Decreased  pe

rcen

tage  

Fig 12: Decreased percentage of testing accuracy for HE’s and our approach

Experiment U5251881 XuanYing ZHU

l  Experiment 3: with fewer fuzzy neurons

50  

55  

60  

65  

70  

75  

80  

85  

90  

95  

100  

6  fuzzy  neurons  

5  fuzzy  neurons  

4  fuzzy  neurons  

3  fuzzy  neurons  

2  fuzzy  neurons  

Accuracy

50  

55  

60  

65  

70  

75  

80  

85  

90  

95  

100  

6  fuzzy  neurons  

5  fuzzy  neurons  

4  fuzzy  neurons  

3  fuzzy  neurons  

2  fuzzy  neurons  

Accuracy

Fig 13: Testing accuracy for KUN’s and our approach as fuzzy neuron numbers decrease

Fig 13(a): Testing accuracy for KUN’s approach as fuzzy neuron numbers decrease

Fig 13(a): Testing accuracy for our approach as fuzzy neuron numbers decrease

Conclusion & Future work U5251881 XuanYing ZHU

l  Conclusion l  This approach achieves stable and robust

good results in extreme situations l  With missing values l  With fewer fuzzy signatures

l  Data-oriented VS semi-random

l  Future work l  Find a more consistent and less time-

consuming fuzzy signature generation method.

l  Implement an another mapping function.