Performance Analysis of a Novel IT2 FCM Algorithm

SHASHANK ANIL HUDDEDAR MAYANK KAGLIWAL BADRINATH SINGHAL FRANK RHEE

1.NEED FOR THIS RESEARCH

Let’s look at what inspired our research initiative

1. The numerous applications

Computer Vision■ 2D to 3D face modeling■ RBF classification■ FCM, Fuzzy ART

Natural Language Processing■ Machines that write

interesting novels in a specified style

Image source: Google

1. The numerous applications

Finance■ Stock price analysis -

Zarandi & Turksen

Computing with Words

Image source: Google

23,067Number of Scopus results with the keywords “fuzzy set applications”

2. Drawbacks

❏ The original Karnik-Mendel (KM) algorithm and variants such as the enhanced KM (EKM), iterative algorithm with stopping condition (IASC), and enhanced IASC (EIASC) algorithms used in type reduction are applicable for multidimensional pattern sets where computation of the centroid is achieved by iterating each dimension of the pattern sets separately (i.e., processed in 1-dimension (1-D)).

❏ These methods ignore the possible correlation among the multiple dimensions and can result in high computational complexity.

“In this paper, we use the concept of embedded lines and

planes as explained in

V. Saxena, N. Yadala, R. Chourasia, and F. C.-H. Rhee,Type Reduction Techniques

for Two-Dimensional Interval

Type-2Fuzzy Sets, IEEE Int. Conf. Fuzzy Syst., pp. 1-6, 2017

Let’s look at the modified IT2 FCM algorithm.

2. Proposed Modified IT2 FCM Algorithm

1. Modified KM Algorithm

❏ The main aim behind the modified KM algorithm is to visually estimate a centroid boundary of 2-D IT2 FSs.

❏ First, we fix θ ∈ [0, π] such that a line creating an angle π /2 + θ with the dimensional feature axis (x1-axis) is used as a reference axis.

❏ Project the entire multidimensional domain to a single dimension.(Dimensionality Reduction).

❏ The modified KM algorithm is analogous to the original KM algorithm where the reference axis corresponds to a reference point, namely the origin and the projection of each point on the reference axis corresponds to the distance of the points from the origin.

1. Modified KM Algorithm

❏ The embedded set obtained as a result of dimensionality reduction is given as

❏ Apply the KM algorithm on this 1-D T2 FS. As a result, convergenceof the modified KM algorithm provides two centroids namely, cl,θ and cr,θ.The embedded lines corresponding to these centroids are defined as

1. Modified KM Algorithm

Figure: Embedded set corresponding to an embedded-curve for direction θ:1) side view (left) 2) top view (right)

2. Modified KM Algorithm

❏ The corresponding embedded set is given by

❏ The centroid of this embedded set is obtained by

❏ The above procedure is repeated for a predetermined number ofdistinct θ, where θ takes discrete values in [0,π].

❏ By connecting the centroids, we obtain the centroid boundary which consists of all the points of the set c(Aθ).

2. Pseudocode and Flowchart

2. Pseudocode and Flowchart

2. Pseudocode and Flowchart

1. Pseudocode and Flowchart

Complexity Analysis

Let’s look at the brief time complexity analysis of KM and Modified KM

KM vs Modified KM

❏ The significant difference in the original IT2 FCM and modified IT2 FCM algorithms is in the type reduction step.

❏ It is observed that the major computational cost involved in the case of the modified KM algorithm is due the number of directions.

❏ If the total number of pattern sets are N and total number of dimensions are m, then the complexity of the KM algorithm isO(m · N log(N)). On the other hand, the complexity of the proposed modified KM algorithm is O(d · N log(N)).

❏ Therefore, by controlling the hyperparameter d, we can achieveimproved results at comparable costs

3. Experiments

Let’s look at the experimental results by applying proposed IT2 FCM algorithm.

EXPERIMENT 1Visualization of the centroid region

1. Motivation

❏ In the case of the original IT2 FCM algorithm, we can visualize the uncertainty of 2-D pattern sets by plotting the cl and cr in 2-D.

❏ By involving only two sets of points, we are able to visualize centroid boundary as a rectangle only.

❏ In case of the modified IT2 FCM algorithm, we plot the T1 FSs ofcentroids considering each direction.

“

2. Construction

We consider a 2-D pattern set that consists of three uniformly distributed gaussian pattern sets with a particular mean and standard deviation. The modified IT2 FCM algorithm was implemented with the generated data and pairs of cl and cr were plotted for every direction

Figure: Visualization of embedded line, embedded plane, and upper and lowermembership functions

3. Visualizations

Figure: Visualization of centroid boundary for 180 directions

4. Observations

❏ As the number of directions are increased the shape of the region was found to tend towards an ellipse.

❏ Shape of the centroid boundary represents the distribution of thepattern sets in corresponding cluster.

❏ This helps in visualizing the centroid boundary better than the original IT2 FCM algorithm

EXPERIMENT 2Estimation of positioning error of cluster centers

1. Motivation

❏ we can consider the center obtained by the modified IT2 FCM algorithm as the “center of centers."

❏ By introducing noise in pattern sets, we check for the robustness of the algorithm

❏ The center obtained by the modified IT2 FCM algorithm is averaged over a set of directions, as a result of which we can expect this algorithm to be more robust compared to the FCM and original IT2 FCM algorithms.

“

2. Construction

We consider two non-overlapping 2-D clusters by sampling 150 points from two uniform distributions with predetermined mean and standard deviation as shown in Fig. below.

The FCM (m = 2.5), original IT2 FCM (m1 = 2, m2 = 3), and modified IT2 FCM (m1 = 2, m2 = 3) algorithms were set to the task of estimating the center of the distributions. To get the maximum clustering result number of directions were set to 180, where θ ∈ [0, π] with resolution of 1 radian.

In the next step, for each iteration, 10 uniformly distributed points were added to the distribution. This process was continued till 16 iterations (160 noise points). The experiment was repeated several times to reduce the variance of the pattern set. The used quality measure was a sum of position errors estimated by the FCM, original IT2 FCM, and modified IT2 FCM algorithms

2. Construction

Figure: Clean and Noisy pattern sets

3. Observation

Figure: Error comparison between the centers obtained from FCM, original IT2 FCM, and modified IT2 FCM algorithms

1. Observation

❏ It is visible that the error in center positioning by the original IT2FCM and modified IT2 FCM algorithms are similar when noisepresent in the pattern sets is low

❏ As noise increases, the error in center positioning is less in themodified IT2 FCM algorithm but is more in case of the original IT2FCM algorithm

❏ Hence, significant improvement in center positioning can be obtained by using the modified IT2 FCM algorithm when pattern set contain high noise.

EXPERIMENT 3Performance analysis of the FCM, original IT2 FCM and modified IT2 FCM algorithms for overlapping pattern sets

1. Motivation

❏ From the previous experiments, results indicate that on average, the modified IT2 FCM algorithm can provide better center as compared to that given by both the FCM and original IT2 FCM algorithms.

❏ Therefore, by applying our proposed modified IT2 FCM algorithm to the task of uncertain pattern recognition, we expect this algorithm to give a better performance than both the FCM and original IT2 FCM algorithms.

“

2. Construction

We draw 400 points from a uniform distribution and create two partially overlapping pattern sets consisting of 100 points in one cluster and 300 points in the other cluster. The two clusters vary in size and density.

25test cases were considered in this experiment. Starting with 100 points in the smaller cluster and 300 points in the larger cluster as shown in Fig. below, we repeatedly remove a fixed number (10 as in our experiment) of data points from the larger cluster and introduce the same number of points in the smaller cluster in subsequent test cases. Removal andintroduction of points was done randomly.

The FCM, original IT2 FCM, and modified IT2 FCM algorithms were applied to the task of pattern recognition with fuzzifier values, m = 2:5, m1 = 2, and m2 = 3 . This experiment wasrepeated 30 times to reduce variance of the pattern set. To optimize the performance, we considered 180 directions in case of the modified IT2 FCM algorithm, where θ 2 [0; π] with resolution of 1 radian

2. Construction

Figure: Overlapping pattern sets

3. Observation

Figure: Performance comparison between the FCM, original IT2 FCM, and modified IT2 FCM algorithms: Magnified view.

3. Observation

❏ The results were found to be consistent as mentioned in “GeneralType-2 Fuzzy C-Means Algorithm for Uncertain Fuzzy Clustering"

❏ The proposed algorithm was observed to perform better than both the FCM and original IT2 FCM algorithms when overlapping was low. After a certain threshold of overlapping, all the algorithms performed almost equally which is evident from above.

❏ Hence, it can be concluded that on an average, the modified IT2FCM algorithm shows better recognition rate than both the FCM and original IT2 FCM algorithms in the case of overlapping pattern sets

. Observation

❏ The results were found to be consistent as mentioned in “GeneralType-2 Fuzzy C-Means Algorithm for Uncertain Fuzzy Clustering"

❏ The proposed algorithm was observed to perform better than both the FCM and original IT2 FCM algorithms when overlapping was low. After a certain threshold of overlapping, all the algorithms performed almost equally which is evident from above.

❏ Hence, it can be concluded that on an average, the modified IT2FCM algorithm shows better recognition rate than both the FCM and original IT2 FCM algorithms in the case of overlapping pattern sets

3. Summary

Let’s look at the summary of the paper in brief.

1. Results

❏ From the experimental results, we conclude that our proposed modified IT2 FCM algorithm is more robust to noisy and overlapping pattern sets as compared to both the FCM and original IT2 FCM algorithms.

❏ Also it provided the surrounding boundary inside which the actual center was located which helps in better visualization purposes.

1. Future Work

❏ In the current implementation of the proposed modified KM algorithm, the directions are chosen uniformly from the discritized domain of θ, where θ ∈ [0, π]. For further study, we will try to tune this hyperparameter, θ based on the domain of the pattern sets to improve the complexity even further.

❏ In addition, rigorous study can be performed for finding the set of directions with minimum cardinality to obtain the best possibleclustering result.

Thanks!Any questions?

You can find us at:Shashank Huddedar- huddedar@iitg.ernet.inMayank Kagliwal –m.kagliwal@iitg.ernet.inBadrinath Singhal – badrinath2602@gmail.comDr. Frank Rhee – frhee@hanyang.ac.kr