A Survey of Fuzzy Clustering

8/2/2019 A Survey of Fuzzy Clustering

1/16

Mathl. Comput. Modelling Vol. 18, No. 11, pp. 1-16, 1993Prin ted in Great Britain. All rights reserved

0895-7177/93 $6.00 + 0.00Copyright@ 1993 Pergamon Press Ltd

A S u r v e y o f F u z zy C l u s t e r i n gM.-S. YANG

Departm ent of Math emat icsChung Yuan Christian UniversityChu ngli, Taiwan 32023, R.O.C.

(Received and accepted October 1993)

Abstract-This paper is a survey of fuzzy set th eory applied in cluster analysis. These fuzzycluster ing algorithm s have been widely studied a nd applied in a variety of substan tive areas. Theyalso become the m ajor techniques in cluster ana lysis. In this paper , we give a survey of fuzzy clusteringin th ree categories. The first category is the fuzzy clustering based on fuzzy relation. The second oneis the fuzzy cluster ing based on objective function. Finally, we give an overview of a nonpar amet ricclassifier. That is the fuzzy generalized k-nearest neighbor r ule.

Keywords-cluster analysis,FUZZYlustering, UZZY-partitions, UZZYelation,FUZZY-means,Fu zzy genera lized -near estneighbor ule, Clust ervalidity.1. INTRODUCTION

Cluster analysis is one of the major t echn iques in pat tern recognition. It is an approach toun supervised learning. The importa nce of cluster ing in various a reas such as taxonomy, medicine,geology, bu siness , engineering syst ems a nd image processing, etc., is well docum ent ed in th elitera tu re. See, for example, [l-3]. The convent iona l (ha rd) clust ering meth ods rest rict th at eachpoint of th e data s et belongs to exactly one clust er. Fu zzy set theory pr oposed by Zadeh [4] in1965 gave an idea of un certain ty of belonging which was described by a mem bersh ip function.The use of fuzzy sets provides impr ecise class members hip inform at ion. Applications of fuzzy settheory in cluster ana lysis were early proposed in the work of Bellman, Kalaba an d Zadeh [5] andRuspin i [S]. These p aper s open th e door of resea rch in fuzzy clust ering. Now fuzzy clust ering ha sbeen widely st udied an d applied in a variety of subst an tive ar eas. See, for example, [7,8]. Thesemethods become the important tools to cluster analysis.

The pur pose of th is pap er is briefly to sur vey th e fuzzy set th eory a pplied in clust er an alysis.We will focus on thr ee cat egories: fuzzy clust ering based on fuzzy relation, fuzzy clust ering basedon objective functions, a nd th e fuzzy genera lized k-nea rest neighbor ru le-one of th e powerfulnonpara metr ic classifiers. We ment ion tha t t his paper is a nonexhaustive survey of fuzzy settheory applied in cluster ana lysis. We may ha ve overlooked the paper s which might be importan tin fuzzy clustering.

In Section 2, we give a brief sur vey of fuzzy c-part itions, (fuzzy) similarit y r elation, an d fuzzyclustering based on fuzzy relation. Section 3 covers a big part of fuzzy clustering based on theobjective functions. In th e litera tu re of fuzzy clust ering, th e fuzzy c-mea ns (FCM) clust eringalgorithm s defined by Dun n [9] an d genera ted by Bezdek [7] ar e the well-known and powerfulmeth ods in cluster ana lysis. Several variat ions an d generalizations of the FCM shall be discussedin th is section. Fina lly, Section 4 cont ain s an overview of a famous nonpar am etr ic classifier-k-nearest neighbor rule in the fuzzy version.

Typeset by AM-&X


2/16

2 M.-S. YANG2. F UZZY CLUST ER ING BASED ON FU ZZY RE LATION

Le t X be a subset of an s-dimensional Euclidean space Ws with its ordinary Euclidean norm1). 11 nd let c be a positive integer bigger than one. A partition of X into c clusters can berepresented by mutually disjoint sets Bi, . . . , II, such that Bi U . . . U B, = X or equivalently bythe indicator function ~1, . . . , pe such that pi(z) = 1 if z E Bi and pi(z) = 0 if z $! Bi for allz E X and all i = l,... ,c. In this case, X is said to have a hard c-partition by the indicatorfunctions p = (pi,. . . ,pc).

The fuzzy set, first proposed by Zadeh [4] in 1965, is an extension to allow pi(x) to be a function(called a membership function) assuming values in the interval [OJ]. Ruspini [6] introduced afuzzy c-partition p = (~1, . . . , ,uc) by the extension to allow pi(z) to be functions assuming valuesin the interval (O,l] such that PI(Z) +. . . + pC(x) = 1 since he first applied the fuzzy set in clusteranalysis.

A (hard) relation T in X is defined to be a function T : X x X -+ (0, 1) where x and y in Xare said to have relation if r(z, y) = 1. A (hard) relation r in X is called an equivalence relationif and only if for all z, y E X,

(1) R(z,z) = 1 (reflexivity),(2) r(z, Y) = T(Y, z) (symmetry), and(3) T(Z,Z) = 7$/-z) = 1 for some z E X ==+ T(z, y) = 1 ( transitivity).

Zadeh [4] defined a fuzzy relation T in X by an extension to allow the values of T in the interval[O,l], where ~(2, y) denotes the strength of the relation between z and y. In Zadeh [lo], he defineda similarity relation S in X if and only if S is a fuzzy relation and for all z, y, z E X, S(z, z) = 1(reflexivity), S(z, y) = S(y, z) (symmetry), and S(z, y) 2 VzEx(S(q z) A S(y, z)) (transitivity)where V and A denote max and min. It is obvious that the similarity relation S is essentially ageneralization of the equivalence relation.

Let us consider a finite data set X = (51,. . . , z,} of W. Denote pi(zj) by /+, i = 1,. . . ,c,j = l,... , n, and r(zj, zle) by rjrc, j, k = 1,. . . , n. Let V,, be the usual vector space of realc x n matrices and let uij be the ijth element of U E V,,,. Following Bezdek [7], we define thefollowing:

Then MC is exactly a (nondegenerate) hard c-partitions space for the finite data set X. DefineA 5 B if and only if aij < bij Vi, where A = [aij] and B = [bij] E V,,. Define R oR = [rij] E V,,with rij = VE==, (~ik A Tkj)o Let

R, = {R E Vn, 1Tij E {O,l} Vi,j; I 5 R; R= RT; R=RoR).Then R, is the set of all equivalence relations in X. For any U = [pgj] E n/i,, let the relationmatrix R = [rjk] in V,, be defined by

1, if bij = /& = 1 for some i,Tjk = { 0, otherwise.Then R is obviously an equivalence relation corresponding to the hard c-partitions U, since itsatisfies reflexivity, symmetry, and transitivity. That is, for any U E MC there is a relationmatrix R in R , such that R is an equivalence relation corresponding to U.

For a fuzzy extension of MC and R ,, let


3/16

A Surveyf Fuzzy Clusteringand

Rfn = {R E V,, ( rij E [0, l] Vi, j; 1 I R; R=RTandR>RoR).Then Mfc is a (nondegenera te) fuzzy c-par tit ions spa ce for X, an d R fn is the set of all similarityrelations in X. Note that Bezdek an d Har ris [ll] showed tha t M, c MC, c MfC an d tha t Mfc isthe convex hull of MC, where MC, is th e set of ma tr ices obta ined by relaxing th e last condit ionof MC to Cj=i pij > 0 Vi.

Above we ha ve described th e so-called h ar d c-par tit ions an d fuzzy c-part itions, also th e equiv-alence relation and the similarity relation. These are main repr esentat ions in clust er a nalysissince cluster ana lysis is just to part ition a data set into c cluster s for which th ere is always anequivalence relat ion correspond ence to th e par tition. Next, we sha ll focus on the fuzzy clust eringmethods based on the fuzzy relation. These can be found in (10,12-161.

Fu zzy clust ering, based on fuzzy relations, was first proposed by Tam ur a et al. [12]. Theyproposed an N-step procedure by usin g th e composition of fuzzy relations beginning with areflexive and symmetric fuzzy relation R in X. They showed that there is an n such thatI < R 5 R2 5 . . . < Rn = Rn+l = . . . = R when X is a finite data set. Then R is usedto define an equivalence relation R x by the rule Rx(z, y) = 1 w X 5 R(z, y). In fact, R is a similarity relation. Consequent ly, we can par tition the dat a set into some clust ers by theequivalence relation R x. For 0 _< & 2 . . < X2 5 X1 5 1, one can obtain a corr esponding k-levelhierarchy of clusters, Di = {equiva lence clust er s of R x, in X}, i = 1 ,. . . , k where Di refines Djfor i < j. This h ierar chy of cluster s is just a single linkage hierar chy which is a well-kn own ha rdclust ering m eth od. Dun n [14] proposed a met hod of compu ting R based on Pr ims minimalspann ing t ree algorithm . Here, we give a simple example to explain it.EXAMPLE. Let

IR= o4 0.8 0.6 1 I0.3 0 0 1Therefore,

x = 0.29 ===+ 1,2,3,4},X = 0.59 + {1,2,3) u {4},x = 0.7 9 * {1,3 } u (2) u {4},x = 0.81 ===+1) u (2) u (3) u (4).

3. FUZZY CLUSTE RING BASE D ON OBJ ECT IVE FUNCTIONSIn Section 2, we int roduced fuzzy clust ering based on fuzzy relat ion. This is one type of fuzzy

cluster ing. But th ese meth ods ar e eventu ally the novel meth ods a nd nothing more than th esingle linkage hiera rchy of (har d) agglomer at ive h ierar chical clust ering met hods. Nothing comesout fuzzy th ere. This is why th ere is not so much research an d very few results in this t ype offuzzy clust ering. If we ad d th e concept of fuzzy c-part itions in clust ering, th e situ at ion sha ll betota lly cha nged. Fu zzy char at erist ics could be repr esent ed by th ese fuzzy c-par titions. In th issection, we give a sur vey of th is kind of fuzzy cluster ing.

Let a dista nce (or dissimilarity) function d be given, where d is defined on a finite data setx = {Xi,... ,Z,} byd:XxX+IW+ with d(z:, z) = 0 and d(z, y) = d(y, cc) for all 2, y E X. Let


4/16

4 M.-S. YANG

dij = d (xi, Zj) and D = [dij] E Vn,. Ruspini [6] first proposed the fuzzy clustering baaed on anobjective function of J&U) to produce an optimal fuzzy c-partition U in Mfcn, where

with d E R. He derived the necessary conditions for a minimizer U* of JR(U). Ruspini [17,18]continued making more numerical experiments. Since JR is quite complicated and difficult tointerpret, the Ruspini method was not so successful but he actually opened the door for furtherresearch, especially since he first put the idea of fuzzy c-partitions in cluster analysis. Notice thatthe only given data in Ruspinis objective function JR(U) is the dissimilarity matrix D, but notthe data set X even though the element of D could be obtained from the data set X.

Based on the given dissimilarity matrix D, Roubens [19] proposed the objective function JROof which the quality index of the clustering is the weighted sum of the total dissimilarity, where

Even though Roubens objective function JRO is much more intuitive and simple than RuspinisJR, the proposed numerical algorithm to optimize JRO is unstable. Libert and Roubens (201 gavea modification of it, but unless the substructure of D is quite distinct, useful results may not beobtained.

To overcome these problems in JRO, Windham [21] proposed a modified objective functionJ AP n n c

JAP(U,B) = ~~~(P;jb;k)d.jk,j=l k=l i=l

where B = [bik] E V,, with bik E [0, l] and CL=1 bik = 1. Such biks are called prototype weights.Windham [21] presented the necessary conditions for minimizers U* and B* of J~p(u, B) andalso discussed numerical convergence, its convergence rate, and computer storage. The Windhamprocedure is called the assignment-prototype (AP) algorithm.

Hathaway et al. [22] and Bezdek et al. (231 gave a new type of objective function JRFCM, calledthe relational fuzzy c-mean (RFCM), where

c (c,,,c;=,P$Jzdjk)JRFCM(~) = c

i= l 2 (CI"=lPid"

with m E [l, co). They gave an iterative algorithm by using a variation of the coordinate decentmethod described in [24]. The objective function JRFCM seems to be hardly interpreted. Infact, it is just a simple variation of the well-known fuzzy clustering algorithms, called the fuzzyc-means (FCM). The FCM becomes the major part of fuzzy clustering. We will give a big part ofsurvey next. The numerical comparison between AP and RFCM algorithms was reported in [25].

The dissimilarity data matrix D is usually obtained indirectly and with difficultly. In general,the only given data may be the finite data set X = {xl,. . . , z,}. For a given data set X ={Xl,..., zn }, Dunn [9] first gave a fuzzy generalization of the conventional (hard) c-means (morepopularly known as k-means) by combining the idea of Ruspinis fuzzy c-partitions. He gave theobjective function JD (U, a), where


5/16

A Survey of Fuzzy Clustering 5with U E Mfcn and a = (al,. . . , a,) E (W) called the clust er center s. Bezdek [7] genera lizedJ D(U, a) to J FCM (U, a) wit h

for which m E (1, co) repr esent s th e degree of fuzziness. The necessar y condit ions for a minimizer(U, a*) of JFCM(~, a ) a re

The itera tive algorithm s for compu tin g minim izers of Jpc~(tY, 3) with these necessary conditionsar e called th e fuzzy c-mean s (FCM) clust ering algorith ms. These FCM clustering proceduresdefined by Dun n [9] an d genera lized by Bezdek [7] ha ve been widely st udied an d applied in avariety of substa ntive area s. There a re so man y papers in the literat ure t ha t concern them selveswith some a spects of th e theory an d applicat ions of FCM. We give some references by the followingcategories. Int erested reader s may refer some of these:

(4(b)Cc)(4Numerical Theorems 126-37)Stochastic Theorems [34,38-411Methodologies [9,37,42-511Applications(1) Im age Pr ocessin g [45,52-581(2) Engineering Systems [59-631(3) Parameter estimation [38,64-681(4) Others [69-74).Her e we ar e int erest ed in describing a variet y of genera lizations of FCM. These include six

generalizations an d two variations.(A) GENERALIZATION1

To add th e effect of differen t cluster sha pes, Gust afson an d Kessel [47] genera lized th e FCMobjective function with fuzzy covaria nce mat rices, wher e

j=l i=l

with U E Mfc n , a = (al ,. . . ,a,) E (R ) and A = (A l,. . . ,A,) for which A i are positivedefinit e s x s ma tr ices with det (Ai) = pi being fixed an d ]]zj - a il], = (zj - oi)T Ai (zj - ai) .The it era tive a lgorit hm s use necessar y condit ions for a minim izer (U,a, A) of J FCM~ (U, a, A) asfollows:

ai =

Pij =

Ai = (p i det(Si))SZ:i,i=l, . . . ,c,j= l,...,n.

wh er e Si = Cj=, jog (zj - ui) (zj - a i).


6/16

6 M.-S. YANG(B) GENERALIZATION 2

JFCM (U, a) is well used to cluster the data set which has the hyper-spherical shapes. Gustafsonand Kessel [47]generalized it to J FCM,, (U, 3, A) for improving the ability to detect different clustershapes (especially in different hyper-ellipsoidal shapes) in the given data set. Another attemptto improve the ability of JFCM(U, a) to detect nonhyper-ellipsoidally shaped substructures wasproposed by Bezdek et al. [42,43], called the fuzzy c-varieties (FCV) clustering algorithms.

Let us denote the linear variety of dimension r(0 2 T < s) in W through the point a andspanned by the vectors {br, . . . , b,} by

Vr(%bl,..., b,) = 1yEW,y=a+&, &ElR .

k=l 1If r = 0, Vo = {u} is a point in W; if T = 1, VI = L(a; b) = {y E IIg 1y = a + tb, t E R} is a linein W; if T = 2, Vz is a plane in W; if r > 3, V, is a hyperplane in W; and V, = W8. In general,the orthogonal distance (in the A-norm on Ws for which A is a positive definite s x s matrix)from z E RS to V,, where the {bi} is an orthonormal basis for VT , is

112DA(x,Vr)=~~{d~(x,y)}= llX-ul~2,-~((x-a,bk)A)2

k=lwhere (z, y)A = xTAy, l]x]]: = (&x)A and

dA(Z,g) = /Ix-Y\lA = ((~-~)*+y))~.Let a = (al,.. ., a,) and bk = (blk,. . . , b&), k = 1,. . , , T. Consider the objective functionJFCV(~, a, by , . . . , b,) which is the total weighted sum of square orthogonal errors between thedata set X = (~1,. . . , xn} and c r-dimensional linear varieties, where

JFCV (u,a, bl ...) b,) = f--j$lr;Dfi,ithj=l i=rD, = DA (j , v,i) ,

T i = l,...,C,yE~]y=oi+~tkb& tkEll% , j = l,... ,n, and

k =l m E [l, oo).The FCV clustering algorithms are iterations through the necessary conditions for minimizers ofJFCV &S ollows:

where sik is the kth unit eigenvector of the fuzzy scatter matrix Si with

(71

Si = Al c p;(q - ai ) (xj - ai )T A2j= l

corresponding to its k th largest eigenvalue, and2 (Dij)2/+1) i=l,...,c,

Pij = k=i (Dkj)2(m-1) j= l,...,n, andk = l,...,r.


7/16

A Surveyof Fuzzy Clusterin g 7(C) GENERALIZATION3

Consider an y probability distr ibution function G on W. Let X = (21,. . . , z,} be a randomsample of six n from G and let G, be the empirical distribution which put s mass l/n at eachan d every point in the sa mple { 21, . . . , z,}. Then the FCM clustering procedures ar e to choosefuzzy c-partitions ~1= (~1,. . . , pc) an d clust er center a to minimize th e objective function

~GFCM('&,~~)= +dh)

Let us repla ce G, by G. Then we genera lized th e FCM wit h res pect to G by th e minimizationof JGFCM (G,cl,a),

JGFCM(G,P,~) = s 2 ~,zmb$ -ai(12G&).i=lConsider a par ticular fuzzy c-par tit ion of a in (lR)c of th e form

i2 Ilz _ ui(121(m-l)-ldz,a) = j=r 115 flj //2(m-1) 1 i=l,...,c.

Define ~(a) = (/J I(z, a), . . . , pc(z, 3)) and define that&(G,a) = JGFCM(G,CL(;L),~~)

Then , for an y fuzzy c-par tition p= (PI(Z), . . . , p,.(z)) and a E (R~)=,JGFCA&,P(~L),~) 5 ~GFCM(G,I_~,;~).

The reason for th e last inequality is tha t

which can be seen from the simple fact that-772c c yyJ4

+l/Cm-l)i=l j=l Yj ) Yi I 2 PT Yii=l

for pk >_ 0, Yk > 0, k = 1,. . . , C, pl + *. . + p, = 1. To claim the simple fact, we may take thegradient of the Lagrangian

and also check that the Hessian matrix of h with respect to pl, . . . ,p, will be positive definite.


8/16

8 M.-S. YANGLet (/A*, ) be a minimizer of JGFCM(G, p, a) and let b* be a minimizer of L,(G, a). Then,

JGFCM (G,p*,a*) L JGFCM (G~(h*),b*)= L, (G,b*)I Lx (G,a*)= JGFCM (G,p(a*),a*)< JGFCM(G,P*,~*).

That is, JGFCM (G,p*, a*) = L, (G,h*). Therefore, we have that solving the minimizationproblem of JGFCM (G, p, a) over p and 3 is equivalent to solving the minimization problemof L, (G, a) over a with the specified fuzzy c-partitions ~(a). Based on this reduced objectivefunction L, (G, a), Yang et aZ. [39-411 created the existence and stochastic convergence propertiesof the FCM clustering procedures. They derived the iterative algorithms for computing theminimizer of L, (G, a) as follows:Set h = (arc,. . . , a,e). For k 2 1, let ak = (ark,. . . , a&), where for i = 1,. . . , c,

They extended FCM algorithms with respect to G, to FCM algorithms with respect to anyprobability distribution function G, and also showed that the optimal cluster centers should bethe fixed points of these generalized FCM algorithms. The numerical convergence properties ofthese generalized FCM algorithms had been created in [36]. These include the global convergence,the local convergence, and its rate of convergence.(D) GENERALIZATION 4

Boundary detection is an important step in pattern recognition and image processing. Thereare various methods for detecting line boundaries; see [75]. When the boundaries are curved,the detection becomes more complicated and difficult. In this case, the Hough transform (HT)(see [76,77]) is th e only major technique available. Although the HT is a powerful technique fordetecting curve boundaries like circles and ellipses, it requires large computer storage and highcomputing time. The fuzzy c-shells (FCS) clustering algorithms, proposed by Dave [45,46], arefine new techniques for detecting curve boundaries, especially circular and elliptical. These FCSalgorithms have major advantages over the HT in the areas of computer storage requirementsand computational requirements.

The FCS clustering algorithms are used to choose a fuzzy c-partition p = (~1,. . . , pc) andcluster centers a = (al,. . . ,a,) and radius T = (~1,. . . , rc) to minimize the objective functionJFCS (p, a, T), which is the weighted sum of squared distance of data from a hyper-ellipsoidalshell prototype

J FCS (P,W) = &?(lj) (11%a&, - ri)2j=li=lamong all a E (IV), /.LE W, and T E (lR+)c with m E [l, co). The necessary conditions for a


9/16

A Survey f Fu zzyC l u s t e r i n gminimizer ($, a*, r*) of JFCS are the following equations satisfied for a* and r*:

1 -

with pf(zj) =

The FCS clustering algorithms are iterations through these necessary conditions. Dave et al.(see [45,53]) applied these FCSs in digital image processing. Bezdek and Hathaway [78] inves-tigated its numerical convergence properties. Krishnapuram et al. [79) gave a new approach tothe FCS.(E) GENE RALIZATION 5

Trauwaert et al. [51] derived a generalization of the FCM based on the maximum likelihoodprinciple for the c members of mutivariate normal distributions. Consider a log likelihood functionJFML (p, a, IV) of a random sample X = {XI,. . . , 5%) from c numbers of mutivariate normalsN(ai,Wi), i = 1,. . . ,c, where

in which ml, m2, ma E [l, 00). To derive iterative algorithms for a maximizer of JFML (p, a, IV),they further limited to the case where ml, m2, and ms are either 1 or 2. They claimed thatonly the case of ml = 2 and m2 = m3 = 1 appeared as an acceptable basis for a membershiprecognition algorithm. In the case of ml = 2 and m2 = ms = 1, Trauwaert et al. [51] gave theiterative algorithm as follows:

whereBij (xj - .i)TW;l(xj - ai), Aij = i In ]Wi], i=l ,**.,c,j=l ,...,n.

This general algorithm is called fuzzy product since it is an optimization of the product offuzzy determinants. In the case of Ai = I for all i the algorithm becomes the FCM.(F) GENE RALIZATION 6

Mixtures of distributions have been used extensively as models in a wide variety of importantpractical situations where data can be viewed as arising from two or more subpopulations mixed invaried proportions. Classification maximum likelihood (CML) procedure is a remarkable mixturemaximum likelihood approach to clustering, see [80]. Let the population of interest be known or beassumed to consist of c different subpopulations, or classes, where the density of an observation xfrom class i is fi(x; e), i = 1,. . , , c for some parameter 6. Let the proportion of individuals in


10/16

1 0 M.-S. YANGth e popula tion which a re from class i be denoted by oi with cri E (0,l) an d x:=,1 oi = 1. Letx = (51,. . . , z,} be a ra ndom sa mple of size n from the population. The CML pr ocedure is th eoptimization problem by choosing a ha rd c-part ition J o* an d a proportion cy* an d an estim at e 6*to m axim ize th e log likelihood I31 (CL,Y,0) of X, wh er e

B1(p,a,e) = eC lnwfi (xj;e)71. c

= CCpi(q)lnaifi (xj;e),j=l k=l

in which ,Q(z) E ( 0 , 1 ) an d pi(~) + ... + ~~(2) = 1 for all 2 E X. Yang [37] gave a ne w t ype ofobjective fun ction Bm +, (11,Q, 0), wher e

in which pi(z) E [0, l] and pi(z) + . . . + &x) = 1 for all z E X with th e fixed const an tsm E [l,co) an d w 2 0. Thu s, B,,, (/J , a, e), called t he fuzzy CML objective fun ction, is a fuzzyext en sion of Bi (cl, (Y,0). Ba sed on B,,, (P, (Y,e), we can der ive a variet y of ha rd an d fuzzyclustering algorithms.

Now we consider th e fuzzy CML objective function B,,, (p, Q, 0) with th e mu ltivaria te norma lsubpopula tion densit ies N (ai, Wi), i = 1,. . . , c. Then, we have that

Based on B m,ul(p, a, IV), we exten d th e fuzzy clust ering algorithm s of Tra uwa ert et al. [51] to amore genera l setting by adding a penalty term ( w C,=, xi=, pyn (zj) In oi . The construction)of th ese penalized types of fuzzy clust ering algorithm s is similar to tha t of [51]. In th e specialcaseofWi=I,i=l,...,c. Wehavethat

= J FCM(~ a) - W 2 2 Py(Xj) In%.j=l i=l

Since JPFCM just adds the penalty term (-w & xk, ,LL; (zj) In ai) to JFCM, J PFCM iscalled the pena lized FCM objective function. We get the following n ecessary conditions for a


11/16

A Survey of Fuzzy Clusteringminimizer (p, a, a) of J PF CM:

a j =

CYi =

Pi(q) = )

11

Thu s, th e penalized F CM clust ering algorithm s for comput ing th e minimizer of J PF CM (p, a, CX)are iterat ions t hrough th ese necessar y conditions. By comparison between the penalized FCMand the FCM, th e difference is the penalty t erm. This pena lized F CM h as been used in para meterestimat ion of the norma l mixtures. The numer ical comparison between the penalized FCM andEM algorithm s for para meter estimat ion of the norma l mixtures ha s been studied in [68].(G) OTHER VARIATIONS

From (A) to (F), we give a sur vey of six genera lizat ions of th e FCM. Th ese gener alizations ar eall based on th e E uclidean norm )I . )I ( i.e., Lz-norm ). I n fact, we can ext end th ese t o L,-norm

JFCM, (uya) 22 CL;&

where dij = (chsl (xjk - aik I) is th e L,-norm of (zj - ai). When p = 2 it is just the FCM.In Math emat ics and its applications, ther e are two extreme L,-norms most concerned. These areLr-norm and L,-norm. Thus,

JFCM~(U,~,) ce~I;"dij with dij = f: (Sjk - a&[ andj=l i=l k=ln c

J FCM, (V 8) = C C p c d ijj=1 i=l

wi th d i j = k z y s ) s j k - a ik 1 .I 1We cann ot find th e necessar y conditions to guide num erical solut ions of th ese nonlinear con-str ained objective functions JFCM~ and JFCM,. Bobrowski an d Bezdek [44] gave a met hod tofind app roximat e minim izers of JFCM~ and JFCM, by using a basis exchange algorithm usedfor discrimina te an alysis with th e perception criterion function by Bobrowski an d Niemiro [Bl].J ajuga [49] also gave a met hod to find approximat e minim izers of JFCM~ based on a meth od forregres sion an alysis with t he Li-norm by Bloomfield an d Steiger 1821. J ajuga [49] gave anoth ersimple method to find approximate minimizers of J FCM, which we shall describe next.

Recall thatJFCM~W,~ ~2 k~;dij with dij = 2 )zjcjk -f&k).j=l i=l k= l

Then U may be a minimizer of JFCM~ for a being fixed only if

Pij =2 (dij)l+-)k=l (dkj) (+)

For U being fixed, let wij = ,zjkP faik,. Then ,c s

J FCM~ (w , a > = c c 2 ij (zjk - aik).j= l=l k=l

Mm 16:11-B


12/16

12 M.-S. YANG

The aik may be a minimizer of J FCM~ w, a) for WQ being fixed only ifaii = c,=lwiixik i= l,...,c,

C&i ij k= l;.. ,s.Jajuga [49] suggested an algorithm for finding approximate minimizers of JFCM~ based on thefollowing steps:

Pgwij= Xik - aikla,k = C,=Iiixik

z EYE=,i.j and2 (&)1-l) i=l,*..,c,

Pij = (dkj) llCna-l) dij =~iZjk--aailc1, where j = l,...,n,k=r k=lk=l,...,s.

4. FUZZY GENERAL k-NEAREST NEIGHBOR RULEIn the conventional nearest neighbor pattern classification problem, X represents the set of

patterns and 0 represents the labeling variable of c categories. We assume that a set of ncorrectly classified samples (ICI, &), (x2,6$) , . . . , (x,, 19,) is given and follows some distributionF(z, Q), where the 5:s take values in a metric space (X,d) and the 0:s take values in the set{1,2 ,..., c}. A new pair (Q,&) is given, where only the measurement ~0 is observable bythe statistician, and it is desired to estimate BO by utilizing the information contained in theset of correctly classified points. We shall call Z; E {xl, ~2,. . . ,z,} a nearest neighbor to zif d(zL,z) = m in d,2 ,,.., n d(xi, x). The nearest neighbor rule (NNR) has been a well-knowndecision rule and perhaps the simplest nonparametric decision rule. The NNR, first proposed byFix and Hodges [83], assigns x to the category 0; of its nearest neighbor &.

Cover and Hart [84] proved that the large sample NN risk R is bounded above by twice the Bayesprobability of error R * under the 0- 1 loss function L, where L counts an error whenever a mistakein classification is made. Cover [85] extended the finite-action (classification) problem of Coverand Hart [84] to the infinite-action (estimation) problem. He proved that the NN risk R is stillbounded above by twice the Bayes risk for both metric and squared-error loss function. Moreover,Wagner [86] considered the posterior probability of error L, = P [e; # eo ) (21, el), . . . , (xn, On )].Here L, is a random variable that is a function of the observed samples and, furthermore,EL,=P[~:,#~], 0 and he proved that under some conditions, L, converges to R with probabilityone as n tends to co. This obviously extended the results of Cover and Hart [84]. There are somany papers about k-NNR in the literature. The k-NNR is a supervised approach to patternrecognition. Although k-NNR is not a clustering algorithm, it is always used to cluster anunlabelled data set through the rule. In fact, Wong and Lane [87] combined clustering withk-NNR and developed the so-called k-NN clustering procedure.

In 1982, Duin [88] first proposed strategies and examples in the use of continuous labelingvariable. Joiwik [89] derived a fuzzy k-NNR in use of Duins idea and fuzzy sets. Bezdek etal. (901 generalized k-NNR in conjunction with the FCM clustering. Keller et al. [91] proposedfuzzy k-NNR which assigns class membership to a sample vector rather than assigning the vectorto a particular class. Moreover, BQreau and Dubuisson [92] gave a fuzzy extended k-NNR bychoosing a particular exponential weight function. But in a sequence of study in the fuzzyversion of k-NNR, there is no result about convergence properties. Recently, Yang and Chen [93]described a fuzzy generalized k-NN algorithm which is a unified approach to a variety of fuzzy


13/16

A Survey of Fuzzy Clustering 13

k-NNRs and then created the stochastic convergence property of the fuzzy generalized NNR,that is, the strong consistency of posterior risk of the fuzzy generalized NNR. Moreover, Yangand Chen [94] gave its rate of convergence. They showed that the rate of convergence of posteriorrisk of the fuzzy generalized NNR is exponentially fast. Since the conventional Ic-NNR is a specialcase of the fuzzy generalized Ic-NNR, the strong consistency in [93] extends the results of Coverand Hart [84], Cover [85], and Wagner [86].

The fuzzy generalized Ic-NNR had been shown that it could improve the performance rate inmost instances. A lot of researchers gave numerical examples and their applications. The resultsin [93,94] just give us the theoretic foundation of the fuzzy generalized Ic-NNR. These also confirmthe value of their applications.

5. CONCLUSIONSWe have already reviewed numerous fuzzy clustering algorithms. But it is necessary to pre-

assume the number c of clusters for all these algorithms. In general, the number c should beunknown. Therefore, the method to find optimal c is very important. This kind of problem isusually called cluster validity.If we use the objective function JFCM (or JFCV, JFC,S, etc.) as a criterion for cluster validity,it is clear that JFCM must decrease monotonically with c increasing. So, in some sense, if thesample of size n is really grouped into E compact, well-separated clusters, one would expect tosee JFCM decrease rapidly until c = 6, but it shall decrease much more slowly thereafter until itreaches zero at c = n. This argument has been advanced for hierarchical clustering procedures.But it is not suitable for clustering procedures based on the objective function. An effectivelyformal way to proceed is to devise some validity criteria such as a cluster separation measure or touse other techniques such as bootstrap technique. We do not give a survey here. But interestedreaders may be directed towards references [95-1031.

Finally, we conclude that the fuzzy clustering algorithms have obtained great success in avariety of substantive areas. Our survey may give a good extensive view of researchers whohave concerns with applications in cluster analysis, or even encourage readers in the appliedmathematics community to try to use these techniques of fuzzy clustering.

REFERENCES1.2.3.4.5.6.7.a.9.

10.11.12.13.14.15.

R.O. Duda and P.E. Hart , Pattern Classification and Scene Analysis, Wiley, New York , (1973).P.A. Devijver a nd J. Kitttler, Pat tern Recognition: A Statistical Approach, Prentice-Hall, Englewood Cliffs,NJ, (1982).A.K. Jain an d R.C. Dubes, Algorithm s for Cluster ing Da ta , Pr ent ice-Hall, En glewood Cliffs, NJ , (1988).L.A. Zadeh, F uzzy sets, Inform. an d Contr ol 8, 338-353 (1965).R. Bellman , R. Kalaba, and L.A. Zadeh, Abstraction and patt ern classification, I. Math . Anal. and Appl.2, 581-586 (1966).E.H. Ruspini, A new app roach to cluster ing, Inform . an d Contr ol 15, 22-32 (1969).J.C. Bezdek, Pattern Recognition with Fuzz y Objective finction Algorithms, Plenum Press, New York,(1981).SK. P al and D.D. Majumdar , fizty Mathema tical Approach to Pattern. Recognition, Wiley (Halsted), NewYork, (1986).J.C. Du nn, A fuzzy relative of the ISODATA process and its use in detecting compact, well-separa tedclusters, J. Cybern etics 3, 32-57 (1974).L.A. Zadeh , Similar itiy relat ions and fuzzy order ings, Inform . Science, 177-200 (1971).J.C. Bezdek an d J.D. Harris, Convex decompositions of fuzzy part itions, J. Math. Anal. and Appl. 67 (2),490-512 (1980).S . Tamur a, S. Higuchi and K. Tanaka , P att ern claasificaiton based on fuzzy relations, IEEE nuns. S yst.Man Cybern . 1 (l), 61-66 (1971).A. Kan del an d L. Yelowitz, Fu zzy chains, IEEE Trans. Syst. Ma n Cybern., 472-475 (1974).J.C. Dunn , A graph t heoretic ana lysis for patt ern classification via Tamur as fuzzy relation, IEEE tins.Syst. Man Cybern. 4 (3), 310-313 (1974).J.C. Bezdek and J.D. Harris, Fuzzy relations an d part itions: An axiomat ic basis for clusterin g, tizzy Setsand Systems 1, 111-127 (1978).


14/16

14 M.-S. YANG

16. B.N. Chatt erji, Char acter recognition using fuzzy similarity relations, In Approximate Reasoning in DecisionAnalysis, (Edited by M.M. Gupt a a nd E. San chez), pp. 131-137, Amst erda m, New York , (1982).

17. E.H. Ruspin i, Nu mer ical met hods for fuzzy cluster ing, I nforma tion Science 2, 319-350 (1970).18. E .H. Ruspin i, N ew experim ent al resu lts in fuzzy cluster ing, Znjonn. Sciences 6, 273-284 (1973).19. M. Roubens, Pa tt ern classificat ion problems with fuzzy sets, fizzy Sets and Systems 1, 239-253 (1978).20. G. Libert and M. Roubens, Non-metric fuzzy clusterin g a lgorithm s and their cluster validity, In Fuzzy

Informat ion and Decision Processes, (Edited by M. Gupt a an d E. San chez), pp. 417-425, Elsevier, NewYork, ( 1982).21. M.P. Windham, Num erical classification of proximity data with assignment measur es, J. Classification 2,

157-172 (1985).22. R.J. H ath away, J .W. Davenport and J.C. Bezdek, Relational duals of the c-mean s algorithm s, Pat tern

Recognition 22, 205-212 (1989).23. J.C. Bezdek and R.J. Hathawa y, Dual object-relation cluster ing models, Znt. J. General Systems 16, 385-396

(1990).24. J .C. Bezdek, R.J. Hat ha way, R.E. Howard, C.E. Wilson a nd M.P. Windha m, Local conver gence an alysis of

a group ed varia ble version of coordina te descent , J . Optim ization Theory an d Applicat ions 54 (3), 471477(1986).

25. J.C. Bezdek, R.J. Hat haway an d M.P. Windham , Num erical comparison of the RFCM and AP algorithmsfor cluster ing relational da ta, Pattenz Recognition 24 (8), 783-791 (1991).

26. J .C. Bezdek, A convergen ce t heorem for th e fuzzy ISODATA cluster ing a lgorithm s, IEEE Zkansactions onPattern Analysis and Machine Intelligence 2 (l), l-8 (1980).27. J .C. Bezdek, R.J. Hat ha way, M.J . Sabin an d W.T. Tucker , Conver gence t heory for fuzzy c-mea ns: Coun-terexamples and repairs, IEEE Bans. Syst. Man Cybern. 17 (5), 873-877 (1987).

28. R. Cann on, J. Dave and J.C. Bezdek, Efficient im plementat ion of the fuzzy c-means cluster ing algorithm s,IEEE Tkuns. Pattern Anal. machine Zntell. 8 (2), 248-255 (1986).

29. R.J. Ha tha way and J .C. Bezdek, Local convergence of the fuzzy c-means a lgorithm s, Pattern Recognition19, 477-480 (1986).

30. M.A. Isma il and S.A Selim, On th e local optim ality of th e fuzzy ISODATA cluster ing algorith m, IEEEnuns . Pattern Anal. Machine Zntell. 8 (2), 284-288 (1986).

31. M.A. I sma il an d S.A Selim, Fuzzy c-mea ns: Optim ality of solutions an d effective ter mina tion of th e algari thm, Pattern Recognition 19 (6), 481-485 (1986).

32. MS Kam el and S.Z. Selim, A th resh olded fuzzy c-mea ns algorith m for semi-fuzzy clust erin g, PatternRecognition 24 (9), 825-833 (1991).

33. T. Kim, J.C. Bezdek an d R. Hat haway, Optimality test for fixed points of the fuzzy c-means algorith m,Pattern Recognit ion 21 (6), 651-663 (1988).34. M.J. Sabin, Convergen ce and consist ency of fuzzy c-mea ns/ISODATA algorith ms, IEEE Trans. PatternAnal. Machine Zntell. 9, 661-668 (1987).

35. S.Z. Selim and M.S. Kamel, On the mat hemat ical and numer ical p roperties of the fuzzy c-means algorith m,Fu zzy Sets an d System s 49, 181-191 (1992).

36. M.S. Yang, Convergence properties of the generalized fuzzy c-mean s clustering algorithm s, Computers Math.Applic. 25 (12), 3-11 (1993).

37. M.S. Yan g, On a class of fuzzy classification ma ximum likelihood p rocedur es, &%zy Sets and Systems 57(3), 365-375 (1993).

38. R.J. Hath away and J.C. Bezdek, On the asymptotic properties of fuzzy c-mean s cluster p rototypes ssestimators of mixtur e subpopulations center s, Commzlm. Statist.-Theory and Method 15 (2), 505-513(1986).39. M.S. Yang and K.F. Yu, On stochastic convergence theorems for the fuzzy c-means cluster ing procedure,Znt. J. General Systems 16, 397411 (1990).

40. M.S. Yan g an d K.F. Yu, On existen ce a nd str ong consist ency of a class of fuzzy c-mea ns cluster ing pr ocedur es,Cybernetics an d System s 23, 583-602 (1992).41. M.S. Yang, On asym ptotic norm ality of a class of fuzzy c-mea ns clust ering procedur es, (submit ted).42. J.C. Bezdek, C. Coray, R. Gunder son and J. Watson, Detection and characterization of cluster substru cture:

I. Linear str ucture: Fuzzy c-lines, SIAM J. Appl. Math. 40 (2), 339-357 (1981).43. J.C. Bezdek, C. Coray, R. Gunder son and J. Watson, Detection and characterization of cluster substru cture:

II. Fuzzy c-varieties and convex combinat ions th ereof, SIAM J. Appl. Math. 40 (2), 358-372 (1981).44. L. Bobrowski an d J.C. Bezdek, C-mean s with 11 an d 1 . norms, IEEE Tkzns. Syst. Man Cybenz. 21 (3),

545-554 (1991).45. N.R. Dave, Fu zzy shell-cluster ing an d applicat ions to circle detect ion in digital images, Int. J. General

Systems 16 (4), 343-355 (1990).46. R.N. Dave, Gener alized fuzzy c-shells clust erin g an d detect ion of circular an d elliptical boun dar ies, Pattern

Recognition 25 (i), 713-721 (1992).47. R.W. Gu nderson, An adapt ive FCV cluster ing algorithm , Int. J. Man A4achine Studies 19 (l), 97-104

(1983).48. D. Gustafson and W. Kessel, Fuzzy cluster ing with a fuzzy covarian ce m atr ix, Proc. IEEE, San Diego, pp.

761-766, (1978).49. K. J ajuga , &-norm based fuzzy cluster ing, Fzlzzy Sets and System 39, 43-50 (1991).


15/16

A Survey of Fuzzy Clustering 15

50.51.52.53.54.

55.56.57.58.59.60.61.62.63.64.65.66.67.68.69.70.71.72.73.74.75.76.77.78.79.80.81.82.83.84.85. T.M. Cover, Estimat ion by the near est neighbor r ule, IEEE tins. Inform. Theory 14, 50-55 (1968).

W. Pedr ycz, Algorit hm s of fuzzy cluster ing wit h par tial super vision, Pat t. &cog. Letter s 3, 13-20 (1985).E. ?kauwaert, L. Kaufman and P. Rousseeuw, Fuzzy cluster ing algorithm s based on the maximum likelihoodprinciple, tizzy Sets and Systems 43, 213-227 (1991).R. Cannon, J. Dave, J.C. Bezdek and M. l%edi, Segmenta tion of a them atic map per image using the fuzzyc-means clustering algorithm, IEEE tins. Ceo. d Remote Sensing 24 (3), 400-408 (1986).R.N. Dave and K. Bha swan , Adap tive fuzzy c-shells clust erin g a nd detect ion of ellipsea, IEEE Tt-ans. onNeural Networks 3 (5), 643-662 (1992).L.O. Hall, A.M. Ben said, L.P. Clark e, R.P. Velth uizen, MS. Silbiger and J.C. Bezdek, A compa rison ofneur al network an d fuzzy cluster ing techniques in segmenting magnet ic r esonance images of the brain,IEEE Trans. on Neural Net works 3 (5), 672-682 (1992).T.L. H unt sberger and M. Descalzi, Color edge detection, Pat tern Recog. Letters 3, 205-209 (1985).T.L. Hun tsberger, C.L. Ja cobs and R.L. Cann on, Itera tive fuzzy image segmentat ion, Pat tern &cog. 18(2), 131-138 (1985).J.M. Keller, D. Subha ngkasen, K. Unklesbay a nd N. Unkleabay, An approximat e reasoning t echnique forrecognition in color ima ges of beef steak s, Int . J. Genem l Systems 16, 331-342 (1990).M. Trivedi a nd J .C. Bezdek, Low level segmentat ion of aerial images with fuzzy cluster ing, IEEE Zinns.Syst. Man Cybem. 16 (4), 589-597 (1986).I. Anderson, J. Bezdek an d R. Dave, Polygonal shape descriptions of plane bounda ries, In Systems Scienceand Science, Vol. 1, pp. 295-301, SGSR Pr ess, Louisville, (1982).J.C. Bezdek and I. Anderson, Curvat ure and tangential deflection of discrete arcs, IEEE tins. PatternAnal. Machine Bell. 6 (l), 27-40 (1984).J.C. Bezdek an d I. Anderson, An application of the c-varieties clustering algorithm s to polygonal curvefitting, IEEE nans. Syst. Man Cybem. 15 (5), 637-641 (1985).K.P. Cha n and Y.S. Cheun g, Cluster ing of cluster , Pa tt ern Recognit ion 25, 211-217 (1992).T.L. Hu ntsberger and P. Ajjimaran gsee, Par allel self-organizing featur e maps for unsu pervised patt ernrecognition, Znt. J. General Systems 16, 357-372 (1990).J.C. Bezdek a nd J.C. Dunn , Optimal fuzzy part itions: A heur istic for estimating the param eters in a mixtur eof normal distributions, IEEE Ifnnsactions on Computers 24 (8), 835-838 (1975).J.C. Bezdek, R.J. Hathawa y an d V.J. H uggins, Param etric estimation for norma l mixtures, Pattern Recog-nition Letters 3, 79-84 (1985).J.W. Davenport, J.C. Bezdek and R.J. Hat haway, Par amet er estimation for finite mixtur e distributions,Computers Math. Applic. 15 (lo), 819-828 (1988).S.K. Pal and S. Mitra, Fuzzy dynamic cluster ing algorithm , Pattern Recognition Letter 11, 525-535 (1990).MS. Yang an d C.F. Su , On param eter estimation for the norma l mixtur es baaed on fuzzy clustering algo-rithms, (submitted).J.C. Bezdek, Nu merical taxonomy with fuzzy sets, J. Math. Bio. 1 (l), 57 -71 (1974).J.C. Bezdek an d P. Castelaz, Pr ototype classification and feature selection with fuzzy sets, IEEE Trans.Syst. Man Cybem. 7 (2), 87-92 (1977).J .C. Bezdek a nd W.A. Ford on, The applicat ion of fuzzy set th eory to medical dia gnosis, In Advances infizzy Set Theory and Applications, pp. 445-461, Nort h-Holland , Amster dam , (1979).G. Gra na th , Applicat ion of fuzzy cluster ing an d fuzzy classification to evalua te pr ovenan ce of glacial till, J .Math Geo. 16 (3), 283-301 (1984).A.B. Mcbra tn ey an d A.W. Moore, Applicat ion of fuzzy sets to clima tic classification, Agricultural and ForestMeteor 35, 165-185 (1985).C. Windha m, M.P. Windha m, B. Wyse and G. Ha nsen , Cluster a nalysis to impr ove food classification with incommodity groups, J . Amer. Diet. Assoc. 85 (lo), 1306-1315 (1985).D.H. Ballard and C.M. Brown, Computer Vision, Pr ent ice-Ha ll, En glewood Cliffs, NJ , (1982).J. Illingworth and J . Kittler, A sur vey of Hough tr ansforms, Computer Vision, Graphics and Image Process-ing, 87-l 16 (1988).E.R. Davies, Finding ellipses using the genera lized Hough tr ansform, Pattern Recognition Lett. 9, 87-96(1989).J.C. Bezdek a nd R.J. Hat haway, Num erical convergence and inter preta tion of the fuzzy c-shells clusteringalgorithm, IEEE 113-ans. on Neural Networks 3 (5), 787-793 (1992).R. Krishna pura m, 0. Na sraoui a nd H. kigui, The fuzzy c spherical shells algorith m: A new approach, IEEEtins on Neuml Networks 3 (5), 663-671 (1992).G.J. McLachlan an d K.E. Basfork, Mixture Models: Inference and Applicaitons to Clustering, MarcelDekker, NY, (1988).L. Bobrowski and W. Niemiro, A meth od of synthesis of linear discriminant function in the case of nonsep-arability, Pattern Recognit ion 17 (2), 205-210 (1984).P. Bloorn iield an d W.L. Steiger, Least Absolut e Deviat ions, Birkh liuser, Boston, MA, (1983).E. Fix and J.L. Hodges, Jr., Discriminat ory ana lysis, nonpar amet ric discriminat ion, consistency properties,Pr oject 21-49-004, Repor t No.4, USAF School of Aviation Medicine, Ra ndolph Field, TX, (1951).T.M. Cover an d P.E. Har t, Nearest n eighbor patt ern classification, IEEE tins. Jnfown. Theory 13, 21-27(1967).


16/16

16 hf.-S. YANG86. T.J. Wagner , Convergence of th e near est neighbor ru le, ZEEE Ibuns. Informat ion Theory 17, 566671

(1971).87. M.A. Wong an d T. Lan e, A kt h-near est neighbor clusterin g procedure, J. R. Statist. Sot. B 45 (3), 362-368

(1983).88. R.P.W. Duin , The u se of cont inu ous var iables for labelling objects, Patt. Recog. Letters 1, 15-20 (1982).89. A. J 6Swik, A lear ning scheme for a fuzzy k-NN rule, Pattern Recognition L&t. 1, 287-289 (1983).90. J .C. Bezdek, S. Chua h and D. Leep, Genera lized k-near est neighbor rules, Z%zzy Sets and Systems 18,237-256 (1986).91. J .M. Keller, MR.. Gr ay and J .A. Givens, A fuzzy k-near est neighbor algorithm, IEEE 2Yuns. Syst. Man

Cybem. 15 (4), 580-585 (1985).92. M. B&au and B. Dubuisson, A fuzzy exten ded k-nearest neighbors ru le, &r.sy Sets and Syst ems 44, 17-32

(1991).93. M.S. Yan g and C.T. Chen, On st rong consistency of the fuzzy genera lized nea rest neighbor rule, ZUzy Setsand Systems 80 (1993) (to appear).

94. MS. Yan g and C.T. Ch en, Convergence ra te of the fuzzy genera lized nea rest neighbor rule, ComputersMath. Applic. (to appear).

95. J .C. Bezdek, Cluster validity with fuzzy sets, J . Cybernetics 3, 58-73 (1974).96. R. Dubes a nd A.K. J ain, Validity stu dies in clusterin g meth odologies, Pattern Recognition 11, 235-254

(1979).97. A.K. J ain and J .V. Moreau, Bootstr ap t echnique in cluster analysis, Pattern Recognition 20 (5), 547-568

(1987).98. F.F. Rivera , E.L. Zapata an d J.M. Car azo, Clust er validity bssed on th e har d ten dency of the fuzzyclassification, Pattern Recognition Let ter8 11, 7-12 (1999).

99. M. Roubens, F uzzy clusterin g algorithms and th eir clust er validity, Eur. J. Oper. Research 10, 294-301(1982).

100. E. Tr au waert , On t he mean ing of Dunns pa rt ition coefficient for fuzzy clust er, Z%z.zy ets and Systems 25,217-242 (1988).

101. M.P. Windha m, Cluster validity for th e fuzzy c-mean s clust ering algorithm, IEEE tins. Pattern Anal.Machine Zntell. 4 (4), 357-363 (1982).

102. M.A. Wong, A bootst ra p t esting procedur e for investigat ing th e nu mber of subpopulat ions, .I. Statist.Comput. Simul. 22, 99-112 (1985).

103. X.L. Xie an d G. Beni, A validit y mea sur e for fuzzy clust erin g, IEEE Ibuns. Pattern Anal. Machine htell.13, 841-847 (1991).

A Survey of Fuzzy Clustering

Documents

Transcript of A Survey of Fuzzy Clustering