FCM CLUSTERING ALGORITHM FOR T-S FUZZY MODEL IDENTIFICATION

PU HAN, JIAN-ZHONG SHI, DONG-FENG WANG, SONG-MING JIAO

Department of Automation, North China Electric Power University, Baoding 071003, China E-MAIL: hp@necpubd.edu.cn, sjz-ha@163.com, wangdongfeng@ncepubd.edu.cn, jiaosongming@126.com

Abstract: An approach for building T-S fuzzy model is proposed

based on fuzzy c-mean clustering algorithm on the basis of nonlinear modeling experience. An alternative T-S fuzzy model is adapted, which has the uniformed premise structure, the premise parameter is decided by fuzzy c-mean clustering algorithm and the consequence parameters is calculated by least square algorithm, and the identification precision is enhanced. Finally the effectiveness and practicability of this method is demonstrated by the simulation result of Box-Jenkins gas furnace data and Mackey-Glass chaos time series.

Keywords: Fuzzy c-mean; T-S fuzzy model; Fuzzy identification;

Least square

1. Introduction

In the identification of nonlinear system, the T-S fuzzy model proposed by Takagi and Sugeno in 1985[1], which can approach the nonlinear function defined in the compact set with arbitrary precision, is gradually widely used.

Fuzzy clustering is an unsupervised learning algorithm, which does not need a priori knowledge, but according to certain rules and requirements to distinguish and classify[2]. The rules of T-S fuzzy model can be summed up as the fuzzy partition to the space product of input and output variables, and the fuzzy clustering, one of fuzzy division methods, has been widely used in recognition of T-S fuzzy model. Some scholar who apply fuzzy clustering algorithm to T-S fuzzy model, have achieved good results[3-5]. Fuzzy c-means algorithm, the most commonly used fuzzy clustering algorithm based on objective function, has the advantages of simple algorithm, fast convergence rate and can handle large data sets, so it is used in this paper to identify the premise parameters of T-S fuzzy model.

The premise structure of general T-S fuzzy model is got through a fuzzy partition to the premise variables respectively, and calculation of membership. The literature [4] presented a new kind of T-S fuzzy model, which makes the premise structure united into a vector structure, so it is

easy to use fuzzy clustering algorithm for the premise parameters identification. But the identification algorithms of consequent parameters in literature [4] are too complex. The least square algorithm is adapted to identify the consequent parameters of fuzzy model in this article, and makes the identification procedures simplified.

2. Alternative T-S fuzzy model

The T-S fuzzy model of this system can be described by the following if-then fuzzy rules:

iR : 1 1iif x is A and … i

r rand x is A

then 0 1 1i i

iy p p x= + + … ir rp x+ ,

and 1

1

ci

ii

ci

i

yy

μ

μ=

=

=�

� 1

( )r

i ij j

j

A xμ=

= ∏

where i=1,2,…,c , c is the number of fuzzy rule, xj(j=1,2,…,r)is the system input, yi is the i-th output,

ijA (j=1,2,…,r) is the fuzzy sets of input spatial and i

jp (j=1,2,…,r) is the consequent parameter of the i-th output.

The premise structure is related to the determination of the number of premise variables in each rule. As a matter of fact, even when some variables are not the premise variables of one rule, fuzzy subsets can be still assumed to them with the membership functions always being 1, which means all the rules has the same premise variables. The uniformed premise structure is advantaged and convenient for identification, analysis and computation of the fuzzy model. As mentioned in paper[4], the alternative fuzzy model can be modified as:

iR : if x is iA then 0 1 1

i i ii r ry p p x p x= + + +�

where x=[x1,x2,…,xr] is the input variables vector, Ai is the degree x belong to the i-th class, which can be decided by fuzzy c-means clustering algorithm. Thus the premise

5632010 IEEE978-1-4244-6527-9/10/$26.00 ©

Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

parameter can be calculated easily.

3. Fuzzy c mean clustering algorithm

Fuzzy c-means (FCM) clustering algorithms extended from hard c-means clustering algorithm, has been one of the most commonly used and discussed fuzzy clustering algorithm[6]. It described as follows:

Let X = {xk, k=1,2,…,n} denotes the training sample, X Rp, and c is the clustering number, then vi(i=1,2,…,c) is the clustering center of cluster i. And uik(i=1,2,…,c, k=1,2,…,n) represents the membership function of the item belongs to the cluster i, and it should satisfy the following constrains pattern k in cluster i, 0 ≤ uik ≤ 1 and

10

n

ikk

u n=

< <� .The objection function for FCM is shown

as Equation (1):

2

1 1( , )

c nm

m ik iki k

J u d= =

=��U v (1)

where22

ik k id = −x v , { }iku=U ,v=(v1,v2,…,vc), m is a constant and m>1, uik fill the following conditions:

11, 1, 2, ,

c

iki

u k n=

= ∀ =� �

10 , 1, ,

n

ikk

u n i c=

< < ∀ =� �

According to the Lagrange multiplier method, ∀ i=1,2,…,c, k=1,2,…,n, the necessary conditions for the objection function to reach its minimum are:

11

2

11

21

1( )

1( )

m

ikik c

m

j jk

du

d

−

−

=

=

� (2)

1

1

nmik k

ki n

mik

k

u

u

=

=

=�

�

xv (3)

4. T-S fuzzy model identification algorithm

The consequent parameters identification algorithms given by literature [4] are to complicate. The least-square algorithm is used to calculate the consequent parameters of

T-S fuzzy model here. According to the new fuzzy model given in chapter 2,

let:

1/

c

i i ii

ω ω ω=

= �

where ( )ii Aω = x .

Therefore, the output of alternative T-S fuzzy model is as follow:

1

c

i ii

y yω=

=�

0 1 1 2 21

( )c

i i i ii r r

ip p x p x p xω

=

= + + + +� �

1 1 1 1 1[ ]r c c c rx x x xω ω ω ω ω ω= � � � 1 1 10 1 0 1[ ]c c c T

r rp p p p p p× � � � By N pairs of input-output date set into the above

equation, the matrix equation can be described as follow: Y = XP (4)

In equation (4), P is the conclusion parameter vector whose dimension is L=(r+1)c, and Y, X are matrixes of N×1,N×L. Then the estimate of P with the least square method is:

* 1( )T T−=P X X X Y (5) The outline of identification algorithm is stated as

follows: Step1: Initial the clustering number c and parameter m

(usually is 2), and determine the input variables and training data of fuzzy model antecedent structure. Then initial the clustering center 0

iν , termination threshold ε >0, and the maximum iterations times Cyc. Let iterations variable l=1;

Step2: Calculate the l-th iterations liku by Equation (2),

and then calculate the l-th iterations liv by l

iku and Equation (3);

Step3: Check for termination in some convenient- induced matrix norms vl-vl-1 <ε , or l> Cyc ,go to Step 4,else l=l+1, go to step 2;

Step4: Calculate the consequent parameters of the fuzzy model by Equation (5).

5. Simulation research

5.1. Example 1

The Box–Jenkins data sets[7] are adapted as the simulation dates, which consists of 296 input–output

564

Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

measurements of a gas-furnace process. It is a SISO system. At each sampling time t, the input u(t) of this process is the gas flow rate and the output y(t) is the output carbon dioxide concentration.

u(t),u(t-1),u(t-2),y(t-1),y(t-2),y(t-3) are selected as the variables of the fuzzy model. The first 100 input–output data were taken as training data and the last 196 as test data, the rule number c=2.Figure 1 depicts the original and the predicted output of our model and the red solid line is

original outputs, Figure 2 draws the errors between original and the predicted output.

Table 1 shows in the same performance MSE (mean square error) between our method and other fuzzy modeling methods, the performance is

2

1

ˆ( ) /N

i i

iJ y y N

=

= −� .

TABLE 1 COMPARISON OF FUZZY IDENTIFICATION METHODS (BOX–JENKINS DATA)

model Input variables Rule number MSE Tong model[8] y(t-1), u(t-4) 19 0.469

Pedrycz model[8] y(t-1), u(t-4) 81 0.320 Xu model[8] y(t-1), u(t-4) 25 0.328 Joo model[9] y(t-1), u(t-4) 6 0.166

Chen model[4] y(t-1), u(t-4) 3 0.2678 Yoshinari model[10] y(t-1), u(t-3) 6 0.299

Sugeno model[8] y(t-1), u(t-3), u(t-4) 6 0.190 Box model[7] y(t-1), y(t-2), y(t-3), u(t), u(t-1), u(t-2) - 0.2020 Lo model[11] y(t-1), y(t-2), y(t-3), u(t), u(t-1), u(t-2) 2 0.062

Nozaki model[12] y(t-1), y(t-2), y(t-3), u(t), u(t-1), u(t-2) 2 0.0613 Our model y(t-1), u(t-4) 6 0.1461 Our model y(t-1), u(t-3), u(t-4) 6 0.1199 Our model y(t-1), y(t-2), y(t-3), u(t), u(t-1), u(t-2) 2 0.0607

0 100 200 30045

50

55

60

65

Time(Sample time 9s)

CO

2(%

)

Figure 1 our model the original and the predicted output (Box–Jenkins data)

0 100 200 300-2

-1

0

1

Time(Sample time 9s)

Erro

r

Figure 2 our model errors between original and the predicted (Box–Jenkins data)

5.2. Example 2

The Mackey–Glass chaotic time series can be found in [11], and is generated from the following delay differential equation.

10

0.2 ( 17)( 1) 0.9 ( )1 ( 17)

x tx t x tx t

−+ = ++ −

In this paper, four inputs x(t-1),x(t-2),x(t-3),x(t-4) are

used to determine the output x(t+1), and here c=2.Initial values of 1.2 are assigned to x(0)~x(17).We collect 1000 input–output data between t=124 and t=1123 for the training data and testing data.

The first 500 input-output pairs are used to train the fuzzy system. The remaining 500 input-output pairs are used to test the fuzzy system. In Fig 3 the solid line shows the actual system and the dotted line shows the fuzzy system. Fig 4 shows the modeling errors and it demonstrates the validity of our approach.

565

Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

0 200 400 600 800 10000

0.5

1

1.5

Time(124-1123)

x(t+

1)

Figure 3 our model the original and the predicted output (M-G chaos model)

0 200 400 600 800 1000-0.015

-0.01

0

0.01

0.02

Time(124-1123)

Erro

r

Figure 4 our model errors between original and the predicted output (M-G

chaos model)

Table 2 shows in the same performance RMSE (root mean square error) between our model and other fuzzy modeling methods, the performance is

2

1

ˆ( ) /N

i i

iJ y y N

=

= −� .

TABLE 2 COMPARISON OF FUZZY IDENTIFICATION METHODS (M-G CHAOS MODEL)

model Input number

Rule number RMSE

ANFIS[13] 4 16 0.0016 Kukolj[14] 4 9 0.0061 Kim[14] 4 9 0.0264

Teskouras[14] 4 2 0.0041 De Souza[14] 4 - 0.0065 Lo model[11] 4 2 0.0161

Our model 4 2 0.0044

6. Conclusions

An identification method for building T-S fuzzy model based on FCM clustering algorithm is proposed in this article. Compared with the general T-S fuzzy model, this alternative

model unifies the premise structure, thus the identification process of premise parameters is simplified with FCM clustering algorithm. And the consequence parameters are calculated by least square algorithm, therefore the identification precision is enhanced. Finally the simulation results show that this algorithm has very good effect.

References

[1] Takagi T, Sugeno M. Fuzzy identification of systems and its application to modeling and control[J] .IEEE Transactions on Systems, Man and Cybernetics, 1985(15):116-132.

[2] Xu R, Donald C Wunsch. Survey of Clustering Algorithms[J]. IEEE Transactions on neural networks, 2005, 16(3):645 -678.

[3] Tsekouras G, Sarimveis H, Kavakli E. A hierarchical fuzzy, clustering approach to fuzzy modeling[J].Fuzzy Sets and Systems, 2005,150(2):245-266.

[4] Chen J Q, Xi Y G, Zhang Z J.A clustering algorithm for fuzzy model identification[J]. Fuzzy Sets and Systems, 1998, 98(3):319-32.

[5] Gomez-Skarmeta A F, Delgado M, Vila M A. About the fuzzy clustering techniques for fuzzy model identification[J]. Fuzzy Sets and Systems,1999,106(2):179-188.

[6] Bezdek J C. Pattern recognition with fuzzy objective function algorithms[M]. New York: Plenum, 1981.

[7] Box G E P, Jenkins G M. Timer Series Analysis, Forecasting and Control. San Francisio, CA:Holden Day,1970.

[8] Sugeno M, Yasukawa T.A fuzzy-logic-based approach to qualitative modeling[J]. IEEE Transactions on Fuzzy Systems, 1993, 1(1):7-31.

[9] Joo Y H, Hwang H S, Kim K B, Woo K B. Fuzzy system modeling by fuzzy partition and GA hybrid schemes[J]. Fuzzy Sets and Systems, 1997, 88:279-288.

[10] Yoshinari Y, Pedrycz W. Construction of fuzzy models through clustering techniques[J].Fuzzy Sets and Systems,1993,52(2):157-685.

[11] Lo J C, Yang C H. A heuristic error-feedback learning algorithm for fuzzy modeling[J].IEEE Transactions on System, Man and Cybernetics 1999, 29(6):686-691.

[12] Nozaki K, Tanaka H.A simple but powerful heuristic method for generating fuzzy rules from numerical data[J]. Fuzzy Sets and Systems, 1997, 86(3):251-270.

[13] Kroll A. Identification of functional fuzzy models using multidimensional reference fuzzy sets[J]. Fuzzy Sets and Systems, 1996, 80:149-158.

[14] Tsekouras G. A hierarchical fuzzy clustering approach to fuzzy modeling[J]. Fuzzy Sets and Systems, 2005, 150:245-266.

566

Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010