10.1.1.12.1988-print

International Journal of Computational Cognition (http://www.YangSky.com/yangijcc.htm)Volume 1, Number 4, Pages 7990, December 2003Publisher Item Identifier S 1542-5908(03)10404-6/$20.00Article electronically published on December 25, 2002 at http://www.YangSky.com/ijcc14.htm. Pleasecite this paper as: Ching-Hung Lee, Jang-Lee Hong, Yu-Ching Lin, and Wei-Yu Lai, Type-2 Fuzzy Neural Network Systems and Learning, International Journal of Computational Cognition(http://www.YangSky.com/yangijcc.htm), Volume 1, Number 4, Pages 7990, December 2003.

TYPE-2 FUZZY NEURAL NETWORK SYSTEMS ANDLEARNING

CHING-HUNG LEE, JANG-LEE HONG, YU-CHING LIN, AND WEI-YU LAI

Abstract. This paper presents a type-2 fuzzy neural network system(type-2 FNN) and its learning using genetic algorithm. The so-calledtype-1 fuzzy neural network (FNN) has the properties of parallel com-putation scheme, easy to implement, fuzzy logic inference system, andparameters convergence. And, the membership functions (MFs) andthe rules can be designed and trained from linguistic information andnumeric data. However, there is uncertainty associated with infor-mation or data. Therefore, the type-2 fuzzy sets are used to treat it.Type-2 fuzzy sets let us model and minimizes the effects of uncertain-ties in rule-base fuzzy logic systems (FLS). In this paper, the previousresults of type-1 FNN are extended to a type-2 one. In addition, thecorresponding learning algorithm is derived by real-code genetic algo-rithm. Copyright c2002 Yangs Scientific Research Institute, LLC.All rights reserved.

1. Introduction

Recently, intelligent systems including fuzzy logic systems, neural net-works, and genetic algorithm, have been successfully used in widely variousapplications. The fuzzy neural network systems (neuro-fuzzy systems) com-bine the advantages of fuzzy logic systems and neural networks have becomea very active subject in many scientific and engineering areas, such as, modelreference control problems, PID controller tuning, signal processing, etc.[2,3,6-11]. In our previous results, the FNN has the properties of parallelcomputation scheme, easy to implement, fuzzy logic inference system, andparameters convergence. The membership functions (MFs) and the rulescan be designed and trained from linguistic information and numeric data.Thus, it is then easy to design an FNN system to achieve a satisfactory level

Received by the editors December 18, 2002 / final version received December 23, 2002.Key words and phrases. Fuzzy neural network, type-2 fuzzy sets, genetic algorithm.This work is supported by the National Science Council, Taiwan, R.O.C., under Grant

NSC-91-2213-E155-012.

c2002 Yangs Scientific Research Institute, LLC. All rights reserved.79

80 LEE, HONG, LIN, AND LAI

of accuracy by manipulating the network structure and learning algorithmof the FNN. However, there is uncertainty associated with information ordata. Therefore, the type-2 fuzzy sets are used to treat it.

Recently, Mendel and Karnik [5,12,14,15] have developed a complete the-ory of type-2 fuzzy logic systems. These systems are again characterized byIF-THEN rules, but their antecedent or consequent sets are type-2. A type-2 fuzzy set can represent and handle uncertain information effectively. Thatis, type-2 fuzzy sets let us model and minimizes the effects of uncertaintiesin rule-base fuzzy logic systems (FLS). The purpose of this paper is to de-velop type-2 fuzzy neural network, i.e., extend our previous results of theFNN the type-2 one. Indeed, The learning algorithm is derived by geneticalgorithm.

Genetic algorithm (GA) was first proposed by Holland in 1975 [13,17,18].It is motivated by mechanism of natural selection, a biological process inwhich stronger individuals are likely be the winners in a competing envi-ronment. It provides an alternative to traditional optimization techniquesby using directed random searches to locate optimal solutions in complexproblems [1,4,13,17,18]. Recently, GA has emerged as a popular family ofmethods for global optimization. Through the use of genetic operations, GAperforms a search by evolving a population of potential solutions [17,18].

The organization of this paper is as follows. In Section 2, we briefly intro-duce the preliminaries- type-1 fuzzy neural network, genetic algorithm andrtype-2 fuzzy set. Section 3 presents the main result- type-2 FNN systemsand learning algorithm. Finally, conclusion is summarized in Section 4.

2. Preliminaries

2.1. Fuzzy Neural Network (Type-1 FNN system). The fuzzy neuralnetwork (FNN) system is one kind of fuzzy inference system in neural net-work structure [2,3,7,10,11]. A schematic diagram of the four-layered FNNis shown in Fig. 1. Obviously, it is a static model of recurrent fuzzy neuralnetwork (RFNN) [7]. The type-1 FNN system has total four layers. Nodesin layer one are input nodes representing input linguistic variables. Nodesin layer two are membership nodes. Here, the Gaussian function is used asthe membership function (MF). Each membership node is responsible formapping an input linguistic variable into a possibility distribution for thatvariable. The rule nodes reside in layer three. The last layer contains theoutput variable nodes. More details about FNNs, convergent theorems andthe learning algorithm, can be found in [6-9]. Also, the FNN used here hasbeen shown to be a universal approximator. That is, for any given realfunction h : Rn Rp, continuous on a compact set K Rn, and arbitrary

TYPE-2 FUZZY NEURAL NETWORK SYSTEMS 81

> 0, there exists a FNN system F (x,W ), such that F (x,W ) h(x) < for every x in K.

Figure 1. Schematic diagram of fuzzy neural networks.

For generality, we must consider m fuzzy rules which can be consideredindependently like dealing with the jth fuzzy rule in Figure 2. Indeed, thesimplified fuzzy reasoning is described as follows.

Given the training input data xk, k = 1, 2, , , n, and the desired outputyp, p = 1, 2, , ,m. Thejth control rule has the following form:

Rj: IF x 1 is Aj1 and . . . . . . xn is A

jn THEN y1 is

j1 and . . . . . . ym is

jm.

where j is the rule number, the Ajqs are membership functions of the an-tecedent part, and jps are real numbers of the consequent part. Whenthe inputs are given, the truth value i of the premise of the jth rule iscalculated by

(1) j = Aj1(x1) Aj2(x2) . . . Ajn(xn).

Among the commonly used defuzzification strategies, the simplified fuzzyreasoning yields a superior result. The output where yp of the fuzzy reason-ing can be derived from the following equation.

(2) yp =i

ipi, p = 1, 2, ,m


where i is the truth value of the premise of the ith rule.

Figure 2. Construction of the jth component of the FNN.

2.2. Type-2 Fuzzy Sets. The concept of type-2 fuzzy set was initiallyproposed as an extension of ordinary (type-1) fuzzy sets by Prof. Zaden [19].And then, the clear definition of type-2 fuzzy set is proposed by Mizumotoand Tanaka [16]. Recently, Mendel and Karnik [5,12,14,15] have developeda complete theory of type-2 fuzzy logic systems (FLSs). These systems areagain characterized by IF-THEN rules, but their antecedent or consequentsets are type-2. A type-2 fuzzy set can represent and handle uncertaininformation effectively. That is, type-2 fuzzy sets let us model and minimizesthe effects of uncertainties in rule-base FLSs. As literature [14,15], there areat least four sources of uncertainties in type-1 FLSs, e.g., antecedents andconsequents of rules, measurement noise, and training date noisy, etc. Allof these uncertainties can be translated into uncertainties about fuzzy MFs.The type-1 fuzzy sets could not treat it because the MFs are crisp. Thatis, type-1 MFs are of two-dimensional, whereas type-2 MFs are of three-dimensional. It is the new third-dimension of type-2 MFs that make itpossible to model the uncertainties.

Subsequently, we use the following notation and terminology to describethe fuzzy sets. Firstly, A is a type-1 fuzzy set and the membership grade ofx X in A is A(x), which is a crisp number in [0,1]; a type-2 fuzzy set inX is A and the membership grade of x X in A is A(x), which is a type-1fuzzy set in [0,1]. The type-2 fuzzy set A X can be represented as


(3) A(x) = fx(u1)/u1 + fx(u2)/u2 + + fx(um)/um =i

fx(ui)/ui

The useful type-2 fuzzy set is the footprint of uncertainty (FOU), e.g.,see Fig. 3 [5,12,14,15]. Figures 3 (a) and 3 (b) show the Gaussian MFs withuncertain STD and Gaussian MFs with uncertain mean. These ones areused to develop the type-2 FNN systems in using on primary and consequentparts. Herein, these MFs with uncertain mean and STD are described as

A(x) = exp( (xm)

2

2

),m [m1,m2] and

A(x) = exp( (xm)

2

2

), [1, 2],(4)

respectively. Obviously, this type membership can be represented as boundedinterval by upper MF and lower MF, denote A(x) and A(x). Details abouttype-2 fuzzy sets can be found in literature [5,12,14,15].

-1 -0.5 0 0.5 1 1.5 2 2.5 30

0.2

0.4

0.6

0.8

1

-1 -0.5 0 0.5 1 1.5 2 2.5 30

0.2

0.4

0.6

0.8

1

(a)(b)

)(xA

)(xA

)(xA

)(xA

Figure 3. Type-2 fuzzy set- (a) Gaussian MFs with un-certain mean (b) Gaussian MFs with uncertain STD.

The basics of fuzzy logic do not change from type-1 to type-2 sets. Thedifference between these two systems is output processing. The type-2 FLSsshould use the type-reducer to reduce the output fuzzy sets degree. As Fig.4 shows [5,12,14,15], the structure of a type-2 FLS is similar to the structureof type-1 one. The structure includes fuzzifier, knowledge base, inferenceengine, type-reducer, and defuzzifier. Based on the block diagram, we willexplanation the FLSs of type-2 FNN systems in next section.


Figure 4. The block diagram of type-2 FLS.

2.3. Genetic Algorithm (GA). GA uses a direct analogy of such naturalevolution. It presumes that the potential solution of a problem is an indi-vidual and can be represented by a set of parameters. These parameterscan be structured by a string of values and are regarded as the genes of achromosome. Herein, we briefly introduce it. A population consists of afinite number of chromosomes (or parameters). The GA evaluates a popu-lation and generates a new one iteratively, with each successive populationreferred to as a generation. Fitness value, a positive value is used to reflectthe degree of goodness of the chromosome for solving the problem, andthis value is closely related to its objective value. In operation process, aninitial population P(0) is given, and then the GA generates a new genera-tion P(t) based on the previous generation P(t-1). The GA uses three basicoperators to manipulate the genetic composition of a population: reprod-uct, crossover, and mutation [1,4, 17,18]. The most common representationin GA is binary [4,13,18]. The chromosomes consist of a set of genes, whichare generally characters belonging to an alphabeta {0,1}. In this paper,the real-coded GA is used to tune the parameters. It is more natural torepresent the genes directly as real numbers since the representations of thesolutions are very close to the natural formulation. Therefore, a chromo-some here is a vector of floating point numbers. The crossover and mutationoperators developed for this coding is introduced below.


3. Type-2 Fuzzy Neural Network and GA

3.1. Type-2 FNN Systems. Herein, we consider a type-2 FLS systemwith a rule base of R rules in type-2 FNN system, e.g., n-input m-outputwith R rules. The jth control rule is described as the following form:

Rj: IF x 1 is Aj1 and. . . xn is A

jn THEN y1 is

j1 and . . . ym is

jm.

where j is a rule number, the Ajqs are type-2 MFs of the antecedent part,and jps are type-1 fuzzy sets of the consequent part. Herein, the antecedentpart MFs are represented as an upper MF and a lower MF, denote A(x)and A(x) (see Fig. 3). The consequent part is a interval set = [, ].The rules let us simultaneously account for uncertainty about antecedentmembership functions and consequent parameters values.

When the input are given, the firing strength of the jth rule is

(5) m = Am1 (x1) Am2 (x2) . . . Amn (xn)where is the meet operation [5,12,14,15]. Herein, the antecedent operationis product t-norm. That is, equation (1) can be calculated by

m

= Am1

(x1) Am2 (x2) . . . Amn (xn) andm = Am1 (x1) Am2 (x2) . . . Amn (xn).(6)

Finally, the type reduction and defuzzification should be considered. Sim-ilar to the FNN, herein the center of sets (COS)-type reduction method isused to find

yil =Mi=1

ilil and y

il =

Mi=1

irir(7)

where il denotes the firing strength membership grad (either i or i).

Hence, the defuzzified output of an interval type-2 FLS is

(8) yi =yil + y

ir

2.

Note that, if rule number R is even M = R2 . On the other hand, R isodd, M = R12 and

(9) y =yl + yr

2+(i

M+1+ iM+1) (iM+1 +

i

M+1)

4.

Herein, we simplify the computation procedure for computing yr and ylwhich is difference from literature [14,15]. Details of comparison can befound in literature [14,15].


Figure 5 summarizes above discussion and shows a fuzzy inference system(jth rule) of type-2-FNN system.

Example: Computation of type-2 FNN system with two rulesIf a type-2 FNN system has two rules as follows:R1: IF x1 is A1 AND x2is B1 THEN y = w1.R2: IF x1 is A2 AND x2 is B2 THEN y = w2.Figure 6 summaries the computation of type-2 FNN system. In the first

layer, the output values are the input x1 and x2, respectively. In layer 2, onedetermines the MF grads by type-2 MFs, i.e., MF grads of upper MF andlower MF. Thus, one obtains [Ai(x1), Ai(x1)] and [Bi(x2), Bi(x2)], i=1,2.Thus, using the operation in layer-product, one can have [

i(x1, x2), i(x1, x2)] =

[A1(x1) B1(x2), A1(x1) B2(x2)]. Finally, yr and yl should be determined.Note that wi = [wi, wi], one has yl = 1w1 + 2w2, yr = 1w1 + 2w2,

and the defuzzified value y =yr + yl

2.

Remark : It is trivial that the type-2 FNN system is a generalization of theFNN system. That is, the type-2 FNN system can be reduce to a type-1 oneif the fuzzy sets is type-1. We can find that details computation of thesesystems are the same.

Figure 5. Fuzzy inference of Type-2 FNN.

3.2. Training of Type-2 FNN- Genetic Algorithm. It is known thatthe type-1 FNN system is a universal approximator [2,3,6-9]. That is, ingeneral, for function mapping or system identification, it is easy to designan FNN system to achieve a satisfactory level of accuracy. By the way, we


Figure 6. Computation example of a Type-2 FNN.

determine the feature parameters to represent a type-2 fuzzy set. Usingthese parameters, a type-2 FNN system can be encoded as a chromosome.Then, the real-code genetic algorithm is used optimize the type-2 FNNsystem, i.e., antecedent and consequent MFs.

Herein, the training process using real-code genetic algorithm is describedas follows.Learning Process

Step 1: Constructing and initializing the type-1 FNN systemStep 2: Using the back-propagation algorithm to train the type-1 FNN

and obtain a set of Gaussian functions (mean, variance) and weighting vec-tor.

Step 3: Using the results of Step 2 and add a uncertainty in antecedentand consequent part, i.e., m1,m2 = m m,,w1, w2 = w w orm,1, 2 = ,w1, w2 = w w.

Step 4: Constructing the chromosome (2Rmean+R STD+2R weight).Step 5: Using the GA to train the type-2 FNN to find the optimal values.


The objective of parameters learning is to optimally adjust the free pa-rameters of the type-2 FNN for each incoming data. Subsequently, in thisphase the chromosome should be defined.Chromosome: the genes of each chromosome (denotes xi) include twoparts. One is the MF and the other is weighting vector. Each MF containstwo means values (upper MF and lower MF), STD, and weighting vector(or mean, two STD values, and weight). Therefore, for a given n-input one-output type-2 FNN with R rules, the number of genes for each chromosomeis 3R n+2R.Fitness function: Herein, the fitness function is defined as

(10) fitness (x) =1E, E =

t

i

(di (t) yi (t))2

where di (t) and yi (t) are the desired output and type-2 FNN system output,respectively.Reproduction: The tournament selection is used in the reproduction pro-cess [13,18].Crossover: Here, the real-coded crossover operation is used.

(11) x1i = x1i + (x2i x1i)

(12) x2i = x1i + (x1i x2i)where fitness (x1) fitness (x2), x1i and x2i are the ith genes of theparents x1 and x2, respectively. x1i and x

2i are the ith genes of the parents

x1 and x2, is a random number and 0 0.5.

Mutation: The mutation operation is

(13) x1i = x1i +

where i denotes the ith gene and it is randomly chosen; x1i and x1i are theith genes of the parents x1 and x1 respectively; is a random number in agiven range.

4. Conclusion

This paper has presented a type-2 FNN system and the correspondinggenetic learning algorithm. This type-2 FNN will be used to treat the un-certainty associated with information or data. That dues to the propertiesof type-2 fuzzy sets, it can represent and handle uncertain information ef-fectively. Therefore, the previous results of the FNN have been extended to


a type-2 one. We determine the feature parameters to represent a type-2fuzzy set. Using these parameters, a type-2 FNN system can be encoded asa chromosome. Then, the real-code genetic algorithm is used optimize thetype-2 FNN system, i.e., antecedent and consequent MFs.

References

[1] T. N. Bui and B. R. Moon, Genetic Algorithm and Graph Partitioning, IEEETrans. on Computers, Vol.45, No.7, July 1996.

[2] Y. C. Chen and C. C. Teng, A Model Reference Control Structure Using A FuzzyNeural Network, Fuzzy Sets and Systems, Vol. 73, pp. 291312,1995.

[3] Y. C. Chen and C. C. Teng, Fuzzy Neural Network Systems in Model ReferenceControl Systems, in Neural Network Systems: Technique and Applications, Vol. 6,Edited by C. T. Leondes, Academic Press, Inc., pp. 285-313, 1998.

[4] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learn-ing, Addison-Wesley, Reading, 1989.

[5] N. Karnik, J. Mendel, and Q. Liang, Type-2 Fuzzy Logic Systems, IEEE Trans.On Fuzzy Systems, Vol. 7, No. 6, pp. 643-658, 1999.

[6] C. H. Lee and C. C. Teng, Fine Tuning of Membership Functions for Fuzzy NeuralSystems, Asian Journal of Control, Vol. 3, No. 3, pp. 18-25, 2001.

[7] C. H. Lee and C. C. Teng, Identification and Control of Dynamic Systems UsingRecurrent Fuzzy Neural Networks, IEEE Trans. on Fuzzy Systems, Vol. 8, No. 4,pp. 349-366, August 2000.

[8] C. H. Lee and C. C. Teng, Approximation of Periodic Function Using a ModifiedFuzzy Neural Network, International Journal of Fuzzy Systems, Vol. 2, No. 3, pp.176-182, 2000.

[9] C. H. Lee and C. C. Teng, Tuning PID Controller of Unstable Processes: A FuzzyNeural Network Approach, Fuzzy Sets and Systems, Vol. 128, No.1, pp. 95-106,2002.

[10] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems, Prentice Hall: Englewood Cliff,1996.

[11] C. T. Lin and C. S. G.Lee, Neural-Network-Based fuzzy logic control and decisionsystem, IEEE Trans. Computers, vol. C-40, no.12, pp1320-1336, 1991.

[12] Q. Liang and J. Mendel, Interval Type-2 Fuzzy Logic Systems: Theory and Design,IEEE Trans. On Fuzzy Systems, Vol. 8, No.5, pp. 535-550, 2000.

[13] K. F. Man, K. S. Tang, S. Kwong, Genetic Algorithms: Concepts and Applica-tions, IEEE Transactions on Industrial Electronics, Vol. 43, No. 5, October 1996.

[14] J. Mendel and R. John, Type-2 Fuzzy Sets Made Simple, IEEE Trans. On FuzzySystems, Vol. 10, No. 2, pp. 117-127, 2002.

[15] J. M. Mendel, Uncertain Rule-Based Fuzzy Logic Systems: Introduction and NewDirections, Prentice-Hall: NJ, 2001.

[16] M. Mizumoto and K. Tanaka, Some Properties of Fuzzy Sets of Type 2, Informa-tion and Control, Vol. 31, pp. 312-340, 1976.

[17] Z. Michalewicz, Genetic Algorithms + Data Structure = Evolutionary Programs,Springer-Verlag, Berlin, 3rd edition 1997.

[18] M. Srinivas, Lalit M. Patnaik, Genetic Algorithms: A Survey, Computer, Vol. 27,pp.17-66, 1994.

[19] L A. Zadeh, The Concept of A Linguistic Variable and Its Application to Approx-imate Reasoning -1, Information Sciences, Vol. 8, pp. 199-249, 1975.


Ching-Hung Leea, Jang-Lee Hongb, Yu-Ching Lina, and Wei-Yu LaiaaDepartment of Electrical Engineering, Yuan Ze University, No. 135, Yuan-Tung Road, Chung-Li, Taoyuan 320, Taiwan, R.O.C.bDepartment of Electronic Engineering, Van Nung Institute of Technology,Chung-Li, Taoyuan 320, Taiwan, R.O.C.

E-mail address: [email protected] (C.-H. Lee)

10.1.1.12.1988-print

Documents

Transcript of 10.1.1.12.1988-print