ANFIS

Neural Networks and Fuzzy Logic

*Fuzzy Logic: Intelligence, control, and Information - J.Yen and R. Langari, Prentice Hall 1999

• Neural networks and fuzzy logic are twocomplimentary technologies

• Neural networks can learn from data andfeedback

– It is difficult to develop an insight about themeaning associated with each neuron and eachweight

– Viewed as “black box” approach (know what thebox does but not how it is done conceptually!)

Online (pattern mode) VS Batchmode of BP learning


• Two ways to adjust the weights usingbackpropagation

– Online/pattern Mode: adjusts the weights basedon the error signal of one input-output pair in thetrainning data.

• Example: trainning set containning 500 input-outputpairs, this mode BP adjusts the weights 500 times foreach time the algorithm sweeps through the trainningset. If the algorithm sweeps converges after 1000sweeps, each weight adjusted a total of 50,000 times.

Online (pattern mode) VS Batchmode of BP learning (cont.)


– Batch mode (off-line): adjusts weights based onthe error signal of the entire training set.

• Weights are adjusted once only after all the trainningdata have been processed by the neural network.

• From previous example, each weight in the neuralnetwork is adjusted 1000 times.

Neural Networks and Fuzzy Logic (cont)


• Fuzzy rule-based models are easy to comprehend(uses linguistic terms and the structure of if-then rules)

• Unlike neural networks, fuzzy logic does not comewith a learning algorithm

– Learning and identification of fuzzy models needto adopt techniques from other areas

• Since neural networks can learn, it is natural tomarry the two technologies.

Neuro- Fuzzy System


• Neuro-fuzzy system can be classified intothree categories:

– A fuzzy rule-based model constructed using asupervised NN learning technique

– A fuzzy rule-based model constructed usingreinforcement-based learning

– A fuzzy rule-based model constructed usingNN to construct its fuzzy partition of the inputspace

ANFIS: Adaptive Neuro-FuzzyInference Systems

*Neuro-fuzzy and Soft Computing - J.Jang, C. Sun, and, E. Mizutani, Prentice Hall 1997

• A class of adaptive networks that arefunctionally equivalent to fuzzy inferencesystems.

• ANFIS architectures representing both theSugeno and Tsukamoto fuzzy models

A two-input first-Order Sugeno FuzzyModel with two rules


Equivalent ANFIS architecture


ANFIS Architecture


• Assume - two inputs X and Y and one output Z

Rule 1: If x is A1 and y is B1, then f1 = p1x + q1y +r1

Rule 2: If x is A2 and y is B2, then f2 = p2x + q2y +r2

ANFIS Architecture: Layer 1


Every node i in this layer is an adaptive node with a node functionO 1,i = µAi (x), for I = 1,2, or O 1,i = µBi-2 (x), for I = 3,4

Where x (or y) is the input to node i and Ai (or Bi) is a linguistic label** O 1,i is the membership grade of a fuzzy set and it specifies the

degree to which the given input x or y satisfies the quantifies

ANFIS Architecture: Layer 1 (cont.)


Typically, the membership function for a fuzzy set canbe any parameterized membership function, such astriangle, trapezoidal, Guassian, or generalized Bellfunction.

Parameters in this layer are referred to as Antecedence Parameters

Triangular MF


Trapezoidal MF


Gaussian MF


Generalized Bell MF


Generalized Bell MF (Cont.)




Every node i in this layer is a fixed node labeled Π, whose outputis the product of all the incoming signals:

O 2,i = Wi = min{µAi (x) , µBi-2 (x)}, i = 1,2

Each node output represents the firing strength of a rule.



Every node in this layer is a fixed node labeled N. The ith nodecalculates the ratio of the ith rule’s firing strength to the sum of allrules’firing stregths:

__O 3,i = Wi = Wi /(W1+W2) , I =1,2 (normalized firing strengths]



Every node i in this layer is an adaptive node with a node function __ __O 4,i = wi fi = wi (pix + qiy +ri) …Consequent parameters



The single node in this layer is a fixed node labeled Σ, whichcomputes the overall output as the summation of all incomingsignals:

__O 5,1 = Σi wi fi

ANFIS Architecture: Alternate


ANFIS architecture for the Sugeno fuzzy model, weightnormalization is performed at the very last layer

ANFIS Architecture: Tsukamoto model


Equivalent ANFIS architecture using the Tsukamoto fuzzy model

ANFIS Architecture: 2 input Sugeno with 9 rules


Backpropagation Learning Appliedto zero-order TSK Fuzzy Modeling


• The algorithm becomes somewhat simplewhen it is applied to the fuzzy model

– Let Assume that we will apply BP learning tothe parameter identification of TSK fuzzymodel with

• Antecedent MFs are of Gaussian type– Antecedent parameters mij and σij

• and the consequents Ci



• In the forward pass, for a given input pattern, the actualresponse of the model is computed directly from theequation below



• In the backward pass, the error signal resultingfrom the difference between the actual outputand the desired output of the model ispropagated backward and the parameters mij,σij, and Ci are adjusted using the error-correction rule.

On-line/Pattern Mode updating


Where η1, η2, η3 are the learning-rate parameters

Summary - Pattern Mode BP applied tozero-order TSK models with Gaussian MFs


1. Set mij (0), σij (0), and Ci (0) to small random numbers.

2. For k=1,2, …., N, do the following steps:3. (Forward Pass) Compute vi(k), where mij, σij, and Ci are

replaced by mij (k-1) and σij (k-1) respectively; computey^(k) where Ci is replaced by Ci (k-1).

4. (Backward Pass) From error signal e(k) update Ci, mij,and σij.

5. If some stopping criterion is satisfied, stop; otherwiseset k=k+1 and goto step2.

Off-line/batch Mode updating


Where η1, η2, η3 are the learning-rate parameters

Summary - Batch Mode BP applied tozero-order TSK models with Gaussian MFs


1. Set mij (0), σij (0), and Ci (0) to small random numbers,and set n = 1.

2. (Forward Pass) Compute vi(k) for k=1, 2, …, N, where mij,σij, and Ci are replaced by mij (n-1) and σij (n-1)respectively; compute y^(k) for k = 1, 2, . . ., N, where Ciis replaced by Ci (n-1).

3. (Backward Pass) From error signal e(k) for k = 1, 2, …, N;update Ci, mij, and σij.

4. If some stopping criterion is satisfied, stop; otherwiseset n=n+1 and goto step2.

Summary - Batch Mode VS Pattern Mode


• Pattern mode of learning is preferred overbatch mode because it requires less localstorage for updating each parameter.

• Batch mode of learning, however, provides amore accurate estimate of the gradients.

• Pattern learning approaches Batch learningprovided that learning-rate parameters aresmall.

Example 1: Modeling nonlinear Function


– y = sin(x)/x• 100 training data pairs (even distributed grid

points of input range [-10,10]• ANFIS zero-order TSK contains 30 rules, whose

antecedent and consequent parameters weretrained using the BP algorithm with the initialparameter values Ci(0) ε [-1,1], mij(0) ε [-10,10], andσij (0) ε [1,2]

Example 1: Modeling sin(x)/x Function


Example 2: Modeling a Two-Input Sinc Function


– z = sinc(x,y) = sin(x) sin(y)/xy• 121 training data pairs (even distributed grid

points of input range [-10,10] x [-10,10]• ANFIS first-order TSK contains 16 rules with 24

antecedent parameters (using generalized BellMFs) and 48 consequent parameters weretrained using Hybrid learning algorithm.



Two passes in the hybrid learning procedure for ANFIS

Forward Pass Backward PassAntecedent parameters Fixed Gradient descentConsequent parameters Least-squares

estimatorFixed

Signals Node outputs Error signals

Neuro-Fuzzy Versus Neural Network


• Neuro-fuzzy system differs from NN in fourmajor ways

1. Nodes and links in a neuro-fuzzy systemcorrespond to a specific component ina fuzzy system

• Example: First Layer describes the antecedent MF

2. Node is usually not fully connected tonodes in an adjacent layer

Neuro-Fuzzy Versus NN (cont.)


3. Nodes in different layers of neuro-fuzzysystem typically perform differentoperations

4. A neuro-fuzzy system typically has morelayers than neural network

ANFIS

Documents

Transcript of ANFIS