Fuzzy Inference Systems Applied to Image Classification in ...
Fuzzy Inference Systems
description
Transcript of Fuzzy Inference Systems
Fuzzy Inference Systems
Review Fuzzy Models
If <antecedence> then <consequence>.
Fuzzification DefuzzificationInferencing
Input Output
Basic Configuration of a Fuzzy Logic System
Target
Error =Target -Output
Types of RulesMamdani Assilian Model
R1: If x is A1 and y is B1 then z is C1
R2: If x is A2 and y is B2 then z is C2
Ai , Bi and Ci, are fuzzy sets defined on the universes of x, y, z respectively
Takagi-Sugeno Model
R1: If x is A1 and y is B1 then z =f1(x,y)
R1: If x is A2 and y is B2 then z =f2(x,y)
For example: fi(x,y)=aix+biy+ci
Types of RulesMamdani Assilian Model
Takagi-Sugeno Model
Mamdani Fuzzy Models
The Reasoning SchemeBoth antecedent and consequent are fuzzy
The Reasoning SchemeBoth antecedent and consequent are fuzzy
1: IF FeO is high & SiO2 is low & Granite is prox & Fault is prox, THEN metal is highImplication (Max)
0
1
0
1
=
2: IF FeO is aver & SiO2 is high & Granite is interm & Fault is prox, THEN metal is aver
30% 50% 70%
0
1
40% 55% 70%
0 km 10 km 20km
0 km 5 km 10km
0t 100t 1000t
3: IF FeO is low & SiO2 is high & Granite is dist & Fault is dist, THEN metal is low
FeO = 60% SiO2 = 60% Granite = 5 km Fault = 1 km Metal = ?
0t 100t 1000t
=
=
Defuzzifier• Converts the fuzzy output of the inference
engine to crisp using membership functions analogous to the ones used by the fuzzifier.
• Five commonly used defuzzifying methods:– Centroid of area (COA)– Bisector of area (BOA)– Mean of maximum (MOM)– Smallest of maximum (SOM)– Largest of maximum (LOM)
Since consequent is fuzzy, it has to be defuzzified
Defuzzifier
Rule 1: Rule 2: Rule 3: Aggregate (Max)
+ + =
Defuzzify (Find centroid)
125 tonnes metal
Formula for centroid
n
ii
n
iii
x
xx
0
0
)(
)(
Sugeno Fuzzy Models• Also known as TSK fuzzy model
– Takagi, Sugeno & Kang, 1985
If x is A and y is B then z = f(x, y)
Fuzzy Rules of TSK Model
Fuzzy Sets Crisp Functionf(x, y) is very often a polynomial
function w.r.t. x and y.The order of a Takagi-Sugeno type fuzzy inference system = the order of the polynomial used.
While antecedent is fuzzy, consequent is crisp
The Reasoning Scheme
ExamplesR1: if X is small and Y is small then z = x +y +1
R2: if X is small and Y is large then z = y +3
R3: if X is large and Y is small then z = x +3
R4: if X is large and Y is large then z = x + y + 2
TAKAGI-SUGENO SYSTEM1. IF x is f1x(x) AND y is f1y(y) THEN z1 = p10+p11x+p12y2. IF x is f2x(x) AND y is f1y(y) THEN z2 = p20+p21x+p22y3. IF x is f1x(x) AND y is f2y(y) THEN z3 = p30+p31x+p32y4. IF x is f2x(x) AND y is f2y(y) THEN z4 = p40+p41x+p42y
The firing strength (= output of the IF part) of each rule is:s1 = f1x(x) AND f1y(y)s2 = f2x(x) AND f1y(y)s3 = f1x(x) AND f2y(y)s4 = f2x(x) AND f2y(y)
Output of each rule (= firing strength x consequent function) :5. o1 = s1 ∙ z16. o2 = s2 ∙ z27. o3 = s3 ∙ z38. o4 = s4 ∙ z4
Overall output of the fuzzy inference system is: o1+ o2+ o3+ o4 s1+ s2+ s3+ s4
z =
18
Sugeno systemRule1: IF FeO is high AND SiO2 is low AND Granite is proximal AND Fault is proximal, THEN Gold =p1(FeO%)+q1(SiO2%) +r1(Distance2Granite)+s1(Distance2Fault)+t1
Rule 2: IF FeO is average AND SiO2 is high AND Granite is intermediate AND Fault is proximal, THEN Gold =p2(FeO%)+q2(SiO2%)+r2(Distance2Granite)+s2(Distance2Fault)+t2
Rule 3: IF FeO is low AND SiO2 is high AND Granite is distal AND Fault is distal, THEN Gold =p3(FeO%)+q3(SiO2%)+r3(Distance2Granite)+s3(Distance2Fault)+t3
Gold(R1) =p1(FeO%)+q1(SiO2%) + r1(Distance2Granite) +s1(Distance2Fault)+t1
1: IF FeO is high X SiO2 is low X Granite is prox X Fault is prox, THEN
0
1
0
12: IF FeO is aver X SiO2 is high X Granite is interm X Fault is prox, THEN
30% 50% 70%
0
1
40% 55% 70%
0 km 10 km 20km
0 km 5 km 10km
3: IF FeO is low & SiO2 is high & Granite is dist & Fault is dist, THEN
FeO = 60% SiO2 = 60% Granite = 5 km Fault = 1 km Metal = ?
s1
Gold(R2) =p2(FeO%)+q2(SiO2%) + r2(Distance2Granite) +s2(Distance2Fault)+t2
s2
Gold(R3) =p3(FeO%)+q3(SiO2%) + r3(Distance2Granite) +s3(Distance2Fault)+t3
s3
Sugeno system
Sugeno system: OutputGold(R1) =p1(FeO%)+q1(SiO2%) +
r1(Distance2Granite) +s1(Distance2Fault)+t1
s1
Gold(R2) =p2(FeO%)+q2(SiO2%) + r2(Distance2Granite) +s2(Distance2Fault)+t2
s2
Gold(R3) =p3(FeO%)+q3(SiO2%) + r3(Distance2Granite) +s3(Distance2Fault)+t3
s3
Firing strength
Rule output
321
321 )3(*)2(*)1(*sss
RGoldsRGoldsRGoldsOutput
A neural fuzzy system
Implements FIS in the framework of NNs
Fuzzification Nodes
Antecedent Nodes
Output Nodes
x y
Fuzzification Nodes
Represents the term sets of the features.
If we have two features x and y and two linguistic variables defined on both of it say BIG and SMALL. Then we have 4 fuzzification nodes.
x y
BIGBIG SMALL SMALL
We use Gaussian Membership functions for fuzzification ---
They are differentiable, triangular and trapezoidal membership functions are NOT differentiable.
Fuzzification Nodes (Contd.)
zx
exp
2
2
and are two free parameters of the membership functions which needs to be determined
How to determine and
Two strategies:
1) Fixed and
2) Update and , through any tuning algorithm
Consequent nodes
kqypxz p, q and k are three free parameters of the consequent polynomial function
How to determine p, q, k
Two strategies:
1) Fixed
2) Update through any tuning algorithm
Fuzzification nodes
x y
BIG BIG SMALLSMALL
μx1 μx2 μy1 μy2
Antecedent nodese.g. If x is Small & y is Small
Consequent nodes
w1 w2w3 w4
e.g. z4 = p4x + q4y + k4
z1 z2 z3z4
Output node O = (w1z1+w2z2+w3z3+w4z4)/
(w1+w2+w3+w4
Target (t)
Error = ½(t-o)2
ANFIS Architecture
Squares: Adaptive nodesCircles: Fixed nodes
ANFIS ArchitectureLayer 1 (Adaptive)Contains adaptive nodes, each with a Gaussian membership function:
Number of nodes = number of variables x number of linguistic values
In the previous example there are 4 nodes (2 variable x 2 linguistic values for each)
Two parameters to be estimated per node: mean (centre) and standard deviation (spread)
These are called premise parameters
Number of premise parameters = 2 x number of nodes = 8 in the example
2
2)(exp)(
xcxf
ANFIS ArchitectureLayer 2 (Fixed)
Contains fixed nodes, each with product operator (T-norm operator). Returns the firing strength of each If-Then Rule.
The firing strength can be normalized. In ANFIS, each node returns a normalized firing strength –
Fixed nodes – no parameter to be estimated.
yxyx
yxyx
ffsffs
ffsffs
224213
122111
;
;
4321
1
ssssss
ANFIS ArchitectureLayer 3 (Adaptive)
Each node contains an adaptive polynomial, and returns output for each fuzzy If-Then rule
Number of nodes = number of If-Then Rules.
The parameters ps are called consequent parameters.
y)pxp(ps zy)pxp(ps zy)pxp(ps z
y)pxp(ps z
42414044
32313033
22212022
12111011
ANFIS ArchitectureLayer 4 (Fixed)
Sums up the output of each node in the previous layer:
A single node in this layer.
No parameter to be estimated.
z z z z z 4321
ANFIS Training
z z z z z 4321
y)pxp(ps zy)pxp(ps zy)pxp(ps z
y)pxp(ps z
42414044
32313033
22212022
12111011
Linear in the consequent parameters Pki, if the premise parameters and, therefore, the firing strengths sk of the fuzzy if-then rules are fixed.
ANFIS uses a hybrid learning procedure (Jang and Sun, 1995) for estimation of the premise and consequent parameters.
The hybrid learning procedure estimates the consequent parameters (keeping the premise parameters fixed) in a forward pass and the premise parameters (keeping the consequent parameters fixed) in a backward pass.
Squares: Adaptive nodesCircles: Fixed nodes
The forward pass:Propagate informationforward until Layer 3 Estimate the consequent parameters by the least square estimator.
The backward pass:Propagate the error signals backwards and update the premise parameters by gradient descent.
ANFIS Training
ANFIS Training : Least Square Estimation
1. Data assembled in form of (xn; yn)
2. We assume that there is a linear relation between x and y:y = ax + b
3. Can be extended to n dimensions:y = a1x1 + a2x2 + a3x3 + … + bThe problem: Given the function f, find values of coefficients ais
such that the linear combination best fits the data
ANFIS Training : Least Square Estimation
Given data {(x1; y1 (xN ; yN)}, we may define the error associated to saying y = ax + b by:
This is just N times the variance of data : {y1 - (ax1+b),…., yn - (axN +b)}The goal is to find values of a and b that minimize the error. In other words minimize the partial derivative of the error wrt a and b:
ANFIS Training : Least Square Estimation
Which gives us:
We may rewrite them as:
The values of a and b which minimize the error satisfy the following matrix equation:
Hence a and b are estimated using:
ANFIS Training : Least Square Estimation
For the following data find least square estimator
SNo X Y X2 XY1 2 9 4 182 3 11 9 333 4 13 16 524 6 17 36 1025 8 21 64 1686 1 7 1 77 2 9 4 188 11 27 121 2979 14 33 196 462
TOTAL 51 147 451 1157
yyx
xxx
xxxba
yyx
xxx
ba
22
12
1.1.
1
1
ANFIS Training : Least Square Estimation
LSE. use andSimplify
0)y.()]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(2p
:
0)s.()]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(2p
)]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(
:
)]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(
xxxxxx
:be data trainingLet they)pxp(psy)pxp(psy)pxp(psy)pxp(ps
zzzz o Output y)pxp(ps z y);pxp(ps z y);pxp(ps z y);pxp(ps z
1142141404132131303122121202112111101142
1142141404132131303122121202112111101110
26426414046326313036226212026126111016
21421414041321313031221212021121111011
666
555
444
333
222
111
4241404323130322212021211101
4321
42414044323130332221202212111011
tE
tE
tE
tE
tytytytytyty
.parameters spread and centre update toused are sexpression above he
Similarly, function. membershipfuzzy theis output, theis where
(spread). and (centre) by drepresente be 1Layer in parameters spread and centre Let the2
Output) Actual-Output (Target E 1layer at Error
valuesy)(x, variable- and aluesstrength v firing -
values,parameter the-:in plug ,parameters consequent all of estimate squareleast After the
2
T
ssO
OO
OEE
fiOc
ssO
OO
OE
cE
sc
i
i
i
i
i
i
ii
i
i
i
i
i
i
ii
ii
ANFIS Training : Gradient descent