Fuzzy Logic and Fuzzy Systems - Trinity College Dublin

1

1

Fuzzy Logic and Fuzzy Systems – Revision Lecture

1

Khurshid Ahmad, Professor of Computer Science,Department of Computer Science

Trinity College,Dublin-2, IRELAND24 February 2008.

https://www.cs.tcd.ie/Khurshid.Ahmad/Teaching.html

2

2

FUZZY LOGIC & FUZZY SYSTEMS

Knowledge Representation & Reasoning

We have covered five topics in this course:

1. Terminology: Uncertainty,

Approximations and Vagueness

2. Fuzzy Sets

3. Fuzzy Logic and Fuzzy Systems

4. Fuzzy Control

5. Neuro-fuzzy systems

3

3


The Written Examination

4

4



Each question

has a preamble

that defines the

scope of the

question.

There is a clear

indication as to

which of the five

topics the

question covers.

5

5



Each question has

two component: A

conceptual part

testing your

comprehension of

terminology and

ontology of the

subject – carrying

no more than 1/3

of the total mark

for the question

6

6



Each question has

two component: A conceptual part testing your

comprehension of terminology and

ontology of the subject

And a problem to

be solved which

shows your ability

to deploy your

knowledge. This

part comprises 2/3

of the mark for

the question

7

7



Each question

has a preamble

that defines the

scope of the

question.

There is a clear

indication as to

which of the five

topics the

question covers.

8

8



Each question has

two component: A

conceptual part

testing your

comprehension of

terminology and

ontology of the

subject – carrying

no more than 1/3

of the total mark

for the question

9

9



Each question has

two component: A conceptual part testing your

comprehension of terminology and

ontology of the subject

And a problem to

be solved which

shows your ability

to deploy your

knowledge. This

part comprises 2/3

of the mark for

the question

10

10



I will prefer

your answer to

the conceptual

part should be

short and

scuccinct.

11

11



For the

problem-

solving part,

please be sure

to show how

you performed

the calculation.

Comment on

the steps you

have taken.

12

12



For the

problem-

solving part,

please be sure

to show how

you performed

the calculation.

Comment on

the steps you

have taken.

13

13



For the

problem-

solving part,

please be sure

to show how

you performed

the calculation.

Comment on

the steps you

have taken.

14

14

FUZZY LOGIC & FUZZY SYSTEMS UNCERTAINITY AND ITS TREATMENT

Theory of fuzzy sets and fuzzy logic has been

applied to problems in a variety of fields:

Taxonomy; Topology; Linguistics; Logic; Automata

Theory; Game Theory; Pattern Recognition;

Medicine; Law; Decision Support; Information

Retrieval;

And more recently FUZZY Machines have been

developed including automatic train control and tunnel

digging machinery to washing machines, rice cookers,

vacuum cleaners and air conditioners.

15

15

FUZZY LOGIC & FUZZY SYSTEMS UNCERTAINITY AND ITS TREATMENT

The term fuzzy logic is used in two senses:

•Narrow sense: Fuzzy logic is a branch of fuzzy set

theory, which deals (as logical systems do) with the

representation and inference from knowledge. Fuzzy

logic, unlike other logical systems, deals with

imprecise or uncertain knowledge. In this narrow, and

perhaps correct sense, fuzzy logic is just one of the

branches of fuzzy set theory.

•Broad Sense: fuzzy logic synonymously with

fuzzy set theory

16

16

FUZZY LOGIC & FUZZY SYSTEMS FUZZY SETS

An Example: Consider a set of numbers: X = {1, 2, ….. 10}. Johnny’s understanding of numbers is limited to 10, when asked he suggested the following. Sitting next to Johnny was a fuzzy logician noting :

0‘Definitely Not’5, 4, 3, 2, 1

0.2‘In some cases, not usually’6

0.5‘Maybe’7

0.8‘Quite poss.’8

1‘Surely’9

1‘Surely’10

‘Degree of membership’

Comment‘Large Number’

17

17


An Example: Consider a set of numbers: X = {1, 2, ….. 10}. Johnny’s understanding of numbers is limited to 10, when asked he suggested the following. Sitting next to Johnny was a fuzzy logician noting :

We can denote Johnny’s notion of ‘large number’ by the

fuzzy set

A =0/1+0/2+0/3+0/4+0/5+ 0.2/6 + 0.5/7 + 0.8/8 + 1/9 + 1/10

0‘Definitely Not’5, 4, 3, 2, 1

0.2‘In some cases, not usually’6

0.5‘Maybe’7

0.8‘Quite poss.’8

1‘Surely’9

1‘Surely’10

‘Degree of membership’

Comment‘Large Number’

18

18


Fuzzy (sub-)sets: Membership Functions

For the sake of convenience, usually a fuzzy set is

denoted as:

A = µA(xi)/xi + …………. + µA(xn)/xnthat belongs to a finite universe of discourse:

whereµA(xi)/xi (a singleton) is a pair “grade of membership element”.

},.......,,{ 21~

nxxxA ⊂

19

19

FUZZY LOGIC & FUZZY SYSTEMS FUZZY SETS: PROPERTIES

α-cutsP5

An empty fuzzy setP4

Cardinality of a fuzzy setP3

Inclusion of one set into another fuzzy set

P2

Equality of two fuzzy setsP1

DefinitionProperties

20

20


FUZZY SETS: OPERATIONS

Example: Recall X = {1, 2, 3} and

A = 0.3/1 + 0.5/2 + 1/3 �A’ = A = 0.7/1 + 0.5/2.

The complementation of a fuzzy set

A ⊂⊂⊂⊂ X (A of X) � A (NOT A of X)

~� µA(x) = 1 - µA(x)

O1

Example: Consider Y = {1, 2, 3, 4} and C ⊂ Y �

~

C = 0.6/1 + 0.8/2 + 1/3; then C’ = (C) = 0.4/1 + 0.2/2 + 1/4then C’ = (C) = 0.4/1 + 0.2/2 + 1/4; C1 contains one member not in C (i.e., 4) and does not contain one member of C (i.e., 3)

Definition & ExampleOperations

21

21



Once we have found that the knowledge of a specialism can be expressed through linguistic variables and rules of thumb, that involve imprecise antecedents and consequents, then we have a basis of a knowledge-base.

•In this knowledge-base ‘facts’ are represented through linguistic variables and the rules follow fuzzy logic.

•In traditional expert systems facts are stated crisply and rules follow classical propositional logic.

22

22



A fuzzy knowledge-based system (KBS) is a KBS that performs approximate reasoning. Typically a fuzzy KBS uses knowledge representation and reasoning in systems that are based on the application of Fuzzy Set Theory. A fuzzy knowledge base comprises vague facts and vaguerules of the form:

IF X

THEN Y

IF X is µXTHEN Y is µY

Rule

X is TRUE or

X is NOT TRUE

X is µXFact

Crisp KBFuzzy KBKB Entity

23

23


Knowledge Representation

There are two challenges:

(a)How to interpret and how to represent vague rules with the help of appropriate fuzzy sets? &

(b)How to find an inference mechanism that is founded on well-defined semantics and that permits approximate reasoning by means of a conjunctive general system of vague rules and case-specific vague facts?

24

24


Knowledge Representation

Linguistic Variables

A linguistic variable is associated with two rules:

(a)A syntactic rule, which defines the well-formed sentences in T( ); and

(b)a semantic rule, by which the meaning of the terms in T( ) may be determined. If X is a term in T( ), then its meaning (in a denotational sense) is a subset of U. A primary fuzzy set, that is, a term whose meaning must be defined a priori, and serves as a basis for the computation of the meaning of the nonprimary terms in T( ).

25

25



R E C A P I T U L A T E

26

26



The operation of a fuzzy expert system depends on the execution of FOUR

major tasks:

Fuzzification, Inference,

Composition, Defuzzification.

27

27

FUZZY LOGIC & FUZZY SYSTEMS Knowledge Representation & Reasoning

Fuzzification involves the choice of variables, fuzzy input and output variables and defuzzified output variable(s), definition of membership functions for the input variables and the description of fuzzy rules.

28

28


Fuzzification :The membership functions defined on the input variables are applied to their actual values to determine the degree of truth for each rule premise.

The degree of truth for a rule's premise is

sometimes referred to as its αααα (alpha) value. If a rule's premise has a non-zero degree of truth, that is if the rule applies at all, then the rule is said to fire.

29

29


Inference: The truth-value for the premise of each rule is computed and the conclusion applied to each part of the rule. This results in one fuzzy subset assigned to each output variable for each rule.

30

30

FUZZY LOGIC & FUZZY SYSTEMS Knowledge Representation & ReasoningInference: MIN and PRODUCT are two inference

methods.

1. In MIN inferencing the output membership function is clipped off at a height corresponding to the computed degree of truth of a rule's premise. This corresponds to the traditional interpretation of the fuzzy logic's AND operation.

2. In PRODUCT inferencing the output membership function is scaled by the premise's computed degree of truth.

31

31


Composition: All the fuzzy subsets assigned to each output variable are combined together to form a single fuzzy subset for each output variable.

32

32


Composition: MAX and SUM are two composition rules:

1. In MAX composition, the combined fuzzy subset is constructed by taking the pointwise maximum over all the fuzzy subsets assigned to the output variable by the inference rule.

2. The SUM composition, the combined output fuzzy subset is constructed by taking the pointwise sum over all the fuzzy subsets assigned to output variable by their inference rule. (Note that this can result in truth values greater than 1).

33

33


Defuzzification: The fuzzy value produced by the composition stage needs to be converted to be converted to a single number or a crisp value.

34

34


Defuzzification: The crisp value is essentially the area under the curve of the new fuzzy subset derived from the composition stage. Such a computation takes into account the effect of each rule inaproportionate manner. Sometimes, however, it is important to take only into account those rules that have the maximum impact.

Hence there are different methods of defuzzication.

35

35


Defuzzification: Two popular defuzzification techniques are the CENTROID and MAXIMUM techniques.

1. The use of CENTROID technique relies on using the centre of gravity of the membership function to calculate the crisp value of the output variable.

2. The MAXIMUM techniques, and there are a number of them, broadly speaking, use one of the variable values at which the fuzzy subset has its maximum truth value to compute the crisp value.

36

36


Knowledge Representation & Reasoning: The Air-conditioner Example

DEFUZZIFICATION: The ‘Centre of Gravity’ (COG) of the output of the rules: Formally, the crisp value is the value located under the centre of gravity of the area that is given by the function

∫∫

=Yy

output

xx

Yy

output

xx

dyyydyy

n

n

ε

ε

µµ

η )()(

1

.

.

.....1

.....1

37

37



DEFUZZIFICATION: The crisp value h can be obtained by approximating the integral with a sum

)()(

1.

.

.....1

.....1

yyy

output

xxoutput

xx

n

n

µµ

η ΣΣ

=

The centre of gravity approach attempts to take the rules into consideration according to their degree of applicability. If a rule dominates during a certain interval then its dominance is discounted in other intervals.

38

38



DEFUZZIFICATION: Another method of defuzzification is

that of Mean of Maxima (MOM) Method. Here again the

weighted sum and weighted membership are worked out,

except that the membership function is given another alpha

level cut corresponding to the maximum value of the output

fuzzy set. The crisp value for MOM method is given as:

yMax output

xnxMaxy

output

xx n

Σ=

=)

. ......1()(

1

.....1 µµη

39

39



What kind of fuzzy logic we have been discussing?

Mamdani calculus where membership functions of both antecedant

and consequent variables are to be considered at the composition stage.

Mamdani calculus involves computation of the consequent fuzzy

variables. This is not always possible –for real-time systems for

example running at high throughput rates- or not always desirable on

the basis of Occam’s logic; things to be kept simple wherever possible.

So if you can approximate a function with a single variable then this is

better than having a function; when possible the approximation of a

constant is better than having a variable.

40

40

SYSTEMS

Knowledge Representation & Reasoning: The Air-conditioner

Example

Risk Membership Functions

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Risk

Membership Value

Low

Medium

High

Salary Membership Function

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100 120 140

Salary in '000 Euros

Membership Function

Excellent

Good

Poor

Debt Membership Function

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-5 5 15 25 35 45 55 65 75

Debt in '000 Euros

Membership Vlaue

Small

Large

41

41

SYSTEMS


Example


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Risk

Membership Value

Low

Medium

High


0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100 120 140


Membership Function

Excellent

Good

Poor


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-5 5 15 25 35 45 55 65 75

Debt in '000 Euros

Membership Vlaue

Small

Large

Original and alpha-cut Membership Functions

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

1.20

0 0.2 0.4 0.6 0.8 1 1.2

Risk

Membership Value

Low

Alpha_Low

Medium

Alpha_Medium

High

Alpha-High

Salary =95K, Debts=60K

42

42

SYSTEMS


Example


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Risk

Membership Value

Low

Medium

High


0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100 120 140


Membership Function

Excellent

Good

Poor


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-5 5 15 25 35 45 55 65 75

Debt in '000 Euros

Membership Vlaue

Small

Large

Salary =50K, Debts=40KOriginal and alpha-cut Membership Functions

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

1.20

0 0.2 0.4 0.6 0.8 1 1.2

Risk

Membership Value

Low

Alpha_Low

Medium

Alpha_Medium

High

Alpha-High

43

43

FUZZY CONTROL

Control Theory?

•The term control is generally defined as a mechanism used to guide or regulate the operation of a machine, apparatus or constellations of machines and apparatus.

44

44

FUZZY CONTROL

CONTROL THEORY?

•'Feedback control' is thus a mechanism for guiding or regulating the operation of a system or subsystems by returning to the input of the (sub)system a fraction of the output.

45

45

FUZZY CONTROL DEFINITIONS

'Feedback control' is thus a mechanism for guiding or regulating the operation of a system or subsystems by returning to the input of the (sub)system a fraction of the output.

The machinery or apparatus etc., to be guided or regulated is denoted by S, the input by W and the output by y, and the feedback controller by C. The input to the controller is the so-called error signal e and the purpose of the controller is to guarantee a desired response of the output y.

C Sye uw

46

46

FUZZY CONTROL FUZZY CONTROLLERS

•Here are some heuristics for making decisions in a feedback control loop:

System ResponsivenessIF the error is positive (negative)& the change in error is approximately zeroTHEN the change in control is positive (negative);

Reduction in overshootingIF the error is approximately zero & the change in error is positive (negative)THEN the change in control is positive (negative);

Steady State ControlIF the error and change in error are approximately zeroTHEN the change in control is approximately zero.

47

47

FUZZY CONTROL

Balancing the Cartpole

The Cartpole Problem is often used to illustrate the use of fuzzy logic.

Basically, we have a pole of length l, with a mass m at its head and mass M at its base, has to be kept upright. The application of a force F is required to control the pole. These two masses are connected by a weightless shaft. The base can be moved on a horizontal axis.

The angle of the pole in relation to the vertical axis (θ), and the angular velocity (dθ/dt) are two OUTPUT variables

Kruse, R., Gebhardt, J., & Klawonn (1994). Foundations of fuzzy systems. Chichester: John Wiley & Sons Ltd

48

48

FUZZY CONTROL

Control Theory?•Typically, rules contain membership functions for both antecedents and consequent.

Mamdani Controller

•If e(k) is positive(e) and ∆e(k) is positive(∆e) then ∆u(k) is positive (∆u)

Takagi-Sugeno Controllers:

•If e(k) is positive(e) and ∆e(k) is positive(∆e) then ∆u(k) =ααααe(k)+ß ∆e(k);

• αααα and ß are obtained from empirical observations by relating the behaviour of the errors and change in errors over a fixed range of changes in control

49

49


A fuzzy logic controller (FLC) with a rule base is defined by the matrix:

PPZP

PZNZ

ZNNN∆∆∆∆e(k)

PZN

e(k)

where the matrix interrelates the error value e(k) in at a given time k, ∆e(k)denotes the change in error (= e(k) - e(k-1)), and the control change ∆u(k) is defined as the difference between u(k) and u(k-1). The term-sets of the input

and output variables of the FLC error e, error change ∆e and control change ∆uby the linguistic labels negative (N), approximately zero (Z) and positive I(P).

The above FLC matrix can equivalent antecedent/consequent rule set

50

50


A CONTROL PROCEDURE

FIND the firing level of each of the rules ��FUZZIFICATION

FIND the output of each of the rules ��INFERENCE

AGGREGATE the individual rule outputs to obtain the overall system output

��COMPOSITION

OBTAIN a crisp value to be input to the controlled system

�� DEFUZZIFICATION

51

51

FUZZY CONTROL

FUZZY CONTROLLERS- An example

The membership functions for the three elements of the term set for the

error e are given as:

==

≤≤−−=

≥−≤=

µ

+≥=

≤≤+=

≤=

µ

−≤=

≤≤−−=

≥=

µ

01

22)sgn(

2&20

)(

21

20

00

)(

21

02

00

)(

2

2

2

e

ee

ee

e

e

ee

e

e

e

ee

e

e

error

zero

error

positive

error

negative

52

52

FUZZY CONTROL FUZZY CONTROLLERS – Another example

For the case where e(k)= -0.9 and ∆e(k)= 0.2, the level or degree of firing for the 9-rule rule set:

e & ∆ e ττττ (=min {e,∆∆∆∆ e})

Negative Zero Positive Negative Zero Positive Output

Rule1 0.45 & 0 0

Rule2 0.45 & 0.6 0.45

Rule3 0.45 & 0.4 0.4

Rule4 0.55 & 0 0

Rule5 0.55 & 0.6 0.55

Rule6 0.55 & 0.4 0.4

Rule7 0 & 0 0

Rule8 0 & 0.6 0

Rule9 0 & 0.4 0

53

53

FUZZY CONTROL FUZZY CONTROLLERS – Takagi-Sugeno Controllers

According to Yager and Filev, ‘a known disadvantage of the

linguistic modules is that they do not contain in an explicit

form the objective knowledge about the system if such

knowledge cannot be expressed and/or incorporated into fuzzy

set framework' (1994:192).

Typically, such knowledge is available often: for example in

physical systems this kind of knowledge is available in the

form of general conditions imposed on the system through

conservation laws, including energy mass or momentum

balance, or through limitations imposed on the values of

physical constants.

54

54


Tomohiro Takagi and Michio Sugeno recognised two important points:

1. Complex technological processes may be described in

terms of interacting, yet simpler sub processes. This is

the mathematical equivalent of fitting a piece-wise

linear equation to a complex curve.

2. The output variable(s) of a complex physical system,

e.g. complex in the sense it can take a number of input

variables to produce one or more output variable, can

be related to the system's input variable in a linear

manner provided the output space can be subdivided

into a number of distinct regions.

Takagi, T., & Sugeno, M. (1985). ‘Fuzzy Identification of Systems and its

Applications to Modeling and Control’. IEEE Transactions on Systems, Man and

Cybernetics. Volume No. SMC-15 (No.1) pp 116-132.

55

55

FUZZY CONTROL FUZZY CONTROLLERS – Takagi-Sugeno ControllersMamdani style inference:The Bad News: This method involves the computation of a two-dimensional shape by summing, or more accurately integrating across a continuously varying function. The computation can be expensive.

For every rule we have to find the membership functions for the linguistic variables in the antecedents and the consequents;

For every rule we have to compute, during the inference, composition and defuzzification process the membership functionsfor the consequents;

Given the non-linear relationship between the inputs and the output, it is not easy to identify the membership functions for the linguistic variables in the consequent

56

56


Takagi and Sugeno (1985) have argued that in order to develop a generic and simple mathematical tool for computing fuzzy implications one needs to look at a fuzzy partition of fuzzy input space.

In each fuzzy subspace a linear input-output relation is formed. The output of fuzzy reasoning is given by the values inferred by some implications that were applied to an input.

57

57


Takagi and Sugeno have described a fuzzy implication R is of the

format:

R: if (x1 is µA(x1),… xk is µA(xk)) then y = g(x1, …, xk), where:

A zero order Takagi-Sugeno Model

will be given as

R: if (x1 is µA(x1),… xk is µA(xk)) then y = k

58

58



Let the temperature be 5 degrees centigrade:Fuzzification: 5 degrees means that it can be COOL and COLD;Inference: Rules 1 and 2 will fire:

Composition:The temperature is ‘COLD’ with a truth value of µ COLD=0.5 � the SPEED will be k1The temperature is ‘COOL’ with a truth value of µCOOL =0.5 � the SPEED will be k2‘DEFUZZIFICATION’: CONTROL speed is(µ COLD*k1+ µCOOL *k2)/(µ COLD+ µCOOL)=(0.5*0+0.5*30)/(0.5+0.5)=15 RPM

59

59

FUZZY LOGIC & FUZZY SYSTEMS Knowledge Representation & Reasoning: The Air-conditioner Example

Zero Order Takagi Sugeno Controller

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 20 40 60 80 100 120 140

Speed

Membership Function

MINIMAL

SLOW

MEDIUM

FAST

BLAST

60

60

FUZZY LOGIC & FUZZY SYSTEMS Knowledge Representation & Reasoning: The Air-conditioner Example

DEFUZZIFICATION: Comparing the results of

two model identification exercises – Mamdani and

Takagi-Sugeno- we get the following results:

50.0050.00Mean of

Maxima

36.9141.43Centre of Area

Mamdani

(RPM)

Takagi-

Sugeno

(RPM)

Controller

61

61

Neuro-fuzzy models

Learn from the input-output data:

• Data mining;•Machine Learning;

• Neural Networks; }• Genetic Algorithms•Hybrids �� Neuro Fuzzy systems

Jang, Jyh-Shing Roger., Sun, Chuen-Tsai & Mizutani, Eiji. (1997). Neuro-Fuzzy & Soft Computing: A Computational Approach to Learning and Machine Intelligence. Upper Saddle River (NJ): Prentice Hall, Inc. (Chapters 8 and 12)

Soft Computing

62

62

Neuro-fuzzy models

Learn from the input-output data:

• If a soft computing system is able to compute the input-output relationships, then it will LEARN to compute the relationshipsJang, Jyh-Shing Roger., Sun, Chuen-Tsai & Mizutani, Eiji. (1997). Neuro-Fuzzy & Soft Computing: A Computational Approach to Learning and Machine Intelligence. Upper Saddle River (NJ): Prentice Hall, Inc. (Chapters 8 and 12)

63

63

Neuro-fuzzy models:A case study

Consider a first-order Sugeno fuzzy model with two inputs (x & y) and one output (z).

There are two fuzzy rules:

R1: IF x is A1 and y is B1 THEN f1=p1x+q1y+r1

R2: IF x is A2 and y is B2 THEN f2=p2x+q2y+r2

64

64



A1

A2

B1

B2

TT

TT

N

N

∑

x

y

w1

w2

w1

w2

w1f1

w2f2

f

Layer 1 Layer 2

Layer 3 Layer 4

Layer 5

65

65


The operation of a fuzzy system depends on the execution of FOUR major tasks:

Fuzzification, Inference,

Composition, Defuzzification.

The different layers in an adaptive network perform one or more of the tasks

66

66



LAYER 5: The single node in this layer is a fixed node

labelled ∑, which computes the overall output as the summation of all incoming signals

∑∑

∑ ==

i

i

i

ii

i

iiw

fw

fwO_

1,5

67

67


The network below is an adaptive network that is functionally equivalent to a Takagi-Sugeno model.

A1

A2

B1

B2

TT

TT

N

N

∑

x

y

w1

w2

w1

w2

w1f1

w2f2

f

Layer 1 Layer 2

Layer 3 Layer 4

Layer 5

68

68

Notes on Artificial Neural Networks:The fan-ins and fan-outs

–

–+

summation

10 fan-in4

1 - 100 meters per sec.

Asynchronous

firing rate,

c. 200 per sec.

10 fan-out4

1010 neurons with 104 connections and an average of 10 spikes per second = 1015 adds/sec. This is a lower bound on the equivalent computational power of the brain.

69

69

Notes on Artificial Neural Networks:Biological and Artificial NN’s

Weighted

Connections Matrix

Plastic Connections

Node to Node via

Arcs

Synaptic Contact (Chemical and Electrical)

Inter-linkage

Network ArcsAxonsOutput

Network ArcsDendritesInput

Network NodesNeuronsProcessing Units

Artificial Neural

Networks

Biological Neural

Networks

Entity

70

70

A single layer perceptron can carry out a number can perform a number of logical operations which are performed by a number of computational devices.

A learning perceptron below performs the AND operation.

An algorithm: Train the network for a number of epochs(1) Set initial weights w1 and w2 and the threshold θ to set of random numbers; (2) Compute the weighted sum:

x1*w1+x2*w2+ θ(3) Calculate the output using a delta function

y(i)= delta(x1*w1+x2*w2+ θ ); delta(x)=1, if x is greater than zero,

delta(x)=0,if x is less than equal to zero

(4) compute the difference between the actual output and desired output:

e(i)= y(i)-ydesired(5) If the errors during a training epoch are all zero then stop otherwise update

wj(i+1)=wj(i)+ α*xj*e(i) , j=1,2

Notes on Artificial Neural Networks: Rosenblatt’s Perceptron

71

71

Neuro-fuzzy models

Adaptive NetworksA network typically comprises a set of nodes connected by directed links.

Each node performs a static node function on its incoming signals to generate a single node output.

Each link specifies the direction of signal flow from one node to another.

An adaptive network is a network structure whose overall input-output behaviour is determined by a collection of modifiable parameters.

72

72

Neuro-fuzzy modelsNeural Networks 'learn' by adapting in accordance with a training regimen: The network is subjected to particular information environments on a particular schedule to achieve the desired end-result.

There are three major types of training regimens or learning paradigms:

REINFORCEMENT or GRADED

UN-SUPERVISED

SUPERVISED

73

73

Supervised Learning

Our focus will be on supervised learning, particularly networks that learn by using the so-called back-propagation algorithm and comprise hidden layers between the input & the output layers .

74

74

Supervised Learning

A situation in which the network is functioning as an input/output system. The network receives a vector

v � and

emits another, v ′ � . Supervised learning

regimen involves the network being supplied with a sequence of examples

v � � �

v � �( )� v

� � �v � �( )�� v � � � ��( )

of "desireable" or "correct" input/output pairs. For each input

v � the network is

supplied v � , the correct output.

75

75

Other Learning SystemsUNSUPERVISED LEARNING or SELF-ORGANISATION:

Under this regimen a network modifies itself in response to

v � inputs. There are no

v ′ � inputs or a

grade/score (see next page).

Therefore, in unsupervised or self-organisation learning there is no EXTERNAL TEACHER or CRITIC to oversee the learning process.

ENVIRONMENT TEACHER

vector describing

state of the

environment

In an unsupervised regimen there are no specific examples of the function to be learned by the network.

76

76

Rosenblatt’s Perceptrons

•A perceptron computes a binary function of its input. A group of perceptrons can be trained on sample input-output pairs until it learns to compute the correct function.

•Each perceptron, in some model, can function independently of others in the group, they can be separately trained – linearly separable.

•Thresholds can be varied together with weights.

•Given values of x1 and x2 to train such that the perceptron outputs 1 for white dots and 0 for black dots.

77

77

Back-propagation Algorithm: Supervised Learning

Backpropagation (BP) is amongst the ‘most popular algorithms for ANNs’: it has been estimated by Paul Werbos, the person who first worked on the algorithm in the 1970’s, that between 40% and 90% of the real world ANN applications use the BP algorithm. Werbos traces the algorithm to the psychologist Sigmund Freud’s theory of psychodynamics. Werbos applied the algorithm in political forecasting.

•David Rumelhart, Geoffery Hinton and others applied the BP algorithm in the 1980’s to problems related to supervised learning, particularly pattern recognition.

•The most useful example of the BP algorithm has been in dealing with problems related to prediction and control.

78

78

Back-propagation Algorithm :Worked Example #1

−−=

3.01.0

1.02.0

2414

2313

ww

ww

[ ] [ ]2.01.01.0050403=www

Consider a 2x2x1 network:

5

42

1 3

First Layer Connectivity

Bias weight

Second Layer Connectivity

[ ] [ ]3.02.04535=ww

The desired vector d=0.9

79

79



5

42

1 3

[ ] [ ]9.01.021=xx

Consider an input vector x:

d=0.9

80

80



5

42

1 3

d=0.9

Fuzzy Logic and Fuzzy Systems - Trinity College Dublin

Documents

Transcript of Fuzzy Logic and Fuzzy Systems - Trinity College Dublin