13 Fuzzy Systems and Computational Intelligence -...

13 Fuzzy Systems andComputational Intelligence

Fuzzy Systems EngineeringToward Human-Centric Computing

13.1 Computational intelligence

13.2 Recurrent neurofuzzy systems

13.3 Genetic fuzzy systems

13.4 Coevolutionary hierarchical genetic fuzzy system

13.5 Hierarchical collaborative relations

13.6 Evolving fuzzy systems

Contents

Pedrycz and Gomide, FSE 2007

13.1 Computational intelligence



Fuzzy set theory

Neuralnetworks

Evolutionary systems

Neuralfuzzy

systems

GeneticFuzzy

systems

Neuralevolutionary

systems

Computational Intelligence

Evolvingsystems

Swarmintelligence

Immunesystems

Computational intelligence

Data processing systems with capabilities of (Bezdek, 1992/1994)

– pattern recognition– adaptive– fault tolerance– performance approximates human performance– no use of explicit knowledge

Framework to design and analyze intelligent systems (Duch, 2007)

– autonomy– learning– reasoning



Computing systems able to (Eberhart, 1996)

– learn– deal with new situations using

• reasoning• generalization• association• abstraction• discovering


– largely human-centered– forms of artificial and synthetic intelligence– collaboration man-machine


Machine learning

Artificial intelligence

Intelligent Systems

Systems science

Control theory

Cognitivesciences

Data analysis recognition

Operationsresearch


13.2 Recurrent neurofuzzysystems



Globally recurrent

– full feedback connections

Partially recurrent

– partial feedback connections


Recurrent neural fuzzy network model

• 11kA

11kA

11kA

• ikiA

ikiA

ikiA

• nNnA

nNnA

NnnA

and

and

and

x1

xi

y1

yk

ym

aj

aji

aj

wji

wjn

wj1 vk1

vkj

vkM

z1

zj

zM xn

r11

rjj

rMM


Recurrent and fuzzy neuron

zj = AND(aj;wj)

)(1

jiji

Mn

ij aswTz

+

==

and

aj1

aji

ajn

wji

wjn

wj1 rj1

rjj

ψand

z-1

rjM

zj


Ni number of fuzzy sets that granulate the ith input

j indexes and neurons; given ki , j is found using

x1, ..., xi,..., xn inputs

−+= ∏∑

−

=−+

=+−

1

1)1(

2)1( )1(

i

rrn

M

iinn Nkkj

)( ikiji xAa i=

wji weight between ith input and jth and neuron

zj output of the jth and neuron

vkj weight jth input of the kth output neuron

rjl feedback connection of the lth input of the jth and neuron

yk = ψ(uk) output kth neuron of the output layer


∑=

ψ=ψ=M

jjkjk zvuy

1)()(

z1

zn

zi

vk1

vkM

vkj ∑ ykψuk

Output layer neuron


Learning algorithm

procedure NET-LEARNING (x, y) returns a network

input:data x, ylocal: fuzzy sets

t, s: triangular normsε: threshold

GENERATE-MEMBERSHIP-FUNCTIONSINITIALIZE-NETWORK-WEIGHTS

until stop criteria ≤ ε dochoose and input-output pair x and y of the data setACTIVE-AND-NEURONSENCODINGUPDATE-WEIGHTS

return a network


Example

Chaotic NH3 laser time series data

• first 1000 samples for learning• predict next 100 steps


−−− Actual ----- NFN

100 steps ahead prediction


Normalized squared forecasting errors (NSE) NH3 laser time series

100 steps ahead

FIR 0.0230 0.0551

NFN 0.0139 0.0306

Model 1 step ahead

2

12

)ˆ(1

∑=

−σ

=N

iii yy

NNSE


13.3 Genetic fuzzy systems


Genetic fuzzy systems

GFS is an approach to design fuzzy models and systems

GFS = fuzzy system + learning using genetic algorithm

Learning of models structure and parameters

– rule base– fuzzy rules– membership functions– operators– inference procedures


Genetic Fuzzy Systems

Encoder Decoder

Processing

Knowledgebase

Geneticalgorithms

Learning

Fuzzy processing

Genetic design


13.4 Coevolutionary hierarchicalgenetic fuzzy system


Coevolution

Considers interactions between population members

Populations hierarchically structured

Hierarchy levels associated with partial solutions of the problem

– individuals of different populations keep collaborative relations– collaboration depends on the fitness of the individuals– hierarchical levels:

• I : membership functions• II : fuzzy rules• III: rule bases• IV: fuzzy systems (models)


Coevolutionary GFS approach


13.5 Hierarchical collaborativerelations


(a)

Collaboration between species


(b)

Collaboration between individuals

Rj: If x1 is A1j and ...and xn is An

j then y = g(wj,x)Pedrycz and Gomide, FSE 2007

Fitness evaluation in hierarchical collaborative evolution


Example: function approximation

Rj: If x1 is A1j and ...and xn is An

j then y = g(wj,x)

and = t-norm

))(1( abbapp

abbta

tt −+−+=

pt : obtained by coevolution

g(wj,x) : least squares + pruning


3.0,0],1,0[

)])7()exp()6.0(13[)exp(35.1(9.1),(

),(),(),(

:

222

11211

211211

1

=σ==Ω−−+=

σ+=→Ω

m

xsinxxsinxxxf

mNxxfxxF

RF

Original function Training dataPedrycz and Gomide, FSE 2007

f1

f1


Result

A11 A2

1 A31

x1

A12 A2

2

x2

A11

x1

A21

A31

x1

A12 A2

2

A32

Partitions

CoevolGFS

ANFIS


RME for function approximation example

Approach Training RME Test RMENumber of

Rules

CoevolGFS 0.25 0.13 8

ANFIS 0.32 0.21 9


Example: classification

Intertwined spirals


2122

2121212

2122

2121213

2122

2121212

2122

2121211

34.028.025.024.10.214.1thenhighishighisIf:

45.078.008.06.72.22.17thenhighsmediumisIf:

46.084.005.07.73.13.15thenlowismediumisIf:

1.017.034.026.06.131.0thenlowislowisIf:

xxxxxxyxandxR

xxxxxxyixandxR

xxxxxxyxandxR

xxxxxxyxandxR

−−−++=

+−−+−−=

−+−+−=

−++−+−=

Classification rules


Approach Cycles MisclassificationNumber of

Rules

CoevolGFS 529 18 9

ANFIS 1000 0.21 9

Classification performance: Intertwined spirals



13.6 Evolving fuzzy systems

Evolving fuzzy systems

Evolving systems: an approach to develop adaptive fuzzy models

Evolving modeling targets nonstationary process and systems

Main properties

– inherit new knowledge– gradual changes– life-long learning– self organization of the system structure– complements GFS approach– may act online



Rule base evolution

Input Output

Data Data

Real-time update (potential -based)

Rule-base

Recursive clustering


Model(update)

Learningprocess

(modeling)

Arousalmechanism

(interpretation)

Data

ρ a

Participatory learning

(Details in Chapter 14)Pedrycz and Gomide, FSE 2007

Functional fuzzy models


∑=

+=n

jjijiiii xaaythenisifR

10: Ax

])(exp[)( 2ijjijj

ij vxkxA −−=

∑=

=c

iii ywy

1

∑=

=c

ii

ii

xλ

xλxw

1)(

)()(

)()()( 2211 nin

iii xAttxAtxA L=λ


procedure EVOLVE-PARTICPATORY- LEARNING (x,y) returns an outputinput : data x,ylocal: antecedent parameters

consequent parameters

INITIALIZE-RULES-PARAMETERSdo forever

read xPL-CLUSTERINGUPDATE-RULE-BASERUN-LEAST-SQUARES(x,y)COMUTE-RULE-ACTIVATIONCOMPUTE-OUTPUT

return y

Evolving participatory learning algorithm

Example

Time series forecasting


Average weekly inflows of a power plant


Estimated partial correlation


2

1)(

1∑=

−=P

k

kd

k xxP

RMSE

||1

1∑=

−=P

k

kd

k xxP

MAD

∑=

−=P

kkd

kd

k

x

xx

PMRE

1

||100

−=kd

kd

k

x

xxRE

|max100max

∑ ∑

∑

= =

=

−−

−−=ρ

P

k

P

k

kd

kd

P

k

kd

kd

xxxx

xxxx

1 1

22

1

)()(

))((

Performance measures


Result


ErrorModels

ePL eTS

RMSE (m3/s) 378.71 545.28

MAD (%) 240.55 356.85

MRE (%) 12.54 18.42

REmax (%) 75.51 111.22

ρ 0.95 0.89

Number of rules 2 2

Forecasting performance average weekly inflow

13 Fuzzy Systems and Computational Intelligence -...

Documents

Transcript of 13 Fuzzy Systems and Computational Intelligence -...