SOFT COMPUTING FUNDAMENTS

á

1

OUTLINE

• Introduction to Soft Computing

• Probabilistic Computing (PC)

• Fuzzy computing (FC)

▫ Fuzzy Theory

▫ Fuzzy Systems

• Neural computing (NC) ▫ Artificial Neural Networks

• Evolutionary Computing (EC)

▫ Genetic Algorithms

• Summary

2

INTRUDUCTION

The idea of soft computing started with Lofti A. Zadeh in 1981. Soft Computing is the fusion of methodologies designed to model and enable solutions to real world problems. * Basically, soft computing is not a homogeneous body of concepts and techniques. *The role model for soft computing is the human mind

3

“The dominant aim of soft computing is to exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness, low solution-cost, and better raptor with reality .” (Zadeh,1981)

Rather than a precise definition for soft computing, it is instead defined by extension.

4

5

AI

SOFT COMPUTING

Machine Learning

Symbolic manipulation

Automatic improvement with experience

Cognitive Psychology

Study of the mind

Statistics

Probability (not possibility)

Uncertainty and Imprecision

Probabilistic Computing (PC)

• Supervised learning

▫ Classification problems

▫ Regression problems

• Unsupervised learning

▫ Clustering

▫ Dimensionality reduction.

*Bayes Theory

*Dampster-Shafer Theory

6

Supervised Learning

• Inputs • Outputs

Training Set

• Linear regression, polynomial regression, logistic regression, etc.

• Optimization functions

Learning algorithm

• Obtain probable solution.

Hypothesis

7

Input

Output

Generate a hypothesis that classifies the data.

A decision boundary can be established using linear regression.

Optimization Algorithms:

• Gradient descendent.

8

X1

X2

Unsupervised Learning

• Input • Not Feedback

DATA

• Generate a probabilistic model.

• Bayes Networks…

Learning algorithm • Obtain

probable solution.

Model

9

Input

Output

There is not correct answer.

There are not parameters to feet.

The only data available is the input

10

X1

X2

In general, the true distribution of the data is unknown

Bayes Decision Theory

Belief Theory:

• 𝑃 𝑦 𝑥 =𝑃 𝑥 𝑦 𝑃(𝑦)

𝑃(𝑥) Classification

• From the Bayes rule is assumed to have a models m where

11

Ethem Alpaydin, 2004

Bayesian Network

• Belief networks or probabilistic networks

• Used as graphical models to represent the interaction between variables.

12

Taken from Wikipedia

Fuzzy computing (FC)

• Introduced by Lofti Zadeh in 1965, as the fuzzy set theory.

13

Human Think

• Vague • Inexact

Nature

• Uncertain • Imprecise

Linguistic Knowledge

• Not quantifiable

Ambiguous

Probabilistic Probable truth

Imprecisely defined

REAL WORLD

Fuzzy theory

Imprecise definition

Membership functions

• Created by experts

Rules • Qualificative

Probable truth

14

Try to reproduce the reasoning to make a decision.

15

Fuzzy set is an extension of classical set theory where elements have degrees of membership.

Classical theory

𝜇𝐴 = 1 𝑥𝜖𝑋0 𝑒𝑙𝑠𝑒

Fuzzy theory

𝜇𝐴 = 𝜇𝐴 𝑥 𝑥𝜖𝑋0 𝑒𝑙𝑠𝑒

Fuzzy Systems

16

• Fuzzification:

With an input variable – find the numeric values of the membership function.

17

18

Fuzzy inference: 1. Using linguistic rules, determine with the inputs which rules apply. 2. Determining conclusions.

• Desfuzzification:

Operation to obtain the most certain output.

19

NEURAL COMPUTING

• Artificial Neural Networks (ANN):

Non-linear approximation

Emulate a simplified biological neuron.

Mathematical or computational model.

Advantage:

Size is not directly related with the number of features.

20

Biological Neuron

21

INPUTS

OUTPUT

McCulloch-Pitts Neuron

22

𝑦𝑖

𝑢𝑖 = 𝑤𝑖𝑗𝑥𝑗

𝑗

𝑦𝑖 = 𝑓(𝑢𝑖) INPUTS

OUTPUT 𝑥1

𝑥𝑛

𝑥𝑗

……

…

…

𝑤1

𝑤𝑗

Synaptic Weights

𝑤𝑛

Activation function

ɸ

Threshold

)()( i

j

iijii xwfty

𝜃𝑖

Bias

1

• Weight factors, Wj

Determine the strain of the input vector.

• Activation Function:

Describes the relationship of input-output.

Could be a threshold function, sigmoid function(smooth), piecewise linear function, tangent hyperbolic function…

23

24

Perceptron: example AND

I1

I2

W1

W2

2

1i

iiIWx O O = x

1

+

F(x) = x

0 0

0 1

1 1

1 0

I1 I2

0

0

1

0

I1 AND I2

-1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

Single layer feed-forward network

25

Multi Layer Feed-forward netowrk

26

LEARNING METHODS IN ANN

27

ANN Learning algorithms

Supervised Learning

Stochastic Gradient descent

EMS Back

propagation

Reinforced Learning

Unsupervised Learning

Hebbian Competitive

Back propagation algorithm

• The artificial neurons are organized in layers, and send their signals “forward”, and then the errors are propagated backwards.`

• The idea of the backpropagation algorithm is to reduce the error in the output.

• The backpropagation algorithm calculates how the error depends on the output, inputs, and weights

28

Classification of ANN

29

Optimization

Optimization Methods

Linear Programming

Non-Linear Programming

Classical Methods

Newton, Fibonacci

Enumerative Methods

Min-max algorithms

Stochastic Methods

EA, GA,SA

30

Evolutionary Computing

• Evolutionary algorithms (EA) are population-based metaheuristics optimization algorithms.

• Use biology-inspired mechanisms like mutation, crossover, natural selection, and survival of the fittest.

• Is more robust than other techniques in cases of noise, or input changes.

31

Basic cycle of evolutionary computing

32

Weise Thomas (2009)

A basic evolutionary algorithm

33

GENETIC ALGORITHMS

Is a subclass of Evolutionary algorithms.

• Mimic natural process as: selection, crosses over, mutation and accepting

• Random search

• Adaptive heuristic search.

• Survival of the fittest

34

Basic process

35

GENETIC OPERATORS

ADJUSTMENT AND EVALUATION

SELECTION

Basic genetic algorithm

• 1. Generate random population

• 2.Evaluate fitness f(x) for each chromosome x

• 3. New Population.

▫ A. Selection select parents – best fitness

▫ B. Crossover crossover the parents –new offspring child

▫ C. Mutation using mutation probability – mutate new offspring

• 4. Raplace

• 5. Test

36

37

Summary

• Soft computing techniques deal with the uncertainty and imprecision of the real world, using algorithms that in contrast to hard computing can treat this problems with robustness and generate low cost solutions.

• Systems based in probabilistic computing and fuzzy logic are approximation to the reasoning, so that it can be obtained a similar model to the real one but not exactly the same.

• Neural networks and evolutionary algorithms create functional approximations, for unknown systems, or very complex systems that can’t be mathematically described.

38

References

• J.-S.R. Jang, K. Goebel, Neuro-Fuzzy and Soft Computing.

• Ethem Alpaydin (2004), Introduction to Machine Learning, MIT Press, Cambridge, Massachustts.

• http://www.uncertainty-in-engineering.net/uncertainty_models/fuzziness

• Weise Thomas (2009), Global Optimization Algorithms, Theory and Application, http://www.it-weise.de/

39