SOFT COMPUTING INTRODUCTION
-
Upload
harshali-y-patil -
Category
Documents
-
view
245 -
download
9
description
Transcript of SOFT COMPUTING INTRODUCTION
8/5/2014
1
INTRODUCTION TO SOFT COMPUTING
Harshali Patil
Introduction
� Evolution of Computing
� Soft computing constituents
� From conventional AI to Computational Intelligence
� Machine Learning basics
8/5/2014
2
Problem Solving
2) How to reuse the previous knowledge?
3) Proposed solution is
valid or not?
1) What kind of similar
problems you have solved?
4) Is it necessary to
retain the knowledge?
New
Problem
Reasoning Solution
Case Based Reasoning
8/5/2014
3
Artificial Intelligence
� If intelligence can be induced in machines it is called as artificial intelligence.
� Soft computing is a part of artificial intelligent techniques
� Closed related to machine intelligence/computational intelligence
Soft Computing
Neural Networks
Fuzzy Inferencesystems
Neuro-Fuzzy
Computing
Derivative-Free
OptimizationSoft ComputingSoft ComputingSoft ComputingSoft Computing
+ =
8/5/2014
4
What is Soft Computing?
� The idea behind soft computing is to model cognitive behaviour of human mind.
� Soft computing is foundation of conceptual intelligence in machines.
� Unlike hard computing , Soft computing is tolerant of imprecision, uncertainty, partial truth, and approximation.
Soft Computing
AccordingAccordingAccordingAccording totototo ProfProfProfProf.... ZadehZadehZadehZadeh::::
"...in contrast to traditional hardcomputing, soft computing exploits thetolerance for imprecision, uncertainty, andpartial truth to achieve tractability,robustness, low solution-cost, and betterrapport with reality”
8/5/2014
5
Soft Vs Hard Computing
Symbolic Logic
Reasoning
Traditional NumericalModeling and Search
Approximate Reasoning
Functional Approximationand Randomized
Search
HARD COMPUTING SOFT COMPUTING
Precise Models Approximate Models
Hard ComputingHard ComputingHard ComputingHard Computing Soft ComputingSoft ComputingSoft ComputingSoft Computing
Conventional computing requires a
precisely stated analytical model.
Soft computing is tolerant of
imprecision.
Often requires a lot of computation
time.
Can solve some real world problems in
reasonably less time.
Not suited for real world problems for
which ideal model is not present.
Suitable for real world problems.
It requires full truth Can work with partial truth
It is precise and accurate Imprecise.
High cost for solution Low cost for solution
8/5/2014
6
Overview techniques of SC
Neural Networks
Fuzzy Logic
Genetic Algorithm
Hybrid Systems
Evolution of Computing
1940s 1947
Cybemetics
1943 McCulloach
Pitts neuron model
1950s 1956 AI 1957 Perceptron
1960s 1960 LISP
language
1960 Adaline
Masaline
1965 Fuzzy
sets
1970s Mid 1970s
knowledge
engineering
(expert system)
1974 Birth of back
propogation
algorithm
1975 Cognitron
Neocognitron
1974 Fuzzy
controller
1970s
Genetic
algorithm
1980s 1980 self organizing
map
1982 hopfield net
1983 Boltzmann
machine
1986
Backpropogation
algorithm boom
1985 Fuzzy
modelling
Mid 1980s
Artificial life
Immune
modelling
1990s 1990s Neuro
fuzzy modelling
1991 ANFIS
1990 Genetic
programming
8/5/2014
7
SC constituents (the first three items) and conventional AI
MethodologyMethodologyMethodologyMethodology StrengthStrengthStrengthStrength
Neural network Learning and adaptation
Fuzzy set theory Knowledge representation via fuzzy if-then rules
Genetic algorithm and simulated annealing
Systematic random search
Conventional AI Symbolic manipulation
Character recognizer
8/5/2014
8
Features of Conventional AI
�Conventional AI manipulates symbols on the assumption that human intelligence behavior can be stored in symbolically structured knowledge bases: this is known as: “ The physical symbol system hypothesis”
�The knowledge-based system (or expert system) is an example of the most successful conventional AI product
What is expert system?
� An expert systemexpert systemexpert systemexpert system is software that uses a knowledge base of human expertise for problem solving, or to clarify uncertainties where normally one or more human experts would need to be consulted
8/5/2014
9
Expert system
Building blocks of expert system
� Knowledge base: Knowledge base: Knowledge base: Knowledge base: factual knowledge and heuristic knowledge
� Knowledge representation: Knowledge representation: Knowledge representation: Knowledge representation: in the form of rules
� Problem solving model: Problem solving model: Problem solving model: Problem solving model: forward chaining or backward chaining
� Note:-
� Knowledge engineering:- building an expert system
� Knowledge engineers:- practitioners.
8/5/2014
10
Definitions of AI
� “AI is the study of agents that exists in an environment and perceive and act” [S. Russel & P. Norvig]
� “AI is the act of making computers do smart things” [Waldrop]
� “AI is a programming style, where programs operate on data according to rules in order to accomplish goals” [W.A. Taylor]
Definitions of AI
� “AI is the activity of providing such machines as computers with the ability to display behavior that would be regarded as intelligent if it were observed in humans” [R. Mc Leod]
� “Expert system is a computer program using expert knowledge to attain high levels of performance in a narrow problem area” [D.A. Waterman]
8/5/2014
11
Definitions of AI
� “Expert system is a caricature of the human expert, in the sense that it knows almost everything about almost nothing” [A.R. Mirzai]
� AI is changing rapidly, these definitions are already obsolete!
Applications of expert system
� Diagnosis and Troubleshooting of Devices and Systems of All Kinds
� Planning and Scheduling
� Configuration of Manufactured Objects from Subassemblies
� Financial Decision Making
� Knowledge Publishing
� Design and Manufacturing
8/5/2014
12
If the facts don't fit the If the facts don't fit the If the facts don't fit the If the facts don't fit the theory, change the facts. theory, change the facts. theory, change the facts. theory, change the facts.
- Albert Einstein
GEMS CBR for Remote Diagnostics
Information
System
Servers
GE RegionalService Team
DiagnosticsSpecialist
ServiceEngineer
Phone, E-mail
FAX, Web
On-site Monitoring
Remote Data
Access
PartsSpecialist
TechnicalAnswer Center
• Call Management/ Commitment Tracking System
• Monitoring & Diagnostics
• Problem/Solution DB• CBR using DB
8/5/2014
13
Machine Learning basics
� Machine learning, a branch of artificial intelligence, concerns the construction and study of systems that can learn from data.
� E.g. a machine learning system could be trained on email messages to learn to distinguish between spam and non-spam messages. After learning, it can then be used to classify new email messages into spam and non-spam folders.
Research Areas
Intelligent RobotsIntelligent RobotsIntelligent RobotsIntelligent Robots� Bayesian inference and design technique for uncertain scene recognition�Combination Image filtering and Bayesian inference�Navigation technique research for autonomous mobile robot�Evolving a mobile robot controller
8/5/2014
14
Research Areas
Intelligent AgentIntelligent AgentIntelligent AgentIntelligent Agent� Intelligent virtual secretary agent�Conversational agent�Intelligent assistants for smart phone service
Research Areas
BioinformaticsBioinformaticsBioinformaticsBioinformatics�Bioinformatics: the collection, classification, storage, and analysis of biochemical and biological information using computers especially as applied in molecular genetics and genomics�Classification techniques in Bioinformatics�Agent driven virtual cell modelling
8/5/2014
15
Research Areas
UbiquitousUbiquitousUbiquitousUbiquitous�Developing an adaptation scheme in the context of middleware and applications�Developing context-aware system for ubiquitous systems�Developing basic theories and algorithms of the advanced intelligent models for ubiquitous environment
Research Areas
Intrusion Detection System (IDS)Intrusion Detection System (IDS)Intrusion Detection System (IDS)Intrusion Detection System (IDS)�HMM-based intrusion detection system�Generation of various intrusion patterns using interactive genetic algorithm�Viterbi algorithm for intrusion type identification�Rule-based integration of multiple measure-models
8/5/2014
16
Research Areas
BiometricsBiometricsBiometricsBiometrics�Analysis and evaluation techniques of fingerprint recognition system�Development of classification and matching algorithm for fingerprint recognition
Neuro Fuzzy and Soft Computing Characteristics
� With NF modeling as a backbone, SC can be characterized as:
� Human expertise (fuzzy if-then rules)
� Biologically inspired computing models (NN)
� New optimization techniques (GA, SA, RA)
� Numerical computation (no symbolic AI so far, only numerical)
� New application domains: mostly computation intensive like adaptive signal processing, adaptive control, nonlinear system identification etc
8/5/2014
17
Neuro Fuzzy and Soft Computing Characteristics
� Model free learning:-models are constructed based on the target system only
� Intensive computation: based more on computation
� Fault tolerance: deletion of a neuron or a rule does not destroy the system. The system performs with lesser quality
� Goal driven characteristics:- only the goal is important and not the path.
� Real world application:- large scale, uncertainties
Neural Network
� DARPA Neural Network Study (1988, AFCEA DARPA Neural Network Study (1988, AFCEA DARPA Neural Network Study (1988, AFCEA DARPA Neural Network Study (1988, AFCEA International Press, p. 60): International Press, p. 60): International Press, p. 60): International Press, p. 60):
... a neural network is a system composed of many simple processing elements operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at computing elements or nodes.
8/5/2014
18
Definition of Neural Network
� According to According to According to According to HaykinHaykinHaykinHaykin (1994), p. 2: (1994), p. 2: (1994), p. 2: (1994), p. 2:
A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:
� Knowledge is acquired by the network through a learning process.
� Interneuron connection strengths known as synaptic weights are used to store the knowledge
� According to According to According to According to NigrinNigrinNigrinNigrin (1993), p. 11: (1993), p. 11: (1993), p. 11: (1993), p. 11:
A neural network is a circuit composed of a very large number of simple processing elements that are neurally based. Each element operates only on local information.
Furthermore each element operates asynchronously; thus there is no overall system clock.
8/5/2014
19
� According to According to According to According to ZuradaZuradaZuradaZurada (1992): (1992): (1992): (1992):
Artificial neural systems, or neural networks, are physical cellular systems which can acquire, store and utilize experiential knowledge.
Multi disciplinary view of Neural Networks
8/5/2014
20
Fuzzy Logic
� Origins: Multivalued Logic for treatment of imprecision Origins: Multivalued Logic for treatment of imprecision Origins: Multivalued Logic for treatment of imprecision Origins: Multivalued Logic for treatment of imprecision and vagueness and vagueness and vagueness and vagueness
� 1930s: Post, Kleene, and Lukasiewicz attempted to represent undetermined, unknown, and other possible intermediate truth-values.
� 1937: Max Black suggested the use of a consistency profile to represent vague (ambiguous) concepts.
� 1965: Zadeh proposed a complete theory of fuzzy sets (and its isomorphic fuzzy logic), to represent and manipulate ill-defined concepts.
FUZZY LOGIC – LINGUISTIC VARIABLES
� Fuzzy logic gives us a language (with syntax and local semantics) in which we can translate our qualitative domain knowledge.
� Linguistic variables to model dynamic systems
� These variables take linguistic values that are characterized by:
� a label - a sentence generated from the syntax � a meaning - a membership function determined by a local
semantic procedure
8/5/2014
21
Linguistic variables
� Linguistic variables associate a linguistic condition with a crisp variable.
� A crispcrispcrispcrisp variable is the kind of variable that is used in most computer programs: an absolute value.
� A linguistic variablelinguistic variablelinguistic variablelinguistic variable, on the other hand, has a proportional nature: in all of the software implementations of linguistic variables, they are represented by fractional values in the range of 0 to 1.
Linguistic variables
Linguistic variables in soup instructions
8/5/2014
22
An old friend comes into your shop asking to buy a few widgets, and wants your best price. The onus is on you to come up with a price given many parameters. Taking this hypothetical case we need to account for:
• Cost of the widgets • Normal markup • Shelf time of the product • Shelf life of the product • Length of the relationship • Customer payment history • Quantity of the sale • Repeat business potential
Parameters to consider pricing a widgetParameters to consider pricing a widgetParameters to consider pricing a widgetParameters to consider pricing a widget
8/5/2014
23
Figure: Linguistic variable HOTFigure: Linguistic variable HOTFigure: Linguistic variable HOTFigure: Linguistic variable HOT
8/5/2014
24
Figure: F_OR operator (Fuzzy OR)
Figure: F_EQ (Fuzzy equal)
8/5/2014
25
FUZZY LOGIC – REASONING METHODS
� The meaning of a linguistic variable may be interpreted as an elastic constraint on its value.
� These constraints are propagated by fuzzy inference operations, based on the generalized modus-ponens.
� An FL Controller (FLC) applies this reasoning system to a Knowledge Base (KB) containing the problem domain heuristics.
� The inference is the result of interpolating among the outputs of all relevant rules.
� The outcome is a membership distribution on the output space, which is defuzzified to produce a crisp output.
� There are two consistent logical argument constructions: modus ponens ("the way that affirms by affirming") and modus tollens ("the way that denies by denying"). Here are how they are constructed:� Modus Ponens: "If A is true, then B is true. A is true. Therefore, B is true."� Modus Tollens: "If A is true, then B is true. B is not true. Therefore, A is
not true.“
� There are two related incorrect and inconsist constructions: affirming the consequent and denying the antecedent.� Affirming the Consequent: "If A is true, then B is true. B is true.
Therefore, A is true."� Denying the Antecedent: "If A is true, then B is true. A is not true.
Therefore, B is not true."
8/5/2014
26
Example
Here is a sensible example, illustrating each of the above:
� "If it is a car, then it has wheels. It is a car. Therefore, it has wheels." (Modus Ponens -CORRECT)
� "If it is a car, then it has wheels. It does not have wheels. Therefore, it is not a car." (Modus Tollens -CORRECT)
� "If it is a car, then it has wheels. It has wheels. Therefore, it is a car." (Affirming the Consequent -INCORRECT.)� Comment: why is this incorrect? Well, the thing might
have wheels but that doesn't mean it has to be a car. It might be a cart, or rollerblades, or a moped. It doesn't have to be a car.
� "If it is a car, then it has wheels. It is not a car. Therefore, it does not have wheels." (Denying the Antecedent - INCORRECT)� Comment: why is this incorrect? Consider
the argument for the "affirming the consequent" example. Rollerblades are not cars, but they DO have wheels.
8/5/2014
27
Genetic algorithm
EVOLUTIONARY PROCESS
Steps involved in Genetic Algorithm
� The genetic algorithms follow the evolution process in the nature to find the better solutions of some complicated problems. Foundations of genetic algorithms are given in Holland (1975) and Goldberg (1989) books.
� Genetic algorithms consist the following steps:
� Initialization� Selection� Reproduction with crossover and mutation
� Selection and reproduction are repeated for each generation until a solution is reached.
� During this procedure a certain strings of symbols, known as chromosomes, evaluate toward better solution.
8/5/2014
28
Hybrid Systems
� Hybrid systems enables one to combine various soft computing paradigms and result in a best solution. The major three hybrid systems are as follows:
� Hybrid Fuzzy Logic (FL) Systems
� Hybrid Neural Network (NN) Systems
� Hybrid Evolutionary Algorithm (EA) Systems
Applications of Soft Computing
� Handwriting Recognition� Image Processing and Data Compression� Automotive Systems and Manufacturing� Soft Computing to Architecture� Decision-support Systems� Soft Computing to Power Systems� Neuro Fuzzy systems� Fuzzy Logic Control� Machine Learning Applications� Speech and Vision Recognition Systems� Process Control and So on
8/5/2014
29