Lecture1: An Introduction to Neural Nets and Fuzzy...
Transcript of Lecture1: An Introduction to Neural Nets and Fuzzy...
![Page 1: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/1.jpg)
1
In the Name of God
Lecture1: An Introduction to Neural Nets and Fuzzy Systems
![Page 2: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/2.jpg)
AI …
• AI: Applying nature-inspired computations in i i th tiengineering, mathematics …
• Bioinformatics: Applying engineering and mathematical tools for biological problemsg p
• AI computing tool▫ Artificial neural networks▫ Artificial neural networks▫ Fuzzy systems▫ Genetic algorithms
S i t lli▫ Swarm intelligence▫ …
![Page 3: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/3.jpg)
Fuzzy Systems and logic
• real-world systems are not crisp• uncertainty in the system• uncertainty in the system• fuzzy approach as a solution• conventionally applied to Control problems• often based on a number of rules• often based on a number of rules• for example:
▫ If it's Sunny and Warm, drive FastSunny(Cover) Warm(Temp) Fast(Speed)Sunny(Cover)Warm(Temp) Fast(Speed)
▫ If it's Cloudy and Cool, drive SlowCloudy(Cover)Cool(Temp) Slow(Speed)
Dri ing Speed is the combination of o tp t of these▫ Driving Speed is the combination of output of these rules...
▫ How fast will I go if it is 21 C° 25 % Cloud Cover ?
![Page 4: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/4.jpg)
Indeed
• Fuzzy Logic provides way to calculate with y g p yimprecision and vagueness
• Fuzzy Logic can be used to represent some kinds of human expertise
• We use Fuzzy Membership SetsW F Li i ti V i bl• We use Fuzzy Linguistic Variables
• We use Fuzzy AND and OR
![Page 5: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/5.jpg)
Fuzzy system applicationsy y
• Pattern recognition and classificationg• Fuzzy clustering• Image and speech processing• Fuzzy systems for prediction• Fuzzy control• Monitoring• Diagnosis• Optimization and decision making• Group decision making• …
![Page 6: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/6.jpg)
Fuzzy system applicationsy y
Vehicle Control
A number of subway systems, particularly in Japan and Europe are using fuzzyEurope, are using fuzzy systems to control braking and speed. One example is thespeed. One example is the Tokyo Monorail
![Page 7: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/7.jpg)
Fuzzy system applications
Appliance control systems
y y
Appliance control systems
• Fuzzy logic is starting to be used to help control y g g pappliances ranging from rice cookers to small-scale microchips (such as the Freescale 68HC12).
• You may have heard ofi t lli t hi hi▫intelligent washing machine▫intelligent refrigerator▫intelligent▫intelligent ....
• Many of them are based on fuzzy logic
![Page 8: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/8.jpg)
Another Example
In order to illustrate some basic concepts in Fuzzy Logic, consider a simplified example of a thermostat controlling a heater fan illustrated in the Figure.
The room temperature detectedThe room temperature detected through a sensor is input to a controller which outputs a control force to adjust the heater fanforce to adjust the heater fan speed.
![Page 9: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/9.jpg)
Conventional Thermostat
• A conventional thermostat works like an on-off switch.
• If we set it at 28oC then the heater is activated only when the temperature falls below 24oC .
• When it reaches 32oC the heater is turned off. A lt th d i d t t i• As a result the desired room temperature is either too warm or too hot.
![Page 10: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/10.jpg)
Fuzzy Thermostaty
• A fuzzy thermostat works in shades of gray where the temperature is treated as a series of overlapping ranges.
• For example 28oC is 60% warm and 20% hot• For example, 28oC is 60% warm and 20% hot. The controller is programmed with simple if-then rules that tell the heater fan how fast to run.
• As a result, when the temperature changes the fan speed will continuously adjust to keep the temperature at the desired leveltemperature at the desired level.
![Page 11: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/11.jpg)
conventional vs fuzzy thermostaty
28oC
![Page 12: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/12.jpg)
fuzzy decision makingy g
• It can be difficult to distinguish between various goals and categories at times▫ Is a goal in an e-commerce decision hard or soft?▫ When is a restaurant crowded, or only slightlyWhen is a restaurant crowded, or only slightly
crowded?• There have been many projects in which fuzzy logic
has been combined with decision support systemshas been combined with decision support systems• One common case is in navigational and sensor
systems for robotics
![Page 13: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/13.jpg)
Artificial neural networks
• Inspired from neurons in the nervous systemp y
• Nowadays has nothing to do with real systems !
A i l d l d t ith i t• A single neuron modeled as a system with input, output and transfer (activation) function
• A network of neurons is formed
• Have applications in both supervised and pp punsupervised cases
• Have applications in modeling dynamic data• Have applications in modeling dynamic data such as time series
![Page 14: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/14.jpg)
Supervised Applications
• Classification/Pattern recognition:Th t k f tt iti i t i i t tt (lik▫ The task of pattern recognition is to assign an input pattern (like handwritten symbol) to one of many classes. This category includes algorithmic implementations such as associative memory.
• Function approximation:▫ The tasks of function approximation is to find an estimate of the pp
unknown function f(.) subject to noise. Various engineering and scientific disciplines require function approximation.
• Prediction/Dynamical Systems:• Prediction/Dynamical Systems:▫ The task is to forecast some future values of a time-sequenced
data. Prediction differs from Function approximation by considering time factorconsidering time factor.Here the system is dynamic and may produce different results for the same input data based on system state (time).
![Page 15: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/15.jpg)
Function Approximation: Body DensityDensity
Predict the real “body density” (e.g. % body fat) calculated using a submersion test using easier to obtain data:g
▫ Age (years), ▫ Weight (lbs), ▫ Height (inches), g ( ),▫ Neck circumference (cm), ▫ Chest circumference (cm), ▫ Abdomen 2 circumference (cm), ▫ Hip circumference (cm), p ( ),▫ Thigh circumference (cm), ▫ Knee circumference (cm), ▫ Ankle circumference (cm), ▫ Biceps (extended) circumference (cm), ( ) ( )▫ Forearm circumference (cm), ▫ Wrist circumference (cm)
• Real relationship is unkown, but if outputs of our system match t l h d d lcorrect values, we have a good model
![Page 16: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/16.jpg)
Classification
• Similar to function approximation except output is a “class”▫ For example: Outputs = on or offp Outputs = Ford, Chevy, or Buick Outputs = Sick or Healthy
![Page 17: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/17.jpg)
Classification: Character Recognition
• NN is used to classify each charactery• Extract features as input to NN• Pixels also can be used as input• Output is the class of the
test character
• S M Razavi M Taghipour “Improvement in Performance of Neural Network for Persian• S.M. Razavi, M. Taghipour, Improvement in Performance of Neural Network for Persian Handwritten Digits Recognition using FCM Clustering”, World Applied Sciences, 2010.
![Page 18: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/18.jpg)
Classification: Speaker Identification
• Determine the speaker identityp y• Selection between a set of known voices
?
??Whose voice is this?
• V. Moonasar, G. K. Venayagamoorthy, "Speaker Identification using a Combination of Different Parameters as Feature Inputs to an Artificial Neural Network Classifier", IEEE AFRICON 1999IEEE AFRICON, 1999.
![Page 19: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/19.jpg)
Prediction: Technical Analysisy
• Technical analysis rests on the assumption that y phistory repeats itself.▫ Example: future market direction can be
determined by examining past prices.▫ Using price, volume, and open interest statistics,
the technical analyst uses charts to predict futurethe technical analyst uses charts to predict future stock movements.
![Page 20: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/20.jpg)
ANN For Prediction
![Page 21: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/21.jpg)
Time-series prediction
• Financial applications:pp▫ Predicting Stock trading
![Page 22: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/22.jpg)
Medical Decision Makingg
• Predicting Blood Transfusion for Emergency g g yPatients
• 1016 patient records areused for training
• Only data of patients upon entry
• ANN is used for predictingth t f bl d d d f t f ithe amount of blood needed for transfusion
• Steven Walczak, "Artificial neural network medical decision support tool: predicting transfusion requirements of ER patients" IEEE Transactions on Informationtransfusion requirements of ER patients", IEEE Transactions on Information Technology in Biomedicine, Sept. 2005.
![Page 23: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/23.jpg)
Estimate the Risk of Mortalityy
• Use of a Probabilistic Neural Network to Estimate the Risk of Mortality after Cardiac Surgery
• Patient records were randomly dividedinto training (732) and validation (380)into training (732) and validation (380)
• The model uses seven variables, each obtainable during routine clinical patient care.
• RICHARD K. ORR, MD, MPH, "Use of a Probabilistic Neural Network to Estimate the Risk of Mortality after Cardiac Surgery", Medical Decision Making April 1997 vol. 17 no. 2 178-185.
![Page 24: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/24.jpg)
Risk of Mortality Resultsy
![Page 25: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/25.jpg)
Medical Diagnosisg
• Diagnosis of Patients based on their symptoms• UCI machine learning benchmark repository
• S. M. Kamruzzaman, Ahmed Ryadh Hasan, Abu Bakar Siddiquee and Md. Ehsanuly qHoque, "MEDICAL DIAGNOSIS USING NEURAL NETWORK", 3rd International Conference on Electrical & Computer Engineering ICECE 2004, 28-30 December 2004, Dhaka, Bangladesh
![Page 26: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/26.jpg)
Medical Diagnosis Resultsg
![Page 27: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/27.jpg)
NN for Diagnosing Breast Cancer from Image
• Extract some features using Image processing g g p gmethods
• Features:▫ Mass Size▫ Mass shape
M d it▫ Mass density▫ Asymmetric density▫ Many other features including patients symptoms▫ Many other features including patients symptoms
and experiment results• Output: Classify Benign or Malignant?p y g g• P. Abdolmaleki,M. Guiti,M. Tahmasebi, "Neural Network Analysis of Breast Cancer
from Mammographic Evaluation", 2006.
![Page 28: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/28.jpg)
Control: Siemens ANN Process ControlControl
• Used in parallel to old system to control the p yparameters of the steel process
• Mainly used to tune the parameters to increase the Steel Grade and Steel Sheet Thickness
• Needs Adaption or else the error will increase ft f dafter a few days
• M. Schlang, "Neural Network for Process Control in Steel Processes", 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'97).
![Page 29: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/29.jpg)
RNN for compensating vibrationg
• A recurrent neural network compensator for psuppressing mechanical vibration in a permanent magnet linear synchronous motor ( S )(PMLSM) is studied.
• H. Yousefia, M. Hirvonena, H. Handroosa, A. Soleymanib, "Application of neural network in suppressing mechanical vibration of a permanent magnet linear motor", Control Engineering Practice 16, 2008.
![Page 30: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/30.jpg)
NN for Oil Spill Detection
• Uses ANN for Oil Spill Detection using Satellite Images (ERS-SAR)
• Several Statistical and Image Processing features are extracted from Imagefeatures are extracted from Image
• F. D. Frate, A. Petrocchi, J. Lichtenegger, G. Calabresi, "Neural Networks for Oil SpillF. D. Frate, A. Petrocchi, J. Lichtenegger, G. Calabresi, Neural Networks for Oil Spill Detection Using ERS-SAR Data", IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 38, NO. 5, SEPTEMBER 2000.
![Page 31: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/31.jpg)
NN for Pharmaceutical Formulation
• Relationship between casual factors and pindividual pharmaceutical responses.
• NN is used to extract nonlinear relationships between these two.
• Several responses relating to the effectiveness, safety and usefulness must be optimized
i lt lsimultaneously
• K Takayama M Fujikawa T Nagai "Artificial Neural Network as a Novel Method to• K Takayama, M Fujikawa, T Nagai, Artificial Neural Network as a Novel Method to Optimize Pharmaceutical Formulations ", Pharmaceutical Research Volume 16, Number 1, 1999.
![Page 32: Lecture1: An Introduction to Neural Nets and Fuzzy Systemsce.sharif.edu/.../resources/root/Lectures/Lecture1.pdf · Fuzzy Thermostat • A fuzzy thermostat works in shades of gray](https://reader033.fdocuments.in/reader033/viewer/2022060513/5f822a1c962d16210f49fe5c/html5/thumbnails/32.jpg)
And …
• Much more examples where ANNs are used for pmodeling, identification, approximation, de-noising, and recognition.