Convolutional Neural Nets Advanced Vision Seminar April 19, 2015.
Seminar Report on "neural network and their applications"
-
Upload
vivek-yadav -
Category
Documents
-
view
5.567 -
download
3
description
Transcript of Seminar Report on "neural network and their applications"
SESSION: 2010-2011
A SEMINAR REPORT ON
“Neural Networks &Their
Application”
UNIVERSITY ROLL NO - 0806331112
SUBMITTED BY - SUBMITTED TO - VIVEK YADAV Mr. MANISH KASHYAP EC(branch) (seminar in-charge) 3rd (year) G (section) 53(roll no.)
ACKNOWLEDGEMENT
I pose my copious gratitude and like to thank the entire staff of
G.L.A.I.T.M, Mathura for their help and kind cooperation during my
entire seminar preparation. I am extremely thankful to them for
providing me with vital information about the topic. I rejoice in
expressing my prodigious gratification to Department of Electronics
&Communication Engineering Department, G.L.A.I.T.M, Mathura for his
indispensable guidance, generous help, perpetual encouragement,
constant attention offered throughout in preparing the seminar.
I take this opportunity to pay my sincere thanks to Mr. MANISH
KASHYAP, seminar in charge & lecturer, Electronics & Communication
Engineering Department, G.L.A. Institute of Technology & Management,
Mathura, for giving me the golden opportunity to present the seminar.
At last but not the least, I would like to thank my parents and all
my peers who have been a constant source of encouragement and
inspiration in every walk of life.
VIVEK YADAV
CERTIFICATE
This is to certify that the work which is being successfully presented in
the seminar report entitled “NEURAL NETWORKS & THEIR
APPLICATIONS” by me in the partial fulfillment of the requirement for
the award of Bachelor Of Technology Degree in Electronics &
Communication Engineering Department at G.L.A. Institute Of
Technology & Management, Mathura from Uttar Pradesh Technical
University, Lucknow.
The matter embodied in this dissertation has not been submitted by
me for award of any other degree.
DATE-: 21 ,APRIL , 2011
This is to certify that the above statement made by the candidate is
correct to the best of my knowledge.
SUBMITTED BY -: SUBMITTED TO-:
VIVEKYADAV (MR. MANISH KASHYAP)
B.TECH. III YEAR (EC) SEMINAR INCHARGE
ROLL NO.: 0806331112
CONTENTS
Topic Page no.
1. Abstract ………………………………………………………………………5
2. Introduction………………………………………………………………… 6
3. Biological neuron…………………………………………………………. 8
4. Artificial neuron…………………………………………………………….9
5. Different models of artificial neuron…………………………….10
6. Classical activation function………………………………………….14
7. Artificial neural network………………………………………………..16
8. Qualities of neural network……………………………………………17
9. Different architecture of ANN…………………………………………19
10.Learning ofANN………………………………………………………………20
11.Advantages&Disadvantages ofANN…………………………………22
12.Applications of ANN…………………………………………………………23
13.Recent advances in field of ANN……………………………………….25
14.Conclusion………………………………………………………………………27
15.Bibliography……………………………………………………………………28
ABSTRACT
Neural network are inspired by biological nervous system and re
composed of many simple computational elements operating in
parallel. In this study basic component of neural network are
introduced and brief on their working. Concept of activation function
is also discussed. Different learning algorithm are also enlisted.Topics
on advantage ,disadvantage and recent development in the field of
ANN’s are mentioned in this study.
Introduction
Neural networks have seen an explosion of interest over the last few
years and are being successfully applied across an extraordinary range
of problem domains, in areas as diverse as finance, medicine,
engineering, geology, physics and biology. The excitement stems from
the fact that these networks are attempts to model the capabilities of
the human brain. From a statistical perspective neural networks are
interesting because of their potential use inprediction and
classification problems.
Artificial neural networks (ANNs) are non-linear data driven self
adaptive approach as opposed to the traditional model based
methods. They are powerful tools for modelling, especially when the
underlying data relationship is unknown. ANNs can identify and learn
correlated patterns between input data sets and corresponding target
values. After training,ANNs can be used to predict the outcome of new
independent input data. ANNs imitate the learning process of the
human brain and can process problems involving non-linear and
complex data even if the data are imprecise and noisy. Thus they are
ideally suited for the modeling of agricultural data which are known to
be complex and often non-linear.
These networks are “neural” in the sense that they may have
been inspired by neuroscience but not necessarily because they are
faithful models of biological neural or cognitive phenomena. In fact
majority of the network are more closely related to traditional
mathematical and/or statistical models such as non-parametric
pattern classifiers, clustering algorithms, nonlinear filters, and
statistical regression models than they are to neurobiology models.
Neural networks (NNs) have been used for a wide variety of
applications where statistical methods are traditionally employed.
They have been used in classification problems, such as identifying
underwater sonar currents, recognizing speech, and predicting the
secondary structure of globular proteins. In time-series applications,
NNs have been used in predicting stock market performance. As
statisticians or users of statistics, these problems are normally solved
through classical statistical methods, such as discriminant analysis,
logistic regression, Bayes analysis, multiple regression, and ARIMA
time-series models. It is, therefore, time to recognize neural networks
as a powerful tool for data analysis.
The biological neuron
Neurons can be of many types and shapes, but ultimately they
function in a similar way and are connected to each other in a rather
complex network stylish way via strands of fibre called axons . A
neurons axon acts as a transmission line and are connected to
another neuron via that neurons dendrites , which are fibres that
emanate from the cell body ( soma ) of the neuron. The junction that
allows transmission between the axons and the dendrites are the
synapse. Synapses are elementary structural and functional units that
creates the signal connection between two or more neurons;
sometimes meaning the connection as whole.
FIG 1-: Biological neuron
The artificial neuron
Artificial neurons are information-processing units that are only
approximations (usually very crude ones) of the biological neuron.
Three basic elements of the artificial neuron can be identified as-:
FIG 2-: Artificial neuron
Input (xi)
Typically, these values are external stimuli from the environment or
come from the outputs of other artificial neurons. They can be
discrete values from a set, such as {0,1}, or real-valued numbers.
Weights (wi)
These are real-valued numbers that determine the contribution of
each input to the neuron's weighted sum and eventually its output.
The goal of neural network training algorithms is to determine the
best possible set of weight values for the problem under
consideration. Finding the optimal set is often a trade-off between
computation time and minimizing the network error.
Threshold (u)
The threshold is referred to as a bias value. In this case, the real
number is added to the weighted sum. For simplicity, the threshold
can be regarded as another input / weight pair, where w0 = u and x0
= -1.
Activation Function (f)
The activation function for the original McCulloch-Pitts neuron was
the unit stepfunction. However, the artificial neuron model has been
expanded to include other functions such as the sigmoid, piecewise
linear, and Gaussian.
Different models of artificial neuron
1.Adaline model
2.Madaline model
3.Rosenballet model
4.Mcculloch pits model
5.Widrow hoff model
6.Kohonen model
1.The adaptive linear element (Adaline)
In a simple physical implementation
Fig3-:The McCulloch-Pitts Model of Neuron
this device consists of a set of controllable resistors connected to a
circuit which can sum up currents caused by the input voltage signals.
Usually the central block, the summer, is also followed by a quantiser
which outputs either +1 of 1,depending on the polarity of the sum.
Although the adaptive process is here exemplified in a case when
there is only one output, it may be clear that a system with many
parallel outputs is directly implementable by multiple units of the
above kind.
If the input conductances are denoted by wi, i = 0; 1; : : : ; n, and
the input and output signals by xi and y, respectively, then the output
of the central block is defined to be:
where θ = w0.
2.Mcculloch pitts modelThe early model of an artificial neuron is introduced by Warren
McCulloch and Walter Pitts in 1943. The McCulloch-Pitts neural model
is also known as linear threshold gate. It is a neuron of a set of inputs
and one output The linear threshold gate simply
classifies the set of inputs into two different classes. Thus the output
is binary. Such a function can be described mathematically using these equations:
(2.1)
(2.2)
are weight values normalized in the range of either
or and associated with each input line, is the
weighted sum, and is a threshold constant. The function is a linear
step function at threshold as shown in figure. The symbolic representation
of the linear threshold gate is shown in figure [
Fig 4: Linear Threshold Function
Fig5-: The McCulloch-Pitts Model of Neuron
Classical Activation functionsWhile it is possible to define some arbitrary, cost function, frequentlya
particular cost will be used, either because it has desirable properties
(such as convexity) or because it arises naturally from a particular
formulation of the problem (e.g., in a probabilistic formulation the
posterior probability of the model can be used as an inverse cost).
Ultimately, the cost function will depend on the desired task.
Different types of activation functions can be used and three of them
are described in [activation functions] . The most commonly used is
nonlinear sigmoid activation functions such as the logistic function . A
logistic function assumes a continuous range of values form 0 and 1 in
contrary to the discrete threshold function. A binary threshold
function was used in the first model of an artificial neuron back in
1943, the so-called McCulloch-Pitts model . Threshold functions goes
by many names, e.g. step-function , heavyside function , hard-limiter
etc. Common for all is that they produce one of two scalar output
values (usually 1 and -1 or 0 and 1) depending on the value of the
threshold. Another type of activation function is the linear function or
some times called the identity function since the activation is just the
input. In general if the task is to approximate some function then the
output nodes are linear and if the task is classification then sigmoidal
output nodes are used
Artificial neural networkComputational models inspired by the human brain-:
1. Massively, parallel, distributed system, made up of simple
processing units.(neurons)
2. Synaptic connection strengths among neurons are used to
store the acquired knowledge.
3. Knowledge is acquired by the network from its environment
through a learning process.
FIG 3-: Simple artificial neural network
Qualities of artificial neural network
1. Real time operation
2. Parallel processing
3. Fault tolerance
4. Self organising
5. Ability to generalize
6. Complete computability
7. Continuous adaptability
Different architectures of ANN
Fig-:Back Propagation network
Fig-:Multi layered Perceptron network
Fig-:Hopfield network
Fig-:Kehonen network
Learning of ANN
An ANN learns from its experience. The usual process of learning
involves three tasks:
1.Compute output(s).
2.Compare outputs with desired patterns and feed-back the
error.
3.Adjust the weights and repeat the process
4.The learning process starts by setting the weights by some
rules . The difference between the actual output (y) and the
desired output(z) is called error (delta).
5.The objective is to minimize delta (error)to zero. The
reduction in error is done by changing the weights
1.Supervised learning-: or Associative learning in which the network
is trained by providing it with input and matching output patterns.
These input-output pairs can be provided by an external teacher, or
by the system which contains the neural network (self-supervised).
2.Unsupervised learning -:or Self-organisation in which an (output)
unit is trained to respond to clusters of pattern within the input. In
this paradigm the system is supposed to discover statistically salient
features of the input population. Unlike the supervised learning
paradigm, there is no a priori set of categories into which the
patterns are to be classified; rather the system must develop its own
representation of the input stimuli.
3.Reinforcement Learning -:This type of learning may be considered
as an intermediate form of the above two types of learning. Here the
learning machine does some action on the environment and gets a
feedback response from the environment. The learning system grades
its action good (rewarding) or bad (punishable) based on the
environmental response and accordingly adjusts its parameters.
Generally, parameter adjustment is continued until an equilibrium
state occurs, following which there will be no more changes in its
parameters. The selforganizing neural learning may be categorized
under this type of learning.
Advantages&Disadvantages of ANN
A.)Advantages
1.Adapt to unknown situations
2.Robustness: fault tolerance due to network redundancy
3.Autonomous learning and generalization
B.)Disadvantages
1.Not exact
2.Large complexity of the network structure.
APPLICATIONS OF ANN
RECENT ADVANCES IN ANN FIELD
Integration of fuzzy logic into neural networks
1. Fuzzy logic is a type of logic that recognizes more than simple true
and false values, hence better simulating the real world. For
example, the statement today is sunny might be 100% true if there
are no clouds, 80% true if there are a few clouds, 50% true if it's
hazy, and 0% true if rains all day. Hence, it takes into account
concepts like -usually, somewhat, and sometimes.
2. Fuzzy logic and neural networks have been integrated for uses as
diverse as automotive engineering, applicant screening for jobs,
the control of a crane, and the monitoring of glaucoma.
Pulsed neural networks
1. "Most practical applications of artificial neural networks are based
on a computational model involving the propagation of continuous
variables from one processing unit to the next. In recent years,
data from neurobiological experiments have made it increasingly
clear that biological neural networks, which communicate
through pulses, use the timing of the pulses to transmit
information and perform computation. This realization has
stimulated significant research on pulsed neural networks,
including theoretical analyses and model development,
neurobiological modeling, and hardware implementation."
Hardware specialized for neural networks
1. Some networks have been hardcoded into chips or analog
devices ? this technology will become more useful as the networks
we use become more complex.
2. The primary benefit of directly encoding neural networks onto
chips or specialized analog devices is SPEED!
3. NN hardware currently runs in a few niche areas, such as those
areas where very high performance is required (e.g. high energy
physics) and in embedded applications of simple, hardwired
networks (e.g. voice recognition).
4. Many NNs today use less than 100 neurons and only need
occasional training. In these situations, software simulation is
usually found sufficient
When NN algorithms develop to the point where useful things can be
done with 1000's of neurons and 10000's of synapses, high
performance NN hardware will become essential for practical
operation.
CONCLUSION
1. All current NN technologies will most likely be vastly improved
upon in the future. Everything from handwriting and speech
recognition to stock market prediction will become more
sophisticated as researchers develop better training methods and
network architectures
2. Although neural networks do seem to be able to solve many
problems, we must put our exuberance in check sometimes ? they
are not magic! Overconfidence in neural networks can result in
costly mistakes: see for a rather funny story about the government
and neural networks.
.
NNs might, in the future, allow: a. Robots that can see, feel, and predict the world around
them
b. Improved stock prediction
c. Common usage of self-driving cars
d. Composition of music
e. Handwritten documents to be automatically transformed into formatted word processing documents
f. Trends found in the human genome to aid in the under standing of the data compiled by the Human Genome Project
g. Self-diagnosis of medical problems using neural networks
h. And much more!
REFRENCES
1. Hertz, J., Palmer, R.G., Krogh. A.S. (1990)
Introduction to the theory of neural computation,
Peruses Books. ISBN 0-201-51560-1
2. B.Yegnarayana(2010) Artificial neural networks
PHI publication. ISBN-978-81-203-1253-1
3. “Neural Networks: A Comprehensive Foundation”
by HAYKINS