Artificial Neural Network

27
Neural Networks Presented By: Akif Burhan Muzafar Junaid Hassan Umar manzoor Muneer AH Saleem zahoor

Transcript of Artificial Neural Network

Page 1: Artificial Neural Network

Neural NetworksPresented By:

Akif Burhan Muzafar

Junaid Hassan Umar manzoor

Muneer AHSaleem zahoor

Page 2: Artificial Neural Network

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by biological nervous systems.

It is composed of a large number of highly interconnected processing elements called neurons.

An ANN is configured for a specific application, such as pattern recognition or data classification

Neural network MEANING

Page 3: Artificial Neural Network

The neuron receives signals from other neurons, collects the input signals, and transforms the collected input signal.

Human information processing takes place through the interaction of many billions of neurons connected to each other, each sending excitatory or inhibitory signals to other neurons (excite in positive/suppress in negative)

Page 4: Artificial Neural Network

Types of Artificial neural networkArtificial neural networks (ANNs) are biologically inspired computer programs designed to simulate the way in which the human brain processes information. ANNs gather their knowledge by detecting the patterns and relationships in data and learn (or are trained) through experience.

Page 5: Artificial Neural Network

Types of artificial neural networks Artificial neural network types vary from those with

only one or two layers of single direction logic, to complicated multi–input many directional feedback loops and layers. On the whole, these systems use algorithms in their programming to determine control and organization of their functions.

Page 6: Artificial Neural Network

Computational power The multi-layer perception (MLP) is a universal function

approximate, as proven by the universa approximation theorem. However, the proof is not constructive regarding the number of neurons required or the settings of the weights. Work by Hava Siegelmann and Eduardo D. Sontag has provided a proof that a specific recurrent architecture with rational valued weights (as opposed to full precision real number-valued weights) has the full power of a Universal Turing Machine using a finite number of neurons and standard linear connections. Further, it has been shown that the use of irrational values for weights results in a machine with super-Turing power.

Page 7: Artificial Neural Network

Capacity Artificial neural network models have a property called

'capacity', which roughly corresponds to their ability to model any given function. It is related to the amount of information that can be stored in the network and to the notion of complexity.

Convergence Nothing can be said in general about convergence since it

depends on a number of factors. Firstly, there may exist many local minima. This depends on the cost function and the model. Secondly, the optimization method used might not be guaranteed to converge when far away from a local minimum. Thirdly, for a very large amount of data or parameters, some methods become impractical. In general, it has been found that theoretical guarantees regarding convergence are an unreliable guide to practical application.

Page 8: Artificial Neural Network

Generalization and statistics In applications where the goal is to create a system that generalizes

well in unseen examples, the problem of over-training has emerged. This arises in convoluted or over-specified systems when the capacity of the network significantly exceeds the needed free parameters. There are two schools of thought for avoiding this problem: The first is to use cross-validation and similar techniques to check for the presence of overtraining and optimally select hyperparameters such as to minimize the generalization error. The second is to use some form of regularization This is a concept that emerges naturally in a probabilistic (Bayesian) framework, where the regularization can be performed by selecting a larger prior probability over simpler models; but also in statistical learning theory, where the goal is to minimize over two quantities: the 'empirical risk' and the 'structural risk', which roughly corresponds to the error over the training set and the predicted error in unseen data due to overfitting.

Page 9: Artificial Neural Network

NEURAL NETWORK APPLICATIONS

Page 10: Artificial Neural Network

APPLICATIONS OF NEURAL NETWORKS

Over the last decade, neural networks have found application across a wide range of areas from business, commerce and industry. Following an overview is provided of the kinds of business problems to which neural networks are suited, with a brief discussion of some of the reported studies Relevant to each area.

Page 11: Artificial Neural Network

The goal of modern marketing exercises is to identify customers who are likely to respond positively to a product, and to target any advertising or solicitation towards these customers. Target marketing involves market segmentation, whereby the market is divided into distinct groups of customers with very different consumer behavior. Market segmentation can be achieved using neural networks by segmenting customers according to basic characteristics including demographics, socio-economic status, geographic location, purchase patterns, and attitude towards a product.

Marketing

Page 12: Artificial Neural Network

Businesses often need to forecast sales to make decisions about inventory, sta$ng levels, and pricing. Neural networks have had great success at sales forecasting, due to their ability to simultaneously consider multiple variables such as market demand for a product, consumers' disposable income, the size of the population, the price of the product, and the price of complementary products. Forecasting of sales in supermarkets and wholesale suppliers has been studied and the results have been shown to perform well when compared to traditional statistical techniques like regression, and human experts.

Retail

Page 13: Artificial Neural Network

One of the main areas of banking and "nance that has been affected by neural networks is trading and financial forecasting. Neural networks have been applied successfully to problems like derivative securities pricing and hedging , futures price forecasting, exchange rate forecasting and stock performance and selection prediction.

Banking and finance

Page 14: Artificial Neural Network

There are many areas of the insurance industry which can benefit from neural networks. Policy holders can be segmented into groups based upon their behaviors, which can help to determine effective premium pricing. Prediction of claim frequency and claim cost can also help to set premiums, as well as find an acceptable mix or portfolio of policy holders characteristics. The insurance industry, like the banking and finance sectors, is constantly aware of the need to detect fraud, and neural networks can be trained to learn to detect fraudulent claims or unusual circumstances.

Insurance

Page 15: Artificial Neural Network

Like other competitive retail industries, the telecommunications industry is concerned with the concepts of churn (when a customer joins a competitor) and win back (when an ex-customer returns). Neural Technologies Inc., is a UK-based company which has marketed a product called DA Churn Manager. Specifically tailored to the telecommunications industry, this product uses a series of neural networks to: analyze customer and call data; predict if, when and why a customer is likely to churn; predict the elects of forthcoming promotional strategies; and interrogate the data to find the most profitable customers. Telecommunications companies are also concerned with product sales, since the more reliant a customer becomes on certain products

Telecommunications

Page 16: Artificial Neural Network

Models of ARTIFICIAL NEURAL NETWORK

Page 17: Artificial Neural Network

Four parts of a typical nerve cell : -

DENDRITES: Accepts the inputs

SOMA : Process the inputs AXON : Turns the

processed inputs into outputs.

SYNAPSES : The electrochemical contact between the neurons.

BIOLOGICAL NEURON MODEL

Page 18: Artificial Neural Network

Inputs to the network are represented by the mathematical symbol, xn

Each of these inputs are multiplied by a connection weight

These products are simply summed, fed through the transfer function, f( ) to generate a result and then output.

ARTIFICIAL NEURON MODEL

f

w1

w2

xn

x2

x1

wn

f(w1 x1 + ……+ wnxn)

Page 19: Artificial Neural Network

ARTIFICIAL NEURAL NETWORK MODEL

output layer

connections

Input layer

Hidden layers

Neural network

Including connections(called weights) between neuron

Compare

Actual output

Desired output

Input output

Figure showing adjust of neural network

: artificial neural network model

CONTD

Page 20: Artificial Neural Network

The neural network in which

every node is connected to every other nodes, and these connections may be either excitatory (positive weights), inhibitory (negative weights), or irrelevant (almost zero weights).

These are networks in

which nodes are partitioned into subsets called layers, with no connections from layer j to k if j > k.

NEURAL NETWORK ARCHITECTURES

Input node

Input node

output node

output node

Hidden node

Layer 1 Layer2 Layer0 Layer3

(Input layer) (Output layer)

Page 21: Artificial Neural Network

Neurons in an animal’s brain are “hard wired”. It is equally obvious that animals, especially higher order animals, learn as they grow.

How does this learning occur? What are possible mathematical models of

learning? In artificial neural networks, learning refers to

the method of modifying the weights of connections between the nodes of a specified network.

The learning ability of a neural network is determined by its architecture and by the algorithmic method chosen for training.

LEARNING

Page 22: Artificial Neural Network

Risk in Artificial Neural Network

TRAINING RISK HARDWARE RISK

HYBRID APPROACH

Page 23: Artificial Neural Network

A common risk of neural networks, particularly in robotics, is that they require a large diversity of training for real-world operation. This is not surprising, since any learning machine needs sufficient representative examples in order to capture the underlying structure that allows it to generalize to new cases. Dean Pomerleau, in his research presented in the paper "Knowledge-based Training of Artificial Neural Networks for Autonomous Robot Driving," uses a neural network to train a robotic vehicle to drive on multiple types of roads (single lane, multi-lane, dirt, etc.). A large amount of his research is devoted to extrapolating multiple training scenarios from a single training experience, and

TRAINING RISK

Page 24: Artificial Neural Network

Preserving past training diversity so that the system does not become over trained (if, for example, it is presented with a series of right turns – it should not learn to always turn right). These issues are common in neural networks that must decide from amongst a wide variety of responses, but can be dealt with in several ways, for example by randomly shuffling the training examples, by using a numerical optimization algorithm that does not take too large steps when changing the network connections following an example, or by grouping examples in so-called mini-batches.

Page 25: Artificial Neural Network

To implement large and effective software neural networks, considerable processing and storage resources need to be committed. While the brain has hardware tailored to the task of processing signals through a graph of neurons, simulating even a most simplified form on Von Neumann technology may compel a neural network designer to fill many millions of database rows for its connections – which can consume vast amounts of computer memory and hard disk space. Furthermore, the designer of neural network systems will often need to simulate the transmission of signals through many of these connections and their associated neurons – which must often be matched with incredible amounts of CPU processing power and time. While neural networks often yield effective programs, they too often do so at the cost of efficiency (they tend to consume considerable amounts of time and money).

HARDWARE RISK

Page 26: Artificial Neural Network

Some other risks come from advocates of hybrid models (combining neural networks and symbolic approaches), who believe that the intermix of these two approaches can better capture the mechanisms of the human mind.

HYBRID APPROACH

Page 27: Artificial Neural Network