Artificial Neural network model & hidden layers in

multilayer artificial neural networks

Outline

• Introduction

• Single Layer • Multi Layer• Learning• Backpropogation Algorithm

Introduction

• Artificial Neural Network are computational models inspired by biological neural network used for processing large number of inputs which are mostly unknown.

What is Problem?

•Our world is full of data. After collection and organization, data, if we are lucky, becomes information. In today's interconnected world, information exists in electronic form that can be stored and transmitted instantly. Challenge is to understand, integrate, and apply information to generate useful knowledge.

What is Solution?

• Interpretation requires data acquisition, cleaning (preparing the data for analysis), analysis, and presentation in a way that permits knowledgeable decision making and action. Key is to extract information about data from relationships buried within the data itself.

Why Human System?

•Human brain is most powerful pattern recognition engine ever invented.

Biological Neuron

Artificial Neuron

Artificial Neuron

History

• 1943: McCulloch–Pitts “neuron”– Started the field

• 1962: Rosenblatt’s perceptron– Learned its own weight values; convergence proof

• 1969: Minsky & Papert book on perceptrons– Proved limitations of single-layer perceptron networks

• 1982: Hopfield and convergence in symmetric networks– Introduced energy-function concept

• 1986: Backpropagation of errors– Method for training multilayer networks

• Present: Probabilistic interpretations, Bayesian and spiking networks

Single Layer

• Input Layer

• Output Layer

Multi Layer

• Input Layer

• Hidden Layer(s)

• Output Layer

Multi Layer

• Instead of programming computational system to do specific tasks, teach system how to perform task

• To do this, generate Artificial Intelligence System- AI

• AI systems must be adaptive – able to learn from data on a continuous basis

ANN Architecture

a. Single Layer Feed Forward

ANN Architecture

Multilayer Feed Forward

ANN Architecture

Recurrent Network

ANN Architecture

a. Single Layer Feed Forward

ANN Architecture

Multilayer Feed Forward

ANN Architecture

Recurrent Network

Clasification of learning strategies

Supervised

• Process of using desired output for training the NN

• It employees a teacher to assist the network by telling the network what the desired response to a given input

• Weights are modified according to the required output

• Not practicable in all cases

Understanding Supervised and Unsupervised Learning

ABA

B A

B

Two possible Solutions…

A

B

A

B

A

B

A

B A

B

A

B

Supervised Learning

• It is based on a labeled training set.

• The class of each piece of data in training set is known.

• Class labels are pre-determined and provided in the training phase.

A

BA

BA

B

ε Class

λ Class

λ Class

λ Class

ε Class

ε Class

26

A single perceptron can be used to represent many boolean functions.• AND function :

x1 x2 output

0 0 -10 1 -11 0 -11 1 1

<Training examples> Decision hyperplane :w0 + w1 x1 + w2 x2 = 0-0.8 + 0.5 x1 + 0.5 x2 = 0

x1 x2 � wixi output

0 0 -0.8 -10 1 -0.3 -11 0 -0.3 -11 1 0.2 1

<Test Results>

-

-

-

+

x1

x2

-0.8 + 0.5 x1 + 0.5 x2 = 0

-

-

-

+

x1

x2

-0.8 + 0.5 x1 + 0.5 x2 = 0

27

OR function • The two-input perceptron can implement the OR function when

we set the weights: w0 = -0.3, w1 = w2 = 0.5

x1 x2 o utput

0 0 - 1

0 1 11 0 11 1 1

<Train ing e x ample s > Decision hyperplane :w0 + w1 x1 + w2 x2 = 0-0.3 + 0.5 x1 + 0.5 x2 = 0

x1 x2 Σwixi output

0 0 - 0.3 - 1

0 1 0.2 - 11 0 0.2 - 11 1 0.7 1

<Te s t Re s ults >

-

+

+

+

x1

x2

-0.3 + 0.5 x1 + 0.5 x2 = 0

-

+

+

+

x1

x2

-0.3 + 0.5 x1 + 0.5 x2 = 0

Unsupervised

• No teacher Required

• Similar to the students learning on their own

• Adaption rules

• Adaption rule generate error signals

Reinforced

• A teacher is assumed to b present but right answer is not given to the network

• Network is given an indication whether output is right or wrong

• Network use this indication to improve performance

Types of Neuron activation function

1. Heaviside

1 if s>=T

F(s)=

0 if s<T

2. Linear

f(s)=s

3. Piecewise Linear

1 if s>T

F(s)= s if –T<s<T

-1 if s<-T

Hardlimiter

Unipolar Sigmoidal

Bipolar Sigmoidal

Unipolar Multimodal

Radial Bessel’s Function

Backpropogation Algorithm

36

NEURAL NETWORK APPLICATION DEVELOPMENT

The development process for an ANN application has eight steps. • Step 1: (Data collection).• Step 2: (Training and testing data separation For a moderately

sized data set, 80% of the data are randomly selected for training, 10% for testing, and 10% secondary testing.

• Step 3: (Network architecture) Important considerations are the exact number of perceptrons and the number of layers.

37

• Step 4: (Parameter tuning and weight initialization)• Step 5: (Data transformation) Transforms the application

data into the type and format required by the ANN. • Step 6: (Training)

38

• Step 7: (Testing)– The testing examines the performance of the network using

the derived weights by measuring the ability of the network to classify the testing data correctly.

– Black-box testing (comparing test results to historical results) is the primary approach for verifying that inputs produce the appropriate outputs.

• Step 8: (Implementation) Now a stable set of weights are obtained.

Thank You for your Attention..!! . .

We welcome any Questions….