Intro to Neural Networks UNIT-II. Neural networks is a branch of "Artificial Intelligence“. ...

43
Intro to Intro to Neural Neural Networks Networks UNIT-II UNIT-II

Transcript of Intro to Neural Networks UNIT-II. Neural networks is a branch of "Artificial Intelligence“. ...

Page 1: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Intro to Neural Intro to Neural NetworksNetworks

UNIT-IIUNIT-II

Page 2: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Neural networks is a branch of "Artificial Neural networks is a branch of "Artificial Intelligence“. Intelligence“.

““Artificial Neural Network” Artificial Neural Network” is a system is a system loosely modeled based on the human loosely modeled based on the human brain. brain.

The field goes by many names, such as The field goes by many names, such as connectionism, parallel distributed connectionism, parallel distributed processing, neuro-computing, natural processing, neuro-computing, natural intelligent systems, machine learning intelligent systems, machine learning algorithms, and artificial neural networks. algorithms, and artificial neural networks.

INTRODUCTIONINTRODUCTIONWhat is a neural network (NN)?What is a neural network (NN)?

Page 3: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

DefinitionDefinition• Neural networks are simplified Neural networks are simplified models of the biological nervous models of the biological nervous system and therefore they have system and therefore they have drawn their motivation from the drawn their motivation from the kindkind of computing performed by a of computing performed by a human brain.human brain.• Neural networks are a powerful technique to Neural networks are a powerful technique to solve many real world problems. solve many real world problems. • They have the ability to learn from They have the ability to learn from experience experience in order to improve their performance in order to improve their performance and to adapt themselves to changes in the and to adapt themselves to changes in the environment. environment.

Page 4: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

• In addition to that they are In addition to that they are able to deal with able to deal with incomplete incomplete information or noisy datainformation or noisy data and can be very effective especially and can be very effective especially in situations where it is not possible in situations where it is not possible to define the rules or steps that lead to define the rules or steps that lead to the solution of a problem.to the solution of a problem.

Continuation………..Continuation………..

Page 5: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

ApplicationsApplicationsNeural Network Applications can be grouped in following Neural Network Applications can be grouped in following

categories:categories:

Clustering:Clustering:A clustering algorithm explores the similarity A clustering algorithm explores the similarity

between patterns and places similar patterns in a cluster. between patterns and places similar patterns in a cluster. Best known applications include data compression and Best known applications include data compression and data mining.data mining.

Classification/Pattern recognition:Classification/Pattern recognition:The task of pattern recognition is to assign an The task of pattern recognition is to assign an

input pattern (like handwritten symbol) to one of many input pattern (like handwritten symbol) to one of many classes. This category includes algorithmic classes. This category includes algorithmic implementations such as associative memory.implementations such as associative memory.   

Page 6: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Continue…Continue…..Function approximation:Function approximation:

The tasks of function approximation is to find The tasks of function approximation is to find an estimate of the unknown function f() subject to an estimate of the unknown function f() subject to noise. Various engineering and scientific disciplines noise. Various engineering and scientific disciplines require function approximation.  require function approximation.  Prediction/Dynamic Systems:Prediction/Dynamic Systems:

The task is to forecast some future values of a The task is to forecast some future values of a time-sequenced data. Prediction has a significant time-sequenced data. Prediction has a significant impact on decision support systems.impact on decision support systems.

Prediction differs from Function Prediction differs from Function approximation by considering time factor.approximation by considering time factor.

Here the system is dynamic and may produce Here the system is dynamic and may produce different results for the same input data based on different results for the same input data based on system state (time).system state (time).

Page 7: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

OverviewOverviewBiological inspiration

Artificial neurons and Artificial neural networks

Learning processes

Learning with artificial neural networks

Page 8: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

The Brain

The Brain as an Information Processing System

The human brain contains about 1010 basic units called neurons or nerve cells.

Each neuron in turn, is connected to about 104 other neurons

On average, each neuron is connected to other neurons through about 10,000 synapses.

Page 9: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Neurons and Synapses• The basic computational unit in the nervous system is the nerve cell, or neuron. • A neuron is a small cell that receives electron chemical signals from its various sources and in turn responds by transmitting electrical impulses to other neurons. • A neuron has:

Dendrites (inputs) Cell body Axon (output)

Page 10: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Continue……Continue……A neuron receives input from other neurons A neuron receives input from other neurons

(typically many thousands). (typically many thousands). Inputs sum (approximately). Inputs sum (approximately). Once input exceeds a critical level, the Once input exceeds a critical level, the neuron discharges a spike - an electrical neuron discharges a spike - an electrical pulse that travels from the body, down the pulse that travels from the body, down the axon, to the next neuron(s) (or other axon, to the next neuron(s) (or other receptors). receptors). This spiking event is also called This spiking event is also called depolarization, and is followed by a depolarization, and is followed by a refractory period, during which the neuron is refractory period, during which the neuron is unableunable to fire. to fire.

Page 11: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

CONTINUE…….The axon endings (Output Zone) almost touch the

dendrites or cell body of the next neuron. Transmission of an electrical signal from one neuron to

the next is effected by neurotransmittors, chemicals which are released from the first neuron and which bind to receptors in the second.

This link is called a synapse. The extent to which the signal from one neuron is

passed on to the next depends on many factors, e.g. the amount of neurotransmittor available, the number and arrangement of receptors, amount of neurotransmittor reabsorbed, etc.

Page 12: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Difference between biological Difference between biological neuron and artificial neuronneuron and artificial neuron

Characteristics

Artificial neural network

Biological neural network

Speed Faster in processing information

Slower in processing information

Processing In a sequential mode(One after another)

Parallel operations

Size and complexity

Small size and less complex

Big size and more complex

Fault tolerance Not fault tolerant(information corrupted in the memory cannot be retrieved)

Tolerant

Control mechanism

Control unit monitors all the activities

There is no central control for processing information in the brain

Page 13: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

STRUCTURE OF NEURONSTRUCTURE OF NEURON

Page 14: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Artificial neuronsArtificial neurons

Neurons work by processing information. They receive and provide information in form of spikes.

The McCulloch-Pitts model

Inputs

Outputw2

w1

w3

wn

wn-1

. . .

x1

x2

x3

xn-1

xn

y)0(;

1

fyxwon

iii

Page 15: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Artificial neural networksArtificial neural networks

Inputs

Output

An artificial neural network is composed of many artificial neurons that are linked together according to a specific network architecture. The objective of the neural network is to transform the inputs into meaningful outputs.

Page 16: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

A vague description is as followsA vague description is as follows: :

• An ANN is a network of many very simple An ANN is a network of many very simple processors ("units"), each possibly having a processors ("units"), each possibly having a (small amount of) local memory. (small amount of) local memory.

• The units are connected by unidirectional The units are connected by unidirectional communication channels ("connections"), communication channels ("connections"), which carry numeric (as opposed to which carry numeric (as opposed to symbolic) data. symbolic) data.

• The units operate only on their local data The units operate only on their local data and on the inputs they receive via the and on the inputs they receive via the connections. connections.

Page 17: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

CONTINUE…..CONTINUE….. The design motivation is what The design motivation is what

distinguishes neural networks from other distinguishes neural networks from other mathematical techniquesmathematical techniques

A neural network is a processing device, A neural network is a processing device, either an algorithm, or actual hardware, either an algorithm, or actual hardware, whose design was motivated by the whose design was motivated by the design and functioning of human brains design and functioning of human brains and components thereof. and components thereof.

Most neural networks have some sort of Most neural networks have some sort of "training" rule whereby the weights of "training" rule whereby the weights of connections are adjusted on the basis of connections are adjusted on the basis of presented patterns. presented patterns.

Page 18: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

CONTINUE…..CONTINUE…..

In other words, neural networks "learn" from In other words, neural networks "learn" from examples, just like children learn to recognize examples, just like children learn to recognize dogs from examples of dogs, and exhibit some dogs from examples of dogs, and exhibit some structural capability for generalization. structural capability for generalization.

Neural networks normally have great potential Neural networks normally have great potential for parallelism, since the computations of the for parallelism, since the computations of the components are independent of each other.  components are independent of each other.  

Biological neural networks are much more Biological neural networks are much more complicated in their elementary structures than complicated in their elementary structures than the mathematical models we use for ANNs.the mathematical models we use for ANNs.

They typically consist of many simple They typically consist of many simple processing units, which are wired together in a processing units, which are wired together in a complex communication network.complex communication network.

Page 19: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Basic ConceptsBasic Concepts

Neural Network

Input 0 Input 1 Input n...

Output 0 Output 1 Output m...

A Neural Network generally A Neural Network generally maps a set of inputs to a set of maps a set of inputs to a set of outputsoutputs

Number of inputs/outputs is Number of inputs/outputs is variablevariable

The Network itself is composed The Network itself is composed of an arbitrary number of nodes of an arbitrary number of nodes with an arbitrary topologywith an arbitrary topology

Page 20: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Activation functionActivation function Activation function is used to calculate the output response Activation function is used to calculate the output response

of a neuron. of a neuron. A few linear and non linear activation functions areA few linear and non linear activation functions are 1. Identify function1. Identify function 2. sigmoidal functions2. sigmoidal functions 3. Binary sigmoidal functions3. Binary sigmoidal functions 4. Bipolar sigmoidal function4. Bipolar sigmoidal function 5. Signum function5. Signum function 6.Binary step function6.Binary step function 7. Bipoloar binary function 7. Bipoloar binary function 8. Uni poloar binary function 8. Uni poloar binary function

Page 21: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Neural network Neural network ArchitecturesArchitectures

1.Single layer feedforward network1.Single layer feedforward network

- only input and output layer- only input and output layer

2. Multi layer Feedforward network2. Multi layer Feedforward network

- Input,output and hidden layer.- Input,output and hidden layer.

3. Recurrent networks3. Recurrent networks

- Contain feedback link - Contain feedback link

Page 22: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Different Forms of Learning:Different Forms of Learning:– Learning agent receives feedback with Learning agent receives feedback with

respect to its actions (e.g. using a teacher)respect to its actions (e.g. using a teacher) Supervised LearningSupervised Learning: feedback is : feedback is

received with respect to all possible received with respect to all possible actions of the agentactions of the agent

Reinforcement LearningReinforcement Learning: feedback is : feedback is only received with respect to the taken only received with respect to the taken action of the agentaction of the agent

– Unsupervised Learning: Unsupervised Learning: Learning when Learning when there is no hint at all about the correct there is no hint at all about the correct actionaction

Learning methodsLearning methods

Page 23: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Classification of learningClassification of learning FigFig

Neural networks learning algorithm

HebbianStochasticError correction

GradientDescent

BackPropagation

Least MeanSquare

Unsupervised learningSupervised learning(Error based)

Competitative

Reinforced learning(output based)

Page 24: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Supervised LearningSupervised Learning Every input pattern that is used to train the n/w Every input pattern that is used to train the n/w

is associated with an output pattern,which is is associated with an output pattern,which is the target or the desired pattern.the target or the desired pattern.

A A teacherteacher is assumed to be present during the is assumed to be present during the learning process, when a comparison is made learning process, when a comparison is made between the networks computed output and between the networks computed output and the correct expected , to deternmine the error.the correct expected , to deternmine the error.

The error can then be used to change n/w The error can then be used to change n/w parameters , which result is an improvement in parameters , which result is an improvement in performanceperformance

Page 25: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Block diagram of supervised Block diagram of supervised learninglearning

XXAdaptive Network

Distance Generator

O

Learning signal

d

Page 26: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Unsupervised LearningUnsupervised Learning The target output is not presented to the n/wThe target output is not presented to the n/w No teacherNo teacher to present the desired patterns. to present the desired patterns. Systems learns of its own by discovering a d Systems learns of its own by discovering a d

adapting to structural features in the input adapting to structural features in the input patterns.patterns.

OO XX

Adaptive NetworkW

Page 27: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Reinforced LearningReinforced Learning A teacher though available, does not A teacher though available, does not

present the expected answer but only present the expected answer but only indicates is the computed output is indicates is the computed output is correct or incorrect.correct or incorrect.

The information provided helps the n/w The information provided helps the n/w in its learning process.in its learning process.

A reward is given for a correct answer A reward is given for a correct answer computed and a penalty for a wrong computed and a penalty for a wrong answer. But , reinforced learning is not answer. But , reinforced learning is not one of the popular forms of learning.one of the popular forms of learning.

Supervised and unsupervised are most Supervised and unsupervised are most popular formspopular forms

Page 28: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Various RulesVarious Rules Those learning methods have found expression Those learning methods have found expression

through various learning rules,through various learning rules, Some of the rules are,Some of the rules are, 1. Hebbian learning rule1. Hebbian learning rule 2. Perceptron learning rule2. Perceptron learning rule 3. Delta learning rule3. Delta learning rule 4. Widrow-Hoff learning rule4. Widrow-Hoff learning rule 5. Correlation learning rule5. Correlation learning rule 6. Winner-Take-All Learning rule6. Winner-Take-All Learning rule 7. Outstar Learning rule7. Outstar Learning rule

Page 29: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Hebbian Learning ruleHebbian Learning rule Proposed by Hebb(1949)Proposed by Hebb(1949) Based on correlative weight adjustmentBased on correlative weight adjustment Oldest learning mechanismOldest learning mechanism The input output patterns pairs (Xi,Yi ) are The input output patterns pairs (Xi,Yi ) are

associated by the weight matrix w, associated by the weight matrix w, unknown as the correlation matrix.unknown as the correlation matrix.

It is computed as,It is computed as,

W W i+1i+1 = W = Wii + C Oi xi + C Oi xi

Page 30: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Perceptron Learning rulePerceptron Learning rule For the perceptron learning rule, the For the perceptron learning rule, the

learning signal is the difference learning signal is the difference between the desired and actual between the desired and actual neuron's response neuron's response

Learning is supervisedLearning is supervised r=d r=d i i - o - o i i

The rule isThe rule is , ,

W W i+1i+1 = W = Wii + C (d + C (dii - O - Oii )xi )xi

Page 31: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Perceptron Learning rulePerceptron Learning rule

TLUTLU XX11 W Wi1 i1

XX22 W Wi2 net ii2 net i

WWi3 i3 OOii

XX33

C (di-oi)X i C (di-oi)X i

XX44 W Wi4i4

XX ddii-O-Oii d dii

CC

+ 1

0 net -1

Page 32: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Delta learning ruleDelta learning rule It is only valid for continuous It is only valid for continuous

activation function.activation function. Supervised learning.Supervised learning.

f’ (net i) is the derivative of the f’ (net i) is the derivative of the activation function f(net)activation function f(net)

W W i+1i+1 = W = Wii + C (d + C (dii - O - Oii ) f’(net i)xi ) f’(net i)xi

Page 33: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Delta Learning ruleDelta Learning rule

XX11 W Wi1 continuous perceptron i1 continuous perceptron

XX22 W Wi2 net ii2 net i

WWi3 i3 OOii

XX33

W i W i

XX44 W Wi4i4

X X ddii-O-Oii d dii

CC

Page 34: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

PerceptronPerceptronThe perceptron is a computational model The perceptron is a computational model

of the retina of the eye and hence, is named of the retina of the eye and hence, is named ‘Perceptron’ .‘Perceptron’ .

The network comprises three units, The network comprises three units, 1. Sensory unit (S)1. Sensory unit (S)2. Association Unit (A)2. Association Unit (A)3. Response Unit (R)3. Response Unit (R)

The S unit comprising 400 photodetectors The S unit comprising 400 photodetectors receives input images and provides a 0/1 receives input images and provides a 0/1 electric signal as output.electric signal as output.

If the output signals exceed a threshold, then If the output signals exceed a threshold, then the photodetector outputs 1 else 0.the photodetector outputs 1 else 0.

Page 35: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Rosenblatt’s original Perceptron modelRosenblatt’s original Perceptron model

Sensory unit(s) Association ResponseSensory unit(s) Association Response

unit(A) unit(R)unit(A) unit(R)

O/PO/P

Fixed W adjustable WFixed W adjustable W

Photo detectorsPhoto detectors

Page 36: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Continue….Continue…. Photo detector are randomly connected Photo detector are randomly connected

to the association unit Ato the association unit A The unit A comprises feature The unit A comprises feature demons or demons or

predicatespredicates TheThe predicates predicates examine the output of the examine the output of the

S unit for specific features of the image.S unit for specific features of the image. The R unit comprises pattern recognizers The R unit comprises pattern recognizers

or perceptrons, which receives the results or perceptrons, which receives the results of the predicate , also in binary form.of the predicate , also in binary form.

While the weights of the S and A units are While the weights of the S and A units are fixed, those of R are adjustable.fixed, those of R are adjustable.

Page 37: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Output calculationOutput calculation The output of the R unit could be such The output of the R unit could be such

that if the weighted sum of its inputs is that if the weighted sum of its inputs is less the or equal to 0, the output is 0 less the or equal to 0, the output is 0 otherwise it is the weighted sum itself.otherwise it is the weighted sum itself.

The training algorithm of the perceptron is The training algorithm of the perceptron is a supervised learning algorithma supervised learning algorithm

Yj = f(net j) = 1, if net j > 0 = 0, otherwiseWhere, net j = ;

1

n

iiixw

Page 38: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

ADALINEADALINE Adaptive Linear network.Adaptive Linear network. Benard widrow of Stanford UniversityBenard widrow of Stanford University Makes use if Supervised learningMakes use if Supervised learning A simple ADALINE networks is belowA simple ADALINE networks is below W0W0

W1 Y OutputW1 Y Output

w2w2

WnWn

Thresholding functionThresholding function

Σx0

x1

x2

xn

Page 39: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Continue…Continue… There is only one output neuron and the There is only one output neuron and the

output values are bipolar(-1 or +1)output values are bipolar(-1 or +1) The learning algorithm is Least mean The learning algorithm is Least mean

square or Delta rule.square or Delta rule.

ADALINE n/w s has had the most ADALINE n/w s has had the most successful applications because it is successful applications because it is virtually in all high speed modems and virtually in all high speed modems and telephone switching systems to cancel telephone switching systems to cancel the echo in long distance the echo in long distance communication circuits.communication circuits.

W W i+1i+1 = W = Wii + C (d + C (dii - O - Oii ) f’(net i)xi ) f’(net i)xi

Page 40: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

MADALINE NetworkMADALINE Network A MADALINE(Many ADALINE) network is A MADALINE(Many ADALINE) network is

created by combining a number of adalinescreated by combining a number of adalines The n/w of ADALINES can span many layers.The n/w of ADALINES can span many layers. A simple MADALINE N/W is shown belowA simple MADALINE N/W is shown below Inputs outputsInputs outputs

:Adaline nw:Adaline nw

A

A

A

A

A A

A

A

Page 41: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Hop field networkHop field network Described by J.J.HopfieldDescribed by J.J.Hopfield Topology is simpleTopology is simple It consists of ‘n’ neurons which are all It consists of ‘n’ neurons which are all

networked with each other.networked with each other. A Hopfield network is able to recognize A Hopfield network is able to recognize

unclear pictures correctly.unclear pictures correctly. The discrete Hopfield net is a fully The discrete Hopfield net is a fully

interconnected neural net with each unit interconnected neural net with each unit connected to every other unit.connected to every other unit.

The net has symmetric weights with no self The net has symmetric weights with no self connections that is all the diagonal elements connections that is all the diagonal elements of the weight matrix of a Hopfield net are of the weight matrix of a Hopfield net are zero.zero.

Page 42: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

continuecontinue Wij = Wji and Wii = 0Wij = Wji and Wii = 0

Weight matrix is,Weight matrix is,

[0 W12 W13 ……. W1n[0 W12 W13 ……. W1n

W = W21 0 W23 ……. W2n]W = W21 0 W23 ……. W2n]

W31 W32 0 …… W3nW31 W32 0 …… W3n

. . . …… .. . . …… .

. . . ….. .. . . ….. .

Wn1 Wn2 Wn3 ….. 0 ]Wn1 Wn2 Wn3 ….. 0 ]

Some times called connectivity matrixSome times called connectivity matrix

Page 43: Intro to Neural Networks UNIT-II.  Neural networks is a branch of "Artificial Intelligence“.  “Artificial Neural Network” is a system loosely modeled.

Single-layer feedback neural Single-layer feedback neural networknetwork

-1 -1

i1 T1i1 T1

VV11 wn1wn1 VV11

i2i2

VV22 w2nw2n T2T2 VV2 2

Wn2Wn2

VVnn -1 -1 VVnn

1 2

n