There are two basic categories:There are two basic categories:1.1. Feed-forward Neural NetworksFeed-forward Neural Networks

These are the nets in which the signals These are the nets in which the signals flow from the input units to the output flow from the input units to the output units, in a forward direction.units, in a forward direction.

They are further classified as:They are further classified as:1.1. Single Layer NetsSingle Layer Nets2.2. Multi-layer NetsMulti-layer Nets

2.2. Recurrent Neural NetworksRecurrent Neural Networks These are the nets in which the signals These are the nets in which the signals

can flow in both directions from the can flow in both directions from the input to the output or vice versa.input to the output or vice versa.

XX11

XX22

XXnn

InputUnits

YY11

YY22

YYmm

OutputUnits

ww1111

ww2121

ww3131

ww1212

ww3232

ww1313

ww2m2m

wwnmnm

ww2222

Biological Neurons In Action

HiddenUnits

ZZ11

ZZjj

ZZpp

OutputUnits

YY11

YYkk

YYmm

InputUnits

XXii

XX11

XXnn

ww1111

wwi1i1

wwn1n1

ww1j1j

wwijij

wwnjnj

ww1p1p

wwipip

wwnpnp

vv1111

vvj1j1

vvp1p1

vv1k1k

vvjkjk

vvpkpk

vv1m1m

vvjmjm

vvpmpm

Supervised TrainingSupervised Training Training is accomplished by presenting a Training is accomplished by presenting a

sequence of training vectors or patterns, each sequence of training vectors or patterns, each with an associated target output vector. with an associated target output vector.

The weights are then adjusted according to a The weights are then adjusted according to a learning algorithm. learning algorithm.

During training, the network develops an During training, the network develops an associative memory. It can then recall a stored associative memory. It can then recall a stored pattern when it is given an input vector that is pattern when it is given an input vector that is sufficiently similar to a vector it has learned.sufficiently similar to a vector it has learned.

Unsupervised TrainingUnsupervised Training A sequence of input vectors is provided, but no A sequence of input vectors is provided, but no

traget vectors are specified in this case.traget vectors are specified in this case. The net modifies its weights and biases, so that The net modifies its weights and biases, so that

the most similar input vectors are assigned to the most similar input vectors are assigned to the same output unit.the same output unit.