COMP305. Part I .

Transcript of COMP305. Part I .

COMP305. Part I.Artificial neural networks.
Topic 3.

Learning Rules of the Artificial Neural Networks.
Hebbs rule (1949).Hebb conjectured that a particular type of use-dependent modification of the connection strength of synapses might underlie learning in the nervous system.
Hebbs rule (1949).Hebb introduced a neurophysiological postulate :

When an axon of cell A is near enough to excite a cell B and repeatedly and persistently tales part in firing it, some growth process or metabolic change takes place in one or both cells, such that As efficiency as one of the cells firing B, is increased.
Hebbs rule (1949).The simplest formalisation of Hebbs rule is to increase weight of connection at every next instant in the way:

(1)where (2)
Hebbs rule (1949). (1)where (2)here wjik is the weight of connection at instant k,

wjik+1 is the weight of connection at the following instant k+1,

Dwjik is increment by which the weight of connection is enlarged,

C is positive coefficient which determines learning rate,

aik is input value from the presynaptic neuron at instant k,

Xjk is output of the postsynaptic neuron at the same instant k.
Hebbs rule (1949). (1)where (2)

Thus, the weight of connection changes at the next instant only if both preceding input via this connection and the resulting output simultaneously are not equal to 0.

Equation (2) emphasises the correlation nature of a Hebbian synapse. It is sometimes referred to as the activity product rule.

Hebbs original learning rule (2) referred exclusively to excitatory synapses, and has the unfortunate property that it can only increase synaptic weights, thus washing out the distinctive performance of different neurons in a network, as the connections drive into saturation..

However, when the Hebbian rule is augmented by a normalisation rule, e.g. keeping constant the total strength of synapses upon a given neuron, it tends to sharpen a neurons predisposition without a teacher, causing its firing to become better and better correlated with a cluster of stimulus patterns.
Normalised Hebbs rule. (1)where (2)

normalisation: (3)

Hebbs rule plays an important role in studies of ANN algorithms much younger than the rule itself, such as unsupervised learning or self-organisation.
Normalised Hebb in practice. Input unit No

1

2

3

4

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=0 C=1

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

Continue

w01w02w03w040.50.50.50.5

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

1

2

3

4t=1 C=1

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

Continue

w11w12w13w140.670.220.670.22

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

1

2

3

4t=2 C=1

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

Continue

w21w22w23w240.700.090.700.09

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

1

2

3

4t=3 C=1

1

2

3

4t=4 C=1

1

2

3

4t=4 C=1

1

2

3

4t=4 C=1

1

2

3

4t=4 C=1

Continue

w31w32w33w340.710.040.710.04

1

2

3

4t=4 C=1

1

2

3

4t=4 C=1

1

2

3

4t=4 C=1

1

2

3

4t=4 C=1

1

2

3

4t=5 C=1

1

2

3

4t=5 C=1

1

2

3

4t=5 C=1

1

2

3

4t=5 C=1

STOP!!!!

w41w42w43w440.710.020.710.02

1

2

3

4

1

2

3

4TestI do not know you

COMP305. Part I .

Documents

Transcript of COMP305. Part I .