1

Financial Informatics –XVII:Unsupervised Learning

1

Khurshid Ahmad, Professor of Computer Science,

Department of Computer Science

Trinity College,Dublin-2, IRELAND

November 19th, 2008.https://www.cs.tcd.ie/Khurshid.Ahmad/Teaching.html

2

PreambleNeural Networks 'learn' by adapting in accordance with a training regimen: Five key algorithms.

ERROR-CORRECTION OR PERFORMANCE LEARNING

HEBBIAN OR COINCIDENCE LEARNING

BOLTZMAN LEARNING (STOCHASTIC NET LEARNING)

COMPETITIVE LEARNING

FILTER LEARNING (GROSSBERG'S NETS)

3

PreambleNeural Networks 'learn' by adapting in accordance with a training regimen: Five key algorithms.

California sought to have thelicense of one of the largest auditing firms(Ernst & Young) removed because of their rolein the well-publicized collapse of Lincoln Savings& Loan Association. Further, regulatorscould use a bankruptcy

4

ANN Learning Algorithms

ENVIRONMENT

LEARNING SYSTEM

Error Signal

Vector describing the environment

Desired response

Actual

Response

- +

TEACHER

5ENVIRONMENTLEARNING SYSTEM

Vector describing state of the

environment

ANN Learning Algorithms

6

ANN Learning Algorithms

ENVIRONMENT CRITIC

LEARNING SYSTEMActions

State-vector input

Primary Reinforcement

Heuristic Reinforcement

7

Hebbian Learning

DONALD HEBB, a Canadian psychologist, was interested in investigating PLAUSIBLE MECHANISMS FOR LEARNING AT THE CELLULAR LEVELS IN THE BRAIN. (see for example, Donald Hebb's (1949) The Organisation of Behaviour. New York: Wiley)

8

Hebbian Learning

HEBB’s POSTULATE: When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic changes take place in one or both cells such that A's efficiency as one of the cells firing B, is increased.

 

9

Hebbian Learning

Hebbian Learning laws CAUSE WEIGHT CHANGES IN RESPONSE TO EVENTS WITHIN A PROCESSING ELEMENT THAT HAPPEN SIMULTANEOUSLY. THE LEARNING LAWS IN THIS CATEGORY ARE CHARACTERIZED BY THEIR COMPLETELY LOCAL - BOTH IN SPACE AND IN TIME-CHARACTER.

 

10

Hebbian Learning

 

LINEAR ASSOCIATOR: A substrate for Hebbian Learning Systems

x1

Output y’

x2 x3

y1y’1 y2y’2 y3y’3

Input x

w11w12w13

w21w22w23

w31w32w33

11

Hebbian Learning

 

A simple form of Hebbian Learning Rule

,jkoldjk

newjk xyww

where h is the so-called rate of learning and x and y are the input and output respectively.

This rule is also called the activity product rule.

12

Hebbian Learning

 

A simple form of Hebbian Learning Rule

x 1,

y 1

x m ,y m

w y 1

x 1

T y 2 x 2

Ty n xn

T

If there are "m" pairs of vectors,

to be stored in a network ,then the training sequence will change the weight-matrix, w, from its initial value of ZERO to its final state by simply adding together all of the incremental weight change caused by the "m" applications of Hebb's law:

13

Hebbian Learning

 

A worked example: Consider the Hebbian learning of three input vectors:

5.1

1

1

0

;

5.1

2

5.0

1

;

0

5.1

2

1

)3()2()1( xxx

in a network with the following initial weight vector:

5.0

0

1

1

w

14

Hebbian Learning

 

A worked example: Consider the Hebbian learning of three input vectors:

5.1

1

1

0

;

5.1

2

5.0

1

;

0

5.1

2

1

)3()2()1( xxx

in a network with the following initial weight vector:

5.0

0

1

1

w

15

Hebbian Learning

 

A worked example: Consider the Hebbian learning of three input vectors:

5.1

1

1

0

;

5.1

2

5.0

1

;

0

5.1

2

1

)3()2()1( xxx

in a network with the following initial weight vector:

5.0

0

1

1

w

16

Hebbian Learning

 

A worked example: Consider the Hebbian learning of three input vectors:

5.1

1

1

0

;

5.1

2

5.0

1

;

0

5.1

2

1

)3()2()1( xxx

in a network with the following initial weight vector:

5.0

0

1

1

w

17

Hebbian Learning

 

The worked example shows that with discrete f(net) and =1, the weight change involves ADDING or SUBTRACTING the entire input pattern vectors to and from the weight vectors respectively.

Consider the case when the activation function is a continuous one. For example, take the bipolar continuous activation function:

.0

;1)( )*(exp12

where

netf net

18

Hebbian Learning

 

The worked example shows that with bipolar continuous activation function indicates that the weight adjustments are tapered for the continuous function but are generally in the same direction:

Vector Discrete Bipolar f(net)

Continuous Bipolar f(net)

x(1) 1 0.905

x(2) -1 -0.077

x(3) -1 -0.932

19

Hebbian LearningThe details of the computation for the three steps with a discrete bipolar activation function are presented below in the notes pages. The input vectors and the initial weight vector are:

5.1

1

1

0

;

5.1

2

5.0

1

;

0

5.1

2

1

)3()2()1( xxx

5.0

0

1

1

w

20

Hebbian LearningThe details of the computation for the three steps with a continuous bipolar activation function are presented below in the notes pages. The input vectors and the initial weight vector are:

5.1

1

1

0

;

5.1

2

5.0

1

;

0

5.1

2

1

)3()2()1( xxx

5.0

0

1

1

w

21

Hebbian Learning

 

Recall that the simple form of Hebbian learning law suggests that the repeated application of the presynaptic signal xj leads to an increase in yk and therefore exponential growth that finally drives the synaptic connection into saturation.

jkjkoldjk

newjk xywww

A number of researchers have proposed ways in which such saturation can be avoided. Sejnowski has suggested that

.

&;

),()(

k

j

jkjk

yofvalueaveragedtimethey

xofvalueaveragedtimethex

wherexxyyw

22

Hebbian Learning

 

The Hebbian synapse described below is said to involve the use of POSITIVE FEEDBACK.

jkjkoldjk

newjk xywww

23

Hebbian Learning

 

What is the principal limitation of this simplest form of learning?

 The above equation suggests that the repeated application of the input signal leads to an increase in , and therefore exponential growth that finally drives the synaptic connection into saturation. At that point of saturation no information cannot be stored in the synapse and selectivity will be lost. Graphically the relationship with the postsynaptic activityis a simple one: it is linear with a slope .

jkjkoldjk

newjk xywww

24

Hebbian Learning

 

The so-called covariance hypothesis was introduced to deal with the principal limitation of the simplest form of Hebbian learning and is given as

 where and denote the time-averaged values of the pre-synaptic and postsynaptic signals.   

))(())(()( xnxynynw jkkj

25

Hebbian Learning

 

  If we expand the above equation:

the last term in the above equation is a constant and the first term is what we have for the simplest Hebbian learning rule:

))()()()()( xyxnynxynxnynw kjjkkj

xy

xny

nxy

nwnw

k

j

kjkj SimpleModified

)(

)(

)()(

))(())(()( xnxynynw jkkj

26

Hebbian Learning

 

Graphically the relationship ∆wij with the postsynaptic activity yk is still linear but with a slope

and the assurance that the straight line curve changes its rate of change at

and the minimum value of the weight change ∆wij is

))(( xnx j

))(( xnx j

y