@

NeuralNetworks, 9 8

i

Memory and Learning of Sequential Patterns byNeural Networks

o

(Received 2 July 1995; 3

Nonmonotone

d i oi

n o i ai b i a

o ia A

i tt a o

t b a a o (

a o

o

o a

a

i a ao

t a

i

a t

b to

oi

b oI i t

ot d s i t a

o

b e

io o

i a i 1 Wi

Ai o

ob

oi i

i i i ao

b e

M

oa i i

u +t K = 1

o a

io

te

e

oa

i

t a

o

a o

oi

i aa a

u

a wo

i *a

t = t

()nXf(f + 1 = S&2 ~ WijXj(t) ,

j = 1

x i o it

= 1 u > – uo i r b a

1X T

W = [Wij], w a

+

o

m . . .= , = t b t

o = O p # v aS S + . . . + * i i

b

w = S’+’S’T.n

i ti a o

S o a i a a

(= (Lwx(o))s’+’ )=s’,n

S i a t = 1 I= = o i

u a ao 2

oX t

S b m

2 o iFlow at

s~ (/.4=0,..., m – ia t

= A a i St

I i b

a oi i

a

ai

X ba

a a X So

X S

aS

a

a

a aX i b

ta b o

a bi

aS i

o

i a

o

A i to

i t

b i ia a

&a a

i io i

i Bt

oo

i ia o i

b B i i i

u ai S i

a ia i b

t si S R I o i

S i a a oI b i

a a3a i S i

I i aS t S

ib a

o a io

a i

3 o# S ai

t

M

o at

o i

a

o

t

i b

M

o~ a

i 1 I

1 – @ 1+=

1 + 1 + J

h ~ i a

~ = 1 I= c = h =

= – 1i

= i t bo i b

x =X a

= ( = o, ... ,m – 1) be n-

a

- + Q ~ Q ~

m i iW Q

Q – > I iQ a

w tso

w Qo 2i I t

a ow a b

1 1 Qv = . . .m – 1 Q = a

i 4 w = Q= 1 1 b

o q q #w b

S = i Z1 1o i

v Wi

T oA

o m = i ao n = t

4

4 o o I = 4 i Ai b i

o

1.0

0.2

0.0

0 2.0 4.0 6.0 8.0 10.0 12.0

o .. . , i ii X=

X

(9)

= 1 X = S i= 1

5 I=

X =1

O Wa o P

p t p aX t

a S t

Oa i 6

s pO = P s OpO t P

p u tP P 2 O

1 Ia t

1 t O Xt

P p1.0

0.8

0.6~mg 0.4

0.2

0.0

0 2.0 4.0 6.0 8.0 10.0 12.0

Wi a i 5 oP =

M

1.0-

0.8-

0.6-

s0.4-

0.2-

0.0t S4

I I 1 1 1 I 1 1 1 I 1 10.0 0.2 0.4 0.6 0.8 1.0

o

b= p = O t

= =

S at & a

7 i aX a

Q1) are

E= ? S oa

XQ

= 1+ 6

Qi t a a

it W

i aa o i

tX i

t ii

i oa i o

a oo o

i Ii b

o iK

i i ~ & 1 i m oi I b

i 1 o oo

o m bi

8 oo i

W a io p~

i 6 Wi pA

iX

Z wt b o

1.0piI Plfj

*~& 0.4

0.2

0.0

0 5.0 10.0 15.0 20.0 25.0 30.0

L ( = 8 Q6 = O ... b

1483

i b a =

io

t

bb

a a < a < iX d

a ba a

I = 4 9

Xp # 1 b

m 1 i aQ Q i

pI–po

ao

1=f a

i oX E = = q: i

= f

t ( ia X Q o i i

( i

W aS w b

oS S i

0X i

b i a o W

aT w

o

Q l Q = 1~

n k

0C Q

1484 M

Q

X Ia

k ? Then sf (O <f <4) are 1 ifi < n – – x = 1

i i X EX a b

X iV= W

i b

v=: ~/A=o

= !+&+l

V < hi p i

~ = . . . = = 1 v v <a t 1 I X

a i< X i

o t

a Xp I

o VX

S a i5 O i

j < t bt O

ratio of

( = O to Y A a Y i i

t a St S i

b i ib o

i

4

b ai i

o o i Ii i t

i wi i( b i

a aI w a

t to

io

ia a z i t

w b

‘U, + ~Wijyj+Zi.j=l

(12)

Ii z a

t

(13)

C a ( > O #i a o ( >

Z = ia o

t b io w

z(t) = @s(r), (14)

S = ~ = + ~ i ao

I ba

,r — = ‘Wij + @/3.Si~j,

d(15)

i i iX S b

S Y

io

a aX

i it a

Xa S

S = aX = S Z = i

a o

i aX i

=S s

S = X

SS t

a os–

S X

S a a ii o X

i S~ i X S

S b oi o

Bo X

a Io X

S i it w

~o X o

ao

a i

oa o n =

S Q a t = O a aa i

t Q a t =Q s i q: # q~+l

b S tt

9 it i

1486 M

.

X Q S QT i o o

a = 2 =

p.

(16)

SX S aW

p.

o S i

t b XS o S i

A o Xo Z = s X t

S i i 4o t

o 6o 1

aPO= 0.3 i n i

Z t i 6 8o p t Q

o S nw o S

1.0)

,

6 o o = O .. . i ih

1.0

0.8

0.6

e0.4

0.2

0.01

x

0.0 0.2 0.4 0.6 0.8 1.0P

o PI-PO plane.

o tP

a a a i aX

S t ad

ii t

S ao o

o S i

o

Xt iQ Q

o

a io X i i 0

o SX S i

bt o

w X S it d s i t

a a ao

o x # s

i b

~ = (l!’[yil, (17)

& a b ao i

t ot

o ii

o p 4o

o X i t o S i W

5

A

a = 5a i

(

1488

Ia

i i i a

A a a

i

i

o nb a

i o aa o

bo

a o

b

i

ti

Ii

1 i ia

i

ba

a a

o oo y

ta b o

t

a

a o a

M

o o

S bo IEEE Transactions on

Computers, C-21, 1197-1206.Y b a

IEEE Transactions on NeuralNetworks, 5, 974-981.

& Ri Journal of Plrysics

A, 26, L507–L513.& S

Neural Networks,2, 37$385.

& M oo

Networks, 8, 391=$04.J J

oProceedings of the National Acaakmy of Sciences of the

USA, 81, 3088–3092.D b

Proceedings of the National Academy of ofthe USA, 83, 946%9473.

K O o a aNetwork, 2, 237–243.

Y oi

M A o o ai Systems and

Computers in Japan, 23-4, 14-24.M

Neural Networks, 6, 11$126.M o a b

Proceedings of the IEEEInternational Conferenceon Neural Networks, 2

M oo Cognitive Brain Research, i

& K obProceedings of the International Neural

Network Conference, 2& I o a

aNeural Networks, 6, 1061–1067.

M A oNeural Networks, 8, 833+38.

B A iNeural Computation, 1

F J o tPhysical Review Letters, 59, 2229-

2232.& T

o oo Physical Review E,

48,867-897.& I i

Physical Review Letters, 57, 2861–2864.

& S

IEEE Transactions aNeural Networks, Networks, 6, 167-176.

& S o