Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez † Giovanni Egidio...

Polynomial Discrete Time Cellular Neural Networks

Eduardo Gomez-Ramirez † Giovanni Egidio Pazienza‡

† LIDETEA, POSGRADO E INVESTIGACION Universidad La Salle – México, D.F. ‡ Department d’Electronica, EALS

Universitat “Ramon Llull” – Barcelona, Spain

Outline Cellular Neural Networks (CNN)

Introduction and Objective Genetic Algorithms (GA) Polynomial Discrete Time CNNs

(PDTCNNs) XOR Problem Game of Life

Learning vs Design Conclusions and future workIntro CNN & GA Polyn. CNN XOR GoL Conclusions

CNN: Introduction CNN for complex task (linearly

nonseparable data)

Multilayer CNNs Include more degrees of freedom for the

output state of each layer Search in a finite set of templates

Single layer: Polynomial CNNs

Intro CNN & GA Polyn. CNN XOR GoL Conclusions

Improve the representation power of a single layer CNN including a simple nonlinear term to solve problems with linearly nonseparable data (XOR)


Objective

CNN: mathematical model


The simplified mathematical model is:

where xc is the state of the cell, uc the input and yc the output

iubtyatxdt

dx dcd

dcd

cc

)()(

)1)(1)((2

1 txtxy ccc

CNN: Activation Function


CNN: Block Diagram


CNN: Discrete Model


Computing x(∞), the model can be represented as

iubnyanx dcd

dcd

c )()(

0)(,1

0)(,1)(

nx

nxny

c

c

c

using the following activation function

Steps: Crossover C(Fg) Mutation M(*) Adding random parent Ag()

GA: main steps proposed


dc

bc

da

ba

FC

dcMbaMM

MF

g

g 212

1 ,,

GA: Crossover


125.0125.

000

000

A

125.125.0

000

000

B I=0

125.00

000

000

A

125.125.125.

000

000

B I=0

2121

2211

2222

2121

2211

1111

1

cccc

bbbb

aaaa

cccc

bbbb

aaaa

P

Individual 1

Individual 2

GA: Crossover


a1

b1

c1

a2

b2

c2

GA: Mutation


mij

mijmij PrF

PrFPFM

)(

)(),(

where rU(0,1) is a random variable with uniform distribution defined on a probability space (,,P),

GA: Mutation (resolution)


'2

'1

'2

'1

'2

'2

'1

'1

'2

'2

'2

'2

'2

'1

'2

'1

'2

'2

'1

'1

'1

'1

'1

'1

'1

cccc

bbbb

aaaa

cccc

bbbb

aaaa

P

125.0125.

000

000

A

125.125.0

000

000

B I=0

125.00

000

000

A

125.125.125.

000

000

B I=0

Individual 1

Individual 2

a1

b1

c1

a2

b2

c2

Population=sons + sons mutated

g

g

g

C FA

M C F

ggn

pgg AOnAS pmin,

GA: Selecting Parents


)(1 wF

FF

g

gg

GA: Adding Random Parent


THEOREM 1: (Weierstrass’s Approximation Theorem)

Let g be a continuous real valued function defined on a closed interval [a,b]. Then, given any positive, there exists a polynomial y (which may depend on ) with real coefficients such that:

For every x [a,b].

Polynomial Discrete Time Cellular Neural Network

)()( xyxg


THEOREM 2 *: Any Boolean Function of n-variables can be realized using a Polynomial Threshold gates of order sn.The quadratic threshold gate can be defined:

And s is the number of inputs and T is the threshold constant.

Polynomial Discrete Time Cellular Neural Network

otherwise

Txxwxwify

n

i

n

ijiijii

0

11 1


* N. J. Nilsson. The Mathematical Foundations of Learning Machines. McGraw Hill, New York, 1990.

PDTCNN: the model (I)

cdd

cNd

dcd

cNd

dcd

c iyuguBkyAkxrr

),()()()()(

0)1(1

0)1(1))1(()(

kxf

kxifkxfky

c

ccc

)0(

)0(

)0(

),(

)(

)(

)(

cd

cNd

cd

cd

cNd

cd

dd

cNd

cd

dd

yuP

uyP

yuP

yug

r

r

r

)(

)(

)(

),(

)(

)(

)(

kyuP

ukyP

kyuP

yug

cd

cNd

cd

cd

cNd

cd

dd

cNd

cd

dd

r

r

r


PDTCNN: the model (II)

)()(),()()(

kuQkyPyug d

cNd

cd

d

cNd

cd

dd

rr

cdd

cNd

dcd

cNd

dcd

c iyuguBkyAkxrr

),()()()()(


PDTCNN: Solving XOR problemSome papers: Z. Yang, Y. Nishio, A. Ushida,

Templates and algorithms for two-layer cellular neural networks. IJCNN’02, 2002.

F. Chen, G. He, G. Chen & X. Xu,

Implementation of Arbitrary Boolean Functions via CNN. CNNA’06, 2006.


PDTCNN:Solving XOR problem M. Balsi, Generalized CNN: Potentials of

a CNN with Non-Uniform Weights. CNNA-92, 2002 .

E. Bilgili, I. C. Göknar and O. N. Ucan, Cellular neural network with trapezoidal activation function. Int. J. Circ. Theor. Appl., 2005


Learning parameters Initialpop=20000 Number of fathers=7 Maximum number of random

parents to be add = 3 Kpro=0.8 Increment=1 Mutation Probability=0.15


PDTCNN:First Scheme U:uij=xijxij+1

1401

150140

iP

BA

0300

010030

iP

BA

0100

010010

iP

BA

)()()( 1 kyuukykx ccccc

)(

)(

)(

),(

)(

)(

)(

kyuP

ukyP

kyuP

yug

cd

cNd

cd

cd

cNd

cd

dd

cNd

cd

dd

r

r

r


PDTCNN:Second Scheme U:uij=xijyij

2281

000000

iP

BA

b)

1151

000001

iP

BAc)

0 1 0 0 0 0

0 1 0 0

A B

P i

)()()( kyukykx cccc

)0(

)0(

)0(

),(

)(

)(

)(

cd

cNd

cd

cd

cNd

cd

dd

cNd

cd

dd

yuP

uyP

yuP

yug

r

r

r


The Game of Life (I) The Game of Life (GoL) is a totalistic cellular

automaton consisting in a two-dimensional grid cells, that may be either alive (black) or dead (white).


The Game of Life (II) The state of each cell varies according to the

following rules: Birth: a cell that is dead at time t becomes

alive at time t + 1 only if exactly 3 of its neighbors were alive at time t;

Survival: a cell that was living at time t will remain alive at t + 1 if and only if it had exactly 2 or 3 alive neighbors at time t.


The Game of Life (III) Every sufficient well-stated mathematical problem can be reduced to a question about Life;

It is possible to make a life computer (logic gates, storage etc.);

Life is universal: it can be programmed to perform any desired calculation;

Given a large enough Life space and enough time, self-reproducing animals will emerge...

The whole universe is a CA! (E.Fredkin, MIT).Intro CNN & GA Polyn. CNN XOR GoL Conclusions

The Game of Life – NOT gate


A

CNN & GoL Multilayer CNN (Chua, Roska) – 1990 Activation function (Chua, Roska) –

1990 CNN-UM (Roska,Chua) -1990 CNN Universal Cells (Dogaru, Chua) –

1999

Simplicity vs. Computational power


Polynomial CNN (I)

),()()( ddede

ede

d yugiubnyanx

0)(,1

0)(,1)(

nx

nxny

d

d

d

What’s g(ud,yd)?


Polynomial CNN (II) In the simplest case g(ud, yd) is a second

degree polynomial, whose general form is

2

0

2 ))()()()((),(i

iedei

iedei

dd yqupyug

200 )()()1()( ed

eed

e yqp

)()()()( 11ed

eed

e yqup

)1()()()( 22

2ed

eed

e qup


Polynomial CNN (III)

Thanks to some considerations we find that

)()()()()()()( 12

0ded

eed

eed

eed

ed yupyqiubnyanx

000

00

000

caA

ppp

pcp

ppp

bbb

bbb

bbb

B

000

00

000

0 0cqQ

ppp

pcp

ppp

ppp

ppp

ppp

P

111

111

111

1


Polynomial CNN (IV)

pcppccccd uypbuypbnx )()()(

iyqya cccc 2

uc and appear in the state equation

direct link with totalistic Cellular Automata

pu


GoL: Rules (I)

GoL: Rules (II)

Rule 1: a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive

Black pixel= +1

White pixel= -1pixel centr. = 1 (black)

Σ neigh. = -2 (5 w, 2 b)

next state = -1 (white)

GoL: Rules (III)

Rule 2: a cell will be alive if at most 3 of its 8 neighbors are alive

Black pixel= +1

White pixel= -1pixel centr. = 1 (black)

Σ neigh. = -2 (5 w, 2 b)

next state = 1 (black)

Design algorithm (I) First iteration: we try to perform the first rule (a

cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive)

If Y(0)=0

bc=1 bp=1 i=3

pcppccccd uypbuypbnx )()()(


Design algorithm (II) Second iteration: we try to accomplish the

second rule (a cell will be alive if at most 3 of its 8 neighbors are alive)

pcpcccd uypuypnx )1()1()(

32 cccc yqya


Design algorithm (III) Hyp: pc=0

Templates found using learning

Coming soon...


Conclusions (I) In general:

In some cases it is possible to reduce a multilayer DTCNN to a single layer PDTCNN

Thanks to the GoL we can explore the capacity of PDTCNNs for Universal Machine


Conclusions (II) About learning:

The resolution used reduces the search space

The step “Add random parent” improves the behavior to avoid local minimas

About design We give a simple algorithm to design

templates for the Polynomial CNN

Future Work Implementations of mathematical

morphology functions with PDTCNNs


Polynomial Discrete Time Cellular Neural Networks

Eduardo Gomez-Ramirez Giovanni Egidio Pazienza

[email protected]@salle.url.edu

Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez † Giovanni Egidio...

Documents

Transcript of Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez † Giovanni Egidio...