Neural Networks for Solving Quadratic Assignment Problems

31

description

Neural Networks for Solving Quadratic Assignment Problems

Transcript of Neural Networks for Solving Quadratic Assignment Problems

Page 1: Neural Networks for Solving Quadratic Assignment Problems
Page 2: Neural Networks for Solving Quadratic Assignment Problems

Agenda Hopfield Neural Network Gaussian Machine Quadratic Assignment Problems How to Solve Problem Computation Results Conclusion

Page 3: Neural Networks for Solving Quadratic Assignment Problems

Artificial Neural Network (ANN) An mathematic model Inspired by the biological nervous systems Acquires knowledge through learning ANN’s knowledge is stored within inter-neuron

connection strengths (Synaptic weights)

Page 4: Neural Networks for Solving Quadratic Assignment Problems

Use continuous activation function Fully connected recurrent network Notion of energy Each state has an energy Computes recursively until a stable state

reached Converges to stable states

Hopfield Network Model:

Hopfield Neural Network

1

( ) ( ) ( )N

ii i j i i

j

du t u t T x t Idt =

= - + +å

0

( )1( ) ( ( )) 1 tanh2

ii i

u tx t f u ta

æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø

Dynamic

Equation:

Output:

Page 5: Neural Networks for Solving Quadratic Assignment Problems

Hopfield Neural Network

Activation Function

a0 parameter determines behavior of the gain function Higher ~ gentle Lower ~ steep

0

( )1( ) ( ( )) 1 tanh2

ii i

u tx t f u ta

æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø

ui

xi =f(ui)

0

1

Page 6: Neural Networks for Solving Quadratic Assignment Problems

Hopfield Network: Energy Function Quantification of current energy of the network Energy surface determines stable states Stable states are local minima

Each update converges to stable state Symmetric connections

2

1

0N

i

i

dxdEdt dt=

æ ö= - £ç ÷è ø

å

1 1 1

1( )2

N N N

i j i j i ii j i

E x T x x I x= = =

= - -åå å

, 0i j ji iiT T T= =

Energy

function:

Lyapunov

Condition:

Page 7: Neural Networks for Solving Quadratic Assignment Problems

Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Objective: Allow the system to escape from local minima Added Gaussian noises that its power vary in time Vary the activation function gain in time

,1

( ) ( ) ( ) ( )N

ii i j j i i

j

du t u t T x t I tdt =

= - + + +å

1( ) ( ( )) 1 tanh2 ( )i ix t f u tæ öæ ö

= = +ç ÷ç ÷ç ÷è øè ø

Dynamic

Equation:

Output:

Page 8: Neural Networks for Solving Quadratic Assignment Problems

Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Added Gaussian noises that its power vary in time

( ) (0, ( ))i it N th s=

Dynamic

Equation:Temperature

0 0.5 1 1.5 2 2.5 3-1.5

-1

-0.5

0

0.5

1

1.5

t

( )i th

,1

( ) ( ) ( ) ( )N

ii i j j i i

j

du t u t T x t I tdt =

= - + + +å

Page 9: Neural Networks for Solving Quadratic Assignment Problems

Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Vary the activation function gain in time

Output:

ui

xi =f(ui)1

1( ) ( ( )) 1 tanh2 ( )i ix t f u tæ öæ ö

= = +ç ÷ç ÷ç ÷è øè ø

Page 10: Neural Networks for Solving Quadratic Assignment Problems

Quadratic Assignment Problems-QAP

12 23 13AB BC ACCost f d f d f d= + +

Assign N Facilities to N locations Minimum sum of product of

“flow between facilities” and

“distance between locations”

N! Possible Solutions

Computationally hard problem, grows exponentially

N = 12, 479001600 solutions

N = 20, 2432902008176640000 solutions

How to find a solution from this ocean ?

Quadratic Assignment Problems-QAP

Page 11: Neural Networks for Solving Quadratic Assignment Problems

Problem Representation: ~ the distance between location k and l and ~ the traffic flow between facility i and j and

A solution: Permutation Matrix (NxN Matrix)

Quadratic Assignment Problems-QAP

[ ]klD d=

[ ]ijF f=

0 ,ii ij jif f f= =

[ ]ikP x=

0 ,kk kl lkd d d= =

1 if is assigned to 0 otherwise ik

i kx ì

= íî

Page 12: Neural Networks for Solving Quadratic Assignment Problems

Objective: Assign N Facilities to N locations Minimize the total cost of assignment

The constraints

Quadratic Assignment Problems-QAP

1 1 1 1

1min2

N N N N

ij kl ik jlx i j k lC f d x x

= = = =

= åååå

1

1

1 for 1,...,

1 for 1,...,

[0,1] ,

N

ikiN

ikk

ik

x i N

x k N

x i k

=

=

= =

= =

Î "

å

å

: Only one location k is assigned in each facility i

: Only one facility i is assigned in each location k

: Output level boundary

Quadratic function QAP

Page 13: Neural Networks for Solving Quadratic Assignment Problems

QAP Example

Page 14: Neural Networks for Solving Quadratic Assignment Problems

Neural Network as QAP solver

Find a representation for the problem

Define a problem energy function

Derive T and I matrixes from the energy function

Construct the network using T and I matrixes

Page 15: Neural Networks for Solving Quadratic Assignment Problems

Representation of QAP Permutation matrix represents an assignment Rows ~ Facilities Columns ~ Locations

12 23 34 45 51DA BC CB BE EDCost f d f d f d f d f d= + + + +

[ ]ikP x= =1 if is assigned to 0 otherwise ik

i kx ì

= íî

Page 16: Neural Networks for Solving Quadratic Assignment Problems

Representation of QAP Use a neuron to represent each entry of the matrix ��If the entry is 1, neuron is on ( ≈ 1) ��If the entry is 0, neuron is off ( ≈ 0)

N-facilities problem represented using N2 neurons

ikx

ikx

Page 17: Neural Networks for Solving Quadratic Assignment Problems

Hopfield Neural Network for QAP Arrange the neuron in a matrix form Neurons addressed with double indices

,1 1 1 1 1 1

1( )2

N N N N N N

ik jl ik jl ik iki j k l i k

E x T x x I x= = = = = =

= - -åååå åå

,1 1

( ) ( ) ( )N N

ikik ik jl jl ik

j l

du t u t T x t Idt = =

= - + +åå

0

( )1( ) ( ( )) 1 tanh2

ikik ik

u tx t f u ta

æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø

Dynamic

Equation:

Output:

Energy

function:

Page 18: Neural Networks for Solving Quadratic Assignment Problems

Gaussian Machine for QAP

Same notation as Hopfield network

,1 1 1 1 1 1

1( )2

N N N N N N

ik jl ik jl ik iki j k l i k

E x T x x I x= = = = = =

= - -åååå åå

Dynamic

Equation:

Output:

Energy

function:

( )1( ) ( ( )) 1 tanh2 ( )

ikik ik

u tx t f u tæ öæ ö

= = +ç ÷ç ÷ç ÷è øè ø

,1 1

( ) ( ) ( ) ( )N N

ikik ik jl jl ik ik

j l

du t u t T x t I tdt = =

= - + + +åå

Page 19: Neural Networks for Solving Quadratic Assignment Problems

Neural Network as QAP solver

Find a representation for the problem

Define a problem energy function

Derive T and I matrixes from the energy function

Construct the network using T and I matrixes

Page 20: Neural Networks for Solving Quadratic Assignment Problems

Energy Function for QAP Its minima must correspond to the valid solutions Shorter paths and flow must have lower energy So, break it down into

penalty cost( )E x E E= +

Page 21: Neural Networks for Solving Quadratic Assignment Problems

Energy Function for QAP Constraint Satisfaction:

Cost:

2 2

penalty1 1 1 1 1 1

1 1 (1 )2 2 2

N N N N N N

ik ik ik iki k k i k i

A A CE x x x x= = = = = =

æ ö æ ö= - + - + -ç ÷ ç ÷

è ø è øå å å å åå

cost, 1 , 12

N N

ij kl ik jli j k l

BE f d x x= =

= åå

Only one “1” in each row Only one “1” in each column Output level close to “1”

Page 22: Neural Networks for Solving Quadratic Assignment Problems

Neural Network as QAP solver

Find a representation for the problem

Define a problem energy function

Derive T and I matrixes from the energy function

Construct the network using T and I matrixes

Page 23: Neural Networks for Solving Quadratic Assignment Problems

Mapping QAP onto Neural Network

Quadratic terms for T values Linear terms for I values

,, 1 , 1 , 1

1( )2

N N N

ik jl ik jl ik iki j k l i k

E x T x x I x= = =

= - -åå åQuadratic term Linear term

Page 24: Neural Networks for Solving Quadratic Assignment Problems

Mapping QAP onto Neural Network QAP Energy Function

Network Energy Function

2 2

1 1 1 1 1 1

, 1 , 1

( ) 1 1 (1 )2 2 2

2

N N N N N N

ik ik ik iki k k i k i

N N

ij kl ik jli j k l

A A CE x x x x x

B f d x x

= = = = = =

= =

æ ö æ ö= - + - + -ç ÷ ç ÷è ø è ø

+

å å å å åå

åå

,, 1 , 1 , 1

1( )2

N N N

ik jl ik jl ik iki j k l i k

E x T x x I x= = =

= - -åå å

Page 25: Neural Networks for Solving Quadratic Assignment Problems

, , 1 , , 1 , , , 1 , 1 , 1

1 1

( )2 2 2 2

2

N N N N N

ik jl ik jl ik jl ij kl ik jli k l k i j i j k l i j k l

N N

ik ik ikk i

A A C BE x x x x x x x f d x x

CA x A x x

= = = = =

= =

= + - +

- - -

å å å åå

å å

Mapping QAP onto Neural Network QAP Energy Function

Network Energy Function

,, 1 , 1 , 1

1( )2

N N N

ik jl ik jl ik iki j k l i k

E x T x x I x= = =

= - -åå å

Page 26: Neural Networks for Solving Quadratic Assignment Problems

Mapping QAP onto Neural Network

Network Energy Function

( )E x T x x I x= - -

Linear term

Page 27: Neural Networks for Solving Quadratic Assignment Problems

Mapping QAP onto Neural Network

Network Energy Function

Derived T and I matrices

( )E x T x x I x= - -

Linear term

, , , , ,

22

ik jl i j k l i j k l ij kl

ik

T A A C Bf dCI A

d d d d= - - + -

= +1 ,0 ,ik

i ji j

d=ì

= í ¹î

Kronecker Delta

Page 28: Neural Networks for Solving Quadratic Assignment Problems

Neural Network as QAP solver

Find a representation for the problem

Define a problem energy function

Derive T and I matrixes from the energy function

Construct the network using T and I matrixes

Page 29: Neural Networks for Solving Quadratic Assignment Problems

Agenda Hopfield Neural Network Gaussian Machine Quadratic Assignment Problems How to Solve Problem Computation Results Conclusion

Page 30: Neural Networks for Solving Quadratic Assignment Problems

Computation Results

Page 31: Neural Networks for Solving Quadratic Assignment Problems

Conclusion