1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

75
1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder

Transcript of 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Page 1: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

1

CHAPTER 3 Optimisation Techniques for Network Problem Solving

M. Pickavet and C. Develder

Page 2: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 2

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 3: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 3

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 4: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 4

Optimisation terminology

variable x

Objective function

f(x)

Solution SNeighbourhood N(x)

move, trajectory

Local optimumSlo

xlo x

Global optimum Sgo

xgo

Solution space X (constraints)

xmin xmax

Page 5: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 5

1. Introduction

minimise or maximise :c(x)

subject to :ai(x) = 0 i = 1, …, m1

ai(x) 0 i = m1 + 1, …, m1 + m2

ai(x) 0 i = m1 + m2 + 1, …, m1 + m2 + m3

wherex: n-dimensional vector (n = number of unknown variables, discrete or continuous)m = m1 + m2 + m3: number of constraintsc, ai: given functions (non-linear in x)

Linear Programming techniques: c and ai linear in x

Heuristic techniques: if global optimisation becomes too complex

Decision variable x

constraints

Objective function c

General optimisation problem

Page 6: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 6

in network design applications :network topology designshortest pathsingle-commodity minimum cost flowmulti-commodity minimum cost flowspare capacity assignment...

in business applications :resource managementrouting and transportationplant allocation…

Examples of optimisation problems

Page 7: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 7

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 8: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 8

General form of LP problem

2. LP techniques

minimise or maximise :z = (c0+) c1.x1 + c2.x2 + … + cn.xn (objective function)

subject to :a1,1.x1 + … a1,n.xn = b1

…am1,1.x1 + … am1,n.xn = bm1

am1+1,1.x1 + … am1+1,n.xn bm1+1

… (functional constraints: )am1+m2,1.x1 + … am1+m2,n.xn bm1+m2

am1+m2+1,1.x1 + … am1+m2+1,n.xn bm1+m2+1

…am1+m2+m3,1.x1 + … am1+m2+m3,n.xn bm1+m2+m3

For each decision variable is specified whether he isnonnegative (xi0), nonpositive (xi0) or

unrestricted in sign (xi urs). (sign restrictions: pos, neg, urs)

Page 9: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 9

Important distinction based on the discrete or continuous character of the decision variables :

Linear Programming problem (LP): all variables are real-valued

Integer (Linear) Programming problem (IP or ILP): all variables are integer-valued

Mixed Integer (Linear) Programming problem (MIP or MILP): some variables are

real-valued, other variables are integer-valued

Binary Integer (Linear) Programming problem (BIP or BILP): all variables are booleans (value = 0 or 1)

LP problems: rather ‘easy’ to solve (see 2.1 LP)

ILP, MIP, BIP problems: much harder to solve (see 2.2 ILP)

Overview of LP techniques

Page 10: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 10

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 11: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 11

LP problems

Main assumptions: Proportionality Additivity Divisibility Certainty

Page 12: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 12

Transformation steps :1. minimisation problem maximisation problem : z’ = -z2. constraint constraint : multiply the constraint by -13. xi 0 xi’ 0 : xi’ = -xi

4. xi urs xi’,xi’’ 0 : replace xi by xi’-xi’’

Standard form of a LP problem :maximise :

z = c1.x1 + c2.x2 + … + cn.xn (objective function: max)subject to :

a1,1.x1 + … a1,n.xn = b1

…am1,1.x1 + … am1,n.xn = bm1

am1+1,1.x1 + … am1+1,n.xn bm1+1 (functional constraints : …am1+m2,1.x1 + … am1+m2,n.xn bm1+m2

xi 0 (sign restrictions: pos)

Transformation from general form to standard form

Page 13: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 13

Simplex algorithmExample with only 2 decision variables :

max z = 8x1 + 5x2

s.t. x1 + x2 6

9x1 + 5x2 45

x1, x2 0

Standard form S of the problem :

z=cte

9x1+5x2=45

x1+x2=6

x1

x2

feasible solution

corner point

isoprofit linex20

x1=0

b

c

da

only the corner points must be checkedto find an optimal solution !!!

Page 14: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 14

Translated problem formulation T :

max z = 8x1 + 5x2 + 0s1 +

0s2

s.t. x1 + x2 + s1 = 6

9x1 + 5x2 + s2 = 45

x1, x2, s1, s2 0

every corner point is a bfs

only the basic feasible solutions must be checkedto find an optimal solution !!!

9x1+5x2=45

x1+x2=6

x1

x2

x20

x1=0

s2=0,

x2=0

n (=4) variables, m (=2) functional contraintsput n-m variables = 0 (=non-basic variables) solve functional (equality) contraints basic variables ( = basic solution)

if basic variables 0: basic feasible solution (bfs)

(s2=0)

(s1=0)

x1=0,

x2=0

s1=0,

x1=0

s1=0,

s2=0

b

c

da

s1 , s2 slack variables

Simplex algorithm: example

Page 15: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 15

-3- new basis: - non basic variable with largest positive coefficient in objective function (8x1) becomes basic

- basic variable which becomes first zero becomes nonbasic (s2)

-4- see -2-

problem form T’’’: corner point c, basis {x1, x2}

max z = 165/4 - 5/4s1 -37/36s2 (7)

s.t. x1 = 15/4 + 5/4s1 +1/36s2 (8)

x2 = 9/4 - 9/4s1 - 1/4s2 (9)

x1, x2, s1, s2 0

9x1+5x2=45, s2=0

x1+x2=6s1=0

x1

x2

x20

x1=0

a

problem form T’: corner point a, basis {s1, s2}

max z = 8x1 + 5x2 (1)

s.t. s1 = 6 - x1 - x2 (2)

s2 = 45 - 9x1 - 5x2 (3)

x1, x2, s1, s2 0

problem form T’’: corner point d, basis {s1,

x1}

max z = 40 + 5/9x2 -8/9s2 (4)

s.t. s1 = 1 - 4/9x2 + 1/9s2 (5)

x1 = 5 - 5/9x2 - 1/9s2 (6)

x1, x2, s1, s2 0

-1- find starting bfs (x1=x2=0, z=0)

-2- objective variable (z) & basic variables (s1,s2)

as a function of nonbasic variables (=0!)

d

c

Simplex algorithm: example

Page 16: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 16

2. LP techniques 2.1 LP

Initialisation :1. Put the LP problem into standard form S.2. Add slack variables to transform the functional inequality constraints into equality constraints ( translated form T).3. Find a starting bfs (readily apparent, Big M or Two-Phase).

Iteration :1. Transformation of the problem formulation: Express the objective and the basic variables as functions of the nonbasic variables using elementary row operations.2. Optimality test: if all nonbasic variables in the objective expression have coefficients 0, the current bfs is optimal.3. Basis change: if any nonbasic variables have coefficients > 0, choose the variable with the most positive coefficient to enter the basis. When augmenting this variable, one of the basic variables will first reach the value 0. This variable leaves the basis.We repeat the steps 1, 2 and 3 till the optimality test in step 2 succeeds.

Termination :An optimal bfs is found (optimal objective value and corresponding values of thedecision variables).

Simplex algorithm: general description

Page 17: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 17

Remarks

1. Selection criterion of entering variable does not always

lead to the highest effective augmentation of the

objective value

2. Unbounded LP problems

3. Avoid cycling by random choice of entering variable

Simplex algorithm: specific properties

Page 18: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 18

• Methods developed for special problems

• techniques exploiting a network structure

• upper bound techniques

• Dual Simplex method

• Interior Point Technique

Other techniques for solving LP problems

Page 19: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 19

Very refined granularity (one cell = 53 bytes typical arc capacity = 600 Mbit/s)

flow variable fk(a) has an almost continuous nature

linear MCMCF problem :

minimize c a f ak

k Ka A

( ) ( )

subject to f a f a b v v V k K

f a u a a A

k

a I v

k

a I v

k

k

k K

( ) '( )

( ) ( ) ( ) , ,

( ) ( ) ,0

where the decision variables fk(a) are real

Solution method: several possibilities

• usually: straightforward Simplex algorithm

• for quite large problems / stringent calculation time restrictions:

special techniques such as problem decomposition

Mathematical formulation: multi-commodity minimum cost flow problem

Application: routing in an ATM-network

Page 20: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 20

x1

x2

EXERCISE

max z = x1 + 2.x2

x1 10 x2 7x1 + x2 15 x1 ,x2 0

Simplex

Page 21: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 21

Simplex

Page 22: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 22

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 23: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 23

ILPILP problem :

max x1 + x2

s.t. -2x1 + 2x2 1

16x1 - 14x2 7

x1, x2 nonnegative integers

optimal solution: zILP*=6 in (3,3)

xLP

xu

xd

8

8

x1

x2

LP relaxation of ILP problem :

LP relaxation :max x1 + x2

s.t. -2x1 + 2x2 116x1 - 14x2 7

x1, x2 0optimal solution: zLP

*=14.5 in (7,7.5)

Rounding :zu=15 in (7,8) infeasible zd=14 in (7,7) infeasible

actual optimal solutionfor the IP problem

3

3

BUT: UPPERBOUND has been found

optimal solution not in corner point !!!

Page 24: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 24

Example with only 2 decision variables

max z = 8x1 + 5x2

s.t. x1 + x269x1 + 5x2 45

x1, x2 nonnegative integers

feasible solutions = black points

Standard form S of the problem :

Solution of LP relaxation: zLP1

*=41.25 in (x1=3.75, x2=2.25)

zIP* 41.25

Subproblem 1

z=cte

9x1+5x2=45

x1+x2=6

x1

x2

Branch-and-bound method

Page 25: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 25

Subproblem 5Subproblem 2

Subproblem 3Subproblem 4

partition feasible region (branch on x1):subproblem 2 = subproblem 1 and x1 4subproblem 3 = subproblem 1 and x1 3

LP relaxation (Simplex): subproblem 2: zLP2

*=41 in (x1=4, x2=1.8) not yet integer

9x1+5x2=45

x1+x2=6x1

partition feasible region (branch on x2):subproblem 4 = subproblem 2 and x2 2subproblem 5 = subproblem 2 and x2 1

LP relaxation (Simplex): subproblem 4: unfeasible ( fathomed)subproblem 5: zLP2

*=40.55... in (x1=4.44, x2=1) not yet integer

Branch-and-bound method: example

Page 26: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 26

Subproblem 6

x1

partition feasible region (branch on x1):subproblem 6 = subproblem 2 and x1 5subproblem 7 = subproblem 2 and x1 4

subproblem 7: zLP7*=37 in (x1=4, x2=1) INTEGER !!!

zILP* 37 (candidate solution) and zILP* 41 (see

subproblem 1)subproblem 6: zLP6*=40 in (x1=5, x2=0) INTEGER !!!

zILP* 40 (candidate solution) and zILP* 41 (see

subproblem 1)

BACK TO: subproblem 3: zLP3*=39 in (x1=3, x2=3) INTEGER !!! BUT: subproblem 6 yielded better solution !CONCLUSION: zILP*= 40 (x1=5,

x2=0)

Subproblem 7

Branch-and-bound method: example

Page 27: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 27

Subproblem 1z

x

x

165

415

49

4

1

2

t=1

Subproblem 2z

x

x

41

4

2

5

1

2

t=2

Subproblem 5z

x

x

365

940

91

1

2

t=4

x14 x13

x22 x21

x15 x14

Subproblem 3z

x

x

LB

39

3

3

40

1

2

t=7 x

Subproblem 4

Infeasiblet=3 x

Subproblem 7

Candidate solution

z

x

x

37

4

11

2

t=5 x

9x1+5x2=45

x1+x2=6x1

zILP*= 40 (x1=5, x2=0)

Subproblem 6

Candidate solution

z

x

x

40

5

01

2

t=6 x

Branch-and-bound method: example

Page 28: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 28

Initialisation

Bounding(upper & lower)

Fathoming ?Select newsubproblem

yesSubproblems

left ?

yes

Stop

no

Branchingno

Branch-and-bound algorithm: general description

Page 29: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 29

Choices to be made :

1. Which variable will be branched on ?

2. Which subproblem will be solved next ? (search strategy)

3. How will we determine upper bounds for the subproblems ?

4. How will we determine lower bounds for the ILP problem ?

Branch-and-bound algorithm: flexible features

Page 30: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 30

• Till now: only B&B on ILP problems

• Extension: B&B also suited for

•MIP problems: only branching on

fractional-valued variables if these

variables must be integer

•BIP problems: branching on a fractional-

valued variable = choosing value 0 or 1

B&B for MIP and BIP problems

Page 31: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 31

•Cutting plane algorithm

•Breaking off B&B if a

satisfactory solution is found

•...

Other techniques for solving ILP problems

Page 32: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 32

Not so refined granularity (smallest unit = 2 Mbit/s typical arc capacity = 155 Mbit/s)

the integer nature of the flow variables fk(a) can not be neglected

integer MCMCF problem :

minimize )a(f)a(cAa Kk

k

subject to

Aa,)a(u)a(f0

Kk,Vv,)v(b)a(f)a(f

Kk

k

k

)v('Ia

k

)v(Ia

k

where the decision variables fk(a) are integer (denote the number of VC-12 units)

Solution method: several possibilities

• straightforward ILP techniques (B&B, cutting plane)

• special techniques

Mathematical model: multi-commodity minimum cost flow problem

Application: routing in SDH network

Page 33: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 33

EXERCISE

z = x1 + 2.x2

x1 10 x2 7x1 + 13/14.x2 15 x1, x2 integer 0

BRANCH-AND-BOUND

Page 34: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 34

Solving network problems using (MI)LP techniques

-1- Analyse communication network problem and model as

mathematical (graph) problem

-2- Well-known graph problem with efficient graph

algorithm ?

-3- If no: linear problem or linearisable ?

-4- If yes: complexity of problem feasible ?

•LP or (M)ILP ?

•# variables, # constraints

•problem-specific characteristics ?

-5- If yes: solve with commercial (MI)LP solver, e.g.

•MS Excel for ‘easy’ problems

•dedicated solver (e.g. CPLEX) for ‘hard’ problems

-6- Check and interpret results

Page 35: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 35

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 36: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 36

Heuristic techniques

• for large scale combinatorial optimisation problems

• optimising a non-linear objective function

• using reasonable means

• finding good local optimal solutions

• trade-off: computation time - solution quality

Introduction

Page 37: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 37

Criteria for classification

• generic - problem dependent

• search - constructive

• stochastic - deterministic

• with or without quality guarantee

• ...

Classification

Page 38: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 38

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 39: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 39

adding a penalty (function P(t)) to the objective function F:• to turn hard constraints into soft constraints• to have a simpler or smoother neighbourhood structure, which makes

describing a trajectory easier• to interconnect separate feasible regions

P(t)

ob

jecti

ve f

un

cti

on

F

xmin,1 xmax,1 xmin,2 xmax,2variable x

X1 X2

Problems :- find initial solution- changes are too complicated- separated regions

Penalty in the objective function

Page 40: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 40

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 41: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 41

Example: design of a Passive Optical Network

hub

hub

hub

hub

service node

Passive Optical Network (PON) :- One service node- Hubs (grouping 100-1000 users, e.g. CATV)- Fiber along the street

Design of a PON: cost factors- Equipment- Fiber cable installation (digging etc.)dominating cost objective

Page 42: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 42

Example: Solution method ?

hub

hub

hub

hub

service node

Passive Optical Network (PON) :- One service node- Hubs (grouping 100-1000 users, e.g. CATV)- Fiber along the street

Design of a PON: cost factors- Equipment- Fiber cable installation (digging etc.)dominating cost objective

EXERCISE

Page 43: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 43

Example: Solution method

Page 44: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 44

Example: ResultGent network: 843 street corners, 1281 street segments, 44 hubs

after 0.5 s + 2.2 s post-opt

Solution quality ?• 3 % cost reduction by post-optimisation• comparison with other algorithms: within 0.7%

Page 45: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 45

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 46: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 46

Algorithm: Steepest Descent

(1) initialisation: select x1 X f* = f(x1); x* = x1;(2) n := 1; STOP := FALSE;(2) While (not STOP) do(3) Find the best x’ in the neighbourhood N(xn)(4) If f(x’) f(xn)(5) xn+1 = x’; f* = f(x’); x* = x’;(6) else(7) STOP=TRUE;(8) n := n+1;(8) od(9) exit;

deterministic, generic, search

Steepest Descent

Page 47: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 47

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 48: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 48

Tabu Search

• describing a trajectory through solution space by choosing the best (not always better) non-tabu neighbour of a solution at each step

• tabu solutions are solutions that have been encountered before excluding tabu solutions from the neighbourhood prevents the solution of getting trapped in a local optimum

• explicit tabu list: full solutions are stored permanentlyattributive (changed characteristic) tabu list: attributes of solutions are stored temporarily

• tabu tenure = time the attribute remains tabu

Principle: improvement of Steepest Descent

deterministic, generic, search

Page 49: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 49

Tabu Search: Algorithm

Algorithm: Tabu Search

(1) initialisation: select x1 X f* = f(x1); x* = x1;(2) While (stopping rule not fulfilled) do(3) Find the best x’ in the neighbourhood N(xn)(4) xn+1 = x’; (5) If f(x’) f* (6) f* = f(x’); x* = x’;(7) Update tabu list(8) Update neighbourhood N(xn+1)

(9) n := n+1;(10) od(11) exit;

update is based on tabu list

Page 50: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 50

• stochastic Tabu Search

• reactive Tabu Search

• aspiration criteria

• diversification

• intensification

Neighbourhood : search in stochastic manner

Adjust the tabu-tenureduring optimisation

Possibility to overrule a tabu status (if very attractive)

Move into new areas (e.g. new move type)

Go back to previously encountered regions which were interesting

Tabu Search: Enhanced features

Page 51: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 51

Example: Ring network design• Given:

the network topology (node locations + cable locations);the demand between each two offices (demand matrix);the cost of all equipment types.

• Asked:where to install rings;which capacity to provide on each ring.

• Constraints:all demand must be routed.

• Objectives:minimisation of the total equipment cost.

• Solution:Tabu Search:

• route traffic first;

• binary variables: placement of cycle or not;

• cost: fibre + ADMs + DXCs;

• move: adding/removing a ring (depending on feasibility of solution);

(= definition of neighbourhood)

• adding penalties to the cost in case a solution is unfeasible.

Page 52: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 52

Example: Ring network design

35600 35400

34000

36000

38000

40000

42000

44000

46000

48000

50000

52000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

iterations

cost

unfeasible solutions

feasible solutions

Course of the solution

• link penalty = 5000• node penalty = 5000• tabu tenure to remove a ring = 3• tabu tenure to add a ring = 6

More rings to be added (many possibilities)than rings to be removed (only a few)

Starting solution: no rings in the network

large penalty

Page 53: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 53

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 54: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 54

Simulated Annealing

Simulated Annealing

Metal Annealing

Goal

Variables

Way

minimising objective functionminimising metal energy(perfect crystal structure)

variables of the objective function

coordinates of the particles

annealing the metal by adapting the temperature

judiciously

changing the variables depending on the value of

the temperature (making random moves)

stochastic, generic, search

Page 55: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 55

Algorithm: Simulated Annealing

(1) initialisation: select x1 X; f * = f(x1); x*=x1;(2) n = 1; STOP := FALSE;(3) While (not STOP) do(4) Draw x’ at random in the neighbourhood V(xn);(5) If f(x’) f(xn)(6) xn+1 = x’;

if f(xn+1)<f *f *=f(xn+1); x*=xn+1;

(7) else(8) draw a number p at random in [0,1] ; (Monte Carlo method)(9) if p pn (pn depends on T and |f(x’)-

f(xn)|)

(10) xn+1 = x’;(11) else xn+1 = xn;

(12) n := n+1;(13) evaluate(STOP);(14) od(15) exit;

Simulated Annealing: Algorithm

Page 56: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 56

• Acceptance probability of solution x’ (neighbour of solution xn):

• Stopping rule

• Cooling schedule

n

T

L

T0

T0

(1)pf x f x

e f x f xf x f x

T

1 if

if

n

n

n

( ' ) ( )

( ' ) ( )( ') ( )

n

Simulated Annealing: Parameters

Page 57: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 58

0

50

100

150

200

250

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

number of plateaus

acceptedsolutions

acceptedbettersolutions

accepted solutions:

36400

30000

35000

40000

45000

50000

55000

60000

65000

050001000015000200002500030000

temperature

cost

course of best feasible solution:

• starting temperature T0 = 30000• length of plateau L = 250• cooling factor = 0.95• penalty for nodes and links = 5000• penalty factor = 1.01

parameters:

Higher penalties at the end

Example: Ring Network design

Page 58: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 59

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 59: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 60

Genetic AlgorithmsAlgorithm derived from natural selection and species evolution

Goal: arrive with a good individual (based on the principle of survival of the fittest)

Way: working with a population of individuals

Genetic Operators: selection, crossover and mutation

Encoding of a solution into a chromosome:

allelechromosome

. . .0 1 1 1 1 01 1

Page 60: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 61

Genetic operators:

crossover: e.g. 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0 1 0 1 0 1

0 0 0 1 0 1 1 1 0 0 0 0 0 1 1 0 1 1 0 0

mutation: e.g.0 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1

selection: with pi =

f x

f x

i

jj

N M

1

4 6,

6

fitness (max)

Genetic Algorithms: Operators

Page 61: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 62

Algorithm: Genetic Algorithm

(1) initialisation: Select an initial population X(1) = {x1(1),..., xN

(1)} X

f * max {f(xi(1)), i=1,...,N}

x* arg max {f(xi(1)), i=1,...,N}

(2) for g = 1, 2, ..., G do(3) Crossover of the good parents with probability

resulting in new chromosomes (‘children’);(4) Mutation of the children and the parents with probability

resulting in new chromosomes (‘mutants’);(5) for j = 1, 2, ..., N+M do

if f(xj(g)) > f *, then f * = f(xj

(g)); x* = xj(g);

(6) od(7) Selection of solutions for the new population (‘parents’)

with a probability proportional to the fitness;(8) od

Genetic Algorithms: Algorithm

Page 62: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 63

• Given:the demand between each two offices (demand matrix);the distance between each two offices (distance matrix);the cost of all equipment types;the routing strategy of the network operator.

• Asked:where to install cables (network topology design);how much capacity to provide on each link (network capacity design).

• Constraints:the edge-connectivity should be two or more (survivability constraint).

• Objectives:maximisation of the survivability;minimisation of the total equipment cost.

• Solution:Genetic Algorithm with extra optimisation routines for each chromosome:

• adding or removing links to bring the degrees of the nodes between some limits;

• replacing crossing links by non-crossing links;

• adding links to obtain a more meshed network (especially important in the case of two connectivity requirements);

• adding links to increase some elements of the connectivity matrix of the topology;

• removing links to decrease the cost of the topology while preserving enough connectivity.

Example: capacitated survivable network design

Page 63: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 64

v1v4

v5

v2

v3

v6

v7

v8

parent network 1

v1v4

v5

v2

v6

v7

v8

parent network 2

v3

v1v4

v5

v2

v6

v7

v8

child network

v3

• Crossover: replacing ‘bad’ part of one topology by ‘good’ part of another topology

• Mutation: changing the network drastically (without lowering the quality too much)

Example: Capacitated survivable network design

Page 64: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 65

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 65: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 66

Natural behaviour of Ant Colony

Principle: initially: random walk of ants from nest to environment deposit of pheromones new ants attracted by pheromones Shorter paths to food are favoured

Nest N FFood F

FN

N

Split S

S

S

Page 66: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 67

Ant Colony Optimisation

Natural behaviour basic idea:• Model the problem as the search for a minimum

cost path in an (artificial) graph.• Artificial ants walk through this graph, looking for

good paths• Each ant has rather simple behaviour

will find rather poor-quality paths on its own• Global cooperation between the ants

better quality solutions

Cooperation via ‘stigmergy’Indirect communication by modifications of environmental states which are only locally accessible by the ants

Page 67: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 68

Ant Colony Optimisation

Improved behaviour (artificial):• An artificial ant can have an internal state, a memory

• Non-immediate update of ‘pheromone trail’• Pheromone can be a function of the quality of the solution

found

• Influence ant trajectory: • Backtracking• Lookahead

• Local optimisation (hybrid implementations)

Page 68: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 69

Ant Colony Optimisation: Algorithm

(1) while (stop criterion not satisfied) do(2) while (available resources) do(3) initialise ant(4) update ant memory M(5) while (current state ≠ target state) do(6) read local ant routing table (7) compute transition probabilities (8) decide upon next state by applying decision

policy(9) move to next state(10) update ant memory M(11) od(12) evaluate solution(13) deposit pheromones on all visited arcs(14) update ant routing tables(15) od(16) pheromone evaporation(17) daemon activities(18) od

Page 69: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 70

Example: AntNet

ACO AntNetMinimum cost in graph Minimal delay in IP network

Ants travelling through graph

Forward and Backward Ants

Environment (Probabilistic) routing tables

Pheromone Updating Updating of the probabilities in the routing tables

ACO applied to (IP) routing: AntNet

Page 70: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 71

Example: AntNet

eth1

eth2

eth3eth4

eth5

eth1

eth2

eth3

eth4

eth5

Page 71: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 72

Outline

1. Introduction: general concepts

2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP

3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks

Page 72: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 73

Concluding remarksSolving an optimisation problem using generic technique

1. Which technique? 2. Technique specific choices (e.g. definition of a move)

1. Which technique

E.g.

• Steepest descent

• Tabu Search

• Simulated Annealing

• Genetic Algorithms

• Ant Colony Optimisation

Page 73: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 74

Concluding remarks

2. Technique specific choices :

E.g.

• Tabu Search - Simulated Annealing:- defining the variables- defining the neighbourhood structure

• Genetic Algorithms:- defining the variables- defining genetic operators

(crossover, mutation, selection)

• Ant Colony Optimisation:- model as minimum cost problem in

graph- drastic or smooth routing table

updates

Page 74: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 75

Concluding remarks

• Adding intelligence to heuristics

e.g. problem-specific move

• Combining heuristics = new heuristic

# heuristics =

Page 75: 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.

Optimisation Techniques for Network Problem Solving 76

[1] W. Winston: Operations Research: Applications and Algorithms (derde editie), Duxbury Press, Pacific Grove, CA, 1997.

[2] G.L.Nemhauser, A.H.G.Kan, M.J.Todd, Handbooks in Operations Research and Management Science, vol. 1, Optimization (North Holland, Amsterdam, 1989).

[3] G. Nemhauser, L. Wolsey: Integer and Combinatorial Optimization (Wiley, New York, 1988).

[4] M. Pirlot: General Local Search Heuristics in Combinatorial Optimization, Belgian Journal of Operations Research, Statistics and Computer Science, vol. 32, nr. 1-2, 1994, p. 7-61.

References