Optimisation Methods Liverpool Hope University. Optimisation.
1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.
-
Upload
hubert-baker -
Category
Documents
-
view
215 -
download
0
Transcript of 1 CHAPTER 3 Optimisation Techniques for Network Problem Solving M. Pickavet and C. Develder.
1
CHAPTER 3 Optimisation Techniques for Network Problem Solving
M. Pickavet and C. Develder
Optimisation Techniques for Network Problem Solving 2
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 3
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 4
Optimisation terminology
variable x
Objective function
f(x)
Solution SNeighbourhood N(x)
move, trajectory
Local optimumSlo
xlo x
Global optimum Sgo
xgo
Solution space X (constraints)
xmin xmax
Optimisation Techniques for Network Problem Solving 5
1. Introduction
minimise or maximise :c(x)
subject to :ai(x) = 0 i = 1, …, m1
ai(x) 0 i = m1 + 1, …, m1 + m2
ai(x) 0 i = m1 + m2 + 1, …, m1 + m2 + m3
wherex: n-dimensional vector (n = number of unknown variables, discrete or continuous)m = m1 + m2 + m3: number of constraintsc, ai: given functions (non-linear in x)
Linear Programming techniques: c and ai linear in x
Heuristic techniques: if global optimisation becomes too complex
Decision variable x
constraints
Objective function c
General optimisation problem
Optimisation Techniques for Network Problem Solving 6
in network design applications :network topology designshortest pathsingle-commodity minimum cost flowmulti-commodity minimum cost flowspare capacity assignment...
in business applications :resource managementrouting and transportationplant allocation…
Examples of optimisation problems
Optimisation Techniques for Network Problem Solving 7
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 8
General form of LP problem
2. LP techniques
minimise or maximise :z = (c0+) c1.x1 + c2.x2 + … + cn.xn (objective function)
subject to :a1,1.x1 + … a1,n.xn = b1
…am1,1.x1 + … am1,n.xn = bm1
am1+1,1.x1 + … am1+1,n.xn bm1+1
… (functional constraints: )am1+m2,1.x1 + … am1+m2,n.xn bm1+m2
am1+m2+1,1.x1 + … am1+m2+1,n.xn bm1+m2+1
…am1+m2+m3,1.x1 + … am1+m2+m3,n.xn bm1+m2+m3
For each decision variable is specified whether he isnonnegative (xi0), nonpositive (xi0) or
unrestricted in sign (xi urs). (sign restrictions: pos, neg, urs)
Optimisation Techniques for Network Problem Solving 9
Important distinction based on the discrete or continuous character of the decision variables :
Linear Programming problem (LP): all variables are real-valued
Integer (Linear) Programming problem (IP or ILP): all variables are integer-valued
Mixed Integer (Linear) Programming problem (MIP or MILP): some variables are
real-valued, other variables are integer-valued
Binary Integer (Linear) Programming problem (BIP or BILP): all variables are booleans (value = 0 or 1)
LP problems: rather ‘easy’ to solve (see 2.1 LP)
ILP, MIP, BIP problems: much harder to solve (see 2.2 ILP)
Overview of LP techniques
Optimisation Techniques for Network Problem Solving 10
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 11
LP problems
Main assumptions: Proportionality Additivity Divisibility Certainty
Optimisation Techniques for Network Problem Solving 12
Transformation steps :1. minimisation problem maximisation problem : z’ = -z2. constraint constraint : multiply the constraint by -13. xi 0 xi’ 0 : xi’ = -xi
4. xi urs xi’,xi’’ 0 : replace xi by xi’-xi’’
Standard form of a LP problem :maximise :
z = c1.x1 + c2.x2 + … + cn.xn (objective function: max)subject to :
a1,1.x1 + … a1,n.xn = b1
…am1,1.x1 + … am1,n.xn = bm1
am1+1,1.x1 + … am1+1,n.xn bm1+1 (functional constraints : …am1+m2,1.x1 + … am1+m2,n.xn bm1+m2
xi 0 (sign restrictions: pos)
Transformation from general form to standard form
Optimisation Techniques for Network Problem Solving 13
Simplex algorithmExample with only 2 decision variables :
max z = 8x1 + 5x2
s.t. x1 + x2 6
9x1 + 5x2 45
x1, x2 0
Standard form S of the problem :
z=cte
9x1+5x2=45
x1+x2=6
x1
x2
feasible solution
corner point
isoprofit linex20
x1=0
b
c
da
only the corner points must be checkedto find an optimal solution !!!
Optimisation Techniques for Network Problem Solving 14
Translated problem formulation T :
max z = 8x1 + 5x2 + 0s1 +
0s2
s.t. x1 + x2 + s1 = 6
9x1 + 5x2 + s2 = 45
x1, x2, s1, s2 0
every corner point is a bfs
only the basic feasible solutions must be checkedto find an optimal solution !!!
9x1+5x2=45
x1+x2=6
x1
x2
x20
x1=0
s2=0,
x2=0
n (=4) variables, m (=2) functional contraintsput n-m variables = 0 (=non-basic variables) solve functional (equality) contraints basic variables ( = basic solution)
if basic variables 0: basic feasible solution (bfs)
(s2=0)
(s1=0)
x1=0,
x2=0
s1=0,
x1=0
s1=0,
s2=0
b
c
da
s1 , s2 slack variables
Simplex algorithm: example
Optimisation Techniques for Network Problem Solving 15
-3- new basis: - non basic variable with largest positive coefficient in objective function (8x1) becomes basic
- basic variable which becomes first zero becomes nonbasic (s2)
-4- see -2-
problem form T’’’: corner point c, basis {x1, x2}
max z = 165/4 - 5/4s1 -37/36s2 (7)
s.t. x1 = 15/4 + 5/4s1 +1/36s2 (8)
x2 = 9/4 - 9/4s1 - 1/4s2 (9)
x1, x2, s1, s2 0
9x1+5x2=45, s2=0
x1+x2=6s1=0
x1
x2
x20
x1=0
a
problem form T’: corner point a, basis {s1, s2}
max z = 8x1 + 5x2 (1)
s.t. s1 = 6 - x1 - x2 (2)
s2 = 45 - 9x1 - 5x2 (3)
x1, x2, s1, s2 0
problem form T’’: corner point d, basis {s1,
x1}
max z = 40 + 5/9x2 -8/9s2 (4)
s.t. s1 = 1 - 4/9x2 + 1/9s2 (5)
x1 = 5 - 5/9x2 - 1/9s2 (6)
x1, x2, s1, s2 0
-1- find starting bfs (x1=x2=0, z=0)
-2- objective variable (z) & basic variables (s1,s2)
as a function of nonbasic variables (=0!)
d
c
Simplex algorithm: example
Optimisation Techniques for Network Problem Solving 16
2. LP techniques 2.1 LP
Initialisation :1. Put the LP problem into standard form S.2. Add slack variables to transform the functional inequality constraints into equality constraints ( translated form T).3. Find a starting bfs (readily apparent, Big M or Two-Phase).
Iteration :1. Transformation of the problem formulation: Express the objective and the basic variables as functions of the nonbasic variables using elementary row operations.2. Optimality test: if all nonbasic variables in the objective expression have coefficients 0, the current bfs is optimal.3. Basis change: if any nonbasic variables have coefficients > 0, choose the variable with the most positive coefficient to enter the basis. When augmenting this variable, one of the basic variables will first reach the value 0. This variable leaves the basis.We repeat the steps 1, 2 and 3 till the optimality test in step 2 succeeds.
Termination :An optimal bfs is found (optimal objective value and corresponding values of thedecision variables).
Simplex algorithm: general description
Optimisation Techniques for Network Problem Solving 17
Remarks
1. Selection criterion of entering variable does not always
lead to the highest effective augmentation of the
objective value
2. Unbounded LP problems
3. Avoid cycling by random choice of entering variable
Simplex algorithm: specific properties
Optimisation Techniques for Network Problem Solving 18
• Methods developed for special problems
• techniques exploiting a network structure
• upper bound techniques
• Dual Simplex method
• Interior Point Technique
Other techniques for solving LP problems
Optimisation Techniques for Network Problem Solving 19
Very refined granularity (one cell = 53 bytes typical arc capacity = 600 Mbit/s)
flow variable fk(a) has an almost continuous nature
linear MCMCF problem :
minimize c a f ak
k Ka A
( ) ( )
subject to f a f a b v v V k K
f a u a a A
k
a I v
k
a I v
k
k
k K
( ) '( )
( ) ( ) ( ) , ,
( ) ( ) ,0
where the decision variables fk(a) are real
Solution method: several possibilities
• usually: straightforward Simplex algorithm
• for quite large problems / stringent calculation time restrictions:
special techniques such as problem decomposition
Mathematical formulation: multi-commodity minimum cost flow problem
Application: routing in an ATM-network
Optimisation Techniques for Network Problem Solving 20
x1
x2
EXERCISE
max z = x1 + 2.x2
x1 10 x2 7x1 + x2 15 x1 ,x2 0
Simplex
Optimisation Techniques for Network Problem Solving 21
Simplex
Optimisation Techniques for Network Problem Solving 22
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 23
ILPILP problem :
max x1 + x2
s.t. -2x1 + 2x2 1
16x1 - 14x2 7
x1, x2 nonnegative integers
optimal solution: zILP*=6 in (3,3)
xLP
xu
xd
8
8
x1
x2
LP relaxation of ILP problem :
LP relaxation :max x1 + x2
s.t. -2x1 + 2x2 116x1 - 14x2 7
x1, x2 0optimal solution: zLP
*=14.5 in (7,7.5)
Rounding :zu=15 in (7,8) infeasible zd=14 in (7,7) infeasible
actual optimal solutionfor the IP problem
3
3
BUT: UPPERBOUND has been found
optimal solution not in corner point !!!
Optimisation Techniques for Network Problem Solving 24
Example with only 2 decision variables
max z = 8x1 + 5x2
s.t. x1 + x269x1 + 5x2 45
x1, x2 nonnegative integers
feasible solutions = black points
Standard form S of the problem :
Solution of LP relaxation: zLP1
*=41.25 in (x1=3.75, x2=2.25)
zIP* 41.25
Subproblem 1
z=cte
9x1+5x2=45
x1+x2=6
x1
x2
Branch-and-bound method
Optimisation Techniques for Network Problem Solving 25
Subproblem 5Subproblem 2
Subproblem 3Subproblem 4
partition feasible region (branch on x1):subproblem 2 = subproblem 1 and x1 4subproblem 3 = subproblem 1 and x1 3
LP relaxation (Simplex): subproblem 2: zLP2
*=41 in (x1=4, x2=1.8) not yet integer
9x1+5x2=45
x1+x2=6x1
partition feasible region (branch on x2):subproblem 4 = subproblem 2 and x2 2subproblem 5 = subproblem 2 and x2 1
LP relaxation (Simplex): subproblem 4: unfeasible ( fathomed)subproblem 5: zLP2
*=40.55... in (x1=4.44, x2=1) not yet integer
Branch-and-bound method: example
Optimisation Techniques for Network Problem Solving 26
Subproblem 6
x1
partition feasible region (branch on x1):subproblem 6 = subproblem 2 and x1 5subproblem 7 = subproblem 2 and x1 4
subproblem 7: zLP7*=37 in (x1=4, x2=1) INTEGER !!!
zILP* 37 (candidate solution) and zILP* 41 (see
subproblem 1)subproblem 6: zLP6*=40 in (x1=5, x2=0) INTEGER !!!
zILP* 40 (candidate solution) and zILP* 41 (see
subproblem 1)
BACK TO: subproblem 3: zLP3*=39 in (x1=3, x2=3) INTEGER !!! BUT: subproblem 6 yielded better solution !CONCLUSION: zILP*= 40 (x1=5,
x2=0)
Subproblem 7
Branch-and-bound method: example
Optimisation Techniques for Network Problem Solving 27
Subproblem 1z
x
x
165
415
49
4
1
2
t=1
Subproblem 2z
x
x
41
4
2
5
1
2
t=2
Subproblem 5z
x
x
365
940
91
1
2
t=4
x14 x13
x22 x21
x15 x14
Subproblem 3z
x
x
LB
39
3
3
40
1
2
t=7 x
Subproblem 4
Infeasiblet=3 x
Subproblem 7
Candidate solution
z
x
x
37
4
11
2
t=5 x
9x1+5x2=45
x1+x2=6x1
zILP*= 40 (x1=5, x2=0)
Subproblem 6
Candidate solution
z
x
x
40
5
01
2
t=6 x
Branch-and-bound method: example
Optimisation Techniques for Network Problem Solving 28
Initialisation
Bounding(upper & lower)
Fathoming ?Select newsubproblem
yesSubproblems
left ?
yes
Stop
no
Branchingno
Branch-and-bound algorithm: general description
Optimisation Techniques for Network Problem Solving 29
Choices to be made :
1. Which variable will be branched on ?
2. Which subproblem will be solved next ? (search strategy)
3. How will we determine upper bounds for the subproblems ?
4. How will we determine lower bounds for the ILP problem ?
Branch-and-bound algorithm: flexible features
Optimisation Techniques for Network Problem Solving 30
• Till now: only B&B on ILP problems
• Extension: B&B also suited for
•MIP problems: only branching on
fractional-valued variables if these
variables must be integer
•BIP problems: branching on a fractional-
valued variable = choosing value 0 or 1
B&B for MIP and BIP problems
Optimisation Techniques for Network Problem Solving 31
•Cutting plane algorithm
•Breaking off B&B if a
satisfactory solution is found
•...
Other techniques for solving ILP problems
Optimisation Techniques for Network Problem Solving 32
Not so refined granularity (smallest unit = 2 Mbit/s typical arc capacity = 155 Mbit/s)
the integer nature of the flow variables fk(a) can not be neglected
integer MCMCF problem :
minimize )a(f)a(cAa Kk
k
subject to
Aa,)a(u)a(f0
Kk,Vv,)v(b)a(f)a(f
Kk
k
k
)v('Ia
k
)v(Ia
k
where the decision variables fk(a) are integer (denote the number of VC-12 units)
Solution method: several possibilities
• straightforward ILP techniques (B&B, cutting plane)
• special techniques
Mathematical model: multi-commodity minimum cost flow problem
Application: routing in SDH network
Optimisation Techniques for Network Problem Solving 33
EXERCISE
z = x1 + 2.x2
x1 10 x2 7x1 + 13/14.x2 15 x1, x2 integer 0
BRANCH-AND-BOUND
Optimisation Techniques for Network Problem Solving 34
Solving network problems using (MI)LP techniques
-1- Analyse communication network problem and model as
mathematical (graph) problem
-2- Well-known graph problem with efficient graph
algorithm ?
-3- If no: linear problem or linearisable ?
-4- If yes: complexity of problem feasible ?
•LP or (M)ILP ?
•# variables, # constraints
•problem-specific characteristics ?
-5- If yes: solve with commercial (MI)LP solver, e.g.
•MS Excel for ‘easy’ problems
•dedicated solver (e.g. CPLEX) for ‘hard’ problems
-6- Check and interpret results
Optimisation Techniques for Network Problem Solving 35
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 36
Heuristic techniques
• for large scale combinatorial optimisation problems
• optimising a non-linear objective function
• using reasonable means
• finding good local optimal solutions
• trade-off: computation time - solution quality
Introduction
Optimisation Techniques for Network Problem Solving 37
Criteria for classification
• generic - problem dependent
• search - constructive
• stochastic - deterministic
• with or without quality guarantee
• ...
Classification
Optimisation Techniques for Network Problem Solving 38
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 39
adding a penalty (function P(t)) to the objective function F:• to turn hard constraints into soft constraints• to have a simpler or smoother neighbourhood structure, which makes
describing a trajectory easier• to interconnect separate feasible regions
P(t)
ob
jecti
ve f
un
cti
on
F
xmin,1 xmax,1 xmin,2 xmax,2variable x
X1 X2
Problems :- find initial solution- changes are too complicated- separated regions
Penalty in the objective function
Optimisation Techniques for Network Problem Solving 40
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 41
Example: design of a Passive Optical Network
hub
hub
hub
hub
service node
Passive Optical Network (PON) :- One service node- Hubs (grouping 100-1000 users, e.g. CATV)- Fiber along the street
Design of a PON: cost factors- Equipment- Fiber cable installation (digging etc.)dominating cost objective
Optimisation Techniques for Network Problem Solving 42
Example: Solution method ?
hub
hub
hub
hub
service node
Passive Optical Network (PON) :- One service node- Hubs (grouping 100-1000 users, e.g. CATV)- Fiber along the street
Design of a PON: cost factors- Equipment- Fiber cable installation (digging etc.)dominating cost objective
EXERCISE
Optimisation Techniques for Network Problem Solving 43
Example: Solution method
Optimisation Techniques for Network Problem Solving 44
Example: ResultGent network: 843 street corners, 1281 street segments, 44 hubs
after 0.5 s + 2.2 s post-opt
Solution quality ?• 3 % cost reduction by post-optimisation• comparison with other algorithms: within 0.7%
Optimisation Techniques for Network Problem Solving 45
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 46
Algorithm: Steepest Descent
(1) initialisation: select x1 X f* = f(x1); x* = x1;(2) n := 1; STOP := FALSE;(2) While (not STOP) do(3) Find the best x’ in the neighbourhood N(xn)(4) If f(x’) f(xn)(5) xn+1 = x’; f* = f(x’); x* = x’;(6) else(7) STOP=TRUE;(8) n := n+1;(8) od(9) exit;
deterministic, generic, search
Steepest Descent
Optimisation Techniques for Network Problem Solving 47
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 48
Tabu Search
• describing a trajectory through solution space by choosing the best (not always better) non-tabu neighbour of a solution at each step
• tabu solutions are solutions that have been encountered before excluding tabu solutions from the neighbourhood prevents the solution of getting trapped in a local optimum
• explicit tabu list: full solutions are stored permanentlyattributive (changed characteristic) tabu list: attributes of solutions are stored temporarily
• tabu tenure = time the attribute remains tabu
Principle: improvement of Steepest Descent
deterministic, generic, search
Optimisation Techniques for Network Problem Solving 49
Tabu Search: Algorithm
Algorithm: Tabu Search
(1) initialisation: select x1 X f* = f(x1); x* = x1;(2) While (stopping rule not fulfilled) do(3) Find the best x’ in the neighbourhood N(xn)(4) xn+1 = x’; (5) If f(x’) f* (6) f* = f(x’); x* = x’;(7) Update tabu list(8) Update neighbourhood N(xn+1)
(9) n := n+1;(10) od(11) exit;
update is based on tabu list
Optimisation Techniques for Network Problem Solving 50
• stochastic Tabu Search
• reactive Tabu Search
• aspiration criteria
• diversification
• intensification
Neighbourhood : search in stochastic manner
Adjust the tabu-tenureduring optimisation
Possibility to overrule a tabu status (if very attractive)
Move into new areas (e.g. new move type)
Go back to previously encountered regions which were interesting
Tabu Search: Enhanced features
Optimisation Techniques for Network Problem Solving 51
Example: Ring network design• Given:
the network topology (node locations + cable locations);the demand between each two offices (demand matrix);the cost of all equipment types.
• Asked:where to install rings;which capacity to provide on each ring.
• Constraints:all demand must be routed.
• Objectives:minimisation of the total equipment cost.
• Solution:Tabu Search:
• route traffic first;
• binary variables: placement of cycle or not;
• cost: fibre + ADMs + DXCs;
• move: adding/removing a ring (depending on feasibility of solution);
(= definition of neighbourhood)
• adding penalties to the cost in case a solution is unfeasible.
Optimisation Techniques for Network Problem Solving 52
Example: Ring network design
35600 35400
34000
36000
38000
40000
42000
44000
46000
48000
50000
52000
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
iterations
cost
unfeasible solutions
feasible solutions
Course of the solution
• link penalty = 5000• node penalty = 5000• tabu tenure to remove a ring = 3• tabu tenure to add a ring = 6
More rings to be added (many possibilities)than rings to be removed (only a few)
Starting solution: no rings in the network
large penalty
Optimisation Techniques for Network Problem Solving 53
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 54
Simulated Annealing
Simulated Annealing
Metal Annealing
Goal
Variables
Way
minimising objective functionminimising metal energy(perfect crystal structure)
variables of the objective function
coordinates of the particles
annealing the metal by adapting the temperature
judiciously
changing the variables depending on the value of
the temperature (making random moves)
stochastic, generic, search
Optimisation Techniques for Network Problem Solving 55
Algorithm: Simulated Annealing
(1) initialisation: select x1 X; f * = f(x1); x*=x1;(2) n = 1; STOP := FALSE;(3) While (not STOP) do(4) Draw x’ at random in the neighbourhood V(xn);(5) If f(x’) f(xn)(6) xn+1 = x’;
if f(xn+1)<f *f *=f(xn+1); x*=xn+1;
(7) else(8) draw a number p at random in [0,1] ; (Monte Carlo method)(9) if p pn (pn depends on T and |f(x’)-
f(xn)|)
(10) xn+1 = x’;(11) else xn+1 = xn;
(12) n := n+1;(13) evaluate(STOP);(14) od(15) exit;
Simulated Annealing: Algorithm
Optimisation Techniques for Network Problem Solving 56
• Acceptance probability of solution x’ (neighbour of solution xn):
• Stopping rule
• Cooling schedule
n
T
L
T0
T0
(1)pf x f x
e f x f xf x f x
T
1 if
if
n
n
n
( ' ) ( )
( ' ) ( )( ') ( )
n
Simulated Annealing: Parameters
Optimisation Techniques for Network Problem Solving 58
0
50
100
150
200
250
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
number of plateaus
acceptedsolutions
acceptedbettersolutions
accepted solutions:
36400
30000
35000
40000
45000
50000
55000
60000
65000
050001000015000200002500030000
temperature
cost
course of best feasible solution:
• starting temperature T0 = 30000• length of plateau L = 250• cooling factor = 0.95• penalty for nodes and links = 5000• penalty factor = 1.01
parameters:
Higher penalties at the end
Example: Ring Network design
Optimisation Techniques for Network Problem Solving 59
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 60
Genetic AlgorithmsAlgorithm derived from natural selection and species evolution
Goal: arrive with a good individual (based on the principle of survival of the fittest)
Way: working with a population of individuals
Genetic Operators: selection, crossover and mutation
Encoding of a solution into a chromosome:
allelechromosome
. . .0 1 1 1 1 01 1
Optimisation Techniques for Network Problem Solving 61
Genetic operators:
crossover: e.g. 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0 1 0 1 0 1
0 0 0 1 0 1 1 1 0 0 0 0 0 1 1 0 1 1 0 0
mutation: e.g.0 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1
selection: with pi =
f x
f x
i
jj
N M
1
4 6,
6
fitness (max)
Genetic Algorithms: Operators
Optimisation Techniques for Network Problem Solving 62
Algorithm: Genetic Algorithm
(1) initialisation: Select an initial population X(1) = {x1(1),..., xN
(1)} X
f * max {f(xi(1)), i=1,...,N}
x* arg max {f(xi(1)), i=1,...,N}
(2) for g = 1, 2, ..., G do(3) Crossover of the good parents with probability
resulting in new chromosomes (‘children’);(4) Mutation of the children and the parents with probability
resulting in new chromosomes (‘mutants’);(5) for j = 1, 2, ..., N+M do
if f(xj(g)) > f *, then f * = f(xj
(g)); x* = xj(g);
(6) od(7) Selection of solutions for the new population (‘parents’)
with a probability proportional to the fitness;(8) od
Genetic Algorithms: Algorithm
Optimisation Techniques for Network Problem Solving 63
• Given:the demand between each two offices (demand matrix);the distance between each two offices (distance matrix);the cost of all equipment types;the routing strategy of the network operator.
• Asked:where to install cables (network topology design);how much capacity to provide on each link (network capacity design).
• Constraints:the edge-connectivity should be two or more (survivability constraint).
• Objectives:maximisation of the survivability;minimisation of the total equipment cost.
• Solution:Genetic Algorithm with extra optimisation routines for each chromosome:
• adding or removing links to bring the degrees of the nodes between some limits;
• replacing crossing links by non-crossing links;
• adding links to obtain a more meshed network (especially important in the case of two connectivity requirements);
• adding links to increase some elements of the connectivity matrix of the topology;
• removing links to decrease the cost of the topology while preserving enough connectivity.
Example: capacitated survivable network design
Optimisation Techniques for Network Problem Solving 64
v1v4
v5
v2
v3
v6
v7
v8
parent network 1
v1v4
v5
v2
v6
v7
v8
parent network 2
v3
v1v4
v5
v2
v6
v7
v8
child network
v3
• Crossover: replacing ‘bad’ part of one topology by ‘good’ part of another topology
• Mutation: changing the network drastically (without lowering the quality too much)
Example: Capacitated survivable network design
Optimisation Techniques for Network Problem Solving 65
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 66
Natural behaviour of Ant Colony
Principle: initially: random walk of ants from nest to environment deposit of pheromones new ants attracted by pheromones Shorter paths to food are favoured
Nest N FFood F
FN
N
Split S
S
S
Optimisation Techniques for Network Problem Solving 67
Ant Colony Optimisation
Natural behaviour basic idea:• Model the problem as the search for a minimum
cost path in an (artificial) graph.• Artificial ants walk through this graph, looking for
good paths• Each ant has rather simple behaviour
will find rather poor-quality paths on its own• Global cooperation between the ants
better quality solutions
Cooperation via ‘stigmergy’Indirect communication by modifications of environmental states which are only locally accessible by the ants
Optimisation Techniques for Network Problem Solving 68
Ant Colony Optimisation
Improved behaviour (artificial):• An artificial ant can have an internal state, a memory
• Non-immediate update of ‘pheromone trail’• Pheromone can be a function of the quality of the solution
found
• Influence ant trajectory: • Backtracking• Lookahead
• Local optimisation (hybrid implementations)
Optimisation Techniques for Network Problem Solving 69
Ant Colony Optimisation: Algorithm
(1) while (stop criterion not satisfied) do(2) while (available resources) do(3) initialise ant(4) update ant memory M(5) while (current state ≠ target state) do(6) read local ant routing table (7) compute transition probabilities (8) decide upon next state by applying decision
policy(9) move to next state(10) update ant memory M(11) od(12) evaluate solution(13) deposit pheromones on all visited arcs(14) update ant routing tables(15) od(16) pheromone evaporation(17) daemon activities(18) od
Optimisation Techniques for Network Problem Solving 70
Example: AntNet
ACO AntNetMinimum cost in graph Minimal delay in IP network
Ants travelling through graph
Forward and Backward Ants
Environment (Probabilistic) routing tables
Pheromone Updating Updating of the probabilities in the routing tables
ACO applied to (IP) routing: AntNet
Optimisation Techniques for Network Problem Solving 71
Example: AntNet
eth1
eth2
eth3eth4
eth5
eth1
eth2
eth3
eth4
eth5
Optimisation Techniques for Network Problem Solving 72
Outline
1. Introduction: general concepts
2. Linear Programming Techniques2.1 LP2.2 ILP - MIP - BIP
3. Heuristics3.1 Introduction3.2 Penalties3.3 Example3.4 Steepest Descent3.5 Tabu Search3.6 Simulated Annealing3.7 Genetic Algorithms3.8 Ant Colony Optimisation3.9 Concluding remarks
Optimisation Techniques for Network Problem Solving 73
Concluding remarksSolving an optimisation problem using generic technique
1. Which technique? 2. Technique specific choices (e.g. definition of a move)
1. Which technique
E.g.
• Steepest descent
• Tabu Search
• Simulated Annealing
• Genetic Algorithms
• Ant Colony Optimisation
Optimisation Techniques for Network Problem Solving 74
Concluding remarks
2. Technique specific choices :
E.g.
• Tabu Search - Simulated Annealing:- defining the variables- defining the neighbourhood structure
• Genetic Algorithms:- defining the variables- defining genetic operators
(crossover, mutation, selection)
• Ant Colony Optimisation:- model as minimum cost problem in
graph- drastic or smooth routing table
updates
Optimisation Techniques for Network Problem Solving 75
Concluding remarks
• Adding intelligence to heuristics
e.g. problem-specific move
• Combining heuristics = new heuristic
# heuristics =
Optimisation Techniques for Network Problem Solving 76
[1] W. Winston: Operations Research: Applications and Algorithms (derde editie), Duxbury Press, Pacific Grove, CA, 1997.
[2] G.L.Nemhauser, A.H.G.Kan, M.J.Todd, Handbooks in Operations Research and Management Science, vol. 1, Optimization (North Holland, Amsterdam, 1989).
[3] G. Nemhauser, L. Wolsey: Integer and Combinatorial Optimization (Wiley, New York, 1988).
[4] M. Pirlot: General Local Search Heuristics in Combinatorial Optimization, Belgian Journal of Operations Research, Statistics and Computer Science, vol. 32, nr. 1-2, 1994, p. 7-61.
References