Post on 01-Apr-2015
Integer Linear Programming
Introduction
• Integer L P problem
(P)
• F(P) = feasible domain of P
• Exemple
1
1
Min
s. t. 1, ,
0, integer 1, ,
n
j jj
n
ij j ij
j
c x
a x b i m
x j n
1 2
1 2
1
1 2
Min 5
s. t. 10 20
2
, 0, integer
z x x
x x
x
x x
Introduction
• Integer L P problem
(P)
• F(P) = feasible domain of P
• Exemple
2010 21 xx
21 x
( ) 0,0 , 1,0 , 2,0 , 0,1 , 1,1 , 2,1 , 0,2F P
1
1
Min
s. t. 1, ,
0, integer 1, ,
n
j jj
n
ij j ij
j
c x
a x b i m
x j n
1 2
1 2
1
1 2
Min 5
s. t. 10 20
2
, 0, integer
z x x
x x
x
x x
Introduction
• Integer L P problem
(P)
• F(P) = feasible domain of P
• denote the relaxtion of (P) where the integrality constraints are relaxed
• Exemple
1 2
1 2
1
1 2
Min 5
s.t. 10 20
2
, 0, integer
z x x
x x
x
x x
P
2010 21 xx
21 x
P1
1
Min
s. t. 1, ,
0, integer 1, ,
n
j jj
n
ij j ij
j
c x
a x b i m
x j n
Introduction
• Integer L P problem
(P)
• F(P) = feasible domain of P
• denote the relaxtion of (P) where the integrality constraints are relaxed
• Exemple
1 2
1 2
1
1 2
Min 5
s.t. 10 20
2
, 0,entiers
z x x
x x
x
x x
P
2010 21 xx
21 x
PF
P 1
1
Min
s. t. 1, ,
0, integer 1, ,
n
j jj
n
ij j ij
j
c x
a x b i m
x j n
Introduction
• Integer L P problem
(P)
• How to solve the problem? Why dont we solve the relaxed
problem, and take the rounded solution?
• Exemple
Solution of the relaxed problem:
(2, 9/5) et z = –11
Rounded solution: (2, 1) et z = –7
1 2
1 2
1
1 2
Min 5
s.t. 10 20
2
, 0,entiers
z x x
x x
x
x x
2010 21 xx
21 x
PF
But (0, 2) is feasible with z = –10
1
1
Min
s. t. 1, ,
0, integer 1, ,
n
j jj
n
ij j ij
j
c x
a x b i m
x j n
Solution methods
• Foundamental principle
Generate a set of linear constraints
to be added to (P)
• Exemple
entiers,0,
2
2010 àSujet
5Min
21
1
21
21
xx
x
xx
xxz
2010 21 xx
21 x
PF
42 21 xx
Méthodes de résolution
• Foundamental principle
Generate a set of linear constraints
to be added to (P) in order to
generate a new problem (PR) such that
Furthermore, when solving the problem , the variables take integer values, and hence it is optimal for (P).
• Exemple
2010 21 xx
21 x
PF
42 21 xx
1 2
1 2
1
1 2
1 2
Min 5
s.t. 10 20
2
2 4
, 0, integer
z x x
x x
x
x x
x x
F PR F P
F PR F P
PR
Cutting plane algorithm
• General statement of the cutting plane algorithm
Introduce new linear constraints to the problem in order to reduce the feasible domain of the relaxed problem but keep all the feasible points of the problem.
• In this procedure, we solve a sequence of relaxed problems until an integer optimal solution is reached
• Any problem of the sequence is obtained by adding an additional linear constraint (a cut) to the current problem.
n
j 1
1
Consider the following integer linear programming problem:
( ) Min
s. t. 1, ,
0, integer, 1, ,
j j
n
ij j ij
j
P c x
a x b i m
x j n
How to generate a Gomory cut.
Let be an optimal basis of ( ), and let be the basic
variable in the row of the optimal tableau that is not integer.k
th
B P x
i
1
1
1 2
11 12 1 1 1
1 2
1 2
basic r.h.s
var.
0 1 0 0
1 0 0 0
0
m
m
k j j j n
j j n
k i i ij in i
j m m
x x x x x x x z
x t t t t b
x t t t t b
x t t
1 2
0 1 0
0 0 0 1
mj mn m
j n
t t b
z c c c c z
The optimal tableau is as follows:
The cooresponding row of the tableau is as follows:
1
where : is an index of a non basic variable
and is not integer.
ik ij jj J
i
x t x b
J j j
b
and consequently. (2)ik ij j k ij j
j J j J
x t x x t x b
The cooresponding row of the tableau is as follows:
1
where : is an index of a non basic variable
and is not integer.
ik ij jj J
i
x t x b
J j j
b
Let the largest integer (floor value) .d d
Since 0 , thenj
ij j ij jj J j J
x jt x t x
If we account for the fact that the variables must take
integer values, it follows from (2) that
. (3)
Hence all solutions of ( ) satisfy (3).
j
ik ij jj J
x
x t x b
P
Now consider the relation obtained by substracting
(1) from (3):
4
Note that
0 et 0
i
i
iij ij jj J
iij ij
t t x b b
t t b b
Since any solution of ( ) satisfies (1) and (3), then it also satifies (4),
and we can introduce this constraint in ( ) without eliminating any solution of ( ).
P
P P
The cooresponding row of the tableau is as follows:
1
where : is an index of a non basic variable
and is not integer.
ik ij jj J
i
x t x b
J j j
b
Let the largest integer (floor value) .d d
Since 0 , thenj
ij j ij jj J j J
x jt x t x
and consequently. (2)ik ij j k ij j
j J j J
x t x x t x b
But the current optimal solution of the relaxed problem ( ) where
0 does not satisfies constraint (4), and thus introducing (4)
reduces the size of the feasible domain of the relaxed problem ( ).
j
P
x j J
P
Now consider the relation obtained by substracting
(1) from (3):
4
Note that
0 et 0
i
i
iij ij jj J
iij ij
t t x b b
t t b b
Since any solution of ( ) satisfies (1) and (3), then it also satifies (4),
and we can introduce this constraint in ( ) without eliminating any solution of ( ).
P
P P
To continue the solution process, we introduce the constraint
where is a slack variable having a cost equal to 0, in the last simplex
tableau
i ii iij ij j ij ij jj J j J
t t x b b t t x x b b
x
to generate a basic solution for the new problem by looking at
as the basic variable in the new row of the tableau.x
This basic solution is not feasible since 0.
Then we continue the solution process using the dual simplex method.
i ix b b
1
1
1 2
11 12 1 1 1
1 2
1
basic r.h.s.
var.
0 1 0 0 0
1 0 0 0 0
m
m
k j j j n
j j n
k i i ij in i
j m
x x x x x x x x z
x t t t t b
x t t t t b
x t t
2
1 1 2 2
1 2
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
i
m mj mn m
ii i i i ij ij in in
j n
t t b
x t t t t t t t t b b
z c c c c z
To continue the solution process, we introduce the constraint
where is a slack variable having a cost equal to 0, in the last simplex
tableau
i ii iij ij j ij ij jj J j J
t t x b b t t x x b b
x
to generate a basic solution for the new problem by looking at
as the basic variable in the new row of the tableau.x
This basic solution is not feasible since 0.
Then we continue the solution process using the dual simplex method.
i ix b b
Remarks:
1) If (i.e., is integer) , et si is not integer, then
1
indicates that ( ) is not feasible since the left hand side take an integer
value for all f
iij ij ij
ik ij jj J
t t t j J b
x t x b
P
easible solution of ( ) while the right hand element
is not integer.
2) The same process is applied at each iteration.
iP b
1 2
1 2 3
1 2 3
Consider the following exemple
Min 21 11
s.t. 7 4 13
, , 0, integer
x x
x x x
x x x
1 2 3
2 3 4
Iteration 1:Basic solution of ( )
4 1 13
7 7 7optimal value 39
New constraint:4 4 1 1 13 13
7 7 7 7 7 7
P
x x x
x x x
2 3 2 3
3 1 2
2 1 2 1
Graphic interpretation:
4 1 64 6.
7 7 7Mais 13 7 4 . Then
4 13 7 4 6 1
x x x x
x x x
x x x x
i iij ij j i ij ij j ij J j J
t t x b b t t x x b b
1 2
1 2 3
1 2 3
Min 21 11s.t. 7 4 13
, , 0, integer
x xx x x
x x x
1 2
1 2 3
2 3 4
1 2 3 4
Iteration 2:
Solve the relaxed problem
Min 21 11
s.t. 7 4 13
4 1 6
7 7 7 , , , 0, integer
x x
x x x
x x x
x x x x
1 4
2 3 4
The result:
1
1 7 3
4 4 21
optimal value 372
x x
x x x
nd2 3 4
3 4 5
3 4 5
1 7 3New constraint specified from the 2 row :
4 4 21 1 7 7 3 3
4 4 4 4 2 21 1 1
4 4 2
x x x
x x x
x x x
3 4
4
2 3 4
3 2 3
3
3 1 2
1 2 2
Graphic interpretation2
Replace by its value obtainedfrom the last constraint added
4 1 6,
7 7 7to obtain
4 1 62.
7 7 7replace
13 7 4to obtain
8 32 4 613 8 2
7 7 7 7
8
x xx
x x x
x x x
xx x x
x x x
1 2 1 28 6
4 2 13 2 37 7
x x x x
iij ij j ij J
iij ij j ij J
t t x b b
t t x x b b
1 2
1 2 3
2 3 4
1 2 3 4
Min 21 11Sujet à 7 4 13
4 1 6
7 7 7, , , 0, entiers
x xx x x
x x x
x x x x
1 3 5
1 2 5
1 4
We obtain
1
2 3
1
x x x
x x x
x x
1 2
1 4
2 3 4
3 4 5
1 2 3 4 5
Iteration 3:
Solve the problem
Min 21 11
s.t. 1
1 7 3
4 4 21 1 1
4 4 2, , , , 0, integer
x x
x x
x x x
x x x
x x x x x
1 2 3
Then the optimal solution is integer:
0, 3, 1
optimal value 33
x x x
Reference:
A. Schrijver, Theory of Linear and Integer Programming,
Wiley & Sons, 1986, 354 - 357
Convergence of Gomory cutting plane algorithm
Under some assumptions for selecting the row of the
tableau to specify the next cut, this author shows that:
"... the cutting plane method terminates"
Branch & Bound method
• Here also we solve a sequence of relaxed problems.
• First we solve . If the optimal solution is integer, then this solution is optimal for the original problem (P).
• Otherwise we use a variable which is not integer.
• Consider two new constraints
P x
having value i ix x
(floor of )
and
(ceeling of )
ii i
ii i
x x x
x x x
1
1
Min
( ) s.t. 1, ,
0, integer 1, ,
n
j jj
n
ij j ij
j
c x
P a x b i m
x j n
1
1
Min
( ) s.t. 1, ,
0, 1, ,
n
j jj
n
ij j ij
j
c x
P a x b i m
x j n
66611 21 xx
4510 21 xx1 2
1 2
1 2
1 2
Min 5
s.t. 11 6 66
10 45
, 0, integer
z x x
x x
x x
x x
1 2
opt. sol. of the relaxed problem:
3.75, 4.125
24.375
x x
z
1
1
New constraints to consider:
3.75 3
and
3.75 4
x
x
With these two new constraints - feasible point of are maintained - a slice of the feasible domain of the relaxed problem is eliminated.
P
66611 21 xx
4510 21 xx1 2
1 2
1 2
1 2
Min 5
s.t. 11 6 66
10 45
, 0, integer
z x x
x x
x x
x x
1 2
Opt. sol. of the relaxed problem:
3.75, 4.125
24.375
x x
z
1
1
New constraints to consider:
3.75 3
and
3.75 4
x
x
Slice of
eliminated
F P
66611 21 xx
4510 21 xx1 2
1 2
1 2
1 2
Min 5
s.t. 11 6 66
10 45
, 0, integer
z x x
x x
x x
x x
2PF 3PF
2
3
But what is left of nis not connected since two subsets
and are left, and their intersection is empty.
F P F P
F P
1 2
Opt. sol. of the relaxed problem:
3.75, 4.125
24.375
x x
z
1
1
New constraints to consider:
3.75 3
and
3.75 4
x
x
Slice of
eliminated
F P
66611 21 xx
4510 21 xx
2PF 3PF
2 1 2
1 2
1 2
1
1 2
Min 5
s.t. 11 6 66
10 45
4
, 0, integer
P z x x
x x
x x
x
x x
3 1 2
1 2
1 2
1
1 2
Min 5
s.t. 11 6 66
10 45
3
, 0, integer
P z x x
x x
x x
x
x x
22
33
Continue the process by defining the problem associated with
and associated with .
P F P
P F P
2 3The best optimal solution of and is the optimal solution of .
P PP
1 2
1 2
1 2
1 2
Min 5
s.t. 11 6 66
10 45
, 0, integer
z x x
x x
x x
x x
1
1
New constraints to consider:
3.75 3
and
3.75 4
x
x
• Next iteration
Select one of the two problems (P2) or (P3 )
Solve this selected problem as we did for (P).
• In our exemple, we select the problem (P3)
2
2
New constraints to considered:
4.2 4
and
4.2 5
x
x
66611 21 xx
4510 21 xx
2PF 3PF
3 1 2
1 2
1 2
1
1 2
Min 5
s.t. 11 6 66
10 45
3
, 0, integer
P z x x
x x
x x
x
x x
3
1 2
Opt. sol. of the relaxed prob. :
3, 4.2
24
P
x x
z
3Slice of
eliminated
F P
3 1 2
1 2
1 2
1
1 2
Min 5
s.t. 11 6 66
10 45
3
, 0, integer
P z x x
x x
x x
x
x x
66611 21 xx
4510 21 xx
2PF 5F P
5 1 2
1 2
1 2
1
2
1 2
Min 5
s.t. 11 6 66
10 45
3
4
, 0, integer
P z x x
x x
x x
x
x
x x
4 1 2
1 2
1 2
1
2
1 2
Min 5 s.t. 11 6 66
10 4535
, 0, integer
P z x xx x
x xxx
x x
2
2
New constraints to considered:
4.2 4
and
4.2 5
x
x
• Next iteration
Select a problem in the set which has not been solved yet.
Solve this selected problem as we did for (P).
• In our exemple, we select the problem (P5)
5 4 2, ,P P P
66611 21 xx
4510 21 xx
2PF 5PF
5 1 2
1 2
1 2
1
2
1 2
Min 5
s.t. 11 6 66
10 45
3
4
, 0, integer
P z x x
x x
x x
x
x
x x
5
1 2
Opt. sol. of the relaxed problem :
3, 4
23
P
x x
z
Since the optimal solution of the relaxed sub problem is integer, then itis a feasible solution of (P).Do not generate new sub problems since we have identified the best feasible solution in this part of the feasible domain of (P). During this solution process, we keep the best feasible integer solution found so far,And its value becomes an upper bound BS for the optimal value in (P).
• Next iteration
Select a problem in the set which has not been solved yet.
Solve this selected problem as we did for (P).
• In our exemple, we select the problem (P4)
4 2,P P
66611 21 xx
4510 21 xx
2PF
4 1 2
1 2
1 2
1
2
1 2
Min 5
s.t. 11 6 66
10 45
3
5
, 0, integer
P z x x
x x
x x
x
x
x x
4
4
Problem is not feasible
P
F P
Stop looking for feasible solutions in the part since it is empty.
• Next iteration
Select a problem in the set which has not been solved yet.
Solve this selected problem as we did for (P).
• In our exemple, we select the problem (P2)
2P
2 1 2
1 2
1 2
1
1 2
Min 5
s.t. 11 6 66
10 45
4
, 0, integer
P z x x
x x
x x
x
x x
66611 21 xx
4510 21 xx
2PF 2
1 2
Opt. sol. of the relaxed problem :
4, 3.667
22.333
P
x x
z
The optimal solution of the relaxed problem a is not integer but its value z = –22.333 > BS = – 23Stop looking for an integer solution in the part of the feasible domain of (P) since it is not possible to find one having a value smaller than the BS = – 23.
• The procedure stops when all the relaxed problems have been solved.
• The integer solution having its value equal to BS in an optimal solution of (P).
Summary of the Branch & Bound approach
• Iterative approach.• At each iteration, - a list of candidate problems is available to be solved. When the procedure starts, the list includes aoly the problem (P). - a candidate problem is selected, and the correponding relaxed problem is solved - the optimal solution of the relaxed allows to update the list of candidate problems or the upper bound and the best integer solution found so far.
e
The Branch & Bound method
• Initialisation The list of candidate problem includes only the problem (P) BS = ∞ Go to Step 2.• Étape 1 Si la liste est vide, terminer. La solution optimale est celle associée à BS, a moins que BS = ∞ dans lequel cas le problème P n’as pas de solution.• Step 2 Select the first candidate problem (PC) on the top of the list.
If the list is empty, stop. The optimal solution is the one
associated with , unless = ,and in this case the problem
has
n
Ste
o s
p 1
olution.
BS BS
Select the first candidate problem (PC) on the top of t
he
Step
l
2
ist.
To analyse , solve the relaxe problem .
If , go to Step1 .
If , go to Step 1.
If the optimal solution
Ste
of is integer,
p 3
then
PC PC
F PC
v PC BS
PC
if , then : ,v PC BS BS v PC
go to Step1.
To analyse , solve the relaxe problem .
If , go to Step1 .
If , go to Step 1.
If the optimal solution
Ste
of is integer,
p 3
then
PC PC
F PC
v PC BS
PC
if , then : ,v PC BS BS v PC go to Step1.
Select a variable which is no
St
t in
ep
te
er.
4
gjx
Generate a first new problem by including the constraint
to the problem and place it on the top of the listj jx x PC
Generate a second new problem by including the constraint
to the problem , and place it on the top of the listj jx x PC Go to Step 2.
If the list is empty, stop. The optimal solution is the one
associated with , unless = ,and in this case the problem
has
n
Ste
o s
p 1
olution.
BS BS
Select the first candidate problem (PC) on the top of t
he
Step
l
2
ist.
To analyse , solve the relaxe problem .
If , go to Step1 .
If , go to Step 1.
If the optimal solution
Ste
of is integer,
p 3
then
PC PC
F PC
v PC BS
PC
if , then : ,v PC BS BS v PC
go to Step1.
Enumeration three of the Branch & Bound
P
2P 3P
4P 5P
41 x 31 x
42 x52 x
Int. sol.BS = – 23
Infeasible solution
Non int. sol.z = – 22.333 > BS
Optimal solution
2.42 x
75.31 x1 2
1 2
1 2
1 2
Min 5 s.t. 11 6 66
10 45, 0,
z x xx x
x xx x
3 1 2
1 2
1 2
1
1 2
Min 5 s.t. 11 6 66
10 453
, 0,
P z x xx x
x xx
x x
5 1 2
1 2
1 2
1
2
1 2
Min 5 s.t. 11 6 66
10 4534
, 0,
P z x xx x
x xxx
x x
4 1 2
1 2
1 2
1
2
1 2
Min 5 s.t. 11 6 66
10 4535
, 0,
P z x xx x
x xxx
x x
2 1 2
1 2
1 2
1
1 2
Min 5 s.t. 11 6 66
10 454
, 0,
P z x xx x
x xx
x x
Candidate problem
P
2P
3P 4P
5P
How to search in the three (how to select the next candidate problem)
:
decend rapidly in the three in order t
a) Depth first search (select the last genera
o reach as soon as
possible a feasible i
ted candidate problem)
nteger solution
require to complete some iterations to solve the candidate
b) Search using the best node (
problem at each node in order t
us
o
ing t
ident
he best ca
ify the no
ndidate pr
de having
oblem):
the best potentiel to improve the upper bound
Selection of the separating variable ,j j j j jx x x x x The variable such that
a) is the largest
j
j j
x
x x b) is the smallestj jx x
c) is closer to 0.5j jx x
The optimal solution of the current relaxation of the candidate pro
Solving the relaxed candidate problem using the dual simplex method
blem
(at the current node) is not feasible for the relaxed problem obtained
by adding a constraint of the type ou .
Then we can use the dual simplex method to solve the
relaxed candidate problem using the optimal solution
of the current problem to g
j j j jx x x x
enerate the initial basic solution.
Reference.
A. Atamturk, M.W.P. Savelsbergh, "Integer-Programming Software
Systems", Annals of Operations Research 140, 67-124, 2005.