Pure Cutting Plane Methods for ILP: a computational perspective
CUTTING PLANE METHOD - sites.ju.edu.josites.ju.edu.jo/sites/Alzalg/Documents/973/CPpres.pdf · The...
Transcript of CUTTING PLANE METHOD - sites.ju.edu.josites.ju.edu.jo/sites/Alzalg/Documents/973/CPpres.pdf · The...
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 1 / 38
The University of JordanDepartment of Mathematics
Cutting Plane Method
Supervised and edited by : Dr. Baha ALZALG
Prepared and presented by :
Nadjiba MANSOURIIlhem KADRISamia MAZOUZAhmed BOUCHENAK
Soraya ABDELAZIZMohamed BENRABIASaid ATLAOUIIbtissem ATTAFI
Integer and Combinatorial Optimization (0301973)
Spring 2018
Sommaire
1 Introduction
2 The Cutting Plane MethodGeneral SchemeAn Example (with the graphical method)
3 The Gomory Cutting Plane MethodThe Algorithm StepsExamples (with the simplex method)How to avoid cycling ?
4 Conclusion
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 2 / 38
Introduction
Classification
An integer programming problem is a mathematical optimization orfeasibility program in which some or all of the variables are restrictedto be integers which is very difficult to solve. However, there arethree different types of algorithms :
1 Exact algorithms.
2 Approximation algorithms.
3 Heuristic algorithms.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 3 / 38
Introduction
Exact Algorithms
Examples of exact algorithms designed and used to solvecombinatorial optimization problems :
1 Cutting Plane Method.
2 Branch and Bound Method.
3 Branch and Cut Method.
4 Dynamic Programming.
In this work, we present a method called theCutting Plane Method.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 4 / 38
Introduction
Relaxation
Let P be a mixed integer linear program :min cTx + dTy + eT zs.t. Ax + By + Cz ≤ b
x , y , z > 0 (P)y ∈ Z.
The linear programming relaxation is :min cTx + dTy + eT zs.t. Ax + By + Cz ≤ b (P ′)
x , y > 0.
Problem (P ′) is called linear relaxation of (P).
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 5 / 38
The Cutting Plane Method General Scheme
General Scheme of the Cutting Plane Method
The integer programming problem
min cTxs.t. Ax ≤ b
x ≥ 0 integer.
The linear programming relaxation
min cTxs.t. Ax ≤ b
x ≥ 0.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 6 / 38
The Cutting Plane Method General Scheme
General Scheme of the Cutting Plane Method
The main idea of the Cutting Plane Method is to solve the integerprogramming problem by solving a sequence of linear programmingproblems, as follows :
1 Solve the linear programming relaxation.
2 If the resulting optimal solution x? is integral, stop (the optimalsolution is found).
3 If not, generate a cut by using the Gomory cutting planealgorithm, i.e, a constraint which is satisfied by all feasibleinteger solution but not by x?.
4 Add this new constraint, resolve problem, and go back to (2).This terminates after a finite number of iterations in (2) and theresulting x? is integral and optimal.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 7 / 38
The Cutting Plane Method An Example (with the graphical method)
Example 1
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 8 / 38
The Cutting Plane Method An Example (with the graphical method)
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 9 / 38
The Cutting Plane Method An Example (with the graphical method)
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 10 / 38
The Cutting Plane Method An Example (with the graphical method)
(We will discuss later how to generate such cuts.)
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 11 / 38
The Cutting Plane Method An Example (with the graphical method)
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 12 / 38
The Cutting Plane Method An Example (with the graphical method)
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 13 / 38
The Gomory Cutting Plane Method
The Gomory Cutting Plane Algorithm
The first finitely terminating algorithm for integer programming was acutting plane algorithm proposed by Ralph Gomory in 1958 at IBM.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 14 / 38
The Gomory Cutting Plane Method The Algorithm Steps
The Gomory Cutting Plane AlgorithmStep 1
We solve the standard form linear programming problem with thesimplex method.Let x? be the optimal basic feasible solution.
We partition x into two subvectors : x =
(xBxN
), where
xB is the subvector corresponding the basic variables, andxN is the subvector corresponding the nonbasic variables.Assume for simplicity that
xB(i) = xi ,∀i = 1, . . . ,m (i.e., the first m variables are basic).
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 15 / 38
The Gomory Cutting Plane Method The Algorithm Steps
From the optimal tableau
B−1Ax = B−1b, · · · · · · · · · (∗)
where• A is the coefficient matrix of the problem in standard form.• B is the submatrix of A whose columns are ai for i in the set ofindices of basic variables.From (∗) we obtain
xB + B−1ANxN = B−1b
where N is the set of indices of nonbasic variables and AN is thesubmatrix of A whose columns are ai for i ∈ N .(Therefore, A = [B AN ]).
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 16 / 38
The Gomory Cutting Plane Method The Algorithm Steps
Step 2
We consider one equality from the optimal tableau
xi +∑j∈N
aijxj = ai0 ,
where
aij = (B−1Aj)i and ai0 = (B−1b)i and ai0 is fractional,
where [t] is floor of t. Since xj ≥ 0 ∀j , we have
xi +∑j∈N
[aij ]xj ≤ ai0, as [aij ] ≤ aij .
Since xj must be integer, we obtain
xi +∑j∈N
[aij ]xj ≤ [ai0].
This inequality is valid for all integer sols, but is not satisfied by x?.Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 17 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
Example 2 (a minimization integer program)
Consider the integer programming problem
min x1 − 2x2s.t. −4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4x1, x2 ≥ 0 and both integers.
The problem in the standard form becomes
min x1 − 2x2s.t. −4x1 + 6x2 + x3 = 9
x1 + x2 + x4 = 4x1, x2, x3, x4 ≥ 0.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 18 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
Solve the linear programming relaxation by using the simplexmethod : Note that
A =
(−4 6 1 01 1 0 1
).
We partition x into two subvectors x = (xB , xN), where
xB = (x1, x2) (basic variables),
xN = (x3, x4) (nonbasic variables).
It follows that
B =
(−4 61 1
)and AN =
(1 00 1
).
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 19 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
The system B−1Ax = B−1b can be written as
x1 − 110x3 + 3
5x4 = 15
10, · · · · · · · · · (1)
x2 + 110x3 + 2
5x4 = 25
10. · · · · · · · · · (2)
So the optimal solution is x1 =(1510, 2510
).
While the two equations have the same fractional part in theright-hand side values (which is 5/10), equation (2) has the biggestright-hand side value in this system.As a result, choosing equation (2) and applying the Gomory cuttingplane, we get
x2 +
[1
10
]x3 +
[2
5
]x4 ≤
[25
10
]. This leads to x2 ≤ 2.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 20 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
We augment the linear programming relaxation by adding theconstraint x2 + x5 = 2 to obtain
min x1 − 2x2s.t. −4x1 + 6x2 + x3 = 9
x1 + x2 + x4 = 4x2 + x5 = 2x1, x2, x3, x4, x5 ≥ 0.
Applying the simplex method, we get
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 21 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
Iteration 1
x1 x2 x3 x4 x5 c Ratiox3 -4 6 1 0 0 9 3
2
x4 1 1 0 1 0 4 4x5 0 1 0 0 1 2 2z 1 -2 0 0 0 0 0
Iteration 2
x1 x2 x3 x3 x5 c Ratiox2
−23
1 16
0 0 32
-1x4
53
0 −16
1 0 52
32
x523
0 −16
0 1 12
34
z −13
0 13
0 0 3
Iteration 3
x1 x2 x3 x4 x5 c Ratiox2 0 1 0 0 1 2x4 0 0 3
121 −5
2−52
x1 0 0 −14
0 32
34
z 0 0 12
0 1 132
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 22 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
So, the new optimal solution is x2 = (34, 2), which is not integer.
The equations in the optimal tableau are
x1 − 14x3 + 3
2x5 = 3
4, · · · · · · · · · (3)
x2 + x5 = 2. · · · · · · · · · (4)
Choosing equation (3) (because it has the biggest fractional partamong all the right-hand side values in the system (3/4 > 0)) andapplying the Gomory cutting plane method, we get
x1 +
[−1
4
]x3 +
[3
2
]x5 ≤
[3
4
]. This leads to x1 − x3 + x5 ≤ 0.
Since x2 + x5 = 2, we have x5 = 2− x2. In term of the originalvariables x1, x2, we get
−3x1 + 5x2 ≤ 7.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 23 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
We add this constraint, together with the previously added constraint.
Applying the simplex method, we get the new optimal solutionx3 = (1, 2), which is integer ! So, it is the optimal solution of theoriginal problem.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 24 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
Example 3 (a maximization integer program)
Consider the following integer programming problem
max 3x1 + 4x2s.t. 2x1 + 5x2 ≤ 15
2x1 − 2x2 ≤ 5x1, x2 ≥ 0 and both integers.
The standard form is
max 3x1 + 4x2s.t. 2x1 + 5x2 + x3 = 15
2x1 − 2x2 + x4 = 5x1, x2, x3, x4 ≥ 0.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 25 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
We solve the LP relaxation by using the simplex method. We have
A =
(2 5 1 02 −2 0 1
),B =
(2 52 −2
), and b =
(155
).
B−1Ax = B−1b ⇐⇒ x1 + 17x3 + 5
14x4 = 55
14,
x2 + 17x3 − 1
7x4 = 10
7.
The optimal solution is
x1 =(55
14,
10
7
).
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 26 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
Choosing the first equation and applying the Gomory cutting planemethod, we obtain
x1 +[1
7
]x3 +
[ 5
14
]x4 ≤
[55
14
], which leads to x1 ≤ 3.
Introducing a new slack variable to get x1 + x5 = 3, x5 ≥ 0.Adding this cut to the linear problem and solving by the simplexmethod, we find the new optimal solution x2 = (3, 9
5), which is not
integer.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 27 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
We choose an equation from the optimal tableau, which is
x2 +1
5x3 −
2
5x5 =
9
5,
and apply the Gomory cutting plane method and find the cut
x1 + x2 ≤ 4 or x2 + x1 + x6 = 4, x6 ≥ 0.
Add this cut to the problem and go back to step 1. The new optimalsolution is x3 = (5
3, 73), which is still not integer.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 28 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
From the optimal tableau, we choose the equation
x2 +1
3x3 −
2
3x6 =
7
3,
and after applying the Gomory cutting plane method, we find the cut
x1 + 2x2 ≤ 6 or x1 + 2x2 + x7 = 6, x7 ≥ 0.
We add this cut to the problem and go back step 1. Finally, find thenew optimal solution x4 = (2, 2), which is integer ! It is an optimalsolution to the original problem.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 29 / 38
The Gomory Cutting Plane Method Examples (with the simplex method)
A graph illustrates the three cuts generated in Example 3.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 30 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
Another cut to avoid the cycling
We apply the following cut to avoid the cycling :∑j∈N
fjxj ≥ f , · · · · · · · · · (∗∗)
wherefj = aij − [aij ] and f = ai0 − [ai0].
This inequality is satisfied by all feasible x but it is not satisfied by x?.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 31 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
Example 4 (on avoiding cycling)
max z = 2x1 + 3x2s.t. 2x1 + 2x2 ≤ 7
x1 ≤ 2x2 ≤ 2x1, x2 ≥ 0 and both integers.
The standard form is
max z = 2x1 + 3x2s.t. 2x1 + 2x2 + x3 = 7
x1 + x4 = 2x2 + x5 = 2x1, x2, x3, x4, x5 ≥ 0.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 32 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
After solving the relaxation by using the simplex method, the optimaltableau is
cj 2 3 0 0 0CBi basic variable x1 x2 x3 x4 x5 Sol
2 x1 1 0 12
0 -1 32
0 x4 0 0 −12
1 1 12
3 x2 0 1 0 0 1 2zj 2 3 1 0 1 9
cj − zj 0 0 -1 0 -1
The optimal solution is
x1 =
(3
2, 2
),
but it is not integer.Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 33 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
We choose the following equation from the optimal tableau
x1 +1
2x3 − x5 =
3
2,
and apply the Gomory cutting plane method to obtain the cut
x1 + x2 ≤ 3.
If we add this constraint to the LPP and use the simplex method, weget the same cut in each sequence, which will lead to cycling !
cj 2 3 0 0 0 0CBi basic variable x1 x2 x3 x4 x5 x6 Sol
2 x1 1 0 12
0 -1 0 32
0 x4 0 0 −12
1 1 0 12
3 x2 0 1 0 0 1 0 23 x6 1 1 0 0 0 1 3
zj 2 3 1 0 1 0 9cj − zj 0 0 -1 0 -1 0
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 34 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
So, we use the cut (∗∗) in our example for solve the ILP. The newcut is
1
2x3 ≥
1
2or−1
2x3 ≤
−1
2,
which is equivalent to
−1
2x3 + x6 =
−1
2, x6 ≥ 0.
Now, we use the dual simplex method to solve the resulting problem.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 35 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
Iteration 1
cj 2 3 0 0 0 0CBi basic variable x1 x2 x3 x4 x5 x6 Sol
2 x1 1 0 12
0 -1 0 32
0 x4 0 0 −12
1 1 0 12
3 x2 0 1 0 0 1 0 23 x6 0 0 −1
20 0 1 −1
2
zj 2 3 1 0 1 0 9cj − zj 0 0 -1 0 -1 0
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 36 / 38
The Gomory Cutting Plane Method How to avoid cycling ?
Iteration 2
cj 2 3 0 0 0 0CBi basic variable x1 x2 x3 x4 x5 x6 Sol
2 x1 1 0 0 0 -1 1 10 x4 0 0 0 1 1 -1 13 x2 0 1 0 0 1 0 23 x6 0 0 1 0 0 -2 2
zj 2 3 0 0 1 2 8cj − zj 0 0 0 0 -1 -2
The tableau is now optimal and the optimal solution is x = (1, 2),which is integer !
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 37 / 38
Conclusion
Conclusion
The Cutting Plane Method is very useful for solving integerprogramming problems, but there is a difficulty lies in the choice ofinequalities which represent the cut of only a very small piece of thefeasible set of the linear programming relaxation.
Integer Optimization (University of Jordan) CUTTING PLANE METHOD 29-04-2018 38 / 38