LP Relaxations of Mixed Integer Programs - Eaton.math.rpi.edu
Transcript of LP Relaxations of Mixed Integer Programs - Eaton.math.rpi.edu
LP Relaxations of Mixed Integer Programs
John E. Mitchell
Department of Mathematical SciencesRPI, Troy, NY 12180 USA
February 2015
Mitchell LP Relaxations 1 / 29
LP Relaxations
LP relaxations
We want to solve an integer program by solving a sequence of linearprograms. We can relax the integrality constraints to give an LPrelaxation. Let
S := {x ∈ Zn : Ax ≥ b, x ≥ 0}= Zn ∩ P
where P := {x ∈ Rn : Ax ≥ b, x ≥ 0}.
Since we restrict x ≥ 0, the polyhedron P has extreme points if it isnonempty.
Mitchell LP Relaxations 2 / 29
LP Relaxations
Convex hull of S
TheoremThe convex hull of S is a polyhedron.
The proof is in Nemhauser and Wolsey.
It is important to note that the extreme rays of conv(S) are the extremerays of P.
Mitchell LP Relaxations 3 / 29
LP Relaxations
Equivalent LP formulation
Our integer program of interest is
min{cT x : x ∈ S} (IP).
We relate this to the linear program
min{cT x : x ∈ conv(S)} (CIP).
We don’t know the constraints of the linear program (CIP) explicitly.
Mitchell LP Relaxations 4 / 29
LP Relaxations
(IP) and (CIP)
r r r r r rr r r r r
r r r
r r
S
������ cT x = z∗
Mitchell LP Relaxations 5 / 29
LP Relaxations
(IP) and (CIP)
r r r r r rr r r r r
r r r
r r
S
������ @
@@
������HH
HHH
Hconv(S)
������ cT x = z∗
Mitchell LP Relaxations 5 / 29
LP Relaxations
Value of (IP) and (CIP)
TheoremFor any c ∈ Rn, we have:
1 The objective value of (IP) is unbounded below if and only if theobjective value of (CIP) is unbounded below.
2 If (CIP) has a bounded optimal value then it has an optimalsolution that is an optimal solution to (IP). This optimal solution isan extreme point of conv(S).
3 If x0 is an optimal solution to (IP) then it is an optimal solutionto (CIP).
The proof is in the text. It relies on the fact that the minimizer of aconcave function over a convex set is at an extreme point.
Mitchell LP Relaxations 6 / 29
LP Relaxations
Value of (IP) and (CIP)
TheoremFor any c ∈ Rn, we have:
1 The objective value of (IP) is unbounded below if and only if theobjective value of (CIP) is unbounded below.
2 If (CIP) has a bounded optimal value then it has an optimalsolution that is an optimal solution to (IP). This optimal solution isan extreme point of conv(S).
3 If x0 is an optimal solution to (IP) then it is an optimal solutionto (CIP).
The proof is in the text. It relies on the fact that the minimizer of aconcave function over a convex set is at an extreme point.
Mitchell LP Relaxations 6 / 29
LP Relaxations
Value of (IP) and (CIP)
TheoremFor any c ∈ Rn, we have:
1 The objective value of (IP) is unbounded below if and only if theobjective value of (CIP) is unbounded below.
2 If (CIP) has a bounded optimal value then it has an optimalsolution that is an optimal solution to (IP). This optimal solution isan extreme point of conv(S).
3 If x0 is an optimal solution to (IP) then it is an optimal solutionto (CIP).
The proof is in the text. It relies on the fact that the minimizer of aconcave function over a convex set is at an extreme point.
Mitchell LP Relaxations 6 / 29
LP Relaxations
Value of (IP) and (CIP)
TheoremFor any c ∈ Rn, we have:
1 The objective value of (IP) is unbounded below if and only if theobjective value of (CIP) is unbounded below.
2 If (CIP) has a bounded optimal value then it has an optimalsolution that is an optimal solution to (IP). This optimal solution isan extreme point of conv(S).
3 If x0 is an optimal solution to (IP) then it is an optimal solutionto (CIP).
The proof is in the text. It relies on the fact that the minimizer of aconcave function over a convex set is at an extreme point.
Mitchell LP Relaxations 6 / 29
LP Relaxations
Possible cases for an integer linear program
Corollary(IP) is either unbounded or infeasible or has an optimal solution.
Mitchell LP Relaxations 7 / 29
LP Relaxations
Set up LP relaxation
So in principle, could solve (IP) by solving the linear program (CIP).But we don’t know a polyhedral description of conv(S).
Look at the LP relaxation
min {cT x : Ax ≥ b, x ≥ 0} = min {cT x : x ∈ P}. (LP)
Mitchell LP Relaxations 8 / 29
LP Relaxations
Compare (IP) to LP relaxation
We can relate (LP) to (IP) in the following theorem. The proof exploitsthe fact that S ⊆ P.
TheoremLet zLP and z IP be the optimal values of (LP) and (IP) respectively.
1 If P = ∅ then S = ∅.2 If zLP is finite then either S = ∅ or z IP is finite. If S 6= ∅ then
z IP ≥ zLP .3 If xLP is optimal for (LP) and if xLP ∈ Zn then xLP is optimal
for (IP).
Mitchell LP Relaxations 9 / 29
LP Relaxations
Compare (IP) to LP relaxation
We can relate (LP) to (IP) in the following theorem. The proof exploitsthe fact that S ⊆ P.
TheoremLet zLP and z IP be the optimal values of (LP) and (IP) respectively.
1 If P = ∅ then S = ∅.2 If zLP is finite then either S = ∅ or z IP is finite. If S 6= ∅ then
z IP ≥ zLP .3 If xLP is optimal for (LP) and if xLP ∈ Zn then xLP is optimal
for (IP).
Mitchell LP Relaxations 9 / 29
LP Relaxations
Compare (IP) to LP relaxation
We can relate (LP) to (IP) in the following theorem. The proof exploitsthe fact that S ⊆ P.
TheoremLet zLP and z IP be the optimal values of (LP) and (IP) respectively.
1 If P = ∅ then S = ∅.2 If zLP is finite then either S = ∅ or z IP is finite. If S 6= ∅ then
z IP ≥ zLP .3 If xLP is optimal for (LP) and if xLP ∈ Zn then xLP is optimal
for (IP).
Mitchell LP Relaxations 9 / 29
LP Relaxations
Compare (IP) to LP relaxation
We can relate (LP) to (IP) in the following theorem. The proof exploitsthe fact that S ⊆ P.
TheoremLet zLP and z IP be the optimal values of (LP) and (IP) respectively.
1 If P = ∅ then S = ∅.2 If zLP is finite then either S = ∅ or z IP is finite. If S 6= ∅ then
z IP ≥ zLP .3 If xLP is optimal for (LP) and if xLP ∈ Zn then xLP is optimal
for (IP).
Mitchell LP Relaxations 9 / 29
LP Relaxations
Cutting plane algorithm illustration
HHHHHH
HH
�������
PPPPP
�����
���
PPP
�����
Solve linear program relaxation
{x ∈ Rn+ : Ax ≥ b}
r r r r r rr r r r r
r r r
r r������ @
@@
������HH
HHH
Hconv(S) tSoln to LP relaxation
Mitchell LP Relaxations 10 / 29
LP Relaxations
Cutting plane algorithm illustration
HHHHHH
HH
�������
PPPPP
�����
���
PPP
�����
Solve linear program relaxation
{x ∈ Rn+ : Ax ≥ b}
r r r r r rr r r r r
r r r
r r������ @
@@
������HH
HHH
Hconv(S) tSoln to LP relaxation
��
��
���
Cutting planeaT
1 x = b1
Mitchell LP Relaxations 10 / 29
LP Relaxations
Cutting plane algorithm illustration
HHHHHH
HH
�������
PPPPP
�����
���
PPP
�����
Solve linear program relaxation
{x ∈ Rn+ : Ax ≥ b, aT
1 x ≥ b1}
r r r r r rr r r r r
r r r
r r������ @
@@
������HH
HHH
Hconv(S)
���
���
tSoln to LP relaxation
Mitchell LP Relaxations 10 / 29
LP Relaxations
Cutting plane algorithm illustration
HHHHHH
HH
�������
PPPPP
�����
���
PPP
�����
Solve linear program relaxation
{x ∈ Rn+ : Ax ≥ b, aT
1 x ≥ b1}
r r r r r rr r r r r
r r r
r r������ @
@@
������HH
HHH
Hconv(S)
���
���
tSoln to LP relaxation
Cutting planeaT
2 x = b2
Mitchell LP Relaxations 10 / 29
LP Relaxations
Cutting plane algorithm illustration
HHHHHH
HH
�������
PPPPP
�����
���
PPP
�����
Solve linear program relaxation
{y ∈ Rn+ : Ax ≥ b, aT
1 x ≥ b1, aT2 x ≥ b2}
r r r r r rr r r r r
r r r
r r������ @
@@
������HH
HHH
Hconv(S) �
��
d Solution to LP relaxation
Mitchell LP Relaxations 10 / 29
LP Relaxations
Cutting plane algorithm illustration
HHHHHH
HH
�������
PPPPP
�����
���
PPP
�����
Solve linear program relaxation
{x ∈ Rn+ : Ax ≥ b, aT
1 x ≥ b1, aT2 x ≥ b2}
r r r r r rr r r r r
r r r
r r������ @
@@
������HH
HHH
Hconv(S) �
��
d Solution to LP relaxation
Solution is integral,so optimal to IP
Mitchell LP Relaxations 10 / 29
LP Relaxations
A cutting plane approach
An integer program has many LP relaxations.
The aim of a cutting plane approach is to add linear constraints to theoriginal LP relaxation in order to tighten up the LP relaxation.
These linear constraints are satisfied by every point in S but they mightnot be satisfied by every point in P.
Adding these constraints gives an improved LP relaxation of theinteger program.
We would like to improve the relaxation so that it resembles conv(S) insome sense, at least in the neighborhood of an optimal solution to (IP).
Mitchell LP Relaxations 11 / 29
Describing Polyhedra by Facets
Polyhedral description of conv(S)
We would like to get a polyhedral description of conv(S).
We’d like as compact a description as possible.
We first set up some notation.
Given a matrix A ∈ Rm×n and a vector b ∈ Rm, we construct thepolyhedron
Q := {x ∈ Rn : Ax ≥ b}.
Note: We have changed notation from Section 1. In particular, anynonnegativity constraints are included in the system Ax ≥ b.
Mitchell LP Relaxations 12 / 29
Describing Polyhedra by Facets
Example 0
Full-dimensional example
Q :=
x ∈ R2 :
x1 + x2 ≤ 6x2 ≤ 6
x1 − x2 ≤ 22x1 + x2 ≤ 10
x1 ≥ 0x2 ≥ 0
Mitchell LP Relaxations 13 / 29
Describing Polyhedra by Facets
Example 1
First example with implicit equality in R2
Q :=
x ∈ R2 :
x1 + x2 ≤ 3x1 + x2 ≥ 3
3x1 + x2 ≥ 3x1 ≥ 0
x2 ≥ 0
Mitchell LP Relaxations 14 / 29
Describing Polyhedra by Facets
Example 2
Second example with implicit equality in R2
Q :=
x ∈ R2 :
2x1 + x2 ≤ 4x1 + x2 ≥ 4x1 ≥ 0
x2 ≥ 0
Mitchell LP Relaxations 15 / 29
Describing Polyhedra by Facets
Implicit equality constraints
We construct three sets of indices:
M := {1, . . . ,m}
M= := {i ∈ M : ATi x = bi ∀x ∈ Q}
M≥ := M \M=
= {i ∈ M : ATi x > bi for some x ∈ Q}.
We also use the notation A=, A≥, b= and b≥ in an analogous way.
Mitchell LP Relaxations 16 / 29
Describing Polyhedra by Facets
Example 0
Full-dimensional example
Q :=
x ∈ R2 :
x1 + x2 ≤ 6 M≥
x2 ≤ 6 M≥
x1 − x2 ≤ 2 M≥
2x1 + x2 ≤ 10 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 17 / 29
Describing Polyhedra by Facets
Example 1
First example with implicit equality in R2
Q :=
x ∈ R2 :
x1 + x2 ≤ 3 M=
x1 + x2 ≥ 3 M=
3x1 + x2 ≥ 3 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 18 / 29
Describing Polyhedra by Facets
Example 2
Second example with implicit equality in R2
Q :=
x ∈ R2 :
2x1 + x2 ≤ 4 M=
x1 + x2 ≥ 4 M=
x1 ≥ 0 M=
x2 ≥ 0 M≥
Mitchell LP Relaxations 19 / 29
Describing Polyhedra by Facets
Valid inequalities
DefinitionThe inequality πT x ≥ π0 is a valid inequality for Q if it is satisfied by allpoints in Q. We also represent the inequality using the pair (π, π0).
DefinitionLet πT x ≥ π0 be a valid inequality for Q. Let H = {x ∈ Rn : πT x = π0}so F := Q ∩ H is a face of Q. Then (π, π0) represents F . If F 6= ∅ then(π, π0) supports Q.
We extend the notation M=, M≥ to the face Fusing the notation M=
F , M≥F .
Mitchell LP Relaxations 20 / 29
Describing Polyhedra by Facets
Valid inequalities
DefinitionThe inequality πT x ≥ π0 is a valid inequality for Q if it is satisfied by allpoints in Q. We also represent the inequality using the pair (π, π0).
DefinitionLet πT x ≥ π0 be a valid inequality for Q. Let H = {x ∈ Rn : πT x = π0}so F := Q ∩ H is a face of Q. Then (π, π0) represents F . If F 6= ∅ then(π, π0) supports Q.
We extend the notation M=, M≥ to the face Fusing the notation M=
F , M≥F .
Mitchell LP Relaxations 20 / 29
Describing Polyhedra by Facets
Impicit equalities for faces
PropositionIf F is a nonempty face of Q then F is a polyhedron and
F = {x ∈ Rn : aTi x = bi ∀i ∈ M=
F , aTi x ≥ bi ∀i ∈ M≥
F },
where M=F ⊇ M= and M≥
F ⊆ M≥.
Mitchell LP Relaxations 21 / 29
Describing Polyhedra by Facets
Facets
Proposition
If F is a facet of Q then there exists some inequality aTk x ≥ bk with
k ∈ M≥ that represents F .
Proof.Now dim(F ) = dim(P)-1, so rank(A=
F ,b=F ) = rank(A=,b=) + 1.
Therefore, we only need one inequality in M≥ that is in M=F , and this
inequality represents the facet.
Mitchell LP Relaxations 22 / 29
Describing Polyhedra by Facets
Facets
Proposition
If F is a facet of Q then there exists some inequality aTk x ≥ bk with
k ∈ M≥ that represents F .
Proof.Now dim(F ) = dim(P)-1, so rank(A=
F ,b=F ) = rank(A=,b=) + 1.
Therefore, we only need one inequality in M≥ that is in M=F , and this
inequality represents the facet.
Mitchell LP Relaxations 22 / 29
Describing Polyhedra by Facets
One inequality per facet
PropositionFor each facet F of Q, one of the inequalities representing F isnecessary in the description of Q.
Proposition
Every inequality aTi x ≥ bi for i ∈ M≥ that represents a face of Q of
dimension less than dim(Q)− 1 is irrelevant to the description of Q.
Thus, later we will emphasize finding facets of the convex hull of S.
Mitchell LP Relaxations 23 / 29
Describing Polyhedra by Facets
One inequality per facet
PropositionFor each facet F of Q, one of the inequalities representing F isnecessary in the description of Q.
Proposition
Every inequality aTi x ≥ bi for i ∈ M≥ that represents a face of Q of
dimension less than dim(Q)− 1 is irrelevant to the description of Q.
Thus, later we will emphasize finding facets of the convex hull of S.
Mitchell LP Relaxations 23 / 29
Describing Polyhedra by Facets
Minimal representation of polyhedron
With these propositions, we can now state two theorems showing howa polyhedron can be minimally represented using linear equalities andinequalities.
Theorem (full dimensional case)A full dimensional polyhedron Q has a unique (to within scalarmultiplication) minimal representation by a finite set of linearinequalities. In particular, for each facet Fi of Q with i = 1, . . . , t , thereis an inequality aT
i x ≥ bi (unique to within scalar multiplication)representing Fi and
Q = {x ∈ Rn : aTi x ≥ bi , i = 1, . . . , t}.
Mitchell LP Relaxations 24 / 29
Describing Polyhedra by Facets
Example 0
Full-dimensional example
Q :=
x ∈ R2 :
x1 + x2 ≤ 6 M≥
x2 ≤ 6 M≥
x1 − x2 ≤ 2 M≥
2x1 + x2 ≤ 10 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 25 / 29
Describing Polyhedra by Facets
Example 0
Full-dimensional example
Q :=
x ∈ R2 :
x1 + x2 ≤ 6 M≥
x2 ≤ 6 M≥
x1 − x2 ≤ 2 M≥
2x1 + x2 ≤ 10 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 25 / 29
Describing Polyhedra by Facets
Minimal representation of polyhedron
Theorem (not full dimensional)If dim(Q) = n − k with k > 0 then let t be the number of facets Fi of Q,and we have
Q = {x ∈ Rn : aTi x = bi for i = 1, . . . , k ,
aTi x ≥ bi for i = k + 1, . . . , k + t}.
The set {(ai ,bi) : i = 1, . . . , k} is a maximal set of linearlyindependent rows of (A=,b=).For i = k + 1, . . . , k + t , (ai ,bi) is any inequality from the equivalenceclass of inequalities representing the facet Fi of Q.
Mitchell LP Relaxations 26 / 29
Describing Polyhedra by Facets
Example 1
First example with implicit equality in R2
Q :=
x ∈ R2 :
x1 + x2 ≤ 3 M=
x1 + x2 ≥ 3 M=
3x1 + x2 ≥ 3 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 27 / 29
Describing Polyhedra by Facets
Example 1
First example with implicit equality in R2
Q :=
x ∈ R2 :
x1 + x2 = 3 M=
x1 + x2 ≥ 3 M=
3x1 + x2 ≥ 3 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 27 / 29
Describing Polyhedra by Facets
Example 1
First example with implicit equality in R2
Q :=
x ∈ R2 :
x1 + x2 ≤ 3 M=
x1 + x2 = 3 M=
3x1 + x2 ≥ 3 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 27 / 29
Describing Polyhedra by Facets
Example 1
First example with implicit equality in R2
Q :=
x ∈ R2 :
x1 + x2 ≤ 3 M=
x1 + x2 = 3 M=
3x1 + x2 ≥ 3 M≥
x1 ≥ 0 M≥
x2 ≥ 0 M≥
Mitchell LP Relaxations 27 / 29
Describing Polyhedra by Facets
Example 2
Second example with implicit equality in R2
Q :=
x ∈ R2 :
2x1 + x2 ≤ 4 M=
x1 + x2 ≥ 4 M=
x1 ≥ 0 M=
x2 ≥ 0 M≥
Mitchell LP Relaxations 28 / 29
Describing Polyhedra by Facets
Example 2
Second example with implicit equality in R2
Q :=
x ∈ R2 :
2x1 + x2 ≤ 4 M=
x1 + x2 = 4 M=
x1 = 0 M=
x2 ≥ 0 M≥
Mitchell LP Relaxations 28 / 29
Describing Polyhedra by Facets
Example 2
Second example with implicit equality in R2
Q :=
x ∈ R2 :
2x1 + x2 = 4 M=
x1 + x2 ≥ 4 M=
x1 = 0 M=
x2 ≥ 0 M≥
Mitchell LP Relaxations 28 / 29
Describing Polyhedra by Facets
Example 2
Second example with implicit equality in R2
Q :=
x ∈ R2 :
2x1 + x2 = 4 M=
x1 + x2 = 4 M=
x1 ≥ 0 M=
x2 ≥ 0 M≥
Mitchell LP Relaxations 28 / 29
Describing Polyhedra by Facets
Summary
The aim in a cutting plane approach is to try to develop a minimaldescription of the polyhedron conv(S), at least in the neighborhoodof an optimal point to (IP).
This pair of theorems tells us that it is enough to determine all thefacets of conv(S), as well as a description of the affine hull of conv(S).
Mitchell LP Relaxations 29 / 29