Linear Programming - cs.iastate.educs511/handout10/LP Basics.pdf · Linear Programming Computer...
Transcript of Linear Programming - cs.iastate.educs511/handout10/LP Basics.pdf · Linear Programming Computer...
Linear Programming
Computer Science 511
Iowa State University
September 24, 2010
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 1 / 27
Example: Profit Maximization
Joe sells two kinds of beer: Regular Duff and Duff Special Brew.
Profit per pint for Regular Duff is $1.
Profit per pint for Duff Special is $2.
Demand for Regular Duff is at most 200 pints per day.
Demand for Duff Special is at most 400 pints per day.
Duff will sell Joe a combined total of at most 500 pints of RegularDuff and Duff Special per day.
How many pints of each beer should Joe buy per day so as tomaximize his profit?
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 2 / 27
Example: LP Formulation
Let x1 and x2 denote the number of pints of Regular Duff and Duff Specialthat Joe will buy per day.
maximize x1 + 2x2
subject tox1 ≤ 200
x2 ≤ 400x1 + x2 ≤ 500x1 , x2 ≥ 0
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 3 / 27
Linear Programming
Maximize (or minimize) a linear function
n∑j=1
cjxj
subject to linear equalities/inequalities of the form
n∑j=1
ajxj = b,
n∑j=1
ajxj ≤ b, orn∑
j=1
ajxj ≥ b.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 4 / 27
Terminology
Feasible solution: Any x = (x1, . . . , xn) that satisfies all the constraints.
Feasible region: Set of all feasible solutions.
The intersection of a collection of halfspaces andhyperplanes.⇒ A convex polyhedral region.
Objective function: The function of x1, . . . , xn that we want to maximize.
Objective value: The value of the objective function for some specificx1, . . . , xn.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 5 / 27
Example
maximize x1 + 2x2
subject tox1 ≤ 200
x2 ≤ 400x1 + x2 ≤ 500x1 , x2 ≥ 0
400
300
100
0100 200 300 400
400
500 !!
!"
Obj. val. = 900
Obj. val. = 400
200
Sunday, September 12, 2010
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 6 / 27
Infeasibility and unboundedness
Definition
A linear program is infeasible if it has no feasible solution.
Example
Maximize x subject to x ≤ −1 and x ≥ 0.
Definition
A linear program is unbounded if its objective value can be madearbitrarily large (small).
Example
Maximize x subject to x ≥ 0.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 7 / 27
The Simplex Algorithm (Dantzig 1947)
Moves from vertex to neighboring vertex of feasible region.
Objective value improves at each step.
Stop when no neighbor gives a higher value.
By convexity, this vertex must be optimum.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 8 / 27
Standard Form
Fact
Every linear program can be put into the following standard form.
maximizen∑
j=1
cjxj
subject ton∑
j=1
aijxj ≤ bi i = 1, . . . ,m
xj ≥ 0 j = 1, . . . n
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 9 / 27
Converting to Standard Form
1 To turn a minimization problem into maximization problem, multiplythe coefficients of the objective function by −1.
2 To reverse the direction of an inequality, multiply both sides by −1.
3 To change an equality constraint into inequalities rewrite∑nj=1 ajxj = b as
n∑j=1
ajxj ≤ b andn∑
j=1
ajxj ≥ b.
4 If x is a variable that is unrestricted in sign, introduce two nonnegativevariables, x+, x− ≥ 0 and replace every occurrence of x by x+ − x−.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 10 / 27
Slack Form
For the ith constraint, define a nonnegative slack variable xn+i :
xn+i = 0 ⇐⇒ constraint i is tight.
At all times, express constraints as a set of equalities where the nvariables in RHS are 0.
I RHS variables correspond to tight inequalities.
Tight inequalities define a vertex of the feasible region.I A point in Rn is defined by n hyperplanes.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 11 / 27
Original
maximizen∑
j=1
cjxj
subject ton∑
j=1
aijxj ≤ bi i = 1, . . . ,m
xj ≥ 0 j = 1, . . . n
Slack Form
z =n∑
j=1
cjxj
xn+i = bi −n∑
j=1
aijxj i = 1, . . . ,m
(x1, . . . , xn+m ≥ 0)
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 12 / 27
Slack Form: Example
Original
maximize 2x1 + 5x2
subject to2x1 − x2 ≤ 4x1 + 2x2 ≤ 9−x1 + x2 ≤ 3
x1 ≥ 0x2 ≥ 0
Slack Form
z = 0 + 2x1 + 5x2
x3 = 4 − 2x1 + x2
x4 = 9 − x1 − 2x2
x5 = 3 + x1 − x2
(x1, x2, x3, x4, x5 ≥ 0)
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 13 / 27
Dictionaries
At all times, simplex maintains a dictionary:
z = v ′ +∑j∈N
c ′jxj
xi = b′i −∑j∈N
a′ijxj for i ∈ B
Variables in RHS are called non-basic.I For each j ∈ N, xj = 0.
Variables in LHS are called basic.I For each i ∈ B, xi = b′
i .
The values of the variables implied by a dictionary give a basicfeasible solution (b.f.s.).
I z = v ′ is the objective value of the b.f.s.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 14 / 27
Pivoting
z = v ′ +∑j∈N
c ′jxj
xi = b′i −∑j∈N
a′ijxj for i ∈ B
Greedy Approach
Find a j ∈ N such that c ′j > 0.I z can be increased by increasing xj .I If no such j exists, current b.f.s. is
optimal.
Increase xj as much as possible,keeping xi ≥ 0 for all i ∈ B.
I xj enters the basis.
For some i ∈ B, xi drops to 0.I xi leaves the basis.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 15 / 27
Dictionary 1
z = 0 + 2x1 + 5x2
x3 = 4 − 2x1 + x2
x4 = 9 − x1 − 2x2
x5 = 3 + x1 − x2
Basic feasible solution:
(x1, x2, x3, x4, x5) = (0, 0, 4, 9, 3)
Objective value: 0
Entering variable: x2
Leaving variable x5
Dictionary 2
z = 15 + 7x1 − 5x5
x3 = 7 − x1 − x5
x4 = 3 − 3x1 − 2x5
x2 = 3 + x1 − x5
Basic feasible solution:
(x1, x2, x3, x4, x5) = (0, 3, 7, 3, 0)
Objective value: 15
Entering variable: x1
Leaving variable x4
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 16 / 27
Dictionary 2
z = 15 + 7x1 − 5x5
x3 = 7 − x1 − x5
x4 = 3 − 3x1 − 2x5
x2 = 3 + x1 − x5
Basic feasible solution:
(x1, x2, x3, x4, x5) = (0, 3, 7, 3, 0)
Objective value: 15
Entering variable: x1
Leaving variable x4
Dictionary 3
z = 22 − 73x4 − 1
3x5
x3 = 6 + 13x4 − 5
3x5
x1 = 1 − 13x4 + 2
3x5
x2 = 4 − 13x4 − 1
3x5
Basic feasible solution:
(x1, x2, x3, x4, x5) = (1, 4, 6, 0, 0)
Objective value: 22
Optimal
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 17 / 27
Unboundedness
Fact
Simplex can be modified to discover unboundedness.
Method
If a linear program is unbounded, at some point, Simplex will find anon-basic xi such that
xi has a positive coefficient in the objective function and
none of the constraints binds xi .
At this point, stop and return unbounded.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 18 / 27
Cycling
Successive pivots may encounter solutions with same objective value.
Reason: the same vertex may be defined by different sets ofconstraints.
Cycling — i.e., getting stuck at a vertex — is a possibility.
Bland’s rule
When choosing entering or leaving variable, always pick the one withsmallest index.
Fact
Using Bland’s rule, Simplex will never cycle.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 19 / 27
Finding an Initial Basic Feasible Solution
The origin, (x1, . . . , xn) = (0, . . . , 0), is not always feasible.
Happens when bi < 0 for some i .
Example
maximize 2x1 + 5x2
subject to2x1 − x2 ≤ 4x1 + 2x2 ≤ 9−x1 + x2 ≤ 3−x1 − x2 ≤ −1
x1 , x2 ≥ 0
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 20 / 27
Finding an Initial Basic Feasible Solution
Set up and solve an auxiliary LP.
Auxiliary LP always has a solution.I An initial b.f.s. for auxiliary problem can be obtained easily.
Optimal solution to auxiliary LP gives an initial b.f.s. for original LP.
Optimal solution to auxiliary problem also tells us if original problemis feasible.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 21 / 27
The Auxiliary Problem: Example
Original Problem
max 2x1 + 5x2
subj. to2x1 − x2 ≤ 4x1 + 2x2 ≤ 9−x1 + x2 ≤ 3−x1 − x2 ≤ −1
x1 , x2 ≥ 0
Auxiliary Problem
max −x0
subj. to−x0 + 2x1 − x2 ≤ 4−x0 + x1 + 2x2 ≤ 9−x0 − x1 + x2 ≤ 3−x0 − x1 − x2 ≤ −1
x0 , x1 , x2 ≥ 0
The auxiliary problem has an obvious solution with x1, x2 = 0.I Not a b.f.s., though.
The original problem also has a solution, but it is less obvious.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 22 / 27
A B.F.S. for the Auxiliary Problem: One Pivot is Enough
Auxiliary Problem in Slack Form
z = 0 − x0
x3 = 4 + x0 − 2x1 + x2
x4 = 9 + x0 − x1 − 2x2
x5 = 3 + x0 + x1 − x2
x6 = −1 + x0 + x1 + x2
Not a legal dictionary:
(x0 = x1 = x2 = 0)⇒ (x6 = −1)
Make x0 enter the basisand x6 leave.
Not a “normal” pivot,since objective valuedecreases.
Legal Starting Dictionary
z = −1 + x1 + x2 − x6
x3 = 5 − 3x1 + x6
x4 = 10 − 2x1 − 3x2 + x6
x5 = 4 − 2x2 + x6
x0 = 1 − x1 − x2 + x6
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 23 / 27
The Auxiliary Problem: General Case
Original
maximizen∑
j=1
cjxj
subject ton∑
j=1
aijxj ≤ bi i = 1, . . . ,m
xj ≥ 0 j = 1, . . . n
Auxiliary
maximize −x0
subject ton∑
j=1
aijxj − x0 ≤ bi i = 1, . . . ,m
xj ≥ 0 j = 0, . . . n
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 24 / 27
The Auxiliary Problem
Fact
The original LP is feasible if and only if optimum objective value for theauxiliary LP is 0.
Proof.
Suppose (x1, . . . , xn) is a feasible solution to the original problem.I ⇒ x = (0, x1, . . . , xn) is a feasible solution to the auxiliary problem.I The objective value of x is 0.I Since we require x0 ≥ 0, x must be an optimal solution.
Suppose the optimum solution to the auxiliary problem has value 0.I ⇒ x0 = 0.I ⇒ (x1, . . . , xn) is a feasible solution for the original problem.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 25 / 27
The Fundamental Theorem of Linear Programming
Theorem
For any linear program in standard form:
1 if there is no optimal solution, then the problem is either infeasible orunbounded,
2 if a feasible solution exists, then a basic feasible solution exists,
3 if an optimum solution exists, then there exists a basic feasiblesolution that is also an optimum solution.
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 26 / 27
Running Time of the Simplex Algorithm
Time per pivot is polynomial.
Number of basic feasible solutions is ≤(m+n
n
).
In the worst case, Simplex may have to run through most of them.I Run time of Simplex is exponential in worst case.
However, this rarely happens.I In practice Simplex is fast.
There are polynomial time algorithms for LP (Khachiyan 1979,Karmarkar 1984)
Computer Science 511 (Iowa State University) Linear Programming September 24, 2010 27 / 27