Post on 23-Nov-2020
Linear Programming
What’s Linear Programming?
Often your try is to maximize or minimize an objective within given constraints
A linear programming problem can be expressed as a linear function of certain variables and the constraints can be expressed as equalities or inequalities on those variables
Linear programming could be a minimization linear program or a maximization linear program
2
Election Spending Problem
You’re running for office
100K urban voters, 200K suburban, and 50K rural voters are your target
You want to win over at least 50% of voters in each area
There’s a catch. These voters may be persuaded differently by advertisements on different issues
3
Election Spending ProblemThe effect of policies (and hence advertising on those policies) on thousands of voters who can be won over (or lost) by spending $1,000 on particular issue.
4
Policy urban suburban rural
building roads -2 5 3
gun control 8 2 -5
farm subsidies 0 0 10
gasoline tax 10 0 -2
Evolving the Linear ProgramRather than trial and error, you can formulate the problem
Let x1 be the number of thousands of dollars spent on advertising on building roads, x2 ... on gun control, x3 ... on farm subsidies, and x4 ... on gasoline tax (for public transit improvements)
You can formulate the requirement to win half urban voters (50k out of 100k voters) as
-2x1 + 8x2 + 0x3 + 10x4 >= 50
...(similarly for other areas)
You have to ensure xi >= 0 (as spending can’t be negative) and minimize x1 + x2 + x3 + x4 5
Linear Program for Election $Minimize x1 + x2 + x3 + x4
subject to
-2x1 + 8x2 + 0x3 + 10x4 >= 50
5x1 + 2x2 + 0x3 + 0x4 >= 100
3x1 - 5x2 + 10x3 - 2x4 >= 25
x1, x2, x3, x4 >= 0
Solution to the linear program yields the optimal strategy.
6
General Linear ProgramsOptimize a linear function subject to a set of linear inequalities
Given a set of real numbers a1, a2, ..., an and a set of variables x1, x2, ..., xn, we define a linear function f on those variables:
f(x1, x2, ..., xn) = a1x1 + a2x2 + ... + anxn = !1<=i<=n aixi
plus linear constraints: either a linear equality f(x1, x2, ..., xn) = b or linear inequalities f(x1, x2, ..., xn) <= b or f(x1, x2, ..., xn) >= b.
7
Canonical Forms of LP
Standard
Maximization of a linear function subject to linear inequalities
Slack
Maximization of a linear function subject to linear equalities
8
Feasible SolutionA linear program has n variables subject to a set of m linear inequalities
A feasible solution is any values for variables that satisfy all the constraints
Feasible region is set of all feasible solutions
The maximization function is called the objective function and the value of this function at particular point in the feasible region an objective value
There may be infinite number of points in a feasible region, so we need a way to practically find the optimal solution for the objective value
9
Solving Two VariablesYou can plot the constraints on a cartesian coordinate and find an optimal solution for the objective value
10
f1f2
f3
Objective function
OptimalSolution
The optimal solution is the vertex of the feasible region
If it is a line, you can use end points of the line segment
Solving Multiple VariablesEach constrain forms a half-space in n-th dimensional space
Feasible region formed by the intersection of these half-spaces is called simplex
The objective function is a hyperplane and the optimal solution occurs at the vertex of the simplex
The simplex algorithm moves through the exterior of the simplex, determining the vertex with maximum value that satisfy the objective function
11
The Simplex AlgorithmIterative algorithm
Set each nonbasic variable to 0 and compute the values of the basic variables from the equality constraints
In each iteration you convert from one slack form to another slack form
You improve the objective value in each iteration
We increase values (subject to constraints) of nonbasic variables from 0 in order to maximize the objective value
You convert the equations into equivalent forms until the objective function has all variables with negative coefficient
12
Example of Simplex AlgGiven the linear program in standard form
maximize 3x1 + x2 + 2x3
subject to
x1 + x2 + 3x3 <= 30
2x1 + 2x2 + 5x3 <= 24
4x1 + x2 + 2x3 <= 36
x1, x2, x3 >= 013
Convert to Slack Form
z = 3x1 + x2 + 2x3
x4 = 30 - x1 - x2 - 3x3
x5 = 24 - 2x1 - 2x2 - 5x3
x6 = 36 - 4x1 - x2 - 2x3
Nonbasic variables on the right, basic variables on the left
14
Giving Valuesz = 3x1 + x2 + 2x3
x4 = 30 - x1 - x2 - 3x3
x5 = 24 - 2x1 - 2x2 - 5x3
x6 = 36 - 4x1 - x2 - 2x3
Giving values of 0 to nonbasic variables, we have (0, 0, 0, 30, 24, 36) and z gets a value of 0.
Now we want to reformulate so the basic solution has a greater objective value.
15
Reformulatingz = 3x1 + x2 + 2x3
x4 = 30 - x1 - x2 - 3x3
x5 = 24 - 2x1 - 2x2 - 5x3
x6 = 36 - 4x1 - x2 - 2x3
Take a nonbasic variable whose coefficient in the objective function is positive and increase the value as much as possible without violating constraints. This variable becomes basic and some other variable becomes nonbasic.
16
Reformulatingz = 3x1 + x2 + 2x3
x4 = 30 - x1 - x2 - 3x3
x5 = 24 - 2x1 - 2x2 - 5x3
x6 = 36 - 4x1 - x2 - 2x3
Increasing x1 will result in decreasing x4, x5, and x6.
However, non of them can become negative due to constraints. x4, x5, and x6, restrict x1 to 30, 12, and 9, respectively.
The third constraint is the most limiting. We switch roles of x1 and x6. 17
Pivoting
z = 3x1 + x2 + 2x3
x4 = 30 - x1 - x2 - 3x3
x5 = 24 - 2x1 - 2x2 - 5x3
x1 = 9 - x2/4 - x3/2 - x6/4
You pivot—that is you chose an entering variable (x1) and a leaving variable (x6) and exchange their roles.
You now replace all x1
18
Pivotingz = 27 + x2/4 + x3/2 - 3x6/4
x1 = 9 - x2/4 - x3/2 - x6/4
x4 = 21 - 3x2/4 - 5x3/2 + x6/4
x5 = 6 - 3x2/2 - 4x3 + x6/2
You’re ready to pick the next pivot, an entering variable with positive coefficient in the objective function and a corresponding leaving variable.
Let’s chose x3 and that will lead to the tightest function and hence the leaving variable x5.
19
Pivoting
z = 111/4 + x2/16 - x5/8 - 11x6/16
x1 = 33/4 - x2/16 + x5/8 - 5x6/16
x3 = 3/2 - 3x2/8 - x5/4 + x6/8
x4 = 69/4 + 3x2/16 + 5x5/8 - x6/16
The lone standing positive coefficient nonbasic variable in objective function is x2
Using that as a pivot,...
20
Pivoting
z = 28 - x3/6 - x5/6 - 2x6/3
x1 = 8 + x3/6 + x5/6 - x6/3
x2 = 4 - 8x3/3 - 2x5/3 + x6/3
x4 = 18 - x3/2 + x5/2
Giving zero values to the nonbasic variables, we get x1 = 8, x2 = 4, x3 = 0, and the optimal solution z = 28.
21
Slack Variablesmaximize 3x1 + x2 + 2x3
x1 + x2 + 3x3 <= 30 12 <= 30, slack is 18 which is x4
2x1 + 2x2 + 5x3 <= 24 24 <= 24, slack is 0 which is x5
4x1 + x2 + 2x3 <= 36 36 <= 36, slack is 0 which is x6
x1, x2, x3 >= 0
Solution x1 = 8, x2 = 4, x3 = 0, and the optimal solution z = 28.
x4, x5, and x6 are the slack variable. The slacks are 18, 0, and 0. If you plug in values for x1, x2, and x3 into original inequalities, you can see the slack.
22
Pivot
23
Simplex Algorigthm
24
Issues to considerIt is possible that the feasible region is unbounded
It is possible that the iteration leaves the objective value unchanged—known as degeneracy
You need to look for cycles where the slack forms simply cycle through transformations
Since Simplex algorithm is deterministic, it will cycle over the same sequence
You can determine it cycles if it does not terminate in at most ( ) iterations
25
n+mm