Today: Linear Programming -...
Transcript of Today: Linear Programming -...
Today: − Linear Programming
COSC 581, Algorithms March 27, 2014
Many of these slides are adapted from several online sources
Reading Assignments
• Today’s class: – Chapter 29.1
• Reading assignment for next Thursday’s class:
– Chapter 29.2-3
• Announcement: Exam #2 on Tuesday, April 1 – Will cover greedy algorithms, amortized analysis – HW 6-9
Linear Programming Example: A Political problem
Suppose a politician is trying to win an election
3 types of areas --- urban, suburban, rural
Certain issues --- roads, gun control, farm subsidies, gasoline tax
Politician wants to find out the minimum amount of money he/she needs to win 50,000 urban, 100,000 suburban, 25,000 rural votes.
Policy urban suburban rural
Build roads -2 5 3
Gun control 8 2 -5
Farm subsidies 0 0 10
Gasoline tax 10 0 -2
The effects of policies on votes:
Define 4 variables : x1 is the # of thousands of dollars spent on advertising on building roads x2 is the # of thousands of dollars spent on advertising on gun control x3 is the # of thousands of dollars spent on advertising on farm subsidies x4 is the # of thousands of dollars spent on advertising on gasoline tax
We format this problem as a linear program: Minimize: x1 + x2 + x3 + x4 // Total spent
Subject to: −2x1 + 8x2 + 0x3 + 10x4 ≥ 50 // urban votes 5x1 + 2x2 + 0x3 + 0x4 ≥ 100 // suburb. votes 3x1 − 5x2 + 10x3 − 2x4 ≥ 25 // rural votes x1, x2, x3, x4 ≥ 0 // can’t have negative costs
The solution of this linear program will yield an optimal strategy for the politician.
How to Solve?
Applications of Linear Programming
LP is a widely used Mathematical Optimization Model:
• Used frequently in management science (operations research), engineering, technology, industry, commerce, economics.
• Efficient resource allocation technique:
– Airline transportation
– Communication networks – e.g., optimize transmission routing
– Factory inventory/production control
– Fund management, stock portfolio optimization
• Can be used to approximate hard optimization problems
• . . .
History of LP 3000-200 BC: Egypt, Babylon, India, China, Greece: [geometry & algebra]
Egypt: polyhedra & pyramids. India: Sulabha suutrah (Easy Solution Procedures) [2 equations, 2 unknowns] China: Jiuzhang suanshu (9 Chapters on the Mathematical Art) [Precursor of Gauss-Jordan elimination method on linear equations] Greece: Pythagoras, Euclid, Archimedes, …
825 AD: Persia: Muhammad ibn-Musa Alkhawrazmi (author of 2 influential books): “Al-Maqhaleh fi Hisab al-jabr w’almoqhabeleh” (An essay on Algebra and equations) “Kitab al-Jam’a wal-Tafreeq bil Hisab al-Hindi” (Book on Hindu Arithmetic). originated the words algebra & algorithm for solution procedures of algebraic systems.
Fourier [1826], Motzkin [1933] [Fourier-Motzkin elimination method on linear inequalities]
Minkowski [1896], Farkas [1902], De la Vallée Poussin [1910], von Neumann [1930’s], Kantorovich [1939], Gale [1960] [LP duality theory & precursor of Simplex]
George Dantzig [1947]: Simplex algorithm. Exponential time in the worst case, but effective in practice.
Leonid Khachiyan [1979]: Ellipsoid algorithm.
The first weakly polynomial-time LP algorithm: poly(n,d,L).
Narendra Karmarkar [1984]: Interior Point Method. Also weakly polynomial-time. IPM variations are very well studied.
Megiddo-Dyer [1984]: Prune-&-Search method. O(n) time if the dimension is a fixed constant. Super-exponential on dimension.
The General LP Problem
subject to:
dd xcxcxc +⋅⋅⋅++ 2211
ndndnn
dd
dd
bxaxaxa
bxaxaxabxaxaxa
≤+⋅⋅⋅++
≤+⋅⋅⋅++≤+⋅⋅⋅++
2211
22222121
11212111
maximize Linear objective function
Linear constraints (stated as inequalities)
Some terminology • Objective function: value measure used to rank alternatives; either
minimize or maximize this objective • Decision variables: the quantities you can control to improve your
objective function; should completely describe the set of decisions to be made
• Constraints: limitations on the values of the decision variables • Linear program: a mathematical program in which the objective function
is a linear function and the constraints are linear equalities or inequalities • Objective value: value of objective function at a particular point • Feasible solution: satisfies all the constraints • Infeasible solution: doesn’t satisfy the constraints • Optimal solution: best feasible solution • Unbounded solution: solution to LP does not have finite objective value
General Idea of LP – via another example
LP Idea: Minimize or maximize a linear objective, Subject to linear equalities and inequalities Example: Bubba is in a pie eating contest that lasts 1 hour. Each torte that he eats takes 2 minutes. Each apple pie that he eats takes 3 minutes. He receives 4 points for each torte and 5 points for each pie. What should Bubba eat so as to get the most points? Step 1: Determine the decision variables: Let x be the number of tortes eaten by Bubba Let y be the number of pies eaten by Bubba
General Idea of LP – (con’t) Step 2: Determine the objective function: Maximize z = 4x + 5y Step 3: Determine the constraints: 2x + 3y ≤ 60 // time constraints x ≥ 0; y ≥ 0 // non-negativity constraints – i.e., can’t eat negative amounts
Feasible Region
Visualizing LP -- Example in 2D
max x1 + 8x2 subject to: (1) x1 ≥ 3 (2) x2 ≥ 2 (3) –3x1 + 4x2 ≤ 14 (4) 4x1 – 3x2 ≤ 25 (5) x1 + x2 ≤ 15
x2
x1
(1)
(2)
(4)
(5) (3)
(5) (3)
x1 = 46/7 x2 = 59/7
Each constraint is represented by a line and a direction. Intersection of constraints is feasible region.
Visualizing LP -- Example in 3D
subject to:
maximize z Optimum (x,y,z)=(0,0,3)
x
y
z
0002
3
≥≥≥≤
≤++
zyxy
zyx
Multi-dimensional cases (beyond 3D)
• Can’t visualize beyond 3D, but same idea holds
• Each constraint defines half-space in n-dimensional space
• The objective function is a hyperplane
• Feasible region is area formed by intersection of half-spaces
• This region is called a simplex
Important Observation: Optimal Solutions are at a Vertex or Line Segment • Intersection of objective function and feasible region is either
vertex or line segment
• Feasible region is convex – makes optimization much easier! • Simplex algorithm finds LP solution by:
– Starting at some vertex – Moving along edge of simplex to neighbor vertex whose value is at
least as large – Terminates when it finds local maximum
• Convexity ensures this local maximum is globally optimal
Optimum (x,y,z)=(0,0,3)
x
y
z
Feasible Region
Recall – History of LP 3000-200 BC: Egypt, Babylon, India, China, Greece: [geometry & algebra]
Egypt: polyhedra & pyramids. India: Sulabha suutrah (Easy Solution Procedures) [2 equations, 2 unknowns] China: Jiuzhang suanshu (9 Chapters on the Mathematical Art) [Precursor of Gauss-Jordan elimination method on linear equations] Greece: Pythagoras, Euclid, Archimedes, …
825 AD: Persia: Muhammad ibn-Musa Alkhawrazmi (author of 2 influential books): “Al-Maqhaleh fi Hisab al-jabr w’almoqhabeleh” (An essay on Algebra and equations) “Kitab al-Jam’a wal-Tafreeq bil Hisab al-Hindi” (Book on Hindu Arithmetic). originated the words algebra & algorithm for solution procedures of algebraic systems.
Fourier [1826], Motzkin [1933] [Fourier-Motzkin elimination method on linear inequalities]
Minkowski [1896], Farkas [1902], De la Vallée Poussin [1910], von Neumann [1930’s], Kantorovich [1939], Gale [1960] [LP duality theory & precursor of Simplex]
George Dantzig [1947]: Simplex algorithm. Exponential time in the worst case, but effective in practice.
Leonid Khachiyan [1979]: Ellipsoid algorithm.
The first weakly polynomial-time LP algorithm: poly(n,d,L).
Narendra Karmarkar [1984]: Interior Point Method. Also weakly polynomial-time. IPM variations are very well studied.
Megiddo-Dyer [1984]: Prune-&-Search method. O(n) time if the dimension is a fixed constant. Super-exponential on dimension.
Our study – while not polynomial, can be fast in practice; variants are commonly used today
In-Class exercise
Maximize: Subject to:
22 6
2 6, 0
x y
x yx y
x yx y
+
− + ≤+ ≤+ ≤
≥
Consider the following linear programming problem: What does the feasible region look like? What is the optimal solution?
Two Canonical Forms for LP: Standard and Slack
• An LP is in standard form if it is the maximization of a linear function subject to linear inequalities
• An LP is in slack form if it is the maximization of a linear function subject to linear equalities
Standard Form • We’re given:
𝑛 real numbers 𝑐1, 𝑐2, … 𝑐𝑛 𝑚 real numbers 𝑏1, 𝑏2, … 𝑏𝑚 𝑚𝑛 real numbers 𝑎𝑖𝑖, for 𝑖 = 1, 2, …, 𝑚 and 𝑗 = 1, 2, … 𝑛
• We want to find:
𝑛 real numbers 𝑥1, 𝑥2, … 𝑥𝑛 that: Maximize: ∑ 𝑐𝑖𝑥𝑖𝑛
𝑖=1 Subject to:
�𝑎𝑖𝑖𝑥𝑖 ≤ 𝑏𝑖
𝑛
𝑖=1
for 𝑖 = 1, 2, … ,𝑚
𝑥𝑖 ≥ 0 for 𝑗 = 1, 2, … ,𝑛
Compact Version of Standard Form • Let: 𝐴 = 𝑎𝑖𝑖 be 𝑚 × 𝑛 matrix
𝑏 = 𝑏𝑖 be an 𝑚-vector 𝑐 = 𝑐𝑖 be an 𝑛-vector
𝑥 = 𝑥𝑖 be an 𝑛-vector
• Rewrite LP as: Maximize: 𝑐𝑇𝑥 Subject to:
𝐴𝑥 ≤ 𝑏 𝑥 ≥ 0
• Now, we can concisely specify LP in standard form as (A, b, c)
Converting LP to Standard Form
4 reasons an LP might not be in standard form: 1) Objective function might be a minimization instead
of maximization 2) There might be variables w/o non-negativity
constraints 3) There might be equality constraints 4) There might be inequality constraints that are “ ≥ ”
instead of “ ≤ “
We can convert any LP into an equivalent standard form
Step 1: Change min LP to max LP • To convert a minimization linear program L into an
equivalent maximization linear program L′, we simply negate the coefficients in the objective function. Example:
Step 2: Dealing with missing non-negativity constraints
• Suppose that some variable 𝑥𝑖 does not have a non-negativity constraint.
• Then: – We replace each occurrence of 𝑥𝑖 by 𝑥′𝑖 − 𝑥
Step 3: Converting equality constraints into inequality constraints
• Suppose that a linear program has an equality constraint f (x1, x2, ..., xn) = b.
• Since x = y if and only if both x ≥ y and x ≤ y, we can replace f (x1, x2, ..., xn) = b by: f (x1, x2, ...,xn) ≤ b and f (x1, x2, ..., xn) ≥ b.
Step 4: Convert “≥” constraints to “≤” constraints
• We can convert the “≥” constraints to “≤” constraints by multiplying these constraints through by -1.
• That is, any inequality of the form:
�𝑎𝑖𝑖𝑥𝑖 ≥ 𝑏𝑖
𝑛
𝑖=1
is equivalent to:
�−𝑎𝑖𝑖𝑥𝑖 ≤ −𝑏𝑖
𝑛
𝑖=1
Slack Form – Useful for Simplex • In slack form, the only inequality constraints are the non-
negativity constraints – All other constraints are equality constraints
• Let: ∑ 𝑎𝑖𝑖𝑥𝑖 ≤ 𝑏𝑖𝑛
𝑖=1 be an inequality constraint • Introduce new variable s, and rewrite as:
𝑠 = 𝑏𝑖 −�𝑎𝑖𝑖𝑥𝑖
𝑛
𝑖=1
𝑠 ≥ 0 • s is a slack variable; it represents difference between left-hand
and right-hand sides
Slack Form (con’t.)
• In general, we’ll use 𝑥𝑛+𝑖 (instead of s) to denote the slack variable associated with the 𝑖th inequality.
• The 𝑖th constraint is therefore:
𝑥𝑛+𝑖 = 𝑏𝑖 −�𝑎𝑖𝑖𝑥𝑖
𝑛
𝑖=1
along with the non-negativity constraint 𝑥𝑛+𝑖 ≥ 0
Example
Standard form: Maximize 2𝑥1 − 3𝑥2 + 3𝑥3 subject to:
𝑥1 + 𝑥2 − x3 ≤ 7 −x1 − x2 + x3 ≤ −7
x1 − 2x2 + 2x3 ≤ 4 x1, x2, x3 ≥ 0
Slack form: Maximize 2𝑥1 − 3𝑥2 + 3𝑥3 subject to: 𝑥4 = 7 − 𝑥1 − 𝑥2 + 𝑥3 𝑥5 = −7 + 𝑥1 + 𝑥2 − 𝑥3 𝑥6 = 4 − 𝑥1 + 2𝑥2 − 2𝑥3 𝑥1, 𝑥2, 𝑥3, 𝑥4, 𝑥5, 𝑥6 ≥ 0
About Slack Form…
Slack form: Maximize 2𝑥1 − 3𝑥2 + 3𝑥3 subject to: 𝑥4 = 7 −𝑥1 −𝑥2 + 𝑥3 𝑥5 = −7 + 𝑥1 + 𝑥2 − 𝑥3 𝑥6 = 4 − 𝑥1 + 2𝑥2 − 2𝑥3 𝑥1, 𝑥2, 𝑥3, 𝑥4, 𝑥5, 𝑥6 ≥ 0
Basic variables – variables on left-hand side
Non-basic variables – variables on right-hand side
Concise Representation of Slack Form • Can eliminate “maximize”, “subject to”, and non-negativity constraints (all are
implicit) • And, introduce z as value of objective function:
𝑧 = 2𝑥1 − 3𝑥2 + 3𝑥3 𝑥4 = 7 − 𝑥1 − 𝑥2 + 𝑥3 𝑥5 = −7 + 𝑥1 + 𝑥2 − 𝑥3 𝑥6 = 4 − 𝑥1 + 2𝑥2 − 2𝑥3
• Then, define slack form of LP as tuple (N, B, A, b, c, v) where N = indices of nonbasic variables B = indices of basic variables
• We can rewrite LP as:
𝑧 = 𝑣 + �𝑐𝑖𝑥𝑖𝑖∈𝑁
𝑥𝑖 = 𝑏𝑖 −�𝑎𝑖𝑖𝑥𝑖 for 𝑖 ∈ 𝐵𝑖∈𝑁
In-Class Exercise
Time period:
9:00-10:00
10:00-11:00
11:00-12:00
12:00-1:00
1:00-2:00
2:00-3:00
3:00-4:00
4:00-5:00
Tellers required:
4 3 4 6 5 6 8 8
A bank is open Monday-Friday from 9am to 5pm. From past experience, the bank knows that it needs (at least) the following number of tellers: The bank hires two types of tellers. Full time tellers work 9-5 every day, except for 1 hour off for lunch. (The bank determines when a full time teller takes lunch hour, but it must be either 12-1 or 1-2.) Full time employees are paid $8 per hour (this includes payment for the lunch hour). The bank can also hire part time tellers. Each part time teller must work exactly 3 consecutive hours each day, and gets paid $5 per hour. To maintain quality of service, at most 5 part time tellers can be hired. Formulate a LP to minimize the cost of the bank to meet teller requirements.
Reading Assignments
• Reading assignment for next Thursday’s class: – Chapter 29.2-3
• Announcement: Exam #2 on Tuesday, April 1 – Will cover greedy algorithms, amortized analysis – HW 6-9