1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .
-
date post
19-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .
![Page 1: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/1.jpg)
1
Design and Analysis of Algorithms
Yoram MosesLecture 7
April 22, 2010
http://www.ee.technion.ac.il/courses/046002
![Page 2: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/2.jpg)
2
Linear Programming
![Page 3: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/3.jpg)
3
Standard Form
Maximization All constraints are ≤ inequalities All variables are non-negative
Objective function
Constraints
Non-negativity constraints
![Page 4: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/4.jpg)
4
Slack Form Maximization n+m variables and m constraints n “regular variables” and m “slack variables” All variables are non-negative All constraints, except for non-negativity, are equalities Slack variables appear only on LHS of equality constraints Each equality constraint has a single unique slack variable
![Page 5: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/5.jpg)
5
Conversion to Slack Form: Example
Standard LP
Slack LP
![Page 6: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/6.jpg)
6
Basic vs. Non-basic Variables
Slack variables basic variables Regular variables non-basic variables B = set of basic variables, N = set of non-basic variables
Ex: B = { 4,5,6 }, N = { 1,2,3 } |B| = m, |N| = n For a solution x, xB = basic part of x, xN = non-basic part of x
![Page 7: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/7.jpg)
7
Short Matrix Form
|B| = m, |N| = n Constraints can be indexed by members of B For a solution x,
xB = basic part of x (i.e., xi for all i B) xN = non-basic part of x (i.e., xj for all j N)
![Page 8: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/8.jpg)
8
Basic Solutions
Every slack form is associated with a basic solution: All non-basics are set to 0 (i.e., xN = 0)
All basics are set to corresponding free coefficients (i.e., xB = b)
Ex: x1 = 0, x2 = 0, x3 = 0, x4 = 30, x5 = 24, x6 = 36
![Page 9: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/9.jpg)
9
Tight Constraints & Constraint Violation
Tight constraint: One in which the basic variable is forced to 0 in the basic solutionThe corresponding free coefficient is 0.
Violated constraint: One in which the basic variable is forced to be negative in the basic solution. the corresponding free coefficient is negative.
![Page 10: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/10.jpg)
10
Basic Feasible Solutions
Basic Feasible Solution (BFS): A basic solution, which is feasible
Easy fact: A basic solution is feasible if and only if all free coefficients are non-negative.
Lemma: Every BFS corresponds to a vertex of the feasible region polytope.
![Page 11: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/11.jpg)
11
Simplex Algorithm: Overview
Works in iterations At each iteration: transform one slack form
P into an equivalent slack form P’ Objective value of basic solution of P’ is
always at least as good as that of P Stop when reaching a local optimum
![Page 12: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/12.jpg)
12
Moving to an Equivalent Slack Form How to increase the objective value of the BFS?
Increase from 0 the value of some non-basic variable, whose coefficient in the objective function is positive.
By how much? As much as possible without violating any of the constraints.
objective: 0
Can increase x1 by at most 9. Objective increases to 27.
![Page 13: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/13.jpg)
13
Switching Basic with Non-basic Suppose that we increase a non-basic variable xi until
some constraint j becomes tight xj, the basic variable of the constraint j, becomes 0
We can thus switch between xi and xj
xi will become the basic variable of constraint j
xj will become a non-basic
When increasing x1 to 9, x6 becomes 0.
We switch between x1 and x6.
![Page 14: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/14.jpg)
14
Switching Basic with Non-basic
We write x1 as a function of other non-basics and x6:
![Page 15: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/15.jpg)
15
Switching Basic with Non-basic
We rewrite the objective function:
![Page 16: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/16.jpg)
16
Switching Basic with Non-basic
We rewrite all the constraints as well, and obtain the following equivalent linear program:
![Page 17: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/17.jpg)
17
Switching Basic with Non-basic
The new LP is equivalent to the previous LP We just rewrote x1 in terms of other variables
Basic solution: x1 = 9, x2 = 0, x3 = 0, x4 = 21, x5 = 6, x6 = 0 New objective value: 27
![Page 18: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/18.jpg)
18
Example continued
Choose x3
First constraint to become tight is constraint 3.
objective: 27
objective: 27.75
![Page 19: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/19.jpg)
19
Example continued
Choose x2
First constraint to become tight is constraint 2.
objective: 27.75
objective: 28
![Page 20: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/20.jpg)
20
Example continued
No more non-basics whose coefficient in the objective function is positive We stop and output basic solution as the optimal solution Solution: x1 = 8, x2 = 4, x3 = 0, x4 = 18, x5 = 0, x6 = 0. Value: 28.
objective: 28
![Page 21: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/21.jpg)
21
Back to the Standard Form
Solution: x1 = 8, x2 = 4, x3 = 0. Value: 28.
objective: 28
![Page 22: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/22.jpg)
22
Pivoting
Pivot: a single iteration of the simplex algorithm1. Choose a non-basic variable xi whose coefficient in the
objective function is > 0 xi is called the “entering variable” If more than one exists, choose one according to some pivoting
rule
2. Find the first constraint j that will be violated when we increase the value of xi from 0 If more than one exists, choose one according to some pivoting
rule
3. Make xi the basic variable of constraint j, and make xj a non-basic variable xj is called the “leaving variable”
4. Write xi as a function of xj and the other non-basics5. Rewrite the objective function and the constraints
![Page 23: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/23.jpg)
23
Pivoting: Geometric Intuition
Lemma: Pivoting corresponds to moving from one vertex of the feasible region to a neighbor vertex, whose objective value is at least as good.
![Page 24: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/24.jpg)
24
Unbounded Programs
Sometimes it is possible to increase the value of the entering value unboundedly, without violating any constraint
In this case the optimal solution of the LP is unbounded Pivot will return “unbounded”
![Page 25: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/25.jpg)
25
The Simplex Algorithm
1. find an initial BFS2. while there is a non-basic variable whose coefficient in
the objective function is > 03. run pivot4. if pivot returns “unbounded” return “unbounded”5. return BFS of current slack form as the optimal solution
Geometric view: Repeatedly move from a vertex of the feasible region to a better neighbor vertex, until a local maximum is reached.
Initial BFS is found by solving an auxiliary linear program (read section in book)
![Page 26: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/26.jpg)
26
Simplex Analysis: Correctness
If LP is infeasible, S’x will fail to find an initial BFS If LP is unbounded, Pivot will return “unbounded” If LP has a bounded optimal solution, it has one at
a vertex Simplex will reach a local maximum vertex Local maximum vertex must be a global maximum Hence, Simplex will output an optimal solution
![Page 27: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/27.jpg)
27
Simplex Analysis: Running Time
We have not specified the two “pivoting rules” For choosing the entering variable For choosing the leaving variable
Degeneracy: objective value of BFS does not improve in an invocation of Pivot
Unwise pivoting rules may lead to infinite loops (i.e., everlasting degeneracy)
![Page 28: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/28.jpg)
28
Degeneracy
objective: 0 entering: x1
leaving: x4
objective: 8 entering: x3
leaving: x5
objective: 8 entering: x2
leaving: x1
![Page 29: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/29.jpg)
29
Pivoting Rules
Bland’s rule: choose entering/leaving variable with smallest index.
Lemma: If Simplex uses Bland’s rule, it never cycles.
Conclusion: Simplex has at most iterations. Theorem [Kalai]: There is a randomized pivoting
rule, with which Simplex runs for a sub-exponential number of iterations in expectation.
Open problem: Is there a pivoting rule with which Simplex runs in polynomial time?
![Page 30: 1 Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 .](https://reader038.fdocuments.in/reader038/viewer/2022103005/56649d3f5503460f94a18ce3/html5/thumbnails/30.jpg)
30
End of Lecture 8