ORF 307: Lecture 4 Linear Programming: Chapter 3 Degeneracy · A program that generates random 2 4...

19
ORF 307: Lecture 4 Linear Programming: Chapter 3 Degeneracy Robert Vanderbei February 15, 2018 Slides last edited on February 16, 2018 http://www.princeton.edu/rvdb

Transcript of ORF 307: Lecture 4 Linear Programming: Chapter 3 Degeneracy · A program that generates random 2 4...

ORF 307: Lecture 4

Linear Programming: Chapter 3Degeneracy

Robert Vanderbei

February 15, 2018

Slides last edited on February 16, 2018

http://www.princeton.edu/∼rvdb

Solve This...

maximize 2x1 + 3x2

subject to x1 + 2x2 ≤ 2x1 − x2 ≤ 1−x1 + x2 ≤ 1

x1, x2 ≥ 0.

1

Solution

⇓ Enter: x2, Leave: w3

⇓ Enter: x1, Leave: w1

⇓ Enter: w3, Leave: w2

Note: The horizontal axis, which onemight call the x1-axis, is where x2 = 0 andis labeled as such.

In (x1, x2) coordinates, the pivots visit thefollowing vertices:

(0, 0) =⇒ (0, 1) =⇒ (0, 1) =⇒ (4/3, 1/3).

Note that the second pivot went nowhere. 2

Degeneracy

Definitions.

A dictionary is degenerate if one or more “rhs”-value vanishes.

Example:

ζ = 6 + w3 + 5x2 + 4w1

x3 = 1 − 2w3 − 2x2 + 3w1

w2 = 4 + w3 + x2 − 3w1

x1 = 3 − 2w3

w4 = 2 + w3 − w1

w5 = 0 − x2 + w1

A pivot is degenerate if the objective function value does not change.

Examples (based on above dictionary):

1. If x2 enters, then w5 must leave, pivot is degenerate.

2. If w1 enters, then w2 must leave, pivot is not degenerate.

3

Cycling

A cycle is a sequence of pivots that returns to the dictionary from which the cycle began.

Note: Every pivot in a cycle must be degenerate. Why?

Pivot Rules

A pivot rule is an explicit statement for how one chooses entering and leaving variables (whena choice exists).

Some Examples:

Largest-Coefficient Rule. (most common pivot rule for entering variable)

Choose the variable with the largest coefficient in the objective function.

Random Positive-Coefficient Rule.

Among all nonbasic variables having a positive coefficient, choose one at random.

First Encountered Rule.

In scanning the nonbasic variables, stop with the first one whose coefficient is positive.

4

Hope

Some pivot rule, such as the largest coefficient rule, will be proven never to cycle.

An example that cycles using the following pivot rules:

• entering variable: largest-coefficient rule.

• leaving variable: smallest-index rule.

ζ = x1 − 2x2 − 2x4

w1 = − 0.5x1 + 3.5x2 + 2x3 − 4x4

w2 = − 0.5x1 + x2 + 0.5x3 − 0.5x4

w3 = 1 − x1 .

Here’s a demo of cycling (ignoring the last constraint)...

5

Hope

Some pivot rule, such as the largest coefficient rule, will be proven never to cycle.

Hope Fades

An example that cycles using the following pivot rules:

• entering variable: largest-coefficient rule.

• leaving variable: smallest-index rule.

ζ = x1 − 2x2 − 2x4

w1 = − 0.5x1 + 3.5x2 + 2x3 − 4x4

w2 = − 0.5x1 + x2 + 0.5x3 − 0.5x4

w3 = 1 − x1 .

Here’s a demo of cycling (ignoring the last constraint)...

6

⇓ Enter: x1, Leave: w1

⇓ Enter: x2, Leave: w2

⇓ Enter: x3, Leave: x1

⇓ Enter: x4, Leave: x2

7

⇓ Enter: w1, Leave: x3

⇓ Enter: w2, Leave: x4

Cycling is rare for small problems! A program that generates random 2× 4 fully degenerateproblems was run more than one billion times and did not find one example!

However, for larger problems with lots of zeros, cycling is common and can be a real problem.

8

Algebra of a Pivot

b a

d c

pivot−→

−ba

1

a

d− bc

a

c

a

9

Python Code – Create Random Problem

m = 2n = 2

x0 = 4*random.rand()y0 = 4*random.rand()r = 5theta = (3.*random.rand(m)-1.)*pi/2.x1 = x0 + r*cos(theta)y1 = y0 + r*sin(theta)A = zeros(m*n).reshape(m,n)A[:,0] = x1-x0A[:,1] = y1-y0b = (x1-x0)*x1 + (y1-y0)*y1A = ceil(2*A)b = ceil(2*b-0.5)b = matrix(b)b = b.Tc = matrix(ceil(9*random.rand(n,1)))

19

Python Code – Algorithm

while ( (max(c) > eps) ):

print(iter)

col = argmax(c) # find entering variableAcol = A[:,col].reshape(m,1) # the associate entering columntmp = -Acol/b # vector of ratiosrow = argmax(tmp) # the leaving variableif (tmp[row] < eps):

print('unbounded')break

j = nonbasics[col] # the entering var's subscripti = basics[row] # the leaving var's subscript

# update A matrix. See section 5.4.Arow = A[row,:] # the row in A of the entering variablea = A[row,col] # the pivot elementA = A - Acol*Arow/a # the out-of-row/out-of-col update formulaA[row,:] = -Arow/a # update formula for the rowA[:,col] = Acol.reshape(1,m)/a # update formula for the colA[row,col] = 1/a # update formula for the pivot element

# update the right-hand sidebrow = b[row,0]b = b - brow*Acol/ab[row] = -brow/a

# update the objective functionccol = c[col,0]c = c - ccol*(Arow.reshape(n,1))/ac[col] = ccol/a

# swapping variables x_j and x_i position in the dictionarybasics[row] = jnonbasics[col] = iiter = iter+1

21

AMPL Codeparam m := 2;

param n := 4;

param c {1..n}; param A {1..m, 1..n};param nonbasics {1..n}; param basics {1..m};param row; param col;param ii; param jj;param Arow {1..n}; param Acol {1..m};param cj; param bi;param a; param ccol;param iter;

for {k in 1..1000000000} {let {i in 1..m, j in 1..n} A[i,j] := Normal01();let {j in 1..n} c[j] := Normal01();let {j in 1..n} nonbasics[j] := j;let {i in 1..m} basics[i] := n+i;display k;

let iter := 1;repeat while (max {j in 1..n} c[j] > 0) {

let cj := 0;for {j in 1..n} {

if (c[j] > cj) then {let col := j;let cj := c[j];

}}let jj := nonbasics[col];

let bi := m+n+1;for {i in 1..m: A[i,jj] < -1e-8} {

if (basics[i] < bi) then {let bi := basics[i];let row := i;

}}if bi > m+n then {break;} # unbounded polytopelet ii := basics[row];

let {j in 1..n} Arow[j] := A[row,j];

let {i in 1..m} Acol[i] := A[i,col];let a := A[row,col];

let {i in 1..m, j in 1..n}A[i,j] := A[i,j] - Acol[i]*Arow[j]/a;

let {j in 1..n} A[row,j] := -Arow[j]/a;let {i in 1..m} A[i,col] := Acol[i]/a;let A[row,col] := 1/a;

let ccol := c[col];let {j in 1..n} c[j] := c[j] - ccol*Arow[j]/a;let c[col] := ccol/a;

let basics[row] := jj;let nonbasics[col] := ii;

if iter > 15 then {display "found a cycling example";break;

}

let iter := iter+1;}

}

23

Perturbation Method

Whenever a vanishing “rhs” appears perturb it.If there are lots of them, say k, perturb them all.Make the perturbations at different scales:

other data� ε1 � ε2 � · · · � εk > 0.

An Example.

Entering variable: x2

Leaving variable: w2

24

Perturbation Method—Example Con’t.

Recall current dictionary:

Entering variable: x1

Leaving variable: w3

DONE!

25

Perturbation Method Applied to Cycling Example

⇓ x1 enters, w2 leaves

⇓ x2 enters, w1 leaves

⇓ x3 enters, x2 leaves

⇓ w2 enters, problem unbounded!

Note: objective function increases with every pivot: 0 < 2ε2 < 2ε1 <83ε1 − 2

3ε2 26

Other Pivot Rules

Smallest Index Rule.

Choose the variable with the smallest index (the x variables are assumed to be“before” the w variables).

Note: Also known as Bland’s rule.

No cycling (it’s been proved).

Random Selection Rule.

Select at random from the set of possibilities.

No infinite cycles.

Greatest Increase Rule.

Pick the entering/leaving pair so as to maximize the increase of the objective functionover all other possibilities.

Note: Too much computation.

Needs a tie-breaking rule.

27

Theoretical Results

Cycling Theorem. If the simplex method fails to terminate, then it must cycle.

Why?

Fundamental Theorem of Linear Programming. For an arbitrary linear program in standardform, the following statements are true:

1. If there is no optimal solution, then the problem is either infeasible or unbounded.

2. If a feasible solution exists, then a basic feasible solution exists.

3. If an optimal solution exists, then a basic optimal solution exists.

28

Geometry

maximize x1 + 2x2 + 3x3subject to x1 + 2x3 ≤ 3

x2 + 2x3 ≤ 2x1, x2, x3 ≥ 0

1 2x2

x3

x1

x1+2x3=3

x2+2x3=2x2=0

maximize x1 + 2x2 + 3x3subject to x1 + 2x3 ≤ 2

x2 + 2x3 ≤ 2x1, x2, x3 ≥ 0

1x2

x3

x1

x1+2x3=2

x2=0

x2+2x3=2

29