TheRevisedSimplexMethoderodri/webpage/cps/theory/lp/...TableauSimplexMethod 2/30 The simplex method...

The Revised Simplex Method

Combinatorial Problem Solving (CPS)

Javier Larrosa Albert Oliveras Enric Rodrıguez-Carbonell

March 29, 2019

Tableau Simplex Method

2 / 30

■ The simplex method we have seen so far is calledtableau simplex method

■ Some observations:

◆ At each iteration we update the full tableau

xB = B−1b−B−1RxR

for the new basis

◆ But ...

■ For pricing only one negative reduced cost is needed

■ For ratio test, only

◆ the column of the chosen non-basic variable in the tableau, and◆ the current basic solution

are needed

Revised Simplex Method

3 / 30

■ Idea: do not keep a representation of the full tableau, only B−1

■ Advantages over the tableau version:

◆ Time and space are saved

◆ Errors due to floating-point arithmetic are easier to control

Revised Simplex Method

3 / 30

■ Idea: do not keep a representation of the full tableau, only B−1

■ Advantages over the tableau version:

◆ Time and space are saved

◆ Errors due to floating-point arithmetic are easier to control

■ We will revise the algorithm and express it in terms of B−1

■ First for LP’s of the form

min z = cTx

Ax = bx ≥ 0

Basic Solution

4 / 30

■ Let us see how the basic solution is expressed in terms of B−1

■ For any basis B,values of basic variables can be expressed in terms of non-basic variables:

BxB +RxR = bBxB = b−RxRxB = B−1b−B−1RxR

Basic Solution

4 / 30

■ Let us see how the basic solution is expressed in terms of B−1

■ For any basis B,values of basic variables can be expressed in terms of non-basic variables:

BxB +RxR = bBxB = b−RxRxB = B−1b−B−1RxR

■ By definition, the basic solution corresponds toassigning null values to all non-basic variables: xR = 0Then xB = B−1b

■ We will denote the basic solution (projected on basic variables) withβ := B−1b

Optimality Condition

5 / 30

■ Let us see now how to express the reduced costs in terms of B−1

■ Recall the equation of basic variables in terms of non-basic variables:


■ Cost function can be split: cTx = cTBxB + cTRxR, where

cTB are the costs of basic variables,

cTR are the costs of non-basic variables


5 / 30

■ Let us see now how to express the reduced costs in terms of B−1

■ Recall the equation of basic variables in terms of non-basic variables:


■ Cost function can be split: cTx = cTBxB + cTRxR, where

cTB are the costs of basic variables,

cTR are the costs of non-basic variables

■ We can express the cost function in terms of non-basic variables:

cTx =cTBxB + cTRxR =

cTB(B−1b−B−1RxR) + cTRxR =

cTBB−1b− cTBB

−1RxR + cTRxR =

cTBB−1b+ (cTR − cTBB

−1R)xR


6 / 30

■ We found that cTx = cTBB−1b+ (cTR − cTBB

−1R)xR

■ The part that depends on non-basic variables is (cTR − cTBB−1R)xR

■ Let aj be the column in A corresponding to variable xj ∈ xR.

The coefficient of xj in (cTR − cTBB−1R)xR is cj − cTBB

−1aj

We will denote the reduced cost of xj with dj := cj − cTBB−1aj

■ Optimality condition: dj ≥ 0 for all j ∈ R

Cost at Basic Solution

7 / 30

■ Let’s see how to express the value of the cost function at the basic solution

■ We found that cTx = cTBB−1b+ dTRxR, where dj = cj − cTBB

−1aj

■ We will denote the value of the cost function at the basic solution with z

■ Taking xR = 0 in the above equation: z := cTBB−1b

Cost at Basic Solution

7 / 30

■ Let’s see how to express the value of the cost function at the basic solution

■ We found that cTx = cTBB−1b+ dTRxR, where dj = cj − cTBB

−1aj

■ We will denote the value of the cost function at the basic solution with z

■ Taking xR = 0 in the above equation: z := cTBB−1b

■ To avoid repeating computations:

Let us define the simplex multiplier as π := (BT )−1cB

Then πT = cTBB−1

So dj = cj − πTaj and z = πT b

Note z = cTBβ holds too

Improving a Non-Optimal Solution

8 / 30

■ Let us assume the optimality condition is violated

■ Let xq be a non-basic variable such that its reduced cost is dq < 0

■ Current value of xq is 0.We can improve by increasing only this valuewhile non-negativity constraints of basic variables are satisfied.


8 / 30




■ Let t ≥ 0 be the new value for xq.

Let xB(t) / xR(t) be the values of basic / non-basic variables in terms of t

Note that xq(t) = t, and xp(t) = 0 if p ∈ R and p 6= q


8 / 30




■ Let t ≥ 0 be the new value for xq.

Let xB(t) / xR(t) be the values of basic / non-basic variables in terms of t

Note that xq(t) = t, and xp(t) = 0 if p ∈ R and p 6= q

So xB(t) = B−1b−B−1RxR(t) = B−1b−B−1aqt = β − tαq

where β = B−1b is the basic solutionand we denote the column in the tableau of xq as αq := B−1aq


9 / 30

■ How much do we improve?

How does the objective value change as a function of t?

z(t) =cTx(t) =

cTBxB(t) + cTRxR(t) =

cTBxB(t) + cqt =

cTBβ − tcTBαq + cqt =

cTBβ − tcTBB−1aq + cqt =

z + tdq

■ As expected, the improvement in cost is ∆z = z(t)− z = tdq


10 / 30

■ Recall that xB(t) = β − tαq

■ How can we satisfy the non-negativity constraints of basic variables?


10 / 30



■ Basic variables have indices B = (k1, ..., km)

■ Let i ∈ {1, ...,m}. The i-th basic variable is xki■ Value of xki as a function of t is the i-th component of xB(t): βi − tαi

q,where βi is the i-th component of β and αi

q is the i-th component of αq


10 / 30







■ We need βi − tαiq ≥ 0

◆ If αiq ≤ 0 the constraint is satisfied for all t ≥ 0

◆ If αiq > 0 we need βi

αiq≥ t


10 / 30







■ We need βi − tαiq ≥ 0

◆ If αiq ≤ 0 the constraint is satisfied for all t ≥ 0

◆ If αiq > 0 we need βi

αiq≥ t

■ The best improvement is achieved with the strongest of the upper bounds:

θ := min{ βi

αiq| αi

q > 0}

■ We say the p-th basic variable xkp is blocking or tight when θ =βp

αpq.

Then αpq is the pivot


11 / 30

1. If θ = +∞(there is no upper bound, i.e., no i such that 1 ≤ i ≤ m and αi

q > 0):

Value of objective function can be decreased infinitely.LP is unbounded.


11 / 30

1. If θ = +∞(there is no upper bound, i.e., no i such that 1 ≤ i ≤ m and αi

q > 0):

Value of objective function can be decreased infinitely.LP is unbounded.

2. If θ < +∞ and the p-th basic variable xkp is blocking:

When setting xq = θ, the non-negativity of basic variables is respected

In particular the value of xkp , i.e. the p-th component of xB(t), isβp − θαp

q = 0

We can make a basis change:xq enters the basis and xkp leaves, where B = (k1, ..., km)

Update

12 / 30

■ New basic indices: B = (k1, . . . , kp−1, q, kp+1 . . . , km)

Before the p-th basic variable was xkp , now it is xq

■ New basic solution: βp = θ, βi = βi − θαiq if i 6= p

Note that before the p-th component of β corresponded to xkp , now to xq

■ New basis: B = B + (aq − akp)eTp

where eTp = (0, ..., 0,

p︷︸︸︷

1 , 0, ..., 0)︸︷︷︸

m

The p-th column of the basis (which was akp) is replaced by aq.

■ New basic costs: cTB = cTB + (cq − ckp)eTp

The p-th basic cost (which was ckp) is replaced by cq.

■ New objective value: z = z + θdq

Algorithmic Description

13 / 30

1. Initialization: Find an initial feasible basis BCompute B−1, β = B−1b, z = cTBβ

2. Pricing: Compute πT = cTBB−1 and dj = cj − πTaj .

If for all j ∈ R, dj ≥ 0 then return OPTIMALElse let q be such that dq < 0. Compute αq = B−1aq

3. Ratio test: Compute I = {i | 1 ≤ i ≤ m,αiq > 0}.

If I = ∅ then return UNBOUNDEDElse compute θ = mini∈I(

βi

αiq) and p such that θ =

βp

αpq

4. Update:

B = B − {kp} ∪ {q} B = B + (aq − akp)eTp

βp = θ, βi = βi − θαiq if i 6= p z = z + θdq

Go to 2.

Updating Matrix Inverse

14 / 30

■ Actually what we really care about is B−1, not B

We need it for computing π = cTBB−1 and αq = B−1aq at each step

(and also β = B−1b in the initialization)

■ Recomputing B−1 at each iteration is too expensive(e.g. O(m3) arithmetic operations with Gaussian elimination!)

■ Next slides: a more efficient way of computing B−1 using B−1


15 / 30

■ Let us make a diversion into linear algebra

■ Let b1, ..., bm be the columns of an invertible matrix B

Let a, α be such that a = Bα =∑m

i=1αibi

Let p be such that 1 ≤ p ≤ m

■ Ba = (b1, . . . , bp−1, a, bp+1, . . . , bm). Want to compute B−1a


15 / 30

■ Let us make a diversion into linear algebra

■ Let b1, ..., bm be the columns of an invertible matrix B

Let a, α be such that a = Bα =∑m

i=1αibi

Let p be such that 1 ≤ p ≤ m

■ Ba = (b1, . . . , bp−1, a, bp+1, . . . , bm). Want to compute B−1a

■ Note αp 6= 0 as otherwise rank(Ba) < m.

Then a = αpbp +∑

i 6=p αibi ⇒ bp =(

1

αp

)

a+∑

i 6=p

(−αi

αp

)

bi

■ Let ηT =((

−α1

αp

)

, . . . ,(−αp−1

αp

)

, 1

αp,(−αp+1

αp

)

, . . . ,(−αm

αp

))T

.

Then bp = Baη

■ Let E = (e1, . . . , ep−1, η, ep+1, . . . , em).

Then BaE = B =⇒ E−1B−1a = B−1 =⇒ B−1

a = EB−1


16 / 30

■ Application to the simplex algorithm:a = aq, α = αq, where xq is entering variable

Thus to update the inverse we can reuse already computed data!

■ Using this update: B−1 is not actually represented as a square table,but as follows

■ Assume initial basis is B0 (e.g., unit matrix I).Then at the k-th iteration of the simplex algorithmthe inverse matrix is B−1 = EkEk−1 · · ·E2E1B

−1

0,

where Ei is the E matrix of the i-th iteration

■ Each E matrix can be stored compactly (vector η + column index p)

■ We can represent B−1 as the list EkEk−1 · · ·E2E1, B−1

0:

Product Form of the Inverse (PFI)

■ When the list is long we reset: the inverse is computed (reinversion)

■ Other ways of representing B−1: LU factoritzation

Bounded Variables

17 / 30

■ Now we want to revise the simplex algorithm for LP’s of the form

min cTxAx = bℓ ≤ x ≤ u (−∞ ≤ ℓk ≤ uk ≤ +∞)

■ In practice, internally variables are translated so thatℓ, u = −∞ or ℓ = 0 or u = 0 to save arithmetic operations

■ Variable xk is lower bounded if ℓk > −∞

■ Variable xk is upper bounded if uk < +∞

■ Variable xk is free if ℓk = −∞ and uk = +∞

Basic Solution

18 / 30

■ For any basis B, recall thatvalues of basic variables can be expressed in terms of non-basic variables:


■ The values of a non-basic variable xj can be:

If lower bounded: ℓk (we say it is at lower bound; denoted by xj ∈ L)If upper bounded: uk (we say it is at upper bound; denoted by xj ∈ U)If free: 0 (we say it is at zero level; denoted by xj ∈ Z)

■ Basic solution:

β := B−1b−∑

j∈L

B−1ajℓj −∑

j∈U

B−1ajuj


19 / 30

■ Recall the cost function in terms of non-basic variables:

cTx = cTBB−1b+ dRxR

where dj = cj − cTBB−1aj for all variable xj

■ If xj ∈ L: cannot improve if dj ≥ 0

■ If xj ∈ U : cannot improve if dj ≤ 0

■ If xj ∈ Z : cannot improve if dj = 0

■ Optimality condition: no improving non-basic variable

Objective Function

20 / 30

■ Recall the cost function in terms of non-basic variables:

cTx = cTBB−1b+ dRxR

where dj = cj − cTBB−1aj for all variable xj

■ Value z of cost function at current basic solution:

z = cTBB−1b+

∑

j∈L

djℓj +∑

j∈U

djuj


21 / 30

■ Let xq be a non-basic variablethat can improve objective value by increasing its value.

This can happen when

◆ xq is lower bounded and xq ∈ L; or

◆ xq is free (so xq ∈ Z)

■ Since increasing xq can improve objective value: dq < 0

■ Let t ≥ 0 be difference of new value xq wrt old value

xB(t)= B−1b−B−1RxR(t)= B−1b− tB−1aq −

∑

j∈LB−1ajℓj −∑

j∈U B−1ajuj= β − tαq

where β = B−1b−∑

j∈L

B−1ajℓj −∑

j∈U

B−1ajuj , αq = B−1aq


22 / 30

■ How does the objective value change as a function of t?

z(t) =cTx(t) =

cTBxB(t) + cTRxR(t) =

cTBxB(t) + tcq +∑

j∈L

cjℓj +∑

j∈U

cjuj =

cTBB−1b+

∑

j∈L

(cj − cTBB−1aj)ℓj +

∑

j∈U

(cj − cTBB−1aj)uj + tcq − tcTBαq =

cTBB−1b+

∑

j∈L

djℓj +∑

j∈U

djuj + tcq − tcTBαq =

z + tcq − tcTBαq =

z + tcq − tcTBB−1aq =

z + tdq

■ Hence the improvement in cost is ∆z = z(t)− z = tdq


23 / 30



q

■ Let λi := ℓki , µi := uki

We need λi ≤ βi − tαiq ≤ µi

◆ If αiq > 0: βi − tαi

q ≥ λi ⇒ βi−λi

αiq

≥ t

◆ If αiq < 0: βi − tαi

q ≤ µi ⇒ βi−µi

αiq

≥ t

■ But we need xq(t) ≤ uq too!

■ Best improvement achieved with

θ := min( uq − ℓq, min{βi−λi

αiq

| αiq > 0}, min{βi−µi

αiq

| αiq < 0})

■ If θ = +∞ we have unboundedness

■ Else if θ = uq − ℓq we have a bound flip: no pivot needed!


24 / 30

θ := min( uq − ℓq, min{βi−λi

αiq

| αiq > 0}, min{βi−µi

αiq

| αiq < 0})

■ Assume θ 6= +∞, θ 6= uq − ℓq.

■ Thus variable xq enters the basis and variable xkp leaves

■ If θ = min{βi−λi

αiq

| αiq > 0} then xkp leaves the basis at lower bound

■ If θ = min{βi−µi

αiq

| αiq < 0} then xkp leaves the basis at upper bound


25 / 30

■ Let xq be a non-basic variablethat can improve objective value by decreasing its value.

This can happen when

◆ xq is upper bounded and xq ∈ U ; or

◆ xq is free (so xq ∈ Z)

■ Since decreasing xq can improve objective value: dq > 0

■ Let t ≤ 0 be the difference of new value xq wrt old value

■ Again xB(t) = β − tαq

■ Again the improvement in cost is ∆z = z(t)− z = tdq


26 / 30



q

■ Let λi := ℓki , µi := uki

We need λi ≤ βi − tαiq ≤ µi

◆ If αiq > 0: βi − tαi

q ≤ µi ⇒ βi−µi

αiq

≤ t

◆ If αiq < 0: βi − tαi

q ≥ λi ⇒ βi−λi

αiq

≤ t

■ But we need ℓq ≤ xq(t) too!

■ Best improvement achieved with

θ := max( ℓq − uq, max{βi−λi

αiq

| αiq < 0}, max{βi−µi

αiq

| αiq > 0})

■ If θ = −∞ we have unboundedness

■ Else if θ = ℓq − uq we have a bound flip: no pivot needed!


27 / 30

θ := max( ℓq − uq, max{βi−λi

αiq

| αiq < 0}, max{βi−µi

αiq

| αiq > 0})

■ Assume θ 6= −∞, θ 6= ℓq − uq.

■ Thus variable xq enters basis and variable xkp leaves

■ If θ = max{βi−λi

αiq

| αiq < 0}, xkp leaves basis at lower bound

■ If θ = max{βi−µi

αiq

|αiq > 0}, xkp leaves basis at upper bound

Update

28 / 30

■ New objective value: z = z + θdq

■ If bound flip

◆ Flip status of xq (i.e., xq ∈ L ⇔ xq ∈ U)

◆ New basic solution: β = β − θαq

■ Else

◆ New basic indices: B = (k1, . . . , kp−1, q, kp+1 . . . , km)

◆ New basic solution: βp = xq + θ, βi = βi − θαiq if i 6= p

◆ New basis inverse: B−1 = EB−1

◆ If entering variable comes from lower boundL = L − {xq} else U = U − {xq}

◆ If leaving variable leaves to lower boundL = L ∪ {xkp} else U = U ∪ {xkp}

Tableau vs. Revised Simplex

29 / 30

■ Time is saved:

✗ Tableau: all dk, all αk are computed

✓ Revised: no. of non-basic variables xk for whichdk, αk are computed can be adjusted

■ Space is saved:

✗ Tableau: even if A sparse, tableau tends to get filled

✓ Revised: sparsity of A can be exploited for storage,and pivots can be chosen to represent B−1 compactly

■ Better numerical behaviour:

✗ Tableau: errors due to floating-point arithmetic accumulate at eachpivoting step

✓ Revised: reinversion (PFI representation of B−1) orrefactorization (LU representation of B−1) can be used for resetting

Original vs. Bounds Simplex

30 / 30

■ Time is saved:

✗ Original: no special treatment of bounds

✓ Bounds: bound flips are much cheaper thanpivoting steps in simplex iterations (basis does not change)

■ Space is saved:

✗ Original: each bound constraint becomes a row

✓ Bounds: bounds are stored cheaply in arrays

TheRevisedSimplexMethoderodri/webpage/cps/theory/lp/...TableauSimplexMethod 2/30 The simplex method...

Documents

Transcript of TheRevisedSimplexMethoderodri/webpage/cps/theory/lp/...TableauSimplexMethod 2/30 The simplex method...