Simplex Method for LP: the Basic Case (week 3-4) · We ﬁrst solve LP in standard forms and show...

Simplex Method for LP: the Basic Case (week 3-4)

Xiaoxi Li

EMS & IAS, Wuhan University

September 22-29, 2016

Operations Research (Li, X.) Simplex Method for LP: the Basic Case (week 3-4) September 22-29, 2016 1 / 42

Organization of this lecture

Contents:

LP in Standard FormSome Geometry on Linear SystemsBasic Properties of LP: Motivating the Simplex MethodThe Essence of Simplex Method: the Geometric ViewSimplex Method: the Algebraic FormSimplex Method: the Tabular Form


LP in Standard Form

A class of LP problems concern the allocation of limited resources intocompeting activities so as to maximize the total profit, described by:

z = value of overall measure of performancexj = level of activity j

cj = unit contribution to Z of activity j

bi = total capacity of resource i available for allocationaij = amount of resource i consumed by each unit of activity j


Formulate the mathematical model for this class of LP problems:

Maximize z = c1x1 + c2x2 + · · ·+ cnxn

s.t.

a11x1 +a12x2 + · · ·+a1nxn ≤ b1 1a21x1 +a22x2 + · · ·+a2nxn ≤ b2 2

...am1x1 +anmx2 + · · ·+amnxn ≤ bm mx1 ≥ 0, ...,xn ≥ 0 +

(0.1)

Two types of constraints:" 1 , ..., m ": the functional constraints;" + ": the nonnegatives constraints.

Any (x1, ...xn) satisfying all constraints " 1 ,..., m and + " in (0.1)is a feasible solution, and the feasible region is set of allfeasible solutions.


LP in Standard Form

The above LP problem is said to be in the standard form, which isdefined by the following three conditions:

1 The objective z is maximized ;2 The LHS and the RHS entries of the functional constraints are

connected with "≤" and the RHS entries are positive constants;3 All x1, ..., xn are nonnegative.

Other forms include LP problems where:z be minimized,"bj < 0",the LHS and RHS are connected by "≥" or "=",or some xj is not required to be nonnegative.

We first solve LP in standard forms and show later that all other formscan be equivalently transformed to standard form.


Some geometry on linear systems

DefinitionIn Rn, a hyperplane is any set of the form:

H(α,β ) = {x ∈ Rn|αTx = β},

where α ∈ Rn \{0} and β ∈ R.

Hyperplanes in R are points, hyperplanes in R2 are lines,hyperplanes in R3 are planes, and hyperplanes in Rn aretranslates of (n−1) dimensional subspaces.Every hyperplane divides the space in half. This defines twoclosed half-spaces, which are H+(α,β ) = {x ∈ Rn|αTx≥ β} andH−(α,β ) = {x ∈ Rn|αTx≤ β}. H(α,β ) is called the supportinghyperplane of H+(α,β ) and H−(α,β ).


DefinitionA convex polyhedron the intersection of a finite number ofhalf-spaces. It is a convex polytope when it is bounded.

Another definition for a convex polytope is the convex hull of afinite set of points. These extreme points are called its vertices.Any constraint in Equation (0.1) defines a half-space, and a LP isa problem of maximizing or minimizing a linear objective functionover a convex polyhedron.A corner-point/ CP solution to Equation (0.1) is defined to be theintersection of n (linear) independent supporting hyperplanes.It is a corner-point feasible / CPF solution if it is feasible, andeach CPF solution corresponds to an extreme point.Two CP solutions are adjacent to each other if they share (n−1)common supporting hyperplanes.


Basic properties of LP: motivating Simplex Method

The main idea of the simplex method lies in the following threeproperties. Focus on LP with a nonempty and bounded feasible region.

Property 1. Optimal solutions are obtained within the set of CPFsolutions (extreme points).

Property 2. There is a finite number of CPF solutions.

Property 3. A CPF solution is optimal if it is better than all its adjacentCPF solutions.


Implications.

Property 1 & Property 2: an optimal solution can be found byenumeration of all CPF solutions.

Property 3: no need to compare one to all other extreme points;rather, we compare each extreme point only with its adjacentones, and a local optimality will imply the global optimality.

This underlines the main idea how the simplex method works:iteratively, check one CPF solution’s local optimality, and move to anadjacent one if not optimal.


The essence of Simplex Method: a geometric view

Solve the example with graphical analysis.


Simplex Method: a geometric view

List all the CPF solutions and its adjacencies for each solution.

CPF Adjacent Active Supporting Hyperplanes(0, 0) (0, 6) and (4, 0) x2 = 0 and x1 = 0(0, 6) (2, 6) and (0, 0) x1 = 0 and 2x2 = 12(2, 6) (4, 3) and (0, 6) 2x2 = 12 and 3x1 +2x2 = 18(4, 3) (4, 0) and (2, 6) 3x1 +2x2 = 18 and x1 = 4(4, 0) (0, 0) and (4, 3) x1 = 4 and x2 = 0


Initialization. Choose (0,0) as the initial CPF solution.

Optimality Test 1. Not optimal.(0,6)−→ z = 30; (0,0)−→ z = 0; (4,0)−→ z = 12.

Iteration 1. Move to a better adjacent CPF solution.

z = 3x1 +5x2: x2 leads to a faster increasing rate for z than x1 does.increase x2 along the line "x1 = 0" until reaching the boundary;arrive at another CPF solution adjacent to it, (0,6), computed as theintersection of "x1 = 0" and "2x2 = 12".


Note. In this example, "increasing x2" coincides with "moving along theline x1 = 0 upward" due to the fact that:

(0,0) is the intersection of the hyperplanes "x1 = 0" and "x2 = 0"and we choose to increase x2 thus to leave "x2 = 0". We are still onthe remaining hyperplane "x1 = 0" (a single one since n = 2).The hyperplane "x1 = 0" is a line in R2 so there is only onedirection to choose since we are leaving from an extreme point ofthe convex polytope.In a high-dimension space, the intersection of the remaining(n−1) hyperplanes is still a line, so there is only one direction tomove by leaving a hyperplane.


Optimality Test 2. Not optimal.(0,0)−→ z = 0; (0,6)−→ z = 30; (2,6)−→ z = 36.

Iteration 2. Move to a better adjacent CPF solution, and seen fromthe graph, (2,6) is the only choice (along the line "2x2 = 12").

Optimality Test 3. Optimal.(0,6)−→ z = 30; (2,6)−→ z = 36; (4,3)−→ z = 27.

Conclusion: the optimal solution is (x∗1,x∗2) = (3,4) and the optimal

value is z = 27.


The solution path: (0,0)−→ (0,6)−→ (2,6).


Simplex Method: an overview

Questions from solving the example with a geometric view:

In high-dimension spaces, a geometric view is missing: algebraicoperations are needed to identify the (adjacent) CPF solutions.To do the optimality test, all the adjacent CPF solutions areneeded. This is still too costly and is it always needed?Recall the identification of the "direction for moving" in R2: eachCPF solution has only two adjacent points, so

at initial point (0,0), use the equation "z = 3x1 +5x2" to chooseamong two directions;at any other point, "not optimal" implies moving to the next point(the previous is already known to be worse).

What to do when many adjacent CPF solutions exist in Rn?

The above problems are solved within the Simplex Method.



Q.1 To identify the (adjacent) CPF solutions.One CPF solution is identified by a group n independentsupporting hyperplanes (binding constraints). Two adjacent CPFsolutions share (n−1) common supporting hyperplanes.Switching CPF solutions (adjacent)⇐⇒ replacing one hyperplanewithin the group with some one outsideIntroduce a slack variable si ≥ 0 to each functional constraint asan indicator variable for the constraint to bind (si = 0) or not(si > 0). xj is already an indicator variable for xj ≥ 0.The group of n "active" hyperplanes⇐⇒ n indicator variables.This group of indicator variables are equal to zero and are calledthe nonbasic variables (opposed to the basic variables).



Q.2 Optimality test without knowing the adjacent CPF solutions.Q.3 Identify the "direction of moving" in Rn.

Let {x′1, ...,x′n} be the nonbasic variables for the current CPFsolution. The above two problems will be solved if we can rewrite

z = c′1x′1 + · · ·+ c′nx′n for some c′1, ...,c′n.

The objective function contains no basic variables, and c′irepresents the changing rate of z in x′i.

This CPF solution is optimal if c′j ≤ 0,∀j ∈ {1, ...,n}.Otherwise, let c′j be the most positive, and pick x′j as the indicatorvariable to be dropped. Move along the edge determined by"{x′i = 0|i 6= j}" until reaching the boundary.


Simplex Method: the algebraic form

In a LP problem in standard form, add one "slack variable sj ≥ 0" toeach functional constraint " j " to obtain the following LP problem incanonical form:

Maximize z = c1x1 + c2x2 + · · ·+ cnxn

s.t.

a11x1 +a12x2 + · · ·+a1nxn + s1 = b1 1a21x1 +a22x2 + · · ·+a2nxn + s2 = b2 2

...am1x1 +am2x2 + · · ·+amnxn + sm = bm mx1 ≥ 0, ...,xn ≥ 0,s1 ≥ 0, ...,sm ≥ 0 +

(0.2)

Fact: LP in standard form and the corresponding LP in canonical formare equivalent.



A solution to the LP in canonical form is called an augmentedsolution (decision variables + slack variables). It is anaugmented feasible solution if satisfies all the constraints.Call an augmented corner-point solution a basic solution(algebraic meaning). A CPF solution is a basic feasible/BFsolution.A basic solution is composed of

n nonbasic variables: equal to zero, corresponding to the nsupporting hyperplanes defining the corner-point solution;m basic variables: the others, forming a basis.

Two BF solutions are adjacent if their corresponding CPFsolutions are adjacent.Fact: Two BF solutions are adjacent if and only if all but one oftheir nonbasic variables are the same.



The example in canonical form:

Maximize z = 3x1 +5x2

s.t.

x1 + s1 = 4 1

2x2 + s2 = 12 23x1 +2x2 + s3 = 18 3x1,x2,s1,s2,s3 ≥ 0 +

(0.3)



Initialization

The origin as the initial point: {x1,x2} are chosen as the nonbasicvariables, thus equal to zero. The basic variables’ values are read as:

s1 = 4 1s2 = 12 2

s3 = 18 3

s1 = 4≥ 0,s2 ≥ 0 and s3 ≥ 0, so (0,0,4,12,8) is a BF solution.

The above system is in the proper form from Gaussian elimination:each equations contains only one basic variable with its coefficientequal to one.



The principal is that the objective z increases in one nonbasicvariable with the fastest rate, and increases as much as possible.The objective function is z = 3x1 +5x2, containing only nonbasicvariables, thus:

the increasing rate in x1 for z is 3, positive;the increasing rate in x2 for z is 5, positive.

Optimality Test 1

(0,0,4,18,18) is not optimal since increasing in either nonbasic variableleads to improvement of z.



Iteration 1

Step 1 (determining the direction of movement). The direction is toincrease x2 because 5 > 3.x2 is then called the entering variable, which is to enter into thebasis to become a basic variable.Step 2 (where to stop). Increase x2 as far as possible. Let x1 = 0,the following minimum ratio test defines the boundary:

Constraint 1 : s1 = 4≥ 0 =⇒ no upper bound on x2;

Constraint 2 : s2 = 12−2x2 ≥ 0 =⇒ x2 ≤ 122 = 6 > 0 ←−

minimum;Constraint 3 : s3 = 18−2x2 ≥ 0 =⇒ x2 ≤ 18

2 = 9 > 0.

When x2 = 6, s2 = 0, the constraint 2 binds, i.e. "x1 +2x2 = 12". s2is the leaving variable.



Step 3 (solving for the new BF solution)."x2 = 0−→ s2 = 0" moves us from the initial BF solution to the newBF solution:

Initial BF Solutions New BF SolutionsNonbasic variables x1 = 0, x2 = 0 x1 = 0, s2 = 0Basic variables s1 = 4, s2 = 12, s3 = 18 s1 =?,x2 = 6,s3 =?



Write z−3x1−5x2 = 0 (x2,s1,s3 are basic variables):(0) z−3x1−5x2 = 0(1) x1 + s1 = 0(2) 2x2 + s2 = 12(3) 3x1 +2x2 + s3 = 18

Convert the above system into a proper form from Gaussianelimination for the new basic variables s1,x2 and s3 for the aim of:

solving x2,s1,s3;making easy the next-step optimality test and iteration.


Elementary algebraic operations on the above system:1. Multiply (or divide) an equation by a nonzero constant;2. Add (or subtract) a multiple of one equation to (or from) another.

Column coefficient for the entering basic variable x2: change from(5,0,2,2) to (0,0,1,0) (coefficient of leaving variable s2).1. Eq.(2′) = Eq.(2)÷2;2. Eq.(0′) = Eq.(0)+5Eq.(2′);3. Eq.(3′) = Eq.(3)−2Eq.(2′).


The resulting new system becomes:(0) z−3x1 + 5

2 s2 = 30(1) x1 + s1 = 4(2) x2 + 1

2 s2 = 6(3) 3x1 − s2 + s3 = 6

x1 and s2 are the new nonbasic variables, thus equal 0. The new BFsolution is (x1,x2,s1,s2,s3) = (0,6,4,0,6), yielding z = 30.

Optimality Test 2 The new objective function writes as

z = 30+3x1−52

s2.

Increasing x1 will yield a higher value of z, thus not optimal.


Iteration 2Step 1 (determining the direction of movement). To increase x1because 3 > 0. x1 is the entering variable.Step 2 (where to stop). To increase x1 as far as possible. Lets2 = 0. The boundary is defined by minimum ratio test:

Constraint 1 : s1 = 4− x1 ≥ 0 =⇒ x1 ≤ 41 = 4 > 0;

Constraint 2 : x2 = 6≥ 0 =⇒ no upper bound on x1;

Constraint 3 : s3 = 6−3x1 ≥ 0 =⇒ x1 ≤ 63 = 2 > 0. ←− minimum.

When x1 = 2, s3 = 0, the constraint 3 binds, i.e. "3x1 +2x2 = 18".s3 is the leaving variable.

Nonbasic variables: {x1,s2} −→ {s3,s2};basic variables: {s3,x2,s1} −→ {x1,x2,s1}


Step 3 (Solving for the new BF solution).Column coefficients for x1 in the new system be (0,0,0,1).1. Eq. (3′) = Eq. (3)÷3;2. Eq. (0′) = Eq. (0)+3Eq. (3′);3. Eq. (1′) = Eq. (1)−Eq. (3′).

The resulting new system becomes:(0) z + 3

2 s2 + s3 = 36(1) s1 + 1

3 s1 − 13 s3 = 2

(2) x2 + 12 s2 = 6

(3) x1 − 13 s2 +

13 s3 = 2

x1 = 2,x2 = 6,s1 = 2,s2 = 0,s3 = 0, i.e. (2,6,2,0,0).



Optimality Test 3 The new objective function is

z = 36−3/2s2−s3.

Increasing neither s2 nor s3 will improve z. The solution is optimal.

Conclusion: the optimal decision variables are (x∗1,x∗2) = (2,6) and the

optimal value is z = 36.

Interpretations:s∗1 = 2: with (x∗1,x

∗2) = (2,6) the optimal solution, the first resource’s

capacity is not exhausted, and 2 units remained;s∗2 = 0: with (x∗1,x

∗2) = (2,6) the optimal solution, the second

resource’s capacity is exhausted;s∗3 = 0: with (x∗1,x

∗2) = (2,6) the optimal solution, the third resource’s

capacity is exhausted.


Simplex method: the tabular form

It is possible to put the algebraic operations of simplex method intabular form.Many computation details can be omitted and the conditions ateach step are given directly.


Initialization

Introduce slack variables si: LP in standard form −→ LP incanonical form.Nonbasic variables = decision variables {x1, ...,xn}; Basicvariables: slack variables {s1, ...,sm}.Nonbasic variables are set zero; basic variables’ values are readdirectly: equal to the RHS entries of functional constraints.The initial BF solution is (x1, ...xn,s1, ...,sm) = (0, ...,0,b1, ...,bm).Write the objective function in the form

z− c1x1− c2x2−·· ·− cnxn = 0,

and put the equation together with other constraint equations.


Complete the tabular form:correspond variables’ entries to coefficients in the constraintequations;on the left, put one column "basic variable" for {z,s1,s2,s3}, where zis regarded as an auxiliary basic variable.

Remark. The initial tableau is in proper form from Gaussianelimination (canonical form):

in Eq. (0): coefficients for basic variables are all zero;in Eq. (i): contains only one basic variable, with the coefficient 1.


Optimality Test 1

The current BF solution is optimal if and only if the coefficient in

Eq. (0) : ”z− c1x1− c2x2−·· ·− cnxn = 0”

for each nonbasic variable is nonnegative (−ci ≥ 0,∀i).

Stop if it is;Otherwise, proceed to an iteration to obtain a new BF solution,which includes the following three steps.

For the example: z = 3x1 +5x2⇐⇒ z−3x1−5x2 = 0. Not optimal so weproceed to an iteration to find a new BF solution.


Iteration

Step I. Identify the entering variable as the nonbasic variablehaving the most negative coefficient in Eq.(0).

The column of the entering variable is the pivot column.

For the example: −5 <−3 < 0, so x2 is the leaving variable.

Step II. Determine the leaving variable by applying the minimumration test.

Pick out the coefficients that are strictly positive;Divide each of these coefficients into the RHS entry of same row;Identify the row with the smallest positive ratio;The basic variable of the identified row is the leaving variable. Inthe "basic variable" column of the new tableau, replace the leavingvariable by the entering variable.


The row with the smallest positive ratio is the pivot row, and theintersection entry of the pivot row/column is the pivot number.

For the example:


Step III. Solve for the new BF solutions: use the elementary rowoperations to construct a new simplex tableau in proper formfrom Gaussian elimination: basically, the pivot column(including Eq. (0)

)will be (0, ...1, ...0), where the only "1"

coefficient appears in the entry of pivot number.For the example:


Read from the simple tableau:(x1,x2,s1,s2,s3) = (0,6,4,0,3) where {s1,x2,s3} are the basicvariables, and {x1,s2} are the nonbasic variables.The new objective (function) equation is

”Eq. (0) : z−3x1 +52

s2 = 0”.

Optimality Test 2 In Eq. (0), "−3 < 0" so it is not optimal. Proceed to asecond iteration.


Iteration 2x1 is the entering variable;6÷3 = 2 < 4 = 4÷1, so s3 is the leaving variable.


The new simplex tableau by the end of Iteration 2 is:

(x1,x2,s1,s2,s3) = (2,6,2,0,0) where {s1,x2,x1} are the basicvariables, and {s3,s3} are the nonbasic variables.The new objective equation is ”Eq. (0) : z+3/2s2 + s3 = 0”.

Optimality Test 3 In Eq. (0), "3/2 > 0 and 1 > 0" so it is optimal.

Conclusion. Optimal decision variables are (x1,x2) = (2,6) and z = 36.


Simplex Method for LP: the Basic Case (week 3-4) · We ﬁrst solve LP in standard forms and show...

Documents

Transcript of Simplex Method for LP: the Basic Case (week 3-4) · We ﬁrst solve LP in standard forms and show...