Notes: Deterministic Models in Operations ResearchNotes: Deterministic Models in Operations Research...

Notes: Deterministic Modelsin Operations Research

J.C. ChrispellDepartment of Mathematics

Indiana University of PennsylvaniaIndiana, PA, 15705, USA

E-mail: [email protected]://www.math.iup.edu/~jchrispe

April 19, 2016

http://www.math.iup.edu/~jchrispe

OR Notes Draft: April 19, 2016

ii

Preface

These notes will serve as an introduction to the basics of solving deterministic models inoperations research. Topics discussed will included optimization techniques and applicationsin linear programming. Specifically a discussion of sensitivity analysis, duality, and thesimplex method will be given. Additional topics to be considered include non-linear anddynamic programming, transportation models, and network models.

The majority of this course will follow the presentation given in the Operations Research:Applications and Algorithms text by Winston [9]. I will supplement the Winston text withadditional material from other popular books on operations research. For further readingyou may wish to consult:

• Introduction to Operations Research by Hillier and Lieberman [3]

• Operations Research: An Introduction by Taha [8]

• Linear Programming and its Applications by Eiselt and Sandblom [1]

• Linear and Nonlinear Programming by Luenberger and Ye [5]

• Linear and Nonlinear Programming by Nash and Sofer [6]

My Apologies in advance for any typographical errors or mistakes that are present in thisdocument. That said, I will do my very best to update and correct the document if I ammade aware of these inaccuracies.

-John Chrispell

iii


iv

Contents

1 Introduction 1

1.1 Table and Chair Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Basic Linear Algebra 7

2.0.1 The Gauss-Jordan Method for Solving Linear Systems . . . . . . . . 8

3 Linear Programming Basics 11

3.1 Parts of a Linear Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1.1 Linear Programming Assumptions . . . . . . . . . . . . . . . . . . . . 12

3.1.2 Feasible Region and Optimal Solution . . . . . . . . . . . . . . . . . . 12

3.2 Special Cases: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2.1 Example: Multiple Optimal Solutions . . . . . . . . . . . . . . . . . 15

3.2.2 Example: Infeasible Linear Program . . . . . . . . . . . . . . . . . . 16

3.2.3 Example: Unbounded Optimal Solution . . . . . . . . . . . . . . . . 17

3.3 Setting up Linear Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3.1 Work-Scheduling Problem: . . . . . . . . . . . . . . . . . . . . . . . . 18

4 Examples of Linear Programs 19

4.1 Diet Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.1.1 Solution Diet Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.2 Scheduling Problem: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.2.1 Solution Scheduling Problem: . . . . . . . . . . . . . . . . . . . . . . 22

4.3 A Budgeting Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.3.1 Solution Budgeting Problem . . . . . . . . . . . . . . . . . . . . . . . 23

v


5 The Simplex Algorithm 27

5.1 Standard Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5.1.1 Example: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.2 Basic and Nonbasic Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.2.1 Directions of Unboundedness . . . . . . . . . . . . . . . . . . . . . . 33

5.3 Implementation of the Simplex Method . . . . . . . . . . . . . . . . . . . . . 36

5.4 Simplex for Minimization Problems . . . . . . . . . . . . . . . . . . . . . . . 42

5.5 Big-M Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.6 Degeneracy and Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.7 Two-Phase Simplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6 Simplex Method Using Matrix-Vector Formulas 57

6.1 Problem Using Tableaus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6.2 Matrix Format of Linear Program . . . . . . . . . . . . . . . . . . . . . . . . 58

6.2.1 Initial Tableau (Matrix Form): . . . . . . . . . . . . . . . . . . . . . . 59

7 Duality 69

7.1 A Motivating Example: My Diet Problem: . . . . . . . . . . . . . . . . . . . 69

7.1.1 Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8 Basic Duality Theory 75

8.1 Relationship Between Primal and Dual . . . . . . . . . . . . . . . . . . . . . 75

8.2 Weak Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

8.3 Strong Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

8.4 The Dual Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

9 Sensitivity Analysis 91

9.1 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

9.1.1 Verify this graphically: . . . . . . . . . . . . . . . . . . . . . . . . . . 93

9.2 Sensitivity Analysis Using Matrices . . . . . . . . . . . . . . . . . . . . . . . 94

9.2.1 Illustrating Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

10 Non Linear Programming 103

vi


10.1 Data Fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

10.1.1 Linear Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

10.1.2 Taylor series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

10.1.3 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

10.2 Non-linear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

11 Network Flows 111

11.1 Dijkstra’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

11.2 Max Flow Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Bibliography 121

vii

Chapter 1

Introduction

“If you choose not to decide you still have made a choice.”

−Neil Peart

Operations Research, at its heart, uses modeling to solve applied mathematical problems inthe hopes of making optimal decisions efficiently.

The term Operations Research can trace its roots back to World War II when a scientificapproach was adopted to distribute resources between various military operations (scientistsdoing research on military operations)[3, 9].

In order to do any scientific research on a problem a mathematical model is needed. TheWinston text states:

• Mathematical Model: - is a mathematical representation of an actual situationthat may be used to make a better decision or simply to understand the actual situationbetter.

The goal with most models considered in this course will be to optimization somequantity. Here we will use:

– objective function based on

– decision variables) subject to problem

– constraints.

Several examples where Operations Research techniques were implemented in order fordifferent organizations to obtain optimal solutions and distribution of resources (CITGOPetroleum manufacturing, The San Francisco Police Department Scheduling, GE Capitalcredit card bill repayment).

1


1.1 Table and Chair Example

Consider the following modification of an example given in [7]:

A furniture manufacturer produces two products: wooden tables and chairs. Theunit profit for the tables is $6, and the unit profit for the chairs is $8. In orderto simplify the problem assume that only resources used in the production of thetables and chairs are wood (in board feet, bf) and labor (in hours, h). It takes6 bf and 1 hour labor to make a table, and 4 bf and 2 hours labor to make achair. There are 20 bf of wood available and 6 hours of labor available. If youwish to maximize profit what is the optimal distribution of these resources?

Formulate a Mathematical Model of the Problem

The idea now becomes to solve the given problem using a linear model or linear programming.Thus, we need to translate the real world problem into a format with mathematical equationsthat represent:

• An objective function: In this case maximize the profit.

• Decision variables: Number of Tables and Chairs to produce.

• Constraint set: We only have a limited number of resources available to use whenmanufacturing the tables and chairs.

For this example the objective function and the constraints will all be linear functions. Weshould also impose non-negativity conditions on the resources and decision variables in thisinstance. If we do not enforce these conditions it may be possible to achieve non-physicalsolutions to the given problem.

The problem definition, and formulation are often the most difficult and import step infinding an optimal solution. Lets define the decision variables x1 and x2 as the number oftables and the number of chairs to produce respectively. It is sometimes useful to place allthe information into a table:

2


Resource Table x1 Chair x2 AvailableWood (bf) 6 4 20Labor (hr) 1 2 6Unit Profit $6 $8

The goal is to obtain the largest profit from the objective function by:

Maximize 6x1 + 8x2

subject to:

6x1 + 4x2 ≤ 20

x1 + 2x2 ≤ 6

x1 ≥ 0

x2 ≥ 0

the problem constraints.

Solving the problem using a Graphical Approach

Lets assume that the number of chairs to be produced, x1 is on the x-axis and the numberof tables to be produced x2 is on the y−axis. First write the constraints as equalities, andthen finding the intercepts of the feasible region. Thus,

6x1 + 4x2 ≤ 20 =⇒ x2 = −3

2x1 + 5 (1.1.1)

x1 + 2x2 ≤ 6 =⇒ x2 = −1

2x1 + 3 (1.1.2)

x1 ≥ 0 x1 = 0 (1.1.3)x2 ≥ 0 x2 = 0 (1.1.4)

(1.1.5)

If all the wood is used to produce chairs (set x1 = 0 in 1.1.1):

x2 = 5 this yields point C.

If all the wood is used to produce tables (set x2 = 0 in 1.1.1):

x1 =10

3this yields point E.

If all the labor is used to produce chairs (set x1 = 0 in 1.1.2):

x2 = 3 this yields point B.

If all the labor is used to produce tables (set x2 = 0 in 1.1.2):

x1 = 6 this yields point D.

Note we obtain a feasible region given by the polygon FEAB.

3


Figure 1.1.1: Plot of the constraints for the table and chair example.

Plot instances of the objective function

If we now pick two values for the objective function we can determine its slope and a directionof improvement. Let z be the value of the objective function:

z = 6x1 + 8x2

Thus, if z = 0 then

6x1 + 8x2 = 0 =⇒ x2 = −3

4x1,

and if z = 8 we see6x1 + 8x2 = 8 =⇒ x2 = −3

4x1 + 1.

The direction of improvement is d = [4, 3]T , and is illustrated in Figure 1.1.2. The optimalsolution will be obtained at the corner or edge of the feasible region that is farthest awayfrom the origin on a line parallel to the objective function contours plotted for z = 0, andz = 8. Specifically from the figure the objective function is maximized at point A when 2tables and 2 chairs are manufactured using the resources available, and yields a profit of $28.

The optimal solution could have also been found by testing the objective function at eachcorner of the feasible region. Note if the same value of the objective function has beenobtained at multiple corners the optimal solution of would have been any convex combinationof the two corners. In this problem we are manufacturing discrete items (building a fractionof a table or chair doesn’t make since) so we are lucky that we obtained an integer solutionto the mathematical model.

4


Here we will solved this simple Linear Program graphically. In the future we will use moreadvanced algebraic techniques (such as the simplex method ) to obtain the optimal solution.For more complicated problems it will become necessary to use software packages such asLINGO. In the future we will also look to see how sensitive our optimal solution is to smallchanges.

Figure 1.1.2: Illustration of the direction of improvement.

5


6

Chapter 2

Basic Linear Algebra

“Our weary eyes still stray to the horizon. Though down this road we’ve been so many

times.”

−Pink Floyd

Before looking into more complicated methods of solving linear programs we need to recallsome of the basics of linear algebra.

We will use the following notation. A matrix is a rectangular array of numbers, with atypical m× n matrix A written in the form:

A =

a11 a12 . . . a1n

a21 a22 . . . a2n...

... . . . ...am1 am2 . . . amn

(2.0.1)

The element of a matrix in the ith row and jth column of matrix A will be denoted by aij.Thus, if the matrix

A =

1 1 2 35 8 13 2134 55 89 144

then a23 = 13.

We will think of a matrix with only one column as vectors (or column vectors). Similarlya matrix with only one row will be a row vector. The number of rows in a vector will defineits dimension, and the number of columns in a row vector will define its dimension.

An m-dimensional vector (row or column) where all elements are zero will be called azerovector and denoted by 0. In two dimensions

0 = [0 0] and 0 =

[00

]7


are zero vectors.

Vectors correspond to a directed line segment from the origin in the m-dimensional plane.Thus the vectors:

u =

[12

], and w =

[−3−4

]are illustrated in Figure 2.0.1. Readers should reacquaint themselves with matrix multipli-

Figure 2.0.1: Illustration of two vectors in two-dimensional plane.

cation, matrix transposition and inner products. The use of matrices and vectors will allowfor mathematical models of linear systems to be developed and solved using linear algebra.

2.0.1 The Gauss-Jordan Method for Solving Linear Systems

The three basic types of elementary row operations (ERO) are used when solving systemsof linear equation using the Gauss-Jordan Method.

1. Multiply any row of a matrix system by a nonzero scalar.

2. Multiply any row of a matrix system by a nonzero scalar and then add that row to adifferent row of the matrix.

3. Interchange any two rows of a matrix system.

8


Consider the following example:

x1 − 2x2 + 3x3 = 9

−x1 + 3x2 = −4

2x1 − 5x2 + 5x3 = 17

We may write this linear system in the augmented matrix system notation as

A|b =

1 −2 3 9−1 3 0 −42 −5 5 17

9


10

Chapter 3

Linear Programming Basics

“Remember to remember meStanding still in your past

Floating fast like a hummingbird”

−Wilco

3.1 Parts of a Linear Program

We detailed earlier that the major components of a linear program:

• Decision variables.

• The objective function.

• The problem constraints.

It is also worthy to note that the coefficient of a variable in the objective function is referredto as an objective function coefficient, and less obviously the coefficients in the constraintfunctions are sometimes referred to as technological coefficients.

It will often be useful to denote the coefficients on the right-hand-side of the constraints withrhs (representing the amount of a distinct resource that is available).

We will separate non-negativity of decision variables as a separate type of constraint, andmake note that decision variables may be unrestricted in sign.

Definitions:

• A function f(x1, x2, . . . , xn) of x1, x2, . . . , xn is a linear function if and only if forsome set of constants c1, c2, . . . , cn, f(x1, x2, . . . , xn) = c1x1 + c2x2 + . . .+ cnxn.

11


• For any linear function f(x1, x2, . . . , xn) and any number b, the inequalities f(x1, x2, . . . , xn) ≤b and f(x1, x2, . . . , xn) ≥ b are linear inequalities.

With these definitions we can define a linear programming problem or LP as an opti-mization problem for which we do the following:

• Maximize or minimize a linear function of some decision variables (our objective func-tion).

• The decision variables are subject to satisfying a set of constraints each of which is alinear inequality.

• There is also a set of sign restrictions that will be satisfied. For example decisionvariables may be non-negative or unrestricted in sign.

3.1.1 Linear Programming Assumptions

In all linear programs the objective function and constraints are effected proportionallyby the decision variables (a constant coefficient multiplies the variable in each). Linearprogramming problems also have the benefit of each decision variable being independent ofothers in its contribution to the objective function. Similarly linear constraints have thebenefit of independence. For example in the table and chair example we considered thenumber of hours of labor required to make a chair did not effect the number of hours laborrequired to manufacture a table. Thus, the left hand side of the labor constraint was theadditive sum of constants multiples of the decision variables.

In addition to the proportionality and additivity assumptions in order for LP’s to representreal situations, the divisibility assumption. That is the decision variables must be allowedto take on fractional values (we can not sell a fraction of a chair). In the future we may look atinteger programming as a way of handling problems that require a discrete valued solution.We are also going to make a certainty assumption. That is that the model parametersobjective function coefficients, the right hand side of constraints and technological coefficientsare known with certainty.

3.1.2 Feasible Region and Optimal Solution

Assuming that a point is a set of distinct values for each decision variable in an LP. Thefeasible region is the collection of all possible points that satisfy the constraints and signrestrictions on the decision variables. Points not within the feasible for a linear program areconsidered infeasible.

The optimal solution for a maximization problem then becomes the point in the LP’sfeasible region that obtains the largest value of the objective function. Similarly for a mini-mization problem the point inside the LP’s feasible region that obtains the smallest value ofthe objective function is considered the optimal solution.

12


A second graphical example

Consider the Linear Program:

max z = x1 + x2

s.t. x1 + 5x2 ≤ 25

6x1 + 5x2 ≤ 40

x1 ≤ 5

with x1, x2 ≥ 0.

The graphical solution to the linear program is shown in Figure 3.1.1. Note the value of theobjective function is is 7.4, and that only two of the constraints are binding at the optimalsolution.

• We consider constraints binding if the left hand side of the constraint is equal to theright hand side when an optimal solution is achieved.

• The constraint x1 ≤ 5 is an example of a nonbinding constraint.

Figure 3.1.1: Graphical solution to LP with bounded Feasible Region.

13


The feasible region for this LP is a convex set. A set of points is aConvex set provided aline segment between any pair of points in the set is wholly contained in the set.

For any convex set S, a point P is an extreme point if each line segment that lies completelyin S and contains the point P has P as an endpoint of the line segment. Extreme pointsmay also be called corner points.

We make note that:

Any LP that has an optimal solution has an extreme point that is optimal.

See the Winston text for a proof of this result. It is an important result because it narrowsour search for an optimal solution down from the whole feasible region to just the extremepoints of the set.

A Minimization Problem

Consider the following minimization linear programming problem:

min z = x1 + x2

s.t. x1 + 4x2 ≥ 8

4x1 + x2 ≥ 6

with x1, x2 ≥ 0.

Here we again solve the LP graphically. Note that the optimal solution is found at point(1615, 26

15

)giving the objective function a value of 14

5. The solution is illustrated in Figure 3.1.2.

Note that we could have used matrix notation to represent the previous problems. It willsometimes be useful to write our linear programs in this form.

Figure 3.1.2: Graphical solution to minimization LP with bounded Feasible Region.

14


3.2 Special Cases:

Linear programs are not always this nice! Sometimes the linear program has an infinitenumber of solutions. Sometimes the linear program has no feasible solution, and at othertimes the solution to the linear program is unbounded.

3.2.1 Example: Multiple Optimal Solutions


max z = 3x1 + 2x2

s.t. 6x1 + 4x2 ≤ 32

−x1 + 2x2 ≤ 10

x1 ≤ 4

with x1, x2 ≥ 0.

When this LP is solved graphically as is done in Figure 3.2.3 it can be seen that there existmultiple optimal solutions. Thus, the solution to the LP becomes the convex combinationof the two extreme points represented in the graph by C and D. We can write the optimalsolution to the LP by letting α ∈ [0, 1], and stating it as a convex set:

optimal set ={

(x, y)∣∣ α (4, 2) + (1− α) (1.5, 5.75)

}. (3.2.1)

We can note that any values in the optimal set will yield a value of 16 in the objectivefunction. Use the following link to explore this problem using an interactive java applet:

http://www.math.iup.edu/~jchrispe/MATH445_545/MultipleOptimalSolutionExample.html

Figure 3.2.3: Graphical solution to maximization LP with bounded Feasible Region andmultiple optimal solutions.

15

http://www.math.iup.edu/~jchrispe/MATH445_545/MultipleOptimalSolutionExample.html


3.2.2 Example: Infeasible Linear Program


max z = 3x1 + 2x2

s.t. x1 + x2 ≥ 10

x2 ≤ 5.8

x1 ≤ 4

with x1, x2 ≥ 0.

Figure 3.2.4 shows the binding constraint contours and a plot of the objective functions

Figure 3.2.4: Plot of binding constraint contours and objective function contour when z = 10.

contour for a value z = 10. Note that there is no region that will satisfy all of the linearprograms constraints making this an infeasible LP.

16


3.2.3 Example: Unbounded Optimal Solution

Consider one last example (straight out of the Winston text). Solve again the followinglinear program graphically.

max z = 2x1 − x2

s.t. x1 − x2 ≤ 1

2x1 + x2 ≥ 6

with x1, x2 ≥ 0.

In Figure 3.2.5 it can be seen that the feasible region is unbounded in the direction of

Figure 3.2.5: Plot of the feasible region for the maximization problem. Note that the feasibleregion is unbounded in the direction of increasing z contours.

increasing z contours. Thus the optimal solution is unbounded.

17


3.3 Setting up Linear Programs

In many cases the difficult part of linear programming is translating a physical situationinto a mathematical model. The following are a collection of word problems that may beformulated as a linear program.

3.3.1 Work-Scheduling Problem:

This example is from the Winston text: Consider a chain of computer stores. The number ofskilled repair time the company requires during the next five months is given in the followingtable:

Month 1 (January): 6,000 hoursMonth 2 (February): 7,000 hoursMonth 3 (March): 8,000 hoursMonth 4 (April): 9,500 hoursMonth 5 (May): 11,000 hours

At the beginning of January, 50 skilled technicians work for the company. Each skilledtechnician can work up to 160 hours per month. To meet future demands, new techniciansmust be trained. It takes one month to train a new technician. During the month of trainingthe trainee must be supervised for 50 hours by an experienced technician. Each experiencedtechnician is paid $2000 a month regardless of how much he works. During the month oftraining the trainee receives $1000 for the month. At the end of each month 5% of thecompanies experienced technicians quit to find other employment. Formulate an LP for thecompany that will minimize the labor costs incurred and meet the service demands for thenext five months.

18

Chapter 4

Examples of Linear Programs

“ The tall one wants white toast, dry, with nothin’ on it...

And the short one wants four whole fried chickens, and a Coke.”

−Mrs. Murphy, The Blues Brothers

In this section our goal will be to consider setting up some common types of linear programs.The list of examples given here is by no means exhaustive, and we should not for get aboutthe work scheduling problem stated in the previous section.

4.1 Diet Problem

My diet may not be nearly as poor as that of the blues brothers; however, it is probably stillpretty bad. Lets assume that my diet requires that all of my food come from my favoritefood groups: pez candy, pop tarts, Cap’N crunch, and cookies. I have the following foodsavailable for my consumption: red pez, strawberry pop-tarts, peanut butter crunch, andchocolate chip cookies. The red pez costs $0.20 per (per package), strawberry pop-tarts cost$0.50 (per package), peanut butter crunch costs $0.70 per bowl, and chocolate chip cookiescost $0.65 cents each. I must ingest at least 2500 calories a day in order to maintain mysugary lifestyle. I must also meet the following requirements I need 8 oz. of sugar, 10 oz.of fat, and 50 mg of yellow-5 food coloring. Each of my chosen foods has these requirednutrients in the following quantities:

Food Calories Sugar (oz.) Fat (oz.) yellow-5 (mg)red pez candy (per package) 50 0 .1 0.01 5strawberry pop-tarts (per package) 250 0.5 0.3 0peanut butter Cap’N crunch (per bowl) 350 1 1 4chocolate chip cookies (1 cookie) 150 0.3 0.7 1

How can I achieve my dietary constraints at a minimum cost?

19


4.1.1 Solution Diet Problem

The first step will be do define some decision variables. Here we will define

x1 as the number of pez packages to include in my diet per dayx2 as the number strawberry pop-tart packages to consume per dayx3 as the number of bowls of peanut butter crunch to consume per dayx4 as the number of chocolate chip cookies to consume each day

Our objective function can then be defined as:

min z = 0.20x1 + 0.50x2 + 0.70x3 + 0.65x4

which will minimize the cost of our diet. The next aim is to meet the special dietary needs.Here a constraint is set up for each of the “nutrients” (calories, sugar, fat, and yellow 5).

50x1 + 250x2 + 350x3 + 150x4 ≥ 2500 (calories required)0.1x1 + 0.5x2 + x3 + 0.3x4 ≥ 8 (sugar required)

0.01x1 + 0.3x2 + x3 + 0.7x4 ≥ 10 (fat required)5x1 + 0x2 + 4x3 + x4 ≥ 50 (yellow-5 required)

It can also be noted that the decision variables will be non-negative in sign giving the

xi ≥ 0 for all i ∈ 1, 2, 3, 4

sign restriction.

This problem can be set up in vector notation using the above variable definitions anddefining

x =

x1

x2

x3

x4

, c =

0.200.500.700.65

, b =

2500

81050

, and

A =

50 250 350 1500.1 0.5 1 0.30.01 0.3 1 0.7

5 0 4 1

.The optimal diet can now be found by considering:

min z = cTx

s.t. Ax ≥ b

with x ≥ 0.

The optimal solution can be quickly found using the LINGO software package and the oldLINDO syntax. Open LINGO and type:

20


min 0 .2x_1 + 0 .5x_2 + 0 .7x_3 + 0.65x_4s . t . 50x_1 + 250x_2 + 350x_3 + 150x_4 >= 25000 .1x_1 + 0 .5x_2 + x_3 + 0 .3x_4 >= 80.01x_1 + 0 .3x_2 + x_3 + 0 .7x_4 >= 105x_1 + 0x_2 + 4x_3 + x_4 >= 50

This yields the optimal objective function value of $7.39 when

x1 = 2.016129, x2 = 0.000000, x3 = 9.979839, and x4 = 0.000000. (4.1.1)

So the minimum cost diet has me eating nothing but approximately 2 packages of pez candyand 10 bowls peanut butter crunch to meet my dietary constraints.

21


4.2 Scheduling Problem:

Suppose that the math department wants to schedule tutors for a cram-day before finalexams. The number of tutors needed for each four hour shift on this cram-day are:

Shift Number Time Number of Tutors Needed1 12:00 am - 4:00 am 32 4:00 am - 8:00 am 43 8:00 am - 12:00 pm 64 12:00 pm - 4:00 pm 55 4:00 pm - 8:00 pm 86 8:00 pm - 12:00 am 4

Each tutor will work two consecutive 4 hour shifts. Formulate a linear program that can beused to minimize the number of tutors needed to meet the cram-day demands.

4.2.1 Solution Scheduling Problem:

Its always best to think about the decision variable first! Here we can think about thestarting shift for the tutors.

Let xi be the number of tutors that start working on shift i.

The objective of the linear program is to minimize the number of tutors needed to meet thedemands, and we make note that summing the decision variables will count the total numberof tutors used. We should also note that since we are only concerning ourselves with 8 hourshifts for a single day, we only need to start tutors for the first five shifts. This give theobjective function:

minimizez =5∑i=1

xi

Then the constraints become:

x1 ≥ 3

x1 + x2 ≥ 4

x2 + x3 ≥ 6

x3 + x4 ≥ 5

x4 + x5 ≥ 8

x5 ≥ 4

where

xi ≥ 0 for i ∈ {1, 2, . . . , 5}. (4.2.2)

Note we also added the non-negativity constraint for all the decision variables. It may havealso been nice to consider making the schedule for several cram-days in a row that again allhad the same demands for the number of tutors needed during each shift.

22


4.3 A Budgeting Problem

Two investments with varying cash flows (in thousands of dollars) are available.

Investment Cash Flow Year 0 Cash Flow Year 1 Cash Flow Year 2 Cash Flow Year 31 -6 - 5 7 92 -8 -3 9 7

Assuming at time 0 that $10,000 is available to invest, and after one year there will be $7,000available to invest. We will also assume an annual interest rate of 10% is available. We shallalso assume that any fraction of an investment may be purchased. Lets find a linear programthat will determine the maximum net present value obtained from the two investments.

4.3.1 Solution Budgeting Problem

We set up the linear program as follows. First the decision variables:

Let xi be the amount of investment i to purchase (i = 1, 2).

Our goal is to maximize the net present value of the investments purchased so we shouldfind the net present value of each investment for use in our objective functions.

Net present value: We need to discount each of the cash flows back to the present usinga discount factor based on the interest rate. Note that

$1.00 Today −→ $1.00 + 0.10($1.00) = $1.00(1.10) One Year From Now

Thus,

$1.00 one year from now −→(

1

1.10

)$1.00 ≈ $0.9090909 today.

We can use (1

1 + r

)nas a discount factor for rate r in year n.

Let NPV1 denote the net present value of investment 1.

NPV1 = −6000 + (−5000)

(1

1.1

)1

+ 7000

(1

1.1

)2

+ 9000

(1

1.1

)3

≈ 2001.503

Let NPV2 denote the net present value of investment 2.

NPV2 = −8000 + (−3000)

(1

1.1

)1

+ 9000

(1

1.1

)2

+ 7000

(1

1.1

)3

≈ 1969.947

23


The objective function for the linear program is then:

maximize z = 2001.50x1 + 1969.94x2.

The constraints on the linear program are based on the amount of cash we have on hand foreach of the first two years, and the additional constraint that we may only purchase upto100% of each of the two investments. This gives

6000x1 + 8000x2 ≤ 10000

5000x1 + 3000x2 ≤ 7000

x1 ≤ 1

x2 ≤ 1

as our constraints, and we have the sign restriction that x1 and x2 are non-negative:

x1 ≥ 0, and x2 ≥ 0.

Note we can solve this linear program graphically and Figure 4.3.1 illustrates the feasiblesolution to the problem. We can see by looking for the optimal objective function contourthat the optimal investment involves 100% of investment 1 and 50% of investment 2 givingthe optimal net present value of the cash flow as:

z = 2001.50(1) + 1969.94(0.5) = 2986.47

Note that there are an infinite number of different problems that may be formulated as linearprograms. It is strongly recommended that the interested reader look over more examplespresented in [9] and other texts.

24


Figure 4.3.1: Feasible region for the budgeting problem with an illustration of a z contour.

25


26

Chapter 5

The Simplex Algorithm

“Don’t wanna wait ’til tomorrowWhy put it off another day?

One by one, little problems

Build up and stand in our way, oh”

−Van Halen

The linear programs that we have been able to solve with out the assistance of a softwarepackage so far have all been in two variables. In general linear programs have many variablesand the goals of this chapter will be:

• Transform a linear programming problem into a standard form.

• Look at basic and nonbasic variables.

• Introduce the Simplex Algorithm for solving linear programming problems.

5.1 Standard Form

As we have seen there are several different ways of writing the objective function for a linearprogram (maximize profit, minimize cost). Additionally the constraints come in all differentforms too (≤,≥,=) so when discussing a systematic method for solving linear programs itwill be advantages to having written or converted the linear program to a standard form.

We will consider the following linear programs to be in standard form:

minimize z = cTx, maximize z = −cTx,subject to Ax = b, subject to Ax = b,

x ≥ 0. x ≥ 0.

27


Here:

• b, c, and x represent vectors of length n.

• A is an m× n constraint matrix.

• b ≥ 0.

Notice that:

• The decision variables are constrained to be non-negative.

• All of the constraints are set up to be equalities.

• The components of the right hand side vector b are non-negative.

The observation that a maximization problem could be converted to a minimization problemby multiplying every coefficient in the objective function by a ‘-1’ can be made. After theproblem is solved the objective value would then need to be multiplied by ‘-1’ again in orderto obtain the solution to the desired problem. The decision variable values obtained wouldbe the same for both objective functions.

Example: Getting to Standard Form

Lets consider transforming the following linear program into a standard form minimizationproblem:

maximize z = 3x1 +4x2 −7x3

subject to 7x1 +x2 +4x3 ≤ 26−2x1 +4x2 +6x3 ≤ −2

6x1 +3x2 −4x3 ≥ 4x3 ≥ 5

with x1 ≥ 0 and x2 free.

Objective function: The problem could be written using the equivalent objective function:

minimize z = −3x1 − 4x2 + 7x3

where z = −z once the optimal solution has been found.

Non-negative right had side: In order to continue converting the given LP to standardform we next consider the constraints. We would like to have all the right hand side coef-ficients be non-negative. Thus, the third constraint in the given linear program should bemultiplied by a ‘-1’ yielding:

−2x1 + 4x2 + 6x3 ≤ −2 −→ 2x1 − 4x2 − 6x3 ≥ 2

28


Non-zero lower bounds: In the original statement of the problem we note that the 4thconstraint has:

x3 ≥ 5.

To convert this to a standard form constraint we make a change of variables and define:

x3 = x3 − 5

This allows us to replace x3 with x3 throughout and changes the constraint

x3 ≥ 5 −→ x3 + 5 ≥ 5 −→ x3 ≥ 0.

Note we need to modify the other constraints and objective function as well. For our exampleproblem we may redefine

z = −z − 35 −→ z = −z − 35

for the modified objective function. For the moment we will leave upper bounds on variablesright in the coefficient matrix.

Free variables: In the given problem we make note that x2 is a free variable. Free variablesmay be converted to non-negative constraints by defining

x2 = x′2 − x′′2 with x′2, x′′2 ≥ 0.

and we will have x′2 to handle the positive values of x2 and the new variable x′′2 will takecare of the negative pieces. There are other ways to handle free variables, and one is to usea free variable as a way of eliminating a constraint (think about this for a future homeworkassignment).

After applying all of the discussed techniques to our linear programming problem we are leftwith the following LP:

minimize z = −3x1 −4x′2 + 4x′′2 +7x3

subject to 7x1 +x′2 − x′′2 +4x3 ≤ 62x1 −4x′2 + 4x′′2 −6x3 ≥ 326x1 +3x′2 − 3x′′2 −4x3 ≥ 24

with x1, x′2, x′′2, x3 ≥ 0.

Equality constraints: In order for the constraints we consider adding slack and excessvariables to enforce the equality constraint. For the constraint:

7x1 + x′2 − x′′2 + 4x3 ≤ 6

let s1 be a slack variable such that s1 ≥ 0 and

7x1 + x′2 − x′′2 + 4x3 + s1 = 6.

For the other two constraints we use excess variables e1 and e2. Thus,

2x1 − 4x′2 + 4x′′2 − 6x3 ≥ 32 =⇒ 2x1 − 4x′2 + 4x′′2 − 6x3 − e1 = 32

29


and6x1 + 3x′2 − 3x′′2 − 4x3 ≥ 24 =⇒ 6x1 + 3x′2 − 3x′′2 − 4x3 − e2 = 24

with e1, e2 ≥ 0.

We can now restate the full linear program in standard form:

minimize z = −3x1 −4x′2 + 4x′′2 +7x3

subject to 7x1 +x′2 − x′′2 +4x3 +s1 = 62x1 −4x′2 + 4x′′2 −6x3 −e1 = 326x1 +3x′2 − 3x′′2 −4x3 −e2 = 24

with x1, x′2, x′′2, x3, s1, e1, e2 ≥ 0.

5.1.1 Example:

Try writing the following linear programs in standard form:

maximize z = 3x1 +5x2 −4x3

subject to 7x1 −2x2 −3x3 ≥ 4−2x1 +4x2 +8x3 = −3

5x1 −3x2 −2x3 ≤ 9

with x1 ≥ 1, x2 ≤ 7, and x3 ≥ 0.

After manipulation we obtain:

maximize z = 3x1 +5x′2 − 5x′′2 −4x3

subject to −7x1 +2x′2 − 2x′′2 +3x3 ≤ 32x1 −4x′2 + 4x′′2 −8x3 = 15x1 −3x′2 + 3x′′2 −2x3 ≤ 4

x′2 − x′′2 ≤ 7

with x1, x′2, x′′2, x3 ≥ 0 and z = z + 3.

We need to take this one more step and include slack variables. Thus,

maximize z = 3x1 +5x′2 − 5x′′2 −4x3

subject to −7x1 +2x′2 − 2x′′2 +3x3 +s1 = 32x1 −4x′2 + 4x′′2 −8x3 = 15x1 −3x′2 + 3x′′2 −2x3 +s2 = 4

x′2 − x′′2 +s3 = 7

with x1, x′2, x′′2, x3, s1, s2, s3 ≥ 0 and z = z + 3.

30


5.2 Basic and Nonbasic Variables

Lets assume that we have a standard form linear program.

minimize z = cTx,subject to Ax = b,

x ≥ 0.(5.2.1)

where b, c, and x represent vectors of length n, and A is an m× n constraint matrix (withn ≥ m), and b ≥ 0. The constraints to the linear program are given by:

Ax = b.

Definition 5.2.1 A basic solution to Ax = b is obtained by setting n−m variables equalto 0 and solving for the values of the remaining m variables. This assumes that the columnsremaining m variables are linearly independent.

To find a basic solution to Ax = b, we simply choose a set of n−m variables (and set theseto zero) to be the nonbasic variables or NBV. The remaining variables will be our basicvariables or BV that are used to satisfy the constraints.


x1 + x2 = 6

−x2 + x3 = 4

Here we get to pick one nonbasic variable as we have two equations and three unknowns.Thus,

NBV = {x3}, then BV = {x1, x2}.

The values of the basic variables x1, and x2 are found by solving the two equations with x3

set to zero. Thus,x2 = −4 and x1 = 10

If we choose x1 to be the nonbasic variable then the basic variables

x2 = 6 and x3 = 10.

We should note that not all sets of basic variables will yield a feasible solution to a set ofconstraints.

Consider the constraints

x1 + 2x2 + 4x3 = 4

x1 + 4x2 + 8x3 = 6.

31


If x1 is taken to be the nonbasic variable then the system becomes:

2x2 + 4x3 = 4

4x2 + 8x3 = 6

a system with no feasible solution and yielding no basic solution to BV = {x2, x3}.

Definition 5.2.2 Any basic solution to (5.2.1) in which all variables are nonnegative is abasic feasible solution.

When solving linear programs we are interested in sets of variables that will satisfy all of theconstraints given by Ax = b as well as also satisfying the nonnegativity constraint for thelinear program stated in standard form. We also know that

Any LP that has an optimal solution has an extreme point that is optimal.

So a good place to start looking for these optimal extreme points is at basic feasible solutions.It can be shown that:

Theorem 5.2.1 A point in the feasible region of a linear program is an extreme point if andonly if it is a basic feasible solution to the LP.

A proof can be found in [4].

Example: The goal of this example is to show the relationship between extreme points andbasic feasible solutions. Given

maximize z = 4x1 + 3x2

subject to x1 + x2 ≤ 42x1 + x2 ≤ 6

with x1, x2 ≥ 0,

write the LP in standard form by adding two slack variables s1 and s2.

Standard form:maximize z = 4x1 + 3x2

subject to x1 + x2 + s1 = 42x1 + x2 + s2 = 6

with x1, x2, s1, s2 ≥ 0.

The feasible region for the linear program is shown if Figure 5.2.1. Note that all of the cornerextreme points correspond to a basic feasible solution to the linear program. We note thatthe intersection at points E, and F are not basic feasible solutions because not all of thebasic variables satisfy the nonnegativity constraint for the linear program in standard form.

32


Figure 5.2.1:

5.2.1 Directions of Unboundedness

Assume that we have a standard form linear program.

minimize z = cTx,subject to Ax = b,

x ≥ 0.

where b, c, and x represent vectors of length n, and A is an m× n constraint matrix (withn ≥ m), and b ≥ 0. The feasible solutions for this linear program will be denoted by S.

Definition 5.2.3 An n by 1 vector d is a direction of unboundedness if for all x in S,and c ≥ 0 then

x + cd ∈ S.

This means we can move as far as we desire in the direction of d and still have a feasiblesolution to the linear program.

33


Example:

Consider the following

minimize z x1 + x2,subject to 7x1 + 2x2 ≥ 28

2x1 + 12x2 ≥ 24x1, x2 ≥ 0.

Written in standard form by adding excess variables we obtain,

minimize z x1 + x2,subject to 7x1 + 2x2 − e1 = 28

2x1 + 12x2 − e2 = 24x1, x2, e1, e2 ≥ 0.

What are the basic feasible solutions for the LP.

BV NBV bfs (Basic Feasible Solution)x1, x2 e1, e2 YES , x = (3.6, 1.4, 0, 0)T

x1, e1 x2, e2 YES , x = (12, 0, 56, 0)T

x1, e2 x2, e1 NO , x = (4, 0, 0,−16)T

x2, e1 x1, e2 NO, x = (0, 2,−24, 0)T

x2, e2 x1, e1 YES, x = (0, 14, 0, 144)T

e1, e2 x1, x2 NO, x = (0, 0,−28,−24)T

Looking at the problem’s feasible region graphically we have the illustration in Figure 5.2.2.Specifically the feasible region for the linear program can be made up of a convex combina-tion of the basic feasible solution points plus a positive constant multiple of a direction ofunboundedness.

Note that moving from the point basic feasible solution at point3.61.400

and moving toward point

4.62.4914

yields a direction of unboundedness given by:

d1 =

11914

.

Note that the direction of unboundedness is not unique, as

d2 =

242252

34


is a direction of unboundedness found by heading from the basic feasible solution120560

toward interior point

144052

.

Specifically we could write the feasible set by letting α1, α2, α3 ∈ [0, 1] such that∑3

i=1 αi = 1,and c a nonnegative constant then

S = α1

3.61.400

+ α2

120560

+ α3

0140

144

+ c

11914

.

See

Figure 5.2.2: The feasible region is shown. Note that the region may be made up of a convexcombination of basic feasible solutions plus a direction of unboundedness.

http://www.math.iup.edu/~jchrispe/MATH445_545/DirectionOfUnboundedness.html

for an interactive version of the solution.

35

http://www.math.iup.edu/~jchrispe/MATH445_545/DirectionOfUnboundedness.html


5.3 Implementation of the Simplex Method

A basic feasible solution has been previously defined. We make note that:

For an LP with m constraints, two basic feasible solutions are said to be adjacentprovided their sets of basic variables have m− 1 basic variables in common.

Lets consider the basic ideas behind the Simplex algorithm.

• Start by finding a basic feasible solution to the LP. Call this the current basic feasiblesolution.

• Determine if the current basic feasible solution is optimal.

– If not find an adjacent basic feasible solution with a better objective value, andrepeat this step.

– If yes stop.

Given a LP with m constraints and n variables it can be noted that their are( nm

)=

n!

(n−m)!m!

different basic feasible solutions.

Provided that we only examine each basic feasible solution once it is clear that we wouldfind the solution after examining at most ( n

m

)different solutions. Even for a small problem this is a large number!

The simplex method is actually more efficient, and will find the optimal solution in practicein approximately:

3m iterations!

36


Example:

Lets introduce the simplex algorithm as well as a good book keeping technique known as asimplex tableau using the following example problem:

max z = 2x1 + 5x2

s.t. 3x1 + 8x2 ≤ 12

2x1 + 3x2 ≤ 6

x1, x2 ≥ 0

The first step is putting the problem in standard form by adding slack variables.

max z = 2x1 + 5x2

s.t. 3x1 + 8x2 + s1 = 12

2x1 + 3x2 + s2 = 6

x1, x2 ≥ 0

Lets rewrite the objective function:

max z = 2x1 + 5x2 =⇒ max z − 2x1 − 5x2 = 0

Then a basic simplex tableau is given by:

Row z x1 x2 s1 s2 RHS Basic0 1 -2 -5 0 0 0 z1 0 3 8 1 0 12 s1

2 0 2 3 0 1 6 s2

Note the more negative a value is in the zero row the more attractive it is for a maximizationproblem. Thus negative non-basic variables are attractive to become basic, and the solutionis not currently optimal. Here we will add x2 into the basis. We need to choose a value toleave such that the solution remains feasible. This leads to a ratio test that is used to keepthe RHS from becoming negative.

Ratio: s1

12/8 = 3/2

Ratio: s2

3/6 = 1/2

We choose the minimum of these values to be the basic variable that leaves the basis in placeof x2.

Row z x1 x2 s1 s2 RHS Basic0 1 -0.125 0 0.625 0 7.5 z1 0 0.375 1 0.125 0 1.5 x2

2 0 0.875 0 -0.375 1 1.5 s2

37


Note that the x1 variable is still attractive, and can enter the basis. Do the ratio test to findthe leaving basic variable.

Ratio: x2

1.5/0.375 = 4

Ratio: s2

1.5/0.875 = 1.7142857142857142

We choose s2 to be the leaving variable:

Row z x1 x2 s1 s2 RHS Basic0 1 0 0 0.571428571 0.142857143 7.714285714 z1 0 0 1 0.285714286 -0.428571429 0.857142857 x2

2 0 1 0 -0.428571429 1.142857143 1.714285714 x1

Note here that no non-basic variables are attractive in row zero (have negative coefficients)so optimality has been reached.

38


Example:

Use the simplex algorithm to solve the following LP:

max z = x1 + x2 + x3

s.t. x1 + 2x2 + 2x3 ≤ 20

2x1 + x2 + x3 ≤ 20

2x1 + 2x2 + x3 ≤ 20

x1, x2, x3 ≥ 0

Row z x1 x2 x3 s1 s2 s3 RHS Basic0 1 -1 -1 -1 0 0 0 0 z1 0 1 2 2 1 0 0 20 s1

2 0 2 1 2 0 1 0 20 s2

3 0 2 1 0 0 0 1 20 s3

Choose x1 to enter the basis, and do the ratio test:

Ratio: s1

20/1 = 20

Ratio: s2

20/2 = 10

Ratio: s3

20/2 = 10

Pick s2 to leave the basis.

Row z x1 x2 x3 s1 s2 s3 RHS Basic0 1 0 -0.5 0 0 0.5 0 10 z1 0 0 1.5 1 1 -0.5 0 10 s1

2 0 1 0.5 1 0 0.5 0 10 x1

3 0 0 0 -2 0 -1 1 0 s3

Choose x2 to enter the basis and do the ratio test:

Ratio: s1 20/1.5 =20

3Ratio: x1 10/0.5 = 20

Pick s1 to leave the basis:

Row z x1 x2 x3 s1 s2 s3 RHS Basic0 1 0 0 1/3 1/3 1/3 0 131

3z

1 0 0 1 2/3 2/3 -1/3 0 623

x2

2 0 1 0 2/3 -1/3 2/3 0 623

x1

3 0 0 0 -2 0 -1 1 0 s3

39


Example

Use the simplex algorithm to solve the following linear program:

max z = 3x1 + 5x2

s.t. x1 ≤ 4

2x2 ≤ 12

3x1 + 2x2 ≤ 18

withx1, x2 ≥ 0

We add a slack variable to each constraint to put it into standard form, and then manipulatethe objective function to:

=⇒ z − 3x1 − 5x2 = 0

Take the set of slack variables as basic, then the starting simplex tableau is given by:

Row z x1 x2 s1 s2 s3 RHS Basic0 1 -3 -5 0 0 0 0 z1 0 1 0 1 0 0 4 s1

2 0 0 2 0 1 0 12 s2

3 0 3 2 0 0 1 18 s3

Here we look and see that x2 is the most attractive variable to enter into the basis. Doing aratio test of the values in the column associated with x2 yields:

Ratio Test:s1 : NA, s2 : 12/2 = 6, s3 : 18/2 = 9

Thus, we choose s2 to be the variable leaving the basis, as it has the minimum value in theratio test.

Row z x1 x2 s1 s2 s3 RHS Basic0 1 -3 0 0 2.5 0 30 z1 0 1 0 1 0 0 4 s1

2 0 0 1 0 0.5 0 6 x2

3 0 3 0 0 -1 1 6 s3

Now x1 is still attractive to us, and we let it enter the basis. Conducting the ratio test againgives:

s1 : 4/1 = 4, x2 : NA, s3 : 6/3 = 2

Choose s3 to be the variable leaving the basis.

40


Row z x1 x2 s1 s2 s3 RHS Basic0 1 0 0 0 1.5 1 36 z1 0 0 0 1 1/3 -1/3 2 s1

2 0 0 1 0 0.5 0 6 x2

3 0 1 0 0 -1/3 1/3 2 x1

Here we have reached an optimal solution as coefficients in row zero are all positive. Thesolution is:

x1 = 2, x2 = 6, and s1 = 2 with z = 36.

41


5.4 Simplex for Minimization Problems

If we are dealing with a minimization problem we need to do things a little differently.Two options here:

• Convert the objective function to that of a maximization problem. The solution willbe given by

z = −z

and the algorithm proceeds as we know it.

• Change the way we select the entering variable in the algorithm so that we are lookingto make the entries in row zero “non-positive”. We choose the most positive value toenter into the basis, and pivot as normal.

Example:

min z = 2x1 − 5x2

s.t. 3x1 + 8x2 ≤ 12

2x1 + 3x2 ≤ 6

withx1, x2 ≥ 0

42


Example:

Consider the following linear program:

max z = −3x1 + 6x2

s.t. 5x1 + 7x2 ≤ 35

−x1 + 2x2 ≤ 2

withx1, x2 ≥ 0

43


Example:


max z = x1 + x2

s.t. − x1 + x2 ≤ 2

−2x1 + x2 ≤ 1

withx1, x2 ≥ 0.

Setting up the initial simplex tableau after adding two slack variables:

Row z x1 x2 s1 s2 RHS Basic0 1 -1 -1 0 0 0 z1 0 -1 1 1 0 2 s1

2 0 -2 1 0 1 1 s2

Here we see that x1 and x2 are attractive. Do a pivot on x2 as there is no potential for x1

to leave at the moment. Ratio:

s1 : 2/1 = 2, s2 : 1/1 = 1.

Choose s2 to leave.

Row z x1 x2 s1 s2 RHS Basic0 1 -3 0 0 1 1 z1 0 1 0 1 -1 1 s1

2 0 -2 1 0 1 1 x2

Note that x1 is attractive and the only value that can leave the basis here is s1.

Row z x1 x2 s1 s2 RHS Basic0 1 0 0 3 -2 4 z1 0 1 0 1 -1 1 s1

2 0 0 1 2 -1 3 x2

Note that the s2 variable is still attractive; however, the values in the tableau below it areboth negative. Thus, we have no candidate to leave via the ratio test. Mathematically thisyields:

x1 = s2 + 1

x2 = s2 + 3

Note that we don’t care at all about the value of s1 in these expressions, and for any non-negative value of s2 we pick positive values of x1 and x2 may be found that satisfy these

44


constraints. This yields a direction of unboundedness, as for any x in the feasible regionS, then

x + cd ∈ S.

So what is the direction?

d =

1101

The tableau suggests that for each unit that the value of s2 is increased, the values for x1

and x2 (seen as the negative of the values in the ‘s2’ column in the final tableau) should alsobe increased by one unit. Note that

cTd ≥ 0

Example:

Find a direction of unboundedness for the following LP.

min z = −x1 − 3x2

s.t. x1 − x2 ≤ 4

−x1 + x2 ≤ 3

withx1, x2 ≥ 0

45


Example: Find a direction of unboundedness for the following LP.

min z = −5x1 − 7x2

s.t. − 3x1 + 2x2 ≤ 30

−2x1 + x2 ≤ 12

withx1, x2 ≥ 0

This is a min problem so we want Row zero to be non positive when we are done.

Row x1 x2 s1 s2 RHS Basis0 5 7 0 0 0 z1 -3 2 1 0 30 s1

2 -2 1 0 1 12 s2

Choose x2 to enter.

Row x1 x2 s1 s2 RHS Basis0 19 0 0 -7 -84 z1 1 0 1 -2 6 s1

2 -2 1 0 1 12 x2

Choose x1 to enter and s1 to leave the basis.

Row x1 x2 s1 s2 RHS Basis0 0 0 -19 31 -198 z1 1 0 1 -2 6 x1

2 0 1 2 -3 24 x2

Here we have found a direction of unboundedness for the problem. Starting at the basicfeasible solution:

X =

62400

we can head forever in the direction

d =

2301

and have the solution remain feasible and decreasing the value of z without bound.

46


5.5 Big-M Method

Consider solving the following linear program:

min z = 2x1 + 3x2

s.t.1

2x1 +

1

4x2 ≤ 4

x1 + 3x2 ≥ 20

x1 + x2 = 10

withx1, x2 ≥ 0

Moving this problem into standard form gives:

min z = 2x1 + 3x2

s.t.1

2x1 +

1

4x2 + s1 = 4

x1 + 3x2 − e1 = 20

x1 + x2 = 10

withx1, x2, s1, e1 ≥ 0

The idea is to now add artificial variables to all the constraints without slack variables. Thiswill allow us to have a starting basic feasible solution. To ensure that these artificial variablesdo not find there way into the final solution basis we penalize the objective function for havingthem. As this is a minimization problem we will add Mai into the objective function foreach of the i artificial variables added. Note that M is a large positive constant. For ourstated example problem this yields:

min z = 2x1 + 3x2 +Ma1 +Ma2

s.t.1

2x1 +

1

4x2 + s1 = 4

x1 + 3x2 − e1 + a1 = 20

x1 + x2 + a2 = 10

withx1, x2, s1, e1, a1, a2 ≥ 0

Translate this into an initial simplex tableau to obtain:

47


Row z x1 x2 s1 e1 a1 a2 RHS Basis0 1 -2 -3 0 0 −M −M 0 z1 0 0.5 0.25 1 0 0 0 4 s1

2 0 1 3 0 -1 1 0 20 a1

3 0 1 1 0 0 0 1 10 a2

Adjust the zero row to reflect that a1 and a2 are basic variables.

Row z x1 x2 s1 e1 a1 a2 RHS Basis0 1 2M − 2 4M − 3 0 −M 0 0 30M z1 0 1/2 1/4 1 0 0 0 4 s1

2 0 1 3 0 -1 1 0 20 a1

3 0 1 1 0 0 0 1 10 a2

Here we see that x2 is the most positive and will enter the basis. The ratio test yields thata1 should leave the basis.

Row z x1 x2 s1 e1 a1 a2 RHS Basis0 1 (2/3)M − 1 0 0 (1/3)M − 1 −(4/3)M + 1 0 (10/3)M + 20 z1 0 5/12 0 1 1/12 -1/12 0 7/3 s1

2 0 1/3 1 0 -1/3 1/3 0 20/3 x2

3 0 2/3 0 0 1/3 -1/3 1 10/3 a2

Here x1 is attractive. Doing the ratio test we see that a2 will leave the basis.

Row z x1 x2 s1 e1 a1 a2 RHS Basis0 1 0 0 0 -1/2 −M + 1/2 −M + 3/2 25 z1 0 0 0 1 -1/8 1/8 -5/8 1/4 s1

2 0 0 1 0 -1/2 1/2 -1/3 5 x2

3 0 1 0 0 1/2 -1/2 3/2 5 x1

All values in row zero are now non-positive giving an optimal solution has been found. Thereare no alternate optimal solutions. Thus,

z = 25, with x1 = 5, x2 = 5, s1 = 1/4, and e1 = a1 = a2 = 0

Note we could have dropped the artificial variables from the tableau once the leave the basis.If we still have an artificial variable that is basic when an optimal solution is found then thelinear program does not have a feasible solution.

48


Second Example of the Big M-Method.

min z = 3x1

s.t. 2x1 + x2 ≥ 6

3x1 + 2x2 = 4

withx1, x2 ≥ 0

Here we put the problem into standard form to obtain.

min z = 3x1 +Ma1 +Ma2

s.t. 2x1 + x2 − e1 + a1 = 6

3x1 + 2x2 + a2 = 4

withx1, x2, e1, a1, and a2 ≥ 0.

Translating the problem over into tableau notation yields:

Row z x1 x2 e1 a1 a2 RHS Basis0 1 -3 0 0 −M −M 0 z1 0 2 1 -1 1 0 6 a1

2 0 3 2 0 0 1 4 a2

Adjust the tableau to reflect that a1 and a2 are basic.

Row z x1 x2 e1 a1 a2 RHS Basis0 1 5M − 3 3M −M 0 0 10M z1 0 2 1 -1 1 0 6 a1

2 0 3 2 0 0 1 4 a2

Here x1 is the most positive value and is picked to enter the basis. Doing a ratio test we findthat a2 is picked to leave the basis.

Row z x1 x2 e1 a1 a2 RHS Basis0 1 0 (4/3)M + 2 −M 0 (−5/3)M + 1 (10/3)M + 4 z1 0 0 -1/3 -1 1 -2/3 10/3 a1

2 0 1 2/3 0 0 1/3 4/3 x1

Here x2 is the most positive value and is picked to enter the basis. Doing a ratio test we seethat x1 is picked to leave the basis.

49


Row z x1 x2 e1 a1 a2 RHS Basis0 1 −2M − 3 0 −M 0 (−7/3)M (2/3)M z1 0 1/2 0 -1 1 -1/2 4 a1

2 0 3/2 1 0 0 1/2 2 x2

Nothing is positive and a1 is still basic. This leads to an infeasible solution. Consider plottingthe feasible region to see this.

50


5.6 Degeneracy and Convergence

All the linear programs we have encountered so far have been “nice" in that the basic feasiblesolutions have not been degenerate.

An Linear Program is degenerate if it has at least one basic feasible solution in which abasic variable is equal to zero.

• Multiple constraints intersecting to give the same basic feasible solution. (Draw in2D.)

• These can be seen in the RHS values with a non-optimal basic feasible solution havinga value of zero.

A cycling example

Consider the following maximization problem:

max z = 2x1 + 3x2 − x3 + 12x4

s.t.− 2x1 − 9x2 + x3 + 9x4 ≤ 01

3x1 + x2 −

1

3x3 − 2x4 ≤ 0

This leads to the initial tableau:

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 -2 -3 1 12 0 0 0 z1 0 -2 -9 1 9 1 0 0 s12 0 1

31 -1

3-2 0 1 0 s2

The ratio test has x2 enters in place of s2.

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 -1 0 0 6 0 3 0 z1 0 1 0 -2 -9 1 9 0 s1

2 0 13

1 -13

-2 0 1 0 x2

Assume we break ties in the ratio test always choosing row 1 . Then the next tableau isgiven by:

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 0 0 -2 -3 1 12 0 z1 0 1 0 -2 -9 1 9 0 x1

2 0 0 1 13

1 -13

-2 0 x2

51


On the next pivot x4 enters in place of x2.

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 0 3 -1 0 0 6 0 z1 0 1 9 1 0 -2 -9 0 x1

2 0 0 1 13

1 -13

-2 0 x4

Now x3 enters in place of x1 again we assume row 1 is used as the tie breaker.

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 1 12 0 0 -2 -3 0 z1 0 1 9 1 0 -2 -9 0 x3

2 0 -13

-2 0 1 13

1 0 x4

Next s2 will now enter for s1 in the next pivot.

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 0 6 0 3 -1 0 0 z1 0 -2 -9 1 9 1 0 0 x3

2 0 -13

-2 0 1 13

1 0 s2

Here s1 enters in place of x3 and we are back to the beginning.

Row z x1 x2 x3 x4 s1 s2 RHS Basis0 1 -2 -3 1 12 0 0 0 z1 0 -2 -9 1 9 1 0 0 s1

2 0 13

1 -13

-2 0 1 0 s2

Continuing in the standard fashion would result in the cycle that was just observed. In orderto prevent cycling in the simplex method we need to consider other strategies for choosingthe variables that will enter and exit the basis at each pivot.

Cycling in the simplex method can be avoided by following Bland’s Rule. Assuming thatthe slack excess and artificial variables are number consecutively after the decision variables:

• Always choose the entering variable with the smallest subscript.

• When there is a tie in the ratio test choose the variable leaving the basis has thesmallest subscript.

52


5.7 Two-Phase Simplex

Two phase simplex gives a way to find an initial basic feasible solution for a linear pro-gramming problem. The procedure is similar to that of the Big M-method; however, thisprocedure is nicer when coded as it avoids scaling issues when adding numbers that aredifferent orders of magnitude.

Implementation

As the name suggests the solution of linear programming problems are separated into twoparts. Put the desired linear program into standard form.

• Phase 1: Ignore the original objective function and minimize the sum of the artificialvariables used in the problem. If the sum of the artificial variables is zero proceed toPhase 2.

• Adjust the original objective function and continue with simplex method as normal.


min z = 2x1 + 3x2

s.t. 3x1 + 2x2 = 14

2x1 − 4x2 ≥ 2

4x1 + 3x2 ≤ 19

withx1, x2 ≥ 0

53


Try the following example using the two phase simplex method.

max z = 3x1 + x2

s.t. x1 + x2 ≥ 3

2x1 + x2 ≤ 4

x1 + x2 = 3

withx1, x2 ≥ 0

54


Example: Consider solving the following LP using the two phase simplex algorithm.

min z = −4x1 − 2x2

s.t. 3x1 − 2x2 ≥ 4

−2x1 + 3x2 = 2

withx1, x2 ≥ 0

Beginning Phase 1 gives the following tableau:

Row z x1 x2 e1 a1 a2 RHS Basic0 1 0 0 0 -1 -1 0 z1 0 3 -2 -1 1 0 4 a1

2 0 -2 3 0 0 1 2 a2

Adjusting to represent that a1 and a2 are basic gives:

Row z x1 x2 e1 a1 a2 RHS Basic0 1 1 1 -1 0 0 6 z1 0 3 -2 -1 1 0 4 a1

2 0 -2 3 0 0 1 2 a2

As this is a minimization of the artificial variables we look for positive values as beingattractive. We see that x1 will enter the basis. Conducting the ratio test:

a1 : 4/3 and a2 : NA

Row z x1 x2 e1 a1 a2 RHS Basic0 1 0 5/3 -2/3 -1/3 0 14/3 z1 0 1 -2/3 -1/3 1/3 0 4/3 x1

2 0 0 5/3 -2/3 2/3 1 14/3 a2

Here it can be seen that x2 is attractive, and the only value that may leave the basis is a2.

Row z x1 x2 e1 a1 a2 RHS Basic0 1 0 0 0 -1 -1 0 z1 0 1 0 -0.6 0.6 0.4 3.2 x1

2 0 0 1 -0.4 0.4 0.6 2.8 x2

This ends phase one of the algorithm enter phase 2.

Row z x1 x2 e1 RHS Basic0 1 4 2 0 0 z1 0 1 0 -0.6 3.2 x1

2 0 0 1 -0.4 2.8 x2

55


Adjust to show that x1 and x2 are basic.

Row z x1 x2 e1 RHS Basic0 1 0 0 3.2 -18.4 z1 0 1 0 -0.6 3.2 x1

2 0 0 1 -0.4 2.8 x2

Note here that the excess variable is attractive; however, there is no potential to pivot. Wesee this as a direction of unboundedness given by

d = (0.6, 0.4, 1)T .

56

Chapter 6

Simplex Method Using Matrix-VectorFormulas

“Learn to be positive, it’s your only chance.”

−The Kinks

Lets consider looking at the simplex algorithm using matrices and compare that with thetableau format we have been using.

6.1 Problem Using Tableaus



subject to x1 ≤ 42x2 ≤ 12

3x1 + 2x2 ≤ 18x1, x2 ≥ 0.

Solution full Tableau Form:Adding slack variables in the constraints the initial tableau for the given linear program iswritten as:

Initial Tableau:

Row z x1 x2 s1 s2 s3 RHS Basis0 1 -3 -5 0 0 0 0 z1 0 1 0 1 0 0 4 s1

2 0 0 2 0 1 0 12 s2

3 0 3 2 0 0 1 18 s3

57


In the first pivot x2 enters the basis replacing s2. Here we have not explicitly shown theratio test. Thus,

Pivot 1 Tableau:

Row z x1 x2 s1 s2 s3 RHS Basis0 1 -3 0 0 5

20 30 z

1 0 1 0 1 0 0 4 s1

2 0 0 1 0 12

0 6 x2

3 0 3 0 0 -1 1 6 s3

The next iteration of the simplex method shows that it is desirable to have x1 enter the basisand replace s3. (Note again we are not showing the ratio text.) Doing this pivot yields:

Pivot 2 Tableau:

Row z x1 x2 s1 s2 s3 RHS Basis0 1 0 0 0 3

21 36 z

1 0 0 0 1 13

-13

2 s1

2 0 0 1 0 12

0 6 x2

3 0 1 0 0 -13

13

2 x1

Here we have obtained an optimal tableau. We can see here that z is maximized at 36 whenx1 = 2 and x2 = 6.

6.2 Matrix Format of Linear Program

A standard form linear program in matrix form is written as:

maximize z = cTxsubject to Ax = b

x ≥ 0.

Thus, for the example problem we make the following definitions:

x =

x1

x2

s1

s2

s3

, c =

35000

, b =

41218

and

A =

1 0 1 0 00 2 0 1 03 2 0 0 1

.

In order to implement the simplex algorithm in matrix format we break the given coefficientmatrix A into to parts: a basic part and a non-basic part. Thus,

A = [N |B]

58


and similarly we separate the vectors x, and c into their basic and non-basic parts:

x =

(xNxB

), where xN =

(x1

x2

)and xB =

s1

s2

s3

c =

(cNcB

), where cN =

(35

)and cB =

000

.

We can now make note of the correspondence between the simplex method as we know itin tableau format, and the defined matrices. Any current basis may be written as:

xNT xB

T RHS Basis−cNT + cB

TB−1N 0 cBTB−1b z

B−1N I B−1b xB

6.2.1 Initial Tableau (Matrix Form):

For the starting basis xB = (s1 s2 s3)T , and xN = (x1 x2)T we have:

N =

1 00 23 2

, B =

1 0 00 1 00 0 1

=⇒ B−1 =

1 0 00 1 00 0 1

.with

cNT =

(3 5

)cB

T =(

0 0 0)

Evaluating each of the expressions in the tableau we have:

−cNT + cBTB−1N =

(−3 −5

)cB

TB−1b =(

0)

B−1b =

41218

B−1N =

1 00 23 2

59


Ratio Test:

Note that both non-basic variables are attractive, and we choose x2 to enter the basis. Doingthe ratio test using the second column of B−1N and the current right hand side vector B−1b:

min

{12

2= 6,

18

2= 9

}and we choose s2 to leave the basis. Note we have ignored the comparison of 4 over 0 in theratio test.

Compare the above values with the tableau:

Initial Tableau:


2 0 0 2 0 1 0 12 s2

3 0 3 2 0 0 1 18 s3

Pivot 1 Tableau (Matrix Form):

Note we have now replaced s2 in the basis with x2. Thus, xB = (s1 x2 s3)T , and xN =(x1 s2)T we have:

N =

1 00 13 0

, B =

1 0 00 2 00 2 1

=⇒ B−1 =

1 0 00 1

20

0 −1 1

.

with

cNT =

(3 0

)cB

T =(

0 5 0)



(−3 5

2

)cB

TB−1b =(

30)

B−1b =

466

B−1N =

1 00 1

2

3 −1

60


Ratio Test:

Note that the non-basic variable x1 should be picked to enter the basis. Doing the ratio testusing the first column of B−1N and the current right hand side vector B−1b:

min

{4

1= 4,

6

3= 2

}and we choose s3 to leave the basis. Note we have ignored the comparison of 6 over 0 wherex2 would potentially leave the basis in the ratio test.

Compare values with the Piviot 1 Tableau:

Pivot 1 Tableau:


20 30 z

1 0 1 0 1 0 0 4 s1

2 0 0 1 0 12

0 6 x2

3 0 3 0 0 -1 1 6 s3

Pivot 2 Tableau (Matrix Form):

Note we have now replaced s3 in the basis with x1. Thus, xB = (s1 x2 x1)T , and xN =(s3 s2)T we have:

N =

0 00 11 0

, B =

1 0 10 2 00 2 3

=⇒ B−1 =

1 13−1

3

0 12

00 −1

313

.

with

cNT =

(0 0

)cB

T =(

0 5 3)



(1 3

2

)cB

TB−1b =(

36)

B−1b =

262

B−1N =

−13

13

0 12

13−1

3

Note that the non-basic variables are nolonger attractive and we have reached an optimalsolution to the problem with z = 36 when, x1 = 2 and x2 = 6.

Compare with the Pivot 2 Tableau:

61


Pivot 2 Tableau:


21 36 z

1 0 0 0 1 13

-13

2 s1

2 0 0 1 0 12

0 6 x2

3 0 1 0 0 -13

13

2 x1

62


Class Exercise:

Making use of the following:

xNT xB



B−1N I B−1b xB

and knowing that the linear program:

maximize z = x1 + 2x2 + 4x3

subject to 7x1 + 3x2 + 1x3 ≤ 30x1 − 3x2 + 7x3 ≤ 50

2x1 + 4x2 + 8x3 ≤ 70x1, x2, x3 ≥ 0.

has an optimal solution when x2, x3, and the slack variable for the second constraint arebasic, classify all the optimal solutions to the linear program. You should do this by findingthe simplex tableau associated with the given basis, and doing appropriate computations tofind additional solutions if needed.

Associating s1, s2 and s3 as the slack variable for each of the problem constraints we notethat the original A matrix for the problem has the following entries:

x1 x2 x3 s1 s2 s3

A =

7 3 1 1 0 01 −3 7 0 1 02 4 8 0 0 1

c =

124000

b =

305070

Solution 1:Using the given optimal basis yields xB = (x2 x3 s2)T , and xN = (x1 s1 s3)T and we have:

N =

7 1 01 0 02 0 1

, B =

3 1 0−3 7 1

4 8 0

=⇒ B−1 =

25

0 − 120

−15

0 320

135

1 −65

.

with

cNT =

(1 0 0

)cB

T =(

2 4 0)

63




(0 0 1

2

)cB

TB−1b =(

35)

B−1b =

17292

44

B−1N =

2710

25− 1

20

−1110−1

5320

845

135−6

5

This yields an optimal solution given by:

X1 =

017292

0440

It can be seen that x1 and s1 have a zero value in row zero and could enter the basis yieldingalternate optimal solutions. Choosing x1 to enter we can conduct the ratio test:

x2 :(17/2)

(27/10=

85

27x3 :

(9/2)

NAs2 :

44

84/5=

55

21

and we choose s2 to leave the basis.

Solution 2:Updating to the alternate optimal basis yields xB = (x2 x3 x1)T , and xN = (s2 s1 s3)T andwe have:

N =

0 1 01 0 00 0 1

, B =

3 1 7−3 7 1

4 8 2

=⇒ B−1 =

− 156− 9

5617

− 5168

11168

114

1384

584− 1

14

.

with

cNT =

(0 0 0

)cB

T =(

2 4 1)

64




(0 0 1

2

)cB

TB−1b =(

35)

B−1b =

107

155215521

B−1N =

− 956− 1

5617

11168− 5

168114

584

1384− 1

14

This gives the optimal solution:

X2 =

5521

15521

92

000

Here we see that s1 is still a candidate to enter into the basis and we conduct the ratio testto see what will leave.

x2 :(10/7)

NAx3 :

(155/21)

NAx1 :

55/21

13/84=

220

13

and we choose x1 to leave the basis.

Solution 3:Updating to the alternate optimal basis yields xB = (x2 x3 s1)T , and xN = (s2 x1 s3)T andwe have:

N =

0 7 01 1 00 2 1

, B =

3 1 1−3 7 0

4 8 0

=⇒ B−1 =

0 − 213

752

0 113

352

1 513− 6

13

.

with

cNT =

(0 1 0

)cB

T =(

2 4 0)

65




(0 0 1

2

)cB

TB−1b =(

35)

B−1b =

4526

2052622013

B−1N =

− 213

326

752

113

526

352

513

8413− 6

13

This gives the optimal solution:

X3 =

0

4526

20526

22013

00

Classifying all optimal solutions to the problem we let αi ∈ [0, 1] such that

∑3i=1 αi = 1 and

write the optimal solution as the convex set of all points:

S = α1X1 + α2X2 + α3X3.

66


Example:

Use the matrix simplex method to find the optimal solution to the following LP:


subject to x1 + 3x2 ≤ 63x1 + 2x2 ≤ 6x1, x2 ≥ 0.

67


68

Chapter 7

Duality

“I’d like to change your mind by hitting it with a rock.”

−They Might Be Giants

As is often the case you may look at problems form many different perspectives. Thisis the case with linear programs. Lets consider a problem form our past in two differentperspectives. So far we have been looking at linear programs in what we will call the primalform.

7.1 A Motivating Example: My Diet Problem:

Recall from earlier this semester that I have a really bad diet that requires that all of myfood come from my favourite food groups: pez candy, pop tarts, Cap’N crunch, and cookies.I have the following foods available for my consumption: red pez, strawberry pop-tarts,peanut butter crunch, and chocolate chip cookies. The red pez costs $0.20 per (per package),strawberry pop-tarts cost $0.50 (per package), peanut butter crunch costs $0.70 per bowl,and chocolate chip cookies cost $0.65 cents each. I must ingest at least 2500 calories a dayin order to maintain my sugary lifestyle. I must also meet the following requirements I need8 oz. of sugar, 10 oz. of fat, and 50 mg of yellow-5 food coloring. Each of my chosen foodshas these required nutrients in the following quantities:

Food Calories Sugar (oz.) Fat (oz.) yellow-5 (mg) Price ($)pez candy (per package) 50 0 .1 0.01 5 0.20pop-tarts (per package) 250 0.5 0.3 0 0.50Cap’N crunch (per bowl) 350 1 1 4 0.70cookies (1 cookie) 150 0.3 0.7 1 0.65

How can I achieve my dietary constraints at a minimum cost?

69


Primal Solution To Diet Problem:

The first step will be do define some decision variables. Here we will define

x1 as the number of pez packages to include in my diet per dayx2 as the number strawberry pop-tart packages to consume per dayx3 as the number of bowls of peanut butter crunch to consume per dayx4 as the number of chocolate chip cookies to consume each day

Our objective function can then be defined as:

min z = 0.20x1 + 0.50x2 + 0.70x3 + 0.65x4

which will minimize the cost of our diet. The next aim is to meet the special dietary needs.Here a constraint is set up for each of the “nutrients” (calories, sugar, fat, and yellow 5).

50x1 + 250x2 + 350x3 + 150x4 ≥ 2500 (calories required)0.1x1 + 0.5x2 + x3 + 0.3x4 ≥ 8 (sugar required)

0.01x1 + 0.3x2 + x3 + 0.7x4 ≥ 10 (fat required)5x1 + 0x2 + 4x3 + x4 ≥ 50 (yellow-5 required)

It can also be noted that the decision variables will be non-negative in sign giving the

xi ≥ 0 for all i ∈ {1, 2, 3, 4}

sign restriction.

This problem can be set up in vector notation using the above variable definitions anddefining

x =

x1

x2

x3

x4

, c =

0.200.500.700.65

, b =

2500

81050

, and

A =

50 250 350 1500.1 0.5 1 0.30.01 0.3 1 0.7

5 0 4 1

.The optimal diet can now be found by considering:

min z = cTx

s.t. Ax ≥ b

with x ≥ 0.

70


Dual of Diet Problem

A second way to consider the problem. Suppose that I know a nutrient salesman (WillyLoman) who will sell me supplements that taste just as good as the items in my diet.Specifically Mr. Loman sells calories, sugar, Fat, and yellow-5 and wants to sell me theseitems in order to meet my daily needs at the maximum price I’m willing to pay. Thendefining the decision variables:

y1 is the price to charge me per calorie.

y2 is the price to charge me per ounce of sugar.

y3 is the price to charge me per ounce of fat.

y4 is the price to charge me per mg of yellow-5.

Loman’s objective function for my diet would look like:

max w = 2500y1 + 8y2 + 10y3 + 50y4

The constraints for the salesman are found using the available foods. Mr. Loman being agreat salesman needs to set his prices low enough that I will purchase his nutrients ratherthan my regular diet of pez and cookies. Thus, he is subject to the constraints:

50y1 + 0.1y2 + 0.01y3 + 5y4 ≤ 0.20 ( the price of pez)250y1 + 0.5y2 + 0.3y3 + 0y4 ≤ 0.50 ( the price of pop-tarts)

350y1 + y2 + y3 + 4y4 ≤ 0.70 ( the price of Cap’N crunch)

150y1 + 0.3y2 + 0.7yy + y4 ≤ 0.65 ( the price of cookies)

where the sign restriction on the decision variables is:

yi ≥ 0 for all i ∈ {1, 2, 3, 4}

Writing this in matrix form in terms of the values defined for the primal problem we have:

max w = bTy

s.t. ATy ≤ c

with y ≥ 0

wherey = (y1 y2 y3 y4)T .

These two linear programs have the same optimal solution values.

71


7.1.1 Canonical Form

The canonical form of a primal and its corresponding dual linear program are given as follows:

min z = cTx

s.t. Ax ≥ b

with x ≥ 0.

max w = bTy

s.t. ATy ≤ c

with y ≥ 0

Primal Dual

Taking Dual of General Linear Programs

What if we don’t start in canonical form? Lets consider the constraints first:

min z = cTx

s.t. A1x ≥ b1

A2x ≤ b2

A3x = b3

with x ≥ 0.

Transform the above problem into standard canonical form (ignoring that the RHS may benegative for a moment).

min z = cTx

s.t. A1x ≥ b1

−A2x ≥ −b2

A3x ≥ b3

−A3x ≥ −b3

with x ≥ 0.

Taking the dual we have:

max w = bT1 y1 − bT2 y′2 + bT3 y

′3 − bT3 y

′′3

s.t. AT1 y1 − AT2 y′2 + AT3 y′3 − AT3 y′′3 ≤ c

with y1,y′2,y

′3,y

′′3 ≥ 0.

Making the change of variables:

y2 = −y′2 and y3 = y′3 − y′′3

and we have:

max w = bT1 y1 + bT2 y2 + bT3 y3

s.t. AT1 y1 + AT2 y2 + AT3 y3 ≤ c

with y1 ≥ 0,y2 ≤ 0,y3 is free

72


What if the variables are not in standard form?

min z = cT1 x1 + cT2 x2 + cT3 x3

s.t. A1x1 + A2x2 + A3x3 ≥ b

with x1 ≥ 0,x2 ≤ 0,x3 is free

Here we make a change of variables and place the problem back into a canonical form. Let

x2 = −x′2 and x3 = x3′ − x′′3

withx′2,x

′3, and x′′3 ≥ 0.

Then

min z = cT1 x1 − cT2 x′2 + cT3 x

′3 − cT3 x

′′3

s.t. A1x1 − A2x′2 + A3x

′3 − A3x

′′3 ≥ b

with x1,x′2,x

′3,x

′′3 ≥ 0

We can now take the Dual:

max z = bTy

s.t. AT1 y ≤ c1

−AT2 y ≤ −c2

AT3 y ≤ c3

−AT3 y ≤ −c3

with y ≥ 0

Adjusting the RHS to be non-negative:

max z = bTy

s.t. AT1 y ≤ c1

AT2 y ≥ c2

AT3 y = c3

with y ≥ 0

Relationship Between Primal and Dual

The following table summarizes the relationship between the primal and the dual problem.

Primal / Dual Constraint Dual/Primal Variableconsistent with canonical form ⇐⇒ variable ≥ 0reversed from canonical form ⇐⇒ variable ≤ 0

equality constraint ⇐⇒ variable is free

73


Examples:

As an exercise find the dual of the following linear programs:

• Problem 1:

max z = x1 + x2

s.t. x1 − x2 ≤ 1

with x1, x2 ≥ 0

We need a dual variable y for each constraint. Thus,

min w = y1

s.t. y1 ≥ 1

−y1 ≥ 1

with y2 ≥ 0.

• Problem 2:

min z = 4x1 − 9x2 + 10x3

s.t. 5x1 + 6x2 + 7x3 ≥ 3

3x1 + 2x2 + 1x3 ≤ 4

−1x1 + 8x2 + 2x3 ≤ 5

with x1 ≥ 0, x2 ≤ 0, x3 is free

And the dual of the linear program is:

max z = 3y1 + 4y2 + 5y3

s.t. 5y1 + 3y2 − y3 ≤ 4

6y1 + 2y2 + 8y3 ≥ −9

7y1 + y2 + 2y3 = 10

with y1 ≥ 0, y2 ≤ 0, y3 ≤ 0

74

Chapter 8

Basic Duality Theory

“Read dozens of books about heroes and crooks, and I learned much from both of their

styles.”

− Jimmy Buffet

Here we will consider the major results that relate the primal and the dual linear program-ming problems.

8.1 Relationship Between Primal and Dual

As a warm up lets consider finding the dual of the linear program:

max z = 2x1 + 3x2 + 4x3

s.t. 5x1 + 6x2 + 7x3 = 8

9x1 + 10x2 + 11x3 ≥ 12

13x1 + 14x2 + 15x3 ≤ 16

with x1 ≥ 0, x2 ≤ 0, x3 is free

Primal / Dual Constraint Dual / Primal Variableconsistent with canonical form ⇐⇒ variable ≥ 0reversed from canonical form ⇐⇒ variable ≤ 0

equality constraint ⇐⇒ variable is free

min z = 8y1 + 12y2 + 16y3

s.t. 5y1 + 9y2 + 13y3 ≥ 2 ( as x1 ≥ 0 in primal problem)

6y1 + 10y2 + 14y3 ≤ 3 ( as x2 ≤ 0 in primal problem)

7y1 + 11y2 + 15y3 = 4 ( as x3 is free in primal problem)

with y1 is free, y2 ≤ 0, y3 ≥ 0

75


What is the dual of a problem that is written in our typical standard form?

min z = cTx

s.t. Ax = b

with x ≥ 0

Here we write the dual as:

max w = bTy

s.t. ATy ≤ c ( as x ≥ 0 in primal problem).

with y is free

Graph the following Example:

max z = 2x1 + x2

s.t. x1 ≤ 1

x2 ≤ 1

with x1 and x2 ≥ 0

Taking the dual:

min w = y1 + y2

s.t. y1 ≥ 2

y2 ≥ 1

with y1 and y2 ≥ 0

Graphing the feasible region for each illustrates a nice relationship between the primal andthe dual problem. The following plot illustrates the feasible region for each:

76


Both linear programs are optimal when an objective function value of 3 is obtained.

(z = 3, when x1 = 1, and x2 = 1)

(w = 3, when y1 = 2, and y2 = 1)

8.2 Weak Duality

One of the major results of relating the two linear programs is “Weak Duality”. Here theprimal objective values provide bounds for the dual objective values, and vice versa. Considerthe primal problem to be the minimization problem. Then,

Theorem 8.2.1 (Weak Duality) Let x be a feasible point for the primal problem in stan-dard form, and let y be a feasible point for the dual problem. Then,

z = cTx ≥ bTy = w.

Proof: From the dual problem’s constraints and the primal problems sign restrictions wehave

cT ≥ yTA and x ≥ 0.

Thus,z = cTx ≥ (yTA)x = yTb = bTy = w

=⇒ z ≥ w

�

Note this means that for a general primal dual min max pair that a feasible solution for theminimization problem will always have an objective function value that is greater than orequal to the objective function value for a feasible point in the dual maximization problem.Thus,

• If the primal is unbounded then the dual is infeasible. If the dual is unbounded thenthe primal is infeasible.

– In general if the primal is infeasible the dual may be infeasible or unbounded.

• If x is a feasible solution to the primal problem, y is a feasible solution to the dual,and cTx = bTy then x and y are optimal for their respective problems.

Example: Form last class we took the dual of the linear program:

max z = x1 + x2

s.t. x1 − x2 ≤ 1

with x1, x2 ≥ 0

77


It can be seen that this linear program is unbounded. Taking the dual gave us:

min w = y1

s.t. y1 ≥ 1

−y1 ≥ 1

with y1 ≥ 0.

Note that this LP is infeasible.

Primal feasible region is unbounded, so the dual will be infeasible.

8.3 Strong Duality

Consider a pair of primal and dual linear programming problems.

Theorem 8.3.1 (Strong Duality) If one of the problems has an optimal solution then sodoes the other, and the optimal objective values are equal.

Proof: With out loss of generality we can make the following assumptions:

• The primal problem has an optimal solution.

• The primal problem is in standard form (min problem).

• x∗ the solution to the primal problem is an optimal basic feasible solution.

Here we write x∗ in terms of basic and non-basic variables:

x∗ =

(xBxN

)78


and correspondingly we can write

A = [B N ], and c =

(cBcN

).

Recall any current basis may be written as:

xNT xB



B−1N I B−1b xB

Then we can note that xB = B−1b., and x∗ is optimal if

−cNT + cBTB−1N ≤ 0 =⇒ cB

TB−1N ≤ cNT

the reduced costs for the non-basic variables are negative.

For the dual problem we let y∗ = (B−1)TcB.

y∗ = (B−1)TcB =⇒ y∗T = cTBB

−1

The goal now is to show that

• y∗ is feasible for the dual

• bTy∗ = cTx∗ (the objective functions have the same value).

For feasibility we need to show ATy∗ ≤ c. Lets start with

y∗TA = cTBB

−1(B N)

=(cTB cTBB

−1N)

≤(cTB cTN

)= cT

Thus,ATy∗ ≤ c

and we have a feasible solution for the dual.

We now need to compare the value of the dual’s objective function with the value of theoptimal primal objective function:

z = cTx∗ = cTBxB = cTBB−1b

w = bTy∗ = y∗Tb = cTBB

−1b

So y∗ is feasible in the dual and has the same objective value as the optimal primal problemobjective value so y∗ is optimal for the dual.

�

79


More on Duality Theory

Primal Problem

Before starting into additional theory lets consider the following LP and for practice take itsdual.

min z = 20x1 + 15x2 + 54x3

s.t. x1 − 2x2 + 6x3 ≥ 30

x2 + 2x3 ≥ 6

2x1 − 3x3 ≥ −5

x1 − x2 ≥ 18

with x1, x2, x3 ≥ 0

The associated Dual problem is:

max w = 30y1 + 6y2 − 5y3 + 18y4

s.t. y1 + 2y3 + y4 ≤ 20

−2y1 + y2 − y4 ≤ 15

6y1 + 2y2 − 3y3 ≤ 54

with y1, y2, y3, y4 ≥ 0

Insight By Example

Lets consider working through an example start to finish in both the primal and dual for-mulation and see how the two are related.

Primal Problem:

min z = 2x1 + 9x2 + 3x3

s.t. − 2x1 + 2x2 + x3 ≥ 1

x1 + 4x2 − x3 ≥ 6

with x1, x2, x3 ≥ 0

Solving the Primal Problem

Adding excess variables and artificial variables to each of the primal constraints we mayobtain the following initial simplex tableau. Note we will use the two-phase simplex algorithm

80


as we are using artificial variables. The objective function for the first phase of the algorithmis:

min zp1 = a1 + a2

where the index on each artificial variable denotes the constraint to which it corresponds.

Row z x1 x2 x3 e1 e2 a1 a2 RHS Basis0 1 0 0 0 0 0 -1 -1 0 z1 0 -2 2 1 -1 0 1 0 1 a1

2 0 1 4 -1 0 -1 0 1 6 a2

Adjust the initial Phase one tableau so that a1 and a2 are truly basic.

Row z x1 x2 x3 e1 e2 a1 a2 RHS Basis0 1 -1 6 0 -1 -1 0 0 7 z1 0 -2 2 1 -1 0 1 0 1 a1

2 0 1 4 -1 0 -1 0 1 6 a2

Here we see that x2 is attractive and should enter the basis by replacing a1.

Row z x1 x2 x3 e1 e2 a1 a2 RHS Basis0 1 5 0 -3 2 -1 -3 0 4 z1 0 -1 1 0.5 -0.5 0 0.5 0 0.5 x2

2 0 5 0 -3 2 -1 -2 1 4 a2

Next we see that x1 is attractive and should enter the basis in the place of a2.

Row z x1 x2 x3 e1 e2 a1 a2 RHS Basis0 1 0 0 0 0 0 -1 -1 0 z1 0 0 1 -0.1 -0.1 -0.2 0.1 0.2 1.3 x2

2 0 1 0 -0.6 0.4 -0.2 -0.4 0.2 0.8 x1

This completes the first phase of the two-phase simplex method. We can place back in theoriginal objective function, and drop the columns that correspond to the artificial variables.

min zp2 = 2x1 + 9x2 + 3x3

This gives

Row z x1 x2 x3 e1 e2 RHS Basis0 1 -2 -9 -3 0 0 0 z1 0 0 1 -0.1 -0.1 -0.2 1.3 x2

2 0 1 0 -0.6 0.4 -0.2 0.8 x1

Update the tableau so that the basic variables are represented by identity columns. Gives:

81


Row z x1 x2 x3 e1 e2 RHS Basis0 1 0 0 -5.1 -0.1 -2.2 13.3 z1 0 0 1 -0.1 -0.1 -0.2 1.3 x2

2 0 1 0 -0.6 0.4 -0.2 0.8 x1

We have achieved the optimal solution to the primal linear programming problem. Theminimum is achieved when z = 13.3 with x1 = 0.8 and x2 = 1.3 and x3 = 0.

Solving the Dual Problem

The dual for our example problem is given by

max w = y1 + 6y2

s.t. − 2y1 + y2 ≤ 2

2y1 + 4y2 ≤ 9

y1 − y2 ≤ 3

with y1, y2 ≥ 0

Here we need to add a slack variable to each of the constraints. This leads to the initialtableau:

Row w y1 y2 s1 s2 s3 RHS Basis0 1 -1 -6 0 0 0 0 w1 0 -2 1 1 0 0 2 s1

2 0 2 4 0 1 0 9 s2

3 0 1 -1 0 0 1 3 s3

Note here we pick y2 to enter into the basis. It will replace s1.

Row w y1 y2 s1 s2 s3 RHS Basis0 1 -13 0 6 0 0 12 w1 0 -2 1 1 0 0 2 y2

2 0 10 0 -4 1 0 1 s2

3 0 -1 0 1 0 1 5 s3

Here we pick y1 to enter the basis in place of s2. This gives,

Row w y1 y2 s1 s2 s3 RHS Basis0 1 0 0 0.8 1.3 0 13.3 w1 0 0 1 0.2 0.2 0 2.2 y2

2 0 1 0 -0.4 0.1 0 0.1 y1

3 0 0 0 0.6 0.1 1 5.1 s3

The optimal solution has been obtained with w = 13.3 where y1 = 0.1 and y2 = 2.2.

82


Observations

Note we could have solved the Dual problem graphically:

Lets compare the optimal Tableau for each of the two problems:

Row z x1 x2 x3 e1 e2 RHS Basis0 1 0 0 -5.1 -0.1 -2.2 13.3 z1 0 0 1 -0.1 -0.1 -0.2 1.3 x2

2 0 1 0 -0.6 0.4 -0.2 0.8 x1

PRIMAL

Row w y1 y2 s1 s2 s3 RHS Basis0 1 0 0 0.8 1.3 0 13.3 w1 0 0 1 0.2 0.2 0 2.2 y2

2 0 1 0 -0.4 0.1 0 0.1 y1

3 0 0 0 0.6 0.1 1 5.1 s3

DUAL

• Note we can read the primal and dual solutions for the other problem using the reducedcosts in row zero of the optimal tableau.

– The optimal dual variable values are the same as the reduced costs of the slack(and excess variables with with signs reversed).

83


Complementary Slackness

The optimal solutions are given by:

PRIMAL =

x1

x2

x3

e1

e2

=

0.81.3000

and DUAL =

y1

y2

s1

s2

s3

=

0.12.200

5.1

We can line up the constraints and sign restrictions for each problem and note which one isbinding in each of the problems:

PRIMAL PROBLEM BINDING DUAL PROBLEM−2x1 + 2x2 + x3 ≤ 1 PRIMAL y1 ≥ 0x1 + 4x2 − x3 ≤ 6 PRIMAL y2 ≥ 0x1 ≥ 0 DUAL −2y1 + y2 ≤ 2x2 ≥ 0 DUAL 2y1 + 4y2 ≤ 9x3 ≥ 0 PRIMAL y1 − y2 ≤ 3

This illustrates complementary slackness.

Consider the primal-dual pair in standard form:

PRIMAL: min z = cTx

s.t. Ax = b

with x ≥ 0

DUAL: max w = bTy

s.t. ATy ≤ c

with y is free

There is an interdependence between the non-negativity constraints in the primal (x ≥ 0) andthe constraints in the dual (ATy ≤ c). At the optimal solution it is not possible to have both:

xj > 0 and (ATy)j < cj.

At least one of the constraints must be binding giving that either

• xj = 0 or

• the j-th dual slack variable is zero (the corresponding dual constraint is binding).

Note this gives that ∑j

xj(c− ATy)j

84


Complementary slackness is often written as:

xT (c− ATy) = 0.

as the primal and dual constraints ensure that the terms in the summation must be non-negative. Thus, if the sum is zero then each term must be zero.

Theorem 8.3.2 (Complementary Slackness) Consider a pair of primal and dual linearprograms in standard form. If x is optimal for the primal and y is optimal for the dual then

xT (c− ATy) = 0

Proof: If x and y are feasible for there respective problems then:

z = cTx ≥ yTAx = yTb = w.

As x and y are also optimal we know that z = w so

cTx = yTAx =⇒ xTc = xTATy

=⇒ xTc− xTATy = 0

=⇒ xT(c− ATy

)= 0

�

8.4 The Dual Simplex Method


min z = 5x1 + 4x2

s.t. 4x1 + 3x2 ≥ 10

3x1 − 5x2 ≥ 12

with x1, x2 ≥ 0.

If we add excess variables to the constraints and place the problem into a tableau we have:

Row z x1 x2 e1 e2 RHS BASIS0 1 -5 -4 0 0 0 z1 0 4 3 -1 0 10 ?2 0 3 -5 0 -1 12 ?

Note that in the above Tableau we do not have any basic variables yet. However, thegiven tableau does show that the reduced costs for the problem are such that the optimalityconditions are satisfied. Lets consider making e1 and e2 basic (multiply row 1 and row 2 by-1). This gives the following tableau.

85


Row z x1 x2 e1 e2 RHS BASIS0 1 -5 -4 0 0 0 z1 0 -4 -3 1 0 -10 e1

2 0 -3 5 0 1 -12 e2

Note that:

• The optimality conditions are still satisfied. (Nothing is attractive to enter into thebasis in the simplex method as we know it.)

• The right hand side values are negative. This means that he current basis is infeasiblein the primal problem.

To move toward a feasible solution we could scan the RHS and see who is the most negative,and work to get them out of the basis. Here we need a candidate to come in in thereplace. Scan down the row corresponding to the mose negative RHS value and find negativecoefficients. If there is more than one we would do a ‘Ratio’ test (reduced cost divided bythe exiting row coefficient in the potential entering column) taking the minimum value tobe our entering variable.

Row z x1 x2 e1 e2 RHS BASIS0 1 0 -37

30 -5

320 z

1 0 0 -293

1 -43

6 e1

2 0 1 -53

0 -13

4 x1

After doing the update we can now see that the optimality conditions and the primal feasi-bility conditions have now both been satisfied. Yielding the optimal solution of z = 20 whenx1 = 4 and x2 = 0. This technique is called the dual simplex method . We can verify thatthis is indeed the solution to the problem by considering it graphically.

86


A Second Example

Use the dual simplex method to solve the following linear program:

min z = 5x1 + 2x2 + 8x3

s.t. 2x1 − 3x2 + 2x3 ≥ 3

−x1 + x2 + x3 ≥ 5

with x1, x2, x3 ≥ 0.

Setting up the problem in tableau form using two excess variables:

Row z x1 x2 x3 e1 e2 RHS BASIS0 1 -5 -2 -8 0 0 0 z1 0 2 -3 2 -1 0 3 ?2 0 -1 1 1 0 -1 5 ?

We negate the two constraints in order to make e1 and e2 basic.

Row z x1 x2 x3 e1 e2 RHS BASIS0 1 -5 -2 -8 0 0 0 z1 0 -2 3 -2 1 0 -3 e1

2 0 1 -1 -1 0 1 -5 e2

Here we look to get e2 out of the basis. We see that x2 is the winner of the Ratio Test.

Row z x1 x2 x3 e1 e2 RHS BASIS0 1 -7 0 -6 0 -2 10 z1 0 1 0 -5 1 3 -18 e1

2 0 -1 1 1 0 -1 5 x2

Note that after the pivot of the Dual Simplex algorithm there is no attractive candidate toenter into the basis using our normal “Primal Simplex” algorithm (the optimality conditionsare still met). We now use Dual simplex to remove e1 from the basis. Note that x3 is theonly choice to replace e1 in the basis.

Row z x1 x2 x3 e1 e2 RHS BASIS0 1 -8.2 0 0 -1.2 -5.6 31.6 z1 0 -0.2 0 1 -0.2 -0.6 3.6 x3

2 0 -0.8 1 0 0.2 -0.4 1.4 x2

Note that the optimal solution of z = 31.6 is achieved when x1 = 0, x2 = 1.4 and x3 = 3.6.

We make note here that the Dual Simplex method is especially useful when doing sensitivityanalysis. If a change in the right hand side value of a constraint is made this method can beused to update the current simplex solution to one that is feasible if the change causes theprimal problem to have negative right hand side values.

87


A Third Example

Use the dual simplex method to solve the following linear program:

min z = 3x1 + 2x2 + x3

s.t. 3x1 + x2 + x3 ≥ 3

−3x1 + 3x2 + x3 ≥ 6

x1 + x2 + x3 ≤ 3

with x1, x2, x3 ≥ 0.

Adding slack and excess variable and creating an initial simplex tableau gives:

min z = 3x1 + 2x2 + x3

s.t. 3x1 + x2 + x3 − e1 = 3

−3x1 + 3x2 + x3 − e2 = 6

x1 + x2 + x3 + s1 = 3

with x1, x2, x3, e1, e2, s1 ≥ 0.

This can be placed into Tableau Form as:

Row z x1 x2 x3 e1 e2 s1 RHS Basis0 1 -3 -2 -1 0 0 0 0 z1 0 3 1 1 -1 0 0 3 ?2 0 -3 3 1 0 -1 0 6 ?3 0 1 1 1 0 0 1 3 s1

Multiply rows 1 and 2 by negative one to make e1 and e2 basic.

Row z x1 x2 x3 e1 e2 s1 RHS Basis0 1 -3 -2 -1 0 0 0 0 z1 0 -3 -1 -1 1 0 0 -3 e1

2 0 3 -3 -1 0 1 0 -6 e2

3 0 1 1 1 0 0 1 3 s1

Do an iteration of dual simplex to have x2 replace e2 in basis.

Row z x1 x2 x3 e1 e2 s1 RHS Basis0 1 -5 0 -0.333333333 0 -0.666666667 0 4 z1 0 -4 0 -0.666666667 1 -0.333333333 0 -1 e1

2 0 -1 1 0.333333333 0 -0.333333333 0 2 x2

3 0 2 0 0.666666667 0 0.333333333 1 1 s1

Do a second iteration of dual simplex to have x3 replace e1 in the basis and we yield theoptimal tableau.

88


Row z x1 x2 x3 e1 e2 s1 RHS Basis0 1 -3 0 0 -0.5 -0.5 0 4.5 z1 0 6 0 1 -1.5 0.5 0 1.5 x3

2 0 -3 1 0 0.5 -0.5 0 1.5 x2

3 0 -2 0 0 1 0 1 0 s1

89


90

Chapter 9

Sensitivity Analysis

“ I don’t know where the sun beams endand the star light begins it’s all a mystery...”

−The Flaming Lips

9.1 Sensitivity Analysis

Before looking at sensitivity analysis it is good to have a base problem to consider. We willbase our initial pilgrimage into sensitivity analysis of linear programs on the following 2Dlinear program.

Base Problem

Consider maximizing the profit from manufacturing two items on three different assemblylines. The profit from manufacturing item 1 is three thousand dollars, and from item 2 is fivethousand dollars. Letting x1 and x2 be the number of each of these items to manufacture,and letting the following constraints denote the number of hours that it takes to manufactureeach of the items at three different assembly lines with the given restrictions on the maximumnumber of hours available on each line. This gives the following linear program:


subject to x1 ≤ 42x2 ≤ 12

3x1 + 2x2 ≤ 18x1, x2 ≥ 0.

Assuming you are renting time on each assembly line what is the most you should pay foran additional hour of time on each of the assembly lines?

91


Solution:Using the simplex algorithm to solve the given linear program we see that after adding twoslack variables in the constraints the initial tableau for the given linear program is writtenas:


2 0 0 2 0 1 0 12 s2

3 0 3 2 0 0 1 18 s3

Here it becomes desirable to pivot x2 into the basis replacing s2. Thus,


20 30 z

1 0 1 0 1 0 0 4 s1

2 0 0 1 0 12

0 6 x2

3 0 3 0 0 -1 1 6 s3

The next iteration of the simplex method shows that it is desirable to have x1 inter the basisand it will replace s3. Doing this pivot yields:


21 36 z

1 0 0 0 1 13

-13

2 s1

2 0 0 1 0 12

0 6 x2

3 0 1 0 0 -13

13

2 x1

Here we have obtained an optimal tableau. We can see here that profit is maximized at$36,000 when x1 = 2 and x2 = 6 are manufactured between the different assembly lines.

Shadow prices:

We are often interested in the marginal value of each of the given resources and their impacton the objective function. That is:

Shadow Price : measures the marginal value of a given resource or the amountby which z will be increased by slightly increasing the amount of a given resourcebeing made available.

The shadow price for resource i is denoted by y∗i and is the amount the objective function willbe increased if the amount of resource bi is slightly increased. The shadow price for resourcei is the zero row coefficient for the ith constraints slack variable in the optimal tableau.

92


From our given example we can see that the shadow price for hours available for eachassembly line (marginal increase in the objective function for slight increases in the amountthe resource available) is:

y∗1 = 0, y∗2 =3

2y∗3 = 1. (9.1.1)

9.1.1 Verify this graphically:

An interactive version of the solution is found at:

http://www.math.iup.edu/~jchrispe/MATH445_545/ShadowPrices.html

What happens if the constraint b2 goes from 12 to 13?

y∗2 = ∆z = 37.5− 36 = 1.5 (9.1.2)

How do we interpret this with regard to the linear program?

An additional unit of labor on production line two will increase the profits by$1500.

How far could we increase the value of b2 the amount of resource 2 and still stay optimalwith the current basis?

93

http://www.math.iup.edu/~jchrispe/MATH445_545/ShadowPrices.html


We could increase the units of resource b2 up to 18 and keep the same basicfeasible solution. After which the constraint is no longer binding. Here theoptimal objective function would be fixed at z =45. At this point a new basicfeasible solution can be obtained, and a new set of shadow prices will come intoplay.

Note that constraints that have zero shadow prices are not binding the objective function.In the given example we note that the first constraint is not binding and increasing the valueof b1 will have no effect on the objective function’s optimal value. Resources for bindingconstraints are sometimes called scarce goods, and resources for non-binding constraintsare called free goods.

• Shadow prices can be also thought of as the maximum price you would want to payfor an additional unit of a given resource.

• Shadow prices help identify the parameters in the model that need to be estimatedcarefully. If a shadow price is zero the model is not sensitive to (at least small) changesin the given resource.

• You would want to monitor resources more closely if they have large shadow prices.

How far can b2 be changed and keep the same optimal basic feasible solution?

6 ≤ b2 ≤ 18

keeps the same basic feasible solution, and an increase of 6 or a decrease of 6 is allowed.What information can we get out of LINDO/LINGO?

9.2 Sensitivity Analysis Using Matrices

We looked briefly as sensitivity of linear programs graphically. This is nice if the problem issmall enough to handle with a software like Geogebra. However; it becomes more difficult todetermine how the solution to a linear program is effected by perturbations of the objectivefunction coefficients and constraints if the problem of interest is more than two dimensional.

Lets recall that for a linear program in the form:

(min or max) cTxs.t. Ax = b

with x ≥ 0

then any current basis may be written using matrix form (using our previously definednotation) as:

xNT xB



B−1N I B−1b xB

94


With this in mind any change in the initial linear program can be thought of in terms ofthese formulations. Specifically checking the feasibility, and optimality of any current basiswill come down to looking at:

B−1b ≥ 0 for feasibility

and−cNT + cB

TB−1N ≥ 0 for optimality in the maximization problem

−cNT + cBTB−1N ≤ 0 for optimality in the minimization problem.

From these conditions we can now answer the following questions:

• How much can the initial problem data be changed before the optimality conditionsno longer hold?

• If the current basis is no longer optimal we would apply primal simplex to restore itto optimality.

• Changes in the data some times may affect the feasibility of the problem. How muchcan they change before the constraints are violated?

We may use these formulations to do sensitivity analysis for a linear program allowing forthe appropriate changes in the optimal solution to be made based on changes in the originalproblem parameters.

9.2.1 Illustrating Example

Consider the following linear program (A problem form the Nash and Sofer text):

maximize z = 3x1 + 13x2 + 13x3

subject to x1 + x2 ≤ 7x1 + 3x2 + 2x3 ≤ 15

2x2 + 3x3 ≤ 9x1, x2, x3 ≥ 0.

Thus, here the matrix form values are

A =

1 1 0 1 0 01 3 2 0 1 00 2 3 0 0 1

, c =

3

1313000

, and b =

7159

.

95


If the optimal solution to the problem is found the optimal basis xB = (x1 x2 x3)T , andxN = (s1 s2 s3)T we have:

N =

1 0 00 1 00 0 1

, B =

1 1 01 3 20 2 3

=⇒ B−1 =

52−3

21

−32

32−1

1 −1 1

.

with

cNT =

(0 0 0

)cB

T =(

3 13 13)

Lets answer the following questions:

1. What is the solution to the problem?Where we find the values of the decision variables as:

Current RHS B−1b =

52−3

21

−32

32−1

1 −1 1

7159

=

431

.

Objective Value cBTB−1b =

(3 13 13

) 431

=(

64)

Thus, x1 = 4, x2 = 3, and x3 = 1.

2. What are the optimal dual variables?

Dual Variables − cNT + cB

TB−1N

=(

0 0 0)

+(

3 13 13) 5

2−3

21

−32

32−1

1 −1 1

1 0 00 1 00 0 1

=

(1 2 3

)Note these are the shadow prices, and if the problem had more context they would tellus the most we would wish to pay for an additional unit on the right hand side of eachof the constraints.

3. What is the solution obtained by decreasing the right-hand side of the second constraintby 5?Now we add a vector of the form: ∆b = (0,−5, 0)T to the initial b vector:

bnew =

7159

+

0−5

0

=

7109

.

96


Where we find the values of the decision variables as:


52−3

21

−32

32−1

1 −1 1

7109

=

232

−92

6

.Thus, x1 = 23

2, x2 = −9

2, and x3 = 6. Here the RHS is no longer feasible for the

problem as x2 < 0. So we can implement an iteration of the dual simplex methodto find a new basic feasible solution. We need to have a handle on the values of thenon-basic columsn which are given by

Non-Basic Columns B−1N =

52−3

21

−32

32−1

1 −1 1

as well as the new values for the dual variables:


TB−1N =(

1 2 3)

We do the Ratio Test for an iteration of the dual simplex method giving:

min{col1 =

∣∣∣∣ 1−32

∣∣∣∣ =2

3, col3 =

∣∣∣∣ 3

−1

∣∣∣∣ = 3

}So we pick x2 to leave the basis and we choose s1 to enter. This gives the new optimalsolution to be: xB = (x1 s1 x3)T , and xN = (x2 s2 s3)T

Where we now have:

B =

1 1 01 0 20 0 3

and N =

1 0 03 1 02 0 1

.


TB−1N =(

23

3 73

)Current RHS B−1b =

433

.Objective Value cB

TB−1bnew =(

51)

4. By how much can the right-hand side of the first constraint increase and decreasewithout changing the optimal basis?

97


Here we are looking at the first component of B−1(b+ b∆) and want to see what rangeof values will keep the first component non-negative where:

b∆ =

∆b00

Thus,

B−1 (b + b∆) =

52−3

21

−32

32−1

1 −1 1

7 + ∆b159

=

52

(7 + ∆b)− 32(15) + (9) ≥ 0

−32

(7 + ∆b) + 32(15)− (9) ≥ 0

(7 + ∆b)− (15) + (9) ≥ 0

These constraints lead to the following conditions:

∆b ≥ −8

5,∆b ≤ 2 , and ∆b ≥ −1.

so the right hand side of the first constraint may increase by at most 2, and decreaseby at most 1 with out changing the current basis.

5. What is the optimal solution to the linear program obtained by increasing the coeffi-cient of x2 in the objective function by 15? Here the value of

cB =

32813

For the different components of the problem we see that:


(109

),

The values of the decision variables are:


431

.Thus, x1 = 4, x2 = 3, and x3 = 1, and the current dual variables


TB−1N =(−43

2492−12

)As we now have an attractive entering variable we can do a Primal simplex pivot withs1 entering the basis. The ratio test using:

Current Non-Basic Columns B−1N =

52−3

21

−32

32−1

1 −1 1

98


and the Current RHS yields that x3 should leave the basis. This gives xB = (x1 x2 s1)T ,and xN = (x3 s2 s3)T where

N =

0 0 02 1 03 0 1

and B =

1 1 11 3 00 2 0

We can look at the updated dual variables and see that:


(2612

),

The values of the decision variables are:


3292

1

.Thus, x1 = 3

2, x2 = 9

2, and x3 = 0, and the current dual variables are


TB−1N =(

432

3 192

)Note that as the dual variable corresponding to s3 is zero it is also a potential candidateto enter the basis giving us multiple optimal solutions.

6. How much can the objective coefficient of x1 increase and decrease without changingthe optimal basis?Here we consider the value of the dual variables that is:


TB−1N =(

1 2 3)

Note that cN =(

0 0 0)so we need only consider the following:

((3 + ∆c) 13 13)

52−3

21

−32

32−1

1 −1 1

1 0 00 1 00 0 1

≥ 0

=⇒ ((3 + ∆c) 13 13)

52−3

21

−32

32−1

1 −1 1

≥ 0

=⇒ 5

2(3 + ∆c) + (13)(

−3

2) + (13) ≥ 0,

−3

2(3 + ∆c) + (13)(

3

2)− (13) ≥ 0,

(3 + ∆c)− 13 + (13) ≥ 0

This gives that

x ≥ −2

5, x ≤ 4

3, and x ≥ −3.

so the coefficient on x1 in the objective function can be increased by at most 43and

decreased by at most 25if the same optimal basis is to be maintained.

99


Example:

Consider the following linear program and its optimal tableau:

max z = 3x1 + x2 − x3

s.t. 2x1 + x2 + x3 ≤ 8

4x1 + x2 − x3 ≤ 10

with x1, x2, x3 ≥ 0

Row z x1 x2 x3 s1 s2 RHS Basis0 1 0 0 1 0.5 0.5 9 z1 0 0 1 3 2 -1 6 x2

2 0 1 0 -1 -0.5 0.5 1 x1

1. Find the dual of this linear program and its optimal solution. Taking the dual gives:

min w = 8y1 + 10y2

s.t. 2y1 + 4y2 ≥ 3

y1 + y2 ≥ 1

y1 − y2 ≥ −1

with y1, y2 ≥ 0

The optimal solution is w = 9 when y1 = 0.5, and y2 = 0.5.

2. Assuming that this linear program is maximizing profit with units of thousands ofdollars. What is the most you would be willing to pay for an additional unit on theright hand side of each of the constraints. Explain your answer to get full credit. Herethe shadow prices are given by the dual variables y1 and y2 these represent the marginalcost of each of the constraints. Thus you would pay at most $500 for an additionalunit of either of the two constraints.

3. Find the range of values of b2 (the right hand side entry of the second constraint) forwhich the current basis remains optimal. Using LINDO we find that the most we canchange the RHS of the second constraint is an increase of 6, and a decrease of at most2.For the algebra we let ∆b denote the change and consider:

B−1bnew ≥ 0

whereB−1 =

(2 −1−1

212

), and bnew =

(8

10 + ∆b

)This gives the following inequalities that will restrict the values of ∆b.

16− (10 + ∆b) ≥ 0 and −4 +1

2(10 + ∆b) ≥ 0

=⇒ 6−∆b ≥ 0 =⇒ 1 +1

2∆b ≥ 0

=⇒ ∆b ≤ 6 =⇒ ∆b ≥ −2

100


This confirms what was obtained using LINDO above.

101


102

Chapter 10

Non Linear Programming

“ “Our opponent is an alien starship packed with atomic bombs,” I said. “We have a

protractor.” ”

−Neal Stephenson, Anathem

There are many instances where the problem considered involves more that a linear rela-tionship between decision variables and our objective. Here we will examine some of thepreliminaries of non-linear programming starting with fitting functional models to sets ofdata.

10.1 Data Fitting

There are many instances where it would be advantageous to find a best fit model for a set ofdata. Here we have a set of points and we want to find in some sense the model or functionthat “best fits” the set of points 1.

For us best fit will be in the least squares sense. Thus given a set of n points (ti, yi), wewant to find a vector of m parameters x that will minimize the least square residual of:

minimizex

n∑i=1

(yi − f(ti,x))2 ,

wheref : Rm+1 → R.

This may be a linear or non-linear problem depending on whether the function f is linearin its components of the parameter vector x.

1Here we will follow the presentation in Heath’s Text. [2]

103


10.1.1 Linear Problems

Consider for example where we are interested in finding a function:

f(t,x) = x1 + x2t+ x3t2 + . . .+ xmt

m−1

that best fits our set of n points we have to find the m coefficients for vector x that wouldminimize our residual in the least squares sense described above.

This problem and others where the parameters x in f are linear combinations will result ina matrix system of the form

Ax ∼= b.

Lets work a couple of examples.

Example:

Consider finding the best fitting linear and quadratic function for the following set of points:

(1, 0), (2, 5), (3, 8), (4, 17), (5, 24)

For the case of a linear function we want to find values of x = (x1, x2)T in

f(t,x) = x1 + x2t

such that we minimize the sum of

min5∑i=1

yi − f(ti,x)

Writing this out as a matrix system:

Ax =

1 t11 t21 t31 t41 t5

[x1

x2

]∼=

y1

y2

y3

y4

y5

= b.

Or with our given set of points:

Ax =

1 11 21 31 41 5

[x1

x2

]∼=

058

1724

= b.

So a residual vector could be found by looking at

r = b− Ax

104


if we are going look at the minimum of squared Euclidean norm of this residual can considerfinding the minimum of

φ(x) = ‖r‖22

= rT r

= (b− Ax)T (b− Ax)

= bTb− bTAx− (Ax)Tb + (Ax)T (Ax)

= bTb− 2bTAx + xTATAx

If we use methods from multi-variable calculus we know that the residual function will havea potential minimum provided (necessary condition):

∇φ(x) = 0

Thus, we will consider:

0 = ∇φ(x)

= 0− 2bTA+ ATAx + ATAx

= −2bTA+ 2ATAx

=⇒ ATAx = ATb

For this to indeed be a minimum note that we need the Hessian Matrix (matrix of secondpartial derivatives) given by ATA to be positive definite. (This can be proved by showingthat ATA has rank n or that the columns are linearly independent). Note this system ofequations is known as the normal equations.

Looking back at our example we need to consider:(5 15

15 55

)[x1

x2

]=

(54

222

)And solving the equation gives:

x =

[x1

x2

]=

(−36

5

6

)So the minimizing linear function is given by:

f(t) = 6t− 36

5

and we make note that if we evaluate φ(x) gives:

φ(x) = bTb− 2bTAx + xTATAx = 10.8

105


Now lets consider fitting a quadratic function to the data. Thus we want to find the bestfit function of the form:

f(t,x) = x1 + x2t+ x3t2

The new system matrix is:

Ax =

1 t1 t211 t2 t221 t3 t231 t4 t241 t5 t25

x1

x2

x3

∼=y1

y2

y3

y4

y5

= b.

After substituting our point into the matrix A we have:

Ax =

1 1 11 2 41 3 91 4 161 5 25

[x1

x2

]∼=

058

1724

= b.

x =

x1

x2

x3

=

−11512757

Thus, the best fit quadratic function for the set of points is given by

f(t) =5

7t2 +

12

7t− 11

5

Plotting our two found functions and the original data set gives:

106


Non-linear Problems

In order to consider non linear problems we need to consider iterative methods that will helpus find the optimal solution to our data fitting problem.

10.1.2 Taylor series

Lets recall in one dimension that we can expand a function about a close point (x + h)provided we know information about our function at point x. Thus a Taylor series expansionis:

f(x+ h) =f(x)

0!+f ′(x)h

1!+f ′′(x)h2

2!+ · · ·

Truncate the expansion after the first term gives:

f(x+ h) ≈ f(x) + f ′(x)h

For a numerical scheme we could note that h is the distance between to points:

h = x1 − x

Thus, we can make a note that an approximation of the first derivative can be given by:

f(x+ x1 − x) ≈ f(x) + f ′(x)(x1 − x)

=⇒ f ′(x) ≈ f(x1)− f(x)

(x1 − x)

10.1.3 Newton’s Method

Newton’s method is a way of iterating toward the root a function given that you have aninitial guess that is reasonably close to the root. To find the root of

f(x) = 0

we guess that x1 is our root. Then an iterative scheme for approaching a root would begiven by rearranging the first derivative approximation that we just found (with f(x1) ≈ 0)to obtain:

xn = xn−1 −f(xn−1)

f ′(xn−1)

where we iterate from some initial guess until some predetermined tolerance is achieved.

107


In order to solve non-linear equations in multiple dimensions we can look at a generalizationof Newton’s method. Looking at a function f such that f : Rn → Rn, the truncated Taylorseries expansion is given by:

f(x + s) ≈ f(x) + Jf (x)s (10.1.1)

where we are using Jf (x) to denote the Jacobian matrix of x. Specifically,

Jf (x)i,j =∂fi(x)

∂xj.

Noting that s is given as:s = xn − x =⇒ xn = s + x

and that xn is an approximate zero to f , we can rewrite (10.1.1) as:

xn ≈ x− f(x)

Jf (x)

An algorithm for solving a system of f(x) = 0 for some initial guess x0 could be written inthe following manner:

Algorithm for Newton’s Method (System of non-linear equations):

While ‖s‖2 > tolerance.

• Solve Jf (xn−1)sn = −f(xn−1) for sn.

• Update xn = xn−1 + sn

Lets now consider newtons method for a function f : Rm → R. Here a two term in ourTaylor series expansion:

f(x + s) ≈ f(x) +∇f(x)T s +1

2sTHf (x)s.

where Hf (x) is the Hessian matrix of second partial derivatives of f such that:

Hf (x)i,j =∂2f(x)

∂xi∂xj

It follows that the function f(x + s) will have a minimum when

Hf (x)s = −∇f(x). (10.1.2)

To see this you should look at this as if f is a quadratic function in s, and note that (10.1.2)is the vertex of the function. This allows us the oppertunity to think of Newton’s Methodas:

108


Algorithm for Newton’s Method :


• Solve Hf (xn−1)sn = −∇f(xn−1) for sn.• Update xn = xn−1 + sn

10.2 Non-linear Least Squares

Lets start by considering a biological system that involves fitting parameters to a mathe-matical model of a swimming lamprey. The work of Dr. Chia-Yu Hsu involves numericalsimulations of these swimmers:

http://www.ccs.tulane.edu/~chiayu/

Here the goal will be to fit a set of swimming data with a best fit model of the followingform:

f(x, t) = (ax2 + bx+ c) sin(kx+ ωt)

where we need to find the values of a, b, c, k, and ω that will best fit our model.

In order to solve this problem lets assume that we are given a set of m points of the form(ti, yi) and we want to fit a nonlinear function with vector of parameters x ∈ Rk such thatthe function f(t,x), is the best fit to the given data set where f : Rk+1 → R. Thus, our goalis to minimize the residual r where r : Rk → Rm, and

ri(x) = yi − f(ti,x), for i ∈ {1, 2, . . . ,m}.

where we are now hoping to minimize

φ(x) =1

2‖r‖2

2 =1

2r(x)T r(x).

We have placed the 12into the function φ for convenience. In order to apply Newton’s Method

to this problem we need to find the following components:

∇φ(x), and Hφ(x).

Finding the first of these expressions is not troubling. Denote the Jacobian matrix of r(x)by J(x) gives

∇φ(x) = JT (x)r(x).

For the second term we calculate the Hessian Matrix of φ as

Hφ(x) = JT (x)J(x) +m∑i=1

ri(x)Hri(x).

Calculating the Hri(x) is often difficult and computationally expensiveso it is often ignoredor dropped fom computations. The result in what is known as a Gauss-Newton Methodgiven by:

109

http://www.ccs.tulane.edu/~chiayu/


Algorithm for Newton’s Method :


• Solve JT (xn−1)J(xn−1)sn = −JT (xn−1)r(xn−1) for sn.

• Update xn = xn−1 + sn

Note the solve step looks like the normal equations for a nonlinear least squares problem.

The question now is what does J(x) look like for our given problem? This is the Jacobeanmatrix of the residual vector. For the swimmers we have:

f(x, x, t) = (ax2 + bx+ c) sin(kx+ ωt)

where we need to find the values of a, b, c, k, and ω that will best fit our model to a givenset of data along a swimmer. Lets let φ = ωt be a parameter all by itself and then back thevalue of ω out of the time dependent data fit (as we are doing this for a moving swimmer).Thus,

f(x, x) = (ax2 + bx+ c) sin(kx+ φ), and x = (a, b, c, k, φ)T

So at any time we are looking at the residual vector for our m points where the componentsof r(x) are:

ri(x) = yi − f(x, ti) for i ∈ {1, 2, . . . ,m}.

and the components of the residuals Jacobean are

J(x)i,1 =∂ri(x)

∂a= −(t2i ) sin(kti + φ)

J(x)i,2 =∂ri(x)

∂b= −ti sin(kti + φ)

J(x)i,3 =∂ri(x)

∂c= − sin(kti + φ)

J(x)i,4 =∂ri(x)

∂k= −(at2i + bti + c) cos(kti + φ)ti

J(x)i,5 =∂ri(x)

∂φ= −(at2i + bti + c) cos(kti + φ)

At the end of the day we will have a five by five matrix system to iterate on and find ouroptimal parameters. Note as we are going to solve this over different time sets we will beable to plot the value computed for φ as a function of the different input times and back outa value for ω the speed of the swimmer.

110

Chapter 11

Network Flows

“It’s hard to decide who’s truly brilliant; it’s easier to see who’s driven, which in the long

run may be more important.”

−Michael Crichton

In this chapter a collection of Network Algorithms will be examined. The current treatmentwill only serve to illustrate some of the other areas that can be explored in the world ofOperations Research.

11.1 Dijkstra’s Algorithm

Dijkstra’s Algorithm can be used to find the shortest path spanning tree for a given network.Essentially the algorithm follows the following steps:

1. Set the distance to notes.

(a) The starting (or current) node distance is set to zero,(b) All other nodes in the network are set to a distance of ∞.

2. Create a list of the unvisited nodes. Consisting of all nodes.

3. From the current node:

Consider the distance to all the unvisited node neighbours in the network.If the distance from the current node to the neighbour is less than the listedvalue in you “distance list” update the distance and the predecessor array.

4. Mark the current node as visited by removing from “unvisited list”.

5. If the distance to all unvisited nodes is∞ stop. Otherwise choose the closest unvisitednode to be the current node, and go back to step 3.

111


Lets consider the algorithm with the following example:

Example:

Use Dijkstra’s algorithm to find a shortest path tree rooted at node 1 in the network below.Note that two of the arcs are bi-directional. Show at each iteration D(·), pred(·), and theselected node i∗ , as well as the final shortest path tree.

��

��

��

��

��

��>

-

QQQQs

- ��>

ZZZZ~

6

? ?��6

@@@@@@@R

5

32

4

6

1

6

5

2

2

118

5

3

3

��

��5∞ ∞

��

��

��

��>-

QQQQs

- ��>

ZZZZ~

6

? ?��6

@@@@@@@R

Initialize

3

5

32

4

6

1

6 2

2

118

5

3

∞ ∞

∞

node 1∗ 2 3 4 5 6D( ) 0 ∞ ∞ ∞ ∞ ∞

pred( )

Note: The ∗ denotes the node selected and p will denote a nodes predecessor.

��

��

��

��

��

��

��

��

��

��

��

-

- ��>

ZZZZ~

6

? ?��6

@@@@@@@R

QQQQs

��>

-

��>

ZZZZ~

? ?

6@@@@@@@R

��>6

��

-

QQQQs3

5

32

4

6

1

6 2

2

118

5

3

∞

5 ∞6

3

∞

3

5

32

4

6

1

6 2

2

118

5

3

5

3

4

8

11

∞

node 1p 2 3 4 5 6∗

D( ) 0 6 ∞ ∞ ∞ 3pred( ) 0 1 1

node 1 2∗ 3 4 5 6p

D( ) 0 4 11 ∞ 8 3pred( ) 0 6 6 6 1

��

��

��

��

��

��

��

��

��

��

��>

ZZZZ~

?

6��>

��

-

QQQQs

6@@@@@@@R

-

?

ZZZZ~

?

��>

��

QQQQs

6

-

?

6

��>

-

@@@@@@@R3

5

32

4

6

1

6 2

2

118

5

3

5

3

4 9

7

∞

3

5

32

4

6

1

6 2

2

118

5

3

5

3

4

7

8

9

112


node 1 2p 3 4 5∗ 6D( ) 0 4 9 ∞ 7 3

pred( ) 0 6 2 2 1

node 1 2 3∗ 4 5p 6D( ) 0 4 8 9 7 3

pred( ) 0 6 5 5 2 1

��

��

��

��

��

��

?

��>

��

QQQQs

6

-

?��>

-

@@@@@@@R

6

ZZZZ~

3

5

32

4

6

1

6 2

2

118

5

3

5

3

4

7

8

9

��

��

��

��

��

��

?

��>

��

QQQQs

6

-

?-

@@@@@@@R

6

��>

ZZZZ~

3

5

32

4

6

1

6 2

2

118

5

3

5

3

4

7

8

9

node 1 2 3p 4 5 6D( ) 0 4 8 9 7 3

pred( ) 0 6 5 5 2 1

node 1 2 3 4∗ 5p 6D( ) 0 4 8 9 7 3

pred( ) 0 6 5 5 2 1

The minimum distance spanning tree for the network is given by the final path shown inred.

113


Class Activity

Use Dijkstra’s algorithm to fine the shortest path from node “1” to node “5” in the followingnetwork. Does Dijkstra’s algorithm obtain the correct result in each of the cases given?

1

2

3

4

52

12

4

5

8

6

10

114


Use Dijkstra’s algorithm to fine the shortest path from node “1” to node “4” in the followingnetwork. Does Dijkstra’s algorithm obtain the correct result in each of the cases given?

1

2

3

4

2

1-2

1

1

115


11.2 Max Flow Problems

Consider the following example (taken form the Winston Text).

Sunco Oil wants to ship the maximum possible amount of oil (per hour) via pipeline froma source node, s, to a sink node, t. On its way from the source to the sink oil must passthrough some or all of the stations 1, 2, and 3. The various arcs represent piplelines ofdifferent diameters. The maximum number of barrels of oil (millions of barrels per hour)that can be pumped through each arc is shown in the following table. Each number is calledan arc capacity. Formulate a linear program that can be used to determine the maximumnumber of barrels of oil per hour that can be sent from the source to the sink.

Arc Capacity(s, 1) 2(s, 2) 3(1,2) 3(1,3) 4(3,t) 1(2,t) 2

116


Let xi,j be the millions of barrels of oil per hour that will pass through arc (i, j) of thepipeline.

maxx3,t + x2,t

s.t. xs,1 ≤ 2

xs,2 ≤ 3

x1,2 ≤ 3

x1,3 ≤ 4

x3,t ≤ 1

x2,t ≤ 2

xs,1 = x1,2 + x1,3 node 1x1,3 = x3,t node 3

xs,2 + x1,2 = x2,t node 2xs,1 + xs,2 = x3,1 + x2,t Total Flow

With xi,j ≥ 0 for all i, j arcs in the network. We could also consider Matchmaking orassignment problems as network flow problems.

Example

Consider the following Comparability Matrix with workers and different job that requireunique skill sets. For example given the following compatibility of the Snorks with differentjobs find an matching that assigns each Snork a job.

Name\Job Supervisor Programmer Athletic Director Maintenance Chef ArtistAllstar c c c cDimmy c c cCasey c cDaffney c c cWellington c cTooter c c c

Note that this is formulated as a network flow problem by adding a source and a sink termto the problem.

117


The max flow that can be pushed through a network can be thought of as finding a minimumcut.

• A cut is any group of arcs on a network that when deleted separate the source nodeform the ending sink node.

• The flow from source to sink for any feasible flow is less than or equal to the capacityof any cut.

• The cut with the smallest capacity derermins the maximum flow in a given network.

Consider the following Network:

1

4

5

32

10

30

20 30

20

15

20

40

Using this network we can start to find the maximum flow by pushing as much as possibledown feasible arcs. Note we stop when a cut can be found that no more capacity can bepushed across.

118


1

4

5

32

1

4

5

32

1

4

5

32

1

4

5

32

1

4

5

32

1

4

5

32

Your textbook contains very detailed algorithms for pushing flow along these networks. Seethe Ford-Fulkerson Method as an example.

119


120

Bibliography

[1] H. A. Eiselt and C.-L. Sandblom. Linear programming and its applications. Springer,Berlin, 2007.

[2] M.T. Heath. Scientific Computing: An Introductory Survey, 2nd Edition. McGraw-Hill,New York, 2002.

[3] Frederick S. Hillier and Gerald J. Lieberman. Introduction to operations research. Holden-Day Inc., Oakland, Calif., third edition, 1980.

[4] D. Luenberger. Linear and nonlinear programming. Addison-Wesley, Reading, Mass.,second edition, 1984.

[5] David G. Luenberger and Yinyu Ye. Linear and nonlinear programming. InternationalSeries in Operations Research & Management Science, 116. Springer, New York, thirdedition, 2008.

[6] S. Nash and A Sofer. Linear and nonlinear programming. McGraw-Hill, New York, 1996.

[7] J. Reeb and S. Leavengood. Using the graphical method to solve linear programs. Ore-gon State University: Extension Service, Performance Excellence in the Wood ProductsIndustry:1–24, 1998.

[8] Hamdy A. Taha. Operations research. An introduction. The Macmillan Co., New York,1971.

[9] Wayne L. Winston. Operations research: applications and algorithms. Duxbury Press,Boston, MA, 1987.

121

Index

basic solution, 31basic variables, 31Bland’s Rule, 52budgeting problem, 23

canonical form, 72complementary slackness, 84convex combination, 15

decision variables, 11degenerate, 51diet problem, 19, 70direction of unboundedness, 34, 45dual problem, 71dual simplex, 85dual simplex method, 86duality

strong duality, 78weak duality, 77

elementary row operation, 8

feasible region, 12free variables, 29

Gauss-Jordan Method, 8

Hessian matrix, 103

Jacobean matrix, 108

least squares, 101

Newton’s method, 105non-linear least squares, 107nonbasic variables, 31normal equations, 103

objective function, 11optimal solution, 12

primal problem, 69

problem constraints, 11

residual vector, 102

scheduling problem, 22sensitivity analysis, 89shadow price, 90simplex method, 5

cycling, 51dual simplex, 85matrix form, 58

slack variable, 82standard form, 27, 31strong duality, 78

tableau, 57Taylor series, 105

122

Notes: Deterministic Models in Operations ResearchNotes: Deterministic Models in Operations Research...

Documents

Transcript of Notes: Deterministic Models in Operations ResearchNotes: Deterministic Models in Operations Research...