An Introduction to Mathematical...

An Introduction to Mathematical Optimization

Tom Asaki

January 21, 2020

Contents

1 Introduction 7

2 Optimization Concepts and Notation 9

3 Single-Variable Optimization 15

3.1 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Multi-Variable Optimization 25

4.1 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . 25

4.2 Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . 26

4.3 How to Compute Eigenvalues . . . . . . . . . . . . . . . . . . . . . . 28

5 Linear Programming 31

5.1 Elements of Linear Programs . . . . . . . . . . . . . . . . . . . . . . 32

6 LP Standard Forms 37

6.1 Important Standard Forms . . . . . . . . . . . . . . . . . . . . . . . . 37

6.2 Converting LPs between Standard Forms . . . . . . . . . . . . . . . . 40

7 Software Solvers 47

7.1 General MIPs Solved Using octave . . . . . . . . . . . . . . . . . . 47

7.2 Solving MIPs Using Matlab . . . . . . . . . . . . . . . . . . . . . . . 50

8 Geometry of Linear Programs 55

8.1 Geometry of the Feasible Region . . . . . . . . . . . . . . . . . . . . . 55

8.2 Geometry of the Objective Function . . . . . . . . . . . . . . . . . . . 57

8.3 Geometry in Standard Form . . . . . . . . . . . . . . . . . . . . . . . 58

9 Geometric Solution Methods 63

9.1 Vertex Enumeration Method . . . . . . . . . . . . . . . . . . . . . . . 65

9.2 Optimal Solutions and Linear Algebra . . . . . . . . . . . . . . . . . 67

3

4 CONTENTS

10 Modeling Concepts 71

10.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

10.2 A First Modeling Example . . . . . . . . . . . . . . . . . . . . . . . . 72

11 Modeling Examples 77

11.1 A-1 Furniture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

11.2 Greenside Yard & Lawn Care . . . . . . . . . . . . . . . . . . . . . . 79

11.3 Tourist Trap Shop . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

11.4 Woof&Meow Pet Food Company . . . . . . . . . . . . . . . . . . . . 83

11.5 Pack&Ship Air Delivery . . . . . . . . . . . . . . . . . . . . . . . . . 87

11.6 Shoes All Year . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

11.7 Thermo – (Linear Data Fitting) . . . . . . . . . . . . . . . . . . . . . 93

11.8 Atmospheric CO2 – Nonlinear Data Fitting . . . . . . . . . . . . . . . 101

11.9 ExaByte Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

11.10Counterfeit Bank Note Detection . . . . . . . . . . . . . . . . . . . . 108

11.11Water Treatment Plant Planning . . . . . . . . . . . . . . . . . . . . 114

11.12Goldbach’s Conjecture . . . . . . . . . . . . . . . . . . . . . . . . . . 117

11.13Packing Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

11.14CopyMaster Machine Scheduling . . . . . . . . . . . . . . . . . . . . 123

11.15Profit Under Premium Fees . . . . . . . . . . . . . . . . . . . . . . . 127

11.16Starcraft Bread Distributors . . . . . . . . . . . . . . . . . . . . . . . 129

11.17Puzzles and Games Inc. . . . . . . . . . . . . . . . . . . . . . . . . . 132

11.18Pit Mine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

11.19Team Building Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 137

12 Geometry of Standard Form 151

12.1 An Example in Two Variables . . . . . . . . . . . . . . . . . . . . . . 153

12.2 An Example in Three Variables . . . . . . . . . . . . . . . . . . . . . 155

12.3 Summary of Basic Solutions . . . . . . . . . . . . . . . . . . . . . . . 158

13 The Simplex Method 161

13.1 Basic Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . . . 163

13.2 Unbounded Linear Programs . . . . . . . . . . . . . . . . . . . . . . . 168

13.3 Multiple Optimal Solutions . . . . . . . . . . . . . . . . . . . . . . . 170

13.4 Recognizing Infeasible Linear Programs . . . . . . . . . . . . . . . . . 174

13.5 Pivot Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

14 Duality 191

15 Sensitivity Analysis 193

CONTENTS 5

16 Solving Integer Programs 19516.1 LP Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19616.2 Branch and Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19816.3 Cutting Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20616.4 Implicit Enumeration . . . . . . . . . . . . . . . . . . . . . . . . . . . 21016.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

17 Interior Point Methods 21917.1 Affine Scaling Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 21917.2 Primal Path Following Algorithm . . . . . . . . . . . . . . . . . . . . 21917.3 Primal-Dual Path Following Algorithm . . . . . . . . . . . . . . . . . 21917.4 Ellipsoid Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21917.5 Reflection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

6 CONTENTS

Chapter 1

Introduction

I was introduced to the world of personal computers in 1979. My high school had along table on which sat three or four Commodore PETs. Their memory capacity wasa few tens of kilobytes, the operating system relied on the Basic language, the screenwas nice monochrome that could display 40x25 or 80x25 extended ASCII charactertext, and external storage was on magnetic cassette tapes. I soon learned that Icould use Basic to program assembly code and wrote simple games that utilized thedisplay. The next year, the school obtained its first workstation. My friend and Iwrote a program to find perfect numbers.1 We already knew the first few – 6, 28,496, and 8128 – because, well, we were nerds. But we did not know any others. Weran our program overnight on the workstation and in the morning we saw on thescreen that our code had found the number 33550336. Only later did we find anindependent source to confirm our calculation (remember, no internet in 1980). I alsowrote a program to compute prime numbers. For a long time I carried around, in alarge cardboard box, reams of dot-matrix printer paper filled with prime numbers.

These forays into early personal computing had one long term effect – it helpedchannel my interests toward mathematics, computing and physics – and one shortterm effect – I was asked to write a computer program for the school. I attended aboarding school that once a year sponsored a student banquet. I was asked to writea computer dating program to match boys and girls for the banquet. Of course, Itook the assignment. The students each completed a multiple-choice questionnairethat I would use to find “good” matches. How I paired people up was entirely mydecision, but it was assumed that people who answered the questions similarly wouldbe good matches. I had no real idea how to find the best set of matches, but I didknow how to find a reasonable set of matches. I picked one person and found theperson (of the opposite sex) that had the most similar set of answers. These becamea match and these two people were removed from the pool of names. Then all I hadto do was repeat until no more matches were possible. It was clear to me that this

1A perfect number is a positive integer that is equal to the sum of its proper divisors. Forexample, 6 = 3 + 2 + 1.

7

8 CHAPTER 1. INTRODUCTION

procedure might not provide the best set of matches, but I had no idea how thatmight be done. My method provided many very good matches and a few very poormatches. Interestingly, one guy was matched with his twin brother’s girlfriend.

Twenty years later, I realized that the computer dating problem is an exampleof a mathematical optimization problem. And now I can solve it exactly and withcomplete assurance that the best set of matches is found. I can also avoid sociallyawkward matches. I wish I knew then what I know now. The problem can beformulated as what is called an integer program, which you, the reader, will be ableto formulate and solve after reading this text.

At the heart, optimization is an applied science driven by the desire to make optimaldecisions. Such questions arise in a surprising variety of disciplines. After all, our livesare defined by a series of decisions which we make consciously or unconsciously. Thedevelopment of optimization techniques bridges the gap between “this seems prettygood” and “this is the best possible option.” However, the success of optimization,as a mathematical field is the result of both applied and theoretical developments inconvex analysis, numberical analysis, computer science and linear algebra. Modernoptimization has roots in the works of many prominent mathematicians: Newton,Galileo, Bernoulli, Euler, Lagrange, Fourier, Cauchy. In the last century, optimizationbecame it’s own subfield of mathematics with contributions from mathematicians toonumerous to list. Perhaps the birth of applied optimization came in the late 1940s atthe confluence of the development of computers and some key algorithms such as theSimplex Method and Dynamic Programming. Great advancements have been madein the years since.

This textbook is but a very small piece of the puzzle and I cannot provide acomprehensive look at optimization. Instead, I have two major goals for the reader.First, I want to provide an understanding of the key ideas that are essential forunderstanding the majority of optimization problems and algorithms. Then, thereader has a knowledge base for extending their own study and reading. Second,I want to provide a large body of tools for modeling and solving the simplest (butextremely useful) optimization problems: mixed integer programs. Then, the readercan solve real-world problems with relative ease and confidence.

The text assumes that the reader is familiar with Linear Algebra (solutions ofsystems of linear equations, linear independence, rank, eigenvalues, positve definitematrices) and Calculus III (partial derivatives, gradient, Fourier expansion). However,these concepts can be reviewed or presented with care so that these courses need notbe prerequisites. We will make use of facts and remarks that more advanced textswould state and prove as theorems. The interested reader is invited to prove thestated remarks.

Chapter 2

Optimization Concepts andNotation

An mathematical optimization problem is expressed as a goal statement – the mini-mization or maximization of an objective function of decision variables. Optimizationproblems model real-world questions which seek some “best” answer.

I need to decide how many acres of lentils, wheat and barley to plant inorder to maximize my expected profit.

I need to decide how to distribute the raw materials in my furniture factoryin order to manufacture pieces of furniture that will maximize my profit.

I need to visit 16 towns in the Northwest and meet with my regionalmanagers. In what order should I visit them so that my cost is minimized?

How can I schedule my employee work hours so that my customers arebest served?

How should I pack my shipping containers so that my cost is minimized?

What is the best way to segment a city into police beats?

How can I maximize expected returns on investments under a variety ofrestrictions?

What is the best fit curve to my data?

Is there a solution to this Sudoku puzzle?

How do I assign neighborhoods to school districts in order to have themost efficient transportation plan?

9

10 CHAPTER 2. OPTIMIZATION CONCEPTS AND NOTATION

How can I best eliminate noise from a digital picture?

What should I pack for my three-week backpacking trip

Every mathematical optimization problem has certain elements. Let’s look atthese in turn.

Decision variables are those numerical quantities which we, as the person ask-ing the optimization question, have control over. In general, there are many deci-sion variables (as in the above examples). We tyically use an indexed variable set:x1, x2, . . . , xn and collect them into a column vector:

x =

x1

x2...xn

=[x1 x2 . . . xn

]T.

Decision variables may be real valued, xk ∈ R, or integer valued, xk ∈ Z, or somemixture of possibilities. If a vector of n decision variables consists of all integervalues, we write x ∈ Zn. Similarly, a vector of n real-valued decision variables iswritten x ∈ Rn. It is incorrect to write xk ∈ Zn or x ∈ Z. Why?

The objective function quantifies the quality of any choice of decision variables.It is typically a function f : Rn → R which inputs decision variables and outputs anobjective value. The objective value is a measure of the goodness (or badness) ofa particular choice of decision variables. Sometimes an objective function is definedonly on some restricted domain f : D → R, where D ⊆ Rn. This might be the casewhen one or more decision variables are integer valued and the objective functiononly makes sense for integer inputs.

Every optimization problem seeks the best possible choice of decision variables asmeasured by the objective function. The goal is to either maximize or minimizethis function. Finding the “best” answer can be very very difficult. Most algorithmsthat solve general optimization problems cannot guarantee a best answer. Usually,all that can be obtained is a “locally best” answer. Consider the following definitions.

Definition 1. Let f : D → R and x ∈ D. We say that x is a global minimizerof f if f(x) ≤ f(y) for all y ∈ D. We say that x is a global maximizer of f iff(x) ≥ f(y) for all y ∈ D.

Definition 2. Let f : D → R and x ∈ D. We say that x is a local minimizerof f if there exists r > 0 such that f(x) ≤ f(y) for all y ∈ D such that‖y− x‖ < r. We say that x is a local maximizer of f if there exists r > 0 suchthat f(x) ≥ f(y) for all y ∈ D such that ‖y − x‖ < r.

11

-2 -1 1 2

-1

1

2

x

f(x)

Figure 2.1: An example function f : R → R which has one global minimizer (nearx = −1), two local minimizers (near x = −1 and x = 3/4) and one local maximizer(near x = 1/3).

A local minimizer x returns the smallest objective value for all vectors in a localneighborhood of x. Consider the single variable example f : RtoR shown in Figure 2.1.The function has one global minimizer near x = −1, which is also a local minimizer.The function has a second local minimizer near x = 3/4. The function also has onelocal maximizer near x = 1/3.

Remark 1. Let f : D → R and x ∈ D. If x is a global minimizer of f then xis also a local minimizer of f . If x is a global maximizer of f then x is also alocal maximizer of f .

Most interesting optimization problems are constrained problems. Constraintsare inequalities or equalities which limit the possible choices of decision variables toan acceptable set Ω ⊂ Rn called the feasible region. For example, a problem mayrequire g(x) ≤ 0 or h(x) = 0.

With these elements, we can write a fairly general example optimization problem:

minx

z = f(x)

s.t. g(x) ≤ 0 (2.1)

h(x) = 0

x ∈ Rn

In words: Find a set of decision variables x which minimizes the objective value zdefined by function f , subject to the condition that g(x) ≤ 0 and the condition that


h(x) = 0, where all decision variables are real-valued. In this case, we can repre-sent the feasible region as Ω = x ∈ Rn | g(x) ≤ 0, h(x) = 0 and the optimizationproblem can be compactly represented as

minx∈Ω

z = f(x).

Notice that g : Rn → Rm (and similarly for h), representing m constriaints on the ndecision variables. For example, suppose we have the (m = 2) inequality constraintsin (n = 3) variables

3x1 + x2 − x2x23 ≤ 4,

lnx1 − x2 ≤ 10.

We write these constraints as y = g(x) ≤ 0 where

y1(x) := 3x1 + x2 − x2x23 − 4

y2(x) := lnx1 − x2 − 10

Remark 2. We often compare vector quantities to scalar quantities in equalitiesor inequalities. For example, we may write y ≤ 0 where y ∈ Rm and 0 ∈ R.This is a notational convenience which is (standard practice) shorthand for theelement-wise comparison: “yk ≤ 0, k = 1, 2, . . . ,m.”

Exercises

1. Consider the lentil, wheat and barley example. What might your decision vari-ables be? List several practical constraints which may be relevant.

2. Consider the traveling regional manager problem. What might your decisionvariables be? List several practical constraints which may be relevant.

3. Consider the shipping container packing examples. What might your decisionvariables be? List several practical constraints which may be relevant.

4. Write the following optimzation problem in the general form of Equation (2.1),defining all functions.

minx

z = x1 − sinx2 + ex3

s.t. x1 + x2 ≥ 1

3| cosx2| ≤ 1

5x1x2x3 = 1

x ∈ R3

13

5. Let x =

[x1

x2

]∈ R2. Sketch the regions in R2 corresponding to the following

conditions.

(a) x1 ≥ 1 and x2 ≥ 1.

(b) x ≥ 1

(c) x ≤ 2 and x1 ≥ 1.

(d) x = 4.

(e) x21 + x2

2 ≤ 3.

(f) x2 ≥ x21.

(g) x2 ≤ ex1 and x2 ≥ lnx1 and x1 ≥ 1.

6. Solve the following optimization problem by inspection (use your understandingof the problem statement to determine a global maximizer). Report both theoptimal decision variables, x∗, and the optimal objective value, z∗ = f(x∗).

maxx

z = 3x1 + x2

s.t. x1 + x2 ≤ 1

x ≥ 0

x ∈ R2

7. Sketch (and describe) a function f : R → R that has exactly three local mini-mizers, exactly one global maximizer, and exactly two global minimizers.

8. Sketch (and describe) a function f : R → R that has a local minimizer that isalso a local maximizer.

9. Sketch the feasible region Ω = x ∈ R2 | (x21 + (x2 − 3)2) ≤ 4, x1 ∈ Z.

Chapter 3

Optimization on Single-VariableFunctions

In this chapter, we will gather principles and techniques which will help us understandand solve linear programs (see Chapter 5). A full treatment of optimization on smoothfunctions is a year-long graduate course. We will develop the key ideas through a seriesof tasks. The reader is encouraged to explore each task in turn before examining thesupplied solution.

Note: In this chapter, we assume that functions f : D → R are twice continuouslydifferentiable. This means that f(x), f ′(x) and f ′′(x) are continuous functions.

3.1 Unconstrained Optimization

Task: Solve the following optimization problem.

minx∈R

z = f(x) = 2x2 − x.

Solution: Any extremal point (local maximum or minimum) of a function of onevariable must be a stationary point.

Definition 3. Let f : R → R and x ∈ R. We say that x is a stationary pointof f if f ′(x) = 0.

If, in addition, f ′′(x) > 0 (f ′′(x) < 0), then x is a local minimizer (maximizer) off . Why?

Unfortunately, if x is a stationary point of f and f ′′(x) = 0, then x may be aminimizer, a maximizer, neither or both! Using these ideas as guidelines, note thatf ′(x) = 4x− 1 and f ′(x) = 0 when x = 1/4. Thus, x = 1/4 is the unique stationary

15

16 CHAPTER 3. SINGLE-VARIABLE OPTIMIZATION

point of f . Furthermore, f ′′(x) = 4 and in particular, f ′′(1/4) = 4 > 0, so x = 1/4 isa local minimizer of f . We report the solution as

x∗ = 1/4, z∗ = −1/8.

Note: One must also check that f(x) does not achieve a lower value than z∗ asx→ ±∞. In this case, lim

x→±∞f(x) = +∞.


maxx∈R

z = f(x) = e−x2

.

Solution: As before, we consider all stationary points and the function behavior asx → ±∞. We have, f ′(x) = −2xe−x

2and f ′′(x) = (4x2 − 2)e−x

2. f ′(x) = 0 only

when x = 0 and f ′′(0) = −2 < 0. Thus, x = 0 is the unique stationary point, and itis a local maximizer with objective value z∗ = f(0) = 1. Notice that lim

x→±∞f(x) = 0.

Thus,x∗ = 0, z∗ = 1.


minx∈R

z = f(x) = ex + e−x.

Solution: As before, we consider all stationary points and the function behavior asx→ ±∞. We have f ′(x) = ex − e−x and f ′′(x) = ex + e−x. f ′(x) = 0 when ex = e−x

or x = −x, that is x = 0. Since f ′′(0) = e0 + e0 = 2, x = 0 is a local minimizer.Notice also that lim

x→±∞f(x) = +∞. Thus, the unique local (and global) minimizer is

x∗ = 0 with optimial objective value z∗ = 2.


maxx∈R

z = f(x) = ex + e−x.

Solution: In the previous task we found the stationary points and the limitingbehavior of the function. We found that f(x) can acheive as large a value as desiredby either increasing or decreasing the value of x. Thus, the problem is unbounded,and has no solution.

3.1. UNCONSTRAINED OPTIMIZATION 17


maxx∈R

z = f(x) = cos x.

Solution: First, we find all stationary points. We have f ′(x) = sinx. So, f ′(x) = 0for x = kπ and any k ∈ Z. The second derivative is f ′′(x) = − cosx and we find thatf ′′(x) = −1 < 0 whenever k is an even integer. Thus, we have a set of local optimalpoints X = x ∈ R | x = 2kπ, k ∈ Z. These points are also global optimal points forwhich z∗ = f(x∗) = 1. Thus, the solution

x∗ ∈ X, z∗ = 1.


maxx∈R

z = f(x) =x√x2 + 1

.

Solution: First, we locate any stationary points. We have f ′(x) = (x2 + 1)−3/2.Since f ′(x) 6= 0 for any choice of x ∈ R, the function has no stationary points. Sinceany local maximizer must be a stationary point, this function has no solution.

Task: Show that the function f(x) = sin4 x has a global minimizer at x = 0 withf ′′(x) = 0.

Solution: We know that 0 ≤ sin4 x ≤ 1 for all x ∈ R and sin4 0 = 0, so x = 0 isa global minimizer of f . We have f ′(x) = 4 sin3 x cosx (Notice that f ′(0) = 0), andf ′′(x) = 4 sin2 x(3 cos2 x − sin2 x) with f ′′(0) = 0. We see that the second derivativetest would fail to identify this point as even a local minimizer.


The single-variable functions we have used as examples are shown below. Thereader is encouraged to revisit each of the previous tasks and verify visually thecorrectness of each solution.

-2 -1 1 2

-1

1

2

3

x

f(x)=2x2 − x

-2 -1 1 2

0.5

1.5

x

f(x)=e−x2

-2 -1 1 2

1

2

3

x

f(x)=ex + e−x

−2π −π π 2π

-1

1

x

f(x)= cos(x)

-3 -2 -1 1 2 3

-1

1

x

f(x)= x√x2+1

−2π −π π 2π

-1

1

x

f(x)= sin4(x)

3.1. UNCONSTRAINED OPTIMIZATION 19

Let’s summarize what we know about solving optimization problems on functionsof one variable, f : R→ R. First, we have what is called a necessary condition: Everylocal optimum is a stationary point.

Remark 3. If x is a local extremum (maximizer or minimizer) of f : R→ R,then x is a stationary point of f . If x is a stationary point of f , then x mayor may not be a local extremum of f .

Some optimization problems have no solution because of the limiting behavior.

Remark 4. For a function f : R→ R, the optimization problem maxx∈R

z = f(x)

has no solution if either limx→+∞

f(x) > f(y) or limx→−∞

f(x) > f(y) for all y ∈ R.

Furthermore, if limx→+∞

f(x) = +∞ or limx→−∞

f(x) = +∞, then the problem is

also said to be unbounded.

A similar statement for the above remark can be made for a minimization problem.Be sure that you understand the difference between optimization problems with nosolution and unbounded optimization problems.

Next, we have a second order sufficient condition. This condition provides adefinite test for an local optimal point.

Remark 5. If x is a stationary point of f : R→ R and f ′′(x) > 0 (f ′′(x) < 0),then x is a local minimizer (maximizer) of f .

We have also made use of one other sufficient condition.

Remark 6. Let f : R → R and suppose a ≤ f(x) for some a ∈ R and allx ∈ R. If f(y) = a then y is a global minimizer of f .Similarly, let f : R → R and suppose f(x) ≤ b for some b ∈ R and all x ∈ R.If f(y) = b then y is a global maximizer of f .

The above Remarks contain all of our tools for examining twice continuouslydifferntiable single-variable functions. However, they also lead to some Corollaryideas.

Remark 7. If a function f : R → R has no stationary points, then it has nolocal extrema, and therefore no global exrema.


Remark 8. If x is a stationary point of f : R→ R and f ′′(x) = 0, then x maybe a local minimizer, a local maximizer, neither or both.

Putting these ideas together in a formal procedure for solving unconstrained op-timization problems on functions of one variable is the subject of Exercise 1.

3.2 Constrained Optimization

Next, we consider optimization problems with constraints on the domain. Again, wewill understand how best to address these types of problems through a series of tasks.

Task: Solve the following optimization problem (see the related first task of thischapter).

minx∈R

z = f(x) = 2x2 − x

s.t. 0 ≤ x ≤ 1

Solution: We previously found that the function f has a single stationary point atx = 1/4 with objective value z = −1/8. This stationary point was shown to be theglobal minimum of the function over R. We now observe that this stationary pointis a feasible point for the current task.

Definition 4. A point x ∈ Ω, where Ω is the feasible region of an optimizationproblem is called a feasible point.

Since the global optimum of the unconstrained problem is a feasible point, it isalso the solution to the contrained problem: x∗ = 1/4, z∗ = −1/8.


minx∈R

z = f(x) = 2x2 − x

s.t. 1 ≤ x

3.2. CONSTRAINED OPTIMIZATION 21

Solution: Building on the discussion in the previous tasks, we see that the globaloptimum for the unconstrained problem is not a feasible point for the constrainedproblem. (We can only consider x ≥ 1.) The feasible region (feasible interval in thiscase), Ω = [1,∞) contains no stationary points. However, a minimizer does exist(see the figure below). Notice that for x∗ = 1, f(x∗) ≤ f(y) for all y ∈ Ω. Thus,x∗ = 1 is the global minimum for this constrained problem, with objective valuez∗ = f(x∗) = 1. Because of the constraint, we found a minimizer at the end of thefeasible interval. Notice that this global optimizer is not a stationary point of f .

-2 -1 1 2

-1

1

2

3

Ω x

f(x)=2x2 − x


minx∈R

z = f(x) = 2x2 − x

s.t. 1 ≤ x

− 5 ≤ x ≤ 0

Solution: Notice that the feasible region, those points in R that satisfy all con-straints, is the empty set. That is, no value for x satisfies both 1 ≤ x and −5 ≤ x ≤ 0.If an optimization problem has a feasible region that is empty, the problem has nosolution. We say that such a problem is infeasible. We were able to determine thatthis problem is infeasible by inspection. However, in general such determinations canbe quite difficult, especially for decision variables in high dimensions or for problemswith very many constraints.



minx∈R

z = f(x) = 2x2 − x

s.t. x = 3

Solution: Notice that the feasible set contains only one possibility for a decisionvariable x = 3 (in R). Thus, the feasible point must also be optimal. The solution is:x∗ = 3, z∗ = 15.

Two final examples which incorporates many ideas together.


minx∈R

z = f(x) = 4x3 − 3x2 − 6x

s.t. x ≥ 2

Solution: First, we can easily establish that this problem is not infeasible becauseany x ≥ 2 is a feasible point. Next, we consider all of the stationary points of f : wefind that there are two, x = 1 and x = −1/2. By the second derivative test, x = 1is a local minimizer and x = −1/2 is a local maximizer. (Try it.) However, neitherstationary point is feasible, and we need only consider feasible stationary points. Sowhat is the solution? The only remaining possibility is that an optimal point may lieat the end points of the feasible region. At x = 2, z = 8 and for x→∞, z →∞. Sothe solution must be x∗ = 2, z∗ = 8. In fact, we must always check the end points ofthe feasible region. Why?

-1 1 2 3

-5

5

10

Ω x

f(x)=4x3 − 3x2 − 6x



minx∈R

z = f(x) = 4x3 − 3x2 − 6x

s.t. x ≥ 0

Solution: This is the same as the previous problem except the feasible region isdifferent. Now the feasible region includes the local minimizer which we found atx = 1, where z = −5. We must be careful and still check the end points of thefeasible region. We already know that for x → ∞, z → ∞. We also have that forx = 0, z = 0. So, we have two candidate optimizers, one at x = 1 and the other atx = 0. The best point is x∗ = 1, z∗ = −5.

-1 1 2 3

-5

5

10

Ω x

f(x)=4x3 − 3x2 − 6x

Be sure that you understand this fact: It is not always the case that the globalminimizer of an optimization problem is also a stationary point of the objectivefunction, even if stationary points are feasible.

Remark 9. Let f : [a, b] → R where a, b ∈ R. If x ∈ D is a global minimizeror maximizer of f on [a, b], then x is a stationary point of f or x = a or x = b.


The above remark can be generalized for feasible regions of multiple closed inter-vals and for unbounded intervals such as [a,∞).

Exercises

1. Using the remarks from the section, and any other facts you have discovered,create a general procedure for solving any optimization problem with objectivefunction f : R→ R and no constraints. Be careful to address the possibility thattwo local optima have different objective values. Also be careful to recognizethe possibility that f ′′(x) = 0 at a stationary point.

2. Graph the function f(x) = x4 − 2x2 and verify that it has three stationarypoints, one of which is a local maximizer and two of which are local minimizers.

3. Show that the function f(x) = x4 +x3 has exactly two stationary points, one ofwhich is a local minimizer and one of which cannot be classified by the secondderivative test. Graph the function and describe the stationary points.

4. Plot the function f(x) = 11+x2

+ e−x and verify that it has no stationary points.

5. Solve the following optimization problems.

(a) min z = f(x) = x2 + 3x− 4 subject to x ∈ R.

(b) min z = f(x) = xex subject to x ∈ R.

(c) min z = f(x) = ln(x2 + x+ 1) subject to x ∈ R.

(d) max z = f(x) = 1− e−x subject to x ∈ R.

6. Provide four illustrative example functions, one for each of the four possibilitiesof Remark 8. Do not use any functions already discussed in these notes. Justifyyour choices.

7. Using the remarks from the section, and any other facts you have discovered,create a general procedure for solving any optimization problem with objectivefunction f : R → R and domain constraints. Be careful to address the possi-bility that two local optima have different objective values. Also be careful torecognize the possibility that f ′′(x) = 0 at a stationary point.

8. Solve the following optimization problems.

(a) min z = f(x) = x2 + 3x− 4 subject to −1 ≤ x ≤ 2.

(b) min z = f(x) = xex subject to −2 ≤ x ≤ 2.

(c) min z = f(x) = ln(x2 + x+ 1) subject to x ≤ −2.

(d) max z = f(x) = 1− e−x subject to 0 ≤ x ≤ 4.

Chapter 4

Optimization on Multi-VariableFunctions

In this section, we continue our exploration of optimization on smooth functions,considering both unconstrained and constrained problems. We will again proceed bya series of tasks.

4.1 Unconstrained Optimization


minx∈R2

z = f(x) = (1− x1)2 + 10(x2 − x21)2.

Solution: This problem can be solved “by inspection.” Notice that the objectivefunction is the sum of two squares so that f(x) ≥ 0 for any choice of x. Thus, weseek decision variable values for which f(x) = 0, if possible. The first term can onlybe zero valued when x1 = 1. And the second term can only be zero when x1 = x2.Thus, the solution is

x∗ =

[11

], z∗ = 0.


minx∈R2

z = f(x) = x21 + x2

2 − x1x2.

Solution: We first seek stationary points of the objective function.

25

26 CHAPTER 4. MULTI-VARIABLE OPTIMIZATION

Definition 5. Let f : Rn → R and x ∈ Rn. Vector x is said to be astationary point of f if ∇f(x) = 0.

For multivariable functions, stationary points are points at which the gradient ofthe function is the zero vector: ∇f(x) = 0. Recall that the gradient of the functionat x is the direction and magnitude of steepest ascent at x. Thus, it makes sensethat at local minimizers or maximizers, the gradient must be zero. For the functionin question:

∇f(x) =

[∂f/∂x1∂f/∂x2

]=

[2x1 − x2

−x1 + 2x2

],

and x =[0 0

]Tis the unique stationary point. The Hessian matrix of second order

derivatives contains information about the local curvature of the function and can beused to determine if stationary points are local maxima or minima (or neither). Wehave the Hessian matrix

∇2f(x) =

[∂2f/∂x21 ∂2f/∂x1∂x2

∂2f/∂x2∂x1 ∂2f/∂x22

]=

[2 −1−1 2

].

If every eigenvalue of the Hessian matrix is positive (negative) at a stationary point,then the stationary point is a local minimizer (maximizer). If the eigenvalues are ofmixed sign, then the stationary point is a saddle point. If any eigenvalue is zero, thenthen the nature of the stationary point cannot be determined from the eigenvalues.In this case, the eigenvalues of the Hessian are λ = 1, 3, which are both positive.Thus, the stationary point is a local minimizer.

Finally, we note that limt→∞

f(td) = +∞ for all nonzero d ∈ R2. This means that

in any fixed direction d, the function f increases without bound as the distance fromthe origin increases (see Exercise 3). Thus, the (global minimizing) solution

x∗ =

[00

], z∗ = f(x∗) = 0.

4.2 Constrained Optimization


maxx

z = f(x) = (the land elevation at location x)

s.t. x ∈ Ω = (Whitman County, Washington)

Solution: This problem asks us to find the point of highest elevation in WhitmanCounty, Washington, x∗, and the highest elevation, z∗ = f(x∗). To complete thistask, we might naturally check the elevation values of mountain peaks and choose


the highest. Notice the similarities in this procedure and the analytical procedure ofthe previous section. Each peak is a local maxima and for each we must evaluate theobjective function. Notice also that if the elevation were provided as a function, f(x),it is likely the case that computing stationary points is likely to be computationallyintractable. But even so, there is a larger difficulty in that, the solution to this problemmay not correspond to a stationary point at all. The highest point in the county maynot occur at a peak. Be sure that you know how this can happen. This is whatmakes problems with constraints potentially much more difficult than unconstrainedproblems. As it turns out, the solution happens to be a peak.

x∗ = (Tekoa Mountain), z∗ = 4009 feet.


minx

z = f(x, y) = e−(x−1)2−y2

s.t. x+ 2y ≥ 10

Solution: First, compute the stationary points of f .

∇f(x, y) = f(x, y)

[−2(x− 1)−2y

]set= 0,

with unique solution (x, y) = (1, 0). This vector is not feasible, so optimal points (ifthey exist) must be on the boundary of the feasible region.

The feasible region, Ω = (x, y) ∈ R2 | x+ 2y ≥ 10, has two boundary features:the line x + 2y = 10 and the fact that Ω is unbounded in the half-space above thisline. Notice that the objective value approaches zero along any path for which ‖(x, y)‖becomes arbitrarily large. That is

limt→∞

f(ta, tb) = 0 for all nonzero (a, b) ∈ R2.

Next, check the boundary defined by x + 2y = 10. To do this, simply use thisequality to find the objective value function along the boundary line by substitution.

g(y) = f(10− 2y, y) = e−(9−2y)2−y2 = e−81+36y−5y2 .

We seek local maximizers of this function. We have

g′(y) = (−10y + 36)g(y)set= 0.

Since g(y) > 0 for all y ∈ R, we find the single stationary point at y = 18/5. Thesecond derivative may tell us about the nature of this stationary point:

g′′(y) = ((−10y + 36)2 − 10)g(y), and g′′(18/5) = −10g(18/5) < 0.


Thus, y = 18/5 is the global maximizer of g and (x, y) = (14/5, 18/5) is the globalmaximizer of f constrained to be in Ω. The answer is:

(x∗, y∗) = (14/5, 18/5), z∗ = f(x∗, y∗) = e−81/5.

Before we leave this example, consider the related problem

minx

z = f(x, y) = e−(x−1)2−y2

s.t. x+ 2y ≤ 10

Here, the constraints are such that the unique stationary point of f is now feasible.We can now verify that (x, y) = (1, 0) is a local (and global) maximizer of f (for boththe constrained and unconstrained problems). We use the second derivative test andfirst compute the Hessian:

∇2f(x, y) = f(x, y)

[4(x− 1)2 − 2 4y(x− 1)

4y(x− 1) 4y2 − 2

].

∇2f(1, 0) =

[−2 00 −2

].

The eigenvalues of this matrix are −2 and −2. Since they are all negative, thestationary point (1, 0) is a local maximizer with objective value f(x, 0) = 1. As wehave seen, the boundary of the feasible region does not provide any points of largerobjective value, so the answer is:

(x∗, y∗) = (1, 0), z∗ = f(1, 0) = 1.

4.3 How to Compute Eigenvalues

Computing eigenvalues by hand can be a tedious business. We will use two basicmethods. First, note that if the matrix in question is a diagonal matrix, then theeigenvalues, λ, are the diagonal entries. A diagonal matrix is one in which the onlynonzero entries are along the main diagonal. Here are some examples:

A =

−2 0 00 4 00 0 2

, λ = −2, 4, 2

A =

5 0 00 0 00 0 1

, λ = 5, 0, 1

4.3. HOW TO COMPUTE EIGENVALUES 29

A =

1 0 00 0 00 0 1

, λ = 1, 0, 1

If you need the eigenvalues of a matrix which is not diagonal, you can use anycomputing software. Matlab/Octave is useful for this task. Suppose you need tocompute the eigenvalues of the following matrix:

A =

[2 −1−1 3

].

Here is example Matlab/Octave code and output illustrating this task:

>> A=[2 -1 ; -1 3];

>> eig(A)

ans =

1.3820

3.6180

The eigenvalues are (approximately) 1.3820 and 3.6180.

Exercises

1. Verify that the following matrix has one negative and two positive eigenvalues.

M =

1 2 13 1 21 2 2

.2. Write the general forms of the gradient vector ∇f(x) and the Hessian matrix∇2f(x) for functions f : Rn → R.

3. Suppose f(x) = x21 + x2

2 − x1x2. Show that limt→∞

f(td) = +∞ for all nonzero

d ∈ R2.

4. Find the five stationary points of the function f(x, y) = x2 + 2y2 + e−3x2−3y2

and classify each using the second derivative test.

5. Find the four stationary points of the function f(x, y) = 3x2(y−1)+y2(y−3)+2and classify each using the second derivative test. Finally, solve the optimizationproblem min

x∈R2z = f(x, y) subject to x2 ≤ 1− y2.

6. Solve the following optimization problems


(a) minx,y∈R

z = f(x, y) = 2x2 − 2xy + y2 + 2x+ y − 5

(b) minx,y∈R

z = f(x, y) = 2x2 − 2xy + y2 + 2x+ y − 5 subject to x+ y ≥ 0.

(c) minx,y∈R

z = f(x, y) = 2x2 − 3xy + y2 + 2x+ y − 5

(d) maxx,y∈R

z = f(x, y) = e−x2−2y2 subject to 2x+ y ≥ 1.

(e) minx,y∈R

z = f(x, y) = 3x2y + y3 − 3x2 − 3y2 + 2 subject to y ≤ 1.

(f) maxx,y∈R

z = f(x, y) = ex sin(x+ y) subject to |x| ≤ 1 and |y| ≤ 1.

(g) minx,y∈R

z = f(x, y) = ex sin(x+ y) subject to |x| ≤ 1 and |y| ≤ 1.

Chapter 5

Linear Programming

We now consider a special class of constrained optimization problems known as LinearPrograms.

Definition 6. A linear program (LP) is an optimization problem of the form:

minx

z = cTx

s.t. Ax ≤ b

Aex = be

x ∈ Rn

where c ∈ Rn, b ∈ Rm, be ∈ Rm′, A an m×n matrix and Ae and m′×n matrix.

The objective and constraint functions of a linear program are expressed as ma-trix operations and are therefore linear functions. Notationally, we have n decisionvariables, m inequality constraints and m′ equality constraints.

Many linear problems have the natural constraints that some or all decision vari-ables are restricted to be integer-valued. These (very common) problems have specialnames and solution strategies which are the subject of later chapters.

Definition 7. A mixed-integer program (MIP) is a linear program with theadditional restriction that some decision variables are restricted to the set ofintegers. An integer program (IP) is a linear program with the additional re-striction that all decision variables are restricted to the set of integers.

31

32 CHAPTER 5. LINEAR PROGRAMMING

5.1 Elements of Linear Programs

Consider the properties of linear programs. We consider the particular lements: de-cision variables, objective function, inequality constraints, equality constraints, goalstatement.

Decision Variables. The decision variables of a linear program are always realvalued, x ∈ Rn. We always write the set of decision variables as a column vector:

x =

x1

x2...xn

=[x1 x2 . . . xn

]T.

The elements of x are called decision variables and the vector itself is called thedecision variable vector.

Objective Function. The objective function of a LP is always a linear functionz = f(x) = cTx, where c ∈ Rn is a given constant vector. For example, we can have

f(x) = 3x1 + 4x2 − 4x3 + 7x4,

in which case

f(x) = cTx =[3 4 −4 7

] x1

x2

x3

x4

.That is, c is the (constant) column vector given by

c =

34−47

.Objective functions of linear programs are examples of linear functions.

Definition 8. A function f : Rn → R is a linear function if there exists a

constant vector c ∈ Rn such that f(x) = cTx.

Definition 9. A function f : D → R is a linear function if f(ax + y) =af(x) + f(y) for all x, y ∈ D and all a ∈ R.

5.1. ELEMENTS OF LINEAR PROGRAMS 33

Example. Consider the function f : R2 → R defined by f(x) = 3x1 + 4x2. We candefine

c =

[34

]and x =

[x1

x2

], so that f(x) = cTx.

So, by Definition 8, f(x) is a linear function.

Example. Consider the function f : R → R defined by f(x) = 2x + 3. Observe:f(ax+ y) = 2(ax+ y) + 3 = a(2x+ 3) + 2y = af(x) + f(y)− 3 6= af(x) + f(y). Thus,f(x) is not a linear function. This function is an example of an affine function.

Definition 10. A function f : D → R is an affine function if there exists ascalar a such that f(x) + a is a linear function.

Inequality Constraints. The inequality constraints of a linear program can bewritten as Ax ≤ b, where x ∈ Rn, A is an m × n real-valued constant matrix, andb ∈ Rm.

Example. Suppose we have the constraint set

2x1 + 3x2 + 4x3 ≤ 5

6x1 + 7x2 + 8x3 ≤ 9

We can represent this set of inequalities as the matrix inequality Ax ≤ b where

x =

x1

x2

x3

, A =

[2 3 46 7 8

], b =

[59

].

Here the left-hand-side expressions are linear functions. We could write

g1(x) = 2x1 + 3x2 + 4x3 and g2(x) = 6x1 + 7x2 + 8x3,

and define g : R3 → R2 as g(x) = Ax. Because the left-hand-side functions of linearprogram inequalities are linear functions, we will say that the inequality constraintsof linear programs are linear.

Example. The constraint set

3x2 − 4x1 − 5 ≥ 4

x1 − 6x3 + x4 ≤ 2

−x1 − x2 − 2x4 ≤ 5 + x3


can be written Ax ≤ b, where

x =

x1

x2

x3

x4

, A =

4 −3 0 01 0 −6 1−1 −1 −1 −2

, b =

−925

.Equality Constraints. The equality constriants Aex = be are represented similarlyto the inequality constraints. An example of mixed equality and inequality constraintswill round out this discussion.

Example. The constraint set

3x1 + 2x2 − x3 ≤ 4

x1 + x2 + x3 = 5

−x1 + 3x2 = 4

−x1 + 3x2 + x3 ≤ 7

can be written Ax ≤ b and Aex = be, where

x =

x1

x2

x3

, A =

[3 2 −1−1 3 1

], b =

[47

], Ae =

[1 1 1−1 3 0

], be =

[54

].

Maximizing Objective Functions. Thus far, we have expressed linear programsas minimization problems. However, there is no fundamental difference between mini-mizing or maximizing a function. The following two optimization problems are equiv-alent. Be sure that you agree.

minx∈Rn

z = f(x) s.t. x ∈ Ω

maxx∈Rn

w = −f(x) s.t. x ∈ Ω

Exercises

1. Classify each of the following optimization problems as a LP, IP, MIP or noneof the above. Explain your reasoning.

5.1. ELEMENTS OF LINEAR PROGRAMS 35

(a)

minx

z = 3x1 + x2 + 3x3

s.t. x2 − x3 = 5x1

x1 + x2 ≥ 3

x2 ≤ x3

x ∈ Z3

(b)

minx

z = 3x1 + x2 + 3x2x3

s.t. x2 − x3 − 5x1 = 2

x1 + x2 ≥ 3

x1 ∈ Rx2, x3 ∈ Z

(c)

minx

z = x1 + x2 + x3

s.t. x2 − x3 − 2x1 = −4.7713x1 + x3 ≥ 3.32

x ∈ Z3

(d)

minx

z = 0

s.t. x2 − x3 − 2x1 = 4

x1 + x3 ≥ sin(3)

x ∈ R3

(e)

minx

z = x1 + x2

s.t. x1 + x2 = 4

x1 + x2 = 3

x ∈ R2


(f)

minx

z = x1 + x2 − 2x3

s.t. x1 ≤ x2 − x3

x1 + x2 = 3 + x3

x3 ≥ x2

x ∈ R3

2. Express the following set of constraints in matrix form.

x1 − 2x2 + 4x3 − x4 ≤ 2

x1 + 2x2 + x4 = 5

−x1 + 9x2 − 3x3 = 0

−x2 + 3x3 + x4 ≤ −1

x1 = x2 − 3x3

3. Express the constraint |x1 + x2 − x3| ≤ 4 as a set of linear inequalities.

4. For what values of a and b is the function f : R → R, defined by y = f(x) =ax+ b, a linear function?

5. For what values of a and b is the function f : R → R, defined by y = f(x) =ax+ b, an affine function?

6. Suppose f(x) = x1x2 + 3x1 − 4x3. If f(x) is expressed as

f(x) =[3 x1 −4

] x1

x2

x3

,then is f a linear function? Explain.

7. Express the following optimization problem as an integer program.

minx

z = x1 + x2 + 3x3

s.t. x1 − x2 ≥ 4

x1 + x2 + x3 ≥ 0

x ∈ 0, 13

Here, x ∈ 0, 13 indicates that each decision variable, x1, x2, x3, is restrictedto be a binary variable with values either 0 or 1.

Chapter 6

Standard Forms for LinearPrograms

In this section we explore the idea that a given linear program can be expressed inmany ways while still conforming to the general form given in the previous section.This may (and probably should) come as a surprise and make the mathematicalresearcher slightly uncomfortable. After all, uniqueness of objects makes them easierto manipulate. In optimization we have several standard forms which we use mostoften. These standard forms have arisen for two basic reasons.

1. Researchers can define subclasses of linear programs. A common frame of ref-erence is essential in understanding the relationships between these subclasses.Of particular interest is understanding the applicability of various solution al-gorithms to linear programs.

2. Software solvers are developed to solve general linear programs, but they requirethe input data to conform to certain standard forms.

We will describe several such standard forms. Keep in mind that any LP can bewritten in any of these standard forms. This may not be clear as you read throughthem, but wait for the end of this section where we describe how to recast any linearprogram into any standard form.

6.1 Important Standard Forms

Standard Form. Historically speaking, the standard form known as Standard Formis known as such because it was the first form for which a solution algorithm wasdeveloped. Using the notation Mm×n(R) to be the set of m × n matrices with real-valued entries, we present the following definition.

37

38 CHAPTER 6. LP STANDARD FORMS

Definition 11. A standard form linear program is a LP expressed in the form

minx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ Rn

where c ∈ Rn, b ∈ Rm, A ∈Mm×n(R), and rank(A) = m.

Standard form differs from the general form given earlier in that the only inequalityconstraints are sign constraints, x ≥ 0. The Simplex Method (Algorithm) used forsolving LPs begins with this form. We will study the Simplex Method later in thistext.

Inequality Forms. Basic inequality forms are important for understanding someessential geometric features of linear programs and the starting forms for some solvers.

Definition 12. An inequality standard form linear program is a LP expressedin one of the following forms

minx

z = cTx

s.t. Ax ≥ b

x ≥ 0

x ∈ Rn

maxx

z = cTx

s.t. Ax ≤ b

x ≥ 0

x ∈ Rn

where c ∈ Rn, b ∈ Rm, A ∈Mm×n(R).

Matlab Form. Solving linear programs in Matlab makes use of the functionlinprog which assumes the following form.

6.1. IMPORTANT STANDARD FORMS 39

Definition 13. A Matlab standard form linear program is one which is inthe form

minx

z = cTx

s.t. Ax ≤ b

Aex = be

` ≤ x ≤ u

x ∈ Rn

where c, `, u ∈ Rn, b ∈ Rm, A ∈ Mm×n(R), be ∈ Rm′, Ae ∈ Mm′×n(R). Note:

elements of ` are allowed to take on the value −∞ and elements of u are allowedto take on the value +∞ in order to account for variables that are not boundedabove and/or below.

The user supplies the matrices c, `, u, b, be, A,Ae, if they exist. Then the followingfunction call will attempt to solve the corresponding linear program, returning anoptimal decision variable vector x and optimal objective value z.

[x,z]=linprog(c,A,b,Ae,be,l,u);

Matlab also uses the function intprog to solve integer programs, and functionintlinprog to solve mixed-integer programs. These functions have similar syntax.

Definition 14. Constraints of the type ` ≤ x ≤ u, where x ∈ Rn and `, u areconstant vectors in Rn are called bound constraints or box constraints.

It is good practice to use box constraints whenever possible because many algo-rithms for solving optimization problems are more computationally efficient when boxconstraints can be specified.

octave Form. octave uses a single function, glpk, to solve all types of mixed-integer programs.


Definition 15. A octave standard form mixed-integer program is one whichis in the form

minx

or maxx

z = cTx

s.t. Ax 〈compare〉 b` ≤ x ≤ u

xk ∈ R or xk ∈ Z, k = 1, 2, . . . , n

where c, `, u ∈ Rn, b ∈ Rm, A ∈ Mm×n(R). Here 〈compare〉 means that indi-vidual constraints can be specified to be upper bound (≤), lower bound (≥), orequality (=) constraints. Note: elements of ` are allowed to take on the value−∞ and elements of u are allowed to take on the value +∞ in order to accountfor variables that are not bounded above and/or below.

Example solutions, including octave code will be presented as needed. For now,we are simply cataloging some useful forms of linear programs, integer programs andmixed-integer programs. You will notice that each of the standard form definitionsis written using matrix notation. We say that these programs are written in matrixform. Shown below are identical linear programs, the one at right is the matrix formequivalent to the one at left. Both are in standard form.

minx

z = x1 + x2 + 3x3

s.t. 2x1 − x2 = 9

x1 + x2 + x3 = −2

x ≥ 0

x ∈ R3

maxx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ R3

c =

113

, A =

[2 −1 01 1 1

], b =

[9−2

]

Writing linear programs in matrix form is useful and important because softwaresolvers require matrix input values such as c, A and b.

6.2 Converting LPs between Standard Forms

We have already seen how one can convert a minimization problem into a maximiza-tion problem, and vice versa. The key idea is that minimizing an objectve functionis identical to maximizing the negative of the function. The most difficult part ofconverting between various forms is converting one type of constraint into another.

6.2. CONVERTING LPS BETWEEN STANDARD FORMS 41

Here we give several examples.

1. Any upper bound constraint (≤) can be converted to a lower bound constraint(≥) (and vice versa) by multiplying the inequalty by any negative constant,typically −1. For example,

3x1 − 7x2 + x3 ≤ 5

is equivalent to−3x1 + 7x2 − x3 ≥ −5.

2. Any equality constraint can be converted to a pair of inequality constraints.For example,

x1 + x2 + x3 = 1

is equivalent to the constraint pair

x1 + x2 + x3 ≤ 1,

x1 + x2 + x3 ≥ 1.

3. An inequality constraint can be converted to an equality constraint and a signconstraint. For example,

x1 + x2 + x3 ≤ 7


x1 + x2 + x3 + x4 = 7,

x4 ≥ 0.

In this example, we created a new non-negative variable whose value is thedifference between left and right-hand sides of the original inequality. This newvariable is known as a slack variable. Here is another example:

x1 + x2 ≥ 3


x1 + x2 − x3 = 3,

x3 ≥ 0.

In this example, we created a non-negative slack variable x3. Notice that itmust be subtracted from x1 + x2 in order to achieve the right-hand-side value3. Sometimes, slack variables are refered to as either an excess variable or adeficit variable depending on whether it measures an excess or deficit of theleft-hand-side expression relative to the constant right-hand-side value.


4. Suppose we wish a non-positive variable x1 ≤ 0 to be non-negative as is therequirement of standard form. In this case, we replace the variable with a newvariable, say x9 = −x1. Then the constraint x1 ≤ 0 becomes x9 ≥ 0. Thevariable x1 is replaced everywhere in the problem by −x9.

5. Suppose we have a variable that is not restricted in sign (neither x1 ≥ 0 norx1 ≤ 0), and we wish to express our LP in standard form in which all decisionvariables must be non-negative. In this case, we can write this variable asthe difference of two non-negative variables, say x8 and x9. That is, we letx1 = x8 − x9, x8 ≥ 0 and x9 ≥ 0. Then x1 is replaced everywhere in theproblem by x8 − x9.

Conversion Example #1. Convert the following linear program into matrix stan-dard form.

minx

z = x1 + x2 − x3

s.t. x1 + x2 ≤ 3

x1 − x3 = 4

x2 + 2x3 ≥ 1

x1, x2 ≥ 0

x ∈ R3

The problem is already a minimization problem and has a linear objective function.There are three decision variables, one of which is unrestricted in sign. We mustconvert the constraints to be equality constraints and have only non-negative decisionvariables. We begin by adding non-negative slack variables to the two inequalityconstraints:

x1 + x2 + x4 = 3

x2 + 2x3 − x5 = 1

x4, x5 ≥ 0

Next, we introduce two more non-negative decision variables so that x3 can takeon negative values as the original problem allows.

x3 = x6 − x7

x6, x7 ≥ 0


We use this equality to eliminate x3 from the problem (by substitution) arrivingat the new standard form optimization problem:

minx

z = x1 + x2 − x6 + x7

s.t. x1 + x2 + x4 = 3

x1 − x6 + x7 = 4

x2 − x5 + 2x6 − 2x7 = 1

x ≥ 0

x ∈ R6

where x =[x1 x2 x4 x5 x6 x7

]T. Notice that x3 is no longer in the optimiza-

tion problem, while all remaining variables are non-negative. In order to recover x∗3,solve for x∗ and then x∗3 = x∗6 − x∗7. Finally, we can write this LP in matrix standardform.

minx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ R6

x =

x1

x2

x4

x5

x6

x7

, A =

1 1 1 0 0 01 0 0 0 −1 10 1 0 −5 2 −2

, b =

341

.

Conversion Example #2. Convert the following linear program into standardform.

maxx

z = cTx+ 4

s.t. Ax = b

x ≥ 0

x ∈ R2

There are two issues. First, the problem is a maximization problem. Second, theobjective function is not linear – it is an affine function. We can define w = −z + 4


to obtain the following standard form problem.

minx

w = −cTx

s.t. Ax = b

x ≥ 0

x ∈ R2

Then, when w∗ is obtained, we have z∗ = −w∗ + 4.

Exercises

1. Suppose f : Rn → R and g : R → R are linear functions. Is g f a linearfunction? Justify your answer.

2. Convert the following linear programs into matrix Standard Form.

(a)

minx

z = x1 + x2 + x3

s.t. x1 + x2 ≤ 3

x1 + x3 ≤ 4

x2 + 2x3 ≥ 1

x2, x3 ≥ 0

x ∈ R3

(b)

maxx

z = x1 + x2 + x3

s.t. x1 + x2 ≥ 3

x1 − x3 = 4

x ≥ 0

x ∈ R3

(c)

maxx

z = dTx

s.t. Dx ≤ e

x ≥ 0

x ∈ Rn

where D is an m× n real matrix, e ∈ Rm and d ∈ Rn.


3. Convert each of the linear programs from Exercise 2 into octave standard formby providing the necessary matrices c, A, b, `, u and specifying the sense of eachconstraint.

4. Convert each of the linear programs from Exercise 2 into Matlab standardform by providing the necessary matrices c, A, b, Ae, be, `, u.

Chapter 7

Using octave or Matlab to SolveMixed Integer Programs

octave and Matlab both use built-in functions that attempt to solve mixed-integerprograms. octave uses the function glpk to solve general mixed integer programs.Matlab uses the function linprog to solve linear programs, intlinprog to solvemixed integer programs, intprog to solve integer programs, and bintprog to solvebinary integer programs. This section details how to use these functions using exam-ples.

When solving any optimization problem using octave or Matlab , it is recom-mended that you create a text file which contains all of the necessary commands.This text file is called a script file and must have extension “.m” to run properly. Asa specific example, you can create a plain text file called myexample.m and type inthe code as described in the following sections which outline the necessary commandsfor the various scenarios.

7.1 General MIPs Solved Using octave

Suppose we wish to solve the following three-variable MIP.

maxw,x,y

z = w − 2x+ y

s.t. 2w − x+ 2y ≤ 15

w + x− 3y ≤ 6

x− 2w = 5

y ≤ 5

w, x, y ≥ 0

w ∈ Rx, y ∈ Z

47

48 CHAPTER 7. SOFTWARE SOLVERS

The function call that octave will use looks like this:

[dv,ov]=glpk(c,A,b,lb,ub,ctype,vtype,sense);

This function takes up to eight inputs and returns two basic outputs. The inputsdefine the optimization problem and the outputs are the optimal decision variables(dv) and optimal objective value (ov), if they exist. The inputs assume that theproblem has been formulated in octave standard form. So, our first task is toreformulate the MIP. Here is the result (See also Definition 15):

maxw,x,y

z = w − 2x+ y

s.t. 2w − x+ 2y ≤ 15

w + x− 3y ≤ 6

− 2w + x = 5

0 ≤ w

0 ≤ x

0 ≤ y ≤ 5

w ∈ Rx, y ∈ Z

For reference, we specify the collection of decision variables as a vector v =[w x y

]T, which also establishes a variable order. With this designation, we can

write the objective fuction as z = cTv where c =[1 −2 1

]T. In octave we write

c=[1;-2;1];

Notice that a semicolon in a vector/matrix indicates a new row, so c in this exampleis a column vector. The next inputs are the A and b matrices that give the coefficientsof the constraints. In this example, we have

A =

2 −1 21 1 −3−2 1 0

, b =

1565

.In octave we write

A=[2 -1 2 ; 1 1 -3 ; -2 1 0];

b=[15;6;5];

Notice that a space between numbers in a matrix indicates moving to the next positionin the same row. An alternate method for constructing A is as follows.

A = [ 2 -1 2

1 1 -3

-2 1 0 ];

7.1. GENERAL MIPS SOLVED USING OCTAVE 49

In the above example a new line becomes a new row in the matrix. This form is oftenpreferable because the matrix form of A is clearly visible. The next task is to definethe lower and upper bounds on each decision variable. These values are encoded inthe vectors ` and u. In octave we write

lb=[0;0;0];

ub=[inf;inf;5];

Here, lb is the variable containing lower bounds, in this case all zeros. The variableub contains the upper bound values. The third variable, y, has upper bound 5. Thevariables w and x have no upper bound. In octave this is indicated by allowing a“value” of infinity, indicated inf.

Next, we must tell octave the sense of each constraint. The variable ctype isa string of letters, one for each constraint, that indicates its sense. There are fourallowed constraint types, each indicated by a capital letter: U for upper bound (≤); Lfor lower bound (≥); S for equality (=); and F for free (ignore). The example problemhas three constraints, the first two are upper bound constraints and the third is anequality, so we write

ctype=’UUS’;

Be sure and use the vertical single quotes when defining string or character variables.This type of quote is compatible with Matlab and is usually found with the doublequote on a standard keyboard.

Next, we must tell octave about our variable types – whether they are restrictedto be integer valued or not. The variable vtype is a string of letters, one for eachdecision variable, that indicates whether it is allowed to be real (C) or restricted tobe integer (I). In the example, the last two decision variables are restricted to beinteger. So we write

vtype=’CII’;

Finally, we tell octave whether we have a minimization or maximization problem.The variable sense is either +1 (minimization) or −1 (maximization). The exampleproblem is a maximization problem, so we write

sense=-1;

Each of the above lines of code should be in a script file along with the glpk functioncall itself. Your script file should have these lines:

c=[1;-2;1];

A=[2 -1 2 ; 1 1 -3 ; -2 1 0];

b=[15;6;5];

lb=[0;0;0];


ub=[inf;inf;5];

ctype=’UUS’;

vtype=’CII’;

sense=-1;

[dv,ov]=glpk(c,A,b,lb,ub,ctype,vtype,sense);

To run this file (and solve the optimization problem) using octave , take the followingsteps.

1. Verify that your file myexample.m contains the code lines shown above.

2. Place this text file in a working directory of your choice.

3. Start octave and use the file menus to navigate to your working directory. Youshould then see your script myexample.m in the file list.

4. At the command prompt type myexample to run your code.

5. To see the optimal decision variables, type dv at the command prompt.

6. To see the optimal objective value, type ov at the command prompt.

For the example problem, you should take the time to verify the optimal solutionv∗ =

[0 5 5

], z∗ = −5.

7.2 Solving MIPs Using Matlab

We can also use Matlab to solve the same example problem (see page 47). We mustfirst reformulate the linear program into Matlab standard form (see Definition 13).Similar to the reformulation of the octave example, we have

minw,x,y

z = cTx

s.t. Ax ≤ b

Aex = be

` ≤ x ≤ u

w ∈ Rx, y ∈ Z,

where

A =

[2 −1 21 1 −3

], b =

[156

], Ae =

[−2 1 0

], be = 5,

` =

000

, u =

——5

, c =

1−21

.

7.2. SOLVING MIPS USING MATLAB 51

In this formulation, note that the inequality constraints are all upper bound con-straints. The Matlab linear programming functions require this condition. Alsonote that variables w and x have no upper bounds. The Matlab code for solvingthis example problem is:

c=[1;-2;1];

A=[2 -1 2 ; 1 1 -3];

b=[15;6];

Ae=[-2 1 0];

be=5;

lb=[0;0;0];

ub=[inf;inf;5];

intvar=[2 3];

[dv,ov]=intlinprog(c,intvar,A,b,Ae,be,lb,ub);

The function intlinprog understands which variables are restricted to integer valuesthrough the input variable intvar which is a vector list of the indices of the decisionvariables which are integer valued. In this case, the second and third variables (x andy) are integer, so intvar contains the two indices 2 and 3.

Remark 10. Many interesting integer programs contain only binary variableswhere xk ∈ 0, 1 for all k. This binary condition is a special case of integerconstraints modeled as

0 ≤ xk ≤ 1, xk ∈ Z, k = 1, 2, . . . , n.

Thus, the methods of the previous examples in this chapter can be employed.

Remark 11. Functions in Matlab and octave have very specific syntax.For some optimization problems, there may not be all types of constraints. Forexample, if there are no equality constraints and you wish to use the Mat-lab function intlinprog, you must still specify values for Ae and be. In thiscase, set these variables as empty variables: Ae=[] and be=[].

Exercises

1. Solve the following optimization problems using either octave or Matlab .Convert the problems into mixed-integer problems as needed. In each case,provide a printout or copy of your code and results.


(a)

maxx

z = x1 + x2 + x3 + x4

s.t. x1 + 2x2 + x3 + 2x4 ≤ 77

2x1 + x2 + 2x3 + x4 ≤ 52

x1 + x2 + x3 + 3x4 ≤ 62

x ≥ 0

x1, x2, x3 ∈ Rx4 ∈ Z

(b)

minx

z = x1 + 2x2 + x3 − x4

s.t. x1 + 2x2 + x3 + 2x4 ≥ 40

2x1 + x2 + 2x3 + x4 ≥ 52

|x1 + x4| ≤ 20

x1 + x2 + x3 + x4 = 62

x ≥ 0

x ∈ Z4

(c)

maxx

z = (1.43)x1 + (2.10)x2

s.t. (2.09)x1 + (3.97)x2 ≤ 12

(4.32)x1 + (2.55)x2 ≤ 11

(0.97)x1 + (1.23)x2 ≤ 4

(1.10)x1 + (1.59)x2 ≤ 5

x ≥ 0

x ∈ R4

(d)

maxx

z = 8x1 + 5x2

s.t. x1 + x2 ≤ 6

9x1 + 5x2 ≤ 45

x ≥ 0

x ∈ R2

7.2. SOLVING MIPS USING MATLAB 53

(e)

maxx

z = 8x1 + 5x2

s.t. x1 + x2 ≤ 6

9x1 + 5x2 ≤ 45

x ≥ 0

x ∈ Z2

(f)

maxx

z = 60x1 + 30x2 + 20x3

s.t. 8x1 + 6x2 + x3 ≤ 48

4x1 + 2x2 + 32x3 ≤ 20

2x1 + 32x2 + 1

2x3 ≤ 8

x ≥ 0

x ∈ Z3

2. Compare the optimal solutions of problems (1d) and (1e). Make some observa-tions about the relationship between the optimal solutions of a linear programand the integer program with the same constraints.

3. Create a Matlab / octave function that displays the solution to an optimiza-tion problem in a user-readable format. The function inputs should be xstar

and zstar (if they exist) and the function should display z∗ and all x∗k 6= 0.

Chapter 8

Geometry of Linear Programs

Let’s consider the geometric interpretation of a linear program in the decision variablespace Rn. This will help us understand how to solve these optimization problems.Consider a 2-variable example with two inequality constraints and two non-negativesign constraints.

maxx

z = 2x1 + x2

s.t. x1 + 2x2 ≤ 6

5x1 + 4x2 ≤ 20 (8.1)

x ≥ 0

x ∈ R2

8.1 Geometry of the Feasible Region

Consider the following geometric definitions.

Definition 16. A halfspace is a set of the form H =x ∈ Rn | aTx ≤ b

for

some a ∈ Rn and b ∈ R.

Definition 17. A polygon is the intersection of a finite number of halfspaces.

The feasible region of the example problem,

Ω =x ∈ R2 | x1 + 2x2 ≤ 6, 5x1 + 4x2 ≤ 20, x ≥ 0

,

is a polygon in R2 because it is the intersection of four half spaces:

H1 = x ∈ Rn | x1 + 2x2 ≤ 6,

55

56 CHAPTER 8. GEOMETRY OF LINEAR PROGRAMS

H2 = x ∈ Rn | 5x2 + 4x2 ≤ 20,

H3 = x ∈ Rn | − x1 ≤ 0,

H4 = x ∈ Rn | − x2 ≤ 0.

The set of points that is in each of these half-spaces is Ω. The boundaries of Ω coincidewith the constraint boundaries. These boundary lines are given by the inequalityconstraints when the equality holds. They are

x1 + 2x2 = 6, 5x1 + 4x2 = 20, x1 = 0 and x2 = 0.

Figure 8.1 shows the geometric situation of the example LP. The feasible regionis the quadrilateral satisfying all four constraints. Some thought should convince youthat the feasible region must be convex, no matter how many or how few constraintsare in the problem. Consider the following two equivalent definitions.

Definition 18. A set Ω is convex if for every x, y ∈ Ω, ax+ (1− a)y ∈ Ω forall 0 < a < 1.

Definition 19. A set Ω ⊆ Rn is convex if for every x, y ∈ Ω, the line segmentconnecting x and y is wholely in Ω.

In R2, the boundary of any LP feasible region is, in general, a polygon containingsome edges and some vertices. In R3, the boundary of any LP feasible region is ingeneral, a polyhedron consisting of vertices, edges and faces (See Figure 8.2). Inhigher dimensions, the general term for polygon is polytope, and we may use eitherterm to indicate a LP feasible region in Rn.

Definition 20. Consider a general linear program with feasible region definedby inequality constraints

aTk x ≤ bk, k ∈ I

and equality constraintsaTk x = bk, k ∈ Ie,

where I and Ie are index sets for the inequality and equality constraints, re-spectively. We say that a constraint, identified by index k, is active or binding

at x if aTk x = bk. If a constraint is not active, we say that it is inactive ornot binding.

Consider the example from the begining of this section and illustrated in Fig-ure 8.1. We see that the constraints 5x1 + 4x2 ≤ 20 and x2 ≥ 0 are active at (4, 0)

8.2. GEOMETRY OF THE OBJECTIVE FUNCTION 57

1 2 3 4 5 6 7

1

2

3

4

5

6

z=

2z

=3

z=

4

Ωx1 + 2x2 = 6

5x1 + 4x2 = 20

x1

x2

Figure 8.1: Geometric detail of the example LP. The feasible region, Ω, is theintersection of four half-spaces and is shown in light blue shading. Three iso-objectivelines of the objective function are shown as red dashed lines. The optimal solution isx∗ = (4, 0), z∗ = 8.

because 5x1 +4x2 = 20 and x2 = 0 at the point (x1, x2) = (4, 0). Also, the constrinatsx1 + 2x2 ≤ 6 and x1 ≥ 0 are inactive at (4, 0).

Notice that there is no requirement that a point x be a feasible point when describ-ing the nature of constriants at that point. For example, the constraint x1 + 2x2 ≤ 6is binding at (5, 1) because x1 +2x2 = 6 at (x1, x2) = (5, 1), even though this locationis not a feasible point of the linear program.

8.2 Geometry of the Objective Function

The objective function z = f(x) = 2x1 + x2 describes a family of parallel lines – onefor each value of z. For example, if z = 2 then the set of solutions to the objectiveequation 2 = 2x1 +x2 is the line x2 = −2x1 + 2 with x2-intercept (0, 2) and slope −2.


Figure 8.2: (Left:) An example polyhedron in R3 This shape is the intersection of 12halfspaces. The boundary is characterized by vertices, edges and faces. (Right:) Asimilar shape which is not represented by the intersection of half spaces. This shapecannot be the feasible region of a linear program.

Each value of z defines a different line in R2. Lines of constant z value are known asiso-objective lines. Three representative iso-objective lines are shown in Figure 8.1(for z = 2, 3, 4).

Definition 21. A level curve of a function f : Rn → R is the solutionset of f(x) = z for some z ∈ R. We say that the solution set is thelevel curve of f with value z.

These lines are three example level curves of f with values 2, 3, 4. Level curvesfor functions f : R2 → R are typically displayed as contour plots. A level curve for afunction F : R3 → R is typically displayed as a surface plot.

8.3 Geometry in Standard Form

Next, consider the standard form linear program

maxx

z = x1 + x2 + 2x3

s.t. 3x1 + 2x2 + x3 = 6 (8.2)

x ≥ 0

x ∈ R3

8.3. GEOMETRY IN STANDARD FORM 59

The feasible region of this linear program is defined by 3 inequality constraints (halfspaces) and one equality constraint.

Definition 22. A hyperplane is a set of the form H =x ∈ Rn | aTx = b

for

some a ∈ Rn and b ∈ R.

Equalities constrain the feasible region to one or more hyperplanes. The feasibleregion for this linear program is shown in Figure 8.3. The feasible region is thetriangle shown in blue which resides in the positive orthant x ≥ 0 and also satisfiesthe equality constraint 3x1 + 2x2 + x3 = 6. The feasible region does not includeany part of the regions below or above the triangle. Suppose we add one additional

x11 2 3 4

x2

1

2

3

4

x3

1

2

3

4

5

6

7

(2, 0, 0)

(0, 3, 0)

(0, 0, 6)

(0, 2, 2)

(1, 0, 3)

Figure 8.3: The geometric representation of two standard form linear programfeasible regions described in the text. The blue triangle is the portion of the plane3x1 + 2x2 + x3 = 6 which intersects with the non-negative orthant. The violet linesegment is the portion of the blue triangle which intersectsthe plane 3x1 +x2−x3 = 0.

equality constraint to (8.2): 3x1 + x2 − x3 = 0. The new feasible region must now


lie on the intersection of the two hyperplanes and in the positive orthant. The newfeasible region is shown in violet in Figure 8.3. The feasible region is still a convexpolygon, in this case a line segment.

Exercises

1. Show that if Ω = ∅ then Ω is convex.

2. Prove that every half-space in Rn is convex.

3. Use graphing software to display several example level curves of the functiong(x, y) = 1

2(x− 2y+ 1)3− (x− y)3 + 5(x− y)2 + 3(2x− y)− 5 for −1 ≤ x, y ≤ 1.

Use your plot to approximately identify any stationary points.

4. Sketch example level curves of the function h(x, y) = 3x+ 2y.

5. Sketch example level curves of the function h(x, y) = ax + by where a, b ∈ Rare given.

6. Sketch several example level curves of the function h(x, y) =√ax2 + y2 where

a > 0.

7. For each of the following linear programs, draw a fully-labeled plot similar tothat of Figure 8.1.

(a)

maxx

z = x1 − x2

s.t. x1 + 2x2 ≤ 8

5x1 + 4x2 ≤ 20

− x1 + x2 ≤ 2

x ≥ 0

x ∈ R2

(b)

maxx

z = x1 − x2

s.t. x1 + 2x2 ≤ 8

− x1 + x2 = 2

x ≥ 0

x ∈ R2

8.3. GEOMETRY IN STANDARD FORM 61

(c)

maxx

z = 2x1 + x2

s.t. x1 + x2 ≤ 10

x1 ≥ x2

2x2 − 4 = x1

x ≥ 0

x ∈ R2

(d)

maxx

z = 2x1 + 3x2

s.t. a ≤ x1 ≤ b

c ≤ x2 ≤ d

x ∈ R2

8. Solve each of the linear programs of Exercise 7 and determine which constraintsare active and which are inactive at any optimal points.

Chapter 9

Geometric Solution Methods forLinear Programs

We can now begin to develop solution methods for linear programs. We will buildon our knowledge of constrained optimization on smooth functions of many variablesalong with the geometry of linear programs. Recall that the general procedure forfinding optimal solutions is to first identify feasible stationary points of the objectivefunction and then to check the boundary of the feasible region. We can use this sameprocedure for linear programs.

For a linear program, we have objective function f : Rn → R defined by f(x) =cTx. Stationary points of f are identified by ∇f(x) = 0.

f(x) = cTx =[c1 c2 · · · cn

]x1

x2...xn

= c1x1 + c2x2 + . . .+ cnxn.,

∇f(x) =

∂f/∂x1∂f/∂x2

...∂f/∂xn

=

c1

c2...cn

= c.

Stationary points of the objective occur only when ∇f(x) = c = 0. If indeed c = 0,then every feasible point of the problem is an optimal point. This situation doesnot arise in practical applications where the objective is a non-trivial function ofthe decision variables. So, in all linear programs which we may consider, the objec-tive function has no stationary points. Thus, all optimal points x∗ must lie on theboundary of the feasible region Ω.

We can take this observation further. Consider the example shown in Figure 8.1.The feasible region boundary is defined by four line segments:

63

64 CHAPTER 9. GEOMETRIC SOLUTION METHODS

1. x1 = 0, 0 ≤ x2 ≤ 3

2. x1 + 2x2 = 6, 0 ≤ x1 ≤ 8/3

3. 5x1 + 4x2 = 20, 8/3 ≤ x1 ≤ 4

4. x2 = 0, 0 ≤ x1 ≤ 4

Now, along each of these line segments, the objective function is linear or affine. Forexample, along x1 = 0, f(x) = 2x1 + x2 = x2 (linear), and along x1 + 2x2 = 6,f(x) = 2x1 + x2 = 2x1 + 3 − (1/2)x1 = (3/2)x1 + 3 (affine). In general, a linearobjective function with domain restricted to the solution set of a linear equation, isan affine function.

Since, along any edge of this feasible region, the objective is affine, there are nostationary points. Thus, any optimal points must lie at the boundaries of the linesegment – at the end points, or vertices. This type of arguement extends to higherdimensions and we arrive at the following conlcusion.

Remark 12. Consider a linear program minx∈Ω

f(x) = cTx with feasible region

Ω having at least one vertex. If an optimal point y exists, then there exists anoptimal point x∗ which is a vertex of Ω.

Notice that not all optimal points are vertices of the feasible region, but if anoptimal solution exists, then at least one must be a vertex. So, in order to solve alinear program, it would seem we need to check “only” the vertices of Ω. However,such an approach is problematic for several reasons.

Only the smallest problems – those with two or three decision variables andnot too many constraints – can be graphed so that the vertices of Ω are easilyidentified.

For larger problems – with n decision variables, and m inequality constraints –the number of potential vertices is

(mn

)= m!

n!(m−n)!, which can be prohibitively

large.

The feasible region may be unbounded, in which case we need an additionalcheck for unboundedness of the linear program.

The feasible region may be empty, and this may not be easily determined in avertex search.

9.1. VERTEX ENUMERATION METHOD 65

9.1 Vertex Enumeration Method

If the feasible region is known to be bounded and not empty, we have the followingresult.

Remark 13. If the linear program

minx∈Ω

f(x) = cTx

has a nonempty and bounded feasible region Ω, then there exists an optimalpoint x∗ which is a vertex of Ω.

This Remark suggests a method of solving a linear program for which the feasibleregion can be graphed. Here are the steps.

Vertex Enumeration

1. Graph the feasible region Ω

2. Determine the coordinates of each vertex of Ω.

3. Compute the objective value at each vertex of Ω.

4. Let z∗ be the best (max or min) objective value found at a vertex of Ω.

5. X = x ∈ Ω | f(x) = z∗ is the set of optimal decision variables x∗ ∈ X.

This method also works for any linear program for which you can determine allvertices of a bounded feasible region. Notice that the set of optimal decision variablevectors is not necessarily a single point. Consider two examples.

Example #1: Solve the following linear program by the method of vertex enumer-ation.

minx

z = f(x) = −x1 + x2

s.t. x1 − x2 ≥ 2

2x1 + x2 ≤ 8

x ≥ 0

x ∈ R2

1. The feasible region is shown in Figure 9.1. Notice that the feasible region is abounded quadrilateral with four vertices.


1 2 3 4 5

1

2

3

4

5

A

B

C

D

Ω

x1 − x2 = −2

2x1 + x2 = 8

x1

x2

Figure 9.1: Feasible region graph for Examples #1 and #2.

2. The coordinates of each vertex are the intersection points of pairs of constraints.The vertex labels and coordinates are shown in the first two columns of Ta-ble 9.1.

3. The objective value at each of the vertices is shown in the last column of Ta-ble 9.1.

4. The best objective value (for this minimization problem) is z∗ = −4, the mini-mum z-value in the table.

5. The optimal decision variable vector is the unique point x∗ = (4, 0), which isvertex (D).

Thus, the solution is: x∗ = (x∗1, x∗2) = (4, 0), z∗ = −4.

9.2. OPTIMAL SOLUTIONS AND LINEAR ALGEBRA 67

vertex coordinates (x1, x2) z = f(x1, x2)

(A) (0, 0) 0

(B) (0, 2) 2

(C) (2, 4) 2

(D) (4, 0) −4

Table 9.1: Vertex enumeration table for Example #1.

Example #2: Solve the following linear program by the method of vertex enumer-ation.

maxx

z = f(x) = −x1 + x2

s.t. x1 − x2 ≥ 2

2x1 + x2 ≤ 8

x ≥ 0

x ∈ R2

This example is almost the same optimization problem of Example #1. The onlychange is that it is a maximization problem. We can use the same vertex enumerationtable. The maximum objective value is z∗ = 2. However, this optimal value occursat two distinct vertices. Since the objective function is affine along the line segmentconnecting these two vertices, it must be true that z = 2 at every point along thisline segment. Thus, every point on this line segment is optimal. We can write theoptimal decision variable set as a convex combination of the two optimal vertices:

X =x ∈ R2 | x = α(0, 2) + (1− α)(2, 4), 0 ≤ α ≤ 1

.

In this case, there are an infinite number of solutions. In general, if p verticesv1, v2, . . . , vp all evaluate to the same optimal objective value, z∗ = f(v1) = f(v2) =. . . = f(vp), then the optimal vertex set is

X =

x ∈ Rn

∣∣∣∣∣ x =

p∑k=1

βkvk,

p∑k=1

βk = 1, β ≥ 0

.

9.2 Optimal Solutions and Linear Algebra

Consider again the example used in describing the Vertex Enumeration Method andillustrated in Figure 9.1. The solution to the minimization problem was the vertex(x1, x2) = (4, 0). Since any solution lies on the boundary of the feasible region, atleast one constraint will be active. In this case, there are two active constraints at the


optimal solution. Notice that the optimization problem with all inactive constraintsremoved :

minx

z = f(x) = −x1 + x2

s.t. 2x1 + x2 ≤ 8

x2 ≥ 0

x ∈ R2

yields the same optimal solution. Only the active constraints at x∗ make any differencein the problem. Furthermore, all of the active constraints reduce to equalities. Thisleads to the equivalent optimization problem

minx

z = f(x) = −x1 + x2

s.t. 2x1 + x2 = 8

x2 = 0

x ∈ R2.

However, this problem has a feasible region consisting of a single point, the solutionto

2x1 + x2 = 8

x2 = 0,

a system of two linear equalities in two unknowns. This optimization problem reducedto solving a system of linear equations. So why is linear programming not simplylinear algebra in disguise? The answer is that we cannot predict which constraintswill be active at the optimal solution. Many solution methods rely on checkingpotential sets of active constraints. The Vertex Enumeration Method is one suchsolution method. The Simplex Method is another.

Exercises

1. Consider the linear program

minx

z = cTx

s.t Ax ≤ b

x ∈ Rn,

where A is an m× n real matrix, and assume m ≥ n (more on this later).

9.2. OPTIMAL SOLUTIONS AND LINEAR ALGEBRA 69

(a) Explain why the number of potential vertices of the feasible region couldbe as large as (

m

n

)=

m!

n! (m− n)!.

(b) How many vertices must be checked for a problem with 77 decision vari-ables and 86 constraints?

(c) Discuss the fact that a problem of this size was solved in the 1940’s.

2. Show that the optimal decision variable set of Example #2 is the line segmentconnecting vertices (B) and (C).

3. Graph the feasible region of the following linear program. Solve using the vertexenumeration method.

maxx

z = f(x) = 2x1 + x2

s.t. x1 + x2 ≤ 4

x2 ≤ 3

x ≥ 0

x ∈ R2


maxx

z = f(x) = 5x1 + x2

s.t. − x1 + x2 ≤ 2

− x1 − 4x2 ≤ 2

3x1 + x2 ≤ 6

x ∈ R2


maxx

z = f(x) = 6x1 + 2x2

s.t. x1 + x2 ≤ 10

− x1 + x2 ≤ 4

3x1 + x2 ≤ 24

x ≥ 0

x ∈ R2

Chapter 10

Linear Program ModelingConcepts

Modeling in optimization is the process of capturing a meaningful approximation ofa real life situation into a mathematical representation. Modeling is both “easy” and“difficult.” The easy part is formulating mathematical expressions from verbal orwritten conditions and goals. The hard part is meaningfully capturing real life. Wewill focus mostly on the easier part. The harder part becomes easier with experienceand hard work.

This dichotomy between the easy and difficult is illustrated with a bit of humorby George Dantzig (who developed the Simplex Method to solve linear programs) inhis article “The Diet Problem,” Interfaces, 20: 4, July-August 1990. Dantzig tellsa true story of his own attempts to create a weight-loss diet in the early 1950’s forhimself. It was a simple matter to create a linear program that maximized “a feeling offullness” while meeting nutritional guidelines and implementing a calorie restriction.However, the solution(s) to the model were impractical meals which included thingslike 500 gallons of vinegar, 200 boullion cubes or 2 pounds of bran. The problem wasnot Dantzig’s ability to model nutritional constraints, nor the Simplex Algorithm’sability to return an optimal solution. The problem was that the constraints (andpossibly the objective) did not model Dantzig’s diet needs with sufficient accuracyand completeness.

10.1 Methodology

The following is a suggested method for modeling optimization problems. This shouldnot be taken as a recipe. Anyone should feel free to model problems in any way thatmakes sense and results in a well-formulated and notationally correct optimizationproblem.

1. Decision Variables. Identify those problem quantities over which the deicision-

71

72 CHAPTER 10. MODELING CONCEPTS

maker has some control, or ability to choose. All other problem quantities willbe written in terms of these decision variables. You can always add more dec-sions variables as you continue to model a problem, but identifying key variablesat the beginning is important.

2. Objective Function. Construct your goal statement in terms of your decisionvariables. What quantity are you trying to maximize or minimize? Sometimesthe objective is not easily written as a linear function, but you should attemptto do so. We will see many examples of nonlinear objective functions which wecan linearize when a model is complete.

3. Constraints. List out any restrictions imposed on your decision variables.Also express these quantities as linear functions whenever possible. Sometimesin constructing constraints it may be clarifying to add new decision variables.We will see examples of this as well.

4. Sign and Integer Restrictions. Don’t forget that many decision variablesare naturally restricted in sign (non-negativity is the most common). Theseconstraints must be included. Remember that any solution algorithm can onlyuse the information you provide.

5. LP in a Standard Form. Write the optimziation problem as a linear programin a standard form. Whether your model is a linear program or not, you nextmust reformulate it into a standard form LP. This step may be quite involvedor relatively simple depending on your model. However, your goal is a modelwhich can be solved by the software of your choice.

10.2 A First Modeling Example

Consider the following example problem.

Alex and Sam need to write a collaborative paper for their English class.Alex writes at 3.5 pages per hour and makes 3 grammatical errors per page.Sam writes at 2.5 pages per hour and mkaes 1 grammatical error per page.They have discovered that the grade given by the professor is proportionalto the number of written pages in the assignment. However, for every 5grammatical errors the professor deducts the equivalent of 1 written page.The maximum number of grammatical mistakes the professor will tolerateis 50. The total of 8 person-hours is allowed for completing the assign-ment. Previously scheduled engagements limit Sam’s time participationto 5.5 hours. Formulate a linear program which can be used to maximizetheir grade.

Let’s use the suggested modeling methodology to complete the assigned task.

10.2. A FIRST MODELING EXAMPLE 73

1. Decision Variables. Alex and Sam need to decide, for example, how manyhours each needs to spend writing. They cannot change how the professorgrades or how many total person-hours are available. So, we can define

x1 = (number of hours that Alex writes)

x2 = (number of hours that Sam writes)

Notice that each decision variable is a well-defined numerical quantity. Somepoor definition examples: “x1 = Alex”, “x1 = Alex writes”, “x1 = time for Alex”.

We could have used a different set of decision variables. For example, Alex andSam could be thinking in terms of how many pages each could write and use x1

(or x2) as the number of pages Alex (or Sam) will write. This choice is equallyvalid, but we will continue with the choice listed above.

2. Objective Function. The goal is to maximize the grade of the paper. So,we must construct a function f : R2 → R for which the inputs are the decisionvariable values (the number of hours each spends writing) and outputs the“grade.”

z = f(x) = (grade value assigned to the paper)

= (effective number of pages written)

= (number of written pages)− (page reduction imposed by professor)

= (pages Alex writes)− (page reduction due to Alex’s errors)

+ (pages Sam writes)− (page reduction due to Sam’s errors)

=3.5 pages

hourx1 −

1 page

5 errors· 3 errors

page· 3.5 pages

hour· x1

+2.5 pages

hourx2 −

1 page

5 errors· 1 errors

page· 2.5 pages

hour· x2

= 1.4 x1 + 2 x2.

This objective does not actually assign a letter grade, but it does return theeffective length of the paper for any choice of hours which Alex and Sam decideto write.

3. Constraints. There are three restrictions on the decision variables that mustbe met.

(a) There is a total limit on the number of grammatical errors which the


professor will tolerate.

50 ≥ (total number of errors)

= (number of errors in Alex’s writing)

+ (number of errors in Sam’s writing)

=3 errors

page· 3.5 pages

hour· x1 +

1 errors

page· 2.5 pages

hour· x2

= 10.5 x1 + 2.5 x2.

or equivalently,10.5 x1 + 2.5 x2 ≤ 50.

(b) There is a total limit on the number of person-hours that Alex and Samcan use to write. That is, if they add the times they each work it cannotexceed the limit of eight.

8 ≥ (total person-hours spend writing)

= (hours Alex spends writing) + (hours Sam spends writing)

= x1 + x2.

(c) Sam has as a maximum number of hours which he can spend on this writingassignment. This constraint is the most straightforward.

x2 ≤ 5.5

4. Sign Restrictions. Neither Alex nor Sam can write a negative amount oftime:

x ≥ 0.

While these sign constraints may seem unnecessary, such constraints can becritically important to include. It may be possible that if Alex were to write anegative amount of time, this would effectively increase the error budget andeffective number of pages. These types of unrealistic situations can occur inoptimization problems if the modeller is not careful.

5. Standard Form. Finally, we collect all of our model pieces into one place asan optimization problem.

maxx

z = f(x) = 1.4 x1 + 2x2

s.t. 21x1 + 5x2 ≤ 100

x1 + x2 ≤ 8

x2 ≤ 5.5

x ≥ 0

x ∈ R2

10.2. A FIRST MODELING EXAMPLE 75

In the above representation, the first constraint was multiplied by 2 simply todisplay fewer decimals and/or fractions. This model is already a linear programin inequality standard form. We can also write the model in (matrix) inequalitystandard form in order to expedite our use of software. We have

maxx

z = cTx

s.t. Ax ≤ b

x ≥ 0

x ∈ R2

c =

1.4

2

, A =

21 5

1 1

0 1

, b =

100

8

5.5

Exercises

1. For the example of Alex’s and Sam’s English paper, describe why the decisionvariable definitions “x1 = Alex”, “x1 = Alex writes”, “x1 = time for Alex” arenot acceptable.

2. Solve Alex’s and Sam’s paper writing problem using the method of Vertex Enu-meration. How many hours should each write? How many total grammaticalerrors will the paper have? What will be the effective length of the paper?

Chapter 11

Linear Progam Modeling Examples

This chapter presents examples of a variety of mixed-integer programming techniques.Each example introduces one or more new concepts and/or strategies.

11.1 A-1 Furniture

This first modeling example presents a basic profit maximization blending problemwith resource constraints.

The A-1 Furniture factory is tooled to produce four types of furniture pieces:chairs, tables, desks, and dressers. The warehouse holds various quantities of thefive necessary raw materials: oak wood, pine wood, glass, wrought iron, and leather.The table below shows the raw material quantities required to produce each type offurniture piece, the raw material quantities in the warehouse, and the expected profitper manufactured item. Formulate an integer program whose solution is a productionplan for maximizing total expected profit.

chair table desk dresser warehouse stock

oak wood (bdft) 4 8 6 6 2132

pine wood (bdft) 1 0 3 4 906

glass (ft2) 0 0 1 2 330

iron (kg) 1 4 3 3 945

leather (ft2) 2 1 1 0 441

profit ($) 21 33 40 50

77

78 CHAPTER 11. MODELING EXAMPLES

Solution. We let x ∈ Z4 be the number of of each type of furniture piece to man-ufacture. In particular,

x1 = number of chairs to manufacture,

x2 = number of tables to manufacture,

x3 = number of desks to manufacture,

x4 = number of dressers to manufacture.

The expected total profit, z, is the sum of the expected profits for each type offurniture.

z = 21x1 + 33x2 + 40x3 + 50x4.

There are constraints on the total amount of each resource we have available. Forexample, the total oak wood used in production cannot exceed the warehouse total.

4x1 + 8x2 + 6x3 + 6x4 ≤ 2132.

Similarly for the other four resources:

x1 + 3x3 + 4x4 ≤ 906 (pine),

x3 + 2x4 ≤ 330 (glass),

x1 + 4x2 + 3x3 + 3x4 ≤ 945 (iron),

2x1 + x2 + x3 ≤ 441 (leather).

We also have the condition that we cannot manufacture negative quantities of anypiece of furniture.

xk ≥ 0, k = 1, 2, 3, 4.

Thus, the IP:

11.2. GREENSIDE YARD & LAWN CARE 79

max z = 21x1 +33x2 +40x3 +50x4

s.t. 4x1 +8x2 +6x3 +6x4 ≤ 2132

x1 +3x3 +4x4 ≤ 906

x3 +2x4 ≤ 330

x1 +4x2 +3x3 +3x4 ≤ 945

2x1 +x2 +x3 ≤ 441

0 ≤ x

x ∈ Z4

The optimal solution is

x∗ =[

164 30 82 124]T,

z∗ = 13, 914.

Thus, the optimal strategy for A-1 Furniture is to manufacture 164 chairs, 30tables, 82 desks, and 124 dressers. This will lead to a total expected profit of $13,914.This solution utilizes all available lumber and glass, but leaves 43 kg of iron and onesquare foot of leather unused.

11.2 Greenside Yard & Lawn Care

This modeling example is a cost minimization problem under fixed requirements. Boxconsraints are used in the formulation.

Greenside Yard & Lawn Care has received a government grant to create temporaryjobs through civic improvement projects in their home town. The grant outlines fourtypes of projects: lawn refurbishing, tree removal, community garden establishment,and city park cleanup. Greenside must decide how many of each project to complete.The cost to Greenside for each type of job (in the order listed above) is $60, $200,$750, and $250. For each job completed 1/5, 3/4, 2, and 1 temporary jobs are created,respectively. The grant stipulates that at least 25 temporary jobs must be createdand at least 50 projects must be completed. It also requires that at least 20 lawnrefurbishing jobs, one community garden, and three park cleanup jobs be completed.A survey of the town shows that at most 40 lawn refurbishing jobs are available,at most 4 community gardens are feasible, and there are 6 city parks. Formulatean integer program whose solution is a job selection strategy that minimizes cost toGreenside.


Solution. We let x ∈ Z4 be the number of each type of job to complete. In partic-ular,

x1 = number of lawns to refurbish,

x2 = number of trees to remove,

x3 = number of community gardens to develop,

x4 = number of parks to cleanup.

The expected total cost, z, is the sum of the expected costs for each type of job.

z = 60x1 + 200x2 + 750x3 + 250x4.

At least 50 projects must be completed.

x1 + x2 + x3 + x4 ≥ 50.

At least 25 temporary jobs must be created. Each lawn refurbishing job createsone-fifth of a temporary job, so the number of temporary jobs created by all lawnjobs is x1/5. Using this reasoning we have the constraint:

1

5x1 +

3

4x2 + 2x3 + x4 ≥ 25.

Grant and city constraints require the additional restrictions: 20 ≤ x1 ≤ 40,0 ≤ x2, 1 ≤ x3 ≤ 4, and 3 ≤ x4 ≤ 6. Thus, the IP

min z = 60x1 +200x2 +750x3 +250x4

s.t. x1 +x2 +x3 +x4 ≥ 50

15x1 +3

4x2 +2x3 +x4 ≥ 25

20 ≤ x1 ≤ 40

0 ≤ x2

1 ≤ x3 ≤ 4

3 ≤ x4 ≤ 6

x ∈ Z4


x∗ =[

29 15 1 6]T,

z∗ = 6990.

11.3. TOURIST TRAP SHOP 81

The optimal strategy for Greenside is to refurbish 29 lawns, remove 15 trees, createone community garden, and clean up all 6 city parks. The total cost to Greensideis $6990. Hopefully, the government grant covers at least this cost. This strategycompletes 51 projects and creates 25.05 temporary jobs.

11.3 Tourist Trap Shop

This example is a profit maximization problem with both revenue and costs. It illus-trates consumer-demand and business-practice requirements.

The Tourist Trap Shop survives on the tourist trade selling a variety of trinkets.Among other items, they sell T-shirts, keychains, refrigerator magnets, and hats.They can obtain these items from four different wholesalers which supply these itemsin fixed quantities. The table below shows the quantity of each item found in eachsupplier’s standard shipment. It also shows the wholesale cost of each shipment.

Supplier Batch Cost T-shirts Keychains Magnets Hats

($) (#) (#) (#) (#)

A 172 60 26 32 26

B 60 24 0 0 12

C 96 0 20 20 14

D 48 10 0 10 10

In considering the tourist business forecast, the manager of Tourist Trap hasdecided to purchase at least 1000 of these items (in some combination) for the season.The total available purchasing funds are $2950. Keychains do not sell very well, sothe total number of keychains should be limited to a maximum of 15% of the totalnumber of items. T-shirts are always popular, so the total T-shirt count should beat least 40% of the total number of items. The total expected revenue per item forT-shirts, keychains, magnets and hats is $7, $2, $3, and $5, respectively. Formulatean integer program whose solution determines the best strategy for buying batchesof items from the four suppliers with the goal of maximizing profit

Solution. We let x ∈ Z4 be the number of batches to purchase from each supplier.In particular, xi is the number of batches to purchase from supplier i, where i =1, 2, 3, 4.

The expected profit is the total expected revenue less the cost of purchasing items.The expected revenue depends upon how many of each item is purchased as part of


the various batches. For T-shirts, we have

# of T-shirts = 60x1 + 24x2 + 10x4

and the total revenue for T-shirts is

T-shirt revenue = 7 (60x1 + 24x2 + 10x4) .

Similarly, we have

Keychain revenue = 2 (26x1 + 20x3) ,

Magnet revenue = 3 (32x1 + 20x3 + 10x4) ,

Hat revenue = 5 (26x1 + 12x2 + 14x3 + 10x4) .

The total profit, z, is the sum of these individual profits less the cost of batches:

z = 698x1 + 228x2 + 170x3 + 150x4 − 172x1 − 60x2 − 96x3 − 48x1,

z = 526x1 + 168x2 + 74x3 + 102x4.

Alternatively, one could compute the expected profit per batch from each supplier,arriving at the same profit function.

Next, consider the various constraints. First, have purchasing funds limited to$2950. This leads to the constraint:

172x1 + 60x2 + 96x3 + 48x4 ≤ 2950.

Next, we must purchase at least 1000 total items. Each batch contains differingnumbers of items. We have

144x1 + 36x2 + 54x3 + 30x4 ≥ 1000.

The number of keychains should be limited to a maximum of 15% of the totalnumber of items. We have

26x1 + 20x3 ≤ (0.15)(144x1 + 36x2 + 54x3 + 30x4).

Or, equivalently,

4.4x1 − 5.4x2 + 11.9x3 − 4.5x4 ≤ 0.

Notice, in particular that the total number of purchases items is not necessarily equalto 1000. Similarly, we have that the total number of T-shirts should be at least 40%of the total number of items. This leads to the constraint:

2.4x1 + 9.6x2 − 21.6x3 − 2x4 ≥ 0.

11.4. WOOF&MEOW PET FOOD COMPANY 83

Thus, the IP:

max z = 526x1 +168x2 +74x3 +102x4

s.t. 172x1 +60x2 +96x3 +48x4 ≤ 2950

144x1 +36x2 +54x3 +30x4 ≥ 1000

4.4x1 −5.4x2 +11.9x3 −4.5x4 ≤ 0

2.4x1 +9.6x2 −21.6x3 −2x4 ≥ 0

0 ≤ x

x ∈ Z4


x∗ =[

13 11 0 1]T,

z∗ = 8788.

This solutions shows that Tourist Trap should purchase 13 batches from supplier1, 11 batches from supplier 2 and one batch from supplier 4. The total expectedprofit is $8788. With this solution, 2298 items are purchased, including 1054 T-shirts(45.9%) and 338 keychains (14.7%).

11.4 Woof&Meow Pet Food Company

This example illustrates the use of dependent variables to simplify problem presenta-tion – at the expense of having more decision variables.

Woof&Meow has decided to use year-end overstock to produce specialty test-market canned food for cats and dogs. The food will be composed of five ingredients:chicken, beef, wheat, rice, and water. The following table shows for each item, thequantity in stock, the fraction (by weight) of protein and fat, and the calorie content.

Ingredient stock protein fat calories

(g) (wt frac) (wt frac) (per g)

chicken 31750 0.27 0.14 2.4

beef 21780 0.26 0.15 2.5

wheat 15600 0.14 0.024 3.4

rice 21000 0.027 0.0025 1.3

water unlimited 0 0 0


Any food produced must meet certain nutritional requirements. Acceptable rangesare shown in the table below.

protein range fat range calorie range

(wt frac) (wt frac) (per g)

cat food 0.07–0.12 0.04–0.07 1.00–1.25

dog food 0.09–0.13 0.03–0.08 0.75–1.15

In addition, consumer appreciation and marketability requirement suggest thatany food produced must contain at least 30% meat by weight. Formulate a linearprogram whose solution provides a recipe for a batch of dog food and a batch of catfood that maximizes the total weight of overstock used in the process.

Solution. We must determine how much of each ingredient is used in each type offood. Let

xij = the grams of ingredient i used in food j,

where i = 1, 2, 3, 4, 5 corresponds to chicken, beef, wheat, rice and water, respectively;and j = 1, 2 corresponds to cat food and dog food, respectively. We can considerxij ∈ R. Because there are many constraints on various relative quantities, we willalso use the following dependent variables.

Wj = the total weight of food j.

Wpj = the total protein weight in food j.

Wfj = the total fat weight in food j.

Cj = the total calories in food j.

All together there are 18 decision variables.

Remark 14. Whenever dependent variables are created for an optimizationproblem, the dependencies must be included in the constraint set.

The objective is to maximize the use of overstock ingredients (by weight). Thatis, we seek to maximize the objective function

z = x11 + x21 + x31 + x41 + x12 + x22 + x32 + x42.

We begin enumerating the problem constraints by defining our dependent variablesin terms of our initial decision variables.

W1 = x11 + x21 + x31 + x41 + x51.

11.4. WOOF&MEOW PET FOOD COMPANY 85

W2 = x12 + x22 + x32 + x42 + x52.

Wp1 = 0.27x11 + 0.26x21 + 0.14x31 + 0.027x41.

Wf1 = 0.14x11 + 0.15x21 + 0.024x31 + 0.0025x41.

Wp2 = 0.27x12 + 0.26x22 + 0.14x32 + 0.027x42.

Wf2 = 0.14x12 + 0.15x22 + 0.024x32 + 0.0025x42.

C1 = 2.4x11 + 2.5x21 + 3.4x31 + 1.3x41.

C2 = 2.4x12 + 2.5x22 + 3.4x32 + 1.3x42.

Now, our inequality constraints are relatively simply expressed in terms of ouraugmented set of decision variables. For example, the requirement that cat food beat least 30% meat by weight is expressed as

x11 + x21

W1

≥ 0.3 or equivalently − x11 − x21 + 0.3W1 ≤ 0.

Notice that the above equivalence assumes that W1 > 0. For this particular problemwe can be reasonably assured that this will be the case at the optimal solution. Thatis, we expect W ∗

1 > 0. However, the denominator of any such constraint should bechecked once the problem is solved to make sure that the equivalency is met. Thecorresponding constraint for dog food is

x12 + x22

W2

≥ 0.3 or equivalently − x12 − x22 + 0.3W2 ≤ 0.

There are constraints on the weight fraction of protiens. For the cat food we have

0.07 ≤ Wp1

W1

≤ 0.12

which is equivalent to the two constraints

0.07W1 −Wp1 ≤ 0 and − 0.12W1 +Wp1 ≤ 0.

Remark 15. Suppose x and y are decision variables and b is a constant. Aconstraint such as x/y ≥ b is equivalent to x ≥ by only if we can be sure thaty > 0 for any feasible y.

For dog food we have the corresponding constraints

0.09W2 −Wp2 ≤ 0 and − 0.13W2 +Wp2 ≤ 0.


We also have similar constraints on the weight fraction of fats:

0.04W1 −Wf1 ≤ 0 and − 0.07W1 +Wf1 ≤ 0.

0.03W2 −Wf2 ≤ 0 and − 0.08W2 +Wf2 ≤ 0.

Next, there are restrictions on the calorie density for each food. Using the same typeof construction, we have

W1 − C1 ≤ 0 and − 1.25W1 + C1 ≤ 0,

0.75W2 − C2 ≤ 0 and − 1.15W2 + C2 ≤ 0.

Finally, we cannot use more ingredients than are available:

x11 + x12 ≤ 31750,

x21 + x22 ≤ 21780,

x31 + x32 ≤ 15600,

x41 + x42 ≤ 21000.

Thus, the LP:

maxx z = cTx

s.t. Ae = be

A ≤ b

x ≥ 0

x ∈ R18

where

xT =[x11 x21 x31 x41 x51 x12 x22 x32 x42 x52 W1 W2 Wp1 Wf1 Wp2 Wf2 C1 C2

]cT =

[1 1 1 1 0 1 1 1 1 0 0 0 0 0 0 0 0 0

]

Ae =

1 1 1 1 1 0 0 0 0 0 −1 0 0 0 0 0 0 0

0 0 0 0 0 1 1 1 1 1 1 −1 0 0 0 0 0 0

0.27 0.26 0.14 0.027 0 0 0 0 0 0 0 0 −1 0 0 0 0 0

0.14 0.15 0.024 0.0025 0 0 0 0 0 0 0 0 0 −1 0 0 0 0

0 0 0 0 0 0.27 0.26 0.14 0.027 0 0 0 0 0 −1 0 0 0

0 0 0 0 0 0.14 0.15 0.024 0.0025 0 0 0 0 0 0 −1 0 0

2.4 2.5 3.4 1.3 0 0 0 0 0 0 0 0 0 0 0 0 −1 0

0 0 0 0 0 2.4 2.5 3.4 1.3 0 0 0 0 0 0 0 0 −1

11.5. PACK&SHIP AIR DELIVERY 87

bTe =[

0 0 0 0 0 0 0 0]

A =

−1 −1 0 0 0 0 0 0 0 0 0.3 0 0 0 0 0 0 0

0 0 0 0 0 −1 −1 0 0 0 0 0.3 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0.07 0 −1 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 −0.12 0 1 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0.09 0 0 −1 0 0 0

0 0 0 0 0 0 0 0 0 0 0 −0.13 0 0 1 0 0 0

0 0 0 0 0 0 0 0 0 0 0.04 0 0 −1 0 0 0 0

0 0 0 0 0 0 0 0 0 0 −0.07 0 0 1 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0.03 0 0 0 −1 0 0

0 0 0 0 0 0 0 0 0 0 0 −0.08 0 0 0 1 0 0

0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 −1 0

0 0 0 0 0 0 0 0 0 0 −1.25 0 0 0 0 0 1 0

0 0 0 0 0 0 0 0 0 0 0 0.75 0 0 0 0 0 −1

0 0 0 0 0 0 0 0 0 0 0 −1.15 0 0 0 0 0 1

1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0

0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0

0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0

bT =

[0 0 0 0 0 0 0 0 0 0 0 0 0 0 31750 21780 15600 21000

]

11.5 Pack&Ship Air Delivery

This problem is an exmple of a packing problem. The formulation is presented in ageneral form with the use of parameter variables for quantities that are not decisionvariables. It also illustrates the use of summation notation to simplify the modelpresentation.

A transport plane has three compartments for storing cargo: front (j = 1), center(j = 2), rear (j = 3). Each compartment has limits on weight capacity, (wj in tons)and space capacity (sj in m3). Any proportion of any of four cargo types (i = 1, 2, 3, 4)can be placed in these compartments for shipment. Each cargo type has total weightti (tons), volume density vi (m3/ton) and profit pi ($/ton). In order to balancethe plane, the ratio of cargo weight to compartment capacity must be equal for allthree compartments. How should the cargos be distributed on the plane in order tomaximize profit?


Solution. We can choose how much of each cargo type is packed into each com-partment. Let

xij = tons of cargo i placed in compartment j,

where xi,j ∈ R. The objective is to maximize profit. We can build the objectivefuntion as follows.

z = (profit in $)

=4∑i=1

(profit in $ from cargo type i)

=4∑i=1

(profit per ton of cargo i) (tons of cargo i shipped)

=4∑i=1

pi(xi1 + xi2 + xi3)

=4∑i=1

pi

3∑j=1

xij

=4∑i=1

3∑j=1

pixij

Remark 16. The use of summation notation is usually prefered in composingthe objective function and any constraints. This method saves space and canexpress many large constraints efficiently.

There are several constraints. First, each compartment has a weight limit.

4∑i=1

xij ≤ wj, j = 1, 2, 3.

And, each compartment has a volume limit. This set of constraints uses the fact thateach cargo type is described by a volume density, so vixij is the volume of cargo typei packed in compartment j.

4∑i=1

vixij ≤ sj, j = 1, 2, 3.

There is a limited quantity (by weight) of each type of cargo.

3∑j=1

xij ≤ ti, i = 1, 2, 3, 4.

11.5. PACK&SHIP AIR DELIVERY 89

The plane must be properly balanced in that the packed weight to available volumeratio must be equal for each compartment.∑4

i=1 xi1s1

=

∑4i=1 xi2s2

=

∑4i=1 xi3s3

.

This set of equality constraints is equivalent to the two constraints:

s2

4∑i=1

xi1 − s1

4∑i=1

xi2 = 0,

s3

4∑i=1

xi2 − s2

4∑i=1

xi3 = 0,

Remark 17. Notice that a constraint set g1(x) = g2(x) = · · · = gm(x) isequivalent to the m−1 constraints g1(x) = g2(x), g2(x) = g3(x), . . . gm−1(x) =gm(x). All other mixed constraint pairs gk(x) = gj(x) are then automaticallysatisfied.

Finally, we must specify that we cannot pack negative quantities of any cargo.

x ≥ 0.

We next collect our work into a linear program with 12 decision variables, 12 con-straints and 12 sign restrictions. The fact that there are 12 of each is entirely coinci-dental. We have

maxx z = cTx

s.t. Aex = be

Ax ≤ b

x ≥ 0

x ∈ R12,

where we specify the variable order

xT =[x11 x12 x13 x21 x22 x23 x31 x32 x33 x41 x42 x43

],

and the other matrices:

cT =[p1 p1 p1 p2 p2 p2 p3 p3 p3 p4 p4 p4

]


Ae =

s2 −s1 0 s2 −s1 0 s2 −s1 0 s2 −s1 0

0 s3 −s2 0 s3 −s2 0 s3 −s2 0 s3 −s2

bTe =

[0 0

]

A =

1 0 0 1 0 0 1 0 0 1 0 0

0 1 0 0 1 0 0 1 0 0 1 0

0 0 1 0 0 1 0 0 1 0 0 1

v1 0 0 v2 0 0 v3 0 0 v4 0 0

0 v1 0 0 v2 0 0 v3 0 0 v4 0

0 0 v1 0 0 v2 0 0 v3 0 0 v4

1 1 1 0 0 0 0 0 0 0 0 0

0 0 0 1 1 1 0 0 0 0 0 0

0 0 0 0 0 0 1 1 1 0 0 0

0 0 0 0 0 0 0 0 0 1 1 1

bT =

[w1 w2 w3 s1 s2 s3 t1 t2 t3 t4

]11.6 Shoes All Year

This modeling problem is an example of multi-period production planning. We alsoillustrate one method for linearizing objective functions that contain absolute valueterms.

A shoe company needs to set up a monthly production schedule for the year thatmeets predicted monthly demand dk, k = 1, 2, . . . , 12. Surplus shoes at the end of anymonth must be stored at a cost of t dollars per pair. Changing production qunatityfrom one month to the next incurs a positive factory retooling cost of r dollars perpair (increase or decrease). Suppose that the company begins the year with no storedshoes and wishes to end the year with no surplus shoes. Construct an integer programthat can be used to minimize production and storage costs while meeting monthlydemand.

Solution. The company might choose to make all required shoes∑12

k=1 dk in thefirst month, but this would result in high storage costs as shoes are held for futuremonths in the year. The company might also choose to produce just enough shoeseach month to meet the demand for that month. However, this plan might result in

11.6. SHOES ALL YEAR 91

large retooling costs as production is changed month to month. The optimal solutionwill attempt to balance these costs.

Let

xk = (shoe production volume in month k = 1, 2, . . . , 12),

sk = (surplus volume at the end of month k = 1, 2, . . . , 11).

Notice that the surplus variables are dependent variables and need to be expressedin terms of the monthly production volumes and demands. In general, the surplusat the end of any month is equal to the surplus from the previous month plus theproduction in that month minus the demand for that month:

sk = sk−1 + xk − dk.

The first and last months are somewhat different because of the surplus conditionsstated in the problem. We have

s1 = 0 + x1 − d1

s2 = s1 + x2 − d2

s3 = s2 + x3 − d3

...

s11 = s10 + x11 − d11

0 = s11 + x12 − d12

The objective function measures the retooling and storage costs:

z = r |x1|+ r12∑k=2

|xk − xk−1|+ t11∑k=1

sk.

This objective is nonlinear, and we will address this issue shortly. We must alsoinclude constraints that make sure that we meet monthly demand. This demand canbe met by previous month storage and current monthly production:

x1 ≥ d1,

xk + sk−1 ≥ dk, k = 2, 3, . . . , 12.

The only additional constraints we have are x ≥ 0 and s ≥ 0. There are no statedconstraints on production volumes or resource limitations, though these type of con-


straints are typically present. The optimization problem is

minx,s

z = r |x1|+ r

12∑k=2

|xk − xk−1|+ t11∑k=1

sk

s.t. s1 = 0 + x1 − d1

sk = sk−1 + xk − dk, k = 2, 3, . . . , 11

0 = s11 + x12 − d12

x1 ≥ d1

xk + sk−1 ≥ dk, k = 2, 3, . . . , 12

x, s ≥ 0

x ∈ Z12, s ∈ Z11.

The absolute value terms in the objective function are nonlinear. Fortunately,we can reformulate this optimization problem into an equivalent integer program.Consider the first term r |x1|. Because x1 is also constrained to be non-negative,at any feasible point r |x1| = rx1. However, all other retooling costs terms do notnecesarily satisfy xk − xk−1 ≥ 0. For these 11 terms, we introduce correspondingexcess variables yk and deficit variables wk defined by

yk =

xk − xk−1 if xk ≥ xk−1

0 if xk < xk−1

wk =

0 if xk ≥ xk−1

xk−1 − xk if xk < xk−1

With these new variables, notice the following two relations.

yk − wk =


xk − xk−1 if xk < xk−1

= xk − xk−1,

yk + wk =


xk−1 − xk if xk < xk−1

= |xk − xk−1| .

We can use the second set of relations to create a linear objective function. Thefirst set of relations becomes a new set of contraints that describe the relationshipbetween the original (independent) variables and the new (dependent) variables. The

11.7. THERMO – (LINEAR DATA FITTING) 93

equivalent integer program for this problem is

minx,s,y,w

z = rx1 + r12∑k=2

(yk + wk) + t11∑k=1

sk

s.t. s1 = 0 + x1 − d1

sk = sk−1 + xk − dk, k = 2, 3, . . . , 11

0 = s11 + x12 − d12

x1 ≥ d1

xk + sk−1 ≥ dk, k = 2, 3, . . . , 12

yk − wk = xk − xk−1, k = 2, 3, . . . , 11

x, s, y, w ≥ 0

x ∈ Z12; s, y, w ∈ Z11.

Remark 18. The following two minimization problems are equivalent.

minx

z = |x|

s.t. Ax ≤ b

x ∈ R

minx,y,w

z = y + w

s.t. y − w = x

Ax ≤ b

y, w ≥ 0

x, y, w ∈ R

11.7 Thermo – (Linear Data Fitting)

This example demonstrates how linear programming (and least squares methods) canbe used to find an affine function that minimize error between dependent and inde-pendent data values. We also illustrate a second method for linearizing absolute valueterms in the objective function. This example is also accompanied by octave code.

Thermo manufactures rapid-repsonse partial immersion thermometers for use inlaboratory settings. For each thermometer, the company produces a set of calibrationtemperature measurements and supplies these to the customer. They would also liketo provide the best linear fit to the data so that the customer has the option to


k 1 2 3 4 5 6 7 8 9

measured temp, xk (C) 10.4 21.0 29.8 39.7 50.4 60.0 71.2 87.2 90.3

actual temp, yk (C) 9.5 20.1 29.5 38.9 50.0 59.2 71.0 81.8 89.9

Table 11.1: Calibration data for a single thermometer manufactured by Thermo.

automate the use of the thermometer in a more dynamic setting that does not requireconstant user interpretation. Consider the set of calibration data shown in Table 11.1.

Thermo would like to produce a best linear fit to this data y = ax + b so that ameasured temperature x is quickly converted to the a temperature y which is closeto the actual temperature. That is, given the calibration data, what values of a andb provide the best linear fit y = ax+ b?

Before we answer this question, it is interesting to look at the data set. The eighthmeasured temperature, x8, appears to possibly be anomalously high, perhaps by asmuch as five degrees. We will not try to fix this apparent error in reading (or intranscription), nor will we remove the data from the data set. We will consider twomethods of data fitting and compare their behavior with respect to this data set as awhole.

Solution #1. We can solve the Thermo problem using the method of least squares.Using our knowledge of optimization, we can formulate the relevant optimizationproblem and solve it without resorting to looking up formulas from other resources.In other words, we can truly understand why the least squares formulas work.

We seek coefficients a and b so that yk ≈ axk+b for all k. One measure of goodnessis to minimize the sum of the squared deviations:

z =9∑

k=1

(axk + b− yk)2,

where axk + b is the actual temperature predicted by the model and yk is the actualtemperature. The above objective computes all differences between prediction andacutal, squares them, then adds them together. This optimization problem has noconstraints on a and b. We have

mina,b

z =9∑

k=1

(axk + b− yk)2

s.t. a, b ∈ R.

This is an unconstrained optimizationn problem in two variables. Thus, any optimalpoints will be stationary points of the objective function. First, compute the gradient


of the objective with respect to the decision variables.

∇z(a, b) =

∂z/∂a

∂z/∂b

=

2

9∑k=1

xk (axk + b− yk)

29∑

k=1

(axk + b− yk)

= 2

a

9∑k=1

x2k + b

9∑k=1

xk −9∑

k=1

xkyk

a9∑

k=1

xk + b9∑

k=1

1−9∑

k=1

yk

= 2

9∑

k=1

x2k

9∑k=1

xk

9∑k=1

xk

9∑k=1

1

a

b

− 2

9∑

k=1

xkyk

9∑k=1

yk

Stationary points satisfy ∇z(a, b) = 0, which leads to

a∗

b∗

=

9∑

k=1

x2k

9∑k=1

xk

9∑k=1

xk

9∑k=1

1

−1

9∑k=1

xkyk

9∑k=1

yk

=

29980.82 460

460 9

−1 29322.46

449.9

≈

0.9780

0.0007

.This solution represents the linear fit function y = (0.9780)x+ (0.0007). We leave itto the reader to show that this stationary point is a global minimizer of the objectivefunction. Both the data and best fit line are shown in Figure 11.1.

Solution #2. Next, we solve the problem by the method of least deviation usingthe same model fitting function y = ax + b. In this case, we consider an objective


Figure 11.1: Raw data (red), least squares linear fit (black) and least deviation linearfit (blue) for the Thermo calibration data.


function which measures the least total deviation between the prediction and thedata, rather than the square. We have

z =9∑

k=1

|axk + b− yk|.

This objective function could be linearized with the use of excess and deficit variablesas was done with the Shoes All Year problem. We will introduce a second methodwhich requires fewer new decision variables, but more additional constraints. Noticethat for any real value, |w| = max w,−w.

Remark 19. The above discussion suggests that these two optimization prob-lems are equivalent:

minw

z = |w|

s.t. Aw ≤ b

w ∈ R

minw,δ

z = δ

s.t. δ ≥ w

δ ≥ −wAw ≤ b

w ∈ R.

These linear programs return the same z∗ and w∗ though they have differentfeasible regions. Notice that whenever δ > |w|, the LP can choose a smaller δso that either δ = w or δ = −w. That is, δ∗ = max w∗,−w∗ = w∗.

So, we have a linearization of the least deviation problem:

mina,b,δ

z =9∑

k=1

δk

s.t. δk ≥ axk + b− yk k = 1, 2, . . . , 9

δk ≥ −axk − b+ yk k = 1, 2, . . . , 9

a, b ∈ Rδ ∈ R9.

At the optimal solution a∗ and b∗ are the optimal model coefficients and the δ∗k arethe absolute deviations between the model prediction and each data point. For thegiven calibration data, we have (a∗, b∗) = (0.9899, 0.0000). The best fit line is shownin Figure 11.1 as the blue line.


The optimization model can be written somewhat compactly. Suppose that wehave n data points (xk, yk). We can write the optimization problem as

mina,b,δ

z = 0a+ 0b+ δ1 + δ2 + . . .+ δn

s.t. − x1a− b+ δ1 ≥ −y1

− x2a− b + δ2 ≥ −y2

...

− xna− b + δk ≥ −yn+ x1a+ b+ δ1 ≥ +y1

+ x2a+ b + δ2 ≥ +y2

...

+ xna+ b + δk ≥ +yn

a, b ∈ Rδ ∈ Rn

The symmetry of the constraints allows us to write

minw z = cTw

s.t. Aw ≥ B

w ≥ 0

w ∈ Rn+2

wT =[a b δ1 δ2 . . . δn

]cT =

[0 0 1 1 . . . 1

]

A =

−x −1n In

+x +1n In

B =

−y+y

.

(11.1)

Here, x is the column vector of x-values, y is the column vector of y-values, 1n is thecolumn vector of n ones, and In is the n × n identity matrix. Remember that the x


and y values are given data, not problem variables.

Remark 20. The matrix A above in written in block matrix form where entirerectagular subarrays are written as a single variable. This method saves spaceand can greatly facilitate the use of software tools.

Discussion. Notice that the least deviation fit is less influenced by the presenceof the outlier point (x8, y8). We may be more inclined to accept the least deviationresult because it more closely approximates eight of the nine data points. A detaileddescription is beyond the scope of this work,; however, we can make a general ob-servation. The least squares method is appropriate if the uncertainty in the data isstationary random Gaussian. The least deviation method may be desireable if thedata contains some outliers which are difficult to identify.

Octave Solution to Linear Least Deviation. We can use the optimization prob-lem (11.1) to illustrate how to quickly construct large matrices relevant to optimiza-tion problems. The following code solves the linear least deviation problem in Octave.It makes use of the block matrix structure of the various matrices and vectors.

x=[10.4;21.0;29.8;39.7;50.4;60.0;71.2;87.2;90.3];

y=[ 9.5;20.1;29.5;38.9;50.0;59.2;71.0;81.8;89.9];

N=length(x);

w=ones(N,1);

c=[0;0;w];

A=[-x -w eye(N) ; x w eye(N) ];

B=[-y ; y];

lb=zeros(N+2,1);

ub=[];

ct=repmat("L",1,2*N);

vt=repmat("C",1,N+2);

sense=1;

[xstar,zstar]=glpk(c,A,B,lb,ub,ct,vt,sense);


Remark 21. Key Octave/Matlab commands for building matrices:

ones(r,c) creates a matrix (or vector) with r rows and c columns whereeach entry is 1. ones(2,4) returns 1 1 1 1

1 1 1 1

. zeros(r,c) creates a matrix (or vector) with r rows and c columns where

each entry is 0. zeros(1,3) returns[0 0 0

].

inf(r,c) creates a matrix (or vector) with r rows and c columns where eachentry is ∞. inf(3,1) returns

inf

inf

inf

. eye(n) creates the n× n identity matrix. eye(3) returns

1 0 0

0 1 0

0 0 1

. repmat(A,r,c) creates a block matrix with matrix A repeated r times ver-

tically and c times horizontally. repmat([1 2 3],2,1) returns 1 2 3

1 2 3

. repmat(str,1,c) creates a string variable with string entry str repeated c

times. repmat(’hwo’,1,3) returns ’hwohwohwo’.

11.8. ATMOSPHERIC CO2 – NONLINEAR DATA FITTING 101

11.8 Atmospheric CO2 – Nonlinear Data Fitting

This example demonstrates that least-deviation methods can also be used to modeldata using a subclass of nonlinear functions. octave code is provided.

It may come as a surprise that we can use linear programming to solve manynonlinear data fitting problems. The key aspect is that the model of the data isexpressible as a linear combination of non-parametrized – but otherwise arbitrary –functions. Consider the monthly mean atmospheric carbon dioxide (CO2) measure-ment data taken at Mauna Loa in Hawaii1. Data from the last several years is shownin Figure 11.2.

Figure 11.2: Monthly average CO2 concentration as measured in the atmosphere onMauna Loa (red) and a five-parameter least deviation fit to the data (blue).

The data suggests two major time-dependent features. First, there is a generalupward trend that appears roughly linear. Second, there is a cyclic annual variation.A possible model that captures these features is

y = a0 + a1x+ a2x2 + a3 sin (2πx) + a4 cos (2πx) .

Here, x = T − 2010 is the number of years since the beginning of 2010, where T isthe calendar year. y is the CO2 concentration in parts per million. The coefficients

1Source: NOAA Earth Systems Research Laboratory:ftp://aftp.cmdl.noaa.gov/products/trends/co2/co2 mm mlo.txt


a0, a1, a2, a3, a4 are the undetermined coefficients of the model – our decision variables.This model allows for a quadratic trend and annual cyclic variation of variable phaseshift. We want to choose coefficients a so that

yk = a0 + a1xk + a2x2k + a3 sin (2πxk) + a4 cos (2πxk)

for all data points (xk, yk). Using the method of least deviation, we have the opti-mization problem

mina

z =N∑k=1

∣∣a0 + a1xk + a2x2k + a3 sin (2πxk) + a4 cos (2πxk)− yk

∣∣

and the equivalent linear formulation

mina,δ

z =N∑k=1

δk

s.t. δk ≥ +a0 + a1xk + a2x2k + a3 sin (2πxk) + a4 cos (2πxk)− yk k = 1, 2, . . . , N

δk ≥ −a0 − a1xk − a2x2k − a3 sin (2πxk)− a4 cos (2πxk) + yk k = 1, 2, . . . , N.

Upon some rearrangement of terms, we have

mina,δ

z =N∑k=1

δk

s.t. −a0 − a1xk − a2x2k − a3 sin (2πxk)− a4 cos (2πxk) + δk ≥ −yk k = 1, 2, . . . , N

+a0 + a1xk + a2x2k + a3 sin (2πxk) + a4 cos (2πxk) + δk ≥ +yk k = 1, 2, . . . , N.

11.8. ATMOSPHERIC CO2 – NONLINEAR DATA FITTING 103

which we can write in matrix form

minw

z = cTw

s.t. Aw ≥ B

w ≥ 0

w ∈ RN+5

wT =[a0 a1 a2 a3 a4 δ1 δ2 . . . δN

]cT =

[0 0 0 0 0 1 1 . . . 1

]

A =

−1N −x −X −S −C IN

+1N +x +X +S +C IN

B =

−y+y

where 1N is the column vector of N ones, IN is the N ×N identity matrix, X is thecolumn vector of squared x values, S and C are the column vectors of values sin(2πx)and cos(2πx), respectively. The following Octave code solves the CO2 data fittingproblem (assuming vectors x and y are defined.

N=length(x);

c=[zeros(5,1);ones(N,1)];

w=ones(N,1);

X=x.^2;

S=sin(2*pi*x);

C=cos(2*pi*x);

A=[-w -x -X -S -C eye(N) ; w x X S C eye(N)];

B=[-y ; y];

lb=-inf(N+5,1);

ub=[];

ct=repmat("L",1,2*N);

vt=repmat("C",1,N+5);

sense=1;

[xstar,zstar]=glpk(c,A,B,lb,ub,ct,vt,sense);


The optimal model parameters are

a∗ = [ 388.6 2.114 0.0314 2.976 − 0.840 ] .

We can interpret these values as follows.

The seasonally adjusted CO2 level at the beginning of year 2010 was 389 ppm.

The annual increase in CO2 level is 2.11 ppm at the beginning of year 2010.

The annual increase in CO2 level is 2.36 ppm at year 2018 (a1 + 2a2(8) = 2.36).

The phase of annual variation is close to zero, most of the amplitude being inthe sine term.

The seasonal variation in CO2 concentration has amplitude 3.09 ppm (3.09 =√(2.976)2 + (0.840)2).

11.9 ExaByte Networking

This example illustrates how linear programming can answer questions about paths onweighted graphs. Often such problems are difficult to model efficiently. The utility ofbox constraints is emphasized.

The ExaByte company specializes in rapid data transfer systems. They are consid-ering using the existing computer network shown in Figure 11.3 for daily operations.Each network node (labeled with violet lower case roman letters) is connected to twoor more other nodes. Each connection is indicated by a red line. The blue num-bers indicate the maximum operating transfer rate (bandwidth) for each connection.ExaByte would like to know the maximum transfer rate achievable between startingnode s and target node t, and what routing should data take within the network.

Solution. We will assume that there is no data storage at any node, so that dataonly passes through the intermediate nodes a, b, c, d and e. Also, data can traveleither direction along any connection. The maximum possible transfer rate out of thestarting node is 32+13+9 = 54. The maximum possible transfer rate into the targetnode is 43 + 8 = 51. So, whatever the optimal transfer rate is, it will not be largerthan 51. It remains to be seen whether or not the network can support an overalltransfer rate of 51.

We choose decision variables which measure the data transfer rate used along eachnetwork connection. Let

xij = (transfer rate from node i to node j),

11.9. EXABYTE NETWORKING 105

s

a

b

c

d

et

32

13

9

10

12

41

43

8

28

35

Figure 11.3: Representation of a 7-node computer network. Roman letters indi-cate computers, red lines indicate hardwired connections, and red numbers indicatemaximum data transfer rates (bandwidths) along each connection.

where i and j are taken from the node set s, a, b, c, d, e, t. In particular, we havethe 10 decision variables

xT =[xsa xsb xsc xab xad xbe xcd xce xdt xet

].

Each decision variable can take on positive or negative values with the meaning:

xij > 0⇒ data is moving from i to j,

xij < 0⇒ data is moving from j to i.

The objective is to maximize the transfer rate from node s to node t. But as wehave already seen, this can be measured by maximizing the transfer rate either outof node s or into node t. If we choose the later, we have

z = xdt + xet.

Flow along each connection is limited by the connection bandwidth. For example,the connection between node a and node b has a bandwidth of 12, so we have theconstraints

−12 ≤ xab ≤ 12.

Since we need not consider data transfer into the starting node we have

0 ≤ xsa ≤ 32.


And, since we need not consider data transfer out of the target node we have

0 ≤ xdt ≤ 43.

Taken all together, we have the bound constraints on bandwidth

0 ≤ xsa ≤ 32

0 ≤ xsb ≤ 13

0 ≤ xsc ≤ 9

−12 ≤ xab ≤ 12

−10 ≤ xad ≤ 10

−28 ≤ xbe ≤ 28

−41 ≤ xcd ≤ 41

−35 ≤ xce ≤ 35

0 ≤ xdt ≤ 43

0 ≤ xet ≤ 8

We also must consider that no data storage can occur at intermediate nodes. Thismeans, for example, that the total data rate into node a must be equal to the totaldata rate out of node a. This is represented by the constraint

xsa = xab + xad.

This constraint holds whether or not the individual variable values are positive ornegative. The constraints for the other intermediate nodes are similarly constructed.We have no sign restrictions on the decision variables, rather we have the bound con-straints found earlier. The reader should verify that the appropriate LP for answeringExaByte’s questions is:

maxx

z = cTx

s.t. Ax = b

` ≤ x ≤ u

x ∈ R10

11.9. EXABYTE NETWORKING 107

where

xT =[xsa xsb xsc xab xad xbe xcd xce xdt xet

]cT =

[0 0 0 0 0 0 0 0 1 1

]`T =

[0 0 0 −12 −10 −28 −41 −35 0 0

]uT =

[32 13 9 12 10 28 41 35 43 8

]

A =

+1 0 0 −1 −1 0 0 0 0 0

0 +1 0 +1 0 −1 0 0 0 0

0 0 +1 0 0 0 −1 −1 0 0

0 0 0 0 +1 0 +1 0 −1 0

0 0 0 0 0 +1 0 +1 0 −1

bT =

[0 0 0 0 0

]The optimal solution is

x∗ =[22 13 9 12 10 25 33 −24 43 1

]Tz∗ = 44.

The network path is shown in Figure 11.4. The maximum transfer rate is 44.Notice that xce < 0 so data moves from node e to node c, while xad > 0 so datamoves from node a to node d.

Remark 22. If the bounds on every variable are captured using bound con-straints ` ≤ x ≤ u, then one should always use the lower and upper boundinputs provided by Octave/Matlab functions. Placing constraints in the A ma-trix for inequality constraints Ax ≤ b is very inefficient for solvers and it isalso an inefficient way to express these simple constraints.


s

a

b

c

d

et

22

13

9

10

12

33

43

1

25

24

Figure 11.4: Solution of the ExaByte computer network problem with transfer ratesalong each connection indicated by arrows for direction and numerical values fortransfer rates.

11.10 Counterfeit Bank Note Detection

Similarly to data fitting, linear programming can also be used to find an optimalnonlinear functions that separates data into two classes. This model objective is tomaximize a quantity that can only be expressed in terms of inequalities.

MegaBank is studying methods for detection of counterfeit bank notes. Because ofthe volume of bills that pass through the bank daily, they wish to have an automatedsystem based on analysis of visual clues. One strategy is to examine properties of thehistogram of wavelet coefficients which describe an image of a banknote. Histogramsof authentic bills and counterfeit bills show distinct properties.2 Sample data is shownin Figure 11.5. Wavelet coefficient histogram analysis was performed for images of75 authentic bank notes (blue) and 75 counterfeit bank notes (red). The two groupsform distinct sets in the given two-dimensional space. MegaBank would like a simplefunction that separates the two groups so that if a new bill has histogram propertiesbelow this function, it will be flagged as likely counterfeit.

Solution. Each bank note is associated with a point (u, v), where the variable uindicates the sum of the bank note’s histogram variance and skewness, and the variablev indicates the bank notes’s histogram kurtosis. We seek a function f(u) so that if

2L. Volker, J. L. Hoffmann, H. Dorksen, R. Hildebrand, E. Gillich, J. Hofmann, J. Schaede,Banknote authentication with mobile devices, Proc.SPIE, 8665, (2013).

11.10. COUNTERFEIT BANK NOTE DETECTION 109

Figure 11.5: Wavelet coefficient histogram data for images of 75 authentic banknotes (blue) and 75 counterfeit bank notes (red).

the bank note is authentic f(u) ≤ v, but if the bank note is counterfeit f(u) > v.Let (ui, vi) be the set of points associated with m authentic bank notes. Let

(wj, xj) bet the set of points associated with n counterfeit bank notes. Consider thefollowing picture.

ui wj

∗vi

f(ui)

xj

f(wj)

The ‘*’ indicates a point (ui, vi) which should lie above the function f (solid line).The ‘o’ indicates a point (wj, xj) which should like below the function f . If all points(ui, vi) lie above the function f by at least some value δ, then

vi − f(ui) ≥ δ i = 1, 2, . . . ,m.

Similarly, if all points (wj, xj) lie below the function f by at least some value δ, then

f(wj)− xj ≥ δ j = 1, 2, . . . , n.


One possible good separator function is the one that maximizes the gap δ. Consider(for now) functions that can be written as a linear combination of N given arbitraryfunctions φk:

f(x) =N∑k=1

akφk(x),

where the ak are coefficients to be determined. We have the linear program

maxδ,a

z = δ

s.t. vi −N∑k=1

akφk(ui) ≥ δ i = 1, 2, . . . ,m

N∑k=1

akφk(wj)− xj ≥ δ j = 1, 2, . . . , n

δ ∈ R

a ∈ RN .

We can write this LP in matrix form:

maxy

z = cTy

s.t. Ay ≤ B

y ∈ RN+1

yT =[δ a1 a2 . . . aN

]cT =

[1 0 0 . . . 0

]

A =

1m φ1(u) φ2(u) . . . φN(u)

1n −φ1(w) −φ2(w) . . . −φN(w)

B =

v

−x

Linear Classifier. Suppose we wish to solve MegaBank’s problem using a linearclassifier: f(t) = a1 + a2t. In this scenario, we have N = 2, φ1(t) = 1 and φ2(t) = t.


We also have n = m. The LP is:

maxy

z = cTy

s.t. Ay ≤ B

y ∈ R3

yT =[δ a1 a2

]cT =

[1 0 0

]

A =

1n 1n u

1n −1n −w

B =

v

−x

The optimal solution is δ∗ = −0.832, a∗1 = 1.87 and a∗2 = −0.86. That is, the optimalclassifier function is f(t) = 1.87 − 0.86t. This function, is shown in the top-left plotin Figure 11.6. Notice that this classifier function does not satisfy the condition thatall points for authentic bank notes lie above and all points for counterfeit bank noteslie below. This is because no such linear classifier exists. The maximum gap δ∗ isnegative. In this case, δ measures the largest mis-classification, and the best fit lineis the one for which the misclassification is minimal for both types of bank notes.

Higher-Order Classifiers. Optimal classifier functions are shown in Figure 11.6for polynomial orders up to five. The quadratic classifier also does not separate alldata points into their known respective groups (δ∗ = −0.341). However, the higher-order classifiers succeed in dividing the data into the two known groupings (δ∗ > 0).

The usefulness of any classifier is not solely determined by how well it can divideknown data into distint groups. A classifier should also accurately place new datapoints into their correct groups. We can measure the effectiveness of a classifier bythe false positive and false negative rates for data that was not used to construct theclassifier. Consider 400 new data points, 200 for authentic bills and 200 for counterfeitbills. Using the same three classifiers, we find the following false classification data.


Figure 11.6: Five classifier functions for the MegaBank data set: linear (top-left),quadratic (top-right), cubic (middle-left), quartic (middle-right), quintic (bottom).


Figure 11.7: The quintic classifier shown with the set of 400 test data points.

linear quadratic cubic quartic quintic

false positives 17 11 1 1 5

false negatives 4 6 4 3 8

The geometry of the quintic classifier is significantly affected by borderline datapoints and does not accurately represent the nature of the boundary between dataclasses. It is a less accurate classifier than the classifiers of order three or four.Figure 11.7 shows the quintic classifier with the 200 test data points.


11.11 Water Treatment Plant Planning

The state of Washington plans to build pollution removal facilities along the ColumbiaRiver. Four sites are under consideration. The cost of building at each site and thecost of treating each ton of water is given in the table. The legislature requires thatat least 80,000 tons of pollutant 1 and 50,000 tons of pollutant 2 be removed fromthe river. The amounts of each pollutant removed from treated water is given in thefollowing table. Find a lowest-cost strategy that meets the legislatures goals.

cost of treating tons pollutant 1 tons pollutant 2

site cost of building 1 ton of water per ton of per ton of

($) ($) treated water treated water

1 100,000 20 0.40 0.30

2 70,000 30 0.30 0.20

3 60,000 30 0.25 0.20

4 40,000 40 0.20 0.25

Solution. Notice that there is no requirement to build a treatment plant on anyparticular site. First, construct an integer program that takes into account all of theproblem conditions except for deciding where to build. Let

xi = (tons of water processed at site i), i = 1, 2, 3, 4.

We wish to minimize the cost of water treatment. We have the (initial) integerprogram

minx

z = 20x1 + 30x2 + 30x3 + 40x4

s.t. 0.40x1 + 0.30x2 + 0.25x3 + 0.20x4 ≥ 80,000

0.30x1 + 0.20x2 + 0.20x3 + 0.25x4 ≥ 50,000

x ≥ 0

x ∈ R4

Because we do not need to build on each site, we create new binary variables. Fori = 1, 2, 3, 4, we define

yi =

1 if a facility is built on site i

0 if a facility is not built on site i

11.11. WATER TREATMENT PLANT PLANNING 115

Our remaining task is to create appropriate constraints that connect the originalvariabes x with the new variables y. The constraints need to model the condition:

yi =

1 if xi > 0

0 if xi = 0

If we can do this, then the new objective function which measures the total cost is

z = 20x1 + 30x2 + 30x3 + 40x4 + 100,000y1 + 70,000y2 + 60,000y3 + 40,000y4.

Notice that with this objective function, the necessary constraint conditions can bewritten simply yi = 1 if xi > 0, yi ∈ 0, 1. This is because, if xi = 0 then theobjective will force yi = 0 in order to minimize the cost. That is, the optimizationproblem will never choose to build on a site that does not process any water. Thenecessary constraints are

yi ≥xiM, yi ∈ 0, 1 ,

where M is chosen so that 0 ≤ xi ≤ M for all possible xi. Notice that if xi > 0,then yi > 0 and because yi ∈ 0, 1, we have yi = 1. A sufficiently large value for Mcan be chosen based on existing problem constraints. Notice that the first constraintshows that

x1 ≤ 80,000/0.40 = 200,000,

x2 ≤ 80,000/0.30 < 267,000,

x3 ≤ 80,000/0.25 = 320,000,

x4 ≤ 80,000/0.20 = 400,000.

Thus, M = 400,000 is sufficient for any combination of nonzero values for x. Similarly,the second constraint shows that

x1 ≤ 50,000/0.30 < 167,000,

x2 ≤ 50,000/0.20 = 250,000,

x3 ≤ 50,000/0.20 = 250,000,

x4 ≤ 50,000/0.25 = 200,000.

We see that M = 250,000 is sufficient for any combination of nonzero values for x.Overall, we can choose M = 400,000 as a good upper bound on x. Thus, the full


integer program is

minx,y

z = 20x1 + 30x2 + 30x3 + 40x4 + 100,000y1 + 70,000y2 + 60,000y3 + 40,000y4

s.t. 0.40x1 + 0.30x2 + 0.25x3 + 0.20x4 ≥ 80,000

0.30x1 + 0.20x2 + 0.20x3 + 0.25x4 ≥ 50,000

xi − 400,000yi ≤ 0 i = 1, 2, 3, 4

x ≥ 0

x ∈ R4

yi ∈ 0, 1 i = 1, 2, 3, 4

Remark 23. In a cost minimization problem with fixed fee k for initiating someaction x > 0, we introduce a binary variable y which indicates the presence (orabsence) of the fixed cost. We have an objective function z = f(x) + ky withconstraint x−My ≤ 0 for a value M larger than any possible value for x.

Remark 24. Binary variables in mixed integer programs are best treated inOctave/Matlab as integer variables with bounds.

yi ∈ 0, 1 is equivalent to 0 ≤ yi ≤ 1 with yi ∈ Z.

So, an equivalent mixed-integer program is

minx,y

z = 20x1 + 30x2 + 30x3 + 40x4 + 100,000y1 + 70,000y2 + 60,000y3 + 40,000y4

s.t. 0.40x1 + 0.30x2 + 0.25x3 + 0.20x4 ≥ 80,000

0.30x1 + 0.20x2 + 0.20x3 + 0.25x4 ≥ 50,000

xi − 400,000yi ≤ 0 i = 1, 2, 3, 4

0 ≤ x

0 ≤ y ≤ 1

x ∈ R4

y ∈ Z4

11.12. GOLDBACH’S CONJECTURE 117

11.12 Goldbach’s Conjecture

In 1752, the mathematician Christian Goldbach made a conjecture concerning primenumbers. In modern terms, his, as yet unproven, conjecture is written: “Every evennumber greater than 2 can be written as the sum of two prime numbers.” Whilewe are not about to prove or disprove this conjecture, we can construct an integerprogram that can test the conjecture for a particular choice of even number. Forexample, given the even number 10, our integer program may return the two primenumbers 7 and 3 because 10 = 3 + 7.

Suppose we seek two prime numbers which when added together sum to a giveneven number E. The candidate prime numbers must come from the ordered set ofprime numbers pkNk=1 where pN is the largest prime less than E. For example,suppose E = 34, then the ordered set of prime numbers is

pk11k=1 = 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31 ,

and there are a few solutions: 34 = 3 + 31 = 5 + 29 = 11 + 23 = 17 + 17.

The idea is to create an integer program that chooses which two prime numberscan sum to a given even number. Since our list of primes has N elements, we let xkbe the number of times pk is included in the sum. Then

∑Nk=1 xkpk is the sum of a

selection of prime numbers and∑N

k=1 xk is the number of primes selected. We havethe initial integer optimization problem

minx

z =

∣∣∣∣∣E −N∑k=1

xkpk

∣∣∣∣∣s.t.

N∑k=1

xk = 2

x ≥ 0

x ∈ ZN

If the prime sum is equal to E then we have a solution with z∗ = 0. However, if noprime sum adds to E, then we have found an even integer E that does not satisfyGoldbach’s conjecture. We can reformulate this optimization problem as an integer


program.

minδ,x

z = δ

s.t.

N∑k=1

xk = 2

δ ≥ +E −N∑k=1

xkpk

δ ≥ −E +N∑k=1

xkpk

x ≥ 0

δ ∈ Z

x ∈ ZN

For this problem, we seek a specific condition on the objective value – that it equalszero. So, we can alternatively reformulate this problem as a feasibility problem inwhich we seek any feasible point. To do this, we set δ = 0, and our inequalityconstraints become an equality constraint:

minx

z = 0

s.t.N∑k=1

xk = 2

N∑k=1

xkpk = E

x ≥ 0

x ∈ ZN

The objective function is no longer relevant because we already know what objectivevalue we require. The only question is whether or not some set of decision variablesis feasible with this value.

The following Octave code tests an even number E to see if there are two primenumbers with E as the sum. The user only selects the value of E.

E=62; % choose a positive even integer

P=primes(E-1); % P is the set of prime numbers (row vector)

N=length(P); % N is the number of primes in P

c=zeros(N,1);

A=[ones(1,N);P];

b=[2;E];

11.13. PACKING BOOKS 119

lb=zeros(N,1);

ub=2*ones(N,1);

ct=’SS’;

vt=repmat(’I’,1,N);

sense=1;

[xstar,zstar]=glpk(c,A,b,lb,ub,ct,vt,sense);

% These remaining lines produce nice output to the prompt

if isfinite(xstar(1))

idx=find(xstar);

if length(idx)==1,idx=[idx idx];end

fprintf(’%d = %d + %d\n’,E,P(idx(1)),P(idx(2)));

else

fprintf(’No two primes sum to %d\n’,E);

end

11.13 Packing Books

Two friends are planning a cross country drive and are selecting which books to bringto read duing the drive. The choices of books and their length are shown in the tablebelow. Theys have decided to choose books based on the following restrictions.

Bring no more than two of the three mystery stories.

Bring at least one of the three books from the “... of the World” series.

Bring at least two of the four books about cats.

Either bring all three Kangaroo books or bring none.

If “Short Stories for Long Trips” is brought then “Long Stories for Short Trips”should also be brought.

“Math is Fun” must be brought if either “Shipwrecks of the World” or “TheSanta Fe Mystery” is brought.

Bring books whose lengths total as many pages as possible, but limit this lengthto no more than 3000 pages.

Formulate an integer program that can be used to select the books to take on thetrip.


Book Number Book Length (pages) Title

1 231 The Mystery of the Golden Cat

2 156 Jokes of the world

3 462 A Climber’s Guide to Volcanoes

4 312 The Kangaroo Cart Mystery

5 124 Kangaroo Breeds for Dummies

6 296 Math is Fun

7 446 The Santa Fe Mystery

8 374 Short Stories for Long Trips

9 228 Cats of the World

10 374 Long Stories for Short Trips

11 155 Shipwrecks of the World

12 362 Cats are Fun

13 397 Kangaroos and Cats, Friends for Life

Solution. We have 13 choices to make – whether or not to bring each of 13 books.For k = 1, 2, . . . , 13, let

xk =

1 if book number k is to be brought on the trip

0 if book number k is not to be brought on the trip

Also, let pk be the number of pages in book k. Our goal is to maximize the totalnumber of pages in books brought on the trip:

z =13∑k=1

pkxk,

subject to many constraints. Because all of the decision variables are binary, a varietyof conditional constraints can be modeled simply. Let’s consider the list in order.

Bring no more than two of the three mystery stories.

The three mystery stories are books 1, 4 and 7. So we limit these choices toany two or fewer:

x1 + x4 + x7 ≤ 2.

This constraint limits our mystery book choices to any combination of the threeexcept all three together.

11.13. PACKING BOOKS 121

Bring at least one of the three books from the “... of the World” series.

Similarly to the mystery book constraint, we have

x2 + x9 + x11 ≥ 1

Bring at least two of the four books about cats.

x1 + x9 + x12 + x13 ≥ 2.

Either bring all three Kangaroo books or bring none.

Since this is another binary choice (all or none) we introduce a new binaryvariable:

y =

1 if all three Kangaroo books are brought on the trip

0 if no Kangaroo books are brought on the trip

Then, we have the constraint

x4 + x5 + x13 = 3y.

Notice that when y = 0 it must be true that x4 = x5 = x13 = 0. And wheny = 1 it must be true that x4 = x5 = x13 = 1. No other combinations ofy, x4, x5, x13 satisfy this constraint.

If “Short Stories for Long Trips” is brought then “Long Stories for Short Trips”should also be brought.

This condition reads, if x8 = 1 then x10 = 1, but if x8 = 0 then x10 can equaleither 0 or 1. This condition is met using the following constraint:

x10 ≥ x8.

This constraint works because the xk are restricted to be binary.

“Math is Fun” must be brought if either “Shipwrecks of the World” or “TheSanta Fe Mystery” is brought.

Similarly to the previous constraint:

2x6 ≥ x7 + x11.

If x7 + x11 has a value of 1 or 2, then x6 = 1. Otherwise x6 could equal eitherzero or one.


Bring books whose lengths total as many pages as possible, but limit this lengthto no more than 3000 pages.

13∑k=1

pkxk ≤ 3000.

Bringing the objective and constraints together, we have the binary integer pro-gram

minx,y

z =13∑k=1

pkxk

s.t. x1 + x4 + x7 ≤ 2

x2 + x9 + x11 ≥ 1

x1 + x9 + x12 + x13 ≥ 2

x4 + x5 + x13 − 3y = 0

x8 − x10 ≤ 0

2x6 − x7 − x11 ≥ 0

z =13∑k=1

pkxk ≤ 3000

0 ≤ y, xk ≤ 1 1 ≤ k ≤ 13

y, xk ∈ Z 1 ≤ k ≤ 13

The answer is to bring every book except books 1, 3 and 9 (“The Mystery of theGolden Cat,” “A Climbers Guide to Volcanoes” and “Cats of the World”). The totalpage count is z∗ = 2996.

A Greedy Algorithm Approach. Suppose we compare the optimal solution toa book choice determined by a greedy approach. The idea is the following.

1. Begin with the books on the shelf and an empty box of books to take on thetrip.

2. Choose the book on the shelf with the largest number of pages.

3. Place this book in the box if its inclusion would not violate any of the con-straints. Otherwise, set it aside.

4. If there is at least one book still on the shelf, then go to step 2. Otherwise, goto step 5.

11.14. COPYMASTER MACHINE SCHEDULING 123

5. If the box of books satisfies all constraints, then the box contains a greedyselection. Otherwise, the greedy algorithm failed to provide a feasible solution.

The following table shows the results of each pass of the greedy algorithm atstep 3. Numerical values in the table are book numbers.

books on shelf books in box (pages) set aside notes

1,2,3,4,5,6,7,8,9,10,11,12,13 Starting Condition

1,2,4,5,6,7,8,9,10,11,12,13 3 (462) Longest book added to box

1,2,4,5,6,8,9,10,11,12,13 3,7 (908) Next longest added

1,2,4,5,6,8,9,10,11,12 3,7,13 (1305) Third longest added

1,2,4,5,6,9,10,11,12 3,7,8,13 (1679) 8 and 10 are same length

1,2,4,5,6,9,11,12 3,7,8,10,13 (2053) 8 and 10 are same length

1,2,4,5,6,9,11 3,7,8,10,12,13 (2415) Next longest

1,2,5,6,9,11 3,4,7,8,10,12,13 (2722) Next longest

1,2,5,9,11 3,4,7,8,10,12,13 (2722) 6 page constraint

2,5,9,11 3,4,7,8,10,12,13 (2722) 1,6 first constraint

2,5,11 3,4,7,8,9,10,12,13 (2950) 1,6 9 added to box

5,11 3,4,7,8,9,10,12,13 (2950) 1,2,6 page constraint

5 3,4,7,8,9,10,12,13 (2950) 1,2,6,11 page constraint

3,4,7,8,9,10,12,13 (2950) 1,2,5,6,11 page constraint

Unfortunately for this algorithm, not only did it terminate with fewer total pagesthan the optimal solution, the books in the box do not satisfy the fourth and sixthconstraints.

11.14 CopyMaster Machine Scheduling

CopyMaster is a company that specializes in making lots of copies. The company hasm copy machines of varying capabilities and speed. The time required to completeany of n types of jobs varies with machine choice. Let cij be the machine timerequired to complete a job of type j = 1, 2, . . . , n using machine i = 1, 2, . . . ,m.Suppose CopyMaster has Nj jobs of each type j to complete. The machines canrun concurrently. Can we use linear programming to decide how to assign jobs tomachines in order to complete the jobs as quickly as possible? This week begins withthe maching data and job requirements listed in the following tables. Times cij arein minutes.


cij j = 1 j = 2 j = 3 j = 4 j = 5 j = 6

i = 1 6 12 8 12 10 3

i = 2 8 6 12 7 6 10

i = 3 8 11 6 6 7 4

i = 4 3 9 8 10 3 10

j = 1 j = 2 j = 3 j = 4 j = 5 j = 6

Nj 641 190 386 469 395 482

Solution. Our decision is to determine how many of each job to assign to eachmachine. We will assume that breaking up jobs among machines does not significantlyinfluence the total work time. Let

xij = (number of jobs of type j run on machine i)

xij ∈ Z, xij ≥ 0

i = 1, 2, 3, 4 and j = 1, 2, 3, 4, 5, 6.

The objective function must measure the time it takes the busiest machine to completeits set of jobs. The time that machine i is busy with its jobs is

6∑j=1

cijxij.

So, the time of the busiest machine is

z = maxi=1,2,3,4

(6∑j=1

cijxij

)= max

6∑j=1

c1jx1j ,6∑j=1

c2jx2j ,6∑j=1

c3jx3j ,6∑j=1

c4jx4j

.

The objective is to minimize this function. If we introduce a new variable T asthe time when the busiest machine completes its jobs, we have the equivalent linearformulation

minx,T

z = T s.t.6∑j=1

cijxij ≤ T, i = 1, 2, 3, 4.

We also have the process constraints that make sure that all jobs are completed.

4∑i=1

xij = Nj, j = 1, 2, 3, 4, 5, 6.

11.14. COPYMASTER MACHINE SCHEDULING 125

We have the complete integer program

minx,T

z = T

s.t.

6∑j=1

cijxij − T ≤ 0, i = 1, 2, 3, 4∑4i=1 xij = Nj j = 1, 2, 3, 4, 5, 6

xij, T ≥ 0

xij, T ∈ Z

The optimal solution is for the jobs to be divided among the machines according tothe following table.

x∗ij j = 1 j = 2 j = 3 j = 4 j = 5 j = 6

i = 1 72 126 482

i = 2 190 248 2

i = 3 260 221

i = 4 569 393

Machines 1, 3 and 4 run a total of 2886 minutes each. Machine 2 runs for z∗ = 2888minutes (48 hours and 8 minutes).

Octave Code. This optimization problem has constraints that contain a lot ofstructure. A relatively short piece of Octave code can construct large matrices usedby the LP solver. Consider the following code that solves the CopyMaster problem.The user need only change the two inputs C and N.

% Enter the Cij matrix data (this could be automated)

C=[6 12 8 12 10 3

8 6 12 7 6 10

8 11 6 6 7 4

3 9 8 10 3 10];

% Enter the number of jobs data

N=[641;190;386;469;395;482];

% Create some size variables

[NumMachines,NumJobs]=size(C);

n=NumMachines*NumJobs+1;

M=max(C(:))*max(N);


% Create the objective vector

c=[zeros(n-1,1);1];

% Create the A matrix

A1=kron(ones(1,NumJobs),eye(NumMachines));

idx=find(A1);

A1(idx)=C;

A1=[A1 -ones(NumMachines,1)];

A2=kron(eye(NumJobs),ones(1,NumMachines));

A2=[A2 zeros(NumJobs,1)];

A=[A1;A2];

% Create the rhs b vector

b=[zeros(NumMachines,1);N];

% Create the various problelm variables for the solver

lb=zeros(n,1);

ub=M*ones(n,1);

ct=[repmat(’U’,1,NumMachines) repmat(’S’,1,NumJobs)];

vt=repmat(’I’,1,n);

sense=1;

% Call the solver

[x,z]=glpk(c,A,b,lb,ub,ct,vt,sense);

% Report the solution

reshape(x(1:end-1),NumMachines,NumJobs)

Either/Or Box Constraints. One potential difficulty with the optimal solutionfor the CopyMaster problem is that very small subjobs can occur. For example, theset of jobs of type j = 5 is divided into subjobs of 393 and 2. In order to improvethe logistics of dividing and re-combining job types, suppose constraints are added inorder to enforce a lower bound on the size of subjobs at 24. That is, either xij = 0or xij ≥ 24. If we introduce new binary variables

yij =

1 if machine i processes some of job j

0 if not

then we can formulate the needed constraints as

24yij ≤ xij ≤Myij.

11.15. PROFIT UNDER PREMIUM FEES 127

Notice that when yij = 0 we have x = 0 and when yij = 1 we have 24 ≤ xij ≤ M .The only remaining task is to choose M to be larger than any feasible xij. In thiscase, M can equal the maximum number of jobs of any type, M = 641.

This type of constraint can be generalized to limit any decision variable to bewithin one of several intervals. See Exercise 14.

11.15 Profit Under Premium Fees

Sumi is planning a flight to Seattle where she will sell some of her remaining artpieces. She has the option to take as many pieces for which she can pay the shippingcost. The weight and selling price for each type of item are shown in the table below.Shipping volume is not a limiting consideration, but shipping weight is a concern.Shipping costs are as follows: 0 − 50 pounds, $2/lb; 50 − 100 pounds, $100+$4/lbfor every pound above 50; 100+ pounds, $300+$10/lb for every pound above 100pounds. What is the maximum profit Sumi can realized on this venture and howmany of each item should she take to Seattle?

index item shipping weight(lb) selling price ($)

1 oil painting 13 55

2 wood sculpture 40 150

3 batik wall hanging 18 75

4 glazed ceramic vase 25 100

Solution. The objective is to maximize profit. The simplest decision variables areto let

xk = (number of items of type k to pack)

Then the profit function is

z = (profit $)

= (selling income $) - (shipping cost $)

= 55x1 + 150x2 + 75x3 + 100x4 − f(w),

where the shipping cost f(w) is a nonlinear function of weight w = 13x1 + 40x2 +18x3 + 25x4. We have

f(w) =

2w if 0 ≤ w ≤ 50

100 + 4(w − 50) if 50 < w ≤ 100

300 + 10(w − 100) if 100 < w

A graph of this function shows that it is piecewise linear.


What this graph also shows is that the cost for any weight w is the minimum valuethat lies at or above every line. So, the shipping cost for any weight is the solutionto

minδ

z = δ

s.t. δ ≥ 2w

δ ≥ 100 + 4(w − 50)

δ ≥ 300 + 10(w − 100),

or equivalently

maxδ

z = −δ

s.t. δ ≥ 2w

δ ≥ 100 + 4(w − 50)

δ ≥ 300 + 10(w − 100),

That is, the shipping cost for any weight is δ∗. So, we can use this solution to thenonlinear shipping cost in the objective of Sumi’s problem. We arrive at the integer

11.16. STARCRAFT BREAD DISTRIBUTORS 129

program:

maxx,w,δ

z = 55x1 + 150x2 + 75x3 + 100x4 − δ

s.t. w = 13x1 + 40x2 + 18x3 + 25x4

δ ≥ 2w

δ ≥ 100 + 4(w − 50)

δ ≥ 300 + 10(w − 100)

x,w, δ ≥ 0

x ∈ Z4

w, δ ∈ Z


x∗1 = 6, x∗2 = 0, x∗3 = 1, x∗4 = 0, w∗ = 96, δ∗ = 284.

Sumi should take six oil paintings and one batik wall hanging. The shipping weightof the art items will be 96 pounds and the shipping cost will be δ∗ = 284 pounds.Sumi’s profit will be $121.

11.16 Starcraft Bread Distributors

The Starcraft Bread Company has m bakeries which have produced s1, s2, . . . smbatches of bread loaves. Starcraft is contracted to deliver batches of bread in quan-tities d1, d2, . . . , dn to n retail stores. Suppose (for now) that any number of batchesis easily shipped from any bakery to any store. Transporting a batch of bread frombakery i to store j has a known cost cij. The cij need not be identical values dueto, for example, higher costs associated with longer travel times or other unequallogistical conditions. What is the lowest cost strategy for supplying each store withits daily bread?

This problem is an example of a transportation problem. There are other efficientmethods for solving these types of problems that do not use linear programming.

Solution. Since each bakery can supply bread to any store, we define

xij = (number of batches shipped from bakery i to store j),

where xij ≥ 0 and xij ∈ Z. Each possible transportation path has an associated costcij. Notice that

n∑j=1

xij = (total batches originating from bakery Bi),


andm∑i=1

xij = (total batches arriving at store Si).

Recognizing that we must meet consumer demands at each store and not exceed thebaking capacity of any bakery, we have the compactly expressed integer program tosolve the Starcraft Break Company problem:

minx

z =m∑i=1

n∑j=1

cijxij

s.t.n∑j=1

xij ≤ si i = 1, 2, . . . ,m

m∑i=1

xij ≥ dj j = 1, 2, . . . , n

x ≥ 0

x ∈ Znm

Balancing the Starcraft Bread Distribution. One interesting aspect of thedistribution problem is that there is no need to ship more batches than are requiredto meet consumer demand. Let s =

∑mi=1 si be the total number of batches of bread

at the bakeries, and let d =∑j = 1ndj be the total consumer demand. Consider the

following three possibilities:

1. If s < d then the available bread is insufficient to meet consumer demand. Thus,the IP is infeasible – no routing of bread will meet consumer demand.

2. If s = d then we have what is refered to as a “balanced” problem. The breadsupply is exactly sufficient to meet consumer demand. And here is the importantobservation: All of the supply and demand constraints will be active at anyfeasible solution. In this case we have the equivalent (standard form) problem:

minx

z =m∑i=1

n∑j=1

cijxij

s.t.

n∑j=1

xij = si i = 1, 2, . . . ,m

m∑i=1

xij = dj j = 1, 2, . . . , n

x ≥ 0

x ∈ Znm

11.16. STARCRAFT BREAD DISTRIBUTORS 131

3. If s > d then the bread supply exceeds the current demand. In this case, weseek to formulate a balanced problem by cleverly introducing slack variables.Create a dummy store, Sn+1 with demand exactly equal to the excess batches,dn+1 = s−d. Then allow decision variables which allow shipping to the dummydestination. We have the balanced problem

minx

z =m∑i=1

n+1∑j=1

cijxij

s.t.n+1∑j=1

xij = si i = 1, 2, . . . ,m

m∑i=1

xij = dj j = 1, 2, . . . , n+ 1

x ≥ 0

x ∈ Z(n+1)m

withm∑i=1

si −n∑j=1

dj = dn+1.

In this scenario, we have new variables xi,n+1 and new constants ci,n+1. Oneinterpretation of these quantities stems from the fact that we do not actuallyship to the dummy destination. xi,n+1 can be the number of batches of breadleft at bakery i, and ci,n+1 can be the cost of storing a batch of bread at bakeryi.

Fixed-Cost Transportation. Suppose that there are fixed costs for transportingbread. For example, if Starcraft chooses to send some bread from bakery i to storej, there may be a truck rental fee regardless of how many batches are sent. Let eijbe such a fixed fee. In the familiar way, we define

yij =

1 if xij > 0

0 if xij = 0


Then, for sufficiently large value of M , say M = maxisi, we have the integerprogram

minx,y

z =m∑i=1

n+1∑j=1

(cijxij + eijyij)

s.t.n+1∑j=1

xij = si i = 1, 2, . . . ,m

m∑i=1

xij = dj j = 1, 2, . . . , n+ 1

xij −Myij ≤ 0 i = 1, 2, . . . ,m; j = 1, 2, . . . , n+ 1

x ≥ 0

x ∈ Z(n+1)m

yij ∈ 0, 1(n+1)m

withm∑i=1

si −n∑j=1

dj = dn+1

M = maxi si

The various quantities in this integer program are interpreted as follows (for i =1, 2, . . . ,m and j = 1, 2, . . . , n).

si: The number of batches of bread at bakery i that are available for shipping.

dj: The number of batches of bread which are in demand at store j.

xij: The number of batches of bread transported from bakery i to store j.

xi,n+1: The number of batches of bread kept at bakery i.

ci,j: The per batch cost of transporting bread from bakery i to store j.

ci,n+1: The per batch cost of keeping bread at bakery i.

eij: The fixed cost of transporting bread from bakery i to store j.

ei,n+1: The fixed cost of keeping bread at bakery i.

11.17 Puzzles and Games Inc.

Puzzles and Games Inc. has the facilities for manufacturing a variety of small toysand games. However, their operation is small enough in scale that during any one

11.17. PUZZLES AND GAMES INC. 133

production period they need to choose subsets of products which can be manufacured.For example, suppose the factory can make some combination of puzzles and yoyos,OR some combination of boomerangs and frisbees. In the table below are the items,the expected profit per item, the units of raw materials need to make each toy orgame, and the amount of raw materials in stock. How can Puzzles and Games Inc.use its stock materials to maximize profit during the current production cycle?

item item # profit wood glue paint plastic

puzzle 1 5 6 1 2 0

yoyo 2 4 5 0 3 3

boomerang 3 7 9 1 3 2

frisbee 4 8 0 0 3 9

(stock) 102 12 25 57

Solution. One strategy is to solve two optimization problems: one in which onlypuzzles and yoyos are manufactured, and another one in which only boomerangs andfrisbees are manufactured. The optimal solution is then the one of higher profit.However, we wish to find a more general procedure that can be used to treat mul-tiple such constraint simultaneously. First we can formulate the problem as if anycombination of the four products can be manufactured. This IP is

maxx

z = 5x1 + 4x2 + 7x3 + 8x4

s.t. 6x1 + 5x2 + 9x3 ≤ 102

x1 + x3 ≤ 12

2x1 + 3x2 + 3x3 + 3x4 ≤ 25

3x2 + 2x3 + 9x4 ≤ 57

x ≥ 0

x ∈ Z4

where xk is the number of toys of type k to manufacture. The remaining productionconstraint is that

either x1 + x2 = 0 or x3 + x4 = 0.

Now, because x ≥ 0 for any feasible solution, we can rephrase this condition as

either x1 + x2 ≤ 0 or x3 + x4 ≤ 0.


Suppose that we need to enforce the general condition “either f(x) ≤ 0 or g(x) ≤ 0.”We introduce a binary variable:

y =

1 if f(x) ≤ 0

0 if g(x) ≤ 0

Then the necessary constraints are (for sufficiently large M):

f(x) ≤M(1− y), g(x) ≤My, y ∈ 0, 1 .

Notice that if y = 1 we have the constraints

f(x) ≤ 0, g(x) ≤M,

and if y = 0 we have the constraints

f(x) ≤M, g(x) ≤ 0.

The key is to choose M large enough that the constraint g(x) ≤ M is never activewhen y = 1 and so that f(x) ≤M is never active with y = 0.

For the Puzzles and Games problem we have f(x) = x1 + x2 and g(x) = x3 + x4

so that the IP that solves the problem is

maxx,y

z = 5x1 + 4x2 + 7x3 + 8x4

s.t. 6x1 + 5x2 + 9x3 ≤ 150

x1 + x3 ≤ 12

2x1 + 3x2 + 3x3 + 3x4 ≤ 25

3x2 + 2x3 + 9x4 ≤ 57

x1 + x2 ≤M(1− y)

x3 + x4 ≤My

x ≥ 0

x ∈ Z4

y ∈ 0, 1with M = 12

We have chosen M = 12 because this is the maximum number of any item that couldbe manufactured simply based on the materials constraints.

11.18. PIT MINE 135

11.18 Pit Mine

(From “The Basics of Practical Optimization,” by Adam B. Levy) A prospective goldmine has been subdivided into 25 5-ton blocks in the following formation:

Level 1 1 2 3 4 5 6 7 8 9 $200

Level 2 10 11 12 13 14 15 16 $300

Level 3 17 18 19 20 21 $400

Level 4 22 23 24 $500

Level 5 25 $600

where the extraction costs per ton at each level are given in the final column. Theonly exception is block 9, which sits under a wetland and as a result requires anadditional $3,700 per ton to extract. The extraction of any block at a lower levelrequires the extraction of the three nearest blocks above it (so, for example, blocks11, 12, and 13 must be extracted before block 18 can be extracted). The marketvalues in $/ton of each block are as follows:

Level 1 100 300 200 20 10 10 100 10 50

Level 2 400 500 300 200 100 100 150

Level 3 400 300 1000 1500 500

Level 4 2000 2100 1900

Level 5 3000

Formulate an integer program to determine which blocks should be extracted tomaximize profit. Assume that each block is either extracted in whole or not extractedat all.

Solution. Our decisions are whether or not to extract each individual block. Let

xk =

1 if block k is to be extracted

0 if not

for k = 1, 2, . . . , 25. Let ek be the extraction cost per ton for block k, and let vkbe the market value per ton of material removed from block k. Then the objecivefunction is

z =25∑k=1

(vk − ek)xk.


Notice that all but one of the Level 1 extractions result in negative profit. Forexample,

v1 − e1 = 100− 200 = −100

v2 − e2 = 300− 200 = +100

...

v9 − e9 = 50− 200− 3700 = −3850

...

v25 − e25 = 3000− 600 = +2400

The only constraints are the sub-excavation constraints which require, for example,that

If x10 = 1 then x1 + x2 + x3 = 3.

Because all decision variables are binary, we can enforce this condition as

x1 + x2 + x3 ≥ 3x10.

We see that if we wish to excavate block 10 (x10 = 1) then we must first excavateblocks 1,2 and 3 (x1 = x2 = x3 = 1). On the other hand, if x10 = 0 then x1, x2 andx3 are free to be either 0 or 1. There are 16 such constraints, one for each of blocks10 through 25. The integer program is

maxx

z = cTx

s.t. Ax ≥ 0

0 ≤ x ≤ 1

x ∈ Z25

where

x =[

x1 x2 . . . xk . . . x25

]Tc = 5×

[−100 +100 . . . vk − ek . . . +2400

]T

11.19. TEAM BUILDING STRATEGY 137

and

A =

1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 -3 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 -3 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 -3 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 -3 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 -3 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 -3 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 -3 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 -3 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 -3 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 -3

.

The optimal solution is to extract all blocks except numbers 9, 16, 21, 24 and 25.The profit for such a strategy is $18,250. The cost of excavating under the wetlandis not made up by lower high-profit excavations. However, if the wetland excavationcost can be reduced to (for example) $3500, then the optimal solution is to excavateall blocks with a corresponding profit of $18,750.

11.19 Team Building Strategy

Big Bear Investments is creating teams of employees to develop new plans for strategicinvestment strategies. There are 18 employees, each of which has on file results ofaptitude and teamwork ability tests. On a scale of 0 (low) to 100 (high), employeescores are shown in the table below. An employee who has been with the companyat least 10 years is considered a senior employee and is indicated in bold face.

Amiel Biniyah Camua Dalliel Eleth Foculs

Aptitude 59 93 69 81 86 50

Teamwork 67 32 56 35 31 66

Gagavr Hielaph Ikhael Jhamue Kielach Leraie

Aptitude 55 88 81 73 91 77

Teamwork 83 98 50 36 52 61

Maliel Nuriel Obor Peniyah Quima Reilah

Aptitude 30 99 37 71 91 65

Teamwork 64 50 59 93 52 38

Big Bear needs to form three teams satisfying the following conditions.


Each team has three, four or five members.

Each team must have a total aptitude score of at least 240.

Each team must have an average teamwork score of at least 65.

Senior employees cannot make up more than half of any team.

No employee can be on more than one team.

Neither Eleth nor Quima can be on a team that also includes either Biniyah orDalleil.

No subset of two or more from the list (Amiel, Eleth, Foculs, Jhamue) can workon the same team.

If Leraie is on a team then Dalleil must be on the same team.

If Camua and Kielach are on the same team then Heilaph cannot be on thatteam.

Gagavr must be on one team.

Either Eleth or Quima must not be on any team. (One or the other must beavailable to work elsewhere.)

If Peniyah is on some team then Dalliel must also be on a team, not necessarilythe same team.

Suppose productivity is measured as the product of aptitude and teamwork scores.How can Big Bear Investments form exactly three teams that maximize total produc-tivity?

Solution. We need to choose who will be on each of three teams. Let

xij =

1 if employee i is placed on team j

0 if not

where teams are indexed by j = 1, 2, 3 and employees indexed by i = 1, 2, . . . 18 inalphabetical order. Suppose the aptitude scores are ai and the teamwork scores areti. Then we have the objective function

z =18∑i=1

3∑j=1

aitixij.

Consider the given constraints in the order presented. In each case, approriate con-straints are given without explanation. You should verify that the constraints enforcethe given condition.


Each team has three, four or five members.

3 ≤18∑i=1

xij ≤ 5, j = 1, 2, 3

Each team must have a total aptitude score of at least 240.

18∑i=1

aixij ≥ 240 j = 1, 2, 3

Each team must have an average teamwork score of at least 65.

18∑i=1

(ti − 65)xij ≥ 0 j = 1, 2, 3

Senior employees cannot make up more than half of any team.

18∑i=1

xij − 2∑i∈S

xij ≥ 0 j = 1, 2, 3

where S = 4, 5, 6, 9, 10, 14, 15, 18 is the index list for senior employees.

No employee can be on more than one team.

3∑j=1

xij ≤ 1 i = 1, 2, . . . , 18

Neither Eleth nor Quima can be on a team that also includes either Biniyah orDalleil.

2x2j + x5j + x17,j ≤ 2

2x4j + x5j + x17,j ≤ 2

for j = 1, 2, 3.

No subset of two or more from the list (Amiel, Eleth, Foculs, Jhamue) can workon the same team.

x1j + x5j + x6j + x10,j ≤ 1 j = 1, 2, 3


If Leraie is on a team then Dalleil must be on the same team.

x4j ≥ x12,j j = 1, 2, 3

If Camua and Kielach are on the same team then Heilaph cannot be on thatteam.

x3j + x8,j + x11,j ≤ 2 j = 1, 2, 3

Gagavr must be on one team.

x71 + x72 + x73 = 1

Either Eleth or Quima must not be on any team. (One or the other must beavailable to work elsewhere.)

x51 + x52 + x53 + x17,1 + x17,2 + x17,3 ≤ 1

If Peniyah is on some team then Dalliel must also be on a team, not necessarilythe same team.

x41 + x42 + x43 ≥ x16,1 + x16,2 + x16,3

The binary integer program has 54 decision variables and 51 constraints. Anoptimal solution is:

Team Aptitude Teamwork Team Members

(#) (sum) (mean)

1 261 65 Nuriel , Peniyah, Quima

2 272 65 Amiel, Gagavr, Kielach, Maliel, Obor

3 296 65 Dalliel , Foculs, Heilaph, Leraie


Exercises

1. Consider the A-1 Furniture problem. Rework the problem to include the addi-tional restriction that for tables and chairs to realize their profit potential theremust be at least six times as many chairs manufactured as tables. Find the newoptimal solution. Compare this solution with the previous formulation.

2. Consider the Greenside Yard & Lawn Care problem. Under the same grantand city restrictions, reformulate the problem in order to maximize the numberof temporary jobs created under the restriciton that no more than $8000 isavailable to fund projects. Show that the maximum number of temporoary jobscreated is 29. What is the new optimal project strategy?

3. Reformulate the Tourist Trap problem to include the following additional vari-ables

x5 = total number of items purchased

x6 = total number of T-shirts purchased

x7 = total number of keychains purchased

Use these variables to simplify some of the existing constraints, at the expenseof adding new constraints. Show that the solution is the same as the originalformulation. Why might this modified formulation be preferable?

4. Solve the Woof&Meow pet food company problem. Suppose your job is topresent your findings and recommendations to the company CEO. Composeone-page executive summary for your employer outlining the company’s mostprofitable use of overstock resources. Include any findings which may be ofinterest to the company’s decision makers.

5. Solve the Pack&Ship problem using the following data.

w =[10 16 8

]T(tons)

s =[6800 8700 5300

]T(m3)

t =[18 15 23 12

]T(tons)

v =[480 650 580 390

]T(m3/ton)

p =[310 380 350 285

]]T ($/ton)

Determine which problem constraints are active constraints.


6. Consider the Shoes All Year production problem. Interpret the values of theoptimal variables x∗, s∗, y∗, w∗ in terms of the original problem statement.

7. Show that the stationary point for the least squares approach to the Thermocalibration problem is, in fact, a global minimizer of the objective function.

8. Based on the five-parameter model fit to the Mauna Loa CO2 data, suggest adifferent model with no more than seven parameters that may more accuratelyrepresent the data set. Carefully justify your model. You do not need to solvethis new problem.

9. Suppose nodes d, e and t of the ExaByte problem represent computers in asecure environment. If no data is allowed to leave this secure environment,what will the new data transfer model be? Solve this new model and comparethe optimal solution to the optimal solution to the original model.

10. Propose a different class of classifier functions that may perform well with thebank note data. Justify your choice.

11. Solve the Water Treatment Plant problem. Suppose your job is to present yourfindings and recommendations to your local state senator. Compose a one-pageexecutive summary.

12. Modify the integer program formulation which tests single even integer instancesof Goldbach’s Conjecture to only allow sums of distinct primes. For example,62 = 31 + 31 is not allowed, but 62 = 3 + 59 is allowed. Use your integerprogram to determine how many even numbers between 101 and 151 cannot bewritten as the sum of two distinct prime numbers.

13. Suggest and implement a modified Greedy Algorithm for solving the Book Pack-ing problem. Your modified algorithm should try to make sure that all con-straints are satisfied upon termination (if possible). Your new algorithm shouldnever remove a book that is placed in the box.

14. Create a set of linear constraints that enforce the following condition on x:

Either a ≤ x ≤ b or c ≤ x ≤ d.

Build from the discussion in the CopyMaster example.

15. What is the (integer) minimum selling price of a glazed ceramic vase whichmakes it profitable to bring at least one on the trip? Keep all other problemparameters the same as in the original problem statement. Describe the newoptimal solution for this vase selling price.

16. Describe (qualitatively) the solution to the fixed cost Starcraft problem for thecase minijeij maxijcij.


17. Solve the Puzzles and Games Inc. problem (a) ignoring the Either/Or manu-facturing constraints and (b) including these constraints. Compare and discussthe results.

18. Justify the choice of binary decision variables in the Pit Mine problem. Inparticular, consider why it it not necessary to allow excavation of partial blocks.

Modeling Problems

19. A technician in a zoo must formulate a diet from two commercial products,food A and food B, for a certain group of animals. In 200 grams of food Athere are 16 grams of fat, 28 grams of carbohydrate, and 6 grams of protein. In200 grams of food B there are 8 grams of fat, 50 grams of carbohydrate, and10 grams of protein. The minimum daily requirements are 176 grams of fat,980 grams of carbohydrate, and 200 grams of protein. Food A costs 8 centsper 100 grams and food B costs 18 cents per 100 grams. Write and solve anoptimization model to answer the following question: What is the minimumcost diet that meets all daily requirements.

20. For a factory that manufactures trucks and cars, each vehicle must go throughan assembly stage and a painting stage. If the painting was entirely devotedto cars, 500 per day could be painted. On the other hand, 400 trucks per daycould be painted per day if the painting was devoted to trucks. The analagousnumbers for assembly are 80 cars per day and 65 trucks per day. Each carproduces $1300 profit and each truck produces $1500 profit.

Write and solve an optimization model to answer the following question: Howmany cars and trucks should the factory produce to maximize profit?

21. (From “The Basics of Practical Optimization,” by Adam B. Levy) A person’sdaily nutritional requirements include the need for 3 units of vitamin A and 1.5units of vitamin C. Bananas sell for $6 per pound, and each pound contains 5units of vitamin A and 1 unit of vitamin C. Rice sells for $1 per pound, andeach pound contains 1 unit of each vitamin.

(a) Write and solve an optimization model to answer the following question:With only bananas and rice to eat, what is the least expensive scheme thatsatisfies the person’s requirements?

(b) Write and solve an optimization model to answer the following question:What is the least expensive scheme that satisfies the person’s requirementsexactly?


22. You decide to travel to Seattle to sell your remaining inventory of car parts.The weight, number in stock, and profit per item are listed in the table below.Airline regulations limit the total weight of your shipment to 214 pounds. Writeand solve an optimization model to answer the following question: What is themaximum profit that can be realized on this venture and which parts should betaken to Seattle?

item weight(lb) # in stock profit($)

engine 104 2 151

transmission 62 4 68

seat 28 4 44

water pump 21 7 29

23. During each 4-hour period, the (24-hour) Red-Eye Diner requires the followingnumber of on-duty staff:

00:00 - 04:00 : 3 12:00 - 16:00 : 5

04:00 - 08:00 : 8 16:00 - 20:00 : 9

08:00 - 12:00 : 6 20:00 - 24:00 : 8

Each person must work a shift of two consecutive 4-hour periods. Write andsolve an optimization model that can be used to minimize the number of staffthe the Red-Eye must hire. (Use only six decision variables.) Next, model andsolve the modified problem in which no 4-hour period begins with a whole newcrew coming on duty.

24. (Modified from “The Basics of Practical Optimization,” by Adam B. Levy) Asupermarket can be staffed with full-time employees in any way that covers thelabor requirements of 400 total person hours on Saturday, 400 on Sunday, 200on Monday, 200 on Tuesday, 250 on Wednesday, 250 on Thursday, and 300 onFriday. Union rules require that each full-time employee work 5 consecutive fulldays (8 hours each) followed by 2 days off, but the supermarket can requirethem to work 1 day of overtime each week (with only 1 day off). A full-timeemployee costs the supermarket $10 per hour for the first 40 hours and $15 perhour on overtime days.

Write and solve an optimization model to answer the following question: Howcan the supermarket meet its daily labor requirements at least cost?

25. (From “The Basics of Practical Optimization,” by Adam B. Levy) An oil com-pany has 6,000 barrels of oil 1 (with quality level 10) and 11,000 barrels of oil 2


(with quality level 6) and cannot restock for this selling period. The companysells two products made from a combination of the two oils: gasoline, whichmust have an average quality level of at least 8, and heating oil, which musthave an average quality level of at least 7. Each dollar spent advertising gaso-line results in 3 barrels of demand and each dollar spent advertising heating oilresults in 7 barrels of demand, and there is no demand without advertising.

Write and solve an optimization model to answer the following question: Ifgasoline is sold for $35 per barrel and heating oil is sold for $30 per barrel, whatshould the oil company sell to maximize profit?

26. Pots-R-Us obtains its stock of outdoor decorative pots from three suppliers.Each supplier sells pots in four sizes (large, medium, small and tiny) in thepercentage quantities and prices given in the following table. Pots are purchasedfrom each supplier in batches of 20. So, for example, each batch of 20 purchasedfrom supplier 1 contains 7 large, 8 medium, 3 small and 2 tiny pots.

Supplier Cost Large Medium Small Tiny

($/pot) (%) (%) (%) (%)

1 4 35 40 15 10

2 3 30 25 35 10

3 2 10 10 50 30

Each quarter, Pots-R-Us places and order with each supplier. At least 100large, 60 medium, 50 small and 40 tiny pots must be purchased each quarter inorder to meet demand. Because the supplier faciliites have limited production,at most 140 pots can be purchased from any supplier in any quarter. Writeand solve an optimizatin model to answer the following question: How canPots-R-Us minimize the cost of acquiring the needed pots for one quarter?

27. (From “Introduction to Mathematical Programming” by W. Winston and M.Venkataramanan) A portfolio manager has been asked to invest $1,000,000. Theinvestment research development department has identified six mutual fundswith varying investment strategies, resulting in different potential returns andassociated risks, summarized as follows

fund 1 2 3 4 5 6

price per share ($) 455 76 110 17 23 22

expected return (%) 30 20 15 12 10 7

risk category high high high med med low


One way to control the risk is to limit the amont of money invested in the variousfunds. To that end, the management has specified the following guidelines. Thetotal dollar amount invested in high risk funds must be between 50% and 75%of the total investment. The total dollar amount invested in medium risk fundsmust be between 20% and 30% of the total investment. The total dollar amountinvested in low risk funds must be at least 5% of the total investment. A secondway to control risk is to diversify. The management has specified that the sharesin each of funds 1 and 2 should not be less than 10% of the shares in the totalportfolio. With these guidelines, write and solve and optimizatin problem thatmaximizes the expected investment return. Be careful! “total dollar amountinvested” is not necessarily the entire $1,000,000 dollars.

28. (From “The Basics of Practical Optimization,” by Adam B. Levy) A violinmaker expects the following demands for violins: 5 violins in the first quarter,4 violins in the second quarter, 4 violins in the third quarter, 3 violins in thefourth quarter. There is an inventory of 6 violins at the beginning of the firstquarter. During each quarter, the violin maker can produce 2 violins at a costof $200 per violin, and can use overtime employees to make additional violinsif necessary, but at a cost of $300 per violin.

Write and solve an optimization model to answer the following question: If, atthe end of each quarter, a holding cost of $25 per violin is incurred for violinsnot sold, how should the violin maker schedule production in order to minimizethe sum of the production and inventory costs during the four quarters whilemeeting the expected demand?

29. (From “The Basics of Practical Optimization,” by Adam B. Levy) An au-tomaker advertises with commercial spots on three types of TV programs: dra-mas (costing $150,000 per minute and seen by 8 million women and 4 millionmen), sports (costing $180,000 per minute and seen by 2 million women and12 million men), and news (costing $130,000 per minute and seen by 4 millionwomen and 5 million men).

(a) Write and solve an optimization model giving a strategy to reach at least35 million women and 35 million men at least cost.

(b) Now do the same under the further stipulation that only one-minute-longcommercial spots can be used.

30. Reformulate the Pit Mine problem under the assumption that blocks can befractionally extracted, except for block 9 which must be completely extractedif at all.

31. (From “The Basics of Practical Optimization,” by Adam B. Levy) Five fast-food workers are to be assigned to five different stations, the table below givesthe productivity on a scale of 1-10 of each worker at each station.


Worker Fries Burgers Ice Cream Register Drive-thru

1 1 6 6 5 9

2 3 6 9 4 10

3 2 5 6 3 6

4 3 6 6 2 7

5 4 7 7 2 8

Write and solve an optimization model to answer the following question: If thetotal productivity of the restaurant is the sum of the productivities of the work-ers assigned to the stations, what assignment of workers to stations maximizestotal productivity?

32. (From “The Basics of Practical Optimization,” by Adam B. Levy) An onlineDVD rental company would like to measure the efficiency of five of its distribu-tion centers in New England: one each in Maine, Vermont, and New Hampshire,and two in Massachusetts. This is done via “data envelopment analysis,” whichcompares ratios of weighted sums of outputs for a center to weighted sums ofinputs for the same center. The efficiency of any distribution center is defined tobe the maximum ratio over all choices of (nonnegative) weights for which noneof the corresponding five rations (one for each center) exceeds 1. The tablebelow shows the average daily inputs and outputs of each distribution center.

Inputs ME VT NH MA 1 MA 2

Value of DVD stock (in $1000’s) 1200 1100 1300 1500 1400

Wages $2,000 $1,580 $2,100 $3,800 $4,000

Outputs ME VT NH MA 1 MA2

Deliveries 1100 950 1120 1220 1190

Restock requests 100 70 150 210 230

Write and solve five optimization models to find the efficiency of each of thefive distribution centers.

33. (From “The Basics of Practical Optimization,” by Adam B. Levy) A woodenpuzzle manufacturer uses four main processes to produce its finished puzzles:wood preparation, painting, cutting, and sanding. The manufacturer uses thesame four processing stations to make five different puzzles, whose respectivetimes for each state of processing are given in the table below. Once a puzzlehas entered the sequence of processing stations, it cannot be “passed” by apuzzle which entered the sequence later, and puzzles can wait between stations.


Puzzle Type 1 2 3 4 5

Wood preparation 4 3 2 3 3

Painting 7 8 6 9 7

Cutting 5 5 6 4 4

Sanding 3 2 3 1 3

Write and solve an optimization model to find the sequence of processing whichminimizes the total time to complete exactly one of each type of puzzle.

34. You are considering which of ten acquaintances to invite to a party: Abdul,Baji, Carlos, Daniel, Esther, Fouzia, Goro, Hemene, Ingrid and Jie. Write andsolve an optimization model that maximizes the number of guests under thefollowing conditions.

(a) At least two women (Baji, Esther, Fouzia, Ingrid, Jie) and at least twomen (Abdul, Carlos, Daniel, Goro, Hemene) should be invited.

(b) Fouzia and Daniel should not both be invited.

(c) If Abdul is invited then Jie should also be invited.

(d) If Baji and Daniel are invited then Hemene should also be invited.

(e) If Esther and Jie are both invited then neither Goro nor Abdul should beinvited.

35. You are trying to decide which of seven possible classes to take next semester.Your choices (i), their credit load (ci), and the grade points per credit youthink you can achieve (gi) are given in the table below. Write and solve anoptimization model that can be used to maximize the number of credits you canearn while keeping your grade point average (GPA) not below 3.5. Furthermore,if you take class 3 you must also take class 5.

class (i) credits (ci) grade points (gi)

1 3 4.0

2 3 3.3

3 4 4.0

4 4 3.0

5 3 2.7

6 3 3.7

7 4 3.0


36. A producer of trail mix has year-end left-over stock of peanuts, chocolate,raisins, pretzels and dried papayas. They would like to make and sell a hol-iday mix from this stock. The goal is to create a mix of these ingredients thathas as close to 240 calories per bag as possible. Processing machinery dispenseseach ingredient in integer unit amounts. The following table describes the stock.

Item Weight Calories

(grams per unit) (calories per unit)

peanuts 6 35

chocolate 5 27

raisins 7 21

pretzels 4 15

papaya 8 4

Any optimal recipe for trail mix must contain at least one unit of each of thefive items. The total bag weight cannot exceed 57 grams (about 2 oz.). Writeand solve an optimizatin model whose solution provides the optimal trail mixrecipe.

37. A breakfast cereal manufacturer wishes to make a new product from some com-bination of the five ingredients: oats, wheat, flax, honey and blueberries. Thecompany machinery dispenses these ingredients in unit quantities into a largemixing vat. Oats, wheat and berries are dispensed in units of cups; honey andflax are dispensed in units of tablespoons. The table shows the calories con-tained in one unit of each ingredient. Write and solve an optimization modelto find the mixture of ingredients that most closely provides a mixture of 2500calories under the additional constraints that (a) at least on unit of each ingre-dient must be used, (b) no more than 7 units of any ingredient can be used and(c) the total volume of ingredients cannot exceed 9 cups.

oats honey wheat flax blueberries

unit 1 cup 1 tbsp 1 cup 1 tbsp 1 cup

calories 158 64 651 55 85

38. A widget manufacturer ships 12,000 widgets to customers every month. Widgetsmay be puroduced at three different factories. The production capacity, costper widget and fixed cost for factory operation are given in the table below.


Factory Fixed Cost Widget Cost Production Capacity

($) ($/widget) (# widgets)

1 80,000 20 6000

2 40,000 25 7000

3 30,000 30 6000

The fixed cost for any factory is only incurred if that factory makes widgets.Write and solve an optimization model whose solution provides the optimalstrategy for minimizing production cost while meeting monthy customer de-mand.

39. A farmer is planning an orchard of mixed apple, pear and cherry trees that canhold a maximum of 350 trees. The seasonal cost of labor per tree is $150, $200and $250, respectively. The seasonal cost of materials per tree is $275, $175 and$125, respectively. Planting logistics require that if any apple trees are plantedthen at least 150 must be planted. Similar limits on pear and cherry trees are50 and 80, respectively. Write and solve an optimization model that maximizesthe number of trees which the farmer can plant when labor costs are limited to amaximum of $50,000 and materials costs are limited to a maximum of $60,000.

Chapter 12

Geometry of Standard Form

We consider the geometry, and some related linear algebra, of standard form linearprograms. Our focus is on understanding the nature of optimal solutions, which inturn will lead to the Simplex Method Algorithm for solving linear programs. Considerthe standard form linear program

minx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ Rn

where A is an m×n matrix with full rank. This rank condition ensures that each con-straint supplies new restrictions relative to any combination of the other constraints.We know that every LP can be reformulated into a LP in standard form in which thereare at least as many decision variables as constraints n ≥ m, so that rank(A) = m(See Exercise 6). There are two cases to consider: n = m and n > m.

1. First, suppose n = m. The constraint set Ax = b is a system of m = n equationswith n unknowns. Because A is full rank, it is a square invertible matrix andthe system of equations has the unique solution x = A−1b. If this solution alsosatisfies the sign constraints x ≥ 0 then it is also the optimal solution to thelinear program: x∗ = A−1b, z∗ = cTx∗. However, if this unique solution doesnot satisfy the sign constraints, then the linear program is infeasible.

151

152 CHAPTER 12. GEOMETRY OF STANDARD FORM

Example 1a. The standard form linear program

maxx

z = x1 + 2x2

s.t. + x1 + x2 = 4

− x1 + x2 = 2

x ≥ 0

x ∈ R2

has the unique optimal solution x∗ = (1, 3), z∗ = 7.

Example 1b. The standard form linear program

maxx

z = x1 + 2x2

s.t. + x1 + x2 = 4

− x1 + x2 = 6

x ≥ 0

x ∈ R2

is infeasible because the unique solution for the equality constraints is x =(−1, 5) which does not satisfy x ≥ 0.

2. Next, suppose n > m. The constraint set Ax = b is a system of m < n equationsin n unknowns. The rank condition guarantees that a solution exists. This isan underdetermined system of equations. The solution set can be written interms of n−m free variables. If this solution set intesects the positive orthantin Rn (x ≥ 0), then the linear program is feasible.

Example 2a. The standard form linear program

maxx

z = x1 + 2x2

s.t. + x1 + x2 = 4

x ≥ 0

x ∈ R2

has a feasible region which is the line segment connecting the two points (0, 4)and (4, 0). This feasible region is bounded and an optimal solution exists.

12.1. AN EXAMPLE IN TWO VARIABLES 153

Example 2b. The standard form linear program

maxx

z = x1 + 2x2

s.t. + 2x1 + x2 = −2

x ≥ 0

x ∈ R2

is infeasible because the solution set to 2x1 + x2 = −2 does not intersect thepositive quadrant, x ≥ 0.

12.1 An Example in Two Variables

Example. Consider the following linear program and corresponding plot of thefeasible region.

maxx

z = x1 + 2x2

s.t. x1 + x2 ≤ 2

x ≥ 0

x ∈ R2 1 2

1

2

Ω

x1 + x2 = 2

(a)(b)

(c)

x1

x2

The feasible region Ω is bounded by the three constraints x1 + x2 ≤ 2, x1 ≥ 0and x2 ≥ 0. The feasible region is equivalently defined by the three vertices of thepolyhedron, (a) = (0, 0), (b) = (2, 0) and (c) = (0, 2). The solution, found using themethod of vertex enumeration, is shown in the table below.

vertex coordinates z optimal?

(a) (0,0) 0 no

(b) (2,0) 2 no

(c) (0,2) 4 yes

x∗ = (0, 2), z∗ = 4

Now, consider the equivalent three-variable standard form problem and corre-sponding graphical representation.


maxx

z = x1 + 2x2

s.t. x1 + x2 + x3 = 2

x ≥ 0

x ∈ R3

1

2

12

1

2

(b)

(c)

(d)x3

x1

x2

The feasible region is the surface bounded by the triangle with vertices (b), (c)and (d). We can solve this LP using the method of vertex enumeration:

vertex coordinates z optimal?

(b) (2,0,0) 2 no

(c) (0,2,0) 4 yes

(d) (0,0,2) 2 no

x∗ = (0, 2, 0), z∗ = 4

Both the inequality and standard form problems yield the same solution in the com-mon decision variables and in objective function.

The standard form LP has n = 3 decision variables and m = 1 equality constraint.The equality constraints define a hyperplane of dimension n−m = 2 and the feasibleregion is the intersection of this plane with the postive octant of R3. In general, wehave the following result.

Remark 25. Given a standard form linear program with n decision variablesand m equality constraints.

1. The solution set of the equality constraints is a hyperplane in Rn of di-mension dH = n−m.

2. The feasible region Ω of the LP has dimension dΩ ≤ n−m.

3. Every vertex of Ω is a vector with at least n−m zero coordinate entries.

We would like to justify the three statements in the above remark.

1. Consider the matrix equation Ax = b, a set of m equations in n unknowns,where A has rank m. We see that A has full rank and n ≥ m. So, the solution

12.2. AN EXAMPLE IN THREE VARIABLES 155

set X = x ∈ Rn | Ax = b can be described using n−m free variables. Thatis, X has dimension n−m.

2. The feasible region is the intersection of X with the positive orthant of Rn,O = x ∈ Rn | x ≥ 0. That is, Ω = X ∩ O. The dimesion of Ω cannot begreater than the dimensions of either X orO. Thus, dim(Ω) ≤ dim(X) = n−m.Specific examples are the subject of Exercise 3.

3. We have dim(X) = n −m. A vertex of Ω is in X and must be defined by theintersection of X with at least n −m additional hyperplanes. The only otherhyperplanes defining the feasible region are those given by the boundaries ofthe inequality constraints x = 0. Thus, at least n −m coordinate entries in avertex vector must equal zero.

12.2 An Example in Three Variables

We need to better understand how the vertices of the feasible region Ω are relatedto the solution set X of Ax = b. We know that the vertices must have a sufficientnumber of zero entries; does any choice of n−m zero entries suffice? To explore thisquestion, consider the four variable standard form LP:

maxx

z = x1 + 2x2

s.t. x1 + x2 + x3 = 4

− x1 + x2 + x4 = 1

x ≥ 0

x ∈ R4

which we can write in matrix form:

maxx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ R4

x =[x1 x2 x3 x4

]Tc =

[1 2 0 0

]TA =

1 1 1 0

−1 1 0 1

b =

[4 1

]T.

In this example, n = 4 and m = 2, and A has rank 2. So, X is a n − m = 2dimensional hyperplane in R4. The intersection of X with the positive orthant is


represented in the figure below. The feasible region is the polygon outlined in bluewith the four vertices:

(a) = (0, 0, 4, 1)

(b) = (0, 1, 3, 0)

(c) = (32, 5

2, 0, 0)

(d) = (4, 0, 0, 5)

The feasible region is represented in R3 with the fourth coordinate value indicated asred dashed parallel level lines on the polygon.

12

5

1 2

5

1

2

5

(a)

(b)

(c)

(d)

x4 = 1

x4 = 2

x4 = 3

x3

x1

x2

Now that we have established the particular geometry of the feasible region, in-cluding the vertices, we turn to the general solutions of Ax = b. One method is to rowreduce the associated augmented matrix [A|b] and write the solution in parametricform in terms of free variables.

[A|b] =

1 1 1 0

−1 1 0 1

∣∣∣∣∣∣ 4

1

∼ 1 0 1

2−1

2

0 1 12

12

∣∣∣∣∣∣32

52

The solution set can be written in terms of the free variables x3 and x4:

x1 = 32− 1

2x3 + 1

2x4,

x2 = 52− 1

2x3 − 1

2x4.

12.2. AN EXAMPLE IN THREE VARIABLES 157

The solution set written in the equivalent parametric vector form is

x =

x1

x2

x3

x4

=

32

52

0

0

+

−1

2

−12

1

0

x3 +

12

−12

0

1

x4.

Notice that because the rank of A is m, we have exactly n −m(= 2) free variables.Interestingly, if we set the free variables x3 = x4 = 0, we recover a vertex of thefeasible region of the linear program:

x = (32, 5

2, 0, 0) = (c).

This result is also clear in the figure, where vertex (c) lies in the x1x2 plane whereboth x3 = 0 and x4 = 0. A study of the figure also reveals that each of the verticeslies in a coordinate plane in which two variables have value zero. These results areshown in the following table.

vertex coordinates zero variables

(a) (0, 0, 4, 1) x1, x2

(b) (0, 1, 3, 0) x1, x4

(c) (32, 5

2, 0, 0) x3, x4

(d) (4, 0, 0, 5) x2, x3

The reason that each vertex must have at least n−m zeros can be described geomet-rically. The solution set to the equality constraints has dimension n−m – the numberof free variables. Reduction of the set to a single point requires the intersection of anadditional n−m planes. These planes must be a subset of the n inequality constraintsx ≥ 0. That is, n−m of the xk must equal zero.

This result suggests a method for finding the vertices of the feasible region for anystandard form linear program. We can consider all combinations of n−m variables,set them to zero, and solve for the remaining variables. The set of vertices will be asubset of these

(nm

)points (some points are not vertices because they are infeasible).

This method should sound familiar – it is, in fact, the vertex enumeration methodfor standard form linear programs. The one serious drawback is that the number ofpotential vertices can be extremely large for problems of even modest size.

The Simplex Method is introduced in the next section. This method circumventsthe combinatorial problem of vertex enumeration by performing a clever vertex searchstrategy that guarantees that an optimal vertex will be found (if one exists), and doesso in very short time. In practice, the Simplex Method only performs ∼ 3m vertexchecks.


12.3 Summary of Basic Solutions

We now present a series of general definitions and facts illustrated using the four-variable example of the previous subsection.

Definition 23. Let Ax = b be a set of m equalities in n ≥ m unknowns. Abasic solution y of Ax = b is one in which n−m entries in y are set to zero.

Remark 26. Let Ax = b be a set of m equalities in n ≥ m unknowns. Ax = bhas

(n

n−m

), not necessarily unique, basic solutions.

The example problem has(

nn−m

)=(

42

)= 6 basic solutions. They are:

vertex coordinates zero variables

(a) (0, 0, 4, 1) x1, x2

(0, 4, 0,−3) x1, x3

(b) (0, 1, 3, 0) x1, x4

(d) (4, 0, 0, 5) x2, x3

(−1, 0, 5, 0) x2, x4

(c) (32, 5

2, 0, 0) x3, x4

Definition 24. A basic feasible solution of a linear program in standard form isa non-negative solution which is also a basic solution for the equality constraints.

The example problem has four basic feasible solutions – the vertices of the feasibleregion.

Exercises

1. Give an example of a two-variable standard form linear program that is feasiblebut has no solution.

2. Give an example of a two-variable standard form linear program that has oneequality constraint and feasible region consisting of exactly one point.

12.3. SUMMARY OF BASIC SOLUTIONS 159

3. Consider Remark 25 Give examples of standard form LPs whose feasible regionshave dimension

(a) dΩ = n−m− 1

(b) dΩ = n−m− 2

(c) dΩ = 0

4. How is the rank condition for standard form LPs interpreted geomerically inRn?

5. Suppose every basic solution of a LP is also a basic feasible solution. Whatmust be true about the equality constraints Ax = b?

6. Show that the standard form representation of any linear program has at leastas many decision variables as equality constraints. To do this, begin with thegeneral form of a linear program given at the beginning of Section 5 and convertto standard form (See Definition 11).

Chapter 13

The Simplex Method

This example illustrates the Simplex Method for solving linear programs. The in-tuitive approach taken here is intended to illuminate understanding of the process.Consider the following LP and feasile region plot.

maxx

z = x1 + x2

s.t. − x1 + x2 ≤ 2

x1 − x2 ≤ 3

2x1 + 3x2 ≤ 16

x ≥ 0

x ∈ R2

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

Ω

x1

x2

We begin by reformulating the linear program in to an equivalent standard formLP. In this case, we need only add three slack variables, x3, x4, x5.

maxx

z = x1 +x2

s.t. −x1 +x2 +x3 = 2

x1 −x2 +x4 = 3

2x1 +3x2 +x5 = 16

x ≥ 0

x ∈ R5

This LP has n = 5 decision variables and m = 3 constraints. We can write this LPin matrix form as

161

162 CHAPTER 13. THE SIMPLEX METHOD

maxx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ R5

x =[x1 x2 x3 x4 x5

]T

c =[1 1 0 0 0

]T

A =

−1 1 1 0 0

1 −1 0 1 0

2 3 0 0 1

b =

[2 3 16

]TA vertex enumeration method would require us to check the feasibility and objec-

tive value at(nm

)= 10 vertices (basic solutions). The Simplex Method seeks to reduce

the number of vertices that one must check in order to find an optimal solution.

Remark 27. Every bfs of a SFLP corresponds to exactly one vertex of thefeasible region. Every vertex of the feasible region of a SFLP corresponds to atleast one bfs.

So, the method of complete enumeration for solving a SFLP amounts to finding allof the basic solutions and retaining the subset which are both feasible and optimizethe objective function. The Simplex Method is “simply” a very efficient method forsearching the collection of vertices.

Before considering our fist exmaple, we need to introduce some notation and makesome observations.

Definition 25. A basis for a SFLP is a set of m distinct indices B ⊂1, 2, . . . , n. The variables with indices from B are called basic variables.The remaining variables, with indices N = 1, 2, . . . , n \B, are callednon-basic variables. We write xB andxN to indicate the vectors containingonly the basic and non-basic variables, respectively.

Vertices of a feasible region, bases, and basic feasible solutions are related in thefollowing ways.

Definition 26. We say that two bases B1 and B2 are adjacent if they differ inonly one index. We also say that two vertices of the feasible region are adjacentif they have associated bases which are adjacent.

13.1. BASIC SIMPLEX METHOD 163

Remark 28. If two vertices are adjacent, then they are either the same pointin Rn or they are connected by an edge of the feasible region.

13.1 Basic Simplex Method

We now employ the Simplex Method to find an optimal solution. In this first exam-ple, we show the basic method only, leaving a variety of intereting details for laterdiscussion and practice.

The Simplex Method begins with any basic feasible solution and steps sequentiallyfrom vertex to adjacent vertex, at each step guaranteeing that the objective functionis non-decreasing (for a max problem), until an optimal vertex is found. In practice,the number of visited vertices is significantly fewer than the total number of basicsolutions. The number of basic solutions can be as many as

(nm

)= n!

m! (n−m)!, whereas

for a typical SFLP the number of vertices visited is on the order of 3m.

(1) Begin with any basic feasible solution.

Often, this first step is difficult because we may not know whether or not ourLP is feasible, and even if it is, we may have a difficult time finding a bfs. Wewill differ this discussion for later.

The recommended practice is to first try to set B to include the set of slackvariables. In our example, we try B = 3, 4, 5, N = 1, 2. We notice quickly,

that this basis corresponds to the basic solution x =[0 0 2 3 16

]Twhich

is also a bfs because x ≥ 0. So we have our initial bfs, with current basisB = 3, 4, 5.

(2) Build the Simplex Tableau corresponding to the current basis.

The Simplex Tableau is just rearrangement of the equality constraints and ob-jective function as follows.

x3 = 2 +x1 −x2

x4 = 3 −x1 +x2

x5 = 16 −2x1 −3x2

z = 0 +x1 +x2


Every basic variable is written in terms of the nonbasic variables using theequality consraints. The objective is also written in terms of the nonbasicvariables. This tableau is equivalent (contains the same information) to thebasis B = 3, 4, 5 applied to the SFLP. In fact, we can easily extract thelocation of the vertex which corresponds to this tableau. At the current bfs,xN = 0 (x1 = x2 = 0), and therefore x3 = 2, x4 = 3 and x5 = 16. In terms ofthe original two variables (x1, x2) = (0, 0), so this bfs corresponds to the point(0, 0) in the plot on the first page of this section. We also have the objectivevalue at this bfs, z = 0.

(3) Select a nonbasic variable to enter the basis, if possible.

Adjacent bases differ in only one index in the basis, and equivalently, adjecentvertices differ in one basic variable. So, in starting from a current bfs and“traveling” to an adjacent bfs we first choose a nonbasic variable to enter thenew basis. When it enters the basis, it will change from a value of zero toa positive value (no longer a nonbasic variable). Since there are often manynonbasic variables, we choose one for which the objective function increases asthis change is made. In our example, we can choose x1 to enter the basis becauseas x1 increasese, so does z. We can see this from the last line of the tableau.We can also choose x2 to increase for the same reason. We can choose either– the only condition is that z increase. For this example, we will arbitrarilychoose x1.

(4) Determine which basic variable will exit the basis, if any.

We now must understand which basic variable is going to change from a positivevalue to zero (become a nonbasic variable). Consider the current tableau. Wemake the following observations about how each basic variable changes as x1

increases.

(a) x3 is never reduced to a value of zero as x1 increaeses.

(b) x4 is reduced to a value of zero when x1 = 3.

(c) x5 is reduced to a value of zero when x1 = 8.

We can only allow x1 to increase to a value of 3 because this is the mostrestrictive condition on the increase in x1. Increasing x1 farther would makebasic variable x5 < 0, an infeasible solution. So, we see that x4 will exit thebasis, taking on a new value of zero.

Sometimes there will be more than one choice of exiting variable. This canhappen when there are two or more equally most-restrictive conditions. Whenthis occurs, choose arbitrarily.

(5) Create the new basis and go to Step (2).


The old basis is B = 3, 4, 5 and the new basis is B = 3, 1, 5. Now we returnto Step (3) and repeat this process. Steps (2)–(5) describe what is known as aSimplex Pivot.


As before, the Simplex Tableau is an arrangement of equality constraints andobjective function in which the basic variables are written in terms of the non-basic variables. The new Simplex Tableau is obtained from the previous tableauby substitution. The equation relating the entering and exiting variables (x1

and x4) is simply inverted:

x4 = 3− x1 + x2 becomes x1 = 3− x4 + x2.

Then, this last equation can be used to eliminate x1 from all other equalitiesand the objective. We have

x3 = 2 + x1 − x2

= 2 + (3− x4 + x2)− x2

= 5− x4

and

x5 = 16− 2x1 − 3x2

= 16− 2(3− x4 + x2)− 3x2

= 10 + 2x4 − 5x2

and

z = 0 + x1 + x2

= 0 + (3− x4 + x2) + x2

= 3− x4 + 2x2

Collecting these results, yields the following new Simplex Tableau.

x3 = 5 −x4

x1 = 3 −x4 +x2

x5 = 10 +2x4 −5x2

z = 3 −x4 +2x2


This new basis and new tableau correspond to the bfs x =[3 0 5 0 10

]T.

This bfs is the vertex (x1, x2) = (3, 0) in the plot on the first page. The pivotwe have performed corresponds to moving from bfs (0, 0) to bfs (3, 0) along thefeasible region edge connecting them.


The only nonbasic variable which can increase z, as it increases, is x2. So wemust choose x2 to enter the basis.


We observe the following effects on the current basic variables as the nonbasicvariable x2 increases.

(a) x3 remains at value 5 as x2 increaeses.

(b) x1 is never reduced to a value of zero when x2 increases.

(c) x5 is reduced to a value of zero when x2 = 2.

Thus, we recognize that x5 will exit the basis.


The new basis is B = 3, 1, 2. Variable x5 exited the basis and variable x2

entered the basis.


The equation relating the entering and exiting variables (x2 and x5) is simplyinverted:

x5 = 10 + 2x4 − 5x2 becomes x2 = 2 + 25x4 − 1

5x5.

Then, this last equation can be used to eliminate x2 from all other equalitiesand the objective. We have

x3 = 5− x4 (remains unchanged)

and

x1 = 3− x4 + x2

= 3− x4 + (2 + 25x4 − 1

5x5)

= 5− 35x4 − 1

5x5


and

z = 3− x4 + 2x2

= 3− x4 + 2(2 + 25x4 − 1

5x5)

= 7− 15x4 − 2

5x5

Collecting these results, yields the following new Simplex Tableau.

x3= 5 −x4

x1= 5 −35x4 −1

5x5

x2= 2 +25x4 −1

5x5

z= 7 −15x4 −2

5x5

This new basis and new tableau correspond to the bfs x =[5 2 5 0 0

]T.

This bfs is the vertex (x1, x2) = (5, 2) in the plot on the first page. The pivotwe have performed corresponds to moving from bfs (3, 0) to bfs (5, 2) along thefeasible region edge connecting them.


Notice that, in the new tableau, neither x4 nor x5 can be chosen to enter thebasis because if either is chosen, then z will decrease. It is not possible to selectan appropriate nonbasic variable. Thus, the current basis must be optimal.

Definition 27. An optimal basis is a basis for which the corresponding basicfeasible solution is optimal for the associated SFLP.

The solution to the SFLP is x∗ = [5 2 5 0 0]T , z∗ = 7 and the solution to theorignal LP is x∗ = [5 2]T , z∗ = 7.

Remark 29. If a Simplex Tableau for a maximization problem has strictlynegtive coefficients on all nonbasic variables in the objective function, then thecurrent basic feasible solution is the unique optimal solution of the LP.

This first example of the Simplex Method produced an optimal solution and ter-minated nicely. There are several other possibilities which we must consider carefully,leading to necessary modifications of the Simplex Method. These exceptions are con-sidered in the next several subsections.


13.2 Unbounded Linear Programs

A given LP may be unbounded. Recall that this means not only is the feasible regionunbounded, but the objective can increase without bound within this feasible re-gion. We must understand how to recognize unbounded problems within the SimplexMethod. Consider the following linear program.

maxx

z = x1 + 2x2

s.t. − x1 + x2 ≤ 2

x1 − 2x2 ≤ 1

x ≥ 0

x ∈ R2

1 2 3 4

1

2

3

4

Ω

x1

x2

The feasible region is unbounded and the objective value can be arbitrarily largefor some feasible point. So, the LP is itself unbounded and has no solution. Theequivalent standard form LP is

maxx

z = +x1 +2x2

s.t. −x1 +x2 +x3 = 2

+x1 −2x2 +x4 = 1

x ≥ 0

x ∈ R4

Before we employ, the Simplex Method, first consider general tests for unboundedness.

Definition 28. The feasible region Ω of a Standard Form Linear Program issaid to be unbounded if there exists x ∈ Ω and nonzero v ∈ Rn, satisfying v ≥ 0and Av = 0. Such a vector v is said to be an unbounded direction for feasibleregion Ω.

This definition states necessary and sufficient conditions for x + αv ∈ Ω for allα ≥ 0. That is, ‖x+ αv‖ can be arbitrarily large and remain feasible. We make twoobservations.

1. Since x is feasible, x ≥ 0 and Ax = b.

13.2. UNBOUNDED LINEAR PROGRAMS 169

2. Since v ≥ 0 and α ≥ 0 we have (i) x+αv ≥ 0 and (ii) A(x+αv) = Ax+αAv =b+ 0 = b. That is, x+ αv is feasible.

Definition 29. A Standard Form (maximization) Linear Program is said to beunbounded if there exists x ∈ Ω and nonzero v ∈ Rn, satisfying v ≥ 0, Av = 0and cTv > 0.

This definition says that a SFLP is unbounded if the feasible region is unboundedand there the unbounded direction vector points in a direction of increasing objectivevalue (for a maximization problem). Notice that x + αv is feasible for all α > 0 andlimα→∞ c

T (x+ αv) = cTx+ cTv (limα→∞ α) =∞.For the current example, we can choose x = (0, 0, 1, 1) and v = (1, 1, 0, 1) 6=

(0, 0, 0, 0). Notice that Ax = b, x ≥ 0, v ≥ 0, Av = 0 and cTv > 0. Thus, the LP isunbounded. We can verify this result using the Simplex Method.

(1) Begin with any basic feasible solution. As in the basic example, we can beginwith the basis that includes only the indices of the slack variables, B = 3, 4.

(2) Build the Simplex Tableau corresponding to the current basis. We have

x3 = 2 +x1 −x2

x4 = 1 −x1 +2x2

z = 0 +x1 +2x2


We choose variable x1 to enter the basis because in doing so, z wil lincrease.


The most limiting condition on increase in x1 is from the second constraint. So,x4 will exit the basis.


The new basis is B = 3, 1. Variable x4 exited the basis and variable x1 enteredthe basis.



x3 = 3 −x4 +x2

x1 = 1 −x4 +4x2

z = 0 −x4 +4x2


The only choice for entering variable is x2 because this is the only variable thatcan increase the objective value.


As x2 increases neither x3 nor x1 will be reduced to zero. Instead they increasewithout bound. So, there is no limiting condition on which to base a choise ofexiting variable. As x2 increases, z increases and x remains feasible. Thus, theLP is unbounded and no solution exists.

Remark 30. If a Simplex Tableau for a maximization problem has a nonba-sic variable with positive coefficient in the objective function and positive co-efficients on the same nonbasic variable in the constraint equations, then thecorresponding LP is unbounded and no solution exists.

13.3 Multiple Optimal Solutions

A given LP may have multiple optimal solutions. In this case, more than one vertexof the feasible region (and therefore, more than one basis) may be optimal. We willconsider how to recognize when a SFLP has multiple optimal vertices. Consider thefollowing linear program.

maxx

z = −12x1 + x2

s.t. − x1 + 2x2 ≤ 2

2x1 + x2 ≤ 6

x ≥ 0

x ∈ R2

13.3. MULTIPLE OPTIMAL SOLUTIONS 171

1 2 3 4

1

2

3

4

Ωx1

x2

A quick check of the vertices of Ω reveals that both (0, 1) and (2, 2) are optimalvertices with z∗ = 1. Let’s see how the Simplex Method recognizes multiple optimalvertices and multiple optimal solutions in general. We first formulate the standardform equivalent problem.

maxx

z = −12x1 +x2

s.t. −x1 +2x2 +x3 = 2

+2x1 +x2 +x4 = 6

x ≥ 0

x ∈ R4

Next, employ the Simplex Method.

(1) Begin with any basic feasible solution. As in the basic example, we can beginwith the basis that includes only the indices of the slack variables, B = 3, 4.

(2) Build the Simplex Tableau corresponding to the current basis. We have

x3 = 2 +x1 −2x2

x4 = 6 −2x1 −x2

z = 0 −12x1 +x2


The only choice for entering variable is x2 because this is the only variable thatcan increase the objective value.



As x2 increases, the basic variable which becomes zero first is x3, so, x3 will exitthe basis.


The new basis is B = 2, 4. Variable x3 exited the basis and variable x2 enteredthe basis.


x2 = 1 +12x1 −1

2x3

x4 = 5 −52x1 +1

2x3

z = 1 +0x1 −12x3


Notice that no choice of entering variable can increase the objective value, sothe current basis is optimal. We have the optimal solution x∗ = (0, 1, 0, 5),z∗ = 1.

However, if x1 enters the basis, z will neither increase nor decrease. Let’scontinue the Simplex Method by choosing to enter x1.


As x1 increases, the basic variable which becomes zero first is x4, so, x4 will exitthe basis.


The new basis is B = 2, 1.


x2 = 2 +15x4 −2

5x3

x1 = 2 −25x4 +1

5x3

z = 1 +0x4 −12x3

13.3. MULTIPLE OPTIMAL SOLUTIONS 173


Notice that no choice of entering variable can increase the objective value, sothis new current basis is also optimal. We have a second optimal solutionx∗ = (2, 2, 0, 0), z∗ = 1.

Notice also that if we were to enter x4 into the basis, we would pivot back tothe first optimal solution. In this instance, we have two optimal vertices. Wecan express the complete optimal solution set X as the convex combination ofthese two solutions:

X =x ∈ R4 | x = α(0, 1, 0, 5) + (1− α)(2, 2, 0, 0), 0 ≤ α ≤ 1

,

or equivalently,

X =x ∈ R4 | x = (2− 2α, 2− α, 0, 5α), 0 ≤ α ≤ 1

.

It is possible to enumerate all optimal vertex solutions through careful pivots. Inthe general case of m optimal vertex solutions v1, v2, . . . , vm we have the optimalsolution set

X =

x ∈ Rn | x =

m∑k=1

αkvk,m∑k=1

αk = 1, αk ≥ 0 for k = 1, 2, . . . ,m

.

If we let V be the matrix with columns given by v1, v2, . . ., vm and α ∈ Rm the vectorof scalar weights, then we can write

X =x ∈ Rn | x = V α, αT1m = 1, α ≥ 0

.

These types of set descriptions are accurate as long as all optimal pivots have beenenumerated.

Remark 31. If a Simplex Tableau has a nonbasic variable with zero coefficientin the objective function, then the LP may have multiple optimal solutions. Ifthe corresponding pivot step length is (a) zero, then the pivot leads to a newbasis with the same basic feasible solution, (b) othewise finite, then the pivotleads to a distinct optimal basic feasible solution, or (c) infinite, then the pivotis unbounded and multiple optimal solutions exist along the ray defined by thisunbounded step.

The last example LP illustrated case (b) from the remark above. Case (a) will beexplored in more depth later. Case (c) cannot be represented using the optimal setdescription given above.


13.4 Recognizing Infeasible Linear Programs

A given LP may be infeasible. Notice that the Simplex Method begins with “Beginwith any basic feasible solution,” but we may be unable to find one easily or nonemay exist. We will explore methods which find a bfs if it exists.

Remark 32. Every Standard Form Linear Program with a nonempty feasibleregion has at least one basic feasible solution.

By the remark above, if no bfs exists then the SFLP is infeasible. Consider thefollowing example LP.

maxx

z = x1 + 2x2

s.t. x1 + x2 ≥ 1

x1 + x2 ≤ 3

x ≥ 0

x ∈ R2

1 2 3 4

1

2

3

4

Ω

Ω

x1

x2

The feasible region does not include the origin in R2. This is problematic forthe Simplex Method (as described thus far). Let’s see why. First, we formulate theequivalent standard form problem.

maxx

z = +x1 +2x2

s.t. +x1 +x2 −x3 = 1

+x1 +x2 +x4 = 3

x ≥ 0

x ∈ R4

Now, if we choose an initial basis with slack variables chosen to be the initial nonbasicvariables, we arrive at the following Simplex Tableau.

13.4. RECOGNIZING INFEASIBLE LINEAR PROGRAMS 175

x3 = −1 −x1 −x2

x4 = +3 −x1 −x2

z = 0 +x1 +2x2

This tableau corresponds to the basic solution x = (0, 0,−1, 3) which is not a basicfeasible solution. The Simplex Method requires us to start with a basic feasiblesolution. So what do we do? At this point, we cannot even be sure that a basicfeasible solution exists. If one does exist, then we need to find it. If one does notexist, then we need to know this fact. We outline two methods for solving potentiallyinfeasible problems: the Big-M Method and the Two-Step Simplex Method.

Big-M Method

Notice that the source of the infeasibility in our choice of basis is in the first equalityconstraint. We see that ther are opposite signs on the coefficient on the slack variable(negative) and the right-hand-side value (positive). We can temporarily remedy thissituation by adding a new non-negative slack variable x5, giving the new constraints

x1 + x2 − x3 + x5 = 1, x5 ≥ 0.

Unlike other slack variable transformations we have made, this new constraint set isnot equivalent to the original constraint x1 + x2 − x3 = 1. The key idea here is thatif, at an optimal solution, x∗5 = 0, then the remaining variables represent an optimalsolution to the original SFLP. So, it is not enough to simply apply this new constraint,but we must encourage x5 to be zero, if possible, so that the two constraints sets areequivalent for x5 = 0. We can do this through the objective function which we changeto be

z = x1 + 2x2 −Mx5,

where M is a “very large value.” In this way, the Simplex Algorithm will preferentiallymake x5 as small as possible, because it is trying to maximize z. So, without yetspecifying a particular value for M , we have the new SFLP:

maxx

z = +x1 +2x2 −Mx5

s.t. +x1 +x2 −x3 +x5 = 1

+x1 +x2 +x4 = 3

x ≥ 0

x ∈ R5

We can solve this optimization problem with the following interpretations.


1. If at any time x5 becomes a nonbasic variable (x5 = 0), then the current basisis a feasible basis for the original SFLP. We can remove x5 from the problemand continue the Simplex Method.

2. If x∗5 > 0, then no set of variables (x1, x2, x3, x4, 0) satisfy the constraints, be-cause if so the Simplex Method would find such a solution due to the large valueof M . Thus, the original SFLP must be infeasible.

Let’s see how the Big-M Method works on our example. We choose as an initial basisB = 4, 5 and construct the intial Simplex Tableau. In this case, we need to writethe objective function in terms of the nonbasic variables carefully:

z = x1 + 2x2 −Mx5

= x1 + 2x2 −M(1− x1 − x2 + x3)

= −M + (M + 1)x1 + (M + 2)x2 −Mx3

And we have the Tableau:

x4 = 3 −x1 −x2

x5 = 1 −x1 −x2 +x3

z = −M +(M + 1)x1 +(M + 2)x2 −Mx3

Instead of specifying a particular value for M , we use the convention that “M is largerthan any other number to which it is compared.” We then proceed with the SimplexMethod. At the first pivot we can choose to enter x1 into the basis and exit x5 fromthe basis. This leads to the new basis B = 4, 1 and the new Tableau:

x4 = 2 +x5 −x3

x1 = 1 −x5 −x2 +x3

z = 1 −(M + 1)x5 +x2 +x3

Now x5 is a nonbasic variable, so the current basis b = 4, 1 is a feasible basis forthe original SFLP. In fact, the current point in the original two-variable problem is(x1, x2) = (1, 0) which is a vertex of the feasible region (see the graph). We cansimply eliminate x5 from the problem and we have the current Tableau:


x4 = 2 −x3

x1 = 1 −x2 +x3

z = 1 +x2 +x3

The current bfs is not optimal because we can enter x3 into the basis and exit x4 fromthe basis. The new basis is B = 3, 1 and the new Tableau is:

x3 = 2 −x4

x1 = 3 −x2 −x4

z = 3 +x2 −x4

The current bfs is not optimal because we can enter x2 into the basis and exit x1 fromthe basis. The new basis is B = 3, 2 and the new Tableau is:

x3 = 2 −x4

x2 = 3 x1 −x4

z = 6 −x1 −2x4

The current bfs is now optimal. The solution is x∗ = (0, 3, 2, 0), z∗ = 6. In termsof the original problem, x∗ = (0, 3), z∗ = 6. The reader is encouraged to trace theSimplex Algorithm path along the boundary of the feasible region shown in the graph,noting when the current point became feasible.

Two-Step Simplex Method

The Big-M Method is a clean method for solving potentially infeasible linear pro-grams. However, in a computationaly scenario, one must be able to determine asufficiently large value for M without introducing numerical instabilities. Sometimesthis is not a significant issue. But we can avoid this necessity using a similar approachknown as the Two-Step Simplex Method. This method does not rely on a choice ofM , but it does require us to solve an initial auxiliary linear program whose sole jobis to determine either (a) a feasible basis for our LP or (b) that our LP is infeasible.Once a feasible basis is obtained, we can employ the Simplex Method on our original


problem. Thus, we solve two linear programs. Let’s see how this works. The auxiliaryproblem is differs from the Big-M Method problem only in the objective function. Inour example, we consider the auxiliary linear program:

maxx

w = −x5

s.t. +x1 +x2 −x3 +x5 = 1

+x1 +x2 +x4 = 3

x ≥ 0

x ∈ R5

where we add new slack variables as before and consider the new objective functionwhich is the negative sum of the new slack variables. As before, we seek a feasiblebasis for the orignal problem. Such a solution occurs when x5 = 0 for a basic feasiblesolution. The objective function encourages all new slack variables to be as smallas possible. In this case, if x∗5 = 0, then the current basis is a feasible basis for theoriginal SFLP. However, if x∗5 > 0, then the original SFLP is infeasible. Let’s employthe Two-Step Simplex Method on our example problem. We begin by solving thisauxiliary problem using the Simplex Method with initial basis B = 4, 5. The initialTableau is

x4 = 3 −x1 −x2

x5 = 1 −x1 −x2 +x3

w = −1 +x1 +2x2 −x3

We see that w < 0 (x5 > 0) and the current basic feasible solution is not optimal.We can enter x1 into the basis, in which case x5 exits the basis. The new basis isB = 4, 1 with associated tabeau:

x4 = 2 +x5 −x3

x1 = 1 −x5 −x2 +x3

w = 0 −x5 +x2

We see that x5 = 0 so that the current basis is a feasible basis for the original SFLP.We can then remove the x5 dependence from the Tableau, insert the original objectivefunction, and solve. We have

z = x1 + 2x2 = (1− x2 + x3) + 2x2 = 1 + x2 + x3,


and the new tableau:

x4 = 2 −x3

x1 = 1 −x2 +x3

z = 1 +x2 +x3

This tableau is identical to the tableau found using the Big-M Method which was twopivots away from the optimal solution.

As a final example, consider a larger problem solved using the Two-Step SimplexMethod. Solve the following LP.

maxx

z = +x1 +x2 +x3

s.t. +x1 +x2 +x3 ≤ 6

+x1 +4x2 ≥ 4

+x2 +x3 = 2

x ≥ 0

x ∈ R3

First, we reformulate the problem into standard form. In this case, we add two newslack variables x4 and x5, one for each of the inequality constraints.

maxx

z = +x1 +x2 +x3

s.t. +x1 +x2 +x3 +x4 = 6

+x1 +4x2 −x5 = 4

+x2 +x3 = 2

x ≥ 0

x ∈ R5

This LP does not present a quick initial basic feasible solution. If we add two newslack variables, x6 to the second equality and x7 to the third equality, then we canchoose the initial feasible basis B = 4, 6, 7. However, the goal is to maximizethe negative of the sum of the new slack variables (auxiliary variables). Here is the


standard form LP:

maxx

w = −x6 −x7

s.t. +x1 +x2 +x3 +x4 = 6

+x1 +4x2 −x5 +x6 = 4

+x2 +x3 +x7 = 2

x ≥ 0

x ∈ R7

Now, we must solve this auxiliary problem which provides a basic feasible solution tothe original SFLP, if such exists. If we achieve w = 0 and neither x6 nor x7 is in thebasis, then the corresponding basis is an initial basis of the original SFLP. Let’s solvethis auxiliary problem. The initial tableau is:

x4 = 6 −x1 −x2 −x3

x6 = 4 −x1 −4x2 +x5

x7 = 2 −x2 −x3

w = −6 +x1 +5x2 +x3 −x5

Performing the pivot to basis B = 4, 1, 7, we find:

x4 = 2 +x6 +3x2 −x3 −x5

x1 = 4 −x6 −4x2 +x5

x7 = 2 −x2 −x3

w = −2 −x6 +4x2 +x3

Performing the pivot to basis B = 4, 1, 3, we find:

x4 = 0 +x6 +4x2 +x7 −x5

x1 = 4 −x6 −4x2 +x5

x3 = 2 −x2 −x7

w = 0 −x6 +3x2 −x7


Now we have w = 0 with both x6 and x7 not in the basis. So, B = 4, 1, 3 is an initialfeasible basis for our original SFLP. The upper portion of the tableau is exactly thatfor the auxiliary problem with x6 and x7 removed. The objective function is (fromthe original SFLP): z = x1 + x2 + x3 = (4− 4x2 + x5) + x2 + (2− x2) = 6− 4x2 + x5.And we have the tableau:

x4 = 0 +4x2 −x5

x1 = 4 −4x2 +x5

x3 = 2 −x2

z = 6 −4x2 +x5

Now, we can procede with the Simplex Method applied to the problem of interest.Performing the pivot to basis B = 5, 1, 3, we find:

x4 = 0 +4x2 −x4

x1 = 4 −x4

x3 = 2 −x2

z = 6 −x4

The current basic feasible solution is a degenerate optimal point x(1) =[

4 0 2 0 0]T

.

We can perform a pivot to basis B = 5, 1, 2, obtaining:

x5 = 8 −x3 −x4

x1 = 4 −x4

x3 = 2 −x3

z = 6 −x4

The current basic feasible solution is also an optimal point x(2) =[

4 0 2 0 8]T

.

The only valid pivot would return the tableau to the basis B = 5, 1, 3. So, we havethe solution set:

X =αx(1) + (1− α)x(2) | 0 ≤ α ≤ 1

,

z∗ = 6.


13.5 Pivot Rules

The core operation of the Simplex Method is the pivot in which we choose one variableto enter the basis and another to exit the basis. Thus far we have treated all validchoices as equally beneficial. Now we need to take a closer look. Recall that theentering variable is one for which the corresponding coefficient in the objective lineof the tableau is non-negative. And, the exiting variable is one for which the enteringvariable increase is most limited (by feasibility). Either of these choices may notbe unique. Pivot rules are strategies for making these choices in an attempt toimprove computational efficiency: reducing the number of pivots and minimizingcomputational time.

Basic Selection Rules

We first consider some intuitive (and often effective) strategies for choosing pivots.In our discussion, we will consider the following example tableau for a maximiza-

tion problem with objective vector c =[1 1 1 1 1 1 1

]T.

x1 = 6 − x2 + 6x7 − 2x5 + x6

x3 = 2 − 2x2 − 3x7 + 3x5 − 2x6

x4 = 3 + 4x2 − x7 − x5 − x6

z = 11 + 2x2 + 3x7 + x5 − x6

This tableau has associated basis B = 1, 3, 4 and the current basic solution isx = (6, 0, 2, 3, 0, 0, 0) with z = 11. This tableau is not optimal because four validpivots exist which would not decrease the objective value. These pivots are:

(a) Enter x2 and exit x3.

(b) Enter x7 and exit x3.

(c) Enter x5 and exit x1.

(d) Enter x5 and exit x4.

Definition 30. A Simplex pivot is said to be valid if it results in a change fromone feasible basis to an adjacent feasible basis associated with an improved orequal objective value.

We now consider several pivot rules which address the selection of entering variables,exiting variables or both.

13.5. PIVOT RULES 183

Largest Coefficient

When considering a variable to enter the basis, it would seem beneficial to choose thevariable with largest positive coefficient in the objective row of the tableau. With thisselection, the objective value is increasing at the fastest possible rate. In our exampleproblem, this corresponds to the choice of pivot (b) because the coefficient for x7 isthe largest.

This strategy has some drawbacks. It is an incomplete pivot method in that itdoes not address ties, both in entering and exiting variables. This strategy can alsobe very inefficient because athe actual pivot step an be very short, limiting the actualincrease in objective.

Largest Increase

One method for avoiding small-step pivots, which do not significantly increase theobjective, is to choose among all valid pivots the one that increases the objective themost. This choice requires us to compute the new objective value at each potentialnew basis. For each pivot, the objective increase (∆z) is the step length (s) timesthe rate of objective increase (dz/ds). These values are determined using the tableauvalues.

pivot s dzds

∆z = sdzds

(a) 2/2 2 2

(b) 2/3 3 2

(c) 6/2 1 3

(d) 3/1 1 3

The largest possible single-pivot increase in z is 3. We should choose either ofpivots (c) or (d) to achieve this increase. Clearly, this strategy is also incomplete inthat ties are possible. However, it does guarantee that z improves the most a singlestep.

Steepest Edge

We can consider trying to pivot along an edge that is sensitive to the global nature ofthe linear program. Instead of considering optimizing a single step, suppose we pivotalong the edge that most closely aligns with the objective direction c. Suppose we


are considering a pivot from feasible region vertex u to vertex v. The pivot directionis the difference vector d = v − u with step size s = ‖d‖. We consdier the value w ofsuch a step as the magnitude of the projection of the normalized pivot direction ontothe objective vector. That is,

w =cTd

s=cT (v − u)

‖v − u‖.

Then, among all valid pivots, we select the one with the highest value (steepest edgerelative to the objective). For the current example, we have u = (6, 0, 2, 3, 0, 0, 0),c = (1, 1, 1, 1, 1, 1, 1) and

pivot v d s w

(a) (5, 1, 0, 7, 0, 0, 0) (−1, 1,−2, 4, 0, 0, 0)√

22 2√22≈ 0.426

(b) (10, 0, 0, 73, 0, 0, 2

3) (4, 0,−2,−2

3, 0, 0, 2

3) 2

√47

33√47≈ 0.438

(c) (0, 0, 11, 0, 3, 0, 0) (−6, 0, 9,−3, 3, 0, 0)√

135 3√135≈ 0.258

(d) (0, 0, 11, 0, 3, 0, 0) (−6, 0, 9,−3, 3, 0, 0)√

135 3√135≈ 0.258

The steepest edge pivot is pivot (b). This method is also incomplete in that itdoes not account for ties.

Cycling

In addition, because a vertex may correspond to more than one basis, the SimplexMethod may need to make pivots of zero step length (in Rn) in order to make progresspast degenerate vertices.

Definition 31. A degenerate vertex of a system of linear equations Ax = b isa basic solution at which more than n constraints are active.

This results in multiple bases corresponding to the same point in Rn. It twobases correspond to the same feasible point x, then x must have more than n − mzeros because both bases determine n−m zeros which are not all the same. Thus, adegenerate vertex must have at least one basic variable with a value of zero.

Remark 33. Let B be a basis for a SFLP and x the corresponding basic solu-tion. If for some k ∈ B, xk = 0 then x is a degenerate vertex of Ax = b.

It is possible for the basic Simplex Method implementation to get stuck in aninfinite loop of such pivots. Pivot selection should guarantee that progress along thefeasible region edges is eventually made. Consider the optimization problem


maxx

z = 3x4 − 80x5 + 2x6 − 24x7

s.t. 4x1 + x4 − 32x5 − 4x6 + 36x7 = 0

2x2 + x4 − 24x5 − x6 + 6x7 = 0

x3 + x6 = 1

x ≥ 0

x ∈ R7

We will attempt to solve this problem using the Simplex Method and employing theLargest Coefficient pivot rule with any ties broken arbitrarily. An initial basis isreadily available: B = 1, 2, 3. The initial tableau is

x1 = 0 − 14x4 + 8x5 + x6 − 9x7

x2 = 0 − 12x4 + 12x5 + 1

2x6 − 3x7

x3 = 1 − x6

z = 0 + 3x4 − 80x5 + 2x6 − 24x7

The reader should verify that the following tableaus represent subsequent pivotsfrom this initial basis.

x4 = 0 − 4x1 + 32x5 + 4x6 − 36x7

x2 = 0 + 2x1 − 4x5 + 32x6 + 15x7

x3 = 1 − x6

z = 0 − 12x1 + 16x5 + 14x6 − 132x7

x4 = 0 + 12x1 − 8x2 − 8x6 + 84x7

x5 = 0 + 12x1 − 1

4x2 + 3

8x6 + 15

4x7

x3 = 1 − x6

z = 0 − 4x1 − 4x2 + 8x6 − 72x7


x6 = 0 + 32x1 − x2 − 1

8x4 + 21

2x7

x5 = 0 − 116x1 + 1

8x2 + 3

64x4 − 3

16x7

x3 = 1 − 32x1 + x2 − 1

8x4 − 21

2x7

z = 0 + 8x1 − 12x2 − x4 + 12x7

x6 = 0 − 2x1 + 6x2 + 52x4 − 56x5

x7 = 0 − 13x1 + 2

3x2 + 1

4x4 − 16

3x5

x3 = 1 + 2x1 − 6x2 − 52x4 + 56x5

z = 0 + 4x1 − 4x2 + 2x4 − 64x5

x1 = 0 − 12x6 + 3x2 + 5

4x4 − 28x5

x7 = 0 − 16x6 − 1

3x2 − 1

6x4 + 4x5

x3 = 1 − x6

z = 0 +−2x6 + 8x2 + 7x4 − 176x5

x1 = 0 − 14x4 + 8x5 + x6 − 9x7

x2 = 0 − 12x4 + 12x5 + 1

2x6 − 3x7

x3 = 1 − x6

z = 0 + 3x4 − 80x5 + 2x6 − 24x7

Notice that the last tableau is identical to the original tableau. The SimplexMethod completed 5 pivots, returning to the orignal basis. Each of the five distinctfeasible bases corresponded to the single basic feasible solution x = (0, 0, 3, 0, 0, 0, 0)and objective value z = 0. Notice also that this solution is not optimal. This sequenceof pivots is an example of cycling. Certain pivot rules will prevent indefinite cycling.


Pivot Rules that Prevent Cycling

Random Selection

One can also eliminate cycling by randomly breaking ties in both entering and exitingvariable selections.

Minimizing Revisits

One method for preventing cycling is to choose, whenever possible, to enter a variableinto the basis which has been in the basis the least number of times, and breaking tiesby selecting the variable that has been in the basis most recently. In the example ofSection 13.5, the entering variable indices will be (in order) 4, 5, 6, 7, 1, 2 (one completecycle) then 6. This last choice is made because both 4 and 6 have been in the basisan equal number of times and 6 has been in most recently. The resulting tableau is

x1 = 1 − 14x4 + 8x5 + x3 − 9x7

x2 = 12− 1

2x4 + 12x5 − 1

2x3 − 3x7

x6 = 1 − x3

z = 2 + 3x4 − 80x5 − 2x3 − 24x7

Notice that we have left the degenerate point using a nonzero step size and z hasincreased. One more pivot leads to the unique optimal solution.

Bland’s Rule

Another method for preventing cycling is Bland’s Rule, which carefully breaks tiesamong entering and exiting variables. The procedure is as follows

1. Choose among all possible entering variables, the one with the lowest variableindex.

2. Among all equally most limiting exiting variables, choose the one with the lowestvariable index.

Pivot Rule Summary

There is no standard best strategy for choosing efficient pivots that avoid cycling.The efficiency of a method depends largely on the problem structure. For typicalproblems arising in applications, many researchers agree that a realiable stratedy isto use the method of steepest ascent at non-degenerate vertices and employ Bland’sRule at degenerate vertices.


Exercises

1. Consider the following standard form linear program. What is the basic solutionassociated with the basis B = 1, 4, 5? Is this solution a basic feasible solution?

maxx

z = cTx

s.t. Ax = b

x ≥ 0

x ∈ R5

x =[x1 x2 x3 x4 x5

]Tc =

[1 1 2 1 2

]T

A =

−1 1 1 0 1

1 −1 1 1 0

2 3 0 0 1

b =

[6 3 16

]T2. Using Definition 28, show that the following feasible region is unbounded.

Ω =x ∈ R5 | x1 + x2 + x3 ≤ 5, x4 − x2 ≤ 5, x ≥ 0

3. Provide a definition for an unbounded minimization linear program modeled

after Definition 29.

4. Show that in the final step of the unbounded LP example (Page 170) the un-bounded step direction is v = (4, 1, 1, 0).

5. Provide three examples, one for each of the three possibilities discussed in Re-mark 31.

6. Show that every point in the set X on page 181 is an optimal point of the givenlinear program.

7. Solve the following linear programs using the Simplex Method as described inthis section.


(a)

max z = 2x1 − x2 + x3

s.t. 3x1 + x2 + x3 ≤ 55

x1 − x2 + 2x3 ≤ 10

x1 + x2 − x3 ≤ 20

x ≥ 0

x ∈ R3

(b)

min z = −2x1 + x2

s.t x1 + x2 ≤ 4

x2 ≤ 3

x1 − x2 ≤ 2

x ≥ 0

x ∈ R2

(c)

min z = 3x1 − x2

s.t. x1 − x2 ≤ 1

− 5x1 + x2 ≤ 5

x ≥ 0

x ∈ R2

(d)

max z = x1 + x2

s.t. x1 + x2 ≤ 4

x1 − x2 ≤ 2

− 2x1 + x2 ≤ 1

x ≥ 0

x ∈ R2


(e)

max z = x1 + x2 + x3

s.t. x1 + x2 + 2x3 ≥ 2

2x1 + x2 + x3 ≤ 6

x ≥ 0

x ∈ R3

(f)

max z = x1 + 2x2

s.t. x1 + 2x2 ≤ 4

x1 − x2 ≤ 5

x ≥ 0

x ∈ R2

8. Solve the following linear program using the method of largest coefficient andapplying Bland’s Rule when necessary to avoid cycling.

9. Formulate a linear program with at least two optimal vertices, at least one ofwhich is degenerate. Solve using the Simplex Method.

10. Solve the optimization problem in Exercise 7a by the Simplex Method employingthe pivot method of steepest ascent.

11. Show that the example problem used to describe the method of MinimizingRevisits has a unique optimal solution.

12. Using Bland’s Rule with the Simplex Method, show that the example problem(used in describing the Minimizing Revisits method) does not result in cycling.

Chapter 14

Duality

intro

Buyout Example

Example Derivation

General Recipe and Example

Primal/Dual Properties

191

192 CHAPTER 14. DUALITY

Chapter 15

Sensitivity Analysis

intro

B/N Notation, p, Q, etc.

(also relate to tableau)

193

194 CHAPTER 15. SENSITIVITY ANALYSIS

Chapter 16

Solving Integer Programs

In this chapter we will explore why integer programs can be quite difficult and compu-tationally expensive to solve relative to linear programs. The key essential conceptscan be illustrated and explored using a small example problem. Consider the followinginteger programming problem.

Problem Statement. Tom earns extra money during Spring Break by refurbishinglawns. This Spring he has enough grass seed and other supplies to complete up toseven jobs. He can complete a residential lawn job in nine hours and charges $495.He can complete a city park job in five hours (using his larger equipment) and chages$315. Tom can devote up to 54 hours to these projects. Formulate and solve aninteger program that Tom can use to maximize his revenue.

Solution. Let

x1 = (number of residential lawns to refurbish)

x2 = (number of city park lawns to refurbish)

x ∈ Z2

The revenue function (in $) is

z = 495x1 + 315x2.

The job and supplies constraints are:

x1 + x2 ≤ 7,

9x1 + 5x2 ≤ 54.

195

196 CHAPTER 16. SOLVING INTEGER PROGRAMS

The full IP is

maxx

z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x ≥ 0

x ∈ Z2

Consider a solution by careful examination of the plot of the feasi region shownin Figure 16.1. By checking the objective values at integer points of the IP feasibleregion, one can quickly find the optimal point x = (6, 0) where the revenue functiontakes on the value $2970. Notice however, that if the problem were to be sovled bylinear programming methods (i.e. let x ∈ R2), then the optimal solution is x∗ =(19

4, 9

4) where z∗ = $3060. The LP solution is not acceptable because Tom should not

plan on completing partial jobs if he expects to maintain good customer relations andhave satisfied customers who pay their bills.

Also notice that there is no apparent simple relationship between x∗ and x, suchas a rounding or estimation procedure. One can formulate examples, even in lowdimensions, for which x∗ and x are arbitrarily far apart. This, and related, questionsare explored in the exercises. The remainder of this chapter explores methods forsolving integer programs. We will use the lawn refurbishing example to help usunderstand each method.

16.1 LP Relaxation

The first method for solving integer programs is the “hopeful” method of LP Relax-ation. The idea is to allow all variables to be real-valued, find a solution, and thenre-evaluate the problem. This method is a typical first step in a full solution process.

Definition 32. Given a general mixed-integer program over integer variablesxk ∈ Z for k = 1, 2, . . .m and real variables xk ∈ R for k = m + 1, . . . , n, thecorresponding LP Relaxation is the linear program with the same objective andwith the feasible region only modified by allowing xk ∈ R for k = 1, 2, . . . ,m.

16.1. LP RELAXATION 197

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

x∗

x x1

x2

Figure 16.1: Graphical representation of the Lawn Refurbishing problem. Thefeasible region consists of the open circles, which are the integer points containedwithin the problem inequality constraints. The optimal solution is x = (6, 0) wherez = 2970. The solution to the LP relaxation problem is the non-integer point x∗ =(19

4, 9

4) where z∗ = 3060.

For example, the LP relaxation of the lawn refurbishing problem is

maxx

z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x ≥ 0

x ∈ R2

The solution to the LP Relaxation (as noted previously) is x∗ = (194, 9

4) with z∗ = 3060.

This point is not feasible with respect to the integer program, and therefore cannot


be the solution. However, if the solution to the LP Relaxation of a general MIP isinteger-valued, then it must also be an optimal solution of the MIP. This importantfact is reason enough to always solve the LP Relaxation and evaluate the form of thesolution. LP Relaxation also forms a key step in many other solution methods.

Another reason for solving the LP Relaxation is one of practicality. This solutionis relatively very easy to obtain and one may not wish to proceed further even if thegiven solution is not MIP-optimal. For the current example, z∗ is an upper bound onthe revenue that Tom can expect. No integer solution can exceed this revenue valuebecause the IP is more restrictive than the LP Relaxation. Tom may then decidethat the nearby solution x = (4, 3) with z = 2925 is a sufficiently good solutionbecause the revenue is within 5% of the upper bound. On the other hand, Tom maydecide that the solution x = (5, 2) is acceptable even though it is an infeasible pointbecause it only asks for one extra hour of labor with a resulting revenue of $3105. Inlarger problems, considerations such as these play an important role because integerprograms can be very difficult to solve exactly.

16.2 Branch and Bound

Branch and Bound Methods generate and solve a set of subproblems (using LP Re-laxation) which are known to contain the IP solution. Such methods are known toterminate in a finite number of subproblem solutions with the integer optimal solu-tion. “Branching” is the term given to the process of dividing a current LP into twoLP subproblems and “Bounding” is the process of understanding when a subproblemeither provides the optimal solution or is known not to directly contribute towardthe solution. We will illustrate the method by (maximization) example and thenconstruct a complete algorithm.

Branch and Bound Example

We will use as our example the lawn refurbishing problem. Begin by formulating theLP Rexalxation of the given problem and placing this problem as the initial problemin a list of LPs to be considered. We will call this linear program (a), and our list ofproblems is L = (a).

(a) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x ≥ 0

x ∈ R2

16.2. BRANCH AND BOUND 199

Next, remove one of the problems from the current list and solve it. Let xa andza be the solution vector and objective for problem (a). Then we have xa = (19

4, 9

4)

and za = 3060. We make the following observations:

xa 6∈ Z2

za cannot yet be compared to the objective value of any known integer solution.

Because current subproblem (a) yields a non-integer optimal point, we can branchthis subproblem into two new subproblems (b) and (c), where (b) is subproblem (a)with the additional constraint x1 ≤ 4:

(b) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x1 ≤ 4

x ≥ 0

x ∈ R2

and (c) is subproblem (a) with the additional constraint x1 ≥ 5:

(c) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x1 ≥ 5

x ≥ 0

x ∈ R2

These two new subproblems are added to our list of current problems, so L =(b), (c). This branch step was performed by choosing a non-integer valued optimaldecision variable xaj (in this case xa1), and forming the two new constraints x1 ≤ bxajcand x1 ≥ dxaje. These two new subproblems were designed to have the following keyproperties:

1. Every integer feasible point of (a) is a feasible point of either (b) or of (c), butnot both.

2. xa is infeasible for both (b) and (c).


1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

xaxb

xc

x1

x2

Figure 16.2: Graphical representation of subproblems (b) and (c) of the Lawn Re-furbishing problem. The feasible region of (b) is shown outlined in blue. The feasibleregion of (c) is shown outlined in red. Notice that xa is infeasible with respect toboth subproblems and every integer feasible point of subproblem (a) is also feasiblewith respect to either (b) or (c). The solution of (b) is xb = (3, 4) with zb = 2925.The solution of (c) is xc = (5, 9

5) with zc = 3042.

These properties are illustrated in Figure 16.3. The feasible region of (b) is outlinedin red and the feasible region of (c) is outlined in blue. The integer optimal solutionto the original IP is necessarily a feasible point of one of the subproblems (b) or (c).

Now that we have completed a branch step, we return to our current list of prob-lems, choose one, remove it from the list, and solve it. Suppose we choose subproblem(b). The solution is xb = (3, 4) with zb = 2925. Notice that this (LP relaxation) so-lution is integer-valued. This subproblem need not be subdivided further because xb

is the optimal solution to (b), so xb must also be the best integer optimal solutionwithin the feasible region of (b). xb is now our current best integer solution for the


Lawn Refurbishing problem. Let x and z hold the current best integer solution. Wehave x = xb and z = zb.

However, we are not finished because our list of current problems, L = (c),is not empty. We next remove (c) from L, and solve: xc = (5, 9

5) with zc = 3042.

Notice that the optimal solution is not integer-valued and that zc > z. Thus, wemust conclude that there might be some integer feasible point with respect to (c) thathas an optimal objective value greater than z. In this case, we must again performa branch step. (Now, if zc had been not greater than z, then we would not need tofurther consider this subproblem because no integer-valued solution could have anobjective value greater than z.)

We branch subproblem (c) into two new subproblems (d) and (e). As before, weformulate two new constraints derived from any non-integer valued decision variablein xc. In this case, we must choose x2 because xc2 = 9

5, but xc1 is integer. The new

constraints are x2 ≤ 1 and x2 ≥ 2. We add the following two subproblems to L:

(d) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x1 ≥ 5

x2 ≤ 1

x ≥ 0

x ∈ R2

and

(e) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x1 ≥ 5

x2 ≥ 2

x ≥ 0

x ∈ R2

so that L = (d), (e).The next step is to remove any subproblem from L and solve. We can remove

and solve (d), leaving L = (e). The feasible region for subploblem (d) is illustratedin Figure ??. The optimal solution is xd = (49

9, 1) with zd = 3010. Notice that xd is

non-integer and zd > z(= 2925). So again, there might be an integer feasible point


1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

xd

x1

x2

Figure 16.3: Graphical representation of subproblem (d) of the Lawn Refurbishingproblem. The feasible region is shown outlined in blue. The solution of (d) is xd =(49

9, 1) with zd = 3010.

of (d) with objective value greater than z. Thus, we branch this subproblem into


subproblems (f) and (g) with the added constraints x1 ≤ 5 and x1 ≥ 6:

(f) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x1 ≥ 5

x1 ≤ 5

x2 ≤ 1

x ≥ 0

x ∈ R2

and

(g) maxx z = 495x1 + 315x2

s.t. x1 + x2 ≤ 7

9x1 + 5x2 ≤ 54

x1 ≥ 5

x1 ≥ 6

x2 ≤ 1

x ≥ 0

x ∈ R2

Now, L = (e), (f), (g) and z = 2925. Because L is not empty, we remove andsolve any subproblem from L. Notice that subproblem (e) is infeasible, so it cannotcontribute to the solution.

We now have L = (f), (g). Removing (f) and solving, we find that the feasibleregion is the line segment connecting points (5, 0) and (5, 1). The optimal solution isxf = (5, 1) with zf = 2790. Because zf < z, this subproblem also cannot contributeto the solution.

We now have L = (g). Removing (g) and solving, we find that the feasibleregion is the single point xg = (6, 0) with zg = 2970. Notice that xg ∈ Z2 and zg > z.Thus, we have a new current best integer solution x = (6, 0) with z = 2970.

Finally, we note that L is empty – there are no unresolved subproblems. So, thecurrrent best integer solution must be the optimal solution to the original integerprogram: x = x = (6, 0), z = z = 2970.


Branch and Bound Algorithm

The example of the Lawn Refurbishing problem demonstrated the key ideas used in aformal Branch and Bound integer programming Algorithm. Recall these fundamentalideas:

Use the method of LP Relaxation to formulate and solve a set of subproblemsleading to the optimal solution of a given IP.

A subproblem can be branched in to two new subproblems in such a way thattogether they contain all integer feasible points of the parent problem.

Non-integer optimal points of LP Relaxation subproblems are systematicallyeliminated from consideration.

Subproblems are removed from consideration whenever (1) the optimal solutionis integer, (2) the subproblem is infeasible or (3) the optimal objective value isno better than that of an existing integer solution (bounding).

Branching occurs a finite number of times.

The complete Branch and Bound algorithm is shown in Figure 16.4. This algo-rithm begins with the LP relaxation of the IP of interest and systematically solves aseries of relaxation subproblems on subsets of the initial feasible region. If an optimalsolution exists, then the algorithm returns one such solution. If the initial LP relax-ation problem is infeasible, then so is the IP. However, if the initial LP relaxationproblem is unbounded, then the IP may be unbounded or it may be infeasible.


Given: Initial IP and LP relaxation (1)

Set: z ← −∞L← (1)k ← 1

L = ∅ ? z > −∞ ?IP Optimal Solution:

x = x, z = z

IP is infeasibleRemove (j) from L

Solve (j)IP is either

unbounded or infeasible

z(j) > z ?

xj ∈ Zn ?

x = x(j), z = zj

Select xj` 6∈ Z

Add to L:

(k + 1): (j) with x` ≤ bxj`c(k + 2): (j) with x` ≥ dxj`e

Set: k ← k + 2

yes yes

nono

infeasible

unbounded

finite

yes

no

yes

no

Figure 16.4: Branch and Bound Algorithm for solving a maximization IP using aseries of LP relaxation subproblems.


16.3 Cutting Planes

Similar to Branch and Bound methods, Cutting Plane methods also consider a seriesof LP relaxation problems whose solution sets contain integer optimal solutions (if anyexist) of a given integer program. However, Cutting Plane methods do not subdividethe feasible region, but instead attempt to reduce the size of the feasible region byadding problem constraints until the IP and the corresponding LP relaxation havethe same (optimal) solution.

Consider the Lawn Refurbishing problem. Suppose we add the constraint 3x1 +2x2 ≤ 18. Then, this modified IP and the corresponding LP relaxation problem havethe solution x∗ = x = (6, 0) with z∗ = z = 2970. This new situation is illusrated inFigure 16.6. Notice that the new constraint accomplishes the same key constructionas the branch step of the Branch and Bound method. That is, with respect to thenew constraint

every integer feasible point of the IP remains feasible, and

the optimal solution of the LP relaxation problem is infeasible.

New constraints which satisfy these two properties are called cutting planes becausethey “cut off” a portion of the feasible region of the LP relaxation problem with-out eliminating integer feasible points. The difficulty with cutting plane methods isthat finding a suitable cutting plane can be just as difficult as solving the originalIP. Fortunately, iterative methods have been developed that can solve IPs using afinite number of cutting planes which are easily found. We will consider the methodintroduced by Gomory applied to IPs with standard form constraints.

Consider the standard form LP relaxation problem with equality constraints Ax =b. The Gomory Cutting Plane constraints are

f(p)− f(−Q)xN ≤ 0, (16.1)

where p and Q are the familiar tableau variables relative to the optimal LP relaxationsolution, and f : R → [0, 1) is the positive fractional part function defined by f(x) =x− bxc. For example f(5.26) = 0.26, f(7) = 0 and f(−2.36) = 0.64.

In order to show that the Gomory constraints are good cutting plane constraints,we must show (a) x∗ does not satisfy 16.1 whenever x∗ 6∈ Zn and (b) every x ∈ Znsuch that Ax = b and x ≥ 0 does satisfy 16.1.

First, consider optimal solution x∗ 6∈ Zn. x∗N = 0 and p = xB is not integer valued.So, f(pk) > 0 for some k and x∗ does not satisfy 16.1.

Next, consider integer feasible point x and optimal basis B. The equality con-straints Ax = b can be written ABxB + ANxN = b or equivalently (for basis B)xB − Qxn = p. Separating p and Q into their integer and fractional parts yieldsxB + (b−Qc+ f(−Q))xN = bpc+ f(p). Rearranging the terms yields

xB − bpc+ b−QcxN = f(p)− f(−Q)xN .

16.3. CUTTING PLANES 207

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

x x1

x2

Figure 16.5: Graphical representation of the Lawn Refurbishing problem with theadditional constraint 3x1 + 2x2 ≤ 18. The IP and the corresponding LP relaxationhave the same (integer) optimal solution x∗ = x = (6, 0) where z∗ = z = 2970.

Notice that the left-hand-side of this equation is integer-valued, so the right-hand-sidemust also be integer-valued. Also, each term of the right hand side must be strictlyless than 1 because f(p) < 1 and f(−Q)xN ≥ 0. Thus, f(p) − f(−Q)xN ≤ 0 for anyinteger-valued feasible point.


Cutting Plane Example

Consider the Lawn Refurbishing problem. Converting the LP to standard form withslack variables x3 and x4 we have

maxx∈R4

z = cTx

s.t. Ax = b

x ≥ 0

xT =[x1 x2 x3 x4

]cT =

[495 315 0 0

]

A =

1 1 1 0

9 5 0 1

bT =

[7 54

]The optimal solution is characterized by the optimal basis B = 1, 2 for which wefind

AB =

1 1

9 5

, A−1B =

1

4

−5 1

9 −1

, AN =

1 0

0 1

,p = A−1

B b =

194

94

, Q = −A−1B AN =

1

4

5 −1

−9 1

,f(p) =

34

14

, f(−Q) =

34

14

14

34

.The Gomory Cutting Plane constraints 16.1 are3

4

14

−3

414

14

34

x3

x4

≤0

0

or, equivalently

3x3 + x4 ≥ 3

x3 + 3x4 ≥ 1.

16.3. CUTTING PLANES 209

We can write theses two constraints in terms of x1 and x2 using the variable substi-tution given by the equality constraints Ax = b. We have

3x1 + 2x2 ≤ 18

7x1 + 4x2 ≤ 42.

These new constraints are now added to the original IP. By construction, the new IPhas the same (integer) feasible region, illustrated in Figure 16.3. The boundaries ofthe new cutting plane constraints are shown as red lines. This linear program has thesame optimal solution as the original integer program.

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

x∗ x1

x2

Figure 16.6: Graphical representation of the Lawn Refurbishing problem with theadditional Gomory cutting plane constraints. The IP and the corresponding LPrelaxation have the same (integer) optimal solution x∗ = x = (6, 0) where z∗ = z =2970.


Cutting Plane Algorithm

The Lawn Refurbishing example problem was solved using a single set of cuttingplane constraints. In general, Gomory’s method requires more than one set of cuttingplanes. The method is as follows.

1. Given: Initial (IP).

2. Formulate the relaxed standard form (LP) corresponding to (IP).

3. Solve (LP). If x∗k ∈ Z for all k, then go to Step 6.

4. Formulate the cutting plane constraints 16.1 from (LP).

5. Augment (LP) to include the cutting plane constraints. Go to Step 3.

6. x for (IP) is the subset of optimal variables from x∗ for (LP).

Gomory also showed that this algorithm terminates in a finite number of iterations.

16.4 Implicit Enumeration

The method of Implicit Enumeration (IE) is a systematic way to solve binary integerprograms:

maxy

z = cTy

s.t. Ay ≤ b (16.2)

y ∈ 0, 1m

without resorting to a complete enumeration scheme that tests all 2m potential feasiblepoints. IE is a branch and bound type method that systematically considers bothfeasible and infeasible points leading (hopefully quickly) to an optimal solution. Inthis section, we show that a variety of integer programs can be recast into binaryinteger programs, illustrate IE by example, and finally present the full algorithm.

Converting IPs to BIPs

Before we examine how to solve binary integer programs, we make the observationthat any integer program of the form

maxx

z = cTx

s.t Ax ≤ b

0 ≤ x ≤ u

x ∈ Zn

16.4. IMPLICIT ENUMERATION 211

can be written in the equivalent binary integer program form 16.2. Each variablexk ∈ Z is bounded 0 ≤ xk ≤ uk. Introduce binary variables yk,j defined by

xk = yk,0 + 2yk,1 + 4yk,2 + . . .+ 2pyk,p,

where p = blog2 ukc. Then, the IP can be written in terms of the binary variablesonly. Consider the following example of an IP (left) and an equivalent BIP (right):

maxx

z = 3x1 + x2

s.t. x1 + 2x2 ≤ 5

0 ≤ x1 ≤ π

0 ≤ x2 ≤ 4

x ∈ Z2

maxy

w = 3y1,0 + 3y1,1 + y2,0 + y2,1 + y2,2

s.t. y1,0 + y1,1 + 2y2,0 + 2y2,1 + 2y2,2 ≤ 5

y1,0 + 2y1,1 ≤ π

y2,0 + 2y2,1 + 4y2,2 ≤ 4

y ∈ 0, 15

Notice that the second inequality constraint in the BIP is feasible for all possiblechoices of y, we include it here for completeness. Not all integer programs have explicitbounds on the decision variables, but implicit bounds may be readily available fromthe problem constraints. For example, the constraint 3x1 + x2 + 2x3 ≤ 13 guaranteesthat x1 ≤ 13

3, x2 ≤ 13 and x3 ≤ 13

2.

Implicit Enumeration: A First Example

Consider the BIP

maxx

z = 3x1 + x2 − 2x3 + 2x4

s.t. x1 + x2 + x3 + x4 ≤ 3

2x1 − x2 + x3 + 2x4 ≤ 3

− x1 + x2 + x3 + 4x4 ≥ 2

x ∈ 0, 14

As a first attempt at finding an optimal solution, consider the maximum possible

objective value. This is obtained by setting x =(

1 1 0 1)

for which we find

z = 6. Any other choice of decision variable values results in a lower objective value.This is because for every term in the objective with a positive coefficient, the objectivevalues is increased by choosing the corresponding decision variable to have a value1. Similarly, for every term in the objective with a negative coefficient, the objectivevalues is not decreased by choosing the corresponding decision variable to have avalue 0.


We have chosen the decision variable values without regard to the constraints.However, a quick check of the constraints shows that this x is feasible. Thus, x mustbe the optimal set of decision variable values: no other choice of x can improve z andx is feasible. We have the solution

x =(

1 1 0 1), z = 6 .

Implicit Enumeratioin: A Full Example

In the previous example, our first attempt at a maximal objective values solution wassuccessful. Typically, however, the initial x is not feasible, and we have more exploringto do. Next, we consider a full IE example. Consider the following scenario:

Heather wishes to pack some combination of four food items for her back-packing trip. The caloric value, weight and volume of each item is givenin the table below. She wishes to formulate and solve a binary integerprogram whose solution maximizes caloric value, subject to a weight lim-itation of 8 pounds and a volume limitation of 11 cubic feet.

item caloric weight volume

value (pounds) (ft3)

1 2 1 1

2 3 3 3

3 4 3 5

4 5 4 7

Heather defines binary decision variables: For k = 1, 2, 3, 4:

xk =

1 if item k is to be packed,

0 if not.

Her binary integer program is

(1) max z = 2x1 + 3x2 + 4x3 + 5x4

s.t. x1 + 3x2 + 3x3 + 4x4 ≤ 8

x1 + 3x2 + 5x3 + 7x4 ≤ 11,

where the problem is identified as (1). The solution, by the method of ImplicitEnumeration, begins as in the previous example. We attempt to solve (1) by con-sidering the maximum possible objective value regardless of constraints. We have


x(1) =(

1 1 1 1)

with z(1) = 14. We use the notiation x(j) and z(j) to indicate a

potential solution of problem (j). A quick check reveals that x(1) is infeasible – notsatisfying either of the constraints – so x(1) is not optimal.

We then check whether or not (1) might possibly be feasible. In other words, weconsider each constraint separately and determine whether or not some values of xkcould possibly satisfy the constraint. If all constraints can be satisfied, then we call(1) a possible problem, and we retain it for further analysis. In this case, we see that

the first constraint can be satisfied by, for example, x =(

0 0 1 1)

. The second

constraint can be satisfied by, for example, x =(

1 0 0 0)

. The conclusion here is

that (1) might possibly have a feasible solution.The next step is to create two subproblems from the current possible problem.

Subproblem (2) is formed from (1) with the substitution x4 = 1, and subproblem (3)is formed from (1) with the substituation x4 = 0. These two subproblems replace(1) by considering separately all solutions with x4 fixed. The choice of fixing theparticular variable x4 was arbitrary (we could have chosen any of the variables). Wehave two subproblems:

(2) max z = 2x1 + 3x2 + 4x3 + 5

s.t. x1 + 3x2 + 3x3 ≤ 4

x1 + 3x2 + 5x3 ≤ 4,

(3) max z = 2x1 + 3x2 + 4x3

s.t. x1 + 3x2 + 3x3 ≤ 8

x1 + 3x2 + 5x3 ≤ 11,

Problem (1) has been replaced by problems (2) and (3). Consider (2) using the same

maximal objective strategy. We have x(2) =(

1 1 1 1)

with z(2) = 14. We adopt

the notational strategy of including all four variable values, the three variables in (2)and the fixed variable from (1). Again, x(2) is infeasible, but (2) is possible becausethe two constraints can be satisfied (individually) by some choices for x. Thus, wecan replace (2) with two new subproblems (4) and (5) where one variable is fixed, sayx3 = 1 for (4) and x3 = 0 for (5). We have

(4) max z = 2x1 + 3x2 + 9

s.t. x1 + 3x2 ≤ 1

x1 + 3x2 ≤ −1,

(5) max z = 2x1 + 3x2 + 5

s.t. x1 + 3x2 ≤ 4

x1 + 3x2 ≤ 4,


At this stage, it is important to remember that we have three unresolved subproblems:(3), (4) and (5). We know that any optimal solution of (1) will be an optimal solutionfor one (or more) of these three subproblems.

The next step is to choose any one of the three subproblems and perform thesame analysis as before. Suppose we consider (3). The maximal objective solution

is x(3) =(

1 1 1 1)

with z(3) = 9. Notice that x(3) is feasible and so represents

our current best feasible solution to (1). This feasible solution we keep as a potentialoptimal solution of (1).

Now, we have two unresolved subproblems: (4) and (5). Suppose we consider (4).

The maximal objective solution is x(4) =(

1 1 1 1)

with z(4) = 14. This solution

is infeasible. Notice that (4) iteself is infeasible because the second constraint cannotbe satisfied by any choice of (binary) decision variables. This subproblem cannotprovide any feasible solutions and we can safely ignore it.

Now, we have one unresolved subproblem: (5). We have the maximal objective

solution x(5) =(

1 1 0 1)

with z(5) = 10. Recall that x(5)3 = 0 because this

variable was fixed when (5) was created from (2). Notice that x(5) is feasible, andthus represents another potentially optimal solution to (1). Because z(5) > z(3), x(5)

becomes our current best feasible solution.Finally, because we now have no remaining unresolved subproblems, our current

best feasible solution must be the optimal solution. We have

x =(

1 1 0 1), z = 10 .

In order to maximize the caloric value of items taken on the backpack trip, Heathershould take items 1, 2 and 4. The weight is 8 pounds and the volume 11 cubic feet.The total caloric value is 10.

Algorithm

The Backpacking problem demonstrated the key ideas used in a formal Implicit Enu-meration Algorithm. The fundamental ideas are:

BIPs are solved using a branch and bound type method in which subproblemsdo not eliminate potential feasible solutions.

Each subproblem is solved to maximize the objective as if there were no con-traints. Unconstrained BIPs are always feasible and never unbounded.

If a maximal solution of the unconstrained subproblem also satisfies the con-straints, then the solution is feasible for the original BIP.

Whenever a maximal solution has an objective value less than the current bestfeasible solution, then the subproblem cannot contain improved solutions.


The complete Implicit Enumeration Algorithm is shown in Figure 16.7. A feasi-bility check on variables set x(j) is a test on whether x(j) satisfies all constraints. Afeasibility check on a BIP (j) is a test on whether or not each individual constrainthas a nonempty feasible set. When selecting a subproblem from the current list L,the recommended strategy is to preview the maximal objective values and choose aproblem with the largest such value.


Given: Initial BIP (1) with constraints Ax ≤ b

Set: z ← −∞L← (1)k ← 1

L = ∅ ? z > −∞ ?BIP Optimal Solution:

x = x, z = z

BIP is infeasibleRemove (j) from L

Set x(j) to maximize z(j)

x(j) feasible? (j) infeasible?

z(j) > z?

z = z(j)

x = x(j)

Choose any x(j)` . Add to L:

(k + 1): (j) with x` = 0

(k + 2): (j) with x` = 1

Set: k ← k + 2

yes yes

nono

no

yes no

yes

yes

no

Figure 16.7: Implicit Enumeration Algorithm for solving a maximization BIP usinga series of reduced-dimension BIP subproblems.

16.5. EXERCISES 217

16.5 Exercises

1. Solve the following problems by the Branch and Bound Method.

(a)

maxx

z = 2x1 + 3x2

s.t. x1 + x2 ≤ 3

x ≥ 0

x ∈ Z2

(b)

maxx

z = 2x1 + 3x2

s.t. 0 ≤ x ≤ 52

x ∈ Z2

(c)

maxx

z = x1 + x2

s.t. 8x1 − x2 ≥ 0

9x1 + x2 ≤ 8

x ∈ Z2

(d)

maxx

z = 8x1 + 5x2

s.t. x1 + x2 ≤ 6

9x1 + 5x2 ≤ 45

x ≥ 0

x ∈ Z2

(e) Modeling Problem 36 from Chapter 11.

(f) Modeling Problem 22 from Chapter 11.

2. Solve the problems in Exericse 1 by the Cutting Planes Method.

3. Solve the following problems by the Implicit Enumeration Method.


(a)

maxx

z = 3x1 + x2

s.t. x1 + x2 ≥ 1

2x1 − x2 ≥ 1

x ∈ 0, 12

(b) Reconsider Heather’s backpacking problem for minimizing weight subjectto a total caloric content of at least 9. Include the volume constraint. Becareful, this is a minimization problem.

(c)

maxx

z = x1 + x2 + x3

s.t. 2x1 − x2 + x3 ≤ 4

x1 + 3x2 + x3 ≤ 5

0 ≤ x ≤ 2

x ∈ Z3

Chapter 17

Interior Point Methods

17.1 Affine Scaling Algorithm

17.2 Primal Path Following Algorithm

17.3 Primal-Dual Path Following Algorithm

17.4 Ellipsoid Methods

17.5 Reflection Algorithm

219

An Introduction to Mathematical...

Documents

Transcript of An Introduction to Mathematical...