Part 3. Linear Programming

3.2 Algorithm

General Formulation 1 2

1

1

1

min , , ,

subject to

1,2, ,

1, 2, ,

0 1,2, ,

A local minimum is a global minimum. The minimum must lie at the intersection

r

r i ii

r

ji i ji

r

ji i ji

i

f x x x C x

a x b j m

a x b j m m p

x i r

x

of several constraints, but not in the interior of the convex region.

Convex function

Convex region

Example

4 5

1 1

1 1 2 2 3 3 4 4 5 5

4 5 4

max

. .

0

1,2,3,4

c=1,2,3

0 c=1,2,3,4,5

p p c cp c

p p p p p p

p p

c c

c

V q C x

s tq Y x Y x Y x Y x Y x

q D

p

x Sx x S

x

Profit

Amount of product p

Amount of crude c

Graphical Solution

1 2

1 2

1

2

min 40 36. .

5 3 45 8 0 10 0

x xs t

x xxx

Degenerate Problems

Non-unique solutions

Unbounded minimum

Degenerate Problems – No feasible region

1 2

1 2

1 2

1 2

min

. . 2 2 0 , 0

f x x

s tx xx xx x

x

Simplex Method – The standard form

1 1 2 2

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

1 2

1 2

min (or max )

. .

0, 0, , 0 0, 0, ,

n n

n n

n n

m m mn n m

n

f c x c x c x

s ta x a x a x ba x a x a x b

a x a x a x b

x x xb b b

x

0m

or

min (or max )

. .

T

m n

f

s t

x c x

A x bx 0b 0

Simplex Method - Handling inequalities

1 2 3 4

1 2 3 4 1

1

. slack variables2 3 4 25

2 3 4 25where 0

x x x x

x x x x s

s

Ex

1 2 3

1 2 3 2

2

. surplus variables5 7

5 7where 0

x x x

x x x s

s

Ex

Simplex Method - Handling unrestricted variables

1 2 21

(1) Let

unrestricted variable = (i.e., )

where 0 and 0

(2) Use one of the equations in the standard form1

i i i in ini

x s sx

s s

x b a x a xa

ans then substitute it into other equations.

Simplex Method- Calculation procedure

1 2

1 2

1 2

1 2

1 2

min 3 2

. . 2 4 3 2 14 3 , 0

f x x

s tx x

x xx xx x

x

Calculation Procedure- Step 0

1 2

1 2

1 2

Convert all inequality constraints to the form in which the RHSs are positive.

2 4 3 2 14 3

x xx xx x

Calculation Procedure - Step 1

1 2 3

1 2 4

1 2

Introduce slack/surplus variables and convertthe inequality constraints to equality constraints 2 4 3 2 14

x x xx x xx x

5 3

x

Calculation ProcedureStep 2: find a basic solution corresponding to a

corner of the feasible region.

1 1, 1 1 1, 1

2 2, 1 1 2, 2

, 1 1 ,

1 2

1

Gauss-Jordan Elimination Canonocal Form

, , , : basic variables (dependent variables), , : nonbasic

m n

m m n n

m m n n

m m m m m n n m

m

m n

x a x a x bx a x a x b

x a x a x b

x x xx x

A x b

variables (independent variables)

Remarks• The solution obtained from a cannonical system

by setting the non-basic variables to zero is called a basic solution (particular solution).

• A basic feasible solution is a basic solution in which the values of the basic variables are nonnegative.

• Every corner point of the feasible region corresponds to a basic feasible solution of the constraint equations. Thus, the optimum solution is a basic feasible solution.

Full Rank Assumption

The rows of ( ) are linearly independent, i,e,, the rank of is .

m n

m n

m m n

m

A

A

Given the set of m simultaneous linear equationsin n unkowns, i.e. (1)Let be any nonsingular submatrix madeof columns of . Then, if

m n

m m

A x b

B A all - components of

NOT associated with the columns of are setequal to zero, the solution to the resulting set of equations is said to be a to (1) w.r.t.the basis . The componen

n m

basic solution

x B

B

( )

ts of associated withthe columns of are called the .

|

|

m n m m m n m

T TB N

N B

basic variables

xB

A B N

x x x

x 0 Bx b

Fundamental Theorem of Linear Programming

Given a linear program in standard form where A is an mxn matrix of rank m.

1. If there is a feasible solution, there is a basic feasible solution;

2. If there is an optimal solution, there is an optimal basic feasible solution.

Implication of Fundamental Theorem

Search inn !m !( )!

basic feasible solutions

nm n m

Extreme Point

1 2

1 2

A point in a convex set is said to bean extreme point of if there are no twodistinct points and in such that

(1- ) for some and 0 1.

CC

C

x

x xx x x

Theorem (Equivalence of extreme points and basic

solutions)

Let be an matrix of rank and an -vector.Let be the convex polytope consists of all -vectors

satisfying A v

m n m mK n

A b

xAx = bx 0

ector is an extreme point iff is a basic feasible solution.x

CorollaryIf there is a finite optimal solution to

a linear programming problem, there is a finite optimal solution which is an extreme point of the constraint set.

Step 2

3 1 2

4 1 2

5 1 2

1 2

3 4 5

number of basic variables = number of constraints

number of non-basic variables 2 43 2 14 33 2 0

4, 14, 3, 0

m

n mx x xx x xx x xf x x

x x x f

x1 and x2 are selected as non-basic variables

Step 3: select new basic and non-basic variables

1 2

1 2

3 2 0

Exame the coefficients in to determine which variable( or ) decreases the value of the objective function whenthe variable is increased from zero. As a general rule, itis advantageou

f x x

fx x

2

1

s to select as a new basic variable which hasthe largest positive coefficient in the objective function. Thus,

remains at zero increases from zero to a positive number

xx

new basic variable

Which one of x3, x4, x5 should be selected as the new non-basic

variables? 1 2 1,max

1 2 1,max 1

1 2 1,max

5

1

1

2 43 2 14 14 / 3 3

3 3

0

The limiting value for can be calculated analyticallyfor each equation by / , i.e. - 4, 14 / 3 and 3. The limitingconstraint i

i i

x x xx x x xx x x

x

xb a

s the one for which the ratio is positive andhas the smallest value.

Step 4: Transformation of the Equations

1 3 4

2 5

1

2 5

1 2 5

New basic variables: ( , , )New non-basic variables: ( , )

The transformation can be achieved by substituting as a function of and (the new non-basic variables).

Let 3

x x xx x

xx x

x x x

3 1 2 3 2 5

4 1 2 4 2 5

5 1 2 1 2 5

1 2 2 5

2 4 73 2 14 5 3 5 3 33 2 0 5 3 9

x x x x x xx x x x x xx x x x x xf x x f x x

=0

Repeat step 4 by Gauss-Jordan elimination

1

2

3

4

5

Original System:

-1 2 1 0 0 0 43 2 0 1 0 0 141 1 0 0 1 0 33 2 0 0 0 1 0

xxxxxf

Write in terms of the augmented matrix

-1 2 1 0 0 0 43 2 0 1 0 0 141 1 0 0 1 0 33 2 0 0 0 1 0

N N B B B

Step 3: Pivot RowSelect the smallest positive ratio

Step 3: Pivot ColumnSelect the largest positive element in the objective function.

bi/ai1

Pivot element

1 2 3 4 5

3

4

1

By elementary row operation B N B B N0 1 1 0 1 0 70 5 0 1 3 0 51 -1 0 0 1 0 30 5 0 0 3 1 9

To do a better job in book-keeping, a "tableau" can be used

0 1 1 0 1

x x x x x f bxxx

0 70 5 0 1 3 0 51 -1 0 0 1 0 30 5 0 0 3 1 9

Basic variables

1 2 3 4 5

3

2

1

* * * *1 2 3

5

0 0 1 -1/5 8/5 0 60 1 0 1/ 5 3/ 5 0 1

1 0 0 1/ 5 2 / 5 0 40 0 0 1 0 1 14

4, 1, 6, 14

Note that the coefficient of is zero.

x x x x x f bxxx

x x x f

x

Step 5: Repeat Iteration

An increase in x4 or x5 does not reduce f

1 2 3 4 5

5

2

1

* * * *1 2 5

0 0 5/8 -1/8 1 0 15/40 1 3/8 1/8 0 0 13/ 4

1 0 1/ 4 1/ 4 0 0 5/ 20 0 0 1 0 1 14

5/ 2, 13/ 4, 15/ 4, 14

Alternative solution!

x x x x x f bxxx

x x x f

It is necessary to obtain a first feasible solution!

1 2

1 2

1 2

1 2 3

1 2 4

1 2 3

[Example] min ( ) . . 3 7 2 4STEP 0:STEP 1: 3 - 7 2 4

0,

f x xs t x x

x x

x x xx x x

x x x

x

4-7, 4

x

Infeasible!

Phase I – Phase II Algorithm• Phase I: generate an initial basic feasible

solution;• Phase II: generate the optimal basic

feasible solution.

Phase-I Procedure• Step 0 and Step 1 are the same as

before.• Step 2: Augment the set of equations by

one artificial variable for each equation to get a new standard form.

New Basic Variables11 1 12 2 1 1 1

21 1 22 2 2 2 2

1

1 2 3 5

1 2 4 6

5 6

3 7

2 4

One feasible solution can always be found:basic variables: 7, 4; nonbasic va

n n n

n n n

m mn n n m m

a x a x a x x ba x a x a x x b

a x a x x b

x x x xx x x x

x x

1 2 3 4riables: 0.x x x x

New Objective Function

1

5 6

1 1 2 21 1 1 1

1 2 3 4

or

4 3 11

m

n kk

m m m m

i i in n ii i i i

f x

f x x

f a x a x a x b

f x x x x

If the minimum of this objective function is reached,then all the artificial variables should be reduced to 0.

1 2 3 4 5 6

5

6

1 2 3 4 5 6

1

6

1 2 3 4 5 6

1

2

N N N N B B

3 1 1 0 1 0 71 2 0 1 0 1 44 3 1 1 0 0 11

B N N N N B

1 1/ 3 1/ 3 0 1/ 3 0 7 / 30 5 / 3 1/ 3 1 1/ 3 1 5 / 30 5 / 3 1/ 3 1 4 / 3 0 5 / 3

B B N N N NInitial feasib

1 0 2 / 5 1/ 5 2 / 5 1/ 5 20 1 1/ 5 3 / 5 1/ 5 3 / 5 10 0 0 0 1 1 0

x x x x x x bxx

x x x x x x bxx

x x x x x x bxx

1 2

3 4

le solution:2, 1

0x xx x

Step 3 – Step 5