15. Linear Programming / Simplex Methodisdl.cau.ac.kr/.../DOEOPT/15.Linear.programming.pdfSome NLP...

Hae-Jin Choi School of Mechanical Engineering,

Chung-Ang University

15. Linear Programming / Simplex Method

SCHOOL OF MECHANICAL ENG. CHUNG-ANG UNIVERSITY -1-

LP is simpler than NLP, hence, good for a foundation

Linearity has some unique features for optimization

A lot of problems are or can be converted to a LP formulation

Some NLP algorithms are based upon LP simplex method

Why Talk About Linear Programming?

DOE and Optimization

http://me.cau.ac.kr/_include/index.php?SubNum=main


US Air Force wanted to investigate the feasibility of applying mathematical techniques to military budgeting and planning.

George Dantzig had proposed that interrelations between activities of a large organization can be viewed as a LP model and that the optimal program (solution) can be obtained by minimizing a (single) linear objective function.

Air Force initiated project SCOOP (Scientific Computing of Optimum Programs)

NOTE:

SCOOP began in June 1947 and at the end of the same summer, Dantzig and associates had developed:

1) An initial mathematical model of the general linear programming problem.

2) A general method of solution called the simplex method.

Historical Perspective




A large variety of Simplex-based algorithms exist to solve LP problems.

Other (polynomial time) algorithms have been developed for solving LP problems: Khachian algorithm (1979)

Kamarkar algorithm (AT&T Bell Labs, mid 80s)

BUT,

none of these algorithms have been able to beat Simplex in actual practical applications.

HENCE,

Simplex (in its various forms) is and will most likely remain the most dominant LP algorithm for at least the near future

Simplex Today




The objective function is of the minimization type. ‘maximize f = cx’ is converted to ‘minimize f = -cx’

All constraints are of the equality type. If a constraint appears in the form of ‘less than or equal to’ , then it is

converted into equality constraint by adding nonnegative slack variable.

All the decision variables are nonnegative If xi may be unrestricted in sign, then it can be written as xi=xi’- xi’’, where

xi’ ≥ 0 and xi’’ ≥ 0

Characteristics of Linear Programming




Given

At - tensile strength area, function of d

Db - bolt circle diameter

Pt - total load

C - joint constant

Fi - preload (= 0.75 Sp At)

Find

N - number of bolts, Sp - proof strength, d - diameter

Satisfy

3d ≤ π Db/N good wrench rule

π Db / N ≤ 6d good seal rule

C Pt / N ≤ Sp At - Fi static loading constraint

Fi ≥ (1 - C) Pt / N joint separation constraint

Minimize Z= [ f1(N, d, Sp), f2(N, d, Sp), ..]

Bolted Joint Design

Question:

Is this a linear or nonlinear model?




Given

d - diameter

At - tensile strength area, function of d

Db - bolt circle diameter

Pt - total load

C - joint constant

Fi - preload (= 0.75 Sp At)

Find

N - number of bolts, Sp - proof strength

Satisfy

3d ≤ π Db / N good wrench rule

π Db / N ≤ 6d good seal rule

C Pt / N ≤ Sp At - Fi static loading constraint

Fi ≥ (1 - C) Pt / N joint separation constraint

Minimize Z= [ f1(N, Sp), f2(N, Sp), ..]

Bolted Joint Design (2)

Question:

Is this a linear or nonlinear problem?



SCHOOL OF MECHANICAL ENG. CHUNG-ANG UNIVERSITY -7- DOE and Optimization

Question:

Is this a linear or nonlinear problem?

2 21 2

3 2 31 2

4 41 22 31 2

1 2

Minimize Subject to

0, 0

f x x

x x ex x ex x ex x

=

≥

≥

≥≥ ≥

Power Function Example



Linear Programming Problem

Maximize z = 4x1 +5x2

Subject to -x1 + x2 ≤ 4 x1 + x2 ≤ 6 x1, x2 ≥ 0

Minimize f = -4x1 -5x2

Subject to

-x1 + x2 +x3 = 4 x1 + x2 +x4 = 6 xi ≥ 0 ; i=1 to 4 where, x3 and x4 are slack variables Canonical Form

The number of Eqs. are smaller than that of variables; therefore, many solutions exist




Basic and Non-basic Solutions


Subject to

-x1 + x2 +x3 = 4 x1 + x2 +x4 = 6 xi ≥ 0 ; i=1 to 4

x3 = 4 + x1 - x2 x4 = 6 - x1 - x2 when x1 =0, x2 =0 (non-basic) x3 =4, x4 =5 (basic)






Subject to

-x1 + x2 +x3 = 4 x1 + x2 +x4 = 6 xi ≥ 0 ; i=1 to 4

x2= 4 + x1 -x3 x4 = 6 - x1 - x2 when x1 =0, x3 =0 (non-basic) x2 =4, x4 =2 (basic)






Subject to

-x1 + x2 +x3 = 4 x1 + x2 +x4 = 6 xi ≥ 0 ; i=1 to 4

x3= 4 + x1 -x2 x2= 6 - x1 - x4 when x1 =0, x4 =0 (non-basic) x2 =6, x3 =-2 (basic)




Basic and Non-basic Solutions Number of non-basic solutions = Number of variables (including slack variables) – Number of constraints = 4 – 2 = 2

Finding all possible combinations of non-basic solutions, of which number is 4C2= 6

No. x1 x2 x3 x4 f Location

1 0 0 4 6 0 A

2 0 4 0 2 -20 B

3 0 6 -2 0 --- Infeasible

4 -4 0 0 10 --- Infeasible

5 6 0 10 0 -24 D

6 1 5 0 0 -29 C x1

x2

A D

C

B

-x1 + x2 = 4

x1 + x2 = 6




Extreme point (or Simplex filter) theorem:

If the maximum or minimum value of a linear function defined over a polygonal convex region exists, then it is to be found at the boundary of the region.

Fundamental Theorem

Convex set:

A set (or region) is convex if, for any two points (say, x1 and x2) in that set, the line segment joining these points lies entirely within the set.

A point is by definition convex.




A finite number of extreme points implies a finite number of solutions!

Hence, search is reduced to a finite set of points

However, a finite set can still be too large for practical purposes

Simplex method provides an efficient systematic search guaranteed to converge in a finite number of steps.

What does the extreme point theorem imply?




1. Begin the search at an extreme point (i.e., a basic feasible solution).

2. Determine if the movement to an adjacent extreme can improve on the optimization of the objective function. If not, the current solution is optimal. If, however, improvement is possible, then proceed to the next step.

3. Move to the adjacent extreme point which offers (or, perhaps, appears to offer) the most improvement in the objective function.

4. Continue steps 2 and 3 until the optimal solution is found or it can be shown that the problem is either unbounded or infeasible.

Basic Steps of Simplex Method




General Simplex LP model:

Minimize f(x)= cx

Satisfy

A x = b

x ≥ 0

In order to get and maintain this form, use

slack, if x ≤ b, then x + slack = b

artificial variables (sometimes need to be added to ensure all variables ≥ 0)

Step 0 – Obtain Canonical Form

Compare constraint conversion with goal conversions using deviation variables

IMPORTANT: Simplex only deals with equalities




Basic x1 x2 x3 x4 -f b Initial x3 -1 1 1 0 0 4

x4 1 1 0 1 0 6 Cost function

-4 -5 0 0 1 0

-x1 + x2 +x3 = 4 x1 + x2 +x4 = 6 -4x1 - 5x2 - f = 0

Minimize f = -4x1 -5x2 Subject to

-x1 + x2 +x3 = 4 x1 + x2 +x4 = 6 xi ≥ 0 ; i=1 to 4

Formulation of Tableau for Simplex Method

Step 0 – Obtain Canonical Form



Two things to determine:

1) which (nonbasic) column of A should be brought into the basis so that the solution improves?

2) which column can be removed from the basis such that the solution stays feasible?

Movement to Adjacent Extreme Point

Given any basis we move to an adjacent extreme point (another basic feasible solution) of the solution space by exchanging one of the columns that is in the basis for a column that is not in the basis.




General rules:

The one non-basic variable to come in is the one which provides the highest reduction in the objective function.

The one basic variable to leave is the one which is expected to go infeasible first.

NOTE: THESE ARE HEURISTICS!!

Variations on these rules exist, but are rare.

Entering and Departing Vector (Variable) Rules




Step 1 : Find Pivot Element



-4 -5 0 0 1 0

1. Find the column in which the minimum negative coefficient in the cost function. Why? This term will decrease the cost function the most; therefore this term must be a basic variable

2. Find the row in which b/a is the minimum, where a is positive number. Pivot element




Step 2 : Entering Basic Variable

Basic x1 x2 x3 x4 -f b Enter x2 x2 -1 1 1 0 0 4

x4 2 0 -1 1 0 2 Cost function

-9 0 5 0 1 20

Row2’ = Row2 - Row1 Cost F’ = Cost F + 5*Row1

x1

x2

A D

C

B

-x1 + x2 = 4

x1 + x2 = 6




Step 3 : Repeatedly Entering Basic Variable

Basic x1 x2 x3 x4 -f b Enter x2 x2 -1 1 1 0 0 4

x4 2 0 -1 1 0 2

Cost function

-9 0 5 0 1 20

Pivot Element


Row2’ = Row2/2 Row1’ = Row1+Row2’ Cost F’ = Cost F+ 9*Row2’

Basic x1 x2 x3 x4 -f b

x2 0 1 1/2 1/2 0 5 Entering x1 instead of x4

x1 1 0 -1/2 1/2 0 1

Cost function

0 0 3 2 1 29

x1

x2

A D

C

B

-x1 + x2 = 4

x1 + x2 = 6



Step 4 : Stopping and Finding Optimum Row2’ = Row2/2 Row1’ = Row1+Row2’ Cost F’ = Cost F+ 9*Row2’


x2 0 1 1/2 1/2 0 5 Entering x1 instead of x4

x1 1 0 -1/2 1/2 0 1

Cost function

0 0 1/2 9/2 1 29

If all numbers are positive, then Stop the iteration and find Minimum value of f= -b=-29

- Minimum





x2 0 1 1/2 1/2 0 5 Instead of X4

x1 1 0 -1/2 1/2 0 1

Cost function

0 0 1/2 9/2 1 29

Basic x1 x2 x3 x4 -f b Instead of x3

x2 -1 1 1 0 0 4

x4 2 0 -1 1 0 2 Cost function

-9 0 5 0 1 20



-4 -5 0 0 1 0 x1 +x2 +x4 =6

-4x1 - 5x2 - f = 0

2x1 –x3+x4 =2

-9x1 - 5x3 - f = 20

-x1 +x2+x3 =4

-x1 +x2+x3 =4

x1 –0.5x3+0.5x4 =1

0.5x3 +4.5x4 - f = 29

x2+0.5x3+0.5x4 =5




Example 2:

1 2 3

1 2 3

1 2 3

1 2 3

Maximize 2Subject to 2x 2 2 5 6 4 6 0, 1, 2,3i

F x x x

x xx x xx x x

x i

= + +

+ − ≤− + − ≥ −

+ + ≤≥ =

1 2 3

1 2 3

1 2 3

1 2 3

Minimize 2Subject to 2 2 2 5 6 4 6 0, 1, 2,3i

f x x x

x x xx x xx x x

x i

= − − −

+ − ≤− + ≤+ + ≤

≥ =




Example 2: STEP 0 – Canonical Form

1 2 3 4

1 2 3 5

1 2 3 6

1 2 3

Cannonical Form 2 2 2 5 6 4 6 - 2 - 0 0, i

x x x xx x x xx x x xx x f

x

+ − + =− + + =+ + + =− − =

≥ 1, 2,3i =

1 2 3

1 2 3

1 2 3

1 2 3

Minimize 2Subject to 2 2 2 5 6 4 6 0, 1, 2,3i

f x x x

x x xx x xx x x

x i

= − − −

+ − ≤− + ≤+ + ≤

≥ =




Example 2: STEP 0 - Form Tableau

1 2 3 4

1 2 3 5

1 2 3 6

1 2 3

Cannonical Form 2 2 2 5 6 4 6 - 2 - 0 0, i

x x x xx x x xx x x xx x f

x

+ − + =− + + =+ + + =− − =

≥ 1, 2,3i =


x1 x2 x3 x4 x5 x6 -f b

Initial 2 1 -1 1 0 0 0 2

2 -1 +5 0 1 0 0 6

4 1 1 0 0 1 0 6

cost -1 -2 -1 0 0 0 1 0



Example 2: STEP 1 and STEP 2

basic x1 x2 x3 x4 x5 x6 -f b

x4 2 1 -1 1 0 0 0 2

x5 2 -1 +5 0 1 0 0 6

x6 4 1 1 0 0 1 0 6

cost -1 -2 -1 0 0 0 1 0

Basic x1 x2 x3 x4 x5 x6 -f b

x2 2 1 -1 1 0 0 0 2

x5 4 0 4 1 1 0 0 8

x6 2 0 2 -1 0 1 0 4

cost 3 0 -3 2 0 0 1 4




x2 2 1 -1 1 0 0 0 2

x5 4 0 4 1 1 0 0 8

x6 2 0 2 -1 0 1 0 4

cost 3 0 -3 2 0 0 1 4


Example 2: STEP 3


x2 3 1 0 5/4 1/4 0 0 4

x3 1 0 1 1/4 1/4 0 0 2

x6 0 0 0 -3/2 -1/2 1 0 0

cost 6 0 0 11/4 3/4 0 1 10

Any of the two rows can be a pivot row



Example 2: STEP 3 Basic x1 x2 x3 x4 x5 x6 -f b

x2 3 1 0 5/4 1/4 0 0 4

x3 1 0 1 1/4 1/4 0 0 2

x6 0 0 0 -3/2 -1/2 1 0 0

cost 6 0 0 11/4 3/4 0 1 10

All positive Minimum f = -10

At x2=4, x3=2, x6=0 (Basic) x1=x4=x5=0 (Nonbaseic)



Example 3: Infinite Number of Solutions

Basic x1 x2 x3 x4 x5 -f b

x3 10 5 1 0 0 0 2500

x4 4 10 0 1 0 0 2000

x5 2 3 0 0 1 0 900

cost -40 -100 0 0 0 1 0

1 2

1 2

1 2

1 2

1 2

-40 100

10 5 2500 4 10 2000 2 3 900 , 0

Minimizef x x

Subject tox xx xx x

x x

= −

+ ≤+ ≤+ ≤≥

1 2

1 2 3

1 2 4

1 2 5

-40 100

10 5 2500 4 10 2000 2 3 900 0; 1 to 5i

Minimizef x x

Subject tox x xx x xx x x

x i

= −

+ + =+ + =+ + =

≥ =



Example 3: Infinite Number of Solutions Basic x1 x2 x3 x4 x5 -f b

x3 8 0 1 -1/2 0 0 1500

x2 4/10 1 0 1/10 0 0 200

x5 8/10 0 0 -3/10 1 0 300

cost 0 0 0 10 0 1 20,000

x2=200, x3=1500, x5=300, (Basic) x1=x4=0 (Nonbasic) f=-20000

Basic x1 x2 x3 x4 x5 -f b

x1 1 0 1/8 -1/16 0 0 1500/8

x2 0 1 -1/20 1/8 0 0 125

x5 0 0 -1/10 -1/4 1 0 150

cost 0 0 0 10 0 1 20,000

x1=1500/8, x2=125, x5=150 (Basic) x3=x4=0 (Nonbasic) f=-20000

Sol1=[0 200 1500 0 3000]T

Sol2=[1500/8 125 0 0 150]T

Sol = λSol1 + (1- λ )Sol2 where 0≤ λ ≤1, f=-20,000

Infinite number of optimum sol.


15. Linear Programming / Simplex Methodisdl.cau.ac.kr/.../DOEOPT/15.Linear.programming.pdfSome NLP...

Documents

Transcript of 15. Linear Programming / Simplex Methodisdl.cau.ac.kr/.../DOEOPT/15.Linear.programming.pdfSome NLP...