Post on 30-May-2018
8/14/2019 Linear Programming (LP) Linear Programming (LP)
1/25
Optimization in Engineering Design
1
Georgia Institute of Technology
Systems Realization Laboratory
Linear Programming (LP)
Linear Programming (LP)
8/14/2019 Linear Programming (LP) Linear Programming (LP)
2/25
Optimization in Engineering Design
2
Georgia Institute of Technology
Systems Realization Laboratory
Why Talk About Linear Programming?
Why Talk About Linear Programming?
LP is simpler than NLP, hence, good for a foundation
Linearity has some unique features for optimization
A lot of problems are or can be converted to a LP formulation
Some NLP algorithms are based upon LP simplex method
8/14/2019 Linear Programming (LP) Linear Programming (LP)
3/25
Optimization in Engineering Design
3
Georgia Institute of Technology
Systems Realization Laboratory
Bolted Joint Design
Bolted Joint Design
GivenAt - tensile strength area, function of d
Db - bolt circle diameter
Pt - total load
C - joint constant
Fi - preload (= 0.75 Sp At)Find
N - number of bolts, Sp - proof strength, d - diameter
Satisfy
3d Db / N good wrench rule Db / N 6d good seal ruleC Pt / N Sp At - Fi static loading constraint
Fi (1 - C) Pt / N joint separation constraint
Minimize Z= [ f1(N, d, Sp), f2(N, d, Sp), ..]
Question:
Is this a linear ornonlinear model?
Question:
Is this a linear or
nonlinear model?
8/14/2019 Linear Programming (LP) Linear Programming (LP)
4/25
Optimization in Engineering Design
4
Georgia Institute of Technology
Systems Realization Laboratory
Bolted Joint Design (2)
Bolted Joint Design (2)
Given
d - diameterAt - tensile strength area, function of d
Db - bolt circle diameter
Pt - total load
C - joint constant
Fi - preload (= 0.75 Sp At)
Find
N - number of bolts, Sp - proof strength
Satisfy
3d Db / N good wrench rule
Db / N 6d good seal rule
C Pt / N Sp At - Fi static loading constraint
Fi (1 - C) Pt / N joint separation constraint
Minimize Z= [ f1(N, Sp), f2(N, Sp), ..]
Question:
Is this a linear ornonlinear problem?
Question:
Is this a linear or
nonlinear problem?
8/14/2019 Linear Programming (LP) Linear Programming (LP)
5/25
Optimization in Engineering Design
5
Georgia Institute of Technology
Systems Realization Laboratory
Historical Perspective
Historical Perspective
1928 John von Neumann published related central theorem of gametheory
1944 Von Neumann and Morgenstern published Theory of Games andEconomic Behavior
1936 W.W. Leontief published "Quantitative Input and Output Relations
in the Economic Systems of the US" which was a linear model withoutobjective function.
1939 Kantoravich (Russia) actually formulated and solved a LPproblem
1941 Hitchcock poses transportation problem (special LP)
WWII Allied forces formulate and solve several LP problems related tomilitary
A breakthrough occurred in 1947...
8/14/2019 Linear Programming (LP) Linear Programming (LP)
6/25
Optimization in Engineering Design
6
Georgia Institute of Technology
Systems Realization Laboratory
SCOOP
SCOOP
US Air Force wanted to investigate the feasibility of applyingmathematical techniques to military budgeting and planning.
George Dantzig had proposed that interrelations between activities of alarge organization can be viewed as a LP model and that the optimalprogram (solution) can be obtained by minimizing a (single) linearobjective function.
Air Force initiated project SCOOP (Scientific Computing of OptimumPrograms)
NOTE:
SCOOP began in June 1947 and at the end of the same summer, Dantzig
and associates had developed:1) An initial mathematical model of the general linear programming
problem.
2) A general method of solution called the simplex method.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
7/25
Optimization in Engineering Design
7
Georgia Institute of Technology
Systems Realization Laboratory
Simplex TodaySimplex Today
A large variety of Simplex-based algorithms exist to solveLP problems.
Other (polynomial time) algorithms have been developed forsolving LP problems:
Khachian algorithm (1979)
Kamarkar algorithm (AT&T Bell Labs, mid 80s)
See Section 4.10
BUT,
none of these algorithms have been able to beat Simplex inactual practical applications.
HENCE,
Simplex (in its various forms) is and will most likely remainthe most dominant LP algorithm for at least the near future
8/14/2019 Linear Programming (LP) Linear Programming (LP)
8/25
Optimization in Engineering Design
8
Georgia Institute of Technology
Systems Realization Laboratory
Fundamental TheoremFundamental Theorem
Extreme point (or Simplex filter) theorem:
If the maximum or minimum value of alinear function defined over a polygonalconvex region exists, then it is to be found
at the boundary of the region.
Extreme point (or Simplex filter) theorem:
If the maximum or minimum value of alinear function defined over a polygonalconvex region exists, then it is to be found
at the boundary of the region.
Convex set:
A set (or region) is convex if, for any two points (say, x1and x2) in that set, the line segment joining these points
lies entirely within the set.
A point is by definition convex.
Convex set:
A set (or region) is convex if, for any two points (say, x1and x2) in that set, the line segment joining these points
lies entirely within the set.A point is by definition convex.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
9/25
Optimization in Engineering Design
9
Georgia Institute of Technology
Systems Realization Laboratory
What does the extreme point theorem imply?What does the extreme point theorem imply?
A finite number of extreme points implies a finite numberof solutions!
Hence, search is reduced to a finite set of points
However, a finite set can still be too large for practicalpurposes
Simplex method provides an efficient systematic searchguaranteed toconverge in a finite number of steps.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
10/25
Optimization in Engineering Design
10
Georgia Institute of Technology
Systems Realization Laboratory
Basic Steps of SimplexBasic Steps of Simplex
1. Begin the search at an extreme point (i.e., a basic feasible
solution).
2. Determine if the movement to an adjacent extreme canimprove on the optimization of the objective function. If not,the current solution is optimal. If, however, improvement is
possible, then proceed to the next step.
3. Move to the adjacent extreme point which offers (or,perhaps, appears to offer) the most improvement in theobjective function.
4. Continue steps 2 and 3 until the optimal solution is found or itcan be shown that the problem is either unbounded orinfeasible.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
11/25
Optimization in Engineering Design
11
Georgia Institute of Technology
Systems Realization Laboratory
Step 0 Obtain Canonical FormStep 0 Obtain Canonical Form
General Simplex LP model:
min (or max) z = ci xi
s.t.
A x = b
x 0In order to get and maintain this form, use
slack, if x b, then x + slack = b
surplus, if x b, then x - surplus = b
artificial variables (sometimes need to be added to ensure all
variables 0, see page 101)
Compare constraint conversion with goalconversions using deviation variables
Compare constraint conversion with goalconversions using deviation variables
IMPORTANT: Simplex only deals with equalities
8/14/2019 Linear Programming (LP) Linear Programming (LP)
12/25
Optimization in Engineering Design
12
Georgia Institute of Technology
Systems Realization Laboratory
Different "components" of a LP modelDifferent "components" of a LP model
LP model can always be split into a basic and a non-basicpart.
Transformed or reduced model is another good way
to show this.
This can be represented in mathematical terms as well asin a LP or simplex tableau.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
13/25
Optimization in Engineering Design
13
Georgia Institute of Technology
Systems Realization Laboratory
Movement to Adjacent Extreme PointMovement to Adjacent Extreme Point
Given any basis we move to an adjacent extreme point(another basic feasible solution) of the solution space byexchanging one of the columns that is in the basis for acolumn that is not in the basis.
Two things to determine:
1) which (nonbasic) column of A should be brought intothe basis so that the solution improves?
2) which column can be removed from the basis such thatthe solution stays feasible?
8/14/2019 Linear Programming (LP) Linear Programming (LP)
14/25
Optimization in Engineering Design
14
Georgia Institute of Technology
Systems Realization Laboratory
Entering and Departing Vector (Variable) RulesEntering and Departing Vector (Variable) Rules
General rules:
The one non-basic variable to come in is the one whichprovides the highest reduction in the objective function.
The one basic variable to leave is the one which isexpected to go infeasible first.
NOTE: THESE ARE HEURISTICS!!
Variations on these rules exist, but are rare.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
15/25
Optimization in Engineering Design
15
Georgia Institute of Technology
Systems Realization Laboratory
Simplex VariationsSimplex Variations
Various variations on the simplex method exist:
"regular" simplex (see Section 4.4)
two-phase method: Phase I for feasibility and Phase II
for optimality (see Section 4.5.1) condensed/reduced/revised method: only use the non-
basic columns to work with (see Section 4.6)
(revised) dual simplex (see Section 4.8), etc.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
16/25
Optimization in Engineering Design
16
Georgia Institute of Technology
Systems Realization Laboratory
Computational ConsiderationsComputational Considerations
Unrestricted variables (unboundedness)
Redundancy (linear dependency, modeling errors)
Degeneracy (some basic variables = 0)
Round-off errors
8/14/2019 Linear Programming (LP) Linear Programming (LP)
17/25
Optimization in Engineering Design
17
Georgia Institute of Technology
Systems Realization Laboratory
Limitations of SimplexLimitations of Simplex
1. Inability to deal with multiple objectives
2. Inability to handle problems with integer variables
Problem 1 is solved using Multiplex
Problem 2 has resulted in:
Cutting plane algorithms (Gomory, 1958)
Branch and Bound (Land and Doig, 1960)
However,solution methods to LP problems with integer or Booleanvariables are still far less efficient than those which includecontinuous variables only
8/14/2019 Linear Programming (LP) Linear Programming (LP)
18/25
Optimization in Engineering Design
18
Georgia Institute of Technology
Systems Realization Laboratory
Example ProblemExample Problem
Maximize Z = 5x1 + 2x2 + x3
subject to
x1 + 3x2 - x3 6,
x2 + x3 4,
3x1 + x2 7,
x1, x2, x3 0.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
19/25
Optimization in Engineering Design
19
Georgia Institute of Technology
Systems Realization Laboratory
Simplex and Example ProblemSimplex and Example Problem
Step 1. Convert to Standard Form
a11 x1 + a12 x2 + + a1n xn b1,
a21 x1 + a22 x2 + + a2n xn b2,
am1 x1 + am2 x2 + + amn xn bm,
a11 x1 + a12 x2 + + a1n xn + xxn+1n+1 = b1,
a21 x1 + a22 x2 + + a2n xn - xxn+2n+2 = b2,
am1 x1 + am2 x2 + + amn xn + xxn+kn+k = bm,
In our example problem:
x1 + 3x2 - x3 6,
x2 + x3 4,
3x1 + x2 7,
x1, x2, x3 0.
x1 + 3x2 - x3 + x4 = 6,
x2 + x3 + x5 = 4,
3x1 + x2 + x6 = 7,
x1, x2, x3, x4, x5, x6 0.
8/14/2019 Linear Programming (LP) Linear Programming (LP)
20/25
Optimization in Engineering Design
20
Georgia Institute of Technology
Systems Realization Laboratory
Simplex: Step 2Simplex: Step 2
Step 2. Start with an initial basic feasible solution (b.f.s.) and set up the
initial tableau.
In our example
Maximize Z = 5x1 + 2x2 + x3
x1 + 3x2 - x3 + x4 = 6,
x2 + x3 + x5 = 4,
3x1 + x2 + x6 = 7,
x1, x2, x3, x4, x5, x6 0.cj
5 2 1 0 0 0
cB Basis
x1 x2 x3 x4 x5 x6
Constants
0 x4 1 3 -1 1 0 0 6
0 x5 0 1 1 0 1 0 40 x6 3 1 0 0 0 1 7
c row 5 2 1 0 0 0 Z=0
8/14/2019 Linear Programming (LP) Linear Programming (LP)
21/25
Optimization in Engineering Design
21
Georgia Institute of Technology
Systems Realization Laboratory
Step 2: ExplanationStep 2: Explanation
Adjacent Basic Feasible Solution
If we bring a nonbasic variable xs into the basis, our system changes fromthe basis, xb, to the following (same notation as the book):
x1 + 1s xs=
xr + rsxr=
xm + ms xs=
1b
rb
sb
xi = for i =1, , misi ab
xs = 1
xj = 0 for j=m+1, ..., n and j s
The new value of the objective function becomes:
=
+=m
1i
sisii c)ab(cZ
Thus the change in the value of Z per unit increase in xs issc = new value of Z - old value of Z
= ==
+m
1i
ii
m
1i
sisii bcc)ab(c
=
m
1i
isis acc=
This is the Inner Product rule
8/14/2019 Linear Programming (LP) Linear Programming (LP)
22/25
Optimization in Engineering Design
22
Georgia Institute of Technology
Systems Realization Laboratory
Simplex: Step 3Simplex: Step 3
Use the inner product rule to find the relative profit coefficients
cj5 2 1 0 0 0
cB Basis
x1 x2 x3 x4 x5 x6
Constants
0 x4 1 3 -1 1 0 0 6
0 x5 0 1 1 0 1 0 40 x6 3 1 0 0 0 1 7
c row 5 2 1 0 0 0 Z=0
jBjj Pccc =
c1 = 5 - 0(1) - 0(0) - 0(3) = 5 -> largest positivec2 = .
c3 = .
Step 4: Is this an optimal basic feasible solution?
8/14/2019 Linear Programming (LP) Linear Programming (LP)
23/25
Optimization in Engineering Design
23
Georgia Institute of Technology
Systems Realization Laboratory
Simplex: Step 5Simplex: Step 5
Apply the minimum ratio rule to determine the basic variable to leave the basis.
The new values of the basis variables:
xi = for i = 1, ..., msisi xab
= >is
i0a
sa
bminxmaxis
In our example:
Row Basic Variable Ratio
1 x4 62 x5 -
3 x6 7/3
8/14/2019 Linear Programming (LP) Linear Programming (LP)
24/25
Optimization in Engineering Design
24
Georgia Institute of Technology
Systems Realization Laboratory
Simplex: Step 6Simplex: Step 6
Perform the pivot operation to get the new tableau and the b.f.s.
jBjj Pccc=
c2 = 2 - (0) 8/3 - (0) 1 - (5) 1/3 = 1/3
c3 = 1 - (0) (-1) - (0) 1 - (5) 0 = 1
c6 = 0 - (0) 0 - (0) 0 - (5) 1/3 = -5/3
cB = (0 0 5)
New iteration:find enteringvariable:
8/14/2019 Linear Programming (LP) Linear Programming (LP)
25/25
Optimization in Engineering Design
25
Georgia Institute of Technology
Systems Realization Laboratory
Final TableauFinal Tableau
x3 enters basis,x5 leaves basis
Wrong value!4 should be 11/3