Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms...
-
Upload
stephanie-stephany-logan -
Category
Documents
-
view
238 -
download
3
Transcript of Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms...
Discrete Optimization Lecture #3
2008/3/4 1Shi-Chung Chang, NTUEE, GIIE, GICE
Last Time1. Algorithms and Complexity
» Problems, algorithms, and complexity» Polynomial time algorithms» Intractability» NP-complete problems
2. Basic Properties of Linear Programming» Forms of LP» Basic feasible solutions» Geometry of LP
3. The Revised Simplex Method
Today1. The R evised Simplex Algorithm
» Basics» The algorithm» Getting an Initial Feasible Solution
2. Computational Aspects» Updating the basis» Convergence and Degeneracy» The Revised Simplex Method
3. Duality» Dual LP» The Duality Theorem» Complementary Slackness» The Dual Simplex Algorithm
2008/3/4 2Shi-Chung Chang, NTUEE, GIIE, GICE
Reading Assignments:1. Sections 3.1-3.3, 4.1-4.2 of [PaS 8 3]2. Sections 4.1-4.5 of [Lue 8 4]
Next Time: sections 4.3,4.4 and Chapter 5 of [PaS 83]
2008/3/4 3Shi-Chung Chang, NTUEE, GIIE, GICE
§2.2 The Revised Simplex MethodThe simplex method for LP : G.B. Dantzig 1951Key idea:
Phase1:find a bfs of
Note that an LP may not have a solution
Example min s.t.
0x
bxA
21 xx
0,
42
21
21
xx
xx
2008/3/4 4Shi-Chung Chang, NTUEE, GIIE, GICE
Phase 2Allow one of the zero components of the bfs to become positive and force one of the original positive components to become
zero.=> How to pick “entering” and “leaving” component Cost
Traditional form of the simplex method: Tableau=> read by yourself
Here we consider matrix form for conciseness of presentation and later developments.
xcT
2008/3/4 5Shi-Chung Chang, NTUEE, GIIE, GICE
SLP: s.t.
Assume P(A)=m.Partition A=[B:D] B: m linearly indep. columns of A (Assume the first m
cols.)
(SLP)
subject to (3.1)
xcmin
0
x
bxA
bx
xDB
A
B
:
mn
D
m
B
Rx
Rx
ZxCxC DDBB min
0,0
DB
DB
xx
bxDxB
2008/3/4 6Shi-Chung Chang, NTUEE, GIIE, GICE
If and a basic solutionWhen from (3.1)
(note that may not ≥0 ) (3.2)
Substituting(3.2) into the cost function
(3.3)
Define (3.4) as the relative cost vector=> To minimize Z, we need only adjust
Q: How?
0Dx bBx B
1
,0Dx
BxDB xDBbBx 11
DBDBDDDBxDBCCbBCxCxDBbBCZ 1111
DBccrBDD
1
Dx
2008/3/4 7Shi-Chung Chang, NTUEE, GIIE, GICE
Example
subject to =2 + =5
+ =6
In matrix form.
32133min xxx
4321 2 xxxx
32132 xxx 5
x321
22 xxx 6
x6,,1,0 ix
i
6100122
5010321
2001112654321 baaaaaa
000313 c
0B
0
Bc
2008/3/4 8Shi-Chung Chang, NTUEE, GIIE, GICE
0
0
0
0
0
0
1
1
1
)( 100BCB
6
5
2
6
5
4
0
x
x
x
x B
0
0
0
3
2
1
0
x
x
x
xD
2008/3/4 9Shi-Chung Chang, NTUEE, GIIE, GICE
3130 Dcr D
Define the vector of simplex multipliers
Compute in two steps:
Assume that is non degenerate If is optimal and it does not pay to increase
If then the cost Z can possible be decreased by increasing
1 BcB
'r
Dcr
cB
D
B
D
B
x
xx
xr ,0
,0Dxr 0
Dx
)0)(( iBx
,0 kr
kmx
2008/3/4 10Shi-Chung Chang, NTUEE, GIIE, GICE
Dx
Q: (1) any limitations on increasing ? (2) If there are more than one component of r < 0 => which should we change to increase?
Simplex method chooses one entering variable, Normally one with the most negative
Q: which component should leave?the first one to reach 0 => satisfied again. bxA
kr
kmx
2008/3/4 11Shi-Chung Chang, NTUEE, GIIE, GICE
If we consider bringing into the basis, i.e. , as the entering variable
=>
=>
1x
362
55
122
161
151
141
xxx
xxx
xxx
4
4
1
4,4,0,1
6
5
1
1
6541
x
x
x
x
xxxx
B 0
4
3
2
1
x
x
x
x D
2008/3/4 12Shi-Chung Chang, NTUEE, GIIE, GICE
1a
Mathematically, let be the new value vector of=>
=>
The ith component of is zero when
i.e. when
Bx
Bx
BkkB
kBb
xbBxaBx
bxaxB
11
yxxbBx
xbByxx
kBB
BBB
.
.1
1
Bx iikbByx 1
i
Bi
i
i
k y
x
y
bBx
1
2008/3/4 13Shi-Chung Chang, NTUEE, GIIE, GICE
To maintain feasibility, the smallest of these ratios for i where > 0 determines how large can be. Say, it happens at l th components of
The new bfs has and
leaves the basis while enters
iyk
x
i
Bi
i
l
Bl
y
x
ymi
y
x
01
min
x 0
lx 0
kx
la ka
0Bx
R earranging , w e can get x
2008/3/4 14Shi-Chung Chang, NTUEE, GIIE, GICE
Revised Simplex Algorithm( one iteration)Step 1: Giving is the basis B such thatStep 2: Solve to get Step 3 : Calculate If r ≥ 0 => optimal solution achieved, STOP
Find K = arg
Step 4 : Solve for y Step 5 : Find
If , => unbounded solution, STOPStep 6 : Update the basic solution
Step 7 : Update the basis Return to step 2
01 bBx B
BcB Dcr D
iri
min
kayB
i
Bi
il
Bl
y
x
ymiy
x
0,1
min
k
iii
x
kiyxx ,
0i
y
2008/3/4 15Shi-Chung Chang, NTUEE, GIIE, GICE
How to Update BasisNote that in the simplex algorithm we need to solve
(step 2) (step 4)
Where B’s differ by only one column between any two subsequent iterations.=> How to calculate B’s efficiently?
B
T CB kayB
2008/3/4 16Shi-Chung Chang, NTUEE, GIIE, GICE
Recall
(rank one update)
Tllkoldkloldnew eaaBaaBB )( col.col.
11
1
1111 )(
)(1
)(
oldl
Tllkold
lkold
Tl
oldTllkold
oldnew By
eaaBI
aaBe
BeaaBBB
Tll
ll
Tl ee
yyey
IE1
2008/3/4 17Shi-Chung Chang, NTUEE, GIIE, GICE
llold
kold
eaB
yaB
1
1Note:
=>
1
0
010
01
001
2
1
l
m
l
l
l
yy
y
yy
yy
11 oldnew
EBB Product Form of the inverse(PFI)
2008/3/4 18Shi-Chung Chang, NTUEE, GIIE, GICE
Some Computational AspectsLet be the elementary matrices after K pivot iterations.
=>
• For large-scale problems, store ‘s as vector and update and y as followers:
kEEE ,,,
21
1
011
1
BEEEBkkk
iE
kk
kkB
T
aBEEy
BEEEc1
01
1
011)(
2008/3/4 19Shi-Chung Chang, NTUEE, GIIE, GICE
• What if is small in pivoting? Use LU or QR decompositions in computing If B is the current basis
Note that
ly
1B
kmllnew
oldoldoldm
aaaaaaB
BULaaaB
1121
21
newoldBLH 1
koldmll aLuuuuu 1
1121
=> upper Hessenberg
2008/3/4 20Shi-Chung Chang, NTUEE, GIIE, GICE
Using a sequence of elimination steps on H:
• Store • QR-decomposition
Do givens transformation on H
• The simplex method is theoretically an exponential algorithm In practice, 2(n + m) iterations, i.e., a linear function of (n + m)
newmoldnew
emnew
UMMLB
HMMU11
1
1
11
1
oldemnew LMML
)( 111 kmeenew aaaaaB
HBQ newT
old
1
1
meoldnew
newTe
Tm
QQQQ
RHQQ
m
n
2008/3/4 21Shi-Chung Chang, NTUEE, GIIE, GICE
Convergence and DegeneracyConvergence:
If the objective function value strictly decreases after every iteration, the simplex algorithm never repeats a basis and it converges in a finite number of iterations.
why?
Degeneracy:If for a such that
=> x is a degenerate
ixbfs ,
0)()( 1
iiB bBx
2008/3/4 22Shi-Chung Chang, NTUEE, GIIE, GICE
Property:in the pivoting step,
=> i.e. no change in cost.
=> Problem: cycling among degenerate=> no guarantee of convergence
=> need an anti-cycling algorithm
oldioldnewZYZZ
bfses
0)(
i
iB
y
x
2008/3/4 23Shi-Chung Chang, NTUEE, GIIE, GICE
How to get an initial feasible solution?
(p)
(PH1)
subject to
0
x
bxAmn
m
iii
zy1
min
0
0
0
z
y
x
bzIyIxAmm
mRzy ,
2008/3/4 24Shi-Chung Chang, NTUEE, GIIE, GICE
• (P) has a feasible solution in (PH1)Q: • can solve (PH1) using revised simplex with
as the initial feasible solution
izyii
,0,0
,0x
2008/3/4 25Shi-Chung Chang, NTUEE, GIIE, GICE
otherwise.
0 if
0ii
i
bby
otherwise.
0 if
0ii
i
bbz
Solving S-P by the Revised Simplex
Subject to
* Flow balance eq. corresponding to * is dropped due to redundancy.
xxmix
52213312
0
0
0
0
1
111
111
111
111
11
4
3
2
1
)(0
8
7
6
5
4
3
2
1
87654321
x
x
x
x
x
x
x
x
eeeeeeees
8,,1,0 jxj
2008/3/4 26Shi-Chung Chang, NTUEE, GIIE, GICE
Chapter 3 Duality§ 3 .1 Dual LP
Consider (SLP)
Subject to
2008/3/4 27Shi-Chung Chang, NTUEE, GIIE, GICE
minc x
x
0
,
.
Ax b
x
Dual Viewpoint
• From the optimality condition, if optimal solution to SLP => corresponding partition of A matrix
such that
*x[ *: D*] = B A
(3.5)
where *' (3.6)
' ' 1* *
' 1*
( * ) 0
*
D BD
B
r c c B D
c B
Note that in Simplex Method, we have' (3.7)
where [ : and (
' 1
] ) .
mBc B R
B D A B m
Dual Viewpoint (Cont.)
' '[
[
= [
' 1
' ' 1
: ]
: ]
: ]
B
B B
A B D
c B B D
c c B D
and .Dc D
' [
[ = ' (3.8)
' ' 1
' '
: ]
: ]
B B
B D
A c c B D
c c c
Note that
The ’ we are looking for must satisfy (3.5) for anyThe root of (3.5) comes from So,
Note that ’ in (3.5) and (3.6) satisfies (3.7) and (3.8).
0.x
Dual Objective FunctionNow consider the objective function
From inequality (3.8) that then for any bfs
’ is the one that leads to the maximum value while satisfying (3.7) and (3.8)
' ' (3.9a)
(3.9b)
( = 0) (3.9c)
**' *'
*
*' *' *' *1 1
*' *' *' *
*' *
* [ : ]
* ( * )
( )
BB D
D
B D B D
D D
D
xc x c x c c
x
c B b c c B D x
b c D x
b x
'' cA 0xwe have
xcbxA ''' bxcbxA *'*''*' and from (3.9)
b'
Dual SLP
(SLP)
Subject to
2008/3/4 31Shi-Chung Chang, NTUEE, GIIE, GICE
minc x
x
0
,
.
Ax b
x
(DSLP)
Subject to
max b
,
.
mR
A c