Sen Advances in Stochastic Mixeda Integer Programming (1)

7/28/2019 Sen Advances in Stochastic Mixeda Integer Programming (1)

1/58

dData Driven Decisions @ The Ohio State UniversityIndustrial and Systems Engineering

Advances in Stochastic Mixed Integer Programming

Lecture at the INFORMS Optimization Section Conference inMiami, February 26, 2012

Suvrajeet Sen

Data Driven Decisions Lab

Integrated Systems Engineering

Ohio State University


2/58

d

Some Historical Remarks

Classification of SMIP

SMIP Models: Risk, Recourse, Resilience

Structural Properties

Decomposition: Benders and Beyond

Illustrative Computational Results

Overview of this Lecture


3/58

d

Historical Remarks: IOS

Age of the INFORMS Optimization Section is

a) 0 < age 10

b) 10 < age 15

c) 15 < age 20

d) 20 < age 25

e) age > 25

INFORMS OS was founded at the SpringORSA/TIMS Meeting in Los Angeles, April

1995.


4/58

d

Historical Remarks:

My assessment of SIP/SMIP

Stochastic

Integer

Programming

Integer

Programming

Linear

Programming

Stochastic

Linear

Programming

Discrete Choice

Discrete Choice

1950s-

Present

1960s -

Present

Uncertainty Uncertainty

Major

Hurdles StillRemain!!


5/58


6/58

d

Some data: [19802000) prior to 2000

annotated bibliography (Stougie and van der Vlerk)

Theory: (7+) papers

Simple Integer Recourse: 2 Structural Properties of Expected Recourse Function: 4

Complexity: (1+)

General Purpose Algorithms: 17 papers

Benders-type methods: 5 Grobner-basis methods: 2

Convex Approximations for Simple Integer Recourse: 2

Other: 8 (Sampling with first-stage integer, disjunctive cuts)

More Historical Remarks:Walk Before You Can Run


7/58

d

Books/Surveys: 6 altogether

Dissertations: 3-4 (1 prior to 2000 in North America)

Habilitation: 1

Published surveys: 3 (includes hierarchical planning)

Special Purpose Models/Algorithms: 25 papers

Production Planning/Scheduling: 3

Network and Routing: 11

Location: 7

Other: 4

In the 12 years: more than 350 articles listed in

http://mally.eco.rug.nl/index.html?BIBLIO/SIP.HTML

More Historical Remarks:Walk Before You Can Run [1980-2000)


8/58

d

Schultz, R (2003) Stochastic Programming with

Integer Variables, Mathematical Programming-B, 285-309

Stougie, L. and M.H. van der Vlerk (2005)

Approximation in Stochastic Integer Programming

Sen, S. (2005) Stochastic Mixed-Integer

Programming Algorithms, Handbook of Discrete

Optimization, (Aardal, Nemhauser, Weismantel, eds.)

. Some newer surveys are also available .

Now we are running (Survey Articles 2000-)


9/58

d

And you know were serious because of

applications with realistic data

Manufacturing Supply Chain: Two-stage Design(IBM, Intel)

Biofuel Supply Chain: Multi-stage Design (Fan et al)

Homeland SecurityDefender/Attacker/Defender

(Wood et al NPS, Ordonez/Tambe, Smith) Electric PowerUnit Commitment (Birge/Takriti,

Philpott, Guan/Zhang), Fuel Price Hedging(Sen etal)

MilitaryPrioritizing Choices (Morton), UAV/MAV(Evers et al)

Fighting Forest Fire (Ntaimo)


10/58

d

SMIP Classification:

A (B-C-D-E) Notation for SMIP

Two Stage Stochastic Linear Programming

Min cTx + E[f(x, )]

Ax = b, x 0

where,

f(x, ) = Min gTy

Wy r()T()xy 0

Variations depend on where the randomness appears


11/58

d

Stochastic MIP with First Stage Integers

Min cTx +E[f(x, )]

Ax b, x Rn1Z

n2

where,

f(x, ) = Min gTy

Wy r()T()x

y R

n3

Zn

denotes integer vectors of length n.

With second-stage integers, extremely difficult!


12/58

d

Stochastic Combinatorial Optimization

Min cTx +E[f(x, )]Ax b, x B

n1

where,(0l18

f(x, ) = Min gTyWy r() T()x

y Rn

2Bn3

HereBn denotes binary vectors of length n.

Many different structures for SMIP!


13/58

d

Describing SMIP Problems

B = Set of stages with BinaryVars.

C = Set of stages with Continuous Vars.

D = Set of stages with Discrete Vars.(arbitrary integers, not just binary)

E = Endogenous Uncertainty (Y/N)

Louveaux has proposed a notation that covers all

SP problems (e.g. notation includes whetherrandom variables are cont/discrete)

Above notation helps clarify domain of applicabilityof results/algorithms etc.


14/58

d

Traditional Benders Decomposition

SLP: B = {}, C={1,2}, D ={}

Wollmer, Norkin et al, Poojari/Mitra:

B = {1}, C={1,2}, D ={1}

Special Structure: Simple Integer Recourse:

B = {2}, C={1}, D ={2} + structure of secondstage

Global Optimization and IP

Ahmed, Tawarmalani, Sahinidis: B = {2}, C={1,2},D ={2} ; + Fixed Tenders

Grossman & Co. (E = Y)


15/58

d

Disjunctive Programming for Two-Stage

Caroe/Tind, Sherali/Fraticelli, Sen/Higle,Sen/Sherali:

B = {1,2}, C={2}, D ={}

Ntaimo/Sen: B = {1,2}, C={1,2}, D ={}

Lagrangian-based Methods for Multi-stage

Multi-stage SMIPs: Caroe/Schultz, Roemisch et al,

Alonso-Ayuso et al, Lulli/Sen, Guan et al

B = {1,2, N}, C={1,2 N}, D ={1,2 N}b


16/58

d

SMIP Models Modeling Risk

Modeling Recourse

Modeling Resilience

Multi-stage Models

Models not Covered (Chance Constraints withDiscrete Distributions)

Special Structured IP (Knapsack, Mixing etc.) SeePrkopa, Dentcheva, Ruszczynski

Leudtke et al (2010), Kkyavuz (2010), Saxena etal (2009), Shen et al (2010)

Stochastic MIP Models:

Risk, Recourse, and Resilience


17/58

d

Risk in SMIP

We have only stated models via Expected Values

Is the reliance on Expectation a handicap?

Of course! But many risk measures (e.g. down-siderisk, mean absolute deviation, CVaR, etc.) can bere-formulated using expectation of a slightlymodified, though mathematically similar function.


18/58

d

SCO for Modeling Risk

We have only stated models via Expected Values

Is the reliance on Expectation a handicap?

Of course! But many risk measures (e.g. down-siderisk, mean absolute deviation, CVaR, etc.) can bere-formulated using expectation of a slightlymodified, though mathematically similar function

Important: Inequalities are indispensible for riskmodeling


19/58

d

SCO for Modeling Risk

Example: Kahneman/Tversky S curve for risk-

aversion can be linearized using 0-1 variables.

Similar to non-convex piecewise linear programming.

Each piece requires a binary (switch variable)

r


20/58

d

SCO for Modeling Recourse:

Stochastic Server Location Problem


21/58

d

SCO for Modeling Recourse: SSLP

This SCO has two sets of decisions:1. Choose server locations (e.g. bases)2. Once demand nodes (e.g threats) appear, then

assign servers to demand nodes


22/58

d



23/58

d

Our Stochastic Server Location Problem (SSLP) alsoincludes some policy constraints:

Policy that each customer will receive service fromonly one site has been established. Moreover,service site must be locatedwithin a prescribed

zone (z).

Max number to be located is v,with eachzonehaving no more than wzservers.



24/58

d

The SSLP model objective: minimize CostExpected

Revenue Potential (last term denotes Penalty for lostdemand)

Min j cjxjE[ijqijyij() + j QjYj()]

subject to: constraints on supply-side,j xj v, j J(z)xj wz,z

demand-side,

jyij() + Yj() = i,isupply/demand:

i yij()Yj() ujxj, j

Plus: All variables are binary



25/58

d

Modeling Resilience

Logical conditions are as follows:

y0jk 1 xj

y1jkxj


26/58

d

Multi-Stage SMIP ModelsNon-anticipativity in the Two Stage Model

(*) is the non-anticipativity constraintall scenarios must agree on first-stage


27/58

d

Two-stage:

NA only on

Root Node

Multi-stage:

Difficult, unless

Ocotillo-type Trees


28/58

d

Recursive Formulation using State Variables

Challenge ofConiferous Trees


29/58

d

SMIP with Recursive Formulation


30/58

d

Most structural properties and algorithms for

SMIP assume relatively complete and

sufficiently expensive recourse.

- < f(x,) < + with probability 1. Under the above assumption, the expected

recourse function is real-valued and lower

semi-continuous.

Structural Properties


31/58

d

Complexity of Two-Stage SMIP

Two-stage stochastic programs with recourse

having finitely many scenarios is #P-hard

(The class #P asks for the count (i.e. how many,

rather than are there any?) The proof reduces any graph reliability problem to

a two-stage stochastic combinatorial optimization

problem


32/58

d

What Do we Need?

Two Issues in Algorithm Design:- Cuts for Second Stage IP

- Approximation off (also convexification)

A Potent Brew! Decomposition

(SP) and Convexification (IP)


33/58

d

Gomory Cuts for SIP Decomposition Hot off the Printer!

First stage 0-1, Second-stage General Integer,

Disjunctive Decomposition (D2

) First stage: 0-1

Second-stage: mixed 0-1

Disjunctive Decomposition with Branch-and-Cut

(D2

-BAC) First stage 0-1

Second-stage: mixed-integer

Beyond Benders Decomposition:

Second-stage IP


34/58

d

Recall --- SLP

Two Stage Stochastic Linear Programming

Min cTx + E[f(x, )]

Ax = b, x 0

where,

f(x, ) = Min gTy

Wy r()T()x

y 0

Variations depend on where the randomness appears


35/58

d

Recall --- Benders Decomposition or

L-shaped Method

Standard Benders Master (OR 501)

Where denote Non-negative Second-stage

Dual Multipliers


36/58

d

Caroe-Tind extension of Benders or

L-shaped Method for Second-stage SIP

Gomory Cuts to represent Subadditive Value Functions

Where denote Non-decreasing Second-

stage value function approximations


37/58

d

Structure Similar toBenders

And a More General Framework

But Need to overcome bottlenecks

Subproblems are Integer Programs

Master Problems are required to Optimize Non-

convex functions


38/58

d

Our Recommendation: maintain Benders

piecewise linear approximations

Notice the change below!

Where denote Non-negative Second-stage

Dual Multipliers

Notice that

RHS r has

changed to

and T to


39/58

d

Our Suggestion: Solve Second

stage usingUpdated LP

approximations

Each iteration will involve only LP solutions in

the second-stage

Solve LP relaxation TWICE

Once solve with an Old Convexification

Derive a Cut to Update the Convexification

We will have

First-stage is same as Benders original proposal

Second-stage are LPs.


40/58

d

But can this be achieved? Yes

under certain assumptions!

First stage pure binary (B = {1,2})

C = , D={2} Use Gomory Cuts (Gade,Kkyavuz, Sen)

If C= {2}, B = {2} Use Disjunctive Set Convexification(Sen and Higle)

If C= {2}, D = {2} Use Disjunctive ValueApproximations for Branch-and-Cut (Sen and Sherali)

First stage general MILP (Global Optimization)

{}


41/58

d

Second Stage Set Convexification

Original ConstraintsValid Inequalities as Functions of x

Parametric Gomory Cuts: Affine

Parametric Disjunctive Cuts:

Piecewise LinearConcave


42/58

d


43/58

d


44/58

d


45/58

d


46/58

d


47/58

d


48/58

d


49/58

d


50/58

d


51/58

d

Parametric Gomory Cuts

Finiteness with Lexicographic Dual Simplex(Gade, Kkyavuz, Sen)

Scen Obj Vars Cons GDD-S GDD-R B&BNodes B&B + GomNodes

4 -63.50 22 24 7 (13) 7 (32) 54 2 (6)

9 -66.17 47 54 7 (39) 6 (76) 306 8 (13)

36 -67.33 182 216 10 (183) 6 (384) 1.55E7 52 (50)

121 -67.67 607 726 9 (526) 6 (1032) 7.60E6 13224 (167)


52/58

d

Convexify0(x,)by viewing its epigraph as adisjunctive set such as the one shown below.

0(x,)

0 1First stage binary variable

Parametric Disjunctive Cuts

C f Di j i D i i


53/58

d

Convergence for Disjunctive Decomposition

(Set Convexification)

AssumptionsComplete recourseAll integer variables are 0-1Maintain all cuts in Wk

Certain rules of order hold (a la

lexicographic dual simplex in Gomorysproof)

Under these assumptions, the D2methodresults in a convergent algorithm (Sen and Higle).


54/58

d

0(x,)

Value Approximations for Branch-and-Cut in Second Stage (Sen and Sherali)

There will be one piece per node of a

truncated BAC tree in the second-stageDisjunctive Programming lets us

convexify the function (for each outcome )


55/58

d55

Illustrative Computational

Results with D2 and D2-BACComputational Results for Problem Instance SSLP_10_50

CPU Time (secs.)

Scenarios Binaries Constraints % ZIP Gap Iterations D Cuts D

L

DEP DEP % Gap

5 2,510 301 10.49 209 189 78.25 997.74 80.53

10 5,010 601 11.38 264 257 171.49 1284.47 Failed 0.19

25 12,510 1,501 10.81 286 281 248.81 1339.24 Failed 0.34

50 25,010 3,001 10.89 252 250 295.95 1982.60 Failed 0.44

100 50,010 6,001 11.07 300 299 480.46 2782.88 Failed 9.02

500 250,010 30,001 10.75 309 307 1902.20 Failed Failed 38.17

1,000 500,010 60,001 11.07 322 321 5410.10 Failed Failed 99.602,000 1,000,010 120,001 11.01 308 307 9055.29 Failed Failed 46.24

SCALABILITY:D2 scales well with increase in number of scenarios

(linear)D2 does not scale well with increase in size of master

program (x

)

Computational Results for Problem Instance SSLP_15_45

CPU Time (secs.)

Scenarios Binaries Constraints % ZIP Gap Iterations D Cuts D

L

DEP DEP % Gap

5 3,390 301 6.88 146 145 110.34 Failed Failed 1.19

10 6,765 601 6.53 454 453 1,494.89 Failed Failed 0.27

15 10,140 901 5.62 814 813 7,210.63 Failed Failed 0.72


56/58

d56

T = 4.6631S

R2

= 0.9888

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0 500 1000 1500 2000 2500

Number of Scenarios S

CPUTime(s)

T

Computational Results Cont

The D2 Algorithm

Solves some of the largest (0-1) instancesScalability - Linear in the number of scenarios

D2 CPU time for SSLP_10_50 with 100

scenarios

C l l h )


57/58

d

Treating Cut Generation as a SpecializedTwo-Stage LP

Computational Results (with Y. Yuan)

Instance D2 D2-BAC D2-BAC++

5.25.50 1.64 0.70 0.36

5.25.100 2.15 1.73 0.89

5.50.100 7.10 3.70 1.565.50.500 34.50 23.05 12.36

5.50.1000 140.47 64.17 22.77

5.50.2000 603.37 274.40 42.74

10.50.50 295.95 373.98 262.13

10.50.100 396.76 452.31 486.99

10.50.500 1902.2 2772.22 1313.38

10.50.1000 5410.1 5677.80 2139.4710.50.2000 9055.29 >10800 3916.47

15.45.5 110.34 232.30 211.79

15.45.10 1494.89 222.41 153.41

15.45.15 7210.63 1988.26 803.56

C l i


58/58

Conclusions

Decomposition (SP) + Valid Inequalities (IP)

provide a potent potion!

But

Stochastic MIP still needs a lot of work

Specially structured cuts (already at play in

Chance Constrained SP)

Multi-stage extensions (very rich area) Real-world Applications

.

Sen Advances in Stochastic Mixeda Integer Programming (1)

Documents

Transcript of Sen Advances in Stochastic Mixeda Integer Programming (1)