Leveraging Big Data for Robust Process Operation Under Uncertainty

Fengqi You

Joint work with graduate student Chao Ning

Process-Energy-Environmental Systems Engineering (PEESE)School of Chemical and Biomolecular Engineering

Cornell University, Ithaca, New York

www.peese.org

• Let Uncertainty Data “Speak” in Math Programming• Data-driven stochastic programming [Jiang, 13]• Data-driven chance constraint programming [Guan, 15]• Distributionally robust optimization [Delage & Ye, 10]• Data-driven static/adaptive robust optimization

• When Big Data “meets” Robust Optimization• Data-driven static robust optimization [Bertsimas et al., 13]• Adaptive/adjustable robust optimization better balances

conservativeness and has high computational tractability

• Objective: A novel data-driven adaptive robust optimization framework – fill the knowledge gap

Data-Driven Decision Making under Uncertainty

2

• Two main components• Decisions: All the decisions are made “here-and-now”• Uncertainty set: Often constructed based on a priori

and relatively simple assumptions about uncertainty

• Drawback: Solution could be overly conservative

Background: Static Robust Optimization

3

0min max ,

s.t. , 0 ,U

i

f

f U i

x u

x u

x u u

Uncertainty setDecisions

• “Wait-and-see” decisions made after uncertainty is revealed• Well represents sequential decision-making problems• Less conservative than Static Robust Optimization• Recourse decisions address feasibility issues

Two-Stage Adaptive Robust Optimization (ARO)

4

( , )min max min

. . ,

, :

T T

U

s t

x y x uu

x

y

c x b y

Ax d x S

x u y S Wy h Tx Mu “wait-and-see” decisions

“here-and-now” decisions

“here-and-now” decisions

Uncertainty “wait-and-see” decisions

• Linear decision rule

• After replacing y with the linear decision rule, we can solve it as a Static Robust Optimization

• Less adaptive but computationally inexpensive

Decision Rules for ARO

5

y q Qu qQ

“here-and-now” decisions

An affine function of uncertainty

Example When Affine Decision Rule Fails to Work

6

ARO with affine decision rule (AARC)

ARO with general decision rule

Worst-case profit 0 6,600

“over-conservatism by refusing to open any of

the facilities”

Location-Transportation Problem

,max min max

. . , ,

, ,

0, , ,

0 , 0,1 ,

i i i i ij ijUx v yi i i j

ij ji

ij ij

ij

i i

i

c x k v y

s t y j U

y x i U

y i j U

x Mv iv i

• Ardestani-Jaafari, Amir, and Erick Delage. “Linearized RobustCounterparts of Two-stage Robust Optimization Problem withApplications in Operations Management.” 2016

Demand Uncertainty

5.9 5.6 4.95.6 5.9 4.9

ˆ , 0 1, ,j j j j j jj

U j

0.6130,000

icM100,000ik

The parameters of costs

Uncertainty set

ˆ20,000 18,000 2j j

Example: 2 facilities and 3 customers

• Linear decision rule

• After replacing y with the linear decision rule, we can solve it as a Static Robust Optimization

• Less adaptive but computationally inexpensive

• Generalized decision rule (This work)

• Fully adaptive• Challenging to solve

Decision Rules for ARO

7

y y u

y q Qu

A general function of uncertainty, determined

by optimization

qQ

“here-and-now” decisions

An affine function of uncertainty

“wait-and-see” decisionsy

min

s.t.

T

y

y

b y

y S

Wy h Tx Mu

• Box Uncertainty Set• Soyster (1973)

• Pros: Tractable• Cons: Very conservative

• Budgeted Uncertainty Set• Bertsimas and Sim (2003)

• Pros: Control conservatism• Cons: Most suitable for independent and symmetric uncertainty

Uncertainty Sets – “Heart” of Robust Optimization

8

budget , 1 1, , i i i i i i ii

U u u u u z z z i

1

box , L Ui i i iU u u u u i

• The “bridge” between data and uncertainty set

• Dirichlet Process (DP) Mixture Model [Blei & Jordan, 06]• A powerful Bayesian nonparametric model• Ability to adjust its complexity to that of data

Data-Driven Uncertainty Set for ARO

9

0 0

1 2

(1, )

, , ( )

( )

~

~

~

~i

k

k

i

i i i l

Beta

F F

l Mult

o l p o

1 kkkF

“Stick Breaking”

Data Sample

1 11

2 21

Pr new observation Dataset

Predictive PosteriorVariationalinference

Dirichlet Process Mixture Model

Uncertainty Set

Features of DP Mixture Model

10

• Dirichlet Process (DP) mixture model [Blei & Jordan, 06]• Model data with complicated characteristics (e.g. multimode)• Handle data outliers, asymmetry, and correlation

• Why DP mixture model is better?

• Parameter space has infinite dimensions

• Infer the number of components from data

DP mixture model

• Finite number of parameters

• Specify the number of components a priori

Parametric mixture models

6 parameters

Variational Inference for DDANRO Uncertainty Set

11

,,

i i

i i

v

Variationalinference

Inference results

Uncertainty dataq is variational distribution

Update kq

Update

Update q

,k kq η H

1

1

ELBO ELBOELBOt t

t

q qtol

q

Yes

No

Parameters in uncertainty sets

1

1

iji

iji i j j

vv v

1

1 dimi

ii i

s

u

1, , NU u u

, , ,i i i is μ Ψ

Evidence lower bound

Update iq l

1

1 1 1

, , , , ,N M M

i k k ki k k

q q l q q q

l β η H η H

,i iμ Ψ

Example 1: Data-driven uncertainty set for ARO

12

Box type uncertainty set

Budgeted uncertainty set Data-driven uncertainty set

Uncertainty data0 20 40 60 80 100 120 140 160 180

0

50

100

150

200

u1

u 2

Outliers

Uncertainty data

Outliers

0 20 40 60 80 100 120 140 160 1800

50

100

150

200

u1

u 2

Uncertainty Set

0 20 40 60 80 100 120 140 160 1800

50

100

150

200

u1

u 2

Uncertainty Set

0 20 40 60 80 100 120 140 160 1800

50

100

150

200

u1

u 2

Uncertainty Set

128 Olin 128 Olin

Uncertainty Sets under Different Parameters

13

Budget based Uncertainty Set Data Driven Uncertainty Set

0 20 40 60 80 100 120 140 160 1800

50

100

150

200

250

u1

u 2

Uncertainty set (=0.5)Uncertainty set (=1.0)Uncertainty set (=1.5)

0 20 40 60 80 100 120 140 160 1800

50

100

150

200

250

u1u 2

Uncertainty set (=0.50)Uncertainty set (=0.93)Uncertainty set (=0.98)

Data-Driven Adaptive Nested Robust Optimization

14

*

1/21

:

, 1, i

i i i i ii

U s

u u μ Ψ z z z

( , )1, ,

1/21

min max max min

. . ,

, 1,

, :

i

T T

i m U

i i i i i i

s t

U s

x y x uu

x

y

c x b y

Ax d x S

u u μ Ψ z z z

x u y S Wy h Tx Mu

Uncertainty set using l1 and l∞ norms

DDANRO1∩∞

• Size depends on data • Multi-level (min-max-

max-min) optimization

Model Features

• Adaptive to uncertainty• Less conservative • Captures the nature of

uncertainty data

Advantages

component iChallenge: How to solve the multi-level optimization problem?

Tailored Column & Constraint Gen. Algorithm

15

min

. . , ,

, ,

T

T l

ll

l

s tl L

l L

l L

x y

c x

Ax db y

Tx Wy h Mu

x S y S

Master problem

Sub-problems

max min

. .

i

Ti U

Q

s t

yu

y

x b y

Wy h Tx Muy S

First-stage decisions

Optimality or feasibility cuts

• Multi-level optimization to single-level SIP

Example 2: ARO under correlated uncertainties

16

1 2 1 2

1 2

1 1 1

2 2 2

min 3 5 max min 6y 10

. . 100 , 0, 1, 2

U

i i

x x y

s t x xx y ux y ux y i

x yu

Uncertainties

Uncertainty set is constructeddirectly from data.

Motivating Example 2

17

ARO with box uncertainty set

ARO with budgeteduncertainty set

Data-driven ARO with l1and l∞ norms based set

Min. obj. 824.8 732.3 620.3First-stagedecisions

1

2

20.379.7

xx

1

2

32.267.8

xx

1

2

41.458.6

xx

35 40 45 50 55 60 65 70 75 80 8520

30

40

50

60

70

80

90

100

u1

u 2

Data-driven uncertainty setBox based uncertainty setBudgeted based uncertainty set

35 40 45 50 55 60 65 70 75 80 8520

30

40

50

60

70

80

90

100

u1

u 2

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%500

550

600

650

700

750

800

850

Data Coverage of Uncertainty Set

Obj

ectiv

e Fu

nctio

n V

alue

ARO with budgeted setThe proposed DDANRO

• Uncertain parameters from historical data• Demands of 4 products (correlated uncertainty)• Processing times of 3 reactions (with outliers)• Asymmetry, multimode and correlated data

18

Objective• Maximize profit

• Assignment constraint• Time constraint• Batch size constraint• Mass balance constraint• Storage constraint• Demand constraint

Constraints

Application 1: Batch Process Scheduling

19

Affected by outliers in processing time data

Static robustoptimization

box uncertainty

set

ARO budgeted

uncertainty set

DDANRO

• DDANRO yields the highest profit ($46,597)

• Reduces conservatism of ARO solution in the presence of outlier-corrupted data.

Data-Driven Robust Batch Scheduling Results

Application 2: Process Network Planning

20

Process Network

• 38 processes• 28 chemicals

Objective• Maximize NPV

• Supply (10)• Demand (16)

Constraints• Expansion constraint• Investment constraint• Mass balance constraint• Capacity constraint• Demand constraint• Supply constraint

Uncertainty

Data-Driven Robust Process Network Planning

21

Static robust optimization w/ box uncertainty

ARO with budget based uncertainty

(Гd=3, Гs=2)

DDANRO(Φd=3, Φs=2)

Max. NPV(m.u.) 761.79 799.03 857.38

Computational Results for Application 2

22

Int. Variables Cont. Var. Constraints Total CPU (s)Original ARO 152 681 945

466.4Master (last iter.) 152 7,450 9,748Subproblem 112 13,033 38,067

23

http://you.cbe.cornell.edu

Fengqi YouRoxanne E. and Michael J. Zak Professor

Cornell University

318 Olin Hall, Ithaca, New York 14853fengqi.you@cornell.edu (email)

www.peese.org