Mixed Integer Programming Models for Non-Separable Piecewise

School of Chemical & Biomolecular Engineering, Georgia Tech

2008 – Atlanta, GA

Mixed Integer Programming Models for

Non-Separable Piecewise Linear Cost

Functions

Juan Pablo Vielma

H. Milton Stewart School of Industrial and Systems Engineering

Georgia Institute of Technology

Joint work with Shabbir Ahmed and George Nemhauser.

is a piecewise linear

function (PLF) and is any compact set.

Convex = Linear Programming. Non-Convex = NP Hard.

Specialized algorithms (Tomlin 1981, ..., de Farias et al.

2008 ) or Mixed Integer Programming Models (12+ papers)./21

Piecewise Linear Optimization

2

min f0(x)

s.t.

fi(x) ≤0 ∀i ∈ I

x ∈X ⊂ Rn

/21

Mixed Integer Models for PLFs Existing studies are for separable functions:

We emphasize Non-Separable and/or discontinuous:

3

f(x,y)

y

x0

1

3

2

0 2 4

x

y

/21

Outline

Applications of Piecewise Linear Functions.

Modeling Piecewise Linear Functions.

Computational Results.

Extension to Discontinuous Functions.

Final Remarks.

4

/21

Applications of Piecewise Linear Functions

Economies of Scale: Concave

Single and multi-commodity network flow.

Applications in telecommunications, transportation,

and logistics.

(Balakrishnan and Graves 1989, ..., Croxton, et al. 2007).5

0 1 3 50

2

3.5

4

/21


Fixed Charges and Discounts

1.Fixed Costs in Logistics.

2.Discounts (e.g. Auctions: Sandholm,

et al. 2006, CombineNet).

3.Discounts in fixed charges (Lowe

1984).6

x

y

0

2

3

0 1 20

1

2

3

4

1. 2. 3.

0 1 30

1

2

3

Gas Network Optimization

(Martin et al. 2006)./21


Non-Linear and PDE Constraints

7

A∂ρ

∂t+ ρ0

∂q

∂x= 0,

∂p

∂x= −λ

|v|v

2Dρ .

Pipe

Demand Points

Pipes/Valves/

Compressors

Connections

Source

/21


Numerically Exact Global Optimization

Process engineering (Bergamini et al. 2005,

2008, Computers and Chemical Eng.)

Wetland restoration (Stralberg et al. 2009).8

/21


Numerically Exact Global Optimization

Process engineering (Bergamini et al. 2005,

2008, Computers and Chemical Eng.)

Wetland restoration (Stralberg et al. 2009).8

0.5 1.0 1.5 2.0 2.5 3.0

0.2

0.4

0.6

0.8

1.0

0 1 2 4 5

f(4) = 5

0

f(0) = 10

f(1) = 32

f(2) = 40

f(5) = 15

/21

Modeling Piecewise Linear Functions

Piecewise Linear Functions: Definition

9

Definition 1. Piecewise Linear f : D ⊂Rn →R:

f(x) :=

{

mP x+ cP x∈ P ∀P ∈P.

for finite family of polytopes P such that D =⋃

P∈PP

f(x,y)

y

x ∈ P

/21


Piecewise Linear Functions: Definition

9

Definition 1. Piecewise Linear f : D ⊂Rn →R:

f(x) :=

{

mP x+ cP x∈ P ∀P ∈P.

for finite family of polytopes P such that D =⋃

P∈PP

0 1 2 4 5

f(4) = 5

0

f(0) = 10

f(1) = 32

f(2) = 40

f(5) = 15

(a) f .

0 1 2 4 5

5

0

10

32

40

15

(b) epi(f).

/21


Modeling Function = Epigraph

Example: 10

/21



Example: 10

0 1 2 4 5

f(4) = 5

0

f(0) = 10

f(1) = 32

f(2) = 40

f(5) = 15

(a) f .

0 1 2 4 5

5

0

10

32

40

15

(b) epi(f).

0

1

3

2

0 2 4

to the discontinuous case.

0

1

3

2

0 2 4

/21



Example: 10

0 1 2 4 5

f(4) = 5

0

f(0) = 10

f(1) = 32

f(2) = 40

f(5) = 15

(a) f .

0 1 2 4 5

5

0

10

32

40

15

(b) epi(f).

0

1

3

2

0 2 4

to the discontinuous case.

0

1

3

2

0 2 4

Lower

Semicontinuous Closed Set

/21


Convex Comb. Univariate (Dantzig, 1960)

11

0

1

3

2

0 2 4

λP1,

/21



11

0

1

3

=

0 2 4

1

3

0 2

∪

0

3

2 4

λP1,

/21



11

0

1

3

=

0 2 4

1

3

0 2

∪

0

3

2 4

x = 0λ0 + 2λ2 + 4λ4

z ≥ 1λ0 + 3λ2 + 0λ4

1 = λ0 + λ2 + λ4, λ0, λ2, λ4 ≥ 0

λ0 ≤ yP1, λ2 ≤ yP1

+ yP2, λ4 ≤ yP2

1 = yP1+ yP2

, yP1, yP2

∈ {0, 1}

λP1,

/21


CC Multivariate (Lee and Wilson, 2001)

12

x

y

f!x,y"

P1

P2

/21


CC Multivariate (Lee and Wilson, 2001)

12

x

y

f!x,y"

P1

P2

Polytopes that have

(0,0) as a vertex.

/21


Other issues: Log models & Strength

Strength:

Popular models are strong.

Standard MILP techniques can

yield weak models.

Size of existing models is linear

in :

We can get models with “size”

logarithmic in (Vielma and

Nemhauser 2008, Vielma et al.

2009).13

f(x,y)

y

x

0 1 20

1

2

Minimization Problems:

Univariate Objective: Sum of 100

univariate functions, each affine in k

segments.

Multivariate Objective: Sum of 10

bivariate functions, each affine in

a k x k grid.

Solver: CPLEX 11 on 2.4Ghz machine.

/21

Computational Results

Computation: Transportation Problems

14

0 1 20

1

2

/21


Univariate Case (Separable)

15

1

10

100

1000

10000

4 8 16 32

Average Time Solve [s]

Number of Segments

DCCCCMC

DLogLog

/21


Multivariate Case (Non-Separable)

16

1

10

100

1000

10000

4x4 8x8 16x16


Grid Size

DCCCCMC

DLogLog

/21

Discontinuous Case

Lower Semicontinuity (LSC)

Lower Semicontinuity:

17

x≤d

0 2 4 5f(4) = 0

f(0) = f+(4) = 1

f−(2) = 4

f+(2) = f(5) = 3

f(2) = 2 Consider the piecewise linearand f

+(d) = limx→dx≥d

f(x).

single variable function. Considerh f−(d) = limx→d

x≤d

f(x) and

≤

0 2 4 50

1

4

3

2

/21

Discontinuous Case

Lower Semicontinuous PLFs

18

f(x) :=

{

mP x + cP x ∈ P ∀P ∈ P

.P = {x ∈n

: aix ≤ bi ∀i ∈ {1, . . . , p},

aix < bi ∀i ∈ {p, . . . ,m}}

Finite family of

copolytopes

/21

Discontinuous Case

Lower Semicontinuous PLFs

18

f(x) :=

{

mP x + cP x ∈ P ∀P ∈ P

.P = {x ∈n

: aix ≤ bi ∀i ∈ {1, . . . , p},

aix < bi ∀i ∈ {p, . . . ,m}}

Finite family of

copolytopes

} { ∈ } {

f(x, y) :=

3 (x, y) ∈ (0, 1]2

2 (x, y) ∈ {(x, y) ∈ 2 : x = 0, y > 0}

2 (x, y) ∈ {(x, y) ∈ 2 : y = 0, x > 0}

0 (x, y) ∈ {(0, 0)}.

Figure 8(b) is slightly more complicated and its domain is ˜ = con

x

y

0

2

3

/21

Discontinuous Case

Disaggregated CC (Jeroslow and Lowe, 1984)

19

f(x) :=

{

x + 1 x ∈ [0, 2)

4 − x x ∈ [2, 4]

0

1

3

2

0 2 4

(DCC)

/21

Discontinuous Case


19

0

1

3

2 =

0 2 4

1

3

0 2

∪

0

2

2 4

f(x) :=

{

x + 1 x ∈ [0, 2)

4 − x x ∈ [2, 4]

(DCC)

/21

Discontinuous Case


19

0

1

3

2 =

0 2 4

1

3

0 2

∪

0

2

2 4

f(x) :=

{

x + 1 x ∈ [0, 2)

4 − x x ∈ [2, 4]

x = 0λP1,0 + 2λP1,2 + 2λP2,2 + 4λP2,4

z ≥ 1λP1,0 + 3λP1,2 + 2λP2,2 + 0λP2,4

1 = λP1,0 + λP1,2, λP1,0, λP1,2 ≥ 0

1 = λP2,2 + λP2,4, λP2,2, λP2,4 ≥ 0

1 = yP1+ yP2

, yP1, yP2

∈ {0, 1}

λP1,(DCC)

/21

Discontinuous Case


19

0

1

3

2 =

0 2 4

1

3

0 2

∪

0

2

2 4

f(x) :=

{

x + 1 x ∈ [0, 2)

4 − x x ∈ [2, 4]

x = 0λP1,0 + 2λP1,2 + 2λP2,2 + 4λP2,4

z ≥ 1λP1,0 + 3λP1,2 + 2λP2,2 + 0λP2,4

1 = λP1,0 + λP1,2, λP1,0, λP1,2 ≥ 0

1 = λP2,2 + λP2,4, λP2,2, λP2,4 ≥ 0

1 = yP1+ yP2

, yP1, yP2

∈ {0, 1}λP2

(DCC)

/21

Discontinuous Case


19

0

1

3

2 =

0 2 4

1

3

0 2

∪

0

2

2 4

f(x) :=

{

x + 1 x ∈ [0, 2)

4 − x x ∈ [2, 4]

x = 0λP1,0 + 2λP1,2 + 2λP2,2 + 4λP2,4

z ≥ 1λP1,0 + 3λP1,2 + 2λP2,2 + 0λP2,4

1 = λP1,0 + λP1,2, λP1,0, λP1,2 ≥ 0

1 = λP2,2 + λP2,4, λP2,2, λP2,4 ≥ 0

1 = yP1+ yP2

, yP1, yP2

∈ {0, 1}

(DCC)

/21

Discontinuous Case


19

0

1

3

2 =

0 2 4

1

3

0 2

∪

0

2

2 4

f(x) :=

{

x + 1 x ∈ [0, 2)

4 − x x ∈ [2, 4]

x = 0λP1,0 + 2λP1,2 + 2λP2,2 + 4λP2,4

z ≥ 1λP1,0 + 3λP1,2 + 2λP2,2 + 0λP2,4

1 = λP1,0 + λP1,2, λP1,0, λP1,2 ≥ 0

1 = λP2,2 + λP2,4, λP2,2, λP2,4 ≥ 0

1 = yP1+ yP2

, yP1, yP2

∈ {0, 1}

yP1

yP2

(DCC)

/21

Discontinuous Case

Multivariate Lower Semicontinuous

20

1

10

100

1000

10000

4x4 8x8 16x16 32x32


Grid Size

DCCMC

DLog

x

y

/21

Final Remarks

Final Remarks

Many MILP models for PLF: Most are simple.

Popular models are strong.

Caution: Standard MILP techniques can weak

models.

Best model varies: e.g. Log’s best for fine grids.

Advertisement: Papers at

http://www2.isye.gatech.edu/~jvielma

21



/21

Comparison of Formulations

Strength of LP Relaxations

Sharp Models: LP = lower convex envelope.

All popular models are sharp.

Locally Ideal: LP = Integral (All but CC, even Log).

Locally ideal implies Sharp.22

(a) epi(f). (b) conv(epi(f)).

LP relaxation

∑v∈V(P)

λvv = x,∑

v∈V(P)

λv (mP v + cP )≤ z

λv ≥ 0 ∀v ∈ V(P) :=

⋃P∈P

V (P ),∑

v∈V(P)

λv = 1

λv ≤∑

{P∈P :v∈P}

yP ∀v ∈ V(P),∑

P∈P

yP = 1, yP ∈ {0,1} ∀P ∈P

∑P∈P

∑v∈V (P )

λP,vv = x,∑

P∈P

∑v∈V (P )

λP,v (mP v + cP )≤ z

λP,v ≥ 0 ∀P ∈P, v ∈ V (P ),∑

P∈P

∑v(P )

λP,v = 1

∑

v∈V (P )

λP,v = yP ∀P ∈P,∑

P∈P

yP = 1, yP ∈ {0,1} ∀P ∈P

/21


For Multivariate Functions

23

(DCC)

(CC)

/21


Logarithmic DCC (DLog)

New? Direct from ideas in Ibaraki (1976), Vielma and

Nemhauser (2008)24

∑

P∈P

∑

v∈V (P )

λP,vv = x,∑

P∈P

∑

v∈V (P )

λP,v (mP v + cP )≤ z

λP,v ≥ 0 ∀P ∈P, v ∈ V (P ),∑

P∈P

∑

v∈V (P )

λP,v = 1

∑

P∈P+(B,l)

∑

v∈V (P )

λP,v ≤ yl,∑

P∈P0(B,l)

∑

v∈V (P )

λP,v ≤ (1− yl), yl ∈ {0,1} ∀l ∈L(P)

where B :P → {0,1}⌈log2 |P|⌉ is any injective function, L(P) := {1, . . . , ⌈log2 |P|⌉},

P+(B, l) := {P ∈P : B(P )l = 1} and P

0(B, l) := {P ∈P : B(P )l = 0}.

/21


Logarithmic Conv. Comb. (Log)

Requires Independent Branching Scheme.

Vielma and Nemhauser (2008).

25

∑

v∈V(P)

λvv = x,

∑

v∈V(P)

λv (mP v + cP )≤ z

λv ≥ 0 ∀v ∈ V(P),

∑

v∈V(P)

λv = 1

∑

v∈Ls

λv ≤ ys,

∑

v∈Rs

λv ≤ (1− ys), ys ∈ {0,1} ∀s∈ S.

/21


Multiple Choice (MC)

Balakrishnan and Graves (1989), Croxton et al. (2003a),

Jeroslow and Lowe (1984) and Lowe (1984)

26

∑

P∈P

xP = x,∑

P∈P

(

mP xP + cP yP

)

≤ z

AP xP ≤ yP bP ∀P ∈P

∑

P∈P

yP = 1, yP ∈ {0,1} ∀P ∈P,

where AP x ≤ bP is the set of linear inequalities describing P . This

/21


Incremental or Delta (Inc)

Similar for multivariate functions.

Croxton et al. (2003a), Dantzig (1963, 1960), Keha et al.

(2004), Markowitz and Manne (1957), Padberg (2000),

Sherali (2001), Vajda (1964) and Wilson (1998).27

d0 d1 d2 d3 d4

f(d3)

0

f(d0)

f(d1)

f(d2)

f(d4)

d0 +

K∑

k=1

δk (dk − dk−1) = x

f(d0)+

K∑

k=1

δk (f(dk)− f(dk−1))≤ z

δ1 ≤ 1, δK ≥ 0, δk+1 ≤ yk ≤ δk,

yk ∈ {0,1} ∀k ∈ {1, . . . ,K − 1}.

/21


Univariate Case (Separable)

28

1

10

100

1000

10000

DCC CC MC Inc SOS2 DLog Log


Formulation

48

1632

/21


Multivariate Case (Non-Separable)

29

1

10

100

1000

10000

DCC CC MC Inc DLog Log


Formulation

4x48x8

16x16

/21

Discontinuous Case

Multivariate Lower Semicontinuous

30

1

10

100

1000

10000

DCC MC DLog


Formulation

4x48x8

16x1632x32

Mixed Integer Programming Models for Non-Separable Piecewise

Documents

Transcript of Mixed Integer Programming Models for Non-Separable Piecewise