Column Generation with the Primal-Dual Interior Point Method€¦ · ColumnGenerationwithIPM T H E...

Column Generation with IPM

TH

E

U N I V E R SI T

Y

OF

ED I N B U

RG

H

Column Generation with thePrimal-Dual Interior Point Method

Jacek Gondzio

joint work with

Pablo Gonzalez-Brevis and Pedro Munari

Buzios, 23 May 2016 1


Outline

• IPMs for Optimization→ central path, perturbed complementarity

• Column Generation/Cutting Planes with IPM

• Warmstarting IPMs→ theory and practice

• Applications (numerous examples)→ summary results

• Conclusions



What is the best method to solve LPs/QPs:

• the Simplex Method (active-set method) or

• the Interior Point Method ?

Maths question: How to cross a polytope?

... let us ask the expert



Brazil view of LP/QP:

How to solve LP/QP problems?

If we asked Neymar Jr, the likely answer would be:“go through the interior of the polytope”.



Interior Point Methods



Primal-Dual Pair of Quadratic ProgramsPrimal Dual

min cTx+ 12x

TQx max bTy − 12x

TQx

s.t. Ax = b, s.t. ATy+ s = c,x ≥ 0; s ≥ 0.

LagrangianL(x, y) = cTx+

12xTQx− yT (Ax− b)− sTx.

Optimality Conditions

Ax = b,

ATy+ s−Qx = c,

XSe = 0, ( i.e., xj · sj = 0 ∀j),(x, s) ≥ 0,

X=diag{x1, · · · , xn}, S=diag{s1, · · · , sn}, e=(1, · · · ,1)∈Rn.



First Order Opt Conditions for QP

Ax = b,

ATy+ s−Qx = c,XSe = 0,(x, s) ≥ 0,

First Order Opt Conditions for Barrier QP

Ax = b,

ATy+ s−Qx = c,XSe = µe,

(x, s) > 0,



Complementarity in the Interior Point Method

The first order optimality conditions (FOC)

Ax = b,

ATy+ s−Qx = c,

XSe = µe,

x, s ≥ 0,

where X=diag{xj}, S=diag{sj} and e=(1, · · · ,1)∈Rn.

Analytic centre (µ-centre): a (unique) point(x(µ), y(µ), s(µ)), x(µ) > 0, s(µ) > 0 that satisfies FOC.

The interior point method gradually reduces the comple-mentarity products

xj · sj ≈ µ → 0 ∀j = 1,2, ..., n.



Attractive features

IPMs:

• excell on large-scale problems

• can easily control the distance to optimality

• are indifferent to degeneracy

• are able to warm start

JG, Interior Point Methods 25 Years Later,EJOR, 218 (2012) 587–601.



Decomposition Approaches

What IPMs can offer?

• Use IPM to solve the Master

• Use IPM to solve the Subproblemearly termination with ε-optimality→ ε-subgradients (“on-demand accuracy”)

JG and Vial, Warm start and ε-subgradients in cut-ting plane scheme for block-angular linear programs, Com-put Optimization and Applications 14(1999) 17-36.JG and Kouwenberg, High performance computingfor ALM, Operations Research 49 (2001) 879–891.



The First Order Optimality Conditions

Ax = b,

−Qx+ ATy+ s = c,XSe = µe,

(x, s) > 0.

Assume primal-dual feasibility:

Ax = b and −Qx+ ATy+ s = c

Parameter µ controls the distance to optimality.

(cTx+12xTQx)−(bTy−

12xTQx) = xTs = nµ.

On-demand accuracy is readily available.



Primal-Dual Column Generation Method:

Heading-in problems:terminate RMP solution early:→ get stable dual solution u

Tailing-off problems:use IPM to solve the RMP:→ no degeneracy issues

t

t t

mucenter



Primal-DualColumn Generation Method



Column Generation (CG)

Consider an LP, called the master problem (MP):

z⋆ := min∑

j∈N

cjλj,

s.t.∑

j∈N

ajλj = b,

λj ≥ 0, ∀j ∈ N.

• N is too big;

• The columns aj are implicit elements of A;

• We know how to generate them!



CG: Restricted master problem (RMP): N ⊂ N

zRMP := min∑

j∈N

cjλj,

s.t.∑

j∈N

ajλj = b,

λj ≥ 0, ∀j ∈ N.

• Optimal λ for the RMP ⇒ feasible λ for the MP;

• λj = λj, ∀j ∈ N , and λj = 0 otherwise;

• Hence, z⋆ ≤ zRMP = UB (Upper Bound).

• How to know it is optimal?

– Call the oracle!



CG:

• Oracle: check the feasibility of the dual u;

• Reduced costs: sj = cj − uTaj, ∀j ∈ N ;

• But the columns are not explicit and, hence,

zSP := min{cj − uTaj|aj ∈ A}.

• (we reset zSP := 0, if zSP > 0);

• Lower Bound: LB = zRMP + κzSP ≤ z⋆, where

κ ≥∑

i∈N

λ⋆i ,

• If zSP < 0, then new columns are generated;

• Otherwise, an optimal solution of the MP was found!



Appealing features

of IPMs:

• Use IPM to solve the RMP:→ no degeneracy issues

• Terminate RMP solution early:→ use reliable estimate of duality gap→ get stable dual solution u



PDCGM Algorithm Parameters: εmax, D>1, δ, κ

1. set LB = −∞, UB =∞, gap = ∞, ε = 0.5;

2. while (gap > δ) do

3. find a well-centred ε-opt (λ, u) of the RMP;

4. UB = min{UB, zRMP};

5. call the oracle with the query point u;

6. LB = max{LB, κzSP + bT u};

7. gap = (UB− LB)/(1 + |UB|);

8. ε = min{εmax, gap/D};

9. if (zSP < 0) then add new columns to the RMP;

10. end (while)



Theorem

Let z∗ be the optimal solution of the (MP).Given δ > 0, the primal-dual column generation methodconverges in a finite number of steps to a feasible solutionλ of the MP with objective value z that satisfies

z − z∗ ≤ δ(1 + |z∗|).

JG, Gonzalez-Brevis and Munari,New developments in the primal-dual column generationtechnique, European J. of Oper Res 224 (2013) 41–51.



Oscillation in a real instance

Changes of dual variables between iterations j and j+1,‖uj − uj+1‖2 on VRPTW instance (Solomon C207):



Warmstarting IPMs



A need to solve a sequence of similar problems

• column generation

• cutting plane methods

• subproblems in the block-angular LPs(Dantzig-Wolfe decomp., Benders decomp.)

• B&B, (and B&Cut, B&Cut&Price, etc)

• SQP

• any sequence of similar problemsexample: computing efficient frontier in Markowitzportfolio optimization



Warm Starts Which method should be used?• Simplex Method, or• Interior Point Method.

xjsj = 0, ∀j

When is the Simplex Method better?→ few indices change optimal partitionB & B, adding one cut in CPM, etc.

When is the Interior Point Method better?→ many indices change optimal partitionadding many cuts in CPM,dealing with a general change of problem data, etc

Conjecture:The more changes in the (large) problemthe more attractive IPM-based warm starts are.



Difficulty of IPM Warm Starts

Modified Problem

Original Problem



Warmstarting Heuristic

Idea: Start close to the (new) central path, not close tothe (old) solution

Modified Problem

Original Problem

JG, Warm start of the primal-dual method applied inCPM, Mathematical Programming 83 (1998) 125–143



Warm Start in PDCGM context: new results

Theorem

Let n and n+k be the dims of RMP and newRMP, resp.Let a µ0-centre (x0, y0, z0) ∈ NS(γ) of RMP be given.The warm start solution (xw, xw, yw, zw, zw) ∈ NS(βγ)produced by the new warmstarting algorithm satisfies

(xw)Tsw + (xw)T sw ≤6γ

n+ k

n(x0)Ts0.

JG and Gonzalez-Brevis,A new warmstarting strategy for the primal-dual columngeneration method, Mathematical Programming A 152(2015) 113–146



Applications of PDCGM

• UFLP: Uncapacitated Facility Location Problem

• MKL: Multiple Kernel Learning problem

• TSSP: Two-Stage Stochastic Problem

• MCNF: MultiCommodity (MinCost) Network Flowproblem

• CSP: Cutting Stock Problem

• CLSPST: Capacitated Lot-Sizing Problem withSetup Times

• VRPTW: Vehicle Routing Pb with Time Windows

PDCGM Software available at:http://www.maths.ed.ac.uk/~gondzio/software/pdcgm.html



Number of iterations (relative to PDCGM)

CSP VRPTW CLSPSTSCGM 1.52 1.33 1.60ACCPM 2.41 4.86 1.26



CPU time [s] (relative to PDCGM)

CSP VRPTW CLSPSTSCGM 3.50 1.95 1.26ACCPM 8.97 4.01 1.27



Conclusions

A completely new perspective is needed to exploit theinsight offered by IPMs in a number of combinatorialoptimization applications:

• column generation

• cutting plane methods

• B & B, (and B & Cut, B & Cut & Price, etc)

Warmstarting works well in the CG/CPM context:

problems are re-optimized in 3-5 IPM iterations



References

JG, Gonzalez-Brevis, Munari, New developmentsin the primal-dual column generation technique,European Journal of Oper Res, 224 (2013) 41–51.

Munari, JG, Using the primal-dual interior point al-gorithm within the branch-price-and-cut method,Computers and Oper Res, 40 (2013) 2026–2036.

JG, Gonzalez-Brevis, A new warmstarting strategyfor the primal-dual column generation method,Mathematical Programming A, 152 (2015) 113–146.

JG, Gonzalez-Brevis, Munari, Large-scale opti-mization with the primal-dual column generation method,Mathematical Programming Comput, 8 (2016) 47–82.

http://www.maths.ed.ac.uk/~gondzio/software/pdcgm.html



Vehicle Routing Problem with Time Windows

A company delivers goods to customers i ∈ C.The company has vehicles k ∈ V and each of them startsat a depot, travels to several customers and returns tothe depot. The visit of vehicle k to customer i needs totake place in a specific time window: ai ≤ sik ≤ bi, wheresik is the time when vehicle k reaches customer i.

Objective: Minimize the total cost of delivery.

Define binary variable xijk which takes value 1 if vehiclek travels from customer i to customer j (k ∈ V, i, j ∈ C)and takes value zero otherwise.



Vehicle Routing Problem with Time WindowsConstraints:Exactly one vehicle leaves customer i:

∑

k∈V

∑

j∈N

xijk = 1, ∀i ∈ C

Vehicle capacity constraint:∑

i∈C

di∑

j∈N

xijk ≤ q, ∀k ∈ V

Each vehicle leaves the depot and returns to it:∑

j∈N

x0jk = 1 and∑

j∈N

xi(n+1)k = 1, ∀k ∈ V



VRPTW: Constraints (continued)

Time-window constraint

sik + tij −M(1− xijk) ≤ sjk, ∀i, j ∈ N,∀k ∈ V.

Since xijk is binary the above constraint has the followingmeaning: If xijk = 1 (vehicle k travels from customer i

to customer j) then

sik + tij ≤ sjk

that is, the arrival time of vehicle k to customer j isgreater than or equal the sum of time when vehicle karrives to customer i and the time tij it takes to travelfrom i to j.Otherwise (if xijk = 0) the presence of “big” M guaran-tees that the constraint is always inactive.



VRPTW min∑

k∈V

∑

i∈N

∑

j∈N

cijxijk

s.t.∑

k∈V

∑

j∈N

xijk = 1, ∀i ∈ C,

∑

i∈C

di∑

j∈N

xijk ≤ q, ∀k ∈ V,

∑

j∈N

x0jk = 1,∑

i∈N

xi(n+1)k = 1, ∀k ∈ V,

∑

i∈N

xihk −∑

j∈N

xjhk = 0, ∀h ∈ C,∀k ∈ V,

sik + tij −M(1− xijk) ≤ sjk, ∀i, j ∈ N,∀k ∈ V,ai ≤ sik ≤ bi, ∀i ∈ N,∀k ∈ V,xijk ∈ {0,1}, ∀i, j ∈ N,∀k ∈ V.


Column Generation with the Primal-Dual Interior Point Method€¦ · ColumnGenerationwithIPM T H E...

Documents

Transcript of Column Generation with the Primal-Dual Interior Point Method€¦ · ColumnGenerationwithIPM T H E...