Mining the CPLEX Node Log for Faster MIP Performance

Decision Optimization

Mining the CPLEX Node Log for Faster MIP Performance

Ed Klotz, Ph.D ([email protected])

© 2013 IBM Corporation

1.0

CPLEX Timeline

1988 1990 1995 2000 2005 2010 2013

2.0 3.0 4.0 5.06.0 6.6

7.0 8.0 9.0 10.0 11.0 12.1

primal simplex

network simplex

presolve

parallel barrier

clique cuts

cover cuts

QP

parallel MIP

CPLEX Optimization, Inc. ILOG IBM

more cuts

OPL / CP

Gomory cuts

C++ / Java

more cuts

probing

user cuts

MIQP

QP simplex

(MI)QCP

.Net

FeasOpt

indicators

conflicts

symmetry

polishing

solution pools

det. parallel

tuning tool

dynamic search

{0-½} cuts

MATLAB

Python

det. barrier

MCF cuts

ODME

12.212.3

dettime limit

SOCP duals

globalization

64 bit non-zeros

non-convex QP

MIP kappa

parallel root

det. concur. LP

12.4

Performance

6.51.2

dual simplex

MIPmajor simplex

improvements

memory model

12.5

L&P cuts

QCP duals

remote object

random seed

det. tuning

12.5.1

MIP heuristics

12.6

global

non-convex

(MI)QP

Distributed

MIP


CPLEX Timeline

Primary sources of MIP performance improvements– Additional presolve reductions– Additional branching selection

• Pseudo costs based on strong branching– Cuts

• Includes any techniques to fix variables based on integrality (e.g. probing)– MIP heuristics– Increased availability of multiple CPUs/cores

Improvements are based on additional calculations to obtain more MIP information– Additional time must pay for itself

• Cuts and heuristics must reduce node count to compensate for additional time

– Increased node LP solve time may be more significant than cut calculation time

• Multi-core must increase node throughput to compensate for overhead, synchronization time


CPLEX Timeline

Fundamentally, CPLEX has thinned the herd of difficult MIPs by adding more functionality to address challenging aspects of the models– Internal logic to assess tradeoffs between additional computations, node

throughput• Facilitated by recent addition of deterministic clock

– Our internal list of development ideas remains long• Our challenge is not running out of ideas, but efficiently assessing and

implementing the ones that have the most promise

In earlier versions, MIP performance tuning usually involved increasing calculations beyond the default level– We expect to continue adding new algorithmic procedures indefinitely– With the current bag of tricks, performance tuning now involves deciding

when to decrease calculations from default levels as well as deciding when to increase them.


Outline

Brief review of branch and cut

Series of examples illustrating different ways to improve performance– Increasing features above default levels– Decreasing features below default levels– Tightening the formulation directly– Performance variability considerations

Conclusions


Root;

v=3.5

x=2.3

Integer y=0.6

z=0.3

Lower Bound

Integer

Upper Bound

Infeas

z=0.1

G

A

P

Review of Branch and Bound

Fathomed

Branch and Bound for MIP



Progress of the algorithm depends on:– Ability to find integer feasible solutions

• # of integer infeasibilities at each node– Ability to prune nodes

• Objective value of best integer feasible solution– Ability to move lower bound

• # of other node relaxations with same objective value• # of active nodes remaining

– Strength of the model formulation– Node throughput

• Node relaxation solve times• Cut, heuristic computation times


Nodes Cuts/

Node Left Objective IInf Best Integer Best Node ItCnt Gap

...

300 229 22.6667 40 31.0000 22.0000 4433 29.03%

400 309 cutoff 31.0000 22.3333 5196 27.96%

500 387 26.5000 31 31.0000 22.6667 6164 26.88%

...

7800 5260 28.5000 23 31.0000 25.6667 55739 17.20%

7900 5324 28.2500 26 31.0000 25.6667 56424 17.20%

8000 5385 27.3750 30 31.0000 25.7778 57267 16.85%


Optimizer Node Log shows algorithm progress– Here we have progress in best node but not best integer

Node pruning Feasible solns

Strength /

lower bound

Node

throughput


Parameter tuning

Enable non default parameters based on node log

Example: Police patrol scheduling (Capar, Keskin and Rubin, working paper)

CPLEX 12.5.1 node log, default settings:Nodes Cuts/

Node Left Objective IInf Best Integer Best Bound ItCnt Gap

* 0+ 0 0.0000 65624.6162 19 ---0 0 1085.0999 347 0.0000 1085.0999 19 ---

* 0+ 0 686.8500 1085.0999 19 57.98%…

* 0+ 0 984.5000 1076.2743 681 9.32%0 2 1076.2743 180 984.5000 1076.2743 681 9.32%

Elapsed time = 3.48 sec. (1782.68 ticks, tree = 0.01 MB, solutions = 11)…

2600 1732 1076.2743 101 1043.5666 1076.2743 93070 3.13%

* 2602+ 1732 1046.5666 1076.2743 93148 2.84%* 2603+ 1077 1054.2167 1073.3166 97988 1.81%

2603 1078 1073.3166 183 1054.2167 1073.3166 97988 1.81%*2606+ 717 1059.7500 1073.3166 98360 1.28%…

16957 6884 1071.5499 129 1065.6999 1073.3166 1768502 0.71%

17826 7272 1073.0720 137 1065.6999 1073.3166 1863716 0.71%

18556 7515 1071.8125 117 1065.6999 1073.3096 1932436 0.71%

…MIP - Integer optimal solution: Objective = 1.0656998700e+03Solution time = 326.88 sec. Iterations = 2607165 Nodes = 25916Deterministic time = 178344.25 ticks (545.60 ticks/sec)

Best node unchanged


Parameter tuning

Enable non default parameters based on node log

CPLEX 12.5.1 node log on same model, mip emphasis = 3 (moving best bound):

Nodes Cuts/Node Left Objective IInf Best Integer Best Bound ItCnt Gap

* 0+ 0 0.0000 65624.6162 1960 ---0 0 1085.0999 243 0.0000 1085.0999 1960 ---

* 0+ 0 975.9833 1085.0999 2217 11.18%…* 0+ 0 1060.6999 1073.3166 4369 1.19%

0 2 1073.3166 51 1060.6999 1073.3166 4369 1.19%Elapsed time = 10.15 sec. (6053.06 ticks, tree = 0.01 MB, solutions = 8)

…

153 151 1071.7548 176 1065.3832 1073.3166 199082 0.74%

* 162+ 156 1065.5666 1073.3114 219295 0.73%

…

MIP - Integer optimal solution: Objective = 1.0656998700e+03Solution time = 103.45 sec. Iterations = 585868 Nodes = 1173Deterministic time = 61697.50 ticks (596.40 ticks/sec)

Node throughput drops, but nodes have much more informative

Time spent at the root node increases

Better progress in the best node


Parameter tuning

Use Automatic Tuning Tool to find less intuitive parameter settings– Performs multiple runs with different parameter settings– Takes advantage of internal performance metrics not available from the node log– Usage

• Set regular time limit parameter for total time of tuning run• Set tuning time parameter for time allowed for a single optimization (default =

ten million deterministic ticks, roughly 10000 seconds of deterministic time)• Time limits can be deterministic or system time• Specify parameters you want to fix during tuning in a parameter file

– Can require significant amount of time to perform complete tuning run• Requires no user activity after start; just let it run overnight• Unaffected by other processes running concurrently on the machine if run in

deterministic mode


Parameter tuning

Automatic tuning tool recommendations for police patrol scheduling model

Fixed and tuned parameter settings:mip limits cutsfactor 30mip strategy presolvenode 2mip strategy probe 2

Nodes Cuts/


* 0+ 0 0.0000 65624.6162 2717 ---0 0 1085.0999 250 0.0000 1085.0999 2717 ---

* 0+ 0 819.3000 1085.0999 3618 32.44%…

0 0 1073.3166 168 1029.3333 Cuts: 7 4942 4.27%

* 0+ 0 1051.3166 1073.3166 4942 2.09%0 2 1073.3166 69 1051.3166 1073.3166 4942 2.09%

Elapsed time = 5.12 sec. (2397.99 ticks, tree = 0.01 MB, solutions = 9)…

2006 823 1072.4978 101 1065.6999 1073.3166 298244 0.71%

Elapsed time = 31.94 sec. (14863.51 ticks, tree = 2.27 MB, solutions = 14)2182 932 1073.3166 122 1065.6999 1073.3166 328325 0.71%

…MIP - Integer optimal solution: Objective = 1.0656998700e+03Solution time = 45.27 sec. Iterations = 366886 Nodes = 2358Deterministic time = 21584.84 ticks (476.75 ticks/sec)


Parameter tuning

Automatic tuning tool recommendations for police patrol scheduling model(ctd)

– Removed some of the time consuming aspects of mip emphasis 3 settings that didn’t justify the time consumed

• Only need probing = 2 instead of 3• Node probing (presolvenode = 2) provides some additional probing• Limiting cutsfactor probably irrelevant; defaults didn’t add that many cuts

– Node count increased compared to run with mip emphasis = 3• But node throughput increased much more, yielding better performance overall

– Tuning tool assessed lack of progress in best node as we did examining node log• But provided more refined settings that would have been difficult to determine

based purely on node log

Fixed and tuned parameter settings:mip limits cutsfactor 30mip strategy presolvenode 2mip strategy probe 2


Parameter tuning

Disable default parameters based on node log

Model from GAMS, default settings, except for mipgap = .05 – Cuts reduce integer infeasibilities, but don’t improve relaxation objective:

Nodes Cuts/


* 0+ 0 4.98672e+10 1.24432e+10 81415 75.05%0 0 1.87274e+10 8134 4.98672e+10 1.87274e+10 81415 62.45%0 0 1.87276e+10 4904 4.98672e+10 Cuts: 9862 121489 62.45%0 0 1.87276e+10 4975 4.98672e+10 Cuts: 9076 155406 62.45%0 0 1.87276e+10 4402 4.98672e+10 Cuts: 9244 185740 62.44%0 0 1.87276e+10 3959 4.98672e+10 Cuts: 5916 202438 62.44%0 0 1.87277e+10 3967 4.98672e+10 Cuts: 4090 209887 62.44%

Heuristic still looking.0 2 1.87277e+10 3967 4.98672e+10 1.87277e+10 209887 62.44%

Elapsed time = 1879.36 sec. (319034.54 ticks, tree = 0.01 MB, solutions = 1)1 3 1.87277e+10 3962 4.98672e+10 1.87277e+10 209897 62.44%2 4 1.87277e+10 3962 4.98672e+10 1.87277e+10 209898 62.44%

…

1109 1111 1.87277e+10 3433 4.98672e+10 1.87277e+10 238873 62.44%1123 1125 1.87278e+10 3265 4.98672e+10 1.87277e+10 239053 62.44%*1144+ 1144 1.93922e+10 1.87277e+10 239610 3.43%…

MIP - Integer optimal, tolerance (0.05/1e-06): Objective = 1.9392204667e+10Current MIP best bound = 1.8727658614e+10 (gap = 6.64546e+08, 3.43%)Solution time = 4552.00 sec. Iterations = 239910 Nodes = 1144 (1145)Deterministic time = 619473.94 ticks (136.09 ticks/sec)

Still no progress in best node since end of root, despite cuts


Parameter tuning

Disable default parameters based on node log– Default cuts don’t improve the best bound value or make heuristics more effective

• Consider disabling them, since they appear to only slow node throughput

Example: Model from GAMS, all cuts disabled, mipgap = .05

Nodes Cuts/


* 0+ 0 4.98672e+10 1.24432e+10 81415 75.05%0 0 1.87274e+10 8134 4.98672e+10 1.87274e+10 81415 62.45%

Heuristic still looking.Heuristic still looking.

0 2 1.87274e+10 8134 4.98672e+10 1.87274e+10 81415 62.45%Elapsed time = 608.36 sec. (159233.85 ticks, tree = 0.01 MB, solutions = 1)

1 3 1.87274e+10 8133 4.98672e+10 1.87274e+10 81417 62.45%2 4 1.87274e+10 8133 4.98672e+10 1.87274e+10 81419 62.45%

…

1130 1132 1.87275e+10 7842 4.98672e+10 1.87274e+10 85796 62.45%

1162 1164 1.87275e+10 7837 4.98672e+10 1.87274e+10 86029 62.45%*1166+ 1166 1.93925e+10 1.87274e+10 86035 3.43%…

MIP - Integer optimal, tolerance (0.05/1e-06): Objective = 1.9392532385e+10Current MIP best bound = 1.8727389285e+10 (gap = 6.65143e+08, 3.43%)Solution time = 3045.10 sec. Iterations = 86040 Nodes = 1166 (1167)Deterministic time = 443974.82 ticks (145.80 ticks/sec)


Parameter tuning

Tuning tool can identify parameters to disable when node log info insufficient

Example: newdano, from MIPLIB 2010 (576 rows, 505 columns, 56 binaries)

Nodes Cuts/


* 0+ 0 440.0000 0.0000 119 100.00%0 0 11.7241 51 440.0000 11.7241 119 97.34%

* 0+ 0 171.0000 11.7241 119 93.14%0 0 16.0751 55 171.0000 Cuts: 229 935 90.60%

* 0+ 0 83.5000 16.0751 935 80.75%* 0+ 0 77.7500 16.0751 1417 79.32%

0 0 17.1863 56 77.7500 Cuts: 230 1417 77.90%* 0+ 0 71.1667 17.1863 1417 75.85%

0 0 17.2388 56 71.1667 Cuts: 230 1768 75.78%0 0 17.2833 56 71.1667 Cuts: 230 2042 75.71%0 0 17.3373 56 71.1667 Cuts: 230 3031 75.64%0 0 17.3760 56 71.1667 Cuts: 230 3395 75.58%0 0 17.3939 56 71.1667 Cuts: 195 3611 75.56%0 0 17.3996 56 71.1667 Cuts: 160 3754 75.55%0 0 17.4023 56 71.1667 Cuts: 125 3878 75.55%0 0 17.4061 56 71.1667 Cuts: 93 3985 75.54%0 0 17.4088 56 71.1667 Cuts: 84 4100 75.54%0 0 17.4092 56 71.1667 Cuts: 52 4155 75.54%0 0 17.4098 56 71.1667 Cuts: 57 4228 75.54%

* 0+ 0 68.7143 17.4098 4228 74.66%0 2 17.4098 56 68.7143 17.4098 4228 74.66%

…

First two passes of cuts effective, remaining passes much less so

Cuts increase node LP size by more than 3x


Parameter tuning

Example: newdano, from MIPLIB 2010 (576 rows, 505 columns, 56 binaries)– Node log (ctd)

Nodes Cuts/


…

6325 2926 62.1919 22 67.6250 37.0000 1595030 45.29%

6587 3072 49.1172 26 67.6250 40.0000 1659687 40.85%

34728 20338 40.0000 37 67.0000 40.0000 8172856 40.30%

Elapsed time = 185.26 sec. (140467.50 ticks, tree = 188.30 MB, solutions = 11)Nodefile size = 58.61 MB (46.21 MB after compression)35786 20909 50.9259 22 67.0000 40.0000 8426618 40.30%36919 21525 56.3644 24 67.0000 40.5000 8694022 39.55%

…714248 125476 62.1919 25 65.6667 62.1919 1.27e+08 5.29%715105 125454 62.1919 27 65.6667 62.1919 1.27e+08 5.29%715971 125462 65.0078 20 65.6667 62.1919 1.27e+08 5.29%Elapsed time = 2863.93 sec. (2158443.24 ticks, tree = 724.30 MB, solutions = 17)Nodefile size = 595.78 MB (476.13 MB after compression)716982 125349 63.3150 17 65.6667 62.1924 1.27e+08 5.29%718967 124668 cutoff 65.6667 62.2254 1.28e+08 5.24%720407 124246 62.2500 18 65.6667 62.2471 1.28e+08 5.21%…MIP - Integer optimal, tolerance (0.0001/1e-06): Objective = 6.5666666667e+01Current MIP best bound = 6.5660139224e+01 (gap = 0.00652744, 0.01%)Solution time = 4026.61 sec. Iterations = 174685443 Nodes = 1023724 (101)Deterministic time = 3043828.98 ticks (755.93 ticks/sec)

29k nodes with no improvement in best bound

~170 iters per node; fairly large node count given small problem size


Parameter tuning

Example: newdano, from MIPLIB 2010 (576 rows, 505 columns, 56 binaries)

Node log recommends additional computations:– Slow progress in the best node

• Set MIP emphasis to optimality or best bound (2 or 3)• Individual parameter settings that improve the best node

Node log recommends reducing computations– Too many cut passes

• Reduce number of cut passes to 1, 2 or 3– Potential for faster node LP solves

• ~170 dual simplex iterations per node is significant given modest problem size• Reducing number cuts may help as well

Numerous options to consider– Or could start by running the tuning tool while working on something else or taking the rest of

the day off

–


Parameter tuning

Example: newdano, from MIPLIB 2010 (576 rows, 505 columns, 56 binaries)– Tuning tool recommendations and results:

– Disabling cuts and heuristics improved node throughput by over 13x• More than enough to compensate for 1.5x increase in node count• Heuristics found solutions, but were unnecessary because branching had no trouble finding

solutions as well• Both were effective, but the tradeoff between additional computation time and reduced

algorithmic effort was unfavorable– Other settings compared to default of 4062 seconds

• Cutpasses = 2: 2612 seconds• MIP emphasis = 2: 4850.74 seconds• MIP emphasis = 3: 10916.46 seconds

Fixed and tuned parameter settings:mip limits cutpasses -1mip strategy heuristicfreq -1

…

MIP - Integer optimal solution: Objective = 6.5666666667e+01Solution time = 442.67 sec. Iterations = 58261123 Nodes = 1533527Deterministic time = 299752.76 ticks (677.14 ticks/sec)

> 9x speedup Despite 1.5x increase in node count


Parameter tuning – easy models

Disable default parameters that incur overhead on very easy models– Useful when solving long sequences of easy models– Insignificant overhead for models that take seconds, minutes or hours becomes

meaningful on models that solve in fractions of a second

Primary parameters that impose modest overhead at start up– Parallel threads– Presolve

Other parameters to consider disabling– Cuts

• Or limit cutpasses to 1 (or some other small integer value)– Heuristics

• Or apply them less frequently (default = 10 nodes)


Parameter tuning

Disable default parameters that incur overhead on very easy models

Example: neos-501453, defaults (4 threads):

Nodes Cuts/


0 0 47431.6772 2 47431.6772 4

0 0 47451.1722 2 Cuts: 6 9

* 0+ 0 47485.1925 47451.1722 9 0.07%

0 0 47451.3719 2 47485.1925 MIRcuts: 2 10 0.07%

* 0+ 0 47454.6145 47451.3719 10 0.01%

…

MIP - Integer optimal, tolerance (0.0001/1e-06): Objective = 4.7454614499e+04

Current MIP best bound = 4.7451371869e+04 (gap = 3.24263, 0.01%)

Solution time = 0.42 sec. Iterations = 10 Nodes = 0 (1)

Deterministic time = 100.07 ticks (237.28 ticks/sec)


Parameter tuning


Example: neos-501453, threads = 1:

Nodes Cuts/


0 0 47431.6772 2 47431.6772 4

0 0 47451.1722 2 Cuts: 6 9

* 0+ 0 47485.1925 47451.1722 9 0.07%

0 0 47451.3719 2 47485.1925 MIRcuts: 2 10 0.07%

* 0+ 0 47454.6145 47451.3719 10 0.01%

…






Parameter tuning


Example: neos-501453, threads = 1, presolve off:

Nodes Cuts/


0 0 47431.6772 2 47431.6772 10

0 0 47451.1722 2 Cuts: 6 13

* 0+ 0 47454.6145 47451.1722 13 0.01%

…


Current MIP best bound = 4.7451172249e+04 (gap = 3.44225, 0.01%




Parameter tuning – key points

Node log provides extensive info about algorithm progress– Identify lack of progress or performance bottlenecks based on node log output– Set parameters based on source of lack of progress

• MIP emphasis sets numerous parameters at once• Classify other parameters based on whether they can improve progress in best

integer, best node, or both• Tuning tool can provide more refined settings

– Sometimes performance can be improved by disabling parameters (or reducing their default intensity)

• If cut or heuristic computation time slows node throughput by more than any performance gains provided

• Faster node throughput makes branching more effective

Reduce overhead when solving a long sequence of easy models– MIPs – one thread, limit presolve, cuts, heuristics (or disable completely).– LP, QP – limit or disable presolve, use only one thread, group problem modifications

together in as few function calls as possible


Statistics: 559 constraints, 1066 variables (516 binary, 516 general integer)

Node log:

Nodes Cuts/

Node Left Objective IInf Best Int Best Node ItCnt Gap

0 0 101984.7744 28 101984.7744 35

*0+ 0 0 4.10026e+08 101984.7744 35 99.98%

153036.9306 35 4.10026e+08 Cuts: 41 151 99.96%

*0+ 0 0 4.00022e+08 153036.9306 151 99.96%

…

*55950+ 0 1.02822e+07 202475.0432 98.03%

56000 infeasible 1.02822e+07 202518.1842 98.03%

Elapsed time = 186.20 sec. (tree size = 13.36 MB).

Tightening the formulation: Penalty variables


Node log (ctd):

Nodes Cuts/

Node Left Objective IInf Best Int Best Node Gap

7149e4 7726073 infeas 1.02822e+07 307724.1416 97.01%

7150e4 7727024 309418.1 33 1.02822e+07 307728.0479 97.01%

Elapsed time = 161631.76 sec. (tree size = 9072.93 MB).

Nodefile size = 8945.58 MB (3124.04 MB after compression)

7151e4 7727720 357983.4 22 1.02822e+07 307731.4823 97.01%


…


Determine how fractional solutions affect the objective:

Min obj: 10000000 id134 + 10000000 id135 + ...

+ 10000000 id161 + 10000 id168 + 10000 id169

+ 1000 id170 + 1000 id171 + 34.299999237 id200

+ … + 10000 id2309

id78: id134 - id135 + 3 id200 + 3 id204 + 3 id220

+ 3 id228 + 3 id248 + ... + 3 id2096 + 3 id2144

+ 2 id2148 = 4

Tightening the formulation : Penalty Variables

(Implied integer by integrality of other variables in the constraint)


Determine how fractional solutions affect the objective(ctd):

Nodes Cuts/

Node Left Objective IInf Best Int Best Node Itcnt Gap

…

*55950+ 11356 0 1.02822e+07 202475.0432 861218 98.03%

56000 11367 infeasible 1.02822e+07 202518.1842 862287 98.03%

Elapsed time = 186.20 sec.

Comparing the best integer and best node values, we see that

removing integrality enables solutions with the sum of the

expensive penalty variables << 1. But, we don't know yet whether

an integer solution exists with all such penalty variables set to 0.

Can we answer that question?

Tightening the formulation : Penalty variables



Yes we can, by solving a related problem:

Add a constraint that sets all the expensive penalty

variables to 0:

conobj: id134 + id135 + id136 + id137 + … + id161 = 0

Results:

MIP - Integer infeasible or unbounded.

Current MIP best bound is infinite.

Solution time = 18.80 sec. Iterations = 409663

Nodes = 38384


Solve another related problem, using solution objective value:

Nodes Cuts/


…

*55950+ 11356 0 1.02822e+7 202475.0432 861218 98.03%

56000 11367 infeasible 1.02822e+7 202518.1842 862287 98.03%

Elapsed time = 186.20 sec.

conobj: id134 + id135 + id136 + id137 + id138 + … + id161 >= 1


conobj: id134 + id135 + id136 + id137 + id138 + … + id161 = 1


0 0 1.01576e+07 16 1.01576e+07 54

1.01851e+07 20 Cuts: 42 82

…

*205 160 0 1.02922e+07 1.02098e+07 1233 0.80%

…

58200 319 infeasible 1.02822e+07 1.02806e+07 440316 0.02%

58300 226 cutoff 1.02822e+07 1.02811e+07 440441 0.01%

MIP - Integer optimal, tolerance (0.0001/1e-06):

Objective =1.0282191250e+07




Solve another related problem, using solution objective value:

Nodes Cuts/



Tightening the formulation – key points Determine how fractional solutions affect the objective

– This often sheds light on how to tighten the formulation

– Helps identify the significant variables and constraints that make the model

challenging

– Can then often combine the variables and constraints to derive cuts

– Disjunctions can also provide useful cuts

Solve one or more related models

Use infeasibility

Use solution objective value

Assess carefully the benefit of combining multiple objectives into a single objective – Solve separate problems with each individual objective frequently works better


Performance variability in MIPs

Branch and Bound can be affected by the optimal root node LP basis– Most LPs solved in practice have numerous alternate optimal bases

• Alternate optimal bases have different fractional variables and solution values– Factors influencing the path taken by simplex or barrier algorithms during the root

node • Slight differences in precision on different hardware (e.g. Power7 vs. Intel)• Differences in the code generated by different compilers• Slight differences in the precision of the problem data

– MPS vs SAV format

– Any non determinism, difference in precision in the model data calculations• Differences in the ordering of the variables or constraints in the model

– LP format representation may order variables differently • Seemingly irrelevant changes to the solution process

– Solving the root node with barrier instead of dual simplex

– Timing between threads if the parallel MIP algorithm is not implemented in a deterministic way

– Changing the number of threads used

– Changing the random seed parameter



Branch and Bound can be affected by the optimal root node LP basis (ctd)– Branch and Cut features influenced by alternate optimal bases

• Branching selection• Gomory cuts affected explicitly• Other cuts affected implicitly regarding which ones are actually separated (i.e.

violated and hence added to the problem)

Most models don’t exhibit large amounts of variability– But the ones that do can be time consuming regarding legitimate performance

improvements• Performance improvement on one model instance doesn’t carry over to similar

data instances• Changing hardware or operating system suddenly results in slower performance• Other seemingly irrelevant changes lead to significant performance degradation

(or improvement)– Need techniques to identify and remedy performance variability



Example: neos-911970– Noted as highly variable (Fischetti, Monaci, Salvagnin, ISMP 2012,

http://ismp2012.mathopt.org/show-abs?abs=579)– CPLEX 12.5.1, 10 random seeds:

Solution time = 7.98 sec. Iterations = 193900 Nodes = 25729Solution time = 306.77 sec. Iterations = 21448972 Nodes = 2048214Solution time = 10.35 sec. Iterations = 315328 Nodes = 38887Solution time = 29.14 sec. Iterations = 1515915 Nodes = 221650Solution time = 1251.60 sec. Iterations = 95977382 Nodes = 11620625Solution time = 12.13 sec. Iterations = 488977 Nodes = 64391Solution time = 8.59 sec. Iterations = 253203 Nodes = 36728Solution time = 6.50 sec. Iterations = 167506 Nodes = 25383Solution time = 7.04 sec. Iterations = 190506 Nodes = 25711Solution time = 7.24 sec. Iterations = 200383 Nodes = 28903

http://ismp2012.mathopt.org/show-abs?abs=579





Example: neos-911970 (ctd)– Look at node log of the slowest run

– Variability involves best node, notbest integer

Nodes Cuts/Node Left Objective IInf Best Integer Best Bound ItCnt Gap

24273 9581 54.7330 15 54.7600 54.7330 139099 0.05%

49915 21576 54.7472 19 54.7600 54.7330 252354 0.05%Elapsed time = 5.86 sec. (3329.66 ticks, tree = 10.88 MB, solutions = 21)73518 31045 54.7457 17 54.7600 54.7330 373447 0.05%96452 40745 54.7330 23 54.7600 54.7330 504888 0.05%

…

11493139 73139 cutoff 54.7600 54.7495 95264869 0.02%

11548161 50340 cutoff 54.7600 54.7495 95619617 0.02%11603445 17177 cutoff 54.7600 54.7500 95920869 0.02%

Elapsed time = 1249.93 sec. (817299.20 ticks, tree = 18.10 MB, solutions = 21)

Optimal solution within 6 seconds

Lack of progress in best node (including 2 million nodes without change)



Example: neos-911970 (ctd)

Parameter tuning based on node log of the slowest run– Tried various settings to improve progress in best node

• MIP emphasis = 2 or 3, variableselect = 3, aggressive symmetry detection• Did not help

– Take a look at the model– But first, kick off a run with the tuning tool

• It includes a parameter that allows the specification of the number of times to repeat a tuning test

• Run each test with 10 different permutations of constraints and variables• Recommended setting backtrack tolerance to 0 (pure best bound search), but

that did not significantly reduce variability


Performance variability in MIPs Example: neos-911970 (description)

Description: 24*35 = 840 binaries, 48 continuous penalty variables whose sum is to be minimized

C49 C50 C51

C73 C74 C75

C97 C98 C99

C72

C96

C120

…

C865 C866 C867 C888…

…

24 columns

35 rows

• 2 sets of 24 soft knapsack constraints

by column of grid

• Sum of binaries across rows = 1

• Sum of binaries across columns >= 1

• First set of soft knapsacks incur penalty if 2 or more binaries = 1

• At least 35-24 = 11 such penalties must be positive

• First set of knapsacks have identical weights for each column

• Second set of knapsacks have identical weights in consecutive groups of 3 columns



Example: neos-911970 (ctd)

Challenging aspects of model– Penalty variables on knapsack constraints inhibit cut generation

• Cover cuts• Possibly clique cuts

– Model suffers from symmetry and near symmetry• Penalty variables also limit symmetry detection• Binaries tend to have lighter weights in one knapsack constraint but heavier

weights in the other

Why do the run time vary so much?– Suspect some complex groups of highly symmetric solutions depend heavily on the

branching sequence regarding whether they have to be processed.


Performance variability in MIPs – key points

Performance tune based on the worst runs– Examine node log to identify source of variability

• Symmetry• Node heuristics depend on path taken, node at which they are applied• Remaining weakness in the formulation

– Use the tuning tool• Repeat parameter enables multiple runs

Random seed parameter helps assess level of variability– Run with multiple random seeds, check variability in run times

Ill conditioning in the model can contribute to performance variability– Small change to input leads to big change in the output

More info on variability and its effect on benchmarking– Mixed Integer Programming: Analyzing 12 Years of Progress, Roland Wunderling

(Tobias Achterberg) Sunday, SD-02, 4:30PM– Performance Variability in Mixed Integer Programming, Andrea Lodi (Andrea

Tramontani) Tuesday, TA-49, 8AM


Conclusions Node log provides detailed information regarding the different factors that

influence MIP performance

Set parameters based on the sources of lack of progress

As CPLEX has evolved, identifying calculations to reduce from default levels may

improve performance

CPLEX’s tuning tool can identify useful parameter settings that otherwise would

have been hard to find

– May help refine user-derived settings based on node log

Penalty variables on soft constraints pose a particular set of challenges

– They weaken or disable cuts

– They result in blended objectives that may be better solved separately

Performance variability can cause large changes in run time from seemingly

insignificant changes in the model, algorithm or computing environment

– Use CPLEX’s randomseed parameter to assess

– Same techniques to address consistent performance issues apply for

inconsistent ones


References MIP Performance tuning and formulation strengthening

– Klotz, Newman. Practical Guidelines for Solving Difficult Mixed Integer Programs

http://www.sciencedirect.com/science/article/pii/S1876735413000020

LP performance issues– Klotz, Newman. Practical Guidelines for Solving Difficult Linear Programs


Performance Variability

– Emilie Danna, Performance Variability in Mixed Integer Programming

http://coral.ie.lehigh.edu/~jeff/mip-2008/talks/danna.pdf

– Koch et. al., MIPLIB 2010, Mathematical Programming Computation, 3:2 (2011)

103-163

– Fischetti, Monaci, Salvagnin, Randomness and Tree Search, ISMP 2012,






Mining the CPLEX Node Log for Faster MIP Performance

Software

Transcript of Mining the CPLEX Node Log for Faster MIP Performance