Model-Based Decision Making: On Supporting Deliberation with Metaheuristics

Introduction Further Example Extended Example Evolutionary Computation Discussion

Model-Based Decision MakingOn Supporting Deliberation with Metaheuristics

Steven O. Kimbrough

University of Pennsylvania

INFORMS Practice Conference, 2009.04.26–28

1 / 58


Acknowledgements

Collaborators: CHENG Shih-Fen, Gary Koehler, Ann Kuo,Lindawati, LAU Hoong Chuin (HC), Ming Lu, David HarlanWood, D.J. Wu

2 / 58


Outline1 Introduction

DeliberationSimple Example: Simple Knapsack

2 Further ExampleGAP: The Generalized Assignment Problem

3 Extended ExampleGAP4_2 FoIsGAP4_2 IoIs

4 Evolutionary ComputationMotivationBasicsHard Problems

5 DiscussionKey PointsVision

3 / 58


Deliberation

Given an optimization problem. . .

How can we recognize a modeling opportunity?How should we build the model(s)?How can we get a good solution?What should our decision be?

Our focus: this last question. The deliberation problem.

Recognize: The questions interact.

4 / 58


Deliberation

Knapsack Problems

Canonical form for the Simple Knapsack model

max z =n∑

i=0

pixi (1)

subject to the constraints

n∑i=0

wixi ≤ c (2)

xi ∈ {0,1}, i = 0,1,2, . . . ,n. (3)

5 / 58


Deliberation

Canonical GAP (= or ≤ on (6))

max z =m∑

i=1

n∑j=1

cijxij (4)

Subject to the constraints:

n∑j=1

rijxij ≤ bi i = 1, . . . ,m (5)

m∑i=1

xij ≤ 1 j = 1, . . . ,n (6)

xij ∈ {0,1} ∀i , j (7)

6 / 58


Deliberation

Deliberation

Given a good solution, provide information relevant toimplementing the decision and to reconsidering the model.Think: post-solution analysis, sensitivity analysis,post-evaluation analysis, etc.The candle-lighting principle: “It is better to light onecandle than to curse the darkness.”The best response to a tight constraint may well be toloosen it.Generalize the point: other aspects of the model.Uncertainty, action.

7 / 58


Deliberation

Shadow prices and reduced costs

From linear programming. Address the candle-lightingprinciple.How much would it be worth to loosen these constraints?How much would it cost us to tighten these constraints?And similar questions.All in support of deliberation.How can these questions be addressed outside of linearprogramming models?

8 / 58


Simple Example: Simple Knapsack

z to c Response Curve (knapsack101)

160 170 180 190 200 210 220 230 2401000

1050

1100

1150

1200

1250z to c Response Curve for knapsack101 Problem

c: right−hand−side values

z: o

bjec

tive

func

tion

valu

es

9 / 58



z to c Response Curve: Zooming in

195 196 197 198 199 200 201 202 203 204 2051100

1105

1110

1115

1120

1125

1130

1135z:

obj

ectiv

e fu

nctio

n va

lues

c: right−hand−side values

z to c Response Curve for knapsack101 Problem

10 / 58



Two Challenges

We are often interested in z as a function of many varyingparameters, not just one, c in this case.

Response surface, not response line.The resulting search space will often be too large tosupport re-solving the model comprehensively.

The original problem may be solved only expensively,typically with a metaheuristic, so re-solving often is notpossible, even in a relatively small search space.

11 / 58


GAP: The Generalized Assignment Problem

Illustrative Example: GAP5_1

An organization is presented with a resource allocationproblem for which it has formulated a 0-1 integer program,called GAP5_1.The problem has 24 decision variables and 8 capacityconstraints, for a search space (∈ {1,2,3,4,5,6,7,8}) of824 = 272 = 4722366482869645213696 ≈ 4.7× 1021.A standard branch-and-bound solver (CPLEX) has foundthe optimal solution, x∗, for which z∗ = 563.

Note: OR-Library, J.E. Beasley.http://people.brunel.ac.uk/~mastjjb/jeb/info.html,[1].

12 / 58

http://people.brunel.ac.uk/~mastjjb/jeb/info.html



Solving with CPLEX

CPLEX> readName of file to read: GAP5_1.lpProblem ’GAP5_1.lp’ read.Read time = 0.00 sec.CPLEX> mipopt

...

Solution pool: 2 solutions saved.MIP - Integer optimal solution:Objective = 5.6300000000e+02Solution time=0.03 sec. Iterations = 0Nodes = 0CPLEX>

13 / 58



Should we implement x∗?

If this were a linear programming problem. . .We would look at shadow prices, reduced costs, &c.Are there constraints that could be tightened at little costand the resources freed up deployed profitably elsewhere?Are there constraints that could be relaxed for a cost thatwould be covered by the benefit of a better solution?Perhaps we are sellers in an combinatorial auction. Wewant to consider acquiring/releasing resources in thefuture. Or negotiating with certain bidders.And so on. We would engage in deliberation (aka:post-solution analysis, sensitivity analysis, post-evaluationanalysis).But this isn’t LP, so what can we do? How can we getsupport for this sort of deliberation?

14 / 58



Sensitivity and Post-Solution Analysis

CPLEX> display sensitivityNot available for mixed integer problems.Use CHANGE PROBLEM to change problem type.CPLEX>

What to do?

15 / 58



Recall: Solution pool: 2 solutions saved.

CPLEX> display solution list *Change from

Solution Name Objective Value Incumbentp1 555.0000 8.33%p2 563.0000 0.00%CPLEX>

16 / 58



CPLEX> display solution member 1 slacks *Solution p1Constraint Name Slack Valueslack c1 1.000000slack c3 4.000000slack c4 7.000000slack c7 2.000000slack c8 2.000000All other slacks matching ’*’ are 0.

17 / 58



CPLEX> display solution member 2 slacks *Solution p2Constraint Name Slack Valueslack c1 2.000000slack c2 9.000000slack c3 7.000000slack c4 6.000000slack c7 3.000000slack c8 3.000000All other slacks matching ’*’ are 0.CPLEX>

18 / 58



Illustrative Example:

GAP5_1 CPLEX Solution Pool Displayed in Resource UsageMode

Capacity ConstraintsConstraint: C1 C2 C3 C4 C5 C6 C7 C8

∑slacks

Capacity: 36 35 38 34 32 34 31 34 —Sol’n. Obj.2: x∗ 563 2 9 7 6 0 0 3 3 30

1: 555 1 0 4 7 0 0 2 2 16

At optimality, there is a slack of 2 on constraint 1, 9 onconstraint 2, etc.Potentially useful information for deliberation, especially whencombined with the corresponding solutions.

19 / 58



But. . . it would be nice to have more

Are there other interesting feasible solutions?If so, how might we get them?What about interesting infeasible solutions?How might we get them?

20 / 58



Obvious approach: change the constraints andre-solve the problem

In this simple example, the maximal slack valueencountered was 9. If we sweep out the range oftightening from 1–9 on each constraint, there are 98 =43,046,721 possibilities.At 100 solutions per second, it would take

4304672160×60×24×100 ≈ 5 days to make the calculations.

And what about infeasible solutions? If we also sweep these toa depth of 9 we get 188/8640000 = 1,275.45 days.(Efficiencies are possible, these are upper bounds, but thebasic point remains.)

21 / 58



Illustrative Example: GAP5 Optimal Solution Displayedin Resource Usage Mode


∑Slacks

Capacity: 36 35 38 34 32 34 31 34 —Sol’n. Obj.

x∗ 563 2 0 2 5 4 1 3 2 19

At optimality, there is a slack of 2 on constraint 1, 0 onconstraint 2, etc.

22 / 58



Some of the Alternative Feasible Solutions Discoveredby Evolutionary Computation for GAP5_1


∑Slacks

Capacity: 36 35 38 34 32 34 31 34 —Sol’n. Obj.

x∗ 563 2 0 2 5 4 1 3 2 191 561 2 0 7 5 3 0 6 9 322 560 2 0 0 7 11 5 2 2 293 560 2 0 8 5 4 7 3 9 382 559 2 0 12 7 11 5 2 2 413 559 20 0 1 5 4 0 6 9 45

23 / 58



There are (at least) two dozen alternative feasible solutionswhich are close in value to the optimal solution (559/563 =0.993) and that offer seemingly large freeing of resources.Depending on prices (not represented in the model), someof these alternatives may be very attractive.We found about 2000 distinct feasible solutions withobjective function values ≥ 550. These are data fordeliberation. Note that EC also found an alternate optimalsolution.All this with modest computational effort.

24 / 58



Some Interesting Infeasible Solutions Discovered byEvolutionary Computation

Capacity Constraints Sum SumConstraint: C1 C2 C3 C4 C5 C6 C7 C8 Sks SpsCapacity: 36 35 38 34 32 34 31 34 — —

Sol’n. Obj.x∗ 563 2 0 2 5 4 1 3 2 19 01 570 20 0 -17 6 11 13 3 -27 53 -442 569 12 -19 -26 7 21 24 2 -7 66 -523 569 2 0 -41 6 11 18 3 -11 40 -524 569 12 15 -13 7 11 11 2 -22 58 -355 568 12 0 -13 17 4 11 2 -7 46 -206 568 12 0 -13 7 11 11 2 -7 43 -207 567 12 0 -13 18 0 11 2 -11 43 -24

Going beyond mere optimality by juggling constraints.Makes you think. . .Depending on prices and opportunities. . . 25 / 58



An Observation

Varying the capacities on the constraints, relaxing themand tightening them, creates a new search space. Howlarge is that space?The largest differential on the tightening side is 20. Goingwith just 13 on the relaxing side, 20+13 = 33. With 8constraints (and limiting ourselves to integers) this is338 = 1,406,408,618,241 ≈ 1.4× 1012. Too many timesto re-solve the problem. (Again, efficiencies are possible,these are upper bounds, but the basic point remains.)We need another method, even for such small problems.How can we find interesting, non-optimal solutions?

26 / 58


GAP4_2 FoIs

GAP4_2

A representative GAP test problem.OR-Library, J.E. Beasley. http://people.brunel.ac.uk/~mastjjb/jeb/info.html, [1].c530-2 644. 5 machines, 30 jobs. 644 at optimum.Solved with FI-2Pop GA.FoIs: top 1000 feasible solutions, ranked by objectivevalue.IoIs: top 1000 infeasible solutions, ranked by distance tofeasibility.

27 / 58




GAP4_2 FoIs

Can we find an optimal solution? Yes. Two!

Optimal Solution A0 1 2 3 4 5 6 7 8 9

0 – 3 3 5 1 2 1 3 1 41 2 3 2 1 4 5 5 2 4 52 3 4 5 3 4 2 1 4 1 53 2 – – – – – – – – –

Optimal Solution B0 1 2 3 4 5 6 7 8 9

0 – 3 3 5 1 2 1 4 1 41 2 3 2 1 4 4 5 2 2 52 3 4 5 3 5 3 1 4 1 53 2 – – – – – – – – –

28 / 58


GAP4_2 FoIs

Now let’s compare them on their constraint slacks

Comparison of SlacksSolution 1 2 3 4 5A=644 2 5 0 1 1B=644 2 1 1 2 0

Decision makers may well prefer A over B or vice versa.

29 / 58


GAP4_2 FoIs

Shadow-price-like questionsObj. Val. 1 2 3 4 5A=644 2 5 0 1 1B=644 2 1 1 2 0

643 2 5 1 4 1643 2 1 1 2 0643 0 4 1 11 0643 5 4 1 3 0643 2 1 1 3 0642 4 0 1 0 1642 2 1 0 1 0642 2 5 4 1 1642 5 4 3 1 0642 0 4 3 9 0641 2 1 1 4 0641 2 5 0 3 1

Can the additional slackresources be usefullydeployed?On which resources dowe not have muchopportunity forredeployment?(Ans: 2 & 5.)

30 / 58


GAP4_2 FoIs

Shadow-price-like questions

In both of my optimal solutions, constraint 1 has a slack of2. Are there any good solutions available with a slack of atleast 10?Yes. #23 has a slack on constraint 1 of 10 and an objectivevalue of 640; #53, 11, 638; #97, 13, 636.What about constraint 2, which already has a slack of 5?#18, 641, 8; #68, 637, 15; #74, 637, 18.3? At 0 or 1. #10, 642, 4; #39, 639, 8; #48, 638, 10.4? At 1 or 2. #5, 643, 11.5? At 0 or 1. #35, 639, 9.Plus, we can do combinations. . .

31 / 58


GAP4_2 FoIs

Slacks for top 100 feasible solutions

12

34

5

0

20

40

60

80

1000

5

10

15

20

32 / 58


GAP4_2 FoIs

Reduced-cost-like questions

Job 25 is assigned to machine 2 in solution A and machine3 in solution B. What’s the best solution if we assign job 25to machine 1?Answer: It has an objective value of 643. The solution is

Job 25 to machine 10 0 1 2 3 4 5 6 7 8 90 – 3 3 5 1 2 3 4 1 41 2 4 2 1 4 5 5 2 2 52 3 4 3 3 5 1 1 4 1 53 2 – – – – – – – – –

Its slacks:Obj. Val. 1 2 3 4 5

643 2 1 1 3 0

33 / 58


GAP4_2 IoIs

Shadow-price-like questionsObj. 1 2 3 4 5645 0 4 1 -1 5644 -1 5 1 3 0645 -2 2 1 4 1644 2 5 0 -2 0646 -2 2 0 1 1645 0 -2 0 3 1644 -2 8 0 0 2646 5 -2 0 5 0645 -2 0 7 0 1648 0 -2 1 0 1644 2 -2 1 11 -2644 2 -2 1 -2 1646 -2 -2 1 2 0645 -2 8 -2 4 1

Is there opportunity toacquire more resources?To trade in slacks andsurpluses?

34 / 58


GAP4_2 IoIs

Shadow-price-like questions

We’re at 644, but we need to get to 651. Show me what Ineed to do.In the top 1000 IoIs there are 75 discovered solutions withobjective values at or above 651.

35 / 58


GAP4_2 IoIs

Shadow-price-like questions. 651?Obj. 1 2 3 4 5

A=644 2 5 0 1 1B=644 2 1 1 2 0

651 -6 -9 1 1 0651 -3 -10 1 5 0651 5 1 1 -4 -12651 2 5 2 -1 -13651 2 -6 0 -13 1651 -10 -2 1 11 1651 -6 -13 1 5 0651 5 -7 4 -12 0651 -5 -7 -5 -1 -12651 2 -6 -13 6 1651 -13 4 1 8 0651 2 -9 0 5 -12

Is thereopportunity toacquire moreresources?To trade inslacks andsurpluses?

36 / 58


GAP4_2 IoIs

Shadow-price-like questions. >651?Obj. 1 2 3 4 5654 -9 0 -9 5 1654 0 -6 1 -8 1654 -6 -5 1 -4 1653 2 -6 1 -12 1653 -5 -11 0 6 1653 2 -13 1 -3 0653 2 -10 -1 -4 0652 -2 4 -10 -5 0652 -3 4 -9 5 0652 -5 5 0 -8 1652 -8 5 0 -2 1652 2 1 1 -9 0652 -5 1 1 -7 0652 2 1 1 -6 1

Is thereopportunity toacquire moreresources?To trade inslacks andsurpluses?

37 / 58


GAP4_2 IoIs

Reduced-cost-like questions

Job 20 is assigned to machine 3 in solutions A and B, andto 3 (and sometimes 4) in all the FoIs.There are 57 solutions in the IoIs in which we assign job 20to machine 2.What are the most promising of these solutions?

38 / 58


GAP4_2 IoIs

Reduced-cost-like questions. 20 to 2?Obj. 1 2 3 4 5647 2 -2 0 6 -5646 2 -2 -12 -8 1646 2 -2 1 9 -5645 0 -6 0 -5 -12645 -8 -8 1 2 8645 2 -2 -4 -9 9645 4 -7 1 5 -5645 4 -7 -1 6 -3645 -5 -3 1 1 8644 -8 -12 0 -1 -1644 2 1 -4 2 -13644 13 -2 0 1 -3644 2 4 0 1 -12644 2 -2 1 11 -2644 0 1 -7 7 -5644 0 1 1 9 -5644 -2 1 7 0 -4

Is thereopportunity toacquire moreresources?

To trade inslacks andsurpluses?

Note: We do notfind anopportunity with20 assigned to 2and z > 647.

39 / 58


GAP4_2 IoIs

11.5

22.5

33.5

44.5

5

0

10

20

30

40

50−100

−50

0

50

40 / 58


Motivation

How can we, how did we, find the FoIs and the IoIs?

Through evolutionary computation.Observation: Population-based heuristics profusely andintelligently sample the solution space in the vicinity ofgood solutions.Idea: Use these samples to derive the FoIs and IoIs.

41 / 58


Basics

Evolutionary Algorithm Schema

1 Initialize parameters.2 Create, P0, the initial population of solutions.3 P ← P0

4 done← false5 Do while not done:

1 Evaluate and record the fitness of each member of P.2 Select members of P to compose the next generation, P ′.3 Perturb the members of P ′.4 P ← P ′5 If stopping condition is met, done← true

42 / 58


Basics

Comments

There are many, many variations on this schema. I’ll focusprototypically on genetic algorithms, GAs.A solution is typically a string of numbers, serving asarguments to an evaluation function.For example, in a simple knapsack problem, solutionsmight be represented as lists of 0s and 1s, e.g.0 0 1 0 1 1

for a 6-item knapsack problem. Note: many otherencodings and representations are possible, e.g., toinclude representations of parameters.

43 / 58


Basics

Comments (con’t.)

Typically, the initial population in a GA consists ofrandomly-generated solutions.Useful: seed the initial population with promising solutionsobtained otherwise.Fitness evaluation is performed on each solution. This istypically the most expensive operation and is recognizedas the standard point of comparison for different methods.Selection can be done in many different ways, but it isalways done on the basis of fitness.We use tournament selection . . .

44 / 58


Basics

Comments (con’t.)

Perturbation of the next generation population memberscan be done in many, many different ways.We are using just mutation and recombination bysingle-point crossover. Prototypical for a simple GA.Mutation: randomly, with some probability, change thevalue of a locus, e.g.,

0 0 1 0 1 1 −→ 0 1 1 0 1 1

45 / 58


Basics

Comments (con’t.)

Recombination by single-point crossover: randomly, withsome probability, exchange information between twoparents/members of the next generation, e.g.,0 0 1 0 1 1 −→ 0 0 1 0 0 00 0 1 1 0 0 −→ 0 0 1 1 1 1

Analogous to sexual recombination.Stopping condition.Take your pick. We fix the number of generations (newpopulations created) in order fix the number of fitnessevaluations, for comparing different GAs.

46 / 58


Hard Problems

Hard Problems

Require heuristic solvers.With more nuance, most interesting formalized decisionproblems are NP-hard and hence (we assume) have nofast (polynomial) solver in the worst case.But many such problems are routinely solved very well withfast solvers.And many are not. For these we are forced to useheuristics, including meta-heuristics, such as GAs andvarious forms of evolutionary computation, tabu search,simulated annealing, particle swarm optimization, artificialimmune systems, ant colony optimization, bespokeheuristics, . . .

47 / 58


Hard Problems

Constrained Optimization Models: Sources of HardProblems

Integer programming, mixed integer programming,non-linear programming, stochastic programming,multi-objective problems, . . . all typically present hardproblems for which only heuristic solvers are available.Explosion of interest in applying heuristics andmeta-heuristics to ConOptModels.Including extensive use in practice.

48 / 58


Hard Problems

Canonical Form for ConOptModels

max~x

z = d(~x) (8)

subject to the constraints

fi(~x) ≤ ai , i = 1,2, . . . ,nf (9)gj(~x) = bj , j = 1,2, . . . ,ng (10)

xl ∈ Sl , l = 1,2, . . . ,nl (11)

Figure : Canonical form for single-objective, deterministicConOptModels. ~x = 〈x1, x2, . . . , xnl 〉

49 / 58


Hard Problems

But, there’s a problem for GAs (and others) withconstrained optimization

The genetic operators—mutation, recombination,etc.—generally do not respect feasibility.How to handle constraint violations when a GA is beinguse to solve a ConOptModel?Approaches to handling infeasible solutions: death penalty(wasteful), penalty functions (but which?), special encodingto guarantee feasibility (rarely available), repair ofinfeasible solutions (problem-specific; computationallycostly, often).The penalty function approach is perhaps most natural andis widely used (e.g., MATLAB). Still, a number of publishedworries.

50 / 58


Hard Problems

Our New Idea, FI2Pop GA: Separate Feasibility andOptimality Evaluations

Maintain two populations in the GA: a population offeasible solutions and a population of infeasible solutions.Fitness in the feasible population is determined solely bythe objective function.Fitness in the infeasible population is determine solely as afunction of distance to feasibility. (We used the sum ofconstraint violations.)Each population is processed in turn with a conventionalGA. Children go to the feasible or infeasible population,depending on. . . whether they are feasible or infeasible.

51 / 58


Hard Problems

Performance Testing

Over a series of studies, we have found excellentperformance on (apparently) hard problems.We have also investigated, both theoretically andempirically, the behavior of the FI2Pop GA. [2]Note: Mindful of the No Free Lunch Theorems.

Typical behavior: FI-2Pop GA is competitive with penalty GAson FoIs and superior on IoIs.

52 / 58


Key Points

Summary

Key for deliberation support:FoIs: Feasible solutions of Interest: Feasible, non-optimalsolutions with good objective values.IoIs: Infeasible solutions of Interest: Infeasible solutionsnear feasibility with good objective values.

These can be found by intelligent sampling of the solutions.This is just what metaheuristics, such as GAs, do.

53 / 58


Key Points

Concept

If we had all possible solutions, we could fully answer ourpost-solution deliberation questions. (But we can’t havethat.)If we could get a representative random sample of all thesolutions, we could have good estimates for ourpost-solution deliberation questions. (But the searchspaces are too large.)If we could bias our sampling to the regions of interest(near the boundary(ies) of the feasible region(s)), we couldhave good estimates for our post-solution deliberationquestions.That’s what we are reporting. And effective means to do it.

54 / 58


Key Points

To be done

Explore metaheuristics fully for purposes of deliberationsupport.Two categories:

Individual, local search based.Tabu search, simulated annealing, etc.Population based search.Evolutionary computation, particle swarm optimization, antcolony optimization, artificial immune systems, scattersearch, etc.

What works best for producing FoIs and IoIs? Scalingissues? OGIT.

55 / 58


Vision

We envision. . .

A transformation in how complex COModels are used inpractice.Estimates given for all post-solution questions of interest.A great broadening of the scope of use and application ofCOModels.An empowering of users, as information for decisionsupport is made readily available.

56 / 58


Vision

J. E. Beasley.OR-Library.World Wide Web, Accessed July 27, 2009.http://people.brunel.ac.uk/~mastjjb/jeb/info.html.

Steven Orla Kimbrough, Gary J. Koehler, Ming Lu, andDavid Harlan Wood.On a feasible–infeasible two–population (FI-2Pop) geneticalgorithm for constrained optimization: Distance tracing andno free lunch.European Journal of Operational Research,190(2):310–327, 2008.

57 / 58




Vision

$Id: deliberation-COModel-master-beamer.tex 3607 2013-06-23 06:20:09Z sok $

58 / 58

Model-Based Decision Making: On Supporting Deliberation with Metaheuristics

Business

Transcript of Model-Based Decision Making: On Supporting Deliberation with Metaheuristics