IBM 기업가치연구소 IBM 2015년의의료 서비스와모델 · ibm 글로벌비즈니스서비스 ibm 기업가치연구소 2015년의의료 서비스와모델 서비스제공모델개선
Welcome to the IBM Presentation Template — IBM Plex...
Transcript of Welcome to the IBM Presentation Template — IBM Plex...
EWOSeminar
Eray Cakici, Ph.D.
January 23, 2020
IBMConstraint Programming (CP) — An Alternative Approach to Heuristics in Scheduling
Data Science & AI Elite Team
Agenda
1
Introduction
2
Computational Study
3
Conclusions
2Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Motivation
Overview of CP
How It Works?
Modeling
Scheduling Problems
CP vs MIP & Heuristics
Future Research
Motivation
3Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
• The manufacturing of integrated circuits (IC) on silicon wafers is a complex production process
• Companies strive to produce a large number of chips in the least amount of time possible at minimum cost
• An improvement in the semiconductor manufacturing can provide huge financial gains
• “TSMC invested $9.3 billion in its Fab15 300 mm wafer manufacturing facility in Taiwan. The same company estimations suggest that their future fab might cost $20 billion” --- Wikipedia
Constraint Programming (CP) can be an alternative approach to replacewidely used heuristics
Constraint Programming | Overview
4
• Solving complex problems with a set of constraints and unknown discretevariables
• constraint satisfaction (feasibility) and optimization problems
• Natively supports logical constraints, specialized constraints i.e.“all-different”, and has no limitations on the arithmetic constraints (i.e. non-linear)
• A rich domain for Artificial Intelligence, Computer Science, and OperationsResearch
• backtracking, constraint propogation, branching heuristics
• Very powerful on combinatorial feasibility and detailed scheduling problems
Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
IBM Cplex Optimization Studio
5Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
APIs• Java• .NET• C++
Optimization Engines
Math ProgrammingIBM ILOG CPLEX Optimizers (Simplex, Barrier, Mixed Integer)
Model Development ToolsIntegrated Development Environment (IDE)
ILOG Concert Technology (C++, .NET, Java)
Connectors• Microsoft Excel• MATLAB• AMPL• Python• MSF
(Callable libraries)
Optimization Programming Language (OPL) & OPLScript
Complete toolkit for prototyping, building and deploying analytical decision support applications using constraint programming and/or mathematical programming
Custom Algorithms
Constraint Programming (CP) IBM ILOG CPLEX CP Optimizer
How CP works?
6
• CP is a constructive approach
• Values are assigned to variables one at a time toextend a partial solution to a complete solution
• At a point, it may be useless to further extend a partial solution as at least one constraint is alreadyviolated by the partial solution
• The solver backtracks and tries a different value for a previously assigned variable
• All possible assignments of values to variables can be examined in this way
Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
How CP works? (cont’d)
7
• In CP, the basic search behavior is tree search
• Including space reduction via domain filtering
• Domain filtering
• Before each value-variable assignment, domain filtering occurs
• Each value of a variable which cannot be usedin a solution (given the current assignment) can be removed
• Each constraint type has a specializedalgorithm which filters domains
Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
X
Domain Filtering
8Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
1
2
3
4
5
1
3
1
3
4
1
4
1
2
4
5
A B C D E
All different
Domain Filtering
9Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
1
2
3
4
5
1
3
1
3
4
1
4
1
2
4
5
A B C D E
All different
X
X
X
X
X
Uses matching theory
adapted from https://www.slideshare.net/PaulShawIBM/cp-optimizer-may2013 – Paul Shaw – CP Optimizer Development Manager at IBM
CP Constructive Search Process
10Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Select a decision variable
Assign a value after domain filtering
Constraint propagation (reduce domains of all other variables)
backtrack on failurecontinue on success
Selection StrategiesMost constrained variables first
…
Assignment StrategiesExplore branches
Try first, try last
…
A Complete Search Tree
11Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
E ≠ 4
S ∈ {9}E ∈ {4..7}N ∈ {5..8}D ∈ {2..8}M ∈ {1}O ∈ {0}R ∈ {2..8}Y ∈ {2..8}E = 4
E ≠ 6
S ∈ {9}E ∈ {6..7}N ∈ {7..8}D ∈ {2..8}M ∈ {1}O ∈ {0}R ∈ {2..8}Y ∈ {2..8}E = 6
S ∈ {9}E ∈ {5..7}N ∈ {6..8}D ∈ {2..8}M ∈ {1}O ∈ {0}R ∈ {2..8}Y ∈ {2..8}
infeasible
infeasible infeasible
S E N D+ M O R E= M O N E Y
E = 5 E ≠ 5
S ∈ {9}E ∈ {5}N ∈ {6}D ∈ {7}M ∈ {1}O ∈ {0}R ∈ {8}Y ∈ {2}
Solution found!
Assign distinct digits to letters such that equation holds
Agenda
1
Introduction
2
Computational Study
3
Conclusions
12Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Motivation
Overview of CP
How It Works?
Modeling
Scheduling Problems
CP vs MIP & Heuristics
Future Research
13Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Modeling | Optimization Programming Language (OPL)
Provides natural mathematical description of optimization models
High-level syntax for mathematical models • produces substantially simpler and
shorter code than general-purpose programming languages
• reduces effort and improves reliability of development, upgrades, and maintenance
Supports all expressions needed to model and solve problems using both mathematical programming and constraint programming
14Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Modeling | Examples
Logical constraints
And (&&), Or (||), Imply (=>), …
Specialized constraints
i.e. alldifferent(x1,x2, ...,xn)
Compatibility constraints
{pair} possible = {<1,1>, <2,4>};{pair} forbidden = {<3,5>};allowedAssignments(possible, x, y);forbiddenAssignments(forbidden, x, y);
15Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Modeling | Examples cont‘d
Interval variables
dvar interval Activity[a in Activities] optional in a.startmin..a.endmax size a.duration;
Expressions over interval variables
maximize sum(a in Activities) endOf(Activity[a]);
Precedence constraints
i.e. endBeforeStart(xi , xj , zij) => e(xi) + zij ≤ s(xj)
sequence, cumul, noOverlap, alternative, ...
16Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Modeling | Classical Examples
Variables
𝑋𝑋1,1, … ,𝑋𝑋𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛
Constraints
AllDifferent(𝑋𝑋𝑛𝑛,1, … ,𝑋𝑋𝑛𝑛,9) ∀ 𝑟𝑟 ∈ 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅
AllDifferent(𝑋𝑋1,𝑛𝑛 , … ,𝑋𝑋9,𝑛𝑛) ∀ 𝑐𝑐 ∈ 𝐶𝐶𝑅𝑅𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝑅𝑅
AllDifferent(𝑋𝑋1,1, … ,𝑋𝑋3,3)
𝑋𝑋1,1 ≠ 8
…
SEND + MORE = MONEYVariables
S, E, N, D, M, O, R, Y
Constraints
0 ≤ S, E, N, D, M, O, R, Y ≤ 9
AllDifferent(S, E, N, D, M, O, R, Y)
1000 S + 100 E + 10 N + D +1000 M + 100 O + 10 R + E =10000 M + 1000 O + 100 N + 10 E + Y
S ≠ 0M ≠ 0
17Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Scheduling Problems | Job-shop
Minimize makespan (the completion time of latest job),tardiness, weighted tardiness, …
op12
op41
op13
M1 M2 M3
J1
J2
J3
J4
op21 op22 op23
op31 op32
op11
op42 op43
18Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Scheduling Problems | Job-shop cont’d
CP Optimizer model for Job-shop (10 lines of code)
19Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Scheduling Problems | Flexible Job-shop
There are certain qualified machines for a given operation with different processing times
J1
J2
J3
J4
M1, M2 or M3147, 123, 145
M2 or M4130, 140
M4 or M5150, 160
M1, or M3214, 150
M2 or M366, 87
M3 or M5147, 145
M1 or M287, 62
M5173
M1, M2 or M4145, 136, 190
M1, M2 or M3128, 123, 145
M3, M4 or M586, 65, 47
M1 or M3110, 85
Scheduling Problems | Flexible Job-shop with Batching
20Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
• Ham. A and Cakici. E.,2016, Flexible job shop scheduling problem with parallel batch processing machine: MIP and CP approaches, Computers & IndustrialEngineering, 102,160-165
• Comparison of CP Optimizer vs different MIP formulations on a set of 20 instances from literature
• The batching allows multiple jobs to be simultaneously processed as long as the total size of the batch does not exceed machine capacity
• Commonly used in semiconductor manufacturing
• Minimize makespan
Scheduling Problems | Flexible Job-shop with Batching cont’d
21Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
MIP Model Revised MIP Model (Valid Inequalities) CP
Problem Objective ComputationalTime (sec) Objective Computational
Time (sec) Objective ComputationalTime (sec)
SFJS1 66 0.01 66 0.02 66 0.08
SFJS2 107 0.01 107 0.02 107 0.00
SFJS3 208 0.05 208 0.06 208 0.02
SFJS4 272 0.02 272 0.01 272 0.00
SFJS5 100 0.06 100 0.06 100 0.00
SFJS6 320 0.31 320 0.36 320 0.00
SFJS7 397 0.64 397 0.05 397 0.00
SFJS8 216 0.51 216 0.08 216 0.00
SFJS9 210 0.55 210 0.06 210 0.00
SFJS10 516 5.9 516 3.38 516 0.01
Average 0.8 0.4 0.01
CP is able to find optimal solution in all cases
Small size problems
Scheduling Problems | Flexible Job-shop with Batching cont’d
22Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
MIP Model Revised MIP Model (Valid Inequalities) CP
Problem Objective ComputationalTime (sec) Objective Computational
Time (sec) Objective ComputationalTime (sec)
MFJS1 410 300 410 300 410 0.01
MFJS2 410 300 410 300 410 0.01
MFJS3 420 300 420 300 420 0.01
MFJS4 506 300 506 300 503 0.04
MFJS5 488 300 488 300 488 0.03
MFJS6 631 300 614 82.84 614 0.05
MFJS7 916 300 848 300 789 0.67
MFJS8 896 300 804 300 774 0.88
MFJS9 nf 300 876 300 843 24.27
MFJS10 3418 300 1105 300 985 11.19
Average 300 278.3 3.72
CP is able to find better in 6 cases out of 10 with on average 3.72 seconds computational time
Medium/large size problems.
Scheduling Problems | Parallel batching machines with job release times, weights, and sizes
23Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
• Ham. A, Fowler. J.W., Cakici. E., 2017, Constraint programming approach for scheduling jobs with different job release times and incompatible families on parallel batching machines, IEEE Transactions on Semiconductor Manufacturing
• Comparison of CP Optimizer vs different heuristics & MIP models proposed byCakici et. al.,2013 on a set of 2560 instances
• Minimize Total Weighted Completion Time
[1] Cakici, E., Mason, S.J., Fowler, J.W., Geismar, H.N., 2013, Batch Scheduling on Parallel Machines with Dynamic Job Arrivals and Incompatible Job Families, International Journal of Production Research
Scheduling Problems | Parallel batching machines with job release times, weights, and sizes
24Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
15 jobs MIP Model 1 MIP Model 2 Heuristic 1 Heuristic 2 CP
Avg. TWCT 1347 (640)** 1350 (493) 1395 (95) 1396 (91) 1348 (639)Avg. PR 1.0014 1.0347 1.0355 1.0000Avg. Run-time 22.23 299.02 19.14 19.08 1.79
15-job instances *
25-job instances *25 jobs Heuristic 1 Heuristic 2 CP
Avg. TWCT 3085 (24) 3086 (18) 2925 (633)Avg. PR 1.0545 1.0549 1.0001Avg. Run-time 39.83 40.03 21.00
* 640 different instances for 15-job, 25-job, 50-job, and 100-job instances** ( ) denotes number of best solutions achieved
Scheduling Problems | Parallel batching machines with job release times, weights, and sizes
25Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
50-job instances *
100-job instances *100 jobs Heuristic 1 Heuristic 2 CP
Avg. TWCT 33051 (14) 33044 (12) 30798 (622)Avg. PR 1.0732 1.0729 1.0005Avg. Run-time 218.21 230.73 109.90
50 jobs Heuristic 1 Heuristic 2 CP
Avg. TWCT 9670 (9) 9675 (14) 9051 (626)Avg. PR 1.0697 1.0702 1.0001Avg. Run-time 118.14 117.48 57.44
CP is able to find best solution in 98.5% of all cases whereas heuristics can find the best only in 4% of all cases
26Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Scheduling Problems | Others
• See [2] for some comparisons against more sophisticated MIP models for Job-shop scheduling (with similar conclusions)
• Comparison of CP Optimizer vs a disjunctive MIP formulation on a set of 140 classical Job-shop instances (50-2000 tasks), time-limit: 2mn, 4 threads
• See [3] for recent review of CP Optimizer (modeling concepts, examples, performance,…)
[2] W.Y. Ku and J. C. Beck. Mixed Integer Programming Models for Job Shop Scheduling: A Computational Analysis. Computers & Operations Research, 2016[3] P. Labroire, J. Rogerie, P. Shaw, and P. Vilim, IBM ILOG CP Optimizer for scheduling. Constraints, 2018
27Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
3+ times faster as compared to 5 years ago
Constraint Programming Optimizer | Performance
Agenda
1
Introduction
2
Computational Study
3
Conclusions
28Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Motivation
Overview of CP
How It Works?
Modeling
Scheduling Problems
CP vs MIP & Heuristics
Future Research
29Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Conclusions | CP vs MIP
• IBM algorithms outperforms classical MIP approaches drastically in both solutiontime and quality
• Large and hard problems which cannot be solved by MIPs are now solvable
• Difficult business rules can also be modeled easily• i.e. Certain lots/activities must observe a maximum time between 2 steps (consecutive or not)
• Certain steps of oven-type (diffusion step) and batching with size constraints
30Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Conclusions | CP vs Heuristics
• There is again significant performance difference between IBM CP Optimizer’sunique algorithms and widely used heuristic approaches
• Better solutions can be achieved much more quickly
• Heuristics’ development and maintenance is not easy• Solution quality not guaranteed as data and few parameters change
• Can require a lot of coding
31Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Conclusions | Industrial Scheduling Problems
The classical job-shop scheduling problem
• Resource/machines are over-simplified• In reality: setup-times, production modes, activities incompatibilities, batching, cumulative resources,
inventories (reservoirs), execution conditions (e.g. resource safety levels, auxiliary resources),...• All operations are performed in a unique way
• In reality: resource allocation, optional operations, alternative recipes, hierarchical decomposition• The single objective function is completely unrealistic
• In reality: combination of earliness/tardiness costs, nonexecution cost, resource related costs, constraint
violation, job/customer priorities…• Real problems are often much larger than the size of current benchmarks
adapted from https://www.slideshare.net/PhilippeLaborie/planningscheduling-with-cp-optimizer - Philippe Laborie, Principal Scientist at IBM
32Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Future Research | CP & MIP & Heuristics together
A Hybrid Approach for Machine Scheduling
Job assignment to machines with MIP or Heuristics (Master Problem)
+
Job sequencing & scheduling with CP (Subproblem)
33Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation
Future Research | Application Areas
• Workforce Scheduling
• Vehicle Routing
• Advertisement Scheduling
• Sports League Scheduling
• Bridge Building
• Gate Allocation
• Timetabling
Data Science & AI Elite Team / © 2020 IBM Corporation
Thank you !
© Copyright IBM Corporation 2020. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. Any statement of direction represents IBM’s current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and ibm.com are trademarks of IBM Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks ofIBM or other companies. A current list of IBM trademarks is available at Copyright and trademark information.
To Learn More www.ibm.com/community/datascience/elite/