Integer Linear Programming - Adam Furmanek...Declarative programming at a glance. Some theory. Real...
Transcript of Integer Linear Programming - Adam Furmanek...Declarative programming at a glance. Some theory. Real...
About me
.NET developer for 5 years.
Scala Developer at Amazon.
Blogger, public speaker.
Author of .NET Internals Cookbook.
http://blog.adamfurmanek.pl
furmanekadam
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 2
AgendaDeclarative programming at a glance.
Some theory.
Real life example.
Implementation consideration.
Summary.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 3
Declarative programming
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 4
Declarative programmingParadigm that expresses the logic of a computation without describing its control flow.
Declarative programming often considers programs as theories of a formal logic, and computations as deductions in that logic space.
A high-level program that describes what a computation should perform.
SELECT * FROM Orders
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 5
Declarative programming β why?Decouples problem formulation from solving process.
Solving part can be replaced without modifying the formulation.
Easier to optimise algorithms for because βthe goalβ is understandable for the machine.
Can be used with little to no programming skills.
Sometimes we donβt know how to solve it.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 6
Example β RTS gameWe are playing RTS game.
We are allowed to hire footmen and archers.
Every footman costs 10 gold and 30 food. Every archer costs 20 gold, 25 food, and 10 wood.
Footmanβs attack is equal to 5, archerβs attack is equal to 7.
Our population limit is set to 200 units. Every footman βcostsβ 1 unit, every archer βcostsβ 2 units.
We have 1000 gold, 1000 food, and 200 wood. We want to get strongest possible army.
07.02.2020 09:11 ADAM FURMANEK 7
UnitsGold Food Wood
Footman 10 30 0
Archer 20 25 10
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 8
Attack Unit count
Footman 5 1
Archer 7 2
Available gold Available food Available wood Max units count
Constraint 1000 1000 200 200
VariablesVariables:
π β ππππ‘ππππ β πππβπππ
Constraints:
π + 2 β π β€ 200 // population10 β π + 20 β π β€ 1000 // gold30 β π + 25 β π β€ 1000 // food
10 β π β€ 200 // wood
Target value:5 β π + 7 β π
07.02.2020 09:11 ADAM FURMANEK 9
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 10
Example
07.02.2020 09:11 ADAM FURMANEK 11
Example
07.02.2020 09:11 ADAM FURMANEK 12
ExampleVariables:
π, πΈ, π, π·,π, π, π , π β 0,β¦ , 9 and all different
Constraints:1000π + 100πΈ + 10π + π· +1000π + 100π + 10π + πΈ =10000π + 1000π + 100π + 10πΈ + π
07.02.2020 09:11 ADAM FURMANEK 13
Example
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 14
Real life examplesRTS game = factory production allocation:
β¦ We have available resources
β¦ We have facitilies
β¦ We want to plan work to maximize the production
Send more money = pattern recognition:β¦ We know structure of a problem but doesnβt understand the latent factors
Floor tiling = microchip transistor layout:β¦ We have transistors and gates of given size
β¦ We want to minimize heat emission and the board size
Others: BTS placement, power plant rooms layout, scheduling systems, items personalization and many more.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 15
Some theoryMIXED INTEGER LINEAR PROGRAMMING
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 16
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 17
Mixed Integer Linear ProgrammingProgramming in mathematics means finding a solution to optimization problem.
Linear Programming is a class of a problems with only linear constraints and with linear cost function.
Mixed Integer Linear Programming is a class of a problems with integer constraint for some of variables.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 18
ConstraintsWe can add two variables:
π + π
We can multiply variable by constant:
5 β π
We can constrain variable with lower/upper bound:
π β€ 7
We cannot multiply variables:π β π
We cannot use strict constraings (greater than, less than, not equal)
π < 10
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 19
Cost functionNotice that not hiring anyone was also a solution!
Problem may have zero, one, multiple or infinitely many solutions. We need to compare them.
We use cost function to specify which one of them is the best.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 20
Cutting plane method
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 21
AlgorithmsBasic algorithms:β¦ Cutting plane methods β we solve the problem without integer constraints
(continuous version of the problem), and next we cut the plane to get smaller problem
β¦ Branch and bound β we solve the problem without integer constraints, and next we bound some variables and perform next iteration
MILP is NP-complete and we sometimes use heuristics:β¦ Tabu search
β¦ Simulated annealing
β¦ Ant colony optimization
07.02.2020 09:11 ADAM FURMANEK 22
What can we do?WE WILL IMPLEMENT
Booleβs logic.
Multiplication.
Comparison operators.
WE WILL NOT IMPLEMENT
Arithmetic: division, remainder, exponentation, roots.
Comparisons: min, max, absolute value.
Number theory: factorial, GCD.
Algorithms: if condition, sorting, loops, lexicographical comparisons, Grayβs code, linear regression.
Set operators: SOS type 1 and 2, approximation.
Graph operators: MST, vertex/edge cover, max flow, connectivity, shortest path, TSP.
Turing machine.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 23
Conjunction β AND operator (&&)
We have two variables: π, π.
We want to create variable π₯ which is π₯ = π && π.
Formula:
0 β€ π + π β 2π₯ β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 24
Conjunction β AND operator (&&)
If both π and π are 1 then π₯ must be 1.
If π₯ was 0:
0 β€ 1 + 1 β 2 β 0 β€ 1
0 β€ 2 β 0 β€ 1
0 β€ 2 β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 25
0 β€ π + π β 2π₯ β€ 1
Conjunction β AND operator (&&)
If both π and π are 1 then π₯ must be 1.
If π₯ is 1:
0 β€ 1 + 1 β 2 β 1 β€ 1
0 β€ 2 β 2 β€ 1
0 β€ 0 β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 26
0 β€ π + π β 2π₯ β€ 1
Conjunction β AND operator (&&)
If either π or π is 1 then π₯ must be 0.
If π₯ was 1:
0 β€ 1 + 0 β 2 β 1 β€ 1
0 β€ 1 β 2 β€ 1
0 β€ β1 β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 27
0 β€ π + π β 2π₯ β€ 1
Conjunction β AND operator (&&)
If either π or π is 1 then π₯ must be 0.
If π₯ is 0:
0 β€ 1 + 0 β 2 β 0 β€ 1
0 β€ 1 β 0 β€ 1
0 β€ 1 β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 28
0 β€ π + π β 2π₯ β€ 1
Conjunction β AND operator (&&)
If both π or π are 0 then π₯ must be 0.
If π₯ was 1:
0 β€ 0 + 0 β 2 β 1 β€ 1
0 β€ 0 β 2 β€ 1
0 β€ β2 β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 29
0 β€ π + π β 2π₯ β€ 1
Conjunction β AND operator (&&)
If both π or π are 0 then π₯ must be 0.
If π₯ is 0:
0 β€ 0 + 0 β 2 β 0 β€ 1
0 β€ 0 β 0 β€ 1
0 β€ 0 β€ 1
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 30
0 β€ π + π β 2π₯ β€ 1
Conjunction β AND operator (&&)
If both π and π are 1 then π₯ must be 1.
If either π or π is 1 then π₯ must be 0.
If both π or π are 0 then π₯ must be 0.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 31
0 β€ π + π β 2π₯ β€ 1
Conjunction & DisjunctionTwo variables conjunction:
0 β€ π + π β 2π₯ β€ 1
π variables conjunction:0 β€ π1 + π2 + π3 + β¦+ ππ β ππ₯ β€ π β 1
Two variables disjunction:β1 β€ π + π β 2π₯ β€ 0
π variables disjunction goes the same way
07.02.2020 09:11 ADAM FURMANEK 32
Negation, exclusive or, implication
Negation:π₯ = 1 β π
Implication:π β π β‘ ~π β¨ π
Exclusive or:~π β§ π β¨ (π β§ ~π)
07.02.2020 09:11 ADAM FURMANEK 33
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 34
Multiplication of two binary variables is their conjunction.
π β π β‘ π β§ π
Multiplication of integer variablesMultiplication is possible when we know the maximum possible value of a variable.
We set the upper bound and perform the long multiplication.
It is rather slow approach, it requires π(π2) temporary variables.
07.02.2020 09:11 ADAM FURMANEK 35
Value decompositionWe decompose the variable to extract digits.
We assume the maximum possible value, because we need to know the number of digits.
Decomposition is straghtforward:
π = π0 + 2 β π1 + 4 β π2 + 8 β π3 + β¦+ 2πβ1 β ππβ1
07.02.2020 09:11 ADAM FURMANEK 36
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 37
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 38
Multiplicationπ β π =π0 β§ π0 + 2 β π0 β§ b1 + 4 β a0 β§ π2 + β¦+ 2πβ1 β π0 β§ ππβ1
+2 β π1 β§ π0 + 2 β π1 β§ π1 + 4 β π1 β§ π2 +β―2πβ1 β (π1 β§ ππβ1)+4 β π2 β§ π0 + 2 β π2 β§ π1 + 4 β π2 β§ π2 + β¦+ 2πβ1 β π2 β§ bnβ1+β―+ 2πβ1( ππβ1 β§ π0 + 2 β ππβ1 β§ π1 + 4 β ππβ1 β§ π2 + β¦+ 2πβ1 β ππβ1 β§ ππβ1 )
It can be represented as:
π β π =
π=0
πβ1
2π
π=0
πβ1
2πππ β§ ππ
07.02.2020 09:11 ADAM FURMANEK 39
Comparison operators
To calculate whether π > π we can use:
0 β€ π β π + 2πβ1π₯ β€ 2πβ1 β 1
To check whether numbers differ:π₯ = π > π β¨ (π > π)
07.02.2020 09:11 ADAM FURMANEK 40
There are libraries doing that!
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 41
https://github.com/afish/MilpManager
Scheduling SystemREAL LIFE EXAMPLE
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 42
IdeaAt the beginning of every term students must be assigned to classes.
It is hard to make them happy because some of them work, some of them prefer to sleep long, some of them prefer to have lots of days off.
We can try to represent the requirements as a MILP program and find the optimal solution.
07.02.2020 09:11 ADAM FURMANEK 43
UsageWe ask students to assign preferences points to every possible class.
The more points assigned the more student wants to be assigned to that class.
We need to take care of rooms occupancy limits, collisions etc.
07.02.2020 09:11 ADAM FURMANEK 44
ScheduleWe are given a schedule of classes in courses
Every class has associated day (Monday β Friday), hour (e.g., 9:30 AM) and duration (e.g., 1:30 hrs).
Every class has associated room with occupancy limit.
07.02.2020 09:11 ADAM FURMANEK 45
ConstraintsVariables.
Exactly one class in one course.
Collisions.
Rooms occupancy limits.
Cost function.
07.02.2020 09:11 ADAM FURMANEK 46
VariablesFor every person, every course, every class we declare binary variable.
Value 1 means that the student is assigned to this class.
We define:π₯πππ’ππ π,ππππ π ,π π‘π’ππππ‘
07.02.2020 09:11 ADAM FURMANEK 47
Exactly one class for a studentEvery student must attend exactly one class in course.
For every course we need to assign student to exactly one class.
α₯
πππ’ππ π,π π‘π’ππππ‘
ππππ π
π₯πππ’ππ π,ππππ π ,π π‘π’ππππ‘ = 1
07.02.2020 09:11 ADAM FURMANEK 48
CollisionsStudent cannot be in two places at the same time.
α₯ πππ’ππ π1, ππππ π 1 , πππ’ππ π2, ππππ π 2 πππππππππ
α₯
π π‘π’ππππ‘
π₯πππ’ππ π1,ππππ π 1,π π‘π’ππππ‘ + π₯πππ’ππ π2,ππππ π 2,π π‘π’ππππ‘ β€ 1
07.02.2020 09:11 ADAM FURMANEK 49
Room occupancy limitEvery room has an occupancy limit.
α₯
πππ’ππ π,ππππ π
π π‘π’ππππ‘
π₯πππ’ππ π,ππππ π ,π π‘π’ππππ‘ β€ π πππ’ππ π,ππππ π
where π πππ’ππ π,ππππ π means the size of the room
07.02.2020 09:11 ADAM FURMANEK 50
Preferences cost functionEvery student assigned points to classes. We want to maximize the sum of those points.
πππ’ππ π,ππππ π ,π π‘π’ππππ‘
ππππ’ππ π,ππππ π ,π π‘π’ππππ‘ β π₯πππ’ππ π,ππππ π ,π π‘π’ππππ‘
where ππππ’ππ π,ππππ π ,π π‘π’ππππ‘ means number of points
07.02.2020 09:11 ADAM FURMANEK 51
What now?Students: 150
Courses: 15
Classes: 9 per course
Problem size: ~40 000 variables.
Solving time: 2 seconds.
This solution is optimal, we wonβt get any better than that!
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 52
Implementation consideration
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 53
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 54
Licenses
Various approaches:β¦License per workstationβ¦License per personβ¦Licensing server with tickets
Licenses for universities and research work.
Licenses for students.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 55
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 56
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 57
https://neos-server.org/neos/solvers/index.html
Beyond MILP
SAT/SMT.
Quadratic programming.
Prolog.
Specialized models for various problems.
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 58
SummaryDeclarative programming allows you to focus on a problem, not on an algorithm!
If something is slow β just replace the solver.
There are many models, choose as powerful as you can (to make modelling easy) and as primitive as possible (to make solving fast).
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 59
Q&A
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 60
ReferencesAlexander Schrijver β βTheory of Linear and Integer Programmingβ
Dennis Yurichev β βSAT/SMT by exampleβ
Adam Furmanek β β.NET Internals Cookbookβ
http://blog.adamfurmanek.pl/2015/08/22/ilp-part-1/ β a lot about ILP
https://github.com/afish/MilpManager β library for modelling
https://neos-server.org/neos/solvers/index.html β NEOS cloud for solving problems
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 61
HT TP://BLOG.ADAMFURMANEK.PL
@FURMANEKADAM
07.02.2020 INTEGER LINEAR PROGRAMMING, ADAM FURMANEK 62