MDP 631Industrial Operations Research

Lecture 6

Integer Programming

Today’s lecture

• Integer programming

• Branch-and-bound algorithm for MIP

• Binary decision variables and mathematical modeling of logical relationships

• Using branch-and-bound algorithm for solving BIP

Integer Programming

• Integer programming (IP) models are mathematical programming models that contain integer variables.

• Integer programming appears in a wide range of applications.

• In some cases the use of integer decision variables is more realistic than using a continuous variable.

• Examples of this include the number of workers, the number of cars to be produced and the number of courses to be taken …etc.

Why Integer Programs?

• Advantages of restricting variables to take on integer values:

– More realistic

– More flexibility (e.g. modeling logical constraints)

• Disadvantages:

– More difficult to model

– Can be much more difficult to solve

Types of IP Models

• An IP in which all variables are required to be integers is called a pure integer programming (PIP) problem.

• An IP in which only some of the variables are required to be integers is called a mixed integer programming (MIP) problem.

• An MIP that is represented by linear objective function and constraints is called mixed integer linear programming (MILP) problem.

• An integer programming problem in which all the variables must be 0 or 1 is called a 0-1 IP or binary IP (BIP).

• The LP obtained by omitting all integer or 0-1 constraints on variables is called LP relaxation of the IP.

How IPs are defined

• For integer programs: just add constraints that say that all variables that are considered integer are integer.

– In writing the model:

xi ≥ 0 and integer for all i S

– In LINGO:@GIN( <variable name>);

• For binary (0-1) decision variables:

– We write

xj {0,1} or 0 xj 1 and xj integer for all i S

– In LINGO@BIN( <variable name>);

Example: All-Integer LP

• Consider the following all-integer linear program:

Max. 3x1 + 2x2

s.t. 3x1 + x2 < 9

x1 + 3x2 < 7

- x1 + x2 < 1

x1, x2 > 0 and integer

Example: All-Integer LP

• LP Relaxation

Solving the problem as a linear program ignoring the integer constraints, the optimal solution to the linear program gives fractional values for both x1 and x2. From the graph on the next slide, we see that the optimal solution to the linear program is:

x1 = 2.5, x2 = 1.5, z = 10.5

Example: All-Integer LP

• LP Relaxation

LP Optimal (2.5, 1.5)

Max 3x1 + 2x2

-x1 + x2 < 1

x2

x1

3x1 + x2 < 9

1

3

2

5

4

1 2 3 4 5 6

x1 + 3x2 < 7

Example: All-Integer LP

ILP Optimal (3, 0)

Max 3x1 + 2x2

-x1 + x2 < 1

x2

x1

3x1 + x2 < 9

1

3

2

5

4

x1 + 3x2 < 7

1 2 3 4 5 6

Example: All-Integer LP

• Complete Enumeration of Feasible ILP Solutions

There are eight feasible integer solutions to this problem:

x1 x2 z

1. 0 0 0

2. 1 0 3

3. 2 0 6

4. 3 0 9 optimal solution

5. 0 1 2

6. 1 1 5

7. 2 1 8

8. 1 2 7

Solution of IPs

• Because we can’t just round off the solution to the LP, solving an IP is not just a matter of solving the LP relaxation.

• In fact, no algorithm exists for solving IPs that is as simple and robust as is the simplex algorithm for solving LPs.

• Branch-And-Bound (BAB or B&B) is commonly used for solving IPs and MIPs.

The Branch-And-Bound Algorithm

Basic Concepts of branch-and-boundThe basic concept underlying the branch-and-bound technique

is to divide and conquer.

Since the original “large” problem is hard to solve directly,

it is divided into smaller and smaller subproblems

until these subproblems can be conquered.

The dividing (branching) is done by partitioning

the entire set of feasible solutions into smaller and smaller subsets.

The conquering (fathoming) is done partially by

(i) giving a bound for the best solution in the subset;

(ii) discarding the subset if the bound indicates that

it can’t contain an optimal solution.

These three basic steps – branching, bounding, and fathoming –

are illustrated on the following example.

Example of Branch-and-Bound

Max Z = 5x1 + 8x2

s.t. x1 + x2 6

5x1 + 9x2 45

x1 , x2 ≥ 0 integer

x1 + x2 = 6

5x1 + 9x2 = 45

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8

(2.25, 3.75)

Z=41.25

Z=20

Utilizing the information about the optimal solution of the LP-relaxation

➢ Fact: If LP-relaxation has integral optimal

solution x*, then x* is optimal for IP too.

▪ In our case, (x1, x2) = (2.25, 3.75) is the optimal

solution of the LP-relaxation. But, unfortunately,

it is not integral.

• The optimal value is 41.25 .

➢ Fact: OPT(LP-relaxation) ≥ OPT(IP)

(for maximization problems)

That is, the optimal value of the LP-

relaxation is an upper bound

for the optimal value of the integer program.

• In our case, 41.25 is an upper bound for

OPT(IP).

Max Z = 5x1 + 8x2

s.t. x1 + x2 6

5x1 + 9x2 45

x1 , x2 ≥ 0 integer

x1 + x2 = 6

5x1 + 9x2 = 45

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8

(2.25, 3.75)

Z=41.25

Z=20

Branching step• In an attempt to find out more

about the location of the IP’s optimal solution,

partition the feasible region of the LP-relaxation.

• Choose a variable that is fractional in the optimal solution to the LP-relaxation – say, x2 . Observe that every feasible IP point must have either x2 3 or x2 ≥ 4 .

Max Z = 5x1 + 8x2

s.t. x1 + x2 6

5x1 + 9x2 45

x1 , x2 ≥ 0 integer

x1 + x2 = 6

5x1 + 9x2 = 45

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8

(2.25, 3.75)

Z=41.25

Z=20

Branching step

• With this in mind, branch on the variable x2 to create the following two subproblems:Subproblem 1 Subproblem 2Max Z = 5x1 + 8x2 Max Z = 5x1 + 8x2

s.t. x1 + x2 6 s.t. x1 + x2 65x1 + 9x2 45 5x1 + 9x2 45x2 3 x2 ≥ 4 x1 , x2 ≥ 0 x1 , x2 ≥ 0

• Solve both subproblems(note that the original optimal solution (2.25, 3.75) can’t recur)

Max Z = 5x1 + 8x2

s.t. x1 + x2 6

5x1 + 9x2 45

x1 , x2 ≥ 0 integer

x1 + x2 = 6

5x1 + 9x2 = 45

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8

(2.25, 3.75)

Z=41.25

Z=20

Branching step (graphically)

Subproblem 1: Opt. solution (3,3) with value 39

Subproblem 2: Opt. solution (1.8,4) with value 41

0

1

2

3

4

5

1 2 3 4 5 6 7 8

Z=20

Subproblem 1

Subproblem 2

(1.8, 4)

(3, 3)

Z=39

Z=41

Solution tree

For each subproblem, we record

• the restriction that creates the subproblem

• the optimal LP solution

• the LP optimum value

The optimal solution for Subproblem 1 is integral: (3, 3).

➢ If further branching on a subproblem will yield no useful information, then we can fathom (dismiss) the subproblem.

In our case, we can fathom Subproblem 1 because its solution is integral.

➢ The best integer solution found so far is stored as incumbent. The value of the incumbent is denoted by Z*.

In our case, the first incumbent is (3, 3), and Z*=39.

➢ Z* is a lower bound for OPT(IP): OPT(IP) ≥ Z* .

In our case, OPT(IP) ≥ 39. The upper bound is 41: OPT(IP) 41.

All

(2.25, 3.75)

Z=41.25

S1: x2 3

(3, 3)

Z=39

S2: x2 ≥ 4

(1.8, 4)

Z=41

int.

Next branching step (graphically)

- Fathom Subproblem 1.

- Branch Subproblem 2 on x1 :

Subproblem 3: New restriction is x1 1.

Opt. solution (1, 4.44) with value 40.55

Subproblem 4: New restriction is x1 ≥ 2.

The subproblem is infeasible

0

1

2

3

4

5

1 2 3 4 5 6 7 8

Z=20

Subpr. 3Subpr. 4

Z=40.55

(1, 4.44)

Solution tree (cont.)

➢ If a subproblem is infeasible, then it is fathomed.

In our case, Subproblem 4 is infeasible; fathom it.

➢ The upper bound for OPT(IP) is updated: 39 OPT(IP) 40.55 .

➢ Next branch Subproblem 3 on x2 .

(Note that the branching variable might recur).

All

(2.25, 3.75)

Z=41.25

S1: x2 3

(3, 3)

Z=39

S2: x2 ≥ 4

(1.8, 4)

Z=41

int.

S3: x1 1

(1, 4.44)

Z=40.55

S4: x1 ≥ 2

infeasible

Next branching step (graphically)

Branch Subproblem 3 on x2 :

Subproblem 5: New restriction is x2 4.

Feasible region:

the segment joining (0,4) and (1,4)

Opt. solution (1, 4) with value 37

Subproblem 6: New restriction is x2 ≥ 5.

Feasible region is just one point: (0, 5)

Opt. solution (0, 5) with value 40

0

1

2

3

4

5

1 2 3 4 5 6 7 8

Z=20

(1, 4)

(0, 5)

Solution tree (final)

➢ If the optimal value of a subproblem is Z*, then it is fathomed.• In our case, Subproblem 5 is fathomed because 37 39 = Z*.

➢ If a subproblem has integral optimal solution x*,

and its value > Z*, then x* replaces the current incumbent.• In our case, Subproblem 5 has integral optimal solution, and its value

40>39=Z*. Thus, (0,5) is the new incumbent, and new Z*=40.

➢ If there are no unfathomed subproblems left, then the current incumbent is an optimal solution for (IP). • In our case, (0, 5) is an optimal solution with optimal value 40.

All

(2.25, 3.75)

Z=41.25

S1: x2 3

(3, 3)

Z=39

S2: x2 ≥ 4

(1.8, 4)

Z=41

int.

S3: x1 1

(1, 4.44)

Z=40.55

S4: x1 ≥ 2

infeasible

S5: x2 4

(1, 4)

Z=37

S6: x2 ≥ 5

(0, 5)

Z=40

int.

int.

Binary Decision Variables and Mathematical Modeling of Logical

Relationships

Using binary decision variables

• Binary (or 0-1) decision variables are quite helpful in building mathematical models that contain logical constraints and relationships.

• This type of IP models appears in many applications of what is called combinatorial optimization problems.

Indicator variables

• Indicator variables are binary variables used to control the value of a continuous variable or the status of a constraint containing continuous variables in an LP model.

Controlling variables

• in this case we have a continuous decision variable x 0. Another binary variable y is added such that the following conditions hold:

y = 1 → x > 0 (1)

y = 0 → x = 0 (2)

• Since y is a binary variable that takes only two values 0 or 1, then we can say that the first condition is equivalent to:

x > 0 → y = 1 (1’)

similarly condition (2) is equivalent to:

x = 0 → y = 0 (2’)

Controlling variables

• Conditions (1) and (2) can be considered by adding the following two constraints to the original LP model:

x M y (3)

x m y (4)

where M and m are upper and lower bounds on the continuous variable x respectively.

• Note: constraint (4) can be removed from the LP model if the objective function takes the form of minimizing the following function: cx x + cy y.

Controlling constraints

• in this case we have a constraint taking the form and we want to represent the situation in

which this inequality holds or does not hold. To do that we introduce a binary variable y where the following relationships or conditions exist:

y = 0 → (5)

y = 1 → (6)

Qxqi

ii

Qxqi

ii

i i

i

q x Q

Controlling constraints

• Similar to the case of controlling a single continuous variable, the following conditions are direct results of conditions (5) and (6) respectively.

→ y = 0 (5’)

→ y = 1 (6’)

• To represent conditions (5) and (6) in the original LP model, we introduce the following constraints:

(7)

(8)

where m and M are lower and upper bounds on the difference , and is a very small number ( < 1)

Qxqi

ii

Qxqi

ii

QMMyxqi

ii ++

+−− Qymxqi

ii )(

Qxqi

ii −

Logical conditions and zero-one variables

• Logical constraints exist in many applications. They are composed of logical terms such as if A happens then B should happen, or if A or not B happen then C should happen and so on.

• Simple logical statements are constructed of constants or sentences such as A, B, and C, and connectives such as and, or and so on.

Logical relationships

Symbol Meaning

And

Or

Not

→ Implies

If and only if

Truth table

The following truth table represents the meaning for each operator:

A B A A B A B A → B

= A B

A B

F F T F F T T

F T T F T T F

T F F F T F F

T T F T T T T

Using binary variables

• Binary variables can be easily used to represent any logical statement.

• Since logical statements can be interpreted using truth tables into either true or false, binary variables having two values 0 or 1 can be used in the same manner as 0 can be used to represent the false state and 1 to represent the true state.

Using binary variables

• If the binary variables xA and xB are used to represent the logical sentences A and B, the following list shows the logical statement and its equivalent representation using binary variables.

Logical statement Equivalent binary formula

A B xA + xB

A B xA xB

A 1 - xA

A→ B xA - xB 0

A B xA - xB = 0

Example of logical constraints

• In a manufacturing problem the following logical constraint exists. If either of products A or B or both are manufactured, then at least one of the products C, D, or E must also be manufactured.

• In logical terms this constraint is represented as follows:

(A B) → (C D E)

Example of logical constraints

• The equivalent mathematical model for this constraint can be obtained by first assigning binary decision variable for each product: xA, xB, xC, xD, xE.

• These binary variables can be incorporated into the mathematical model the same way the indicator variables are used.

Example of logical constraints

• Then to use a mathematical modeling equivalent of the logical statement (A B) → (C D E), we need to break it step by step.

• We have a single implication statement which is the main part of the logical statement, then we need to introduce two binary variables for each part of this statement.

• The left part will have the binary variable xL and the right part will have the binary variable xR. Then the implication statement is represented mathematically as follows:

xL - xR 0

Example of logical constraints

• Now xL should be linked with the binary variables that represent the left hand side of the implication statement. To do that we use the following constraints:

xL xA + xB 2 xL

Similarly

xR xC + xD + xE 3 xR

Special ordered sets of variables

• The concept of special order sets is specially developed to represent common situations in linear programming. They are two types:– Special ordered sets of type 1 (SOS1): is an ordered set of variables

(continuous or integer) within which exactly one variable must be non-zero.

– Special ordered sets of type 2 (SOS2): is an ordered set of variables within which at most two consecutive variables are non-zero.

• The conditions arising in the SOSs can be easily represented mathematically by using binary variables. The following examples illustrate how this can be done

Special ordered sets of variables (Example 1)

• A depot can be sited in any location A, B, C, D or E. only one depot can be built.

• Here, we use binary variables xA, xB, xC, xD and xE to represent the condition that the depot is built in any one of them.

• Then the SOS in this case is the set of binary variables.

• The following constraint needs to be added to represent the condition that only one location will be assigned to build the depot:

xA + xB + xC + xD + xE = 1

Special ordered sets of variables (Example 2)

• If we have an SOS2 of the variables xA, xB, xC, xD and xE the SOS2 conditions can be represented by using additional decision variables zi where i = 1 to 4 (the number of variables-1). The following constraints will be added:

xA z1

xB z1 + z2

xC z2 + z3

xD z3 + z4

xE z4

And z1 + z2 + z3 + z4 = 1

Illustrative examples on using logical constraints

Example 1: Capital Budgeting IP

• Stockco is considering four investments.

• Each investment:

– Yields a determined NPV.

– Requires a certain cash flow at the present time.

• Currently Stockco has $14,000 available for investment.

• Formulate an IP whose solution will tell Stockco how to maximize the NPV obtained from the four investments.

Example 1: Solution

• Begin by defining a variable for each decision that Stockcomust make.

• The NPV obtained by Stockco is:Total NPV obtained by Stockco = 16x1 + 22x2 + 12x3 + 8x4

• Stockco’s objective function is:max z = 16x1 + 22x2 + 12x3 +8x4

• Stockco faces the constraint that at most $14,000 can be invested.

• Stockco’s 0-1 IP is:

max z = 16x1 + 22x2 + 12x3 +8x4

s.t. 5x1 + 7x2 + 4x3 +3x4 ≤ 14xj = 0 or 1 (j = 1,2,3,4)

How to Model “logical” Constraints

• At most 2 stocks are selected.

• Exactly 3 stocks are selected.

• Either of stock 2 or stock 3 can be selected but not both.

• If stock 1 is selected, then so is stock 2.

• If stock 3 is selected, then stock 4 is not selected.

Some Simple Constraints

• If we have a bound on the number of stocks selected we just insert a constraint like:

x1 + x2 + x3 + x4 2

“and” or “or” Constraints

• Exactly 3 stocks are selected can be modeled using:x1 + x2 + x3 + x4 = 3

• Either of stock 2 or stock 3 can be selected but not both, can be modeled using:x2 + x3 ≤ 1

Basic if-then Constraints

• If stock 1 is selected then stock 2 is also selected can be modelled using:

x1 x2

• If stock 3 is selected then stock 4 is not allowed to be selected can be modelled using:

x3 1-x4

Either-or constraints

• We know how to represent either-or forms in terms of the decision variables.

• Now we want to know how to deal with either-or situations when it is a pair of constraints that must be decided between.

• If we start with two constraints f(x) 0 and g(x) 0 and exactly one of them must be true we add a binary variable ywhich forces one of them to be true.

• We then add the two constraints to the formulation using:f(x) My

g(x) M(1-y)

Either-or constraints

• If the binary variable y=1 then the two constraints are equivalent to:f(x) Mg(x) 0

• And if y=0 then:f(x) 0g(x) M

• Thus we ensure that exactly one of the two original constraints is true by forcing both of the new constraints to be true.

Example 2: Dorian Autos

• In this example the only real concern we have is that Dorian Autos must produce a minimum number of 1000 cars if it produces any at all.

• If we define xi to be the number of cars produced of type i, then we know that either xi=0 or xi≥1000.

• Putting these into the form required by either-or constraints would give:xi 0 and 1000-xi 0

Ex 6 Dorian Autos cont’d

• The two constraints:xi Miyi

1000 – xi Mi(1-yi)

• The value of Mi needs to be selected and may be different for each type of car.

Solving BIP models using Branch and Bound Algorithm

Example of Binary IP

All the decision variables have the binary form

Because the last two decisions represent mutually exclusive alternatives (the

company wants at most one new warehouse), we need the constraint

x1: building a factory in Los Angeles? Yes (x1=1) or No (x1=0)

x2: building a factory in San Francisco? Yes (x2=1) or No (x2=0)

x3: building a warehouse in Los Angeles? Yes (x3=1) or No (x3=0)

x4: building a warehouse in San Francisco? Yes (x4=1) or No (x4=0)

Furthermore, decisions 3 and 4 are contingent decisions, because they are

contingent on decisions 1 and 2, respectively (the company would consider

building a warehouse in a city only if a new factory also were going there). Thus,

in the case of decision 3, we require that x3 = 0 if x1 = 0. This restriction on x3

(when x1 = 0) imposed by adding the constraint

Similarly, the requirement that x4 = 0 if x2 = 0 is imposed by adding the constraint

The complete BIP model for this problem is

California Manufacturing Co. Example

Branching

Original problem

Solution Tree

The variable used to do this branching

at any iteration by assigning values to

the variable is called the branching

variable

Bounding

LP relaxation of the whole problem

LP relaxation of subproblem 1 LP relaxation of subproblem 2

Fathoming

Three cases where a subproblem is conquered (fathomed).

(1) A subproblem is conquered if its LP relaxation has an integer optimal solution

(2) A subproblem is conquered if it is inferior to the current incumbent.

Since Z*=9, there is no reason to consider further any subproblem whose

bound ≤ 9, since such a subproblem cannot have a feasible solution better than

the incumbent. Stated more generally, a subproblem is fathomed whenever its

(3) If the simplex method finds that a subproblem’s LP relaxation has no

feasible solutions, then the subproblem itself must have no feasible solutions, so

it can be dismissed (fathomed).

Using the BIP Branch-and Bround Algorithm to

Solve the California Manufacturing Co. Example

(1) Initiliazation

Set Z*= − ∞. Solve the relaxation of the whole problem by the

simplex method. The optimal solution of the relaxation is

1)The bound of the whole problem is less than Z*.

2)The relaxation of the whole problem has feasible solution.

3)The optimal solution includes a noninteger value of x1.

So the whole problem can not be fathomed and should be divided

(branched) into subproblems.

(2) Iteration 1Subproblem 1 with x1=0. The optimal solution of its relaxation is (0,1,0,1) with Z=9.

The optimal solution is integer, which is the best feasible solution found so far. So this

integer solution with Z*=9 is stored as the first incumbent. Since the optimal solution of

its relaxation is integer, subproblem 1 is fathomed

Subproblem 2 with x1=1. Because subproblem 2 is not fathomed, it should be divided

into subproblems.

Subproblem 1

Subproblem 2

(3) Iteration 2The only remaining subproblem corresponds to the x1=1 node, so we shall branch from

this node to create the two new subproblems.

Subproblem 4 with x1=1, x2=1.

Subproblem 3 with x1=1, x2=0. LP relaxation of subproblem 3

LP relaxation of subproblem 4

Bound for subproblem 3 :

Bound for subproblem 4 :

Subproblem 1

Subproblem 2

Subproblem 3

Subproblem 4

(4) Iteration 3So far, the algorithm has created 4 subproblems. Subproblem 1 has been fathomed, and

subproblem 2 has been replaced by subproblems 3 and 4, but these last two remain under

consideration. Because they were created simultaneously, but subproblem 4 has the larger

bound, the next branching is done from subproblem 4, which creates the following new

subproblems

Subproblem 5 with x1=1, x2=1, x3=0.

Subproblem 6 with x1=1, x2=1, x3=1.

LP relaxation of subproblem 5 :

LP relaxation of subproblem 6 :

No feasible solutions

Bound for subproblem 5 : Z≤ 16

Subproblem 1

Subproblem 2

Subproblem 3

Subproblem 4

Subproblem 5

Subproblem 6

(5) Iteration 4The subproblems 3 and 5 corresponding to nodes (1,0) and (1,1,0) remain under

consideration. Since subproblem 5 was created most recently, so it is selected for

branching.

Since x4 is the last variable, fixing its value at either 0 or 1 actually creates a single

solution rather than subproblems. These single solutions are

(1,1,0,0) with Z=14 is better than

the incumbent with Z*=9, so

(1,1,0,0) with Z*=14 becomes the

new incumbent.

Because a new incumbent has been

found, we reapply fathoming test 1

with the new incumbent to the only

remaining subproblem 3.

Subproblem3:

There are no remaining (unfathomed) subproblems. Therefore, the optimality test

indicates that the current incumbent is optimal.

Subproblem 1

Subproblem 2

Subproblem 3

Subproblem 4

Subproblem 5

Subproblem 6

Last Comments on IP models

• There are often multiple ways of modeling the same combinatorial optimization problem.

• Solvers for integer programs are extremelysensitive to the formulation. (not true for LPs)