1

Optimization TechniquesConstrained Optimizationby Linear Programming

updated 10.11.01

NTUSY-521-N

SMUEMIS 5300/7300

Systems Analysis MethodsDr. Jerrell T. Stracener, SAE Fellow

2

Nature of Linear Programming

3

The Product Mix Problem

• Among the most common in linear programming

• The problem is to find out which products to include in the production plan and in what quantitiesthese should be included (product mix) in orderto maximize the profit.

• A solution tells more generally, in effect, how to allocate scarce resources.

4

The Blending Problem

• Involve the determination of the best blend ofavailable ingredients to form a certain quantity of a product under strict specifications required to meeta designated level of output or given specifications

• Especially important in the process industries suchas petroleum, chemicals and food, and in fields wherea certain level of service is desired at minimum cost

• In blending, the cost of the ingredients is to be minimized , while adhering to certain specificationsand using certain ingredients (resources).

5

The Blending Problem

• An attempt is made to use as few resources aspossible to provide a given product or service level

Note: A blending problem is also considered aproblem of allocating resources in the best mannerand in both cases we try to achieve the highest ratio of outputs to inputs; that is, to maximizeproductivity.

6

Formulation of the Linear Programming Model

Management science models, are composed of three components:

• The decision (controllable) variables

• The environment (uncontrollable) variables

• The result (dependent) variables

• The LP model is composed of the same components, but they assume different names

7

Decision Variables

The decision variables in LP depend on the type ofLP problem being considered. They can be thequantities of the resources to be allocated, or thenumber of units to be produced. The decisionmaker is searching for the value of these unknownvariables which will provide an optimal solution to the problem.

The DecisionVariables

(controllable)

The Constraints(uncontrollable variables

and relationships)

ObjectiveFunction

(result variables)

8

The Objective Function

• A linear programming model attempts to optimizea single goal, written as a linear function; for examplemaximize 5x1 + 7x2 + x3. That is, it attempts tofind either the maximum level of a desirable goal,such as total share of the market or total profit, or the minimum level of some undesirable outcome, such as total cost.

• Two Step Approach1. Select a primary goal whose level is to bemaximized or minimized.2. Transform the other goals into constraints,which must only be satisfied.

9

The Constraints

• The decision maker is searching for the values of the values of the decision variables that will maximize (or minimize) the value of the objective function. Such a process is usually subject to several uncontrollable restrictions, requirements, or regulations that are called constraints. These constraints are expressed as linear equations and/orinequalities.

10

Formulation of a Linear Programming Model

Every LP problem is composed of:

• Decision variables - The variables whose values are unknown and are searched for. Usually they are designated by x1, x2, …

• Objective function - This is a mathematicalexpression, given as a linear function, that showsthe relationship between the decision variables anda single goal (or objective) under consideration. Theobjective function is a measure of goal attainment.

11

Formulation of a Linear Programming Model

• Optimization - Linear programming attempts toeither maximize or minimize the values of theobjective function.

• Profit or cost coefficients - The coefficients of thevariables in the objective function are called theprofit (or cost) coefficients. They express the rateat which the value of the objective function increasesby including in the solution one unit of each of thedecision variables.

12

Formulation of a Linear Programming Model

• Constraints - The maximization (or minimization)is performed subject to a set of constraints. Therefore, linear programming can be defined as a constrained optimization problem. These constraints are expressed in the form of linear inequalities (or, sometimes inequalities). They reflect the fact thatresources are limited (e.g., in a product-mix problem) or the specific product requirements (e.g., in a blending problem).

13

Formulation of a Linear Programming Model

• Input-output (technology) coefficients - Thecoefficients of the constraints’ variables are calledthe input-output coefficients. They indicate the rateat which a given resource is depleted or utilized. Theyappear on the left hand side of the constraints.

• Capacities - The capacities (or availability) of thevarious resources, usually expressed as some upperor lower limit, are given on the right hand side of theconstraints. The right-hand side also expressesminimum requirements.

•Non-Negativity - It is required that only non-negative (zero or positive) values of the decision variables be considered.

14

Formulation of a Linear Programming Model

• The general linear programming model can bepresented in the following mathematical terms

aij = input-output coefficientscj = cost (profit) coefficientsbi = capacities (right hand side)xj = decision variables

Find a vector (x1, …, xn) which minimizes (or maximizes) a linear objective function F(x) where:

F(x) = c1x1 + c2x2 + … + cjxj + … + cnxn

15

Formulation of a Linear Programming Model

subject to the linear constraints:

a11x1 + a12x2 + … + a1nxn b1

a21x1 + a22x2 + … + a2nxn b1

… … … … ai1x1 + ai2x2 + … + ainxn bi

… … … … am1x1 + am2x2 + … + amnxn bm

and the non-negativity constraints:

x1 0, x2 0, …

16

Advantages of Linear Programming Model

• Linear programming is a tool that can be used to solve allocation type problems. Such problems are very common and extremely important.

• Solution is difficult due to the fact that an infinitenumber of possible solutions may exist.

• Linear programming not only provides the optimalsolution but does so in a very efficient manner.

• It provides additional information concerning thevalues of the resources that are allocated.

17

Limitations of Linear Programming Due to Assumptions

The applicability of linear programming is limited byseveral assumptions. As in all mathematical models,assumptions are made for reducing the complexreal-world problem into a simplified form. The majorones are:

• Certainty - It is assumed that all data involved in the linear programming problem are known with certainty

• Linear objective function - It is assumed that the objective function is linear. This means that per unit cost, price or profit are assumed to be unaffected by changes in production methods or quantities produced or sold.

18

Limitations of Linear Programming Due to Assumptions

• Linear constraints - The constraints are also assumed to be linear. This means that all theinput-output coefficients are considered to beunaffected by a change of methods, quantities,or utilization level.

• Nonnegativity - Negative activity levels (or negative production) are not permissable. It isrequired, therefore, that all decision variablestake nonnegative values.

19

Limitations of Linear Programming Due to Assumptions

• Additivity - It is assumed that the total utilizationof each resource is determined by adding togetherthat portion of the resource required for theproduction of each of the various products oractivities. The assumption of additivity also meansthat the effectiveness of the joint performance ofactivities, under any circumstances, equals the sum of the effectiveness resulting from the individualperformance of these activities.

20

Limitations of Linear Programming Due to Assumptions

• Divisibility - Variables can, in general, be classifiedas continuous or discrete. Continuous variables aresubject to measurement (e.g., weight), temperature

• Independence - Complete independence ofcoefficients is assumed, both among activitiesand among resources. For example, the price ofone produce has no effect on the price of another.

21

Limitations of Linear Programming Due to Assumptions

• Proportionality - The requirement that the objective function and constraints must be linear is a proportionality requirement. This means that theamount of resources used, and the resulting value of the objective function, will be proportional to thevalue of the decision variables.

22

Solving Linear Programs

• All solutions to a linear programming problemwhich satisfy all the constraints are called feasible

• The collection of feasible solutions is called thefeasible solution space or area

• Any solution which violates one or more of theconstraints is termed infeasible

23

Solving Linear Programs

The major task in applying linear programming isthe formulation of the problem.

• Once a problem has been formulated, one of several available methods of solution can be applied

• The graphical method, whose main purpose is to illustrate the concepts involved in the solution process

• The general, computationally powerful simplexmethod

• Other approaches

24

The Graphical Method of Solution

The graphical method is used mainly to illustratecertain characteristics of LP problems and to helpin explaining the simplex method. The only casewhere it has a practical value is in the solution ofsmall problems with two decision variables andonly a few constraints, or, problems with twoconstraints or only a few decision variables.

25

Example

The two models of color sets produced by the Sekido Corporation, will be designated as A and B. The company is in the market to make money; that is, its objective is profit maximization. The profit realized is $300 from set A and $250 from set B. Obviously, the more A sets produced and sold, the better. The trouble is that there are certain limitations which prevent Sekido Company from producing and selling thousands of sets. These Limitations are:

1. Availability of only 40 hours of labor each day in the production department (a labor constraint).2. A daily availability of only 45 hours of machine time (a machining constraint).3. Inability to sell more than 12 sets of model A each day (a marketing constraint).

(the constraints are on the next slide)

26

Example

First let A = x1 and B = x2.

So we need to maximize

z = 300x1 + 250x2

with these constraints

2x1 + 1x2 40 (labor constraint)

1x1 + 3x2 45 (machining constraint)

1x1 + 0x2 12 (marketing constraint)

x1 0 nonnegativity

x2 0 nonnegativity

27

Example - solution

• Graphing the feasible area - The graphical method starts with the graphing of a feasible are within whicha search for the optimal solution is to be conducted.The feasible area is established through graphing allof the inequalities and equations that describe the constraints.

1st quadrantfeasible

x1 0, x2 0

4th quadrantinfeasible

x1 0, x2 0

2nd quadrantinfeasible

x1 0, x2 0

3rd quadrantinfeasible

x1 0, x2 0

28

Example - solution

• Graphing the first constraint - This is expressed as 2x1 + x2 40. The steps in drawing the constraint are:

Step 1: Since an inequality of the type less than or equal to has in effect two parts, we will first consider only the equality part of the constraint. In our example, it will be 2x1 + x2 = 40. Since anequation can be shown graphically as a straight line, it is sufficient to find the coordinates of two points to graph the entire equation as a line. To do so, x1 is first set to zero. This will yield a point where the equation intersects the x2 axis.

29

05

1015202530354045

0 5 10 15 20 25 30 35 40 45

x1

x2

Example - solution

if we repeat this for the rest of the constraints we will get a graph that looks like this:

The first constraint

0

10

20

30

40

50

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

x1

x2

121 x

402 21 xx

453 21 xx

30

Example - solution

We limit our selves to the largest region that satisfies all our constraints.

This region has 4 vertices, which we will use in our main equation to try and maximize it.

15

120

a b

c d

Solution Total ProfitPoint (corner) coordinates 300x1 + 250 x2 =

a (0,15) 300 (0) + 250 (0) = 0b (12,11) 300 (0) + 250 (15) = $3,750c (0,0) 300 (12) + 250 (0) = $3,600d (12,0) 300 (12) + 250 (11) = $6,350 maximum

31

Example - solution

So our maximum value of $6,350 is when

x1 = 12 and x2 =11.

32

LP SolutionsThe Graphical Method

prepared and presented by Imran Ismailupdated 10.11.01

NTUSY-521-N

SMUEMIS 5300/7300

Systems Analysis MethodsDr. Jerrell T. Stracener, SAE Fellow

33

Consider the following LP

Maximize X + 2Y Subject to:

X + Y <= 4 X - 2Y <= 2 -2X + Y <= 2 X, Y >= 0

This problem is in 2-dimensions and can be solved graphically

34

Finding the Feasible Region

We begin by graphing the constraints on an X-Y coordinate system to determine the set of all points that satisfy all the constraints. This set of points is known as the feasible region for the LP.

Since both variables must be non-negative, we know that the feasible region must be within the first quadrant.

35

The constraint lines can be constructed by changing all inequalities in to equal signs.

i.e. X + Y <= 4 become X + Y = 4, the inequalities play an important role in defining the feasible region and are implemented later on.

36

The set of solutions to the equation X + Y = 4 can be represented by the straight line passing through the points (4,0) and (0,4) in the XY plane

37

Similarly for constraint X-2Y <= 2, we get

38

And for constraint 3, -2X+Y <= 2:

Since X and Y must be nonnegative, the feasible region for the LP is the bounded by the three lines described (and shown) above and the X and Y axes. Any point inside or on the boundary of the region described above is a feasible solution to the LP

39

Finding an Optimal Solution

The objective function is given as an equation whose value has yet to be determined. The information provided in the LP gives us 2 clues:

• The function has to be maximized• The function must satisfy all constraints i.e. it

must lie in the feasible regions

40

In such an instance, we can start by assuming an objective function value of 0 (zero).

This gives us the following equation

X + 2Y = 0

The line above will provide us with a set of points that will have an objective function value of 0.

When drawn on the graph, the segment of this line that intersects the feasible region will represent the set of feasible solutions with an objective function value of zero. In this case, the set is just the single point (0,0).

41

Similarly, the line X + 2Y = 2 describes the set of points that have an objective function value of two. So, the segment of this line that intersects the feasible region represents the set of feasible solutions with an objective function value of two.

We can continue this method till we reach an objective function value such that it no longer intersects the feasible region.

42

By continuing on in this fashion, we can find an optimal solution for the LP by "pushing" the objective function line up until it last touches the feasible region.

43

This occurs when we graph the line

X + 2Y = 22/3 = 7.333

which intersects the feasible region at the point (2/3, 10/3). Since there are no feasible solutions with a greater objective function value than 22/3, we say that X = 2/3, Y = 10/3 is an optimal solution and that 22/3 is the optimal value for the objective function.

44

It is important to investigate how other objective functions might behave given the same feasible region.

45

It is observed that objective functions with different slopes, while traveling through the feasible region might be maximized at different points.

It is also important to note that the corner points of the feasible region are the only part of this region that do not allow the objective function to move further ahead while still remaining feasible!

Therefore, if an LP has an optimal solution, it is guaranteed that this solution will occur at a corner or extreme point of the feasible region

46

The “Brute Force” Method

Using the fact that if an LP has an optimal solution, then it has an extreme point solution, we can use a "brute force" method to find an optimal solution by:

• testing each extreme point to see if it is feasible and then comparing the objective function values.

Recall that an extreme, or corner-point, solution is formed by the intersection of two of the constraints.

47

In our example we can identify the extreme points by labeling A, B, C, D and E, such that:

B

A

CD

E

48

Point A is formed by the intersection of:X + Y = 4 and –2X + Y = 2

and similarly:B: X - 2Y = 2 and X + Y = 4C: X - 2Y = 2 and Y = 0D: X = 0 and Y = 0E: -2X + Y = 2 and X = 0

Solving these equations and plugging them into the objective function, we get:

49

Here, the co-ordinate points of A give us the maximum objective function value

X-co-ord Y co-ord Obj. Val

A .66 3.33 7.33

B 3.33 .66 4.66

C 2 0 2

D 0 0 0

E 0 2 4

50

Point A is also where the objective function line was graphically maximized.

Therefore, we conclude that point A is the optimal extreme point of the feasible region. It:

• maximizes the value of the objective function and

• satisfies all constraints

The (x,y) co-ordinates of point A are called the optimal decision variables or the optimal solution with optimal objective function value ZA

51

Conclusion

The graphical method and the brute force method will always obtain the same result.

In our example, it was found that the optimal solution is such that:

• Xopt=2/3 = 0.66

• Yopt=10/3 = 3.33

• Zopt = 22/3 = 7.333

52

Systems Analysis MethodsDr. Jerrell T. Stracener, SAE Fellow

SMUEMIS 5300/7300

NTUSY-521-N

NTUSY-521-N

SMUEMIS 5300/7300

Statistical Analysis Other Linear Programming Methods

updated 9.19.01

53

Using Excel

54

Maximization Problem - Using Excel

A company produces two models of color TV sets,designated A and B. The company’s objective is profit maximization. The profit realized is $300 fromset A and $250 from set B. The limitations are:

1. Only 40 hours of labor each day are available.Labor constraint

2. Only 45 hours of machine time are available.Machining constraint

3. Only 12 sets of model A may be sold each day.Marketing constraint

Determine how many sets of each model to produceeach day so that total profit is maximized. Use Excel!

55

Formulation

The Decision Variables:

x1 = number of model A producedX2 = number of model B produced

The Objective Function (total profit):

z = 300x1 + 250x2

This is the function to be maximized, as wasdemonstrated using a graphical technique last week in class.

56

Constraints

Labor Constraint:2x1 + x2 40

Machine Constraint:x1 + 3x2 45

Marketing Constraint:x1 12

Non-negativity Constraint (no negative sales):x1 0 x2 0

57

Solver in Excel

First, make the spreadsheet look nice so wecan understand what the heck you are doing.

58

Solver in Excel

From the Toolsmenu in Excelselect the Solver.

It will spawn themenu on the nextslide, into whichyou put your constraints.

59

Solver in Excel

The Target Cell is where we want the answer.

60

Solver in Excel

Once the constraints are input, push Solve and the cell will change from the previousvalue, to the value the Solver solved for.