DECISION MODELING WITH DECISION MODELING WITH MICROSOFT EXCELMICROSOFT EXCEL

Copyright 2001Prentice Hall Publishers and

Ardith E. Baker

NonlinearNonlinear

Chapter 7Chapter 7

OptimizationOptimizationPart 1Part 1

Introduction to Nonlinear Introduction to Nonlinear Optimization ModelsOptimization Models

Not all ___________relationships in business and economics problems are _________.

In general, some of the prominent reasons for ________________are:

1. ________________Relationships

2. Nonadditive Relationships

3. Efficiencies or Inefficiencies of__________Although common, nonlinear models are

more difficult to ____________than linear models.

Unconstrained Optimization in Unconstrained Optimization in Two Two

or More Decision Variablesor More Decision VariablesConsider the case of two ___________variables, x1 and x2 and a given function f(x1,x2).For the case of 2 decision variables, partial ____________from calculus are used to describe local or global ____________of f.

Let fxi denote the first partial derivative, fxi xi

denote the second partial derivative, etc.

Any point at which all first partial derivatives vanish is called a_____________________.

We have the following ___________________for optimality:

At a local max or min both partial At a local max or min both partial derivatives must equal ______(i.e., derivatives must equal ______(i.e., f f xx11= = f f

xx22 = 0). = 0). That is, a local maximizer or a local That is, a local maximizer or a local

minimizer is always a _____________point.minimizer is always a _____________point.However, not all stationary points provide maxima and___________.

In this case, we can employ the second order ___________condition for optimality.

First and second-order tests (called first-first-order ________conditionsorder ________conditions and ___________optimality conditionsoptimality conditions, respectively) can be applied to locate unconstrained local optima for functions of more than one variable.NOTES:

The first order conditions are____________; the second-order conditions are sufficient.The second-order conditions __________the first order ones (i.e., the second order conditions assume that x1

*,x2* is a ________ point).

The first-derivative test (the necessary condition) says that the _____optima are contained among the stationary points of the__________.

The second-derivative test (the _________ condition) allows us to distinguish between local ___________and minimizers, and points that are neither.

For a ____________function of n variables, each local optimizer is a stationary point.

In order to guarantee that a stationary point is, for example, a _______maximizer, second-order sufficiency conditions must be___________.

These two types of optimality conditions have limited practical ______________because:

1. Setting the first partial derivatives equal to zero gives a system of n equations in n ________. Unless the system is_____, it is not easy to find solutions.

2. The second-order sufficiency conditions are complicated and require the evaluation of _______________of certain entries in the matrix of second partial derivatives.

First-order necessary conditions in nonlinear optimizers serve as a _________criterioncriterion for the hill-climbing optimization methods that search for local__________.

When maximizing a ___________functionfunction, any stationary point is a _________maximizer (for a convex function, any stationary point is a global minimizer).

In the general case, an optimized solution could be a local maximizer or minimizer or neither, in the _________case, we are guaranteed that any solution is a global_____________.

Nonlinear Optimization: Nonlinear Optimization: A Descriptive Geometric A Descriptive Geometric

IntroductionIntroductionSoftware packages (such as Solver) are based on ________hill-climbing (or hill-descent) behavior. For a maximization problem:

An _________point is chosen (i.e., a set of numerical values for the decision variables).An uphill direction is determined by approximating the _________of change in all directions (the first partial derivative) of the objective function at that initial point.

The method __________when the approximated rates of further OV change in all directions (the first ________derivatives) are close to zero (the first order conditions are__________).

For _____________optimization, the method moves from the initial point, along a line in an uphill (____increasing) direction, to the highest point that can be attained on that line.

Then, a new uphill direction is defined, and the ____________is continued.

Although the focus has been on unconstrained ____________unconstrained ____________, we are interested in optimizing an objective function subject to______________. Just as in the case of LP modeling, constraints take the form of __________and/or inequalities. However, ____________of the constraints is not assumed in this case.

Thus, the general NLP ___________model can be written as follows (f and gi’s are just symbols for complicated nonlinear functions of the decision variables, x, to compute the OV and each constraint’s LHS).

Max f(x1, x2, …, xn) (objective)

s.t. g1(x1, …, xn) = b1

g2(x1, …, xn) = b2

gm(x1, …, xn) = bm

m equalityconstraints

k inequalityconstraints

h1(x1, …, xn) < r1

h2(x1, …, xn) < r2

hk(x1, …, xn) < rk

Nonlinear Optimization: Nonlinear Optimization: A Descriptive Geometric A Descriptive Geometric

IntroductionIntroduction

Consider the following symbolic model:Graphical Analysis:Graphical Analysis:

Max x1 - x2

s.t. -x12 + x2 > 1

x1 + x2 < 3

-x1 + x2 < 2

x1, x2 > 0

If even one _____________constraint, objective function, or both exists, then it is a nonlinear model. This is called a ________________(NLP).

To use the _______approach, first we graph each constraint to find the _____________(constraint setconstraint set) which is the set of points that simultaneously satisfy _____the constraints.This set represents the ___________decisions.

We want to find the allowable decision that ___________the objective function. To do this, find the “most uphill” _________of the objective function that still touches the constraint set.

The point at which it touches will be an ________ solution to the model.

x2

x1

2

2There is only one __________constraint and

the solution is not at a corner intersection.

Noncorner Optima:Noncorner Optima:This graph shows

a ________________ nonlinear inequality

constrained ___________model in

which all constraints are linear and the

constraint set is a standard

LP_____________.

x2

x1

The objective function is __________.

Nonlinear Optimization: Nonlinear Optimization: A Descriptive Geometric A Descriptive Geometric

IntroductionIntroduction

The following statements hold in either LP or NLP models:

Comparisons between LP and NLP:Comparisons between LP and NLP:

1. Increasing (decreasing) the ____of a < (>) constraint loosens the constraint. This cannot contract and may expand the _____________set.2. Increasing (decreasing) the RHS of a < (>) constraint __________the constraint. This cannot expand and may ____________the constraint set.

3. ________a constraint cannot hurt and may help the optimal ____________value.4. ___________a constraint cannot help and may hurt the optimal objective value.

In LP, the ____________on a specified constraint was defined as the rate of the rate of change in OV as the RHS of that change in OV as the RHS of that constraint_________constraint_________, with all other data unchanged.

Lagrange Multiplier:Lagrange Multiplier:

In the NLP context, this rate of change is called the__________________.

In an LP, the shadow price is ___________for a range of values for the RHS parameter of interest. In the NLP context, this property does not generally hold true. Consider the following simple NLP:Max x2

s.t. x < b

x > 0

In order to ___________x2, make x as large as possible. Thus, the optimal solution is x* = b, and the optimal value of the _______function (OV) is (x*)2 = b2. Thus, the OV is a function of b;OV(b) = b2 .The rate of change of this OV function as b increases is the _________of OV(b), namely 2b. The Lagrange multiplier is not ____________for a range of values of the RHS, b. It varies continuously with b, as might be expected.

In an LP, it is always true that there cannot be a ______solution that is not also________.

Local versus Global Solutions:Local versus Global Solutions:

This is not usually true for ________NLP models. Such models may have local as well as global solutions. Consider the ___________Max model:x2

x1

In the previous graph, the point called “Local Max” is termed a local constrained local constrained ___________ ___________ because the value of the objective function at this point is no smaller than at its ____________ feasible points.

The point called “__________” is termed a global constrained maximizerglobal constrained maximizer because the value of the ________function at this point is no smaller than at all other feasible points.

In general for NLPs, additional conditions must be imposed upon the model, called ________and concavity conditions. These conditions must be satisfied to guarantee that a local constrained _____________is also global.

Nonlinear Optimization: Nonlinear Optimization: A Descriptive Geometric A Descriptive Geometric

IntroductionIntroduction

Many non-linear problems in business and economics are of the following form:

Equality-Constrained NLPs:Equality-Constrained NLPs:

The goal is to maximize or minimize an objective function in n ___________subject to a set of m equality constraints.

Maximize or Minimize f(x1, …, xn)

s.t. g(x1, …, xn) = bi

i = 1, …, m (m<n)

A manufacturer can make a product on either of two machines.

Example 1.Example 1.

Let x1 denote the _______made on machine 1, x2, the quantity made on machine 2. letax1 + bx1

2 = cost of producing on machine 1

ax2 + bx22 = cost of producing on machine 2

Find the values of x1 and x2 that __________total cost subject to the requirement that total production quantity is to be some given number, say R. The _______________model is:Min ax1 + bx1

2 + ax2 + bx22

s.t. x1 + x2 = R

The classic _________________model. Let Example 2.Example 2.

Determine the ____________mix that maximizes that person’s utility subject to his/her budget constraint.

p1, p2, and p3 denote given ________prices of three goods

s1, s2, and s3 be given person-specific ____________

B, a specified constant, denotes a person’s available spending____________

x1s1 + x2

s2 + x3s3 denote the person’s

“____” derived from consuming x1 units of good 1, x2 units of good 2 and x3 units of good 3.

The symbolic model is:

Max x1s1 + x2

s2 + x3s3

s.t. p1x1 + p2 x2 + p3 x3 = B

Consider the modelExample 3.Example 3.

Max x1 - x2

s.t. -x12 + x2 = 1

x2

x1

1.5

1.0

0.5

0.5 1.0

Solution

EqualityConstraint

Optimal objective contour and constraint

line are tangent at optimal solution.

Here is the Solved spreadsheet model and formulas:

Using Solver for NLP ModelsUsing Solver for NLP Models

Solver can be used to enter and ________a model that could contain a nonlinear objective or nonlinear constraint functions or_______.For LP optimization, Solver uses the _________ method to move from corner to corner in the __________region.

For NLP optimization, Solver uses a hill-climbing technique based on a “_______________” procedure, called GRG.

The following steps describe the GRG procedure:Using the _________values of the decision

variables (specified in Solver’s Changing Changing CellsCells field), the procedure finds a feasible solution. From that initial starting point, a ________is computed that most rapidly improves the OV. ___________ (i.e., changes in values of the decision variables), is made in that direction until a constraint boundary is encountered or until the ______no longer improves.A new direction is __________from that new point, and the process is repeated and continues until no further improvement in any ____________occurs.

A restaurant’s average daily expense for advertising is $100, all of which is to be allocated to newspaper ads and radio commercials. Let

Example Nonlinear ModelsExample Nonlinear Models

Marketing Expenditures:Marketing Expenditures:

x1 = avg. no. $/day spent on newspaper adsx2 = avg. no. $/day spent on radio ads

Total annual cost of running the advertising dept:Cost = C(x1, x2) = 20,000 – 440x1 – 300x2 +

20x1 + 12x2 + x1x2

The goal is to find the restaurant’s ________ allocation that will minimize the total annual cost while adhering to the desired ad ___________of $100 per day. The symbolic model is as follows:Min 20,000 – 440x1 – 300x2 + 20x1 + 12x2 +

x1x2s.t. x1 + x2 = 100

x1, x2 > 0

Here is the Solved Excel spreadsheet:

Here are the Solver parameters and the Sensitivity Report: This value

indicates that the initial ___of increase in the annual cost of the adv. dept.

would be about $1195

for each additional

budget dollar spent daily on

____________.

Lagrange MultipliersLagrange Multipliers in NLP have almost the same interpretation as the ____________in LP.

Example Nonlinear ModelsExample Nonlinear Models

Economic Interpretation of Lagrange Economic Interpretation of Lagrange Multipliers and Reduced Gradients:Multipliers and Reduced Gradients:

At_______, the value of the Lagrange multiplier is the ___________rate of change in the optimal value of the objective function as the ith RHS, bi, is increased, with all other data______________.In economic terminology, the ith Lagrange multiplier reflects the _______________of the ith resource.

The units of measure are:units of the objective function

units of RHS of constraint i

A __________Lagrange multiplier would indicate that increasing the RHS would initially increase the_____.

A __________Lagrange multiplier would indicate that decreasing the RHS would initially ________ the OV.

The Lagrange multiplier can be used to ________ what will happen to the _________value if the RHS is changed.

Similar to Reduced Cost, the _______________of a variable relates to the upper or lower bound constraints on __________variables.

A ____________Reduced Gradient indicates that increasing the variable will initially decrease the OV.

A ____________Reduced Gradient indicates that increasing the variable will initially increase the OV.

If a decision variable is at its _______bound, the Reduced Gradient should be ___________for the solution to be optimal in a Max model.Otherwise, decreasing the variable would improve the __________function value.

If a decision variable is at its lower bound, the Reduced gradient should be __________in a Max model.

Otherwise, _____________the variable would improve the objective function value.

If a decision variable is between its upper and lower bounds, the Reduced gradient should be ______for the solution to be_________.

Consider the following NLP model of a previous LP model. Let

Astro and Cosmo Revisited:Astro and Cosmo Revisited:

Profit = (PA – 210)A + (PC – 230)C

A = daily production of Astro model TV setsPA = selling price of Astros = 314 – 1.9A +.01A2

C = daily production of Cosmo model TV setsPC = selling price of Cosmos = 243 - .14C

If the unit variable cost of an Astro is $210 and the unit variable cost of a Cosmo is $230, then the total profit is

Max (PA – 210)A + (PC – 230)C

Here is the symbolic model:

PA = .01A2 – 1.9A + 314 (selling price of Astros)

s.t.

PC = -.14C + 243 (selling price of Cosmos)A < 70 (capacity of Astro line)C < 50 (capacity of Cosmo line)A + 2C < 120 (department A labor hours)A + C < 90 (department B labor hours)A, PA, C, PC > 0

Nonlinearconstraints

Here is the Excel spreadsheet model:

Here are the Solver parameters :

Here is the Sensitivity Report:

This constraint is_______. The Lagrange multiplier indicates that _________the OV increases at the rate of about $0.86 per unit of additional labor hours in Dept. A.

This graph shows that for the Astro and Cosmo NLP model, the optimal solution does not occur at a _______of the feasible region, though it is on the boundary.

0

20

50

90

C

A70 90 120

Solver may not provide the optimal solution with NLP models. Here are some examples:

Example Nonlinear ModelsExample Nonlinear Models

Optimality in NLPs:Optimality in NLPs:

Gulf Coast Oil blends gasoline from three components:

Gulf Coast Oil Model:Gulf Coast Oil Model:

Domestic blend

Foreign blend (a blending of two sources)Octane Additive

(used only in premium gasoline)

The Foreign Blend is transported monthly to Gulf Coast Oil in the single 8,000,000 gallon storage compartment of a large tanker.

Component Octane Cost Availability

No. per Gallon (000s Gal/Mth)

Domestic Blend 85 $0.6510,000

Foreign Blend Source 1 93 $0.80

*

Source 2 97 $0.90 *

Premium Additive 900 $30 50

*Because of the way Gulf Coast Oil obtains Source 1 and Source 2, no more than 8,000,000 gallons of Source 1 plus Source 2 may be obtained per month.

Because the oil purchased from the two sources loses its separate identities when “_______” into the storage compartment of the tanker, the model is called a_______________.The goal is to determine how many gallons of Regular, Midgrade, and Premium gasoline to ________each month, given that it must honor minimum supply contracts of 100 thousand gallons of each type of gasoline.

In addition, each gasoline is subject to a ________octane requirement. The octane number is a weighted average of the octane numbers of its components where the weights are the ________of each component in the blend.

Minimum Octane No Wholesale Price Per Gallon

Regular 87 $1.18

Midgrade 89 $1.25

Premium 94 $1.40

Let

R = thousand gal. of regularregular gasoline producedM = thousand gal. of midgrademidgrade gas producedP = thousand gallons of premiumpremium gas producedD = thousand gallons of domesticdomestic blend producedA = thousand gallons of premium additiveadditive produced

RD = thousand gallons of domestic blend in regular gasolineRF = thousand gallons of foreign blend in regular gasoline

MD = thousand gallons of domestic blend in midgrade gasoline

PD = thousand gallons of domestic blend in premium gasolinePF = thousand gallons of foreign blend in premium gasoline

MF = thousand gallons of foreign blend in midgrade gasoline

S1= thous. gal. purchased, foreign source 1S2= thous. gal. purchased, foreign source 2

OCT = octane number of pooled foreign blend= 93S1 + 97S2

S1 + S2

OCT(S1 + S2) = 93S1 + 97S2In addition to the nonnegativity constraints, the symbolic model is:Max 1.18R + 1.25M + 1.40P - .65D - .8S1 - .9S2

– 30As.t. R = RD + RF (composition of reg. gas)M = MD + MF (composition of

midgrade gas)P = PD + PF + A (composition of premium gas)D = RD + MD + PD (total domestic blend used)S1 + S2 < 8,000

(tanker capacity for foreign sources)

85RD + OCT*RF > 87R (min octane number for regular

gasoline)

The next 4 constraints are nonlinear:

85MD + OCT*MF > 89M (min octane number for midgrade

gasoline)85PD + OCT*PF + 900A > 94P

(min octane number for premium gasoline)OCT(S1 + S2) = 93S1 + 97S2

(pooling constraint for foreign sources)

D < 10,000 (supply limit for domestic blend)A < 50 (supply limit for premium blend)R, M, and P each > 100 (contract delivery min.)

Here is an Excel model with example decision values:

Solver will use these values to find an initial starting point.

Here are the Solver parameters:

This is the first solution found by Solver:

The previous Solver ResultsSolver Results message means that Solver has __________its search because the rate of change in the OV was below the Solver _________Solver _________value for 5 iterations (i.e., the rate of improvement in the OV was too low to continue the optimization method).

For NLPs, Solver always starts from a given ______point. Now, run Solver again to force it to begin optimization again to see if it will _______ upon its solution.

This is the second solution found by Solver:

Because of the form of the nonlinear constraints, this particular Gulf Coast Oil model is called a ___________modelmodel.

The starting point for the NLP method is very important for a nonconcave model. You may need to try several different _______points. The best starting points are those near the ______ optimal solution to the model.For nonconcave models, there is no ___________ that the Solver solution is the global optimal one.In the Gulf Coast Oil example, after two attempts to _______the model, Solver has converged to a locallocal optimum and not the globalglobal optimum.

The global optimum for this model is ________ to be the following:

This solution was found by re-optimizing the model dozens of times, each time using a different starting point set of _______cell values.

Here is the Sensitivity Report for the optimal solution:

For __________or highly nonlinear models, some of the Solver OptionsSolver Options can be used to try and improve Solver’s GRG’s ____________tactics.

Solver Options SettingsSolver Options Settings

This value is used to ________ Solver’s search when the OV is improving very slowly. If the improvement is < the default value of .00001 for

5________, Solver stops. Setting this value smaller

forces Solver to continue the optimization method even if the rate of change in ____is

small.

Setting the ________option to QuadraticQuadratic forces Solver to approximate its estimates of the variable equations in its one-dimensional searches by a __________function instead of a linear (tangent) one.

Selecting _______forces Solver to produce a more accurate

approximation by estimating each directional

___________using two adjacent points to each iterative

solution point instead of just one.

These SearchSearch options determine how

Solver chooses the search ________along

which an improvement in the OV will be sought.

This setting determines how closely the _____

calculations must match the RHS in order for a given constraint to be

____________.If a constraint’s LHS differs from its RHS by an amount less than this setting, then the two are considered equal,

and thus, the constraint is_______.

The settings shown below are suggested for highly __________or nonconcave NLPs.

If the NLP model involves some integer decision variables, then setting the ____________to 0% will force Solver to continue its search.

End of Part 1Please continue to Part 2