Nonlinear Programming

A nonlinear program (NLP) is similar to a linear program in that it is composed of an objective function, general constraints, and variable bounds.

A nonlinear program includes at least one nonlinear function, which could be the objective function, or some or all of the constraints. Z = x1

2 + 1/x2

Many real systems are inherently nonlinear Unfortunately, nonlinear models are much more

difficult to optimize

Maximize Z = f(x)

Subject to.

gi(x) ≤ b1

xi ≥ 0

General Form of Nonlinear

No single algorithm will solve every specific problem

Different algorithms are used for different types of problems

Wyndor Glass example with nonlinear constraint

Maximize Z = 3x1 + 5x2

Subject tox1 ≤ 49x1

2 + 5x22 ≤ 216

x1, x2 ≥ 0

The optimal solution is no longer a CPF anymore, but it still lies on the boundary of the feasible region.

We no longer have the tremendous simplification used in LP of limiting the search for an optimal solution to just the CPF solutions.

Wyndor Glass example with nonlinear objective function

Maximize Z = 126x1 – 9x1

2 + 182x2 – 13 x22

Subject to x1 ≤ 4

x2 ≤ 123x1

+ 2x2 ≤ 18x1, x2 ≥ 0

The optimal solution is no longer a CPF anymore, but it still lies on the boundary of the feasible region.

Wyndor Glass example with nonlinear objective function

The optimal solution lies inside the feasible region.

That means we need to look at the entire feasible region, not just the boundaries.

Maximize Z = 54x1 – 9x1

2 + 78x2 – 13 x22

Subject to x1 ≤ 4

x2 ≤ 123x1

+ 2x2 ≤ 18x1, x2 ≥ 0

Unlike linear programming, solution is often not

on the boundary of the feasible solution space.

Cannot simply look at points on the solution space

boundary, but must consider other points on the

surface of the objective function.

This greatly complicates solution approaches.

Solution techniques can be very complex.

Nonlinear programming problems come in many different shapes and forms Unconstrained Optimization Linearly Constrained Optimization Quadratic Programming Convex Programming Separable Programming Nonconvex Programming Geometric Programming Fractional Programming

These problems have no constraints,

so the objective is simply to maximize

the objective function Basic function types

Concave Entire function is concave down

Convex Entire function is concave up

Basic calculus Find the critical points Unfortunately this may be difficult for

many functions Estimate the maximum

Bisection Method Newton’s Method

Gradient Search procedure: Z = 2x1x2 + 2x2 – x12 – 2x2

2

f / x1 = 2x2 – 2x1

f / x2 = 2x1 + 2 – 4x2

Initialization: set x1*, x2* = 0.

1) Set x1 = x1* + t(f / x1) = 0 + t(2×0 – 2×0) = 0.

Set x2 = x2* + t(f / x2) = 0 + t(2×0 + 2 – 4×0) = 2t.

f (x1 , x2) = (2)(0)(2t) +(2)(2t) – (0)(0) – (2)(2t)(2t) = 4t – 8t2.

2) f ’(x1 , x2) = 4 – 16t. Let 4 – 16t = 0 then t* = ¼.

3) ReSet x1* = x1* + t (f / x1) = 0 + ¼(2×0 – 2×0) = 0.

x2* = x2* + t(f / x2) = 0 + ¼(2×0 + 2 – 4×0) = ½.

Stopping rule: f / x1 = 1, f / x2 = 0

Gradient Search procedure: Z = 2x1x2 + 2x2 – x12 – 2x2

2

f / x1 = 2x2 – 2x1

f / x2 = 2x1 + 2 – 4x2

Iteration 2: x1* = 0 x2* = ½ .

1) Set x = (0,1/2) + t(1,0) = (t,1/2).

f (t,1/2) = (2)(t)(1/2) +(2)(1/2) – (t)(t) – (2)(1/2)(1/2) = t – t2 + ½.

2) f ’(t,1/2) = 1 - 2t. Let 1 – 2t = 0 then t* = ½ .

3) ReSet x* = (0,1/2) + ½ (1,0) = (½, ½).

Stopping rule: f / x1 = 0, f / x2 = 1.

Gradient Search procedure: Z = 2x1x2 + 2x2 – x12 – 2x2

2

f / x1 = 2x2 – 2x1

f / x2 = 2x1 + 2 – 4x2

Continue for a few more iterations:

Iteration 1: x* = (0, 0)

Iteration 2: x* = (½, ½)

Iteration 3: x* = (½, ¾)

Iteration 4: x* = (¾, ⅞)

Iteration 5: x* = (⅞, ⅞)

Notice the value is converging toward x* = (1, 1)

This is the optimal solution since the gradient is 0 f (1,1) = (0,0)

Nonlinear objective function with linear constraints

Karush-Kuhn Tucker conditions KKT conditions