NLP-intro
Transcript of NLP-intro
-
8/6/2019 NLP-intro
1/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory1
Introduction to Non-Linear Optimization
-
8/6/2019 NLP-intro
2/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory2
Nonlinear Optimization
Note:Unlike for linear problems, a global optimum for a nonlinearproblem cannot be guaranteed, except for special cases, e.g.,if you know the space is unimodal, or convex, or monotonicityexists.
Two standard heuristics that most people use:
1) find local extrema starting from widely varying starting points of thevariables and then pick the most extreme of these extrema (if theyare not the same)
2) perturb a local extremum by taking a finite amplitude step away
from it, and then see whether your routine returns you to a betterpoint or "always" to the same one.
Question: How would you "automate" a search for a globalextremum?
-
8/6/2019 NLP-intro
3/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory3
Basic Steps in Nonlinear Optimization
In its simplest form, a numerical search procedure consistsof four steps when applied to unconstrained minimizationproblems:
1) Selection of an initial design in the n-dimensional space, where nis the number of design variables
2) A procedure for the evaluation of the function (objective function)at a given point in the design space.
3) Comparison of the current design with all of the precedingdesigns.
4) A rational way to select a new design and repeat the process.
Constrained minimization requires step for evaluation ofconstraints as well. Same applies for evaluating multipleobjective functions.
-
8/6/2019 NLP-intro
4/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory4
Nonlinear Optimization Process
Simplistically, most design tasks seek to find a perturbation to an existing designwhich will lead to an improvement. Thus, we seek a new design which is the old
design plus a change
:
Xnew=Xold+HX
Optimization algorithms use the same formula, but apply a two step process:
Xk=Xk-1+ESk
You (the engineer) have to provide an initial design X0.
The optimization will then determine a search direction Skthat will improve thedesign.
Next question is how far we can move in direction Skbefore we must find a newsearch direction. This is a one-dimensional search since we only have to determine
the value of the scalarE to improve the design as much as possible.
-
8/6/2019 NLP-intro
5/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory5
A Good Algorithm
A good algorithm is (among others):
Robust algorithm must be reliable for general design applicationsand (thus) must theoretically converge to the solution point startingfrom any given starting point.
General Should not impose restrictions on the model's constraintsand objective functions.
Accurate Ability to converge to precise mathematical optimumpoint is important, though it may not be required in practice.
Easy to use by both experienced and inexperienced users. Shouldnot have problem dependent tuning parameters.
Efficient To be efficient, the number of repeated analyses shouldbe kept to a minimum. Hence, an efficient algorithm has 1) a fasterrate of convergence requiring fewer iterations, and 2) least number ofcalculations within one (design) iteration.
Note: Tradeoffs have to be made
-
8/6/2019 NLP-intro
6/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory6
Zero and first order algorithms
You often must choose between algorithms which need onlyevaluations of the objective function or methods that alsorequire the derivatives of that function.
Algorithms using derivatives are generally more powerful, butdo not always compensate for the additional calculations ofderivatives.
Note that you may not be able to compute the derivatives.
-
8/6/2019 NLP-intro
7/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory7
Basic Descent Methods
Basic descent methods are the basic techniques for iterativelysolving unconstrained minimization problems.
Important for practical situations because they offer the simplestand most direct alternatives for obtaining solutions.
Also good as a benchmark.
-
8/6/2019 NLP-intro
8/8
Optimization in Engineering Design
Georgia Institute of Technology
Systems Realization Laboratory8
General Basic Descent Method Algorithm
Basic steps:
1. start at an initial point;
2. determine according to a fixed rule a direction of movement; and
3. move in that direction to a (relative) minimum of the objective functionon that line.
4. At the new point, a new direction is determined and the same processis repeated.
The primary difference between algorithms (steepest descent,Newton's method, etc) is the rule by which successive directions ofmovement are selected.
The process of determining the minimum point on a line is called linesearch.