Introduction to Optimization - Nc State...

Introduction to Optimization

Anjela GovanNorth Carolina State UniversitySAMSI NDHS Undergraduate workshop 2006

What is Optimization?

Optimization is the mathematical discipline which is concerned with finding the maxima and minima of functions, possibly subject to constraints.

Where would we use optimization?

ArchitectureNutritionElectrical circuitsEconomicsTransportationetc.

What do we optimize?

A real function of n variables

with or without constrains

),,,( 21 nxxxf K

Unconstrained optimization

22 2),(min yxyxf +=

Optimization with constraints

22),(min

1,522),(min

02),(min

22

22

22

or

or

=++=

≥<<−+=

>+=

yxyxyxf

yxyxyxf

xyxyxf

Lets Optimize

Suppose we want to find the minimum of the function

Review max-min for ú2

What is special about a local max or a local min of a function f (x)?at local max or local min f’(x)=0 f”(x) > 0 if local minf”(x) < 0 if local max


Second Derivative TestLocal min, local max, saddle pointGradient of f – vector (df/dx, df/dy, df/dz)direction of fastest increase of fGlobal min/max vs. local min/max

Gradient Descent Method Examples

Minimize function

11,11)(5.0),( 22

≤≤−+=

yxyxyxf α

Minimize function

4,4)cos()cos(),(

≤≤−=

yxyxyxf

Gradient Descent Method Examples

Use function gd(alpha,x0)Does gd.m converge to a local min? Is there a difference if α > 0 vs. α < 0?How many iterations does it take to converge to a local min? How do starting points x0 affect number of iterations?

Use function gd2(x0)Does gd2.m converge to a local min?How do starting points x0 affect number of iterations and the location of a local minimum?

How good are the optimization methods?

Starting pointConvergence to global min/max.Classes of nice optimization problems Example: f(x,y) = 0.5(αx2+y2), α > 0Every local min is global min.

Other optimization methods

Non smooth, non differentiable surfaces ⇒can not compute the gradient of f ⇒can not use Gradient MethodNelder-Mead MethodOthers

Convex Hull

A set C is convex if every point on the line segment connecting xand y is in C.

The convex hull for a set of points X is the minimal convex setcontaining X.

Simplex

A simplex or n-simplex is the convex hull of a set of (n +1) . A simplex is an n-dimensional analogue of a triangle. Example:

a 1-simplex is a line segmenta 2-simplex is a trianglea 3-simplex is a tetrahedrona 4-simplex is a pentatope

Nelder-Mead Method

n = number of variables, n+1 pointsform simplex using these points; convex hullmove in direction away from the worst of these points: reflect, expand, contract, shrink

Example: 2 variables ⇒ 3 points ⇒ simplex is triangle3 variables ⇒ 4 points ⇒ simplex is tetrahedron

Nelder-Mead Method – reflect, expand

Nelder-Mead Method-reflect, contract

A tour of Matlab: Snapshots from the minimizationAfter 0 steps

A tour of Matlab: Snapshots from the minimizationAfter 30 steps (converged)

fminsearch function

parameters: q =[C,K]cost function:

Minimize cost function[q,cost]=fninsearch(@cost_beam, q0,[],time,y_tilde)

2N

1ii )y_tilde]),[,(y(t cost iKC −∑=

=

Our optimization problem

In our problemOur function: cost function “lives” in ú3

2 parameters C and K, n=2 Simplex is a triangle

2N

1ii )y_tilde]),[,(y(t cost iKC −∑=

=

Introduction to Optimization - Nc State...

Documents

Transcript of Introduction to Optimization - Nc State...