MATLABOptimization

Greg Reese, Ph.D

Research Computing Support Group

Miami University

MATLAB Optimization

© 2010-2013 Greg Reese. All rights reserved 2

3

Function SummaryFunction Parameter Linearity Constraints Location

fgoalattain

fminbnd

fmincon

fminimax

fminsearch

fminunc

fzero

lsqlin

lsqnonlin

lsqnonneg

4

Optimization

Optimization - finding value of a parameter that maximizes or minimizes a function with that parameter

– Talking about mathematical optimization, not optimization of computer code!– "function" is mathematical function, not MATLAB language function

5

Optimization

Optimization - finding value of a parameter that maximizes or minimizes a function with that parameter

– Can have multiple parameters– Can have multiple functions– Parameters can appear linearly or nonlinearly

6

Linear programming

Linear programming– "programming" means determining feasible programs (plans, schedules, allocations) that are optimal with respect to a certain criterion– Most often used kind of optimization–Tremendous number of practical

applications\

Won't discuss further

7

fminbnd

The basic optimizing 1D routine is fminbnd, which attempts to find a minimum of a function of one variable within a fixed interval.• function must be continuous• only real values (not complex)• only scalar values (not vector)• might only find local minimum

8

fminbnd x = fminbnd( @fun, x1, x2 )x1 < x2 is the minimization interval

x is the x-value in [x1,x2] where the minimum occurs

fun is the function whose minimum we’re looking for. fun must accept exactly one scalar argument and must return exactly one scalar value

Because there are no restrictions on the value of fun, finding the minimum is called unconstrained optimization.

9

fminbnd

x = fminbnd( @fun, x1, x2 )@fun is called a function handle. You must include the “@” when you call fminbnd.

10

fminbnd

Try ItFind the minimum of f(x) = ( x – 3 )2 – 1

on the interval [0,5]

Step 1 – make an m-file with the function

11

fminbnd

Try ItStep 2 – call fminbnd with your function

>> x = fminbnd( @parabola, 0, 5 )x = 3

If you want to get the value of the function at its minimum, call the function with the minimum x value returned by fminbnd>> parabola(x)ans = -1

12

fminbndTry ItStep 3 – verify result is a global minimum by plotting function over a range, say [-10 10]

>> ezplot( @parabola, [-10 10] )

13

fminunc

The multidimensional equivalent of fminbnd is fminunc. It attempts to find a local minimum of a function of one or more variables.• function must be continuous• only real values (not complex)• only scalar values (not vector)• might only find local minimum

14

fminuncx = fminunc( @fun, x0 )x0 is your guess of where the minimum is

fun is the function whose minimum we’re looking for. fun must accept exactly one scalar or vector argument and must return exactly one scalar value

15

fminuncx = fminunc( @fun, x0 )fun only has one argument. If the argument is a vector, each element represents the corresponding independent variable in the function

Examplef(x,y,z) = x2 + y2 + z2

function w = f( x ) w = x(1)^2 + x(2)^2 + x(3)^2

16

fminuncOptimizing routines such as fminunc are often sensitive to the initial value. For bad initial values they may converge to the wrong answer, or not converge at all.

Tip – if fun is a function of one or two variables, plot it first to get an idea of where the minimum is

17

fminunc

Try ItFind the minimum of f(x) = ( x – π )2 – 1

Step 1 – make an m-file with the function

18

fminunc

Try ItStep 2 – plot the function to get an estimate of the location of its minimum

Looks like min isat about 3

19

fminunc

Try ItStep 3 – call fminunc with your function and an initial guess of 3>> xmin = fminunc( @pi_parabola, 3 )Warning: Gradient must be provided for trust-region method; using line-search method instead.> In fminunc at 247Optimization terminated: relative infinity-norm of gradient less than options.TolFun.xmin = 3.1416

Ignore this stuff

20

fminunc

Try ItFind the minimum of

f(x,y) = ( x–22.22 )2 + ( y-44.44 )2 - 17

Step 1 – make an m-file with the function

21

fminuncTry ItStep 2 – plot the function to get an estimate of the location of its minimum

ezsurf expects a function that takes two arguments. Since parabola only takes one, we need to make a different version that takes two. Let's call it paraboloid_xy

22

fminunc

Try ItStep 2 – plot the function to get an estimate of the location of its minimum>> ezsurf( @parabola_xy, [ 0 50 0 50] )

After zooming andpanning some, itlooks like the minis at about x=20and y=50

23

fminunc

Try ItStep 3 – call fminunc with your function and an initial guess of [20 50]

>> x = fminunc( @paraboloid, [20 50] )x = 22.2200 44.4400>> paraboloid( x )ans = -17.0000

24

fminunc

Try ItFind the minimum of a 5D hypersphere of centered at [ 1 2 3 4 5 ]

f(x,y,z,a,b) = (x–1)2 + (y–2)2 + (z–3)2 +

(a–4)2 + (b–5)2

Step 1 – make an m-file with the functionUse 3 dots to continue a line in a file.

25

fminuncTry ItStep 2 – can't plot 5D, so guess [ 0 0 0 0 0]

Step 3 – call fminunc with your function and an initial guess of [0 0 0 0 0]

>> xmin = fminunc( @hypersphere, [0 0 0 0 0] )xmin = 1.0000 2.0000 3.0000 4.0000 5.0000>> hypersphere( xmin )ans = 3.9790e-012

≈ 0

26

MATLAB Optimization

Questions?

27

The End