MATH 6026 - UPRMacademic.uprm.edu/~pvasquez/mate6026/clases1516II/2.1.pdf · MATH 6026 What is a...

MATH 6026

Dr. Pedro Vásquez

UPRM

P. Vásquez (UPRM) Conferencia 1 / 17

MATH 6026

Fundamentals of unconstrained optimization

The mathematical formulation of an unconstrained optimization problemis:

minxf (x)

where x 2 Rn is a real vector with n ≥ 1 components and f : Rn ! R isa smoothed function.

Usually, there is no enough information related to f . All we know are thevalues of f and maybe some of its derivatives at a set of pointsx0, x1, x2, · · · .


MATH 6026

Example


MATH 6026

What is a solution?

Ideally we would like to obtain a global minimizer of f , a point where thefunction obtains its least value.

Definitiona point x∗ is a global minimizer if f (x∗) ≤ f (x) for all x 2 Rn

or over the domain of interest.

Definitiona point x∗ is a local minimizer if there is a neighborhood @ ofx∗ such that f (x∗) ≤ f (x) for all x 2 @.

Definitiona point x∗ is a strict local minimizer (also called strong localminimizer) if there is a neighborhood @ of x∗ such that f (x∗) < f (x) forall x 2 @ with x 6= x∗.

Definitiona point x∗ is an isolated local minimizer if there is aneighborhood @ of x∗ such that x∗ is the only minimizer in @.


MATH 6026

Example


MATH 6026

Reconizing a local minimumOne way to find out if a point x∗ is a local minimum is to examine all thepoints in its inmediate vicinity, to make sure that none of them has asmaller function value. If the function f is smoothed, in particular if itstwice continuously di§erentiable, we will be able to tell that x∗ is a localmimizer by examining the gradient rf (x∗) and the Hessian r2f (x∗) .

Theorem

(Taylor 0s Theorem) Suppose that f : Rn ! R is continuouslydi§erentiable and that p 2 Rn. Then we have that:f (x + p) = f (x) +rf (x + tp)T pfor some t 2 (0, 1) . Moreover, if f is twice continuously di§erentiable, wehave that:rf (x + p) = rf (x) +

R 10 r

2f (x + tp) pdtand thatf (x + p) = f (x) +rf (x)T p + 1

2pTr2 (x + tp) p

for some t 2 (0, 1) .


MATH 6026

Theorem

(First order necessary conditions)If x∗ is a local minimizer and f is continuously di§erentiable in an openneighborhood of x∗, then rf (x∗) = 0.

Proof.


MATH 6026

Theorem

(Second order necessary conditions)If x∗ is a local minimizer of f and r2f exists and is continuouslydi§erentiable in an open neighborhood of x∗, then rf (x∗) = 0 andr2f (x∗) is positive semidefinite.

Proof.


MATH 6026

Theorem

(Second order su¢cient conditions)Suppose that r2f is continuously di§erentiable in an open neighborhoodof x∗ and that rf (x∗) = 0 and r2f (x∗) is positive definite. Then x∗ isa local minimizer of f .

Proof.


MATH 6026

Theorem

When f is convex, any local minimizer x∗ is a global minimizer of f . If inaddition f is di§erentiable, then any stationary point x∗ is a globalminimizer of f .

Proof.


MATH 6026

Example


MATH 6026


MATH 6026

Example


MATH 6026


MATH 6026 - UPRMacademic.uprm.edu/~pvasquez/mate6026/clases1516II/2.1.pdf · MATH 6026 What is a...

Documents

Transcript of MATH 6026 - UPRMacademic.uprm.edu/~pvasquez/mate6026/clases1516II/2.1.pdf · MATH 6026 What is a...