Crash Course in Mathematics Constrained...
Transcript of Crash Course in Mathematics Constrained...
Crash Course in Mathematics Constrained Optimisation
Crash Course in Mathematics
Constrained Optimisation
Serçin �ahin
Y�ld�z Technical University
9 October 2012
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Formally we write the problem as follows:
maxx1,x2f (x1, x2) s.t. g(x1, x2) = 0
f (x1, x2) is called the objective function or maximand,
The x1, x2 are called choice variables,
The function g(x1, x2) is called the constraint and
The set of all x1, x2 that satisfy the constraint are sometimes
called the constraint set or the feasible set.
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
One way to solve this problem is by substitution.
Suppose that g(x1, x2) = 0 can be written to isolate x2 on one
side as:
x2 = g̃(x1)
We substitute it into the objective function:
maxx1f (x1, g̃(x1))
The usual �rst-order conditions required that we set the total
derivative,∂f
∂xequal to zero and solve for the optimal x∗1 .
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Lagrange's Method
Let us reconsider our original problem:
maxx1,x2f (x1, x2) s.t. g(x1, x2) = 0
Let's form a new function, called Lagrangian, which has three
variables instead of two: namely x1, x2, λ:
L(x1, x2, λ) ≡ f (x1, x2)− λg(x1, x2)First-order conditions give the critical points of L:
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Lagrange's Method
The General Form of the Lagrange's Method:
Suppose we have a function of n variables and we face mconstraints, where m < n. Our problem is:
For x = (x1, ..., xn) and λ = (λ1, ..., λn) , we obtain the
function of n +m variables:
The �rst-order conditions again require that all partial
derivatives of L be equal to zero at the optimum (x∗, λ∗):
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Geometric Interpretation
We can represent the objective function geometrically by its
level sets
L(y0) ≡ {(x1, x2) | f (x1, x2) = y0}At any point along any level set of the function, total
di�erential must be zero:
Then the slope of the level set through (x1, x2) will be
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Geometric Interpretation
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Geometric Interpretation
We can think of the constraint function , as a kind of level set.
It is the set of all (x1, x2) such that
g(x1, x2) = 0
Again, for any (x1, x2) satisfying the constraint, the following
relation must hold:
Then the slope of the constraint at the point (x1, x2) is:
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Geometric Interpretation
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Geometric Interpretation
Simple rearrangement of �rst-order conditions for a critical
point of the Lagrangian function gives
Suppose λ∗ 6= 0.
Dividing the �rst of the equations by the second gives
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Geometric Interpretation
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Second-Order Conditions
The bordered Hessian of the Lagrangian function involves the
second-order partials of L bordered by the �rst-order partials
of the constraint equation and a zero.
Let's call the determinant of the bordered Hessian D.
If (x∗1 , x∗2 , λ
∗) solves the �rst-order conditions, and if D > 0
when evaluated at (x∗1 , x∗2 , λ
∗), then (x∗1 , x∗2 ) is a local
maximum of f (x1, x2) subject to the constraint g(x1, x2) = 0.
If (x∗1 , x∗2 , λ
∗) solves the �rst-order conditions, and if D < 0
when evaluated at (x∗1 , x∗2 , λ
∗), then (x∗1 , x∗2 ) is a local
minimum of f (x1, x2) subject to the constraint g(x1, x2) = 0.
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Second-Order Conditions
In the multivariable, multiconsraint case, the bordered Hessian
is again formed by bordering the matrix of second-order
partials of L by all the �rst-order partials of the constraints
and enough zeros to form a partials of L by all the �rst-order
partials of the constraints and enough zeros to form a
symmetric matrix.
Its principal minors are the determinants of submatrices
obtained by moving down the principal diagonal.
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Second-Order Conditions
The n −m principal minors of interest here are those
beginning with the (2m + 1) st and ending with the
(n +m)-th,i.e., the determinant of H.
Crash Course in Mathematics Constrained Optimisation
Equality Constraints
Second-Order Conditions
Su�cient Conditions for Local Optima with Equality
Constraints
Let the objective function be f (x) and the m < n constraints
be g j(x) = 0, j = 1, ...m.Given the Lagrangian function, let (x∗, λ∗) solve the �rst-orderconditions of the Lagrange's method. Then,
1 x∗ is a local maximum of f (x) subject to the constraints if the
(n −m) principal minors alternate in sign beginning with
positive Dm+1 > 0,Dm+2 < 0, ..., when evaluated at (x∗, λ∗).2 x
∗ is a local minimum of f (x) subject to the constraints if the
(n −m) principal minors are all negative
Dm+1 < 0,Dm+2 < 0, ..., when evaluated at (x∗, λ∗).
Crash Course in Mathematics Constrained Optimisation
Inequality Constraints
Let's begin with the simplest possible problem: maximising a
function of one variable subject to a non-negativity constraint
on the choice variable.
maxx f (x) subject to x ≥ 0.
Crash Course in Mathematics Constrained Optimisation
Inequality Constraints
All together, we have identi�ed three conditions thatcharacterise the solution to the simple maximisation problemwith non-negativity constraints.
Condition 1: f ′(x∗) ≤ 0
Condition 2: x∗ [f (x∗)]Condition 3: x∗ ≥ 0
Crash Course in Mathematics Constrained Optimisation
Inequality Constraints
Crash Course in Mathematics Constrained Optimisation
Inequality Constraints
Kuhn-Tucker Conditions
Let us now consider optimisation subject to general inequality
constraints
Crash Course in Mathematics Constrained Optimisation
Optimality Theorems
Consider the maximisation problem,
where x is a vector of choice variables, and a = (a1, ..., al) is avector of parameters that may enter the objective function, the
constraints, or both.
Suppose that for each a ∈ A there is at least one solution,
x(a) to the maximisation problem stated above.
Then, for that vector of parameters, a, the maximised value of
the objective function is f (x(a), a). This de�nes a new
function, V (a), called the value function.
Crash Course in Mathematics Constrained Optimisation
Optimality Theorems
The Envelope Theorem
Consider the maximisation problem when there is just one
constraint and suppose the objective function, f , and the
constraint function, g are continuously di�erentiable in (x, a).Let L(x, a, λ) be the problem's associated Lagrangian function
and let (x(a), λ(a)) solve the Kuhn-Tucker conditions.
�nally, let V (a) be the problem's associated value function.
Then, the Envelope theorem states that for every a ∈ U,
where the right-hand side denotes the partial derivative of the
Lagrangian function with respect to the parameter ajevaluated at the point (x(a), λ(a)).