Kuhn-tucker Theory Slides
-
Upload
imstillarockstar -
Category
Documents
-
view
216 -
download
0
Transcript of Kuhn-tucker Theory Slides
-
8/9/2019 Kuhn-tucker Theory Slides
1/21
3/26/201
INTRODUCTION TO KUHN-TUCKER THEORY
OPTIMIZATION WITH INEQUALITYCONSTRAINTS
Max f(x)
s.t. gk(x) bk, k= 1,2, , m
Remarks:
If the objective is Minf(x), we may changeit to Max f(x).
If the constraint is gk(x) bk, we maychange it to gk(x) bk.
-
8/9/2019 Kuhn-tucker Theory Slides
2/21
3/26/201
Graphical Solution
Max f x x x
s t x x x
x
x
x x
( )
. .
,
x
12
22
2
12
1 2
2
1
1 2
2 1
2 0
1 0
1 0
0
Example
Graphical Solution
f x x x
x x
( )
( )
x
12
22
2
12
22
2 1
1
Rewrite the objective function as
This is the equation of a circle of radius rwith center
at (0,1).
x x r12
22 21 ( )
Let the objective function be equal to r2, i.e.,
-
8/9/2019 Kuhn-tucker Theory Slides
3/21
3/26/201
Level (Contour) Sets of the
Objective Function
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Center:
(0,1) r
Constraint Curve x2= x12+ 2x1
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
-
8/9/2019 Kuhn-tucker Theory Slides
4/21
3/26/201
The Feasible Set
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Circle of radius r= 1
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
r= 1
-
8/9/2019 Kuhn-tucker Theory Slides
5/21
3/26/201
Optimal Solution
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
r= 1
(x1*,x2*)= (1,1)
(x1**,x2**)
= (0,0)
Optimal Solution
There are two optimal solutions:
(x1*,x2*) = (1,1)
VOF(x1*,x2*) =x1*2+x2*
22x2* + 1 = 1
(x1**,x2**) = (0,0)
VOF(x1**,x2**) =x1**2
+x2**2
2x2** + 1 = 1
-
8/9/2019 Kuhn-tucker Theory Slides
6/21
3/26/201
Example 2
Min x y
s t x y
x y
x y
( ) ( )
. .
,
5 6
2 6
2 8
0
2 2
( ) ( )x y r 5 62 2 2Let
Example 2: The Objective Function
( ) ( )x y r 5 6 2 2
This is the equation of a circle centered at (5,6) with radius r.
Thus, we wish to minimize r, i.e., we want to determine the
circle centered at (5,6) with the smallest radius that touchesthe feasible set.
-
8/9/2019 Kuhn-tucker Theory Slides
7/21
3/26/201
Example 2: The Objective Function
0 3 5 8 x
4
6
y
Example 2: The Feasible Set
0 3 5 8 x
4
6
y
2x+y= 6
x+ 2y= 8
-
8/9/2019 Kuhn-tucker Theory Slides
8/21
3/26/201
Example 2: The Optimal Solution
0 3 5 8 x
4
6
y
2x+y= 6
x+ 2y= 8
(x*,y*) (optimal solution)
Example 2. The Optimal Solution
The optimal solution is the intersection of the
two lines given by the equations
2x+y= 6
x+ 2y= 8
The optimal solution isx* = 4/3,y* = 10/3
VOF(x*,y*) = (4/35)2+ (10/36)2= 185/9
-
8/9/2019 Kuhn-tucker Theory Slides
9/21
-
8/9/2019 Kuhn-tucker Theory Slides
10/21
3/26/201
1
Slaters Constraint Qualification:
Example
2 6x y
x y 2 8
g x y x y1 6 2 0( , )
g x y x y2 8 2 0( , )
Consider the constraints:
Rewrite the constraints as:
Slaters Constraint Qualification:Example
Both functions are concave (because they are
linear) on R2+, the nonnegative quadrant.
Choose (x0,y0) = (1,1). Then
g1(x0,y0) = 62(1)1 = 3 > 0
g2(x0,y0) = 812(1) = 5 > 0 Hence, the constraints satisfy Slaters
Constraint Qualification.
-
8/9/2019 Kuhn-tucker Theory Slides
11/21
3/26/201
Kuhn-Tucker Necessary Conditions
for Optimality
Given the problem
Max f(x), x Rn
s.t. gk(x) bk, k= 1,2, . . ., m
wheref,gk(k= 1,2, ..., m) have continuous partial
derivatives on an open convex subset D of Rnand
thegks satisfy Slater's constraint qualification.
If x* is an optimal solution, then there exist scalars
1*, 2*, ..., m* such that
Kuhn-Tucker Necessary Conditionsfor Optimality
( ) ( *) ( *) , , , . . . ,*a f g j nj kk
m
jk
x x
1
0 1 2
( ) ( *) , , , . . . ,*b g b k mk k k x 0 1 2
( ) , , , . . . ,*c k mk 0 1 2
( ) ( *) , , , . . . ,d g b k mk kx 0 1 2
-
8/9/2019 Kuhn-tucker Theory Slides
12/21
3/26/201
1
Example 1
1. Min 4 + 2 4
s.t. 4
2. Max 2 2 + +
s.t. 1
3. Max 4 2 + +
s.t. + 0.25
Steps
1. Write in standard form.
2. Check if constraints are concave. If
yes,
3. Check if Slaters constraint qualification
is met. If yes,4. Findxs that satisfy Kuhn-Tucker
Necessary Conditions
-
8/9/2019 Kuhn-tucker Theory Slides
13/21
-
8/9/2019 Kuhn-tucker Theory Slides
14/21
3/26/201
1
The Expenditure Minimization
Problem
Min E(x, y) =p1x+p2y
s.t. U(x,y) u
x, y 0
The optimal solution is called an expenditure-minimizer.
Transform to a max problem:
Max E(x, y) = p1xp2y
s.t. U(x,y) u 0 x, y 0
Kuhn-Tucker Conditions whenNonnegativity Constraints are Explicit
Given the problem
Max f(x), x Rn
s.t. gk(x) bk, k= 1,2, . . ., m
x 0
wheref,gk
(k= 1,2, ..., m) have continuous partialderivatives on an open convex subset D of Rnand
thegks satisfy Slater's constraint qualification.
If x* is an optimal solution, then there exist scalars
1*, 2*, ..., m* such that
-
8/9/2019 Kuhn-tucker Theory Slides
15/21
3/26/201
1
Kuhn-Tucker Conditions when
Nonnegativity Constraints are Explicit
( ) ( *) ( *) ,*a f gj kk
m
jk
x x
1
0
( ) ( *) ( *)* *b f g xj k jk
k
m
jx x
1
0
( ) ( *)*c g bk k k x 0
( ) *d k 0
( ) ( *)e g bk
kx 0( ) *f x 0
Example
Maxf(x,y) =x+ 2y
s.t. g(x,y) = x2y+ 2 0
x,y 0
It is clear thatfandghave continuous partial
derivatives.
We have already shown thatgis concave
and satisfies Slaters ConstraintQualification.
-
8/9/2019 Kuhn-tucker Theory Slides
16/21
3/26/201
1
Kuhn-Tucker Conditions when
Nonnegativity Constraints are Explicit
(a) 1 2x 0 2 0 (b) (1 2x)x= 0 (2)y= 0 (c) (x2y+ 2) = 0 (d) 0 (e) x2y+ 2 0 (f ) x,y 0
Case 1. = 0 This is not possible from
(a).
Case 2. > 0 x= 0; not possible from
(a); hence,x> 0.
From (c): x2y+ 2 = 0 y= 0 => x2+ 2 = 0 => x= 2 => 1 22 = 0
=> = 1/(22), contrary to 2 from (a); hence,y> 0.
Kuhn-Tucker Conditions whenNonnegativity Constraints are Explicit
(a) 1 2x 0 2 0 (b) (1 2x)x= 0 (2)y= 0 (c) (x2y+ 2) = 0 (d) 0
(e) x2y+ 2 0 (f ) x,y 0
From (b),
(2) = 0 => * = 2 =>12(2)x= 0; => x* = ;
Solving foryin (c):
y* = 31/16
It can be shown that
x* = ,y* = 31/16
is an optimal solution byshowing that the objectivefunction is concave (Kuhn-Tucker SufficiencyCondition).
-
8/9/2019 Kuhn-tucker Theory Slides
17/21
3/26/201
1
Example
1. Min 3 + 4
s.t. + 15
, 0
Kuhn-Tucker Sufficient Condition forOptimality: Example
Min x y( ) ( ) 2 32 2
s t x y. . 3 2 6
x y, 0
Transform the problem to conform to the standard form.
Max f x y x y( , ) ( ) ( ) 2 32 2
s t g x y x y. . ( , ) 6 3 2 0
x y, 0
-
8/9/2019 Kuhn-tucker Theory Slides
18/21
3/26/201
1
Kuhn-Tucker Sufficient Condition for
Optimality: Example
Max f x y x y( , ) ( ) ( ) 2 32 2
s t g x y x y. . ( , ) 6 3 2 0
x y, 0
Note: Constraintgis concave since it is linear.
Choosex0= 0,y0= 0.
g(x0,y0) =g(0,0) = 6 > 0.
Hence, Slaters Constraint Qualification is satisfied.
Kuhn-Tucker Sufficient Condition forOptimality: Example
Kuhn-Tucker conditions
(a) 2(x2)3 0 2(y3)20 (b) [2(x2)3]x= 0 [2(y3)2]y= 0
(c) (63x2y) = 0 (d) 0 (e) 63x2y 0 (f ) x,y 0
Case 1. = 0 From (a):
2(x2) 0 => x 2 2(y3) 0 => y 3 From (b):
2(x2) = 0 => x= 2 2(y3) = 0 => y= 3
From (e): 63x2y= 63(2)2(3)
=6 Hence,x= 2,y= 3; this
violates (e).
Therefore, = 0 is notpossible.
-
8/9/2019 Kuhn-tucker Theory Slides
19/21
3/26/201
1
Kuhn-Tucker Sufficient Condition for
Optimality: Example
Kuhn-Tucker conditions
(a) 2(x2)3 0 2(y3)20 (b) [2(x2)3]x= 0 [2(y3)2]y= 0 (c) (63x2y) = 0 (d) 0 (e) 63x2y 0 (f ) x,y 0
Case 2. > 0 From (c):
63x2y= 0 x= 0 => y= 3
=> = 0 (from (b))contrary to > 0.
Hence,x> 0.
y= 0 => x= 2
=> = 0 (from (b))contrary to > 0.
Hence,y> 0.
From (b) & (c):
2(x2)3= 0 2(y3)2= 0 63x2y= 0
Kuhn-Tucker Sufficient Condition forOptimality: Example
x y* , * , * 8
13
27
13
12
13
H( , )x y
2 0
0 2
The Hessian matrix of the objective functionfis:
His a negative definite matrix; hencefis strictly concave.
By the Kuhn-Tucker sufficiency theorem, (x*,y*) is an
optimal solution.
Solution:
-
8/9/2019 Kuhn-tucker Theory Slides
20/21
3/26/201
2
Example
Min ( 5) + ( 4)
s.t. 2 + 3 10
, 0
Example
Min 42 + 5 3
s.t. + 2 20
, 0
-
8/9/2019 Kuhn-tucker Theory Slides
21/21
3/26/201
Example
Max 52 4 + 10 +
s.t. 3 2 5
, 0
The Envelope Theorem forConstrained Optimization
dzd
f g dx
df g
dy
df gx x y y
**
**
**
dz
df g
**
Use the first-order conditions on (4):
(4)
Note that the right hand side of equation (5) is the derivative
of the Lagrangean with respect to , i.e.,
(5)
dz
dL x y
*( *, *, *, )
Add (3) to the right hand side of (1):