gSLM Service Delivery & Service Level Management in Grid Infrastructures
1 OR II GSLM 52800. 2 Outline classical optimization – unconstrained optimization dimensions of...
-
Upload
edith-ryan -
Category
Documents
-
view
230 -
download
0
Transcript of 1 OR II GSLM 52800. 2 Outline classical optimization – unconstrained optimization dimensions of...
2
OutlineOutline
classical optimization – unconstrained optimization dimensions of optimization
feasible direction
3
Classical Optimization ResultsClassical Optimization Results Unconstrained Optimization Unconstrained Optimization
different dimensions of optimization conditions nature of conditions
necessary conditions (必要條件 ): satisfied by any minimum (and possibly by some non-minimum points)
sufficient conditions (充分條件 ): if satisfied by a point, implying that the point is a minimum (though some minima may not satisfy the conditions)
order of conditions first-order conditions: in terms of the first derivatives of f & gj
second-order conditions: in terms of the second derivatives of f & gj
general assumptions: f, g, gj C1 (i.e., once continuously differentiable) or C2 (i.e., twice continuously differentiable) as required by the conditions
4
Feasible Direction Feasible Direction
S n: the feasible region
x S: a feasible point
a feasible direction d of x: if there exists > 0 such that x+d S for 0 < <
5
Two Key Concepts Two Key Concepts for Classical Results for Classical Results
f: the direction of steepest accent
gradient of f at x0 being orthogonal to the tangent of the contour f(x) = c at x0
6
The Direction of Steepest Accent The Direction of Steepest Accent ff contours of f(x1, x2) = ff: direction of steepest accent in some sense, increment of unit
move depending on the angle with f f within within 9090 of of ff: increasing: increasing
closer to 0closer to 0: increasing more: increasing more
beyond beyond 9090 of of ff: decreasing: decreasing closer to 180closer to 180: decreasing more: decreasing more
above results generally true for above results generally true for any any ff
x2
x1
2 21 2x x
ff((xx11, , xx22) =) =
ff((xx1010, , xx2020) = ) = cc
d on the tangent plane at xd on the tangent plane at x00
ff(x(x00++d) d) cc for small for small
roughly speaking, for roughly speaking, for ff(x(x00) = ) = cc, , ff(x(x00++d) = d) = cc for for
small small when d is on the tangent plane at x when d is on the tangent plane at x00
7
Gradient of Gradient of ff at x at x00 Being Orthogonal to Being Orthogonal to
the Tangent of the Contour the Tangent of the Contour ff(x) = (x) = cc at x at x00
2 21 2x x
8
First-Order Necessary Condition First-Order Necessary Condition (FONC)(FONC)
ff CC11 on S and x on S and x** a local minimum of a local minimum of ff then for any feasible direction d at xthen for any feasible direction d at x**, , TTff(x(x**)d )d
0 0 increasing of increasing of ff at any feasible direction at any feasible direction
ff((xx) = ) = xx22 for 2 for 2 xx 5 5 ff((xx, , yy) = ) = xx22 + + yy22 for 0 for 0 xx, , yy 2 2
ff((xx, , yy) = ) = xx22 + + yy22 for for xx 3, 3, yy 3 3
9
FONC for Unconstrained NLPFONC for Unconstrained NLP
ff CC11 on S & x on S & x** an interior local minimum an interior local minimum (i.e., without touching any boundary) (i.e., without touching any boundary) TTff(x(x**) = 0) = 0
10
FONC Not SufficientFONC Not Sufficient
Example 3.2.2: f(x, y) = -(x2 + y2) for 0 x, y Tf((0, 0))d = 0 for all feasible direction d
(0, 0): a maximum point
Example 3.2.3: f(x) = x3
f(0) = 0
x = 0 a stationary point
11
Feasible Region with Feasible Region with Non-negativity ConstraintsNon-negativity Constraints
Example 3.2.4. (Example 10.8 of JB) Find candidates of the minimum points by the FONC. min f(x) =
subject to x1 0, x2 0, x2 0
* ** *( ) ( )
0, if 0; 0, if 0.j jj j
f fx x
x x
x x
2 2 21 2 3 1 2 1 3 13 2 2 2x x x x x x x x
* **( ) ( )
0, 0jj j
f fx
x x
x xor, equivalently
12
Second-Order ConditionsSecond-Order Conditions
another form of Taylor’s Theorem f(x) = f(x*)+Tf(x*)(x-x*)
+0.5(x- x*)TH(x*)(x - x*)+ ,
where being small, dominated by other terms
if Tf(x*)(x-x*) = 0, f(x) f(x*) (x- x*)TH(x*)(x - x*) 0
13
Second-Order Necessary ConditionSecond-Order Necessary Condition
ff CC22 on S on S
if xif x** is a local minimum of is a local minimum of ff, then for any , then for any feasible direction d feasible direction d nn at x at x**, ,
(i).(i). TTff(x(x**)d )d 0, and 0, and
(ii). if (ii). if TTff(x(x**)d = 0, then d)d = 0, then dTTH(xH(x**)d )d 0 0
14
Example 3.3.1(a)Example 3.3.1(a)
SONC satisfied
ff((xx) = ) = xx22 for 2 for 2 xx 5 5 ff((xx, , yy) = ) = xx22 + + yy22 for 0 for 0 xx, , yy 2 2
ff((xx, , yy) = ) = xx22 + + yy22 for for xx 3, 3, yy 3 3
15
Example 3.3.1(b)Example 3.3.1(b)
SONC: more discriminative than FONCSONC: more discriminative than FONC
ff((xx, , yy) = -() = -(xx22 + + yy22) for 0 ) for 0 xx, , y y in Example in Example 3.2.2 3.2.2
(0, 0), a maximum point, failing the SONC(0, 0), a maximum point, failing the SONC
16
SONC for Unconstrained NLPSONC for Unconstrained NLP
ff CC22 in S in S
xx** an interior local minimum of an interior local minimum of ff, then, then
(i).(i). TTff(x(x**) = 0, and ) = 0, and
(ii). for (ii). for allall d, d d, dTTH(xH(x**)d )d 0 0 (ii) H(xH(x**) being positive semi-definite ) being positive semi-definite
convex convex f f satisfying (ii) (and actually more)satisfying (ii) (and actually more)
17
Example 3.3.2Example 3.3.2
identity candidates of minimum points for the f(x) =
Tf(x*) =
x = (1, -1) or (-1, -1)
H(x) =
(1, -1) satisfying SONC but not (-1, -1)
3 21 2 1 23 2x x x x
21 1 2(3 3 ,2 2)x x x
16 0
0 2
x
18
SONC Not SufficientSONC Not Sufficient
ff((xx, , yy) = -() = -(xx44 + + yy44))
TTff((0, 0))d = 0 for all d((0, 0))d = 0 for all d
(0, 0) a maximum(0, 0) a maximum
19
SOSC for Unconstrained NLPSOSC for Unconstrained NLP
ff CC22 on S on S nn and and xx** an interior point an interior point
if if (i). (i). TTff(x(x**) = 0, and ) = 0, and
(ii). H(x(ii). H(x**) is positive definite) is positive definite
xx** a strict local minimum of a strict local minimum of ff
20
SOSC Not NecessarySOSC Not Necessary
Example 3.3.4.
x = 0 a minimum of f(x) = x4
SOSC not satisfied
21
Example 3.3.5Example 3.3.5
In Example 3.2.4, is (1, 1, 1) a minimum?
.
6 > 0;
positive definite, i.e., SOSC satisfied
6 2 2
( ) 2 2 0
2 0 2
H x
6 212 ( 2)( 2) 8;
2 2
6 2 2
2 2 0 (6)(2)(2) ( 2)(2)( 2) ( 2)( 2)(2) 8
2 0 2
2 2 21 2 3 1 2 3 1 2 1 3 1( , , ) 3 2 2 2f x x x x x x x x x x x
22
Effect of ConvexityEffect of Convexity
If for all y in the neighborhood of x* S, Tf(x*)(y-x*) 0
convexity of f implies f(y) f(x*) + Tf(x*)(y-x*) f(x*)
x* a local min of f in the neighborhood of x*
x* a global minimum of f
23
Effect of ConvexityEffect of Convexity
f C2 convex H positive semi-definite everywhere
Taylor's Theorem, when Tf(x*)(x-x*) = 0, f(x) = f(x*) + Tf(x*)(x-x*)
+ (x- x*)TH(x* + (1-)x)(x - x*)
= f(x*) + (x- x*)TH(x* + (1-)x)(x - x*)
f(x*)
x* a local min a global min
24
Effect of ConvexityEffect of Convexity
facts of convex functions (i). a local min = a global min (ii). H(x) positive semi-definite everywhere (iii). strictly convex function, H(x) positive definite
everywhere
implications for f C2 convex function, the FONC Tf(x*) = 0 is
sufficient for x* to be a global minimum if f strictly convex, x* the unique global min