11. Optimum Design Concepts - CAUisdl.cau.ac.kr/education.data/DOEOPT/11.opt.design.concepts.pdf ·...
Transcript of 11. Optimum Design Concepts - CAUisdl.cau.ac.kr/education.data/DOEOPT/11.opt.design.concepts.pdf ·...
Hae-Jin ChoiSchool of Mechanical Engineering,
Chung-Ang University
11. Optimum Design Concepts
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-1-
Introduction to basic ideas, concepts, and
theories used for optimization
Once the problems from different fields
have been transcribed into mathematical
statements, they become the same
mathematical problem to solve using
optimization theories.
Optimization methods are categorized into
indirect or direct (search) methods
Indirect methods: seeking solution from
optimality criteria
Direct methods: seeking solution from
initial estimation and iterative process
DOE and Optimization
Introduction
Optimization
methods
Indirect
methods
Direct
methods
Constrained
problem
Unconstrained
problem
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-2-
Global (absolute) Minimum
A function f(x) of n variables has global
(absolute) minimum at x* if f(x*)≤ f(x) for all
x in the feasible region.
Local (relative) Minimum
A function f(x) of n variables has a local
(relative) minimum at x* if f(x*)≤ f(x) for all
x in a small neighborhood N of x* in the
feasible region, where
DOE and Optimization
Global and Local Minimum
{ | with }N S *x x x x
x
f(x)
x
f(x)
x=a x=b
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-3-
Geometrically, the gradient vector is normal to the tangent plane at
the point x* for a function. -> the direction of maximum increase in
the function
At a given point (x*), the gradient vector is
DOE and Optimization
Gradient Vector
1
2
1 2
( )
( )( ) ( ) ( )
( ) ....
....
( )
T
n
n
f
x
ff f f
f xx x x
f
x
*
*
* * **
*
x
xx x x
x
x
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-4-
Hessian matrix is obtained
differentiating the gradient vector
once again where all derivatives are
calculated at a given point x*
Hessian is an n x n matrix usually
denoted as H or .
Hessian is always an symmetric
matrix since
Hessian plays an important role in the
sufficiency condition of optimality
DOE and Optimization
Hessian Matrix
2
2 2 2
2
1 1 2 1
2 2 2
2
2 1 2 2
2 2 2
2
1 2
where i=1 to n and j=1 to n
or
...
...
... ... ... ...
...
i j
n
n
n n n
f
x x
f f f
x x x x x
f f f
x x x x x
f f f
x x x x x
H
2 f
2 2
i j j i
f f
x x x x
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-5-
Approximated function by polynomial in a neighborhood of any
point in terms of its value and derivative.
Taylor’s series expansion about the point x* is
where R is remainder (higher order terms)
Let x-x*=d
DOE and Optimization
Taylor Series Expansion
* 2 ** * * 2
2
( ) 1 ( )( ) ( ) ( ) ( )
2
df x d f xf x f x x x x x R
dx dx
* 2 ** * 2
2
( ) 1 ( )( ) ( )
2
df x d f xf x d f x d d R
dx dx
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-6-
For two variables
In general
DOE and Optimization
Taylor Series Expansion
* * * ** * * *1 2 1 2
1 2 1 2 1 1 2 2
1 2
2 * * 2 * * 2 * ** 2 * * * 21 2 1 2 1 2
1 1 1 1 2 2 2 22 2
1 1 2 2
( , )) ( , ))( , ) ( , ) ( ) ( )
( , ) ( , ) ( , )1( ) 2 ( )( ) ( )
2
f x x f x xf x x f x x x x x x
x x
f x x f x x f x xx x x x x x x x R
x x x x
*
1( ) ( ) ( ) ( ) ( )
2
1( ) ( )
2
T T
T T
f f f R
or
f f f R
* * * *
*
x x x - x x - x H x - x
x + d x d d Hd
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-7-
Quadratic form is a special nonlinear function having only second-order
terms. e.g.,
In general
or
Replacing P with a symmetric matrix A
H in the Taylor Series Expansion is associated with the quadratic form
DOE and Optimization
Matrix of Quadratic Form
2 2 2
1 2 3 1 2 2 3 3 1( ) 2 3 2 2 2f x x x x x x x x xx
1 1
1( ) are known constants
2
n n
ij i j ij
i j
f p x x where px
1( )
2
is called matix of quadatic form
Tf
where
x x Px
P
1 1( )
2 2
T Tf x x Px x Ax
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-8-DOE and Optimization
Example: Matrix of Quadratic Form
Identify a matrix associated with the quadratic form
Dividing the coefficients equally between pij and pji,
2 2 2
1 2 3 1 1 2 1 3 2 2 3 3
1( , , ) (2 2 4 6 4 5 )
2F x x x x x x x x x x x x
1 1
1 2 3 2 1 2 3 2
3 3
2 2 4 2 0.5 11 1 1
( ) [ ] 0 6 4 [ ] 1.5 6 62 2 2
0 0 5 3 2 5
T
x x
F x x x x x x x x
x x
x x Px
1
1 2 3 2
3
2 1 21 1
( ) [ ] 1 6 22 2
2 2 5
T
x
F x x x x
x
x x Ax
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-9-DOE and Optimization
Matrix of Quadratic Form
If for all x except x=0, then the
quadratic form is positive definite
If for all x except x=0, then the
quadratic form is negative definite
If for all x (at lease one that
makes ) then the quadratic form
is positive semidefinite
If for all x (at lease one that
makes ) then the quadratic form
is negative semidefinite
0Tx Ax
0Tx Ax
0Tx Ax
0Tx Ax
x 0
0Tx Ax
x 0
0Tx Ax
2 2 2
1 2 3
1 2 3
2 0 0
0 4 0
0 0 3
2 4 3 0
for all unless 0
Therefore is positive definite
T x x x
x x x
A
x Ax
x
A
2 2 2
1 2 1 2 3
2 2
3 1 2
1 2 3
1 1 0
1 1 0
0 0 1
( 2 )
( ) 0
for all and
0 when and 0
Therefore is negative semidefinite
T
T
x x x x x
x x x
x x x
A
x Ax
x
x Ax
A
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-10-
F(x)=xTAx is a positive definite if and only if all eigenvalues of A are
strictly positive, i.e., λi > 0, i=1 to n
F(x) =xTAx is a positive semidefinite if and only if all eigenvalues of A
are non-negative, i.e., λi ≥ 0, i=1 to n (at least one eigenvalue is
zero)
F(x) =xTAx is a negative definite if and only if all eigenvalues of A are
strictly negative, i.e., λi < 0, i=1 to n
F(x) =xTAx is a negative semidefinite if and only if all eigenvalues of A
are non-positive, i.e., λi ≤ 0, i=1 to n (at least one eigenvalue is
zero)
F(x) =xTAx is indefinite if some λi < 0 and some other λi > 0
DOE and Optimization
Method of Checking - Eigenvalues
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-11-
To differentiate the Quadratic Form with respective to xi
Therefore, the Gradient of the quadratic form is
Differentiating once again with respect to xj, we get
The Hessian matrix is
DOE and Optimization
Differentiation of a Quadratic Form
1 1
1( )
2
n n
ij i j
i j
F a x xx
1
n
ij j
ji
Fa x
x
( )F x Ax2
ij
i j
Fa
x x
H A
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-12-
Calculate gradient and Hessian of following quadratic form
DOE and Optimization
Example
2 2 2
1 1 2 1 3 2 2 3 3
1( ) (2 2 4 6 4 5 )
2F x x x x x x x x xx
1
1 2 3 2
3
1
2
3
2 1 21 1
( ) [ ] 1 6 22 2
2 2 5
2 1 2
( ) 1 6 2
2 2 5
2 1 2
1 6 2
2 2 5
T
x
F x x x x
x
x
F x
x
x x Ax
x Ax
H = A
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-13-
Minimization of f(x) without any constraint on x.
Not a practical case, but good for understanding constrained
optimization concept.
Optimality Condition for Function of Single Variable
First-order necessary condition
Sufficient condition
Optimality Condition for Functions of Multiple Variables
DOE and Optimization
Unconstrained Optimum Design Problem
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-14-
Necessary condition for optimality: this
condition must be satisfied for a point to be
optimum
If any point does not satisfy the necessary condition, it
cannot be optimum.
Satisfaction of the necessary condition does not guarantee an
optimum point
Sufficient condition for optimality: this condition
provides tests to distinguish between optimum and
non-optimum points
If a candidate optimum point satisfies the sufficient
conditions, then it is indeed optimum
Even if no point that satisfies the sufficient condition, we
still have a chance to have an optimum.
DOE and Optimization
Optimality Conditions
Optimum condition
Sufficient
condition
Necessary condition
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-15-
* 2 ** 2
2
( ) 1 ( )( ) ( )
2
df x d f xf x f x d d R
dx dx
DOE and Optimization
Necessary Condition for Function of Single Variable
Taylor series expansion near x*
Change in the function near x*
* 2 ** 2
2
( ) 1 ( )( ) ( )
2
df x d f xf f x f x d d R
dx dx
The change must be non-negative value (≥0) when x* is local minimum
Therefore, ignoring the remainder R
Since d could be positive or negative value,
f
*( )0
df xd
dx
*( )0
df x
dx
x* is stationary
point
that could be
maximum,
minimum, or
saddle point
The first term is the dominant term
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-16-DOE and Optimization
Sufficient Condition for Function of Single Variable
2 *2
2
1 ( )
2
d f xf d R
dx
Since d2 is always positive, for x* to be minimum, ignoring
the remainder, R.
2 *
2
( )0
d f x
dx
Since f’(x)=0
Now, this term is dominant term
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-17-
For the general case of a function of multiple variables, f(x) where x is an
n-vector,
The necessary condition using the multidimensional form of Taylor series
expansion is
Since the first term is dominant, the necessary condition is
With the second term the sufficient condition is
DOE and Optimization
Optimality Condition for Functions of Multiple
Variables
* 1( ) ( ) ( *) ( *)
2
T Tf f f f R*x + d x x d d H x d
( *)f x 0
( *) is poisitive definiteH x
0Td H(x*)d
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-18-
Find a local minimum point for the function
With necessary condition
DOE and Optimization
Example: Local minima for a function
1 2
1 2
(4.0 06)( ) 250
Ef x x
x xx
2
1 2
2
1 2
2 2
1 2 1 2
2 2
1 2 1 2 1 2 1 2
1 2
4.0 061
0( )
4.0 06 0250
Solving the following two equations
4.0 06 0 250( ) 4.0 06 0
250( ) ( 250 ) 0
250
Substituting this t
E
x xf
E
x x
x x E and x x E
x x x x x x x x
x x
x
1 2
o the original equation,
1000 and 4
The stationary point is (1000, 4)
x x
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-19-
Find a local minimum point for the function
(solution)
With necessary condition
DOE and Optimization
Example: Local minima for a function
1 2
1 2
(4.0 06)( ) 250
Ef x x
x xx
2
1 2
2
1 2
2 2
1 2 1 2
2 2
1 2 1 2 1 2 1 2
1 2
4.0 061
0( )
4.0 06 0250
Solving the following two equations
4.0 06 0 250( ) 4.0 06 0
250( ) ( 250 ) 0
250
Substituting this t
E
x xf
E
x x
x x E and x x E
x x x x x x x x
x x
x
1 2
o the original equation,
1000 and 4
The stationary point is (1000, 4)
x x
General approach for
solving simultaneous
non-linear equations
Newton-Raphson method
(See Appendix)
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-20-
The Hessian Matrix
Checking sufficient condition at the stationary point (1000, 4)
H is positive definite since the two eigenvalues of the Hessian matrix are
0.006 and 500.002, which are all positive values.
Therefore, (1000, 4) is indeed local minimum, and the local minimum is
f(1000, 4) = 3000
DOE and Optimization
Example: Local minima for a function
2
1
2 3
11 2
2
21
4.0 06
21
x
xE
xx x
x
H
2
0.008 1(4.0 06)(1000,4)
1 5004000
EH
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-21-DOE and Optimization
Example: Local minima for a function
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-22-
Appendix: Newton-Raphson
Numerical method for simultaneous non-linear equations.
For a one-variable system, the Taylor series approximation and resulting Newton-Raphson equations are:
For a two-variable system,
)()()()( 11 iiiii xfxxxfxf 1
( )
( )
ii i
i
f xx x
f x
f1,i1 f1,i x1,i1 x1,i f1,i
x1 x2,i1 x2,i
f1,i
x2x1,i1 x1,i
f1,if2,ix2
f2,if1,ix2
f1,ix1
f2,ix2
f1,ix2
f2,ix1
f2,i1 f2,i x1,i1 x1,i f2,i
x1 x2,i1 x2,i
f2,i
x2x2,i1 x2,i
f2,if1,ix1
f1,if2,ix1
f1,ix1
f2,ix2
f1,ix2
f2,ix1
DOE and Optimization
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-23-
1[ ]{ } { } i iZ x x f
n
ininin
n
iii
n
iii
x
f
x
f
x
f
x
f
x
f
x
f
x
f
x
f
x
f
Z
,
2
,
1
,
,2
2
,2
1
,2
,1
2
,1
1
,1
][
1, 2, ,{ }T
i i i n ix x x x
1 1, 1 2, 1 , 1{ }T
i i i n ix x x x
1, 2, ,{ }T
i i n if f f f
1, 1,
1, 1 1, 1, 1 1, 2, 1 2,
1 2
2, 2,
2, 1 2, 1, 1 1, 2, 1 2,
1 2
i i
i i i i i i
i i
i i i i i i
f ff f x x x x
x x
f ff f x x x x
x x
1[ ]{ } { } [ ]{ }, where [ ] i iZ x f Z x Z Jacobian matrix
DOE and Optimization
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-24-DOE and Optimization
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-25-
Ex.Use the Newton-Raphson method to determine the roots of the equations. Use the initial guesses of x1 =1.5 and x2 = 3.5.
573
10
2
212
21
2
1
xxx
xxx
5.32)5.3)(5.1(6161 75.36)5.3(33
5.1 5.65.3)5.1(22
21
2
0,222
2
1
0,2
1
2
0,1
21
1
0,1
xxx
fx
x
f
xx
fxx
x
f
6.5(32.5) 1.5(36.75) 156.125Jacobian
DOE and Optimization
SCHOOL OF MECHANICAL ENG.
CHUNG-ANG UNIVERSITY-26-
5.210)5.3(5.1)5.1( 2
0,1 f
625.157)5.3)(5.1(35.3 2
0,2 f
The values of the functions can be evaluated at the initial guesses as
These values can be substituted to give
03603.2125.156
)5.1(625.1)5.32(5.25.11
x
84388.2125.156
)75.36)(5.2()5.6(625.15.32
x
The computation can be repeated until an acceptable accuracyis obtained.
DOE and Optimization