L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4...
-
Upload
alexia-sharp -
Category
Documents
-
view
212 -
download
0
Transcript of L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4...
![Page 1: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/1.jpg)
L22 Numerical Methods part 2
• Homework• Review• Alternate Equal Interval• Golden Section• Summary• Test 4
1
![Page 2: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/2.jpg)
2
![Page 3: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/3.jpg)
Problem 10.4
3
2 2 21 2 3 1 2 2 3: ( ) 2 2 2 2
( 3,10, 12) (1,2,3)
Determine whether is a descent direction. . 0
given f x x x x x x xand at
i e
xd x
dc d
1 2
2 1 3
3 2 *
2 2 2(1) 2(2) 6( *) 4 2 2 4(2) 2(1) 2(3) 16
4(3) 2(2) 164 2x
x xf x x x
x x
c x
1 1 2 2 3 3
03
6 16 16 10
12
6( 3) 16(10) 16( 12)18 160 19250 0
c d c d c d
T
c d
c d c d
Yes, descent direction
![Page 4: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/4.jpg)
Prob 10.10
4
2 21 2 2 3
:( ) ( ) ( )
(4,8,4) (1,1,1)
Determine whether is a descent direction. . 0
givenf x x x xand at
i e
xd x
dc d
1 2
1 2 2 3
2 3 *
2( ) 2(1 1) 4( *) 2( ) 2( ) 2(1 1) 2(1 1) 8
2(1 1) 42( )x
x xf x x x x
x x
c x
1 1 2 2 3 3
04
4 8 4 8
4
4(4) 8(8) 4(4)16 64 1696 0
c d c d c d
T
c d
c d c d
No, not a descent direction
![Page 5: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/5.jpg)
Prob 10.19
5
1 2
2 1 3
3 2 *
2 2 2(2) 2(4) 12( *) 4 2 2 4(4) 2(2) 2(10) 40
4(10) 2(4) 484 2
1212 40 48 40
4812( 12) 40( 40) 48( 48)
(144 1600 2304)4048
x
x xf x x x
x x
c x
c d
2 2 21 2 3 1 2 2 3
:
( ) 2 2 2 2( 12, 40, 48) (2,4,10)
What is the slope of the function at the point?Find the optimum step size.
given
f x x x x x x xand at
xd x
Slope
![Page 6: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/6.jpg)
Prob 10.19 cont’d
6
( 1) ( ) ( )
( ) ( )
1 1 1
2 2 2
3 3 3( ) ( )
1
2
3
1
2
2
2 124 4010 48
2 ( 12) 2 124 ( 40) 4 4010 ( 48) 10 48
k k k
new old
new old
x x dx x dx x d
xxxxxx
x x d
2 2 21 2 3 1 2 2 3
2 2 2
( ) 2 2 2 2
(2 12 ) 2(4 40 ) 2(10 48 ) 2(2 12 )(4 40 ) 2(4 40 )(10 48 )
( ) 4048 25504 0* 4048 / 25504 0.15873
f x x x x x x x
f
x
![Page 7: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/7.jpg)
Prob 10.19 cont’d
7
alpha 0.15872
x1 0.095358x2 -2.34881x3 2.38143
f(x) 10.75031
( 1) ( ) ( )
( ) ( )
1
2
3
1
2
2
2 124 0.15873 4010 48
2 0.15873( 12) 0.0954 0.15873( 40) 2.3510 0.15873( 48) 2.38
k k k
new oldxxx
xxx
x x d
![Page 8: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/8.jpg)
Prob 10.30
8
( 1) ( ) ( )
( ) ( )
1
2
3
4
1
2
3
4
2221
4 23 2
2 ( 2)1 ( 2)4 ( 2)3 ( 2)
k k k
new oldxxxx
xxxx
x x d
2 2 2 2
2
( ) ((2 2 ) 1) ((1 2 ) 2) ((4 2 ) 3) ((3 2 ) 4)
( ) 16 16 4
f
f
2 2 2 21 2 3 4( ) ( 1) ( 2) ( 3) ( 4)
(2,1,4,3) ( 2,2, 2,2)( )
f x x x x
find f
xx d
![Page 9: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/9.jpg)
The Search Problem
• Sub Problem AWhich direction to head next?
• Sub Problem BHow far to go in that direction?
9
( )kx
![Page 10: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/10.jpg)
Search Direction… Min f(x): Let’s go downhill!
10
( *)Tf f x d
1( ) ( *) ( *)( *) ( *) ( *)( *)
2T Tf f f R x x x x x x x H x x x
1( * ) ( *) ( *)
2T Tf f f R x d x x d d H d
( ) ( )let ( *) then new oldor d x x x x * d x x d
( *) 0Tf x dDescent condition
( *)Tlet fc x
0c d
![Page 11: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/11.jpg)
Step Size?
How big should we make alpha?Can we step too “far?”
i.e. can our step size be chosen so big that we step over the “minimum?”
11
![Page 12: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/12.jpg)
12
Figure 10.2 Conceptual diagram for iterative steps of an optimization method.
We are hereWhich direction should we head?
![Page 13: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/13.jpg)
Some Step Size Methods
• “Analytical”Search direction = (-) gradient, (i.e. line search)Form line search function f(α)Find f’(α)=0
• Region Elimination (“interval reducing”)Equal intervalAlternate equal intervalGolden Section
13
![Page 14: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/14.jpg)
14
Figure 10.5 Nonunimodal function f() for 0
Nonunimodal functions
Unimodal if stay in locale?
![Page 15: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/15.jpg)
Monotonic Increasing Functions
15
![Page 16: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/16.jpg)
Monotonic Decreasing Functions
16
continous
![Page 17: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/17.jpg)
17
Figure 10.4 Unimodal function f().
Unimodal functions
monotonic increasing then monotonic decreasing
monotonic decreasing then monotonic increasing
![Page 18: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/18.jpg)
Some Step Size Methods
• “Analytical”Search direction = (-) gradient, (i.e. line search)Form line search function f(α)Find f’(α)=0
• Region Elimination (“interval reducing”)Equal intervalAlternate equal intervalGolden Section
18
![Page 19: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/19.jpg)
19
Figure 10.3 Graph of f() versus .
Analytical Step size
( 1) ( )( ) ( + ) ( )k kf f f x x d
( )
( ) ( ) ( )
( 1) ( ) ( )
given , and letthen
old
new old k
k k k
d xx x dx x d
( 1) ( )( ) ( + ) ( )'( )=0
k kf f ff
x x dSlope of line
search=c d
Slope of line at fmin0c d
![Page 20: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/20.jpg)
Analytical Step Size Example
20
2 21 2: ( ) ( 2) ( 1)
44
find optimal step size *and ( *)!
given f x x
and let and at
f
x
d c x
2 21 2
2 2
2
( ) ( 2) ( 1)
( ) ((4 4 ) 2) ((4 6 ) 1)( ) 52 52 13( ) 2(52 ) 52 0
* 1/ 2
f x x
fff
1
2
2 21 2
2 2
4 1/ 2( 4) 24 1/ 2( 6) 12
*1
( *) ( 2) ( 1)
(2 2) (1 1) 0
xx
f x x
x
x
1
2
2( 2) 2(4 2)2( 1) 2(4 1)
46
xx
d c
( 1) ( ) ( )
( ) ( )
1 1 1
2 2 2
1
2
4 ( 4) 4 44 ( 6) 4 6
k k k
new oldx x dx x dxx
x x d
2 2( ) ((4 4 ) 2) ((4 6 ) 1)( ) 2((4 4 ) 2)( 4) 2((4 6 ) 1)( 6) 0( *) (2 4 )( 4) (3 6 )( 6) 0
8 16 18 36 26 52 0* 26 / 52 1 / 2
fff
![Page 21: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/21.jpg)
Alternative Analytical Step Size
21
( 1) ( )
( 1) ( 1) ( 1)
( 1) ( ) ( )
( 1)( )
( 1) ( )
( 1) ( )
( ) ( + ) ( )'( )=0( ) ( ) ( )
0
since( )
( ) 0
0
k k
k T k k
k k k
kk
k k
k k
f f ffdf f d
d d
d
df
x x d
x x x
xx x d
xd
x d
c d
( 1) ( ) ( )
1
2
4 ( 4) 4 44 ( 6) 4 6
k k k
xx
x x d
( 1) ( ) 04
4 8 , 6 126
4(4 8 ) 6(6 12 ) 016 32 36 72 052 104 0
52 / 104 1/ 2
k k
T
c d
( 1) 1
2
2( 2)2( 1)
2(4 4 2)2(4 6 1)
4 86 12
k xx
c
New gradient must be orthogonal to d for ' ( )=0f
![Page 22: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/22.jpg)
Some Step Size Methods
• “Analytical”Search direction = (-) gradient, (i.e. line search)Form line search function f(α)Find f’(α)=0
• Region Elimination (“interval reducing”)Equal intervalAlternate equal intervalGolden Section
22
![Page 23: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/23.jpg)
23
Figure 10.6 Equal-interval search process. (a) Phase I: initial bracketing of minimum. (b) Phase II: reducing the interval of uncertainty.
“Interval Reducing”Region elimination
“bounding phase”
Interval reductionphase”
( 1)( 1)
2
l
u
u l
I
![Page 24: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/24.jpg)
2 delta!
24
![Page 25: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/25.jpg)
Successive-Equal Interval Algorithm
25
x f(x)-5.0000 22.0067-4.0000 18.0183-3.0000 14.0498-2.0000 10.1353-1.0000 6.36790.0000 3.00001.0000 0.71832.0000 1.38913.0000 10.08554.0000 40.59825.0000 130.4132
( ) 2 - 4 exp( )f x x x x f(x)
0.0000 3.00000.2000 2.42140.4000 1.89180.6000 1.42210.8000 1.02551.0000 0.71831.2000 0.52011.4000 0.45521.6000 0.55301.8000 0.84962.0000 1.3891
x f(x)1.2000 0.52011.2400 0.49561.2800 0.47661.3200 0.46341.3600 0.45621.4000 0.45521.4400 0.46071.4800 0.47291.5200 0.49221.5600 0.51881.6000 0.5530
x f(x)1.3600 0.4561931.3680 0.4554881.3760 0.4550341.3840 0.4548331.3920 0.4548881.4000 0.4552001.4080 0.4557721.4160 0.4566051.4240 0.4577021.4320 0.4590651.4400 0.460696
x lower -5x upper 5
delta 1
02
0.2
1.21.6
0.04
“Interval” of uncertainty
1.361.44
0.008
![Page 26: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/26.jpg)
More on bounding phase I
Swan’s method
Fibonacci sequence
26
( 1) ( )
( 1) ( )
210
k k k
k k k
0
(1.618)k
jq
j
![Page 27: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/27.jpg)
Successive Alternate Equal Interval
27
Assume bounding phase has found Min can be on either side of
But for sure its not in this region!
1
32 1
3 3
b l
b l u
I
I I
u land
b
Point values… not a line
![Page 28: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/28.jpg)
Successive Alt Equal Int
28
ME 482 Optimal Design alt_eq_int.xls RJE 4/11/2012
Alternate Equal Interval Search for locating minimum of f(x)x lower 0.5x upper 2.618
Iteration xl 1/3(xu+2xl) 1/3(2xu+xl) xu Interval Optimal
1 x 0.5000000 1.2060000 1.9120000 2.6180000 2.118000 1.2060000f(x) 1.6487213 0.5160975 1.1186085 5.2362796 0.5160975
2 x 0.5000000 0.9706667 1.4413333 1.9120000 1.412000 1.4413333f(x) 1.6487213 0.7570370 0.4609938 1.1186085 0.4609938
3 x 0.9706667 1.2844444 1.5982222 1.9120000 0.941333 1.2844444 f(x) 0.7570370 0.4748826 0.5513460 1.1186085 0.47488264 x 0.9706667 1.1798519 1.3890370 1.5982222 0.627556 1.3890370 f(x) 0.7570370 0.5344847 0.4548376 0.5513460 0.45483765 x 1.1798519 1.3193086 1.4587654 1.5982222 0.418370 1.3193086
f(x) 0.5344847 0.4635997 0.4655851 0.5513460 0.4635997
( ) 2 - 4 exp( )f x x x
Requires two function evaluations per iteration
![Page 29: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/29.jpg)
29
Figure 10.8 Initial bracketing of the minimum point in the golden section method.
Fibonacci Bounding
![Page 30: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/30.jpg)
30
Figure 10.9 Graphic of a section partition.
Golden section
22
1,2
1 1 4(1)( 1)4
2 2(1)
1 5 1 2.236
2 20.618, 1.618
b b ac
a
2
leftside = rightside(1 )
[ ] (1 )[ ] (1 )
1 0
I II I
![Page 31: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/31.jpg)
Golden Section Example
31
ME 482 Optimal Design alt_gold.xls RJE 4/11/2012
Golden Section Region Elimiation Search for locating minimum of f(x)x lower 0.5 1-τ= 0.381966x upper 2.618 τ = 0.618034
Iteration xl xL+(1-τ) I xL+ τ I xu Interval I Optimal
1 x 0.5 1.309004 1.808996 2.618 2.118 1.30900399f(x) 1.6487213 0.4664682 0.8683316 5.2362796 0.4664682
2 x 0.5 0.999992 1.309004 1.808996 1.308996 1.30900404f(x) 1.6487213 0.7182921 0.4664682 0.8683316 0.46646819
3 x 0.999992 1.309004 1.499984 1.808996 0.809004 1.30900401 f(x) 0.7182921 0.4664682 0.4816814 0.8683316 0.46646824 x 0.999992 1.1909719 1.309004 1.499984 0.499992 1.30900403 f(x) 0.7182921 0.5263899 0.4664682 0.4816814 0.466468195 x 1.1909719 1.309004 1.3819519 1.499984 0.3090121 1.38195187
f(x) 0.5263899 0.4664682 0.4548602 0.4816814 0.45486022
( ) 2 - 4 exp( )f x x x
![Page 32: L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1.](https://reader036.fdocuments.in/reader036/viewer/2022081603/5697bfd21a28abf838cab998/html5/thumbnails/32.jpg)
Summary• General Opt Algorithms have two sub problems:
search direction, and step size
• Descent condition assures correct direction• For line searches…in local neighborhood…
we can assume unimodal!
• Step size methods: analytical, region elimin.• Region Elimination (“interval reducing”)
Equal interval Alternate equal interval Golden Section 32