Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of...
Transcript of Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of...
![Page 1: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/1.jpg)
Numerical AnalysisMath 465/565
Order and Rates of Convergence
1
Saturday, September 14, 13
![Page 2: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/2.jpg)
“Speed of convergence”
2
We now have two algorithms which we can compare - bisection and the fixed-point method. Which is faster?
Hard to answer : Depends on what interval we start with, how close to a root we start with, etc. So for any two particular instances one method might converge in fewer iterations than the other.
How can we qualify more generally which method might be better in some sense?
Saturday, September 14, 13
![Page 3: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/3.jpg)
Review of bisection
3
We showed that bisection has the property that
Or, bisection ”behaves” like the sequence
�k = 2�k
If we define the error as ek = |xk � x̄|, we can write
ek |b� a| 2�k
|xk � x̄| |b� a|2k
Saturday, September 14, 13
![Page 4: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/4.jpg)
How fast does an algorithm converge?
4
Suppose that an algorithm produces iterates that converge as
limk!1
xk = x̄
If there exists a sequence �k that converges to zero and
a positive constant C, such that
ek = |xk � x̄| C|�k|
then xk is said to converge with rate �k.
Saturday, September 14, 13
![Page 5: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/5.jpg)
Convergence rate - Bisection
5
rate 2�k
|xk � x̄| |b� a| 12
k
So the bisection method has a convergence rate of
12k
with |b� a| as the asymptotic convergence constant.
Saturday, September 14, 13
![Page 6: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/6.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 7: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/7.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49;
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 8: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/8.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 9: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/9.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 10: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/10.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 11: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/11.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
>> log(2)
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 12: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/12.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
>> log(2)
ans =
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 13: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/13.jpg)
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
>> log(2)
ans =
0.693147180559945
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
![Page 14: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/14.jpg)
Fixed point method
7
Suppose over the interval [x0 � ⇢, x0 + ⇢], a function
g(x) satisfies a Lipschitz condition, and furthermore,
x0 is su�ciently close so that a sequence
xk+1 = g(xk), k = 0, 1, 2, . . .
converges to a root x̄ in [x0 � ⇢, x0 + ⇢]. Then
ek = |xk � x̄| �
k⇢
where |g0(x)| � < 1.
Saturday, September 14, 13
![Page 15: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/15.jpg)
Convergence of fixed point iteration
8
So the convergence rate is �
kwhere |g0(x)| � < 1.
Saturday, September 14, 13
![Page 16: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/16.jpg)
Estimated rate for fixed point
9
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 17: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/17.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 18: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/18.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 19: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/19.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 20: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/20.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 21: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/21.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 22: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/22.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 23: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/23.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 24: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/24.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
ans =
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 25: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/25.jpg)
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
ans =
1.3514
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
![Page 26: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/26.jpg)
Comparison of convergence rates
10
Bisection ⇡ 2
�k
Fixed point ⇡ �k
Which is better?
Saturday, September 14, 13
![Page 27: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/27.jpg)
Order of convergence
11
Suppose we have that
Then the convergence of the sequence xk to x̄ is said
to be of order ↵.
limk!1
|xk+1 � x̄||xk � x̄|↵ = lim
k!1
ek+1
e
↵k
= µ
Saturday, September 14, 13
![Page 28: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/28.jpg)
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13
![Page 29: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/29.jpg)
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13
![Page 30: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/30.jpg)
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
ab =
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13
![Page 31: Order and Rates of Convergence - Boise State University · 2013-10-07 · “Speed of convergence” 2 We now have two algorithms which we can compare - bisection and the fixed-point](https://reader033.fdocuments.in/reader033/viewer/2022041918/5e6a9ff86b71831c1d0fdec7/html5/thumbnails/31.jpg)
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
ab =
0.9885 -0.8910
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13