Numerical Analysis

1

EE, NCKUTien-Hao Chang (Darby Chang)

In the previous slide Rootfinding

– multiplicity

Bisection method– Intermediate Value Theorem

– convergence measures

False position– yet another simple enclosure method

– advantage and disadvantage in comparison with bisection method

2

In this slide Fixed point iteration scheme

– what is a fixed point?

– iteration function

– convergence

Newton’s method– tangent line approximation

– convergence

Secant method3

Rootfinding Simple enclosure

– Intermediate Value Theorem

– guarantee to converge• convergence rate is slow

– bisection and false position

Fixed point iteration– Mean Value Theorem

– rapid convergence• loss of guaranteed convergence

4

2.3

5

Fixed Point Iteration Schemes

6

7

There is at least one point on the graph at which the tangent lines is parallel to the secant line

Mean Value Theorem

We use a slightly different formulation

An example of using this theorem– proof the inequality

8

9

Fixed points Consider the function

– thought of as moving the input value of to the output value

– the sine function maps to • the sine function fixes the location of

– is said to be a fixed point of the function

10

11

Number of fixed points According to the previous figure, a

trivial question is– how many fixed points of a given

function?

12

13

14

𝑔 ′ (𝑥 )≤𝑘<1

Only sufficient conditions

Namely, not necessary conditions– it is possible for a function to violate one or more of the

hypotheses, yet still have a (possibly unique) fixed point

15

Fixed point iteration

16

Fixed point iteration If it is known that a function has a

fixed point, one way to approximate the value of that fixed point is fixed point iteration scheme

These can be defined as follows:

17

18

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

19

20

Any Questions?

21

About fixed point iteration

Relation to rootfinding Now we know what fixed point

iteration is, but how to apply it on rootfinding?

More precisely, given a rootfinding equation, f(x)=x3+x2-3x-3=0, what is its iteration function g(x)?

22

hint

Iteration function Algebraically transform to the form

– …

Every rootfinding problem can be transformed into any number of fixed point problems– (fortunately or unfortunately?)

23

24

25

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

26

Analysis #1 iteration function converges

– but to a fixed point outside the interval

#2 fails to converge– despite attaining values quite close to #1

#3 and #5 converge rapidly– #3 add one correct decimal every iteration

– #5 doubles correct decimals every iteration

#4 converges, but very slow

27

Convergence This analysis suggests a trivial question

– the fixed point of is justified in our previous theorem

28

29

30

31

𝑘 demonstrates the importance of the

parameter – when , rapid

– when , dramatically slow

– when , roughly the same as the bisection method

32

Order of convergence of fixed point iteration schemes

33

All about the derivatives,

34

35

36

37

38

Stopping condition

39

40

Two steps

41

The first step

42

The second step

43

Any Questions?

44

2.3 Fixed Point Iteration Schemes

2.4

45

Newton’s Method

46

47

Newton’s Method

Definition

48

49

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

50

In the previous example Newton’s method used 8 function

evaluations Bisection method requires 36

evaluations starting from False position requires 31 evaluations

starting from

51

52

Any Questions?

53

Initial guess Are these comparisons fair?

– , converges to after 5 iterations

– , fails to converges after 5000 iterations

– , converges to after 42 iterations

54

example

answer

Initial guess Are these comparisons fair?

– , converges to after 5 iterations

– , fails to converges after 5000 iterations

– , converges to after 42 iterations

55

answer

Initial guess Are these comparisons fair?

– , converges to after 5 iterations

– , fails to converges after 5000 iterations

– , converges to after 42 iterations

56

in Newton’s method Not guaranteed to converge

– , fails to converge

May converge to a value very far from – , converges to

Heavily dependent on the choice of

57

Convergence analysis for Newton’s method

58

59

The simplest plan is to apply the general fixed point iteration convergence theorem

Analysis strategy To do this, it is must be shown that

there exists such an interval, , which contains the root , for which

60

61

62

63

64

Any Questions?

65

Newton’s Method

Guaranteed to Converge?

Why sometimes Newton’s method does not converge?

This theorem guarantees that exists But it may be very small

66

hint

answer

Newton’s Method

Guaranteed to Converge?

Why sometimes Newton’s method does not converge?

This theorem guarantees that exists But it may be very small

67

answer

Newton’s Method

Guaranteed to Converge?

Why sometimes Newton’s method does not converge?

This theorem guarantees that exists But it may be very small

68

69

Oh no! After these annoying analyses, the Newton’s method is still not guaranteed to converge!?

http://img2.timeinc.net/people/i/2007/startracks/071008/brad_pitt300.jpg

Don’t worry Actually, there is an intuitive method Combine Newton’s method and bisection

method– Newton’s method first

– if an approximation falls outside current interval, then apply bisection method to obtain a better guess

(Can you write an algorithm for this method?)

70

Newton’s Method

Convergence analysis At least quadratic

–

– , since

Stopping condition–

71

72http://www.dianadepasquale.com/ThinkingMonkey.jpg

Recall that

73

Is Newton’s method always faster?

74

75

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

76

Any Questions?

77

2.4 Newton’s Method

2.5

78

Secant Method

Secant method Because that Newton’s method

– 2 function evaluations per iteration

– requires the derivative

Secant method is a variation on either false position or Newton’s method– 1 additional function evaluation per iteration

– does not require the derivative

Let’s see the figure first

79

answer

80

Secant method Secant method is a variation on

either false position or Newton’s method– 1 additional function evaluation per

iteration

– does not require the derivative

– does not maintain an interval

– is calculated with and

81

82

83

84

85

Any Questions?

86

2.5 Secant Method