Nonlinear Equations Jun-Feng Yinenglish-c.tongji.edu.cn/_SiteConf/files/2014/05/07/... · Jun-Feng...

Nonlinear Equations

Jun-Feng Yin

Tongji University

[email protected]

http://www.tongji.edu.cn/∼yin

Copyright c©2011, NA�Yin Last Modification: Oct. 2011 1

Nonlinear Equations

⇒ Motivation

• Bisection Method

• Newton’s Method

• Secant Method

• Summary


Motivation

• For a given function f(x), find its root(s), i.e.:

⇒ find x (or r=root) such that f(x) = 0

• BVP: dipping of suspended power cable. What is λ?

λ cosh50

λ− λ− 10 = 0

• (Some) simple equations ⇒ solve analytically

6x2 − 7x+ 2 = 0 cos 3x− cos 7x = 0(3x− 2)(2x− 1) = 0 2 sin 5x sin 2x = 0

x = 23,

12 x = nπ

5 ,nπ2 , n ∈ Z


Motivation(cont.)

• In genetal, we cannot exploit the function, e.g.:

2x2− 10x+ 1 = 0

andcosh(

√x2 + 1− ex) + log | sinx| = 0

• Note: at times ∃ multiple roots

∗ e.g., previous parabola and cosine

∗ we want at least one

∗ we may only get one(for each search)

Need a general, function-independent algorithm.


Nonlinear Equations

• Motivation

⇒ Bisection Method


• Secant Method

• Summary


Bisection Method–Example

Intuitive, like guessing a number ∈ [0, 100].


Restrictions and Max Error Estimate

• Restrictions

∗ function slices x-axis at root? start with two points a and b 3 f(a)f(b) < 0? graphing tool (e.g., Matlab) can help to find a and b

∗ require C0[a, b] (why? note: not a big deal)

• Max error estimate

∗ after n steps, guess midpoint of current range∗ error: ε ≤ b−a

2n+1 (think of n = 0, 1, 2)∗ note: error is in x; can also look at error in f(x) or combination

? enters entire world of stopping criteria

Question: Given tolerance (in x), what is n? · · ·


Convergence Rate

• Given tolerance τ (e.g., 10−6), how many steps are needed?

• Tolerance restriction (ε from before):

(ε ≤ b− a2n+1

) < r

• log (any base)

log(b− a)− n log 2 < log 2r

or

n >log(b− a)− log 2r

log 2

Rate is independent of function.


Convergence Rate (cont.)

• Base 2 (i.e., bites of accuracy)

n > log2(b− a)− 1− log2 r

i.e., number of steps is a constant plus one step per bit

• Linear convergence rate: ∃C ∈ [0, 1)

|xn+1 − r| ≤ C|xn − r|, n ≥ 0

i.e., monotonic decreasing error at every step, and

|xn+1 − r| ≤ Cn+1|x0 − r|


• Bisection convergence

∗ not linear (examples?), but compared to init. max error:

∗ similar form:|xn+1 − r| ≤ Cn+1(b− a), with C = 12

Okay, but restrictive and slow.


Exercise

1. Find where the graphs of y = 3x and y = ex intersect by finding roots ofex − 3x = 0 correct to four decimal digits.

2. If a = 0.1 and b = 1.0, how many steps of the bisection method are needed todetermine the roots with an error of at most 1

2 × 10−8?

3. Find a root of the equation 6(ex− x) = 6 + 3x2 + 2x3 between −1 and 1 usingthe bisection method.


Nonlinear Equations

• Motivation


⇒ Newton’s Method

• Secant Method

• Summary


Newton’s Method

• Approximate f(x) near x0 by tangent l(x)

f(x) ≈ f(x0) + f ′(x0)(x− x0) ≡ l(x)

Want l(r) = 0⇒ r = x0 − f(x0)f ′(x0)

, ∴ x1 := r, likewise:

xn+1 = xn −f(xn)

f ′(xn)

• Alternatively (Taylor’s): have x0, for what h is

f(x0 + h︸︷︷︸≡x1

) = 0

f(x0 + h) ≈ f(x0) + hf ′(x0) or h = f(x0)f ′(x0)


Newton’s Method-Example

Intuitive, like guessing a number ∈ [0, 100].


Convergence Rate

• Theorem: With the following three conditions:

1)f(r) = 0,2) f ′(r) 6= 0,3) f ∈ C2(B(r, δ))⇒

∃δ 3 ∀x0 ∈ B(r, δ) and ∀n we have

|xn+1 − r| ≤ C(δ)|xn − r|2

∗ for a given δ, C is a constant (not necessarily < 1)

• English: With enough continuity and proximity ⇒ quadratic convergence!

• Note: again, use graphing tool to seed x0.

Newton’s Method can be very fast.


Convergence Rate Example

f(x) = x3 − 2x2 + x− 3, x0 = 4

n xn f(xn)0 4 331 3 92 2.4375 2.0368652343753 2.21303271631511 0.2563633850614184 2.17555493872149 0.006463361488813065 2.17456010066645 4.47906804996122e-066 2.17455941029331 2.15717547991101e-12

• Stopping criteria∗ theorem: uses x; above: uses f(x)-often all we have∗ possibilities: absolute/relative, size/change, x or f(x)(combos,. . .)


Sample Newton Failure 1

Runaway process



Division by zero derivative–recall algorithm



Loop-d-loop (can happen over m points)


Exercise

1. Verify that when Newton’s method is used to computed√R, the sequence of

iterates is defined by

xn+1 =1

2(xn +

R

xn)

and then (Quadratic convergence)

x2n+1 −R = [x2n −R2xn

]2

Perform the scheme for computing√2 with x0 = 1 to obtain 10−6 accuracy.

2. Establish Newton’s iterative scheme in simplified form, not involving thereciprocal of x, for the function f(x) = xR− x−1, and carry out four steps ofthis procedure using R = 4.


3. Find the root of the equation

2x(1− x2 + x) lnx = x2 − 1

in the interval [0, 1] by Newton’s method. Make a table that shows the numberof correct digits in each step.


Nonlinear Equations

• Motivation



⇒ Secant Method

• Summary


Secant Method

• Motivation: avoid derivatives

• Taylor (or derivative): f ′(xn) ≈ f(xn)−f(xn−1)xn−xn−1

• ∴ xn+1 = xn − f(xn) xn−xn−1f(xn)−f(xn−1)

• Bisection requirements comparison:∗ 2 previous point∗ f(a)f(b) < 0

• Additional advantage vs. Newton:∗ only one function evaluation per iteration

• Superlinear convergence:|xn+1 − r| ≤ C|xn − r|1.618...(recognize the exponent?)


Nonlinear Equations

• Motivation



• Secant Method

⇒ Summary


Root Finding–Summary

• Performance and requirements

f ∈ C2 nbhd(r) init.pts. � � speedyBisection × × 2

√1 ×

Newton√ √

1 × 2√

Secant ×√

2 × 1

� requirement that f(a)f(b) < 0

� function evaluations per iteration

• Often methods are combined (how?), with restarts for divergence or cycles

• Recall: use graphing tool to seed x0 (and x1)


Exercise

1. Show that in the case of a zero of multiplicity m, the modified Newton’smethod

xn+1 = xn −mf(x)

f ′(x)

is quadratically convergent. Use it solve x2 − 2x+ 1 = 0. (Hint: Use Taylorseries for each of f(r + en) and f ′(r + en))

2. Use Newton’s method solve the nonlinear system

{f(x, y) = x2 + y2 − 25 = 0g(x, y) = x2 − y − 2 = 0.


3. The following method has third-order convergence for computing√R:

xn+1 =xn(x

2n + 3R)

3x2n +R

Carry out numerical experiment for computing√10 and compare with Newton

scheme.

4. Use the Steffensen method for find the solution of the equation2x(10x2 + x) lnx− x2 + 1 = 0 in (0, 1] by the formula

xn+1 = xn −f(xn)

g(xn)

in which g(x) = {f [x+ f(x)]− f(x)}/f(x). Prove it is quadraticallyconvergent.


Nonlinear Equations Jun-Feng Yinenglish-c.tongji.edu.cn/_SiteConf/files/2014/05/07/... · Jun-Feng...

Documents

Transcript of Nonlinear Equations Jun-Feng Yinenglish-c.tongji.edu.cn/_SiteConf/files/2014/05/07/... · Jun-Feng...