LMAM and School of Mathematical Sciences Peking University

24
Numerical Analysis Zhiping Li LMAM and School of Mathematical Sciences Peking University

Transcript of LMAM and School of Mathematical Sciences Peking University

Page 1: LMAM and School of Mathematical Sciences Peking University

Numerical Analysis

Zhiping Li

LMAM and School of Mathematical SciencesPeking University

Page 2: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�S��ª

��{Ä�g��´±��­, � Newton {�«O3u^L­�þü:����O­�, Cq¦)­�� x- ¶��:.

� xk−1, xk �1w¼ê f (x) ":��¥�ü�:, KLü:(xk−1, f (xk−1)), (xk , f (xk)) ���� x-¶�:��I´

xk+1 = xk −xk − xk−1

f (xk)− f (xk−1)f (xk).

ùÒ´��{�S��ª, §��±w�´3 Newton {¥ò���Ç f ′(xk) �����Ç

f (xk )−f (xk−1)xk−xk−1

����,

��{I�kü�Ð��UåÚ, Ïd¡�üÚ{. ù����ØÄ:S�{ØÓ.

2 / 24

Page 3: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�Âñ5ÚÂñ�Ý

½nµ � f (x) 3Ù": x∗ �,���S��ëY��, �f ′(x∗) 6= 0. XJÐ� x0 6= x1 ¿©�C x∗, K��{½Â�S�S� {xk+1}∞k=1 Âñu x∗, �Ù(²þ)Âñ�Ý�±����

r = 1+√

52 ≈ 1.618 �.

y²µ Äk·�F"�Ñ x∗ �����, ¦�±T��¥?Ûü�ØÓ:�Ð�d��{S��ª�Ñ�:Eá3T��¥.

d®�^�§�3 x∗ ��� ∆1 = {x : |x − x∗| ≤ δ1}, ¦�f (x) ∈ C2(∆1), f ′(x) 6= 0, ∀x ∈ ∆1. P M1 =

maxx∈∆1|f ′′(x)|

2 minx∈∆1|f ′(x)| , �

δ2 < 1/M1, - δ = min{δ1, δ2}, ∆ = {x : |x − x∗| ≤ δ}. ·��y²�� ∆ ÷v�¦. 5¿§M = maxx∈∆ |f ′′(x)|

2 minx∈∆ |f ′(x)| ≤ M1, Mδ < 1.

3 / 24

Page 4: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�Âñ5ÚÂñ�Ý�y²£Y¤

é?¿�Ð� x0, x1 ∈ ∆, P {xk}∞k=2 �d��S�{�)�S�. � xk−1, xk ∈ ∆, d x∗ ∈ ∆, f (x∗) = 0, Ú���§

P1(x) = f (xk)x − xk−1

xk − xk−1+ f (xk−1)

x − xkxk−1 − xk

(= f ��5��¼ê)�{��O (5¿§ùp x∗ Ø73 xk−1

Ú xk �m), �3 ξ1 ∈ ∆ ¦�

P1(x∗) = −1

2f ′′(ξ1)(x∗ − xk)(x∗ − xk−1).

,��¡, d���§k

P1(xk+1)−P1(x∗) =f (xk)− f (xk−1)

xk − xk−1(xk+1−x∗) = f ′(ξ2)(xk+1−x∗),

Ù¥ ξ2 3 xk−1, xk �m§Ïd ξ2 ∈ ∆.

4 / 24

Page 5: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�Âñ5ÚÂñ�Ý�y²£Y¤

5¿� P1(xk+1) = 0, ¿P ej = |xj − x∗|, Kd±þüª�

ek+1 =∣∣∣ f ′′(ξ1)

2f ′(ξ2)

∣∣∣ekek−1 ≤maxx∈∆ |f ′′(x)|2 minx∈∆ |f ′(x)|

ekek−1 ≤ (Mδ)δ < δ.

ùÒy² xk+1 ∈ ∆. dþª�k

ek+1 ≤ (Mek−1)ek ≤ (Mδ)ek ≤ · · · ≤ (Mδ)ke1 ≤ (Mδ)kδ ≤ 1

M(Mδ)k+1.

dd9 Mδ < 1 � limk→∞ xk = x∗. Âñ5�y.

��©ÛÂñ�. P E0 = Me0, E1 = Me1, - Ek+1 = EkEk−1,k = 1, 2, · · · , K8B�y Ek+1 ≤ (Mδ)Ek ≤ (Mδ)kE1, ∀k ≥ 1,Ïd, limk→∞ Ek = 0. P yk = lnEk .

5 / 24

Page 6: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�Âñ5ÚÂñ�Ý�y²£Y¤

K yk ÷v�©�§ yk+1 = yk + yk−1, k = 1, 2, · · · , ÙÏ)�L«� Fibonacci ê�yk = C1p

k1 + C2p

k2 , Ù¥ p1, p2 ´A��§

p2 = p + 1

��, p1 = 1+√

52 ≈ 1.618, p2 = 1−

√5

2 ≈ −0.618. u´k

Ek+1

Ep1k

= eC1pk+11 eC2p

k+12

eC1pk+11 eC2p

k2p1

= eC2pk2 (p2−p1). d |p2| < 1, k

limk→∞

Ek+1

Ep1

k

= 1.

dd� Ek Âñu"��Ý� p1 = 1+√

52 ≈ 1.618 ��.

dc©Û�, Mek ≤ Ek , Ïd, ek Âñu"��ÝØ$up1 ≈ 1.618 �.

6 / 24

Page 7: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�Âñ5ÚÂñ�Ý�y²£Y¤

,��¡§�Ä Ek+1 = Ek Ek−1(1 + δ), P yk = ln Ek , K yk ÷v�©�§ yk+1 = yk + yk−1 + ln(1 + δ), k = 1, 2, · · · . T�§���A)� y = − ln(1 + δ), Ïd§�§�Ï)�

yk = C1pk1 + C2p

k2 − ln(1 + δ).

u´k Ek = Ek(1 + δ)−1. q p1 + p2 = 1, ¤±

limk→∞

Ek+1

Ep1

k

= (1 + δ)−p2 .

ù`²§é?¿� δ, Ek Âñu"��Ý�´ p1.

7 / 24

Page 8: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��{

��{�Âñ5ÚÂñ�Ý�y²£Y¤

é k � 1, M∗ek+1 ≈ (M∗ek)(M∗ek−1), Ù¥ M∗ =∣∣∣ f ′′(x∗)2f ′(x∗)

∣∣∣. Ïd§zk = ln(M∗ek) ìCÂñ��A� Fibonacci ê�. Ïd§ekÂñu"��Ý� p1 ≈ 1.618 �.

5 1µ � Newton {�'§��{Ø^O��ê�§ÏdO�þ�~§¦+Âñ�Ývk����§�E,´��5�"

5 2µ ��{�g��?�Úÿ2�^L­�þn:��g­�� x-¶��:�Ñ#S�:��Ô�{§Ù(²þ)Âñ�Ý´ p3 − p2 − p − 1 = 0 ���¢� ≈ 1.84. ¦+Âñ�ÝEvk����§��Ô�{�`("):��´µ=B�Ñn�¢�Щ� x0, x1, x2, �k�US�� f (x) �E�.

8 / 24

Page 9: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��5�§|�S�){

��5�§|�¦)— ¯K!(J�]

�Ĺ n > 1 ���þ!n ��§��§|µf1(x1, x2, · · · , xn) = 0,

f2(x1, x2, · · · , xn) = 0,

· · · · · · · · · · · ·fn(x1, x2, · · · , xn) = 0,

Ù¥ fi : Rn → R, i = 1, 2, · · · , n, ¥��k��´��5�.

(J: ØO�þìO�§�æ��´1 "y°()�êÆnØ (�35!��5!· · · );

2 Nõ���¦)g�!�nÚ�{Ã{E��õ�;

] :1 ÛÜ�5z�{!¦)�5�§|�S�{;

2 Ø N��nÚØÄ:S����êÃ'�g���{.

9 / 24

Page 10: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��5�§|�S�){

��5 Jacobi S�

�¦)�5�§|� Jacobi S�{aq/, 31 i fÚ§3�½ xj , j 6= i �^�e, ò fi �� xi ���5¼ê¦�; é¤k1 ≤ i ≤ n, ѦÑ���§�å�# {xi}ni=1. �{Xeµ

for k = 0, 1, 2, · · ·for i = 1, 2, · · · , n) fi (x

(k)1 , · · · , x (k)

i−1, u, x(k)i+1, · · · , x

(k)n ) = 0 � u;

x(k+1)i = u;

endXJ÷vS�Ê�^�§Kª�Ì�¿ÑÑ x (k+1)

end

10 / 24

Page 11: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��5�§|�S�){

��5 Gauss-Seidel S�

�¦)�5�§|� Gauss-Seidel S�{aq/, 31 i fÚ§3�½ xj , j 6= i �^�e, ò fi �� xi ���5¼ê¦�, ¦���á=^Ù�# xi . �{Xeµ

for k = 0, 1, 2, · · ·for i = 1, 2, · · · , n) fi (x

(k+1)1 , · · · , x (k+1)

i−1 , u, x(k)i+1, · · · , x

(k)n ) = 0 � u;

x(k+1)i = u;

endXJ÷vS�Ê�^�§Kª�Ì�¿ÑÑ x (k+1)

end

11 / 24

Page 12: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

��5�§|�S�){

��5 SOR S�

�¦)�5�§|� SOR S�{aq/, 31 i fÚ§3�½xj , j 6= i �^�e, ò fi �� xi ���5¼ê¦�, ¦���á=^Ù(ÜtµÏf ω �# xi . �{Xeµ

for k = 0, 1, 2, · · ·for i = 1, 2, · · · , n) fi (x

(k+1)1 , · · · , x (k+1)

i−1 , u, x(k)i+1, · · · , x

(k)n ) = 0 � u;

x(k+1)i = x

(k)i + ω(u − x

(k)i );

endXJ÷vS�Ê�^�§Kª�Ì�¿ÑÑ x (k+1)

end

5µ ù�{3�½^�eUÂñ§���Âñ�ú.

12 / 24

Page 13: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

Newton S�{9ÙU?.�{

ÛÜ�5z���5�§|� Newton S�{

� f : Rn → Rn �1w¼ê�, d Taylor Ðmªk

f(x∗) = f(x(k)) + ∇f(x(k))(x∗ − x(k)) + O(‖x∗ − x(k)‖2),

Ù¥ ∇f(x(k)) =(∂fi (x

(k))

∂x(k)

)´ f � Jacobi Ý.

dd£¡�ÛÜ�5z¤=�Cq�5�§|

∇f(x(k))(x∗ − x(k)) ≈ −f(x(k)).

½Â��5�§|� Newton S�S�µé k = 0, 1, · · ·1 )�5�ê�§| ∇f(x(k))y(k) = −f(x(k));

2 - x(k+1) = x(k) + y(k).

¢SO��, ��±Ú\�5|¢ minλ ‖f(x(k) + λy(k))‖.

13 / 24

Page 14: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

Newton S�{9ÙU?.�{

Newton S�{�C/ Broyden �{

����/aq§Newton S�{���kÛÜÂñ5§é1w¼ê�ü�§Âñ�Ý����;

zÚÑIO� Jacobi Ý, $�þã�, cÙ´p��/.

XÛ;�O� Jacobi Ý, ��±$�þ��CqO�´U? Newton S�{�Ä�Ñu:.

Broyden �{3 ∇f(x(0))−1 �Ä:þ§zÚ�I�é�AgÝ��þ¦{=¼�e�ÚS�¤I�Cq_Ý, l ^é���d¼�S�?��þ y(k).

14 / 24

Page 15: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

Newton S�{9ÙU?.�{

Broyden �{�nØ| — Sherman-Morrison Ún

Únµ � A � n ��_Ý, x, y ∈ Rn. XJyTA−1x 6= −1,KA + xyT ��_, �

(A + xyT )−1 = A−1 − A−1xyTA−1

1 + yTA−1x.

� y(k−1) = x(k) − x(k−1) é��, Cq/k

∇f(x(k))y(k−1) = ∇f(x(k))(x(k)−x(k−1)) ≈ f(x(k))−f(x(k−1)) =: g(k−1).

·�F"é�UCq�O ∇f(x(k)) �Ý A(k), ¦�A(k)y(k−1) = g(k−1), � A(k) = A(k−1) + u(k−1)(y(k−1))T , Ù¥ u(k−1) �½.

15 / 24

Page 16: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

Newton S�{9ÙU?.�{

Broyden �{�g�9�EL§

d^��

g(k−1) = A(k)y(k−1) = A(k−1)y(k−1) + u(k−1)(y(k−1))Ty(k−1).

dd� u(k−1) = g(k−1)−A(k−1)y(k−1)

(y(k−1))T y(k−1) , u´

A(k) = A(k−1) +g(k−1) − A(k−1)y(k−1)

(y(k−1))Ty(k−1)(y(k−1))T .

ØJ�yÚn�^� (y(k−1))T (A(k−1))−1u(k−1) 6= −1 �du (y(k−1))T (A(k−1))−1g(k−1) 6= 0 (Ø�b�o¤á).

16 / 24

Page 17: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

Newton S�{9ÙU?.�{

Broyden �{�g�9�EL§

- A(0) = ∇f(x(0)), � (A(0))−1 ®�. dÚn 4.3.1 �

(A(k))−1

= (A(k−1))−1−[(A(k−1))−1g(k−1)−y(k−1)

](y(k−1))T (A(k−1))−1

(y(k−1))T (A(k−1))−1g(k−1).

k (A(k))−1, Ò�±O�1 k Ú�S�?��þ

y(k) = (A(k))−1f (x(k)).

5 1µ ùÒ´ Broyden �{�Ä�Ú½. dd��§ Broyden�{�O�þ�(é�. �±y² Broyden �{, � f ÷v�½^��, ´ÛÜ��5Âñ�.

5 2µ U?.� Newton S�{�kNõ. ��Ñäk�½^�e�ÛÜ��5Âñ5. éJ'�§��`�.

17 / 24

Page 18: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

����{{0— ÓÔ�{

ÓÔ�{— ���Âñ�{��«kÃ}Á

±þ0���«��5�§Ú��5�§|�S��{Ñ�kÛÜÂñ5§=���Âñ�S�S�§7L�Ñl) x∗ ¿©�C�Ð� x0, ù3¢SA^¥ Ø´��. Ï~�³/²�!�E}Á!±9N$í.

é,AÏa.���5�§½�§|§�±�Eäk�ÛÂñ5��{. ÓÔ{£homotopy¤, �¡�òÿ{£continuation¤,Ò´ù�¡�«kÃ�}Á.

18 / 24

Page 19: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

����{{0— ÓÔ�{

ÓÔ�{�Ä�g�

�¦) F(x) = 0, ��ЩXÚ G(x) = 0, Ù�®�� x0.

½ÂÓÔ¼êH(x, λ) = λF(x) + (1− λ)G(x).

w,k H(x, 0) = G(x), H(x, 1) = F(x).

·�F"H(x, λ) = 0 k) xλ, �§ëY/�6uëêλ ∈ [0, 1].

ù�, �� 0 = λ0 < λ1 < · · · < λn = 1, ¦� λi+1 − λi ¿©�,xi , xλi Ò�±�� xi+1 , xλi+1

�v�S��.

^ÛÜÂñ�S�{�gCq¦) xi , i = 1, 2, · · · , n, �ª��F(x) = 0 �) xn.

19 / 24

Page 20: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

����{{0— ÓÔ�{

ÓÔ�{�¢y

Ú 1: � G(x) = 0 ��®�� x0;Ú 2: é i = 1, 2, · · · , n, ± xi−1 �Ð�,

^ Newton {¦)��5�§| H(x, λi ) = 0,��Cq) xi ;

Ú 3: ��� xn =�¤¦.

5 1µ Ú� λi − λi−1 �>�>½§=g·A�N�.

5 2µ Newton {�±��Ù§?Û�«ÛÜÂñ�S�{.

5 3µ é i = 1, 2, · · · , n − 1, Ø7�Ñp°Ý�ê�). 3Ú�À�·���¹e, xi �°ÝU��

15 |xi+1 − xi | Òv.

5 4µ ý��]Ô5gu G(x) �À�.

20 / 24

Page 21: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

����{{0— ÓÔ�{

ÓÔ�{¢yL§¥�UÑy��¹

éuØÓ�ÓÔ¼ê H(x, λ) ÓÔ�{�U¬�ѱeA«ØÓ�(Jµ

1 �{�l�^n��ÓÔ­�§�ª�ÑF(x) = 0 ����.

2 �{3,� λ0 ?uѧ= limλ→λ0−0 ‖xλ‖ =∞.

3 �{3,� λ0 ?Ñy=ò:, d�, ¤¦�� xλ0 Ø3?Û xλ, λ > λ0, �vC���¥.

4 �{3,� λ0 ?Ñy©�, d�, ¤¦�� xλ0 Ø3?Û­½� xλ, λ > λ0, �vC���¥.

3���/ (2), (3), (4) �§�l��¬�}§d�I���ÓÔ¼ê.

21 / 24

Page 22: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

����{{0— ÓÔ�{

�{ü�ÓÔ¼ê�{

1 H(x, λ) = F(x) + (λ− 1)F(x0).

2 H(x, λ) = λF(x) + (1− λ)A(x− x0), Ù¥A ��_Ý.

5µ ��5`§ÓÔ¼ê�À�vk�½�5. ØL§éõ�ª¼ê®²/¤�@k���{£ë�ë�©z[26]¤.

22 / 24

Page 23: LMAM and School of Mathematical Sciences Peking University

Lecture 11: Numerical Solution for Nonlinear Equations

��5�§�S�){

����{{0— ÓÔ�{

��5�§|¦���`z�{

�¦) F(x) = 0, - G (x) = F(x)TF(x). K¦�¯K�du¦¼ê G (x) ����:. Ïd§�A^�`z�nØÚ�{¦).~^��{k

1 ��eü{�FÝ.�{§ÛÜ�5z�{.

2 Úî{!&6�{§ÛÜ�g%C��{.

3 �[ò»{.

4 · · · · · · · · ·

23 / 24

Page 24: LMAM and School of Mathematical Sciences Peking University

SKoµ3, 6, 8; þÅSKoµ3, 4, 5 (2), (5).

Thank You!