MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.
-
Upload
piers-weaver -
Category
Documents
-
view
218 -
download
2
Transcript of MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.
![Page 1: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/1.jpg)
MATH 685/ CSI 700/ OR 682 Lecture Notes
Lecture 8.
Nonlinear equations
![Page 2: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/2.jpg)
Nonlinear Equations Given a function f, we are looking for a value x, s.t.
f(x)=0
(a root of the equation, or a zero of the function f). The problem is called root finding or zero finding.
![Page 3: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/3.jpg)
Example
![Page 4: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/4.jpg)
Existence/uniqueness
![Page 5: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/5.jpg)
Examples in 1d
![Page 6: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/6.jpg)
Example of a system in 2d
![Page 7: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/7.jpg)
Multiplicity
![Page 8: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/8.jpg)
Sensitivity and conditioning
![Page 9: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/9.jpg)
Sensitivity and conditioning
![Page 10: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/10.jpg)
Sensitivity and conditioning
![Page 11: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/11.jpg)
Convergence rate
![Page 12: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/12.jpg)
Convergence rate
![Page 13: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/13.jpg)
Bisection method
![Page 14: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/14.jpg)
Example: bisection iteration
![Page 15: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/15.jpg)
Bisection method
![Page 16: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/16.jpg)
Fixed-point iterations
![Page 17: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/17.jpg)
Examples
![Page 18: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/18.jpg)
Example: fixed point problems
![Page 19: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/19.jpg)
Examples: FPI
![Page 20: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/20.jpg)
Example: FPI
![Page 21: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/21.jpg)
Convergence of FPI
![Page 22: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/22.jpg)
Newton’s method
![Page 23: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/23.jpg)
Newton’s method
![Page 24: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/24.jpg)
Newton’s method
![Page 25: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/25.jpg)
Convergence of Newton’s method
![Page 26: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/26.jpg)
Newton’s method
![Page 27: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/27.jpg)
Secant method
![Page 28: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/28.jpg)
Secant method
![Page 29: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/29.jpg)
Example
![Page 30: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/30.jpg)
Higher-degree interpolation
![Page 31: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/31.jpg)
Inverse interpolation
![Page 32: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/32.jpg)
Inverse quadratic interpolation
![Page 33: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/33.jpg)
Example
![Page 34: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/34.jpg)
Linear fractional interpolation
![Page 35: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/35.jpg)
Example
![Page 36: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/36.jpg)
Safeguarded methods Rapidly convergent methods for solving nonlinear equations may
not converge unless started close to solution, but safe methods are slow
Hybrid methods combine features of both types of methods to achieve both speed and reliability
Use rapidly convergent method, but maintain bracket around solution
If next approximate solution given by fast method falls outside bracketing interval, perform one iteration of safe method, such as bisection
Fast method can then be tried again on smaller interval with greater chance of success
Ultimately, convergence rate of fast method should prevail Hybrid approach seldom does worse than safe method, and
usually does much better Popular combination is bisection and inverse quadratic
interpolation, for which no derivatives required
![Page 37: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/37.jpg)
Zeros of polynomials
![Page 38: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/38.jpg)
Systems of nonlinear equations
Solving systems of nonlinear equations is much more difficult than scalar case because:
Wider variety of behavior is possible, so determining existence and number of solutions or good starting guess is much more complex
There is no simple way, in general, to guarantee convergence to desired solution or to bracket solution to produce absolutely safe method
Computational overhead increases rapidly with dimension of problem
![Page 39: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/39.jpg)
Fixed-point iteration (FPI)
![Page 40: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/40.jpg)
Newton’s method
![Page 41: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/41.jpg)
Example
![Page 42: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/42.jpg)
Example
![Page 43: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/43.jpg)
Convergence of Newton’s method
![Page 44: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/44.jpg)
Cost of Newton’s method
![Page 45: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/45.jpg)
Secant updating methods
![Page 46: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/46.jpg)
Broyden’s method
![Page 47: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/47.jpg)
Broyden’s method
![Page 48: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/48.jpg)
Example
![Page 49: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/49.jpg)
Example
![Page 50: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/50.jpg)
Example
![Page 51: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/51.jpg)
Example (cont)
![Page 52: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/52.jpg)
Robust Newton-like methods
![Page 53: MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 8. Nonlinear equations.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649de75503460f94ae1676/html5/thumbnails/53.jpg)
Trust-region methods