Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire,...
-
Upload
tara-cozzens -
Category
Documents
-
view
220 -
download
3
Transcript of Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire,...
![Page 1: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/1.jpg)
Modified Newton Methods
Lecture 9
Alessandra Nardi
Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy
![Page 2: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/2.jpg)
Last lecture review
• Solving nonlinear systems of equations– SPICE DC Analysis
• Newton’s Method– Derivation of Newton
– Quadratic Convergence
– Examples
– Convergence Testing
• Multidimensonal Newton Method– Basic Algorithm
– Quadratic convergence
– Application to circuits
![Page 3: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/3.jpg)
0 Initial Guess,= 0x k
Repeat {
1 1Solve for k k k k kFJ x x x F x x
} Until 11 , small en ug o hkk kx x f x
1k k
Compute ,k kFF x J x
Multidimensional Newton MethodAlgorithm
Newton Algorithm for Solving 0F x
![Page 4: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/4.jpg)
If
1) Inverse is boundedkFa J x
) Derivative is Lipschitz ContF Fb J x J y x y
Then Newton’s method converges given a sufficiently close initial guess (and
convergence is quadratic)
Multidimensional Newton Method Convergence
Local Convergence Theorem
![Page 5: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/5.jpg)
Last lecture review
• Applying NR to the system of equations we find that at iteration k+1:– all the coefficients of KCL, KVL and of BCE of
the linear elements remain unchanged with respect to iteration k
– Nonlinear elements are represented by a linearization of BCE around iteration k
This system of equations can be interpreted as the STA of a linear circuit (companion network) whose elements are specified by the linearized BCE.
![Page 6: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/6.jpg)
Note: G0 and Id depend on the iteration count k G0=G0(k) and Id=Id(k)
Application of NR to Circuit EquationsCompanion Network – MNA templates
![Page 7: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/7.jpg)
Application of NR to Circuit EquationsCompanion Network – MNA templates
![Page 8: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/8.jpg)
Modeling a MOSFET(MOS Level 1, linear regime)
d
![Page 9: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/9.jpg)
Modeling a MOSFET(MOS Level 1, linear regime)
![Page 10: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/10.jpg)
Last lecture review
• Multidimensional Case: each step of iteration implies solving a system of linear equations
• Linearizing the circuit leads to a matrix whose structure does not change from iteration to iteration: only the values of the companion circuits (the nonlinear elements) are changing.
![Page 11: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/11.jpg)
DC Analysis Flow DiagramFor each state variable in the system
![Page 12: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/12.jpg)
Implications• Device model equations must be continuous
with continuous derivatives (not all models do this - - be sure models are decent - beware of user-supplied models)
• Watch out for floating nodes (If a node becomes disconnected, then J(x) is singular)
• Give good initial guess for x(0)
• Most model computations produce errors in function values and derivatives. Want to have convergence criteria || x(k+1) - x(k) || < such that > than model errors.
![Page 13: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/13.jpg)
Improving convergence
• Improve Models (80% of problems)
• Improve Algorithms (20% of problems)
Focus on new algorithms:Limiting Schemes
Continuations Schemes
![Page 14: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/14.jpg)
Outline
• Limiting Schemes– Direction Corrupting– Non corrupting (Damped Newton)
• Globally Convergent if Jacobian is Nonsingular
• Difficulty with Singular Jacobians
• Continuation Schemes– Source stepping– More General Continuation Scheme– Improving Efficiency
• Better first guess for each continuation step
![Page 15: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/15.jpg)
At a local minimum, 0f
x
LocalMinimum
Multidimensional Case: is singularFJ x
Multidimensional Newton MethodConvergence Problems – Local Minimum
![Page 16: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/16.jpg)
f(x)
X0x
1x
Must Somehow Limit the changes in X
Multidimensional Newton MethodConvergence Problems – Nearly singular
![Page 17: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/17.jpg)
Multidimensional Newton MethodConvergence Problems - Overflow
f(x)
X0x 1x
Must Somehow Limit the changes in X
![Page 18: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/18.jpg)
0 Initial Guess,= 0x k
Repeat {
1 1Solve for k k k kFJ x x F x x
} Until 1 1 , small enoughkkx F x 1k k
Compute ,k kFF x J x
Newton Algorithm for Solving 0F x
1 1limitedk k kx x x
Newton Method with Limiting
![Page 19: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/19.jpg)
1limited k
ix
• NonCorrupting
1 1 if k ki ix x
1 otherwisekisign x
1kx 1limited kx
• Direction Corrupting
1 1limited k kx x
1min 1,
kx
1kx 1limited kx
Heuristics, No Guarantee of Global Convergence
Newton Method with LimitingLimiting Methods
![Page 20: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/20.jpg)
General Damping Scheme
Key Idea: Line Search
21
2Pick to minimize k k k kF x x
Method Performs a one-dimensional search in Newton Direction
21 1 1
2
Tk k k k k k k k kF x x F x x F x x
1 1k k k kx x x
1 1Solve for k k k kFJ x x F x x
Newton Method with LimitingDamped Newton Scheme
![Page 21: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/21.jpg)
If
1) Inverse is boundedkFa J x
) Derivative is Lipschitz ContF Fb J x J y x y
Then
There exists a set of ' 0,1 such thatk s
1 1 with <1k k k k kF x F x x F x
Every Step reduces F-- Global Convergence!
Newton Method with LimitingDamped Newton – Convergence Theorem
![Page 22: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/22.jpg)
0 Initial Guess,= 0x k Repeat {
1 1Solve for k k k kFJ x x F x x
} Until 1 1 , small enoughkkx F x 1k k
Compute ,k kFF x J x
1Find 0,1 such that is minimi e z dk k kk F x x 1 1k k k kx x x
Newton Method with LimitingDamped Newton – Nested Iteration
![Page 23: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/23.jpg)
X0x1x
2x1Dx
Damped Newton Methods “push” iterates to local minimumsFinds the points where Jacobian is Singular
Newton Method with LimitingDamped Newton – Singular Jacobian Problem
![Page 24: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/24.jpg)
Solve , 0 where:F x
a) 0 ,0 0 is easy to solveF x
b) 1 ,1F x F x
c) is sufficiently smoothx
Starts the continuation
Ends the continuation
Hard to insure!
Newton with Continuation schemes Basic Concepts - General setting
Newton converges given a close initial guess
Idea: Generate a sequence of problems, s.t. a problem is a good initial guess for the following one
0)(xF
![Page 25: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/25.jpg)
Solve 0 ,0 , 0prevF x x x
While 1 {
}
=0.01,
0prevx x
Try to Solve , 0 with NewtonF x
If Newton Converged
, 2,prevx x Else
,1
2 prev
Newton with Continuation schemes Basic Concepts – Template Algorithm
![Page 26: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/26.jpg)
Newton with Continuation schemes Basic Concepts – Source Stepping Example
![Page 27: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/27.jpg)
+-Vs
R
Diode
1, 0diode sf v i v v V
R
v
, 1Not dependent!diodef v i v
v v R
Source Stepping Does Not Alter Jacobian
Newton with Continuation schemes Basic Concepts – Source Stepping Example
![Page 28: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/28.jpg)
0 ,0= 0 00 F x x Observations
0 ,0F xI
x
=1 1 ,1 1F x F x
0 ,0 1F x F x
x x
Problem is easy to solve and Jacobian definitely
nonsingular.
Back to the original problem and original Jacobian
, 1F x F x x
Newton with Continuation schemes Jacobian Altering Scheme
![Page 29: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/29.jpg)
Solve 0 ,0 , 0prevF x x x
While 1 {
}
=0.01,
0 ?prevx x
Try to Solve , 0 with NewtonF x
If Newton Converged
, 2,prevx x Else
,1
2 prev
Newton with Continuation schemes Jacobian Altering Scheme – Basic Algorithm
![Page 30: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/30.jpg)
,F x ,F x
,F xx x
x
,F x
0
0,, xF x
x xF
x
Have From last step’s Newton
Better Guess for next step’s
Newton
Newton with Continuation schemes Jacobian Altering Scheme – Update Improvement
![Page 31: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/31.jpg)
, 1F x F x x
Easily Computed
, F x
F x x
If
Then
Newton with Continuation schemes Jacobian Altering Scheme – Update Improvement
![Page 32: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/32.jpg)
1
0, ,F x
xx
xx
F
x
0 1
0x
Graphically
Newton with Continuation schemes Jacobian Altering Scheme – Update Improvement
![Page 33: Modified Newton Methods Lecture 9 Alessandra Nardi Thanks to Prof. Jacob White, Jaime Peraire, Michal Rewienski, and Karen Veroy.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649cb85503460f9497f594/html5/thumbnails/33.jpg)
Summary• Newton’s Method works fine:
– given a close enough initial guess
• In case Newton does not converge:– Limiting Schemes
• Direction Corrupting
• Non corrupting (Damped Newton)– Globally Convergent if Jacobian is Nonsingular
– Difficulty with Singular Jacobians
– Continuation Schemes• Source stepping
• More General Continuation Scheme
• Improving Efficiency– Better first guess for each continuation step