Newtons Method Notes -...
Transcript of Newtons Method Notes -...
Local Op)miza)on: Newton’s Method
Classes of Op)miza)on Method There are two class of local op)miza)on methods: 1. First-‐order methods
1. Only u)lizes first deriva)ves 2. Gradient Descent is an example of a first order
method
2. Second-‐order methods 1. U)lizes first and second deriva)ves 2. Newton’s method is a second-‐order method First we will go over the equa)ons for the
Newton’s Method local op)miza)on algorithm and compare them to Gradient Descent
Review: Algorithm for Gradient Descent
The general procedure for numerical geometry op)miza)on is as follows: 1. Calculate the force on all atoms for some
configura)on of an atomic system. 2. If the magnitude of the force is less than
threshold, you have found a cri)cal point! STOP. 3. If not, move the atoms such that they go
towards a cri)cal points 4. Repeat.
rn+1 = rn +αF(rn )
Algorithm for Newton’s Method The general procedure for numerical geometry op)miza)on is as follows: 1. Calculate the force on all atoms for some
configura)on of an atomic system. 2. If the magnitude of the force is less than
threshold, you have found a cri)cal point! STOP. 3. If not, move the atoms such that they go
towards a cri)cal points. In NM you will need to calculate second deriva4ves, V’’, as well.
4. Repeat.
rn+1 = rn +F(rn )V ''(rn )
Newton’s Method (NM)
• So far, I have just shown you the equa)ons for Newton’s Method.
• I will show you the deriva)on of NM in a few slides, but let’s start with the same example given in class with gradient descent to see how this algorithm works…
Example Newton’s Method
Step #, n rn F(rn) rn+1=rn+F(rn)/V’’ (rn) 1 1 2 3
V(r)
r
Start with a ini)al guess of the local minimum:
r=1
V (r) = (r −3)2
r1=1
Example Newton’s Method
Step #, n rn F(rn) rn+1=rn+F(rn)/V’’ (rn) 1 1 4 2 3
V(r)
r
1. Calculate the force on all atoms for some configura)on of an
atomic system.
V (r) = (r −3)2
!F(r) = − d
drV (r) = −2(r −3)
!F(1) = −2(1−3) = 4
r1=1
Example Newton’s Method
Step #, n rn F(rn) rn+1=rn+F(rn)/V’’ (rn) 1 1 4 2 3
V(r)
r
V (r) = (r −3)2 2. If the magnitude of the force is less than threshold (0.01eV/Angstrom), you have found a
cri)cal point! STOP. !F(1) = 4!F(1) = 4*4 = 4
4 < 0.01− > False
!F(r1)r1=1
Example Newton’s Method
Step #, n rn F(rn) rn+1=rn+F(rn)/V’’ (rn) 1 1 4 2 3
V(r)
r
3. If not, move the atoms such that they go towards a cri)cal points. First calculate V’’
V (r) = (r −3)2
rn+1 = rn +αF(rn )V ''(rn )
V ''(r) = 2
r1=1
Example Newton’s Method
Step #, n rn F(rn) rn+1=rn+F(rn)/V’’ (rn) 1 1 4 3 2 3 3
V(r)
r
3. If not, move the atoms such that they go towards a cri)cal
points.
V (r) = (r −3)2
rn+1 = rn +F(rn )V ''(rn )
r2 = r1 +F(rn )V ''(rn )
=1+ 42= 3
r1=1
r2=3
Example Newton’s Method
Step #, n rn F(rn) rn+1=rn+F(rn)/V’’ (rn) 1 1 4 3 2 3 3
V(r)
r
3. If not, move the atoms such that they go towards a cri)cal
points.
V (r) = (r −3)2
r2 = r1 +F(rn )V ''(rn )
=1+ 42= 3
r1=1
r2=3 You already found the cri)cal point in only one step!
Newton’s Method (NM) • Here is why:
– NM assumes that a second order Taylor expansion is accurate around your current posi)on:
– Then a near-‐by cri)cal point is approximated by taking the deriva)ve of Vapprox , seang it equal to zero, and solving for r
– Below is the equa)on found when solving for r in the equa)on above. No)ce that this is the same as the Newton’s Method equa)ons introduced earlier. We just derived NM! (If you want to see all the algebra, check out the last slide!).
Vapprox (r) =V (r0 )+V '(r0 )(r − r0 )+12V ''(r0 )(r − r0 )
2
Vapprox '(r) =V '(r0 )+V ''(r0 )(r − r0 ) = 0
rn+1 = rn +F(rn )V ''(rn )
Newton’s Method algorithm!
r = r0 −V '(r0 )V ''(r0 )
Newton’s Method
• Since a second order Taylor expansion works perfectly for a parabola, NM works in one step!
• In the extra credit assignment, you will implement Newton’s method and apply it to the L-‐J poten)al (it will not be as simple as the example shown in these slides!)
• You will explore some of the issues that come up in Newton’s Method and start to think about how we can address these problems.
Newton’s Method (NM) • Here is why:
– NM assumes that a second order Taylor expansion is accurate around your current posi)on:
– Then a near-‐by cri)cal point is approximated by taking the deriva)ve of Vapprox , seang it equal to zero, and solving for r
Vapprox (r) =V (r0 )+V '(r0 )(r − r0 )+12V ''(r0 )(r − r0 )
2
Vapprox '(r) =V '(r0 )+V ''(r0 )(r − r0 ) = 0V '(r0 )+V ''(r0 )(r − r0 ) = 0V ''(r0 )(r − r0 ) = −V '(r0 )
r − r0 =−V '(r0 )V ''(r0 )
r = r0 +F(r0 )V ''(r0 )
rn+1 = rn +F(rn )V ''(rn )
Newton’s Method algorithm!