Particle-Part. Part.-Mesh Ewald Method Accuracy Investigationscalkins/uofr/exammaxi.pdf ·...

1
Particle-Part. Part.-Mesh Ewald Method Accuracy Investigations Keith G. Calkins (Department of Chemistry, University of Rochester, Summer 2006 RET Program), (Keith teaches public high school at the Berrien County Math & Science Center at Andrews University) Advisor/Mentor: Harry Stern (Assistant Professor, Department of Chemistry, University of Rochester) Abstract We report the effects of mesh spacing and as- signment order on the estimated total RMS er- ror, thermodynamic properties, and timing for the molecular dynamics software program msim used by Harry Stern’s group at the University of Rochester Department of Chemistry. This pro- gram can optionally use the Ewald-sum method to substantially reduce the execution time required to handle the long-range potential. Additional sav- ings in execution time can be realized by using higher assignment orders with larger mesh spac- ing. Where this trade-off is valid is of great in- terest as computer runs fill available resources. 1E-07 1E-06 1E-05 0.0001 0.001 0.01 0.1 1 0.1 0.2 0.4 0.8 1.6 RMS Error Sum vs. Mesh and Assignment Order Mesh Spacing (A) Total RMS Error (kcal/(mol A)) Association Order and Fit Order 1 0.36*x**0.79 Order 2 6.4e-2*x**1.73 Order 3 1.7e-2*x**2.61 Order 4 6.6e-3*x**3.48 Order 5 4e-3*x**4.57 Figure 1. The Total Estimated RMS Error De- pends via Power Laws on Mesh Spacing and As- signment Order. Introduction The Stern group at the University of Rochester Department of Chemistry uses a molecular dynam- ics simulation package developed by Harry Stern which has many adjustible parameters. Some of these parameters were documented while learning to use the Mac/Unix/msim work environment. Molecular dynamics uses a computer program to mathematically model a physical system, in these cases atoms and molecules. One starts with cer- tain initial values and evolves the system in small time steps by applying the various forces to the in- teracting particles to calculate new positions and momenta. The Ewald-sum method reduces com- putation by splitting any long-range force into a short-range force done via particle-particle calcu- lations and a smoothly varying force which is ap- proximated by particle-mesh calculations. For this reason the hybrid system is known as the Particle- Particle Particle-Mesh Ewald Method. We examined how the total estimated RMS error affected the calculations of the average potential energy per molecule, the radial distribution func- tion, and the dielectric constant. Since the total estimated RMS error is strongly dependent on the mesh spacing and assignment order, we first deter- mined this dependence. Then various parameter ranges were explored further to find where prob- lems arose. Lastly, collected timing information was analyzed to help optimize future runs. The TIP4P potential model for water has been well studied and has some known limitations. However, it has the advantage of fast evaluation over more complex models. This is important when dealing with proteins in an aqueous environ- ment. The 58 amino acid residue Bovine Pancre- atic Trypsin Inhibitor (BPTI) is a small, simple, well-studied globular protein. Simulating it within an aqueous environment of 2684 water molecules is now computationally accessible without a super- computer. RMS Error Vs Mesh and Assignment Figure 1 at left shows how the total estimated RMS error (the sum of both the real and reciprocal space estimates) varies based on assignment order and mesh spacing. Apparently several options are available to obtain a total estimated RMS error of 0.001 kcal/(mol ˚ A), for instance. Specifically, for an assignment order of 2, a mesh spacing of 0.1 ˚ A is required; for an assignment order of 3, a mesh spacing of 0.4 ˚ A; for an assignment order of 4, a mesh spacing of 0.6 ˚ A; and finally, for an assignment order of 5, a mesh spacing of 0.7 ˚ A. It is estimated that with an assign- ment order of 1 a mesh spacing of 0.0001 ˚ A would be required to obtain that level of estimated error. However, the program encountered bus errors or segmentation faults due to memory allocation for mesh spacings below 0.05 ˚ A where the power law equation fit reasonably well. PE Vs RMS Error -10 -9.95 -9.9 -9.85 -9.8 -9.75 0.4 0.8 1.6 P.E. and Uncertainty vs. Mesh Spacing Mesh Spacing (A) Ave. Potential Energy (kcal/mol) Run Length 50 ps 2 ns 10 ns Exp. P.E. Figure 2. The Potential Energy per Molecule is In- sensitive to the Mesh Spacing. Figure 2 shows how potential energy per molecule varied by mesh spacing. Figure 3 presents the same data as above but the x- axis is now total estimated RMS error. -10 -9.95 -9.9 -9.85 -9.8 -9.75 1e-05 0.0001 0.001 0.01 0.1 PE and Uncertainty vs. Total Est. RMS Error Total Est. RMS Error (kcal/(mol A)) Ave. Potential Energy (kcal/mol) Run Length 50 ps 2 ns 10 ns Exp. P.E. Figure 3. Potential Energy per Molecules is Largely Independent of Estimated Total RMS Error. The experimental potential energy per molecule has been included for reference. The potential en- ergy is related to RT (the gas constant times the temperature) minus the enthalpy, under certain as- sumptions. Longer runs reduced the uncertainty but remain within 1% of the experimental value. The TIP4P model can be tuned for this. Perhaps additional tuning is indicated. This value is gener- ally only known experimentally to within 0.1% in this region. Radial Distribution Function Another interesting property of water, especially in the liquid state, is how far apart, on average the various atoms are from each other. The radial dis- tribution function (rdf) for the O-O distance was output and viewed over a wide range of mesh spac- ings and assignment orders. This property showed no dependence on these input parameters, even for absurdly large mesh spacing. Since all graphs were indistinguishable, only one typical graph is shown. 0 0.5 1 1.5 2 2.5 3 0 2 4 6 8 10 Radial Distribution Function vs Distance Distance (A) Radial Distribution Function (O-O) typical Figure 4. Typical Radial Distribution Function. Dielec. Const. Vs Mesh & RMS Error The dielectric constant for water is anomalously high. It is a measure of how a material concentrates the electrostatic lines of flux relative to a vacuum where the electric field intensity (E) and electric flux density (D) are identical. It is also tempera- ture dependent. This property is very sensitive to the model used and the length of time the dynam- ics simulation is run. 0 10 20 30 40 50 60 70 80 0.2 0.4 0.8 1.6 3.2 6.4 Dielectric Const. vs Mesh Spacing and Assign. Ord. Mesh Spacing (A) Dielectric Constant assn. ord. 2 assn. ord. 4 assn. ord. 5 2+ns ao=3 2ns ao#3 10+ns ao=3 Exp. di. const. at 25C TIP4P model: Neumann Figure 5. Convergence of the Dielectric Constant. The dielectric constant is very sensitive to mesh spacing above 1 ˚ A. The dielectric constant converged to a value close to that expected for the TIP4P model, for mesh spacings be- low 1 ˚ A. That value is significantly less than the experimental value at that temperature. Figure 6. Bovine Pancreatic Trypsin Inhibitor. Timing Dependencies The original intent was to do most simulation runs on the Mac. The sources and makefile were provided and the executable generated. Li- braries for BLAS (Basic Linear Algebra System) and LINPACK (Linear Algebra Package) as well as FFTW3, or the Fastest Fourier Transform in the West, version three, had to be obtained and configured. The Apple developer website had highly optimized LINPACK and BLAS versions and FFTW3 is available via FINK Commander un- der the GNU General Public License as published by the Free Software Foundation. The makefile as provided did not specify any optimization so many runs were done be- fore it was recompiled with the -O3 option. To model the dielectric constant, longer runs were re- quired so usage of the program on the 24 node, two 1.6 GHz Opteron processor per node, oc- curred. This gave the opportunity to not only compare a 1.6 GHz PowerPC Mac running unopti- mized and optimized versions, but also gave the opportunity to compare the POWERPC to the Opteron. Optimization provided about a 40% im- provement as did going from the PowerPC (Mac) to the Opteron (Cluster). Of course, on the cluster, multiple simultaneous runs were possible, whereas runs on the Mac were single-threaded. 10 100 1000 0.1 0.2 0.4 0.8 1.6 3.2 6.4 Run Times vs Mesh Spacing Mesh Spacing (A) Run Times (Hours) Machine/Run Length 1.6 GHz Opteron 2ns 1.6 GHz PowerPC 2ns 1.6 GHz Opteron 50ps*40 1.6 GHz PowerPC 50ps*40 1.6 GHz PowerPC unopt. Figure 7. Timing was Dependent on Mesh Spacing and Machine Type and Optimization. Figure 7 shows the time for runs with a mesh spacing above 1 ˚ A as independent of the mesh spac- ing. However, as mesh spacing decreased, the com- putation time increased. It is assumed that the Ewald-sum method combined with the use of fast fourier transforms reduces this increase from be- ing proportional to N 2 to being proportional to N log N . We use N here to refer to particle num- bers not the number of grid locations. Some con- fusion in usage in the literature is apparent. It is important to note that with higher assign- ment order the mesh spacing need not be as small for a given RMS error so run times can be reduced as indicated in Figure 8. Disclaimer Multibillionaire enrichment was minimized in the execution of this program, the writing of the final report, and the development of this poster. Free Software Foundation and TeX products were used as much as practical. Avoid doing business with monopolistic companies to reduce economic dis- parancy and global warming. Long run CP-V on a SDS/Xerox Sigma 7! Σ7 Conclusions Estimated total RMS error is dependent on mesh spacing with a power law relationship. Sig- nificantly different power laws apply for different association orders. In the TIP4P potential model for water, ra- dial distribution fuctions were independent of mesh spacing and assignment order as was the average potential energy which depended on run length. The dielectric constant was dependent on mesh spacing and required a coarseness of less than 1 ˚ A to stablize close to a value which was significantly lower than experimental but reasonably close to prior TIP4P investigations. Significant execution time savings can be real- ized by utilizing the higher assignment orders with larger mesh spacings. Figure 8 below indicates just how substantial some of these savings can be. 10 100 1000 1E-05 0.0001 0.001 0.01 0.1 Run Times vs RMS Error (Assignment Order=3) RMS Error (kcal/(mol A)) Run Times (Hours) Asgn. Ord=5 Asgn. Ord=4 Asgn. Ord=2 Machine/Run Length 1.6 GHz Opteron 2 ns 1.6 GHz PowerPC 2 ns 1.6 GHz Opteron 50 ps*40 1.6 GHz PowerPC 50 ps*40 1.6 GHz PowerPC unopt. Figure 8. Time can be Saved due to Decreased Es- timated Total RMS Error with Increased Assign- ment Order. References Allen and Tildesley, Computer Simulation of Liquids, Clarendon Press, 1987. R. W. Hockney and J. W. Eastwood, Computer Simulation Using Particles, In- stitute of Physics Publishing, 1988. Zuowei Wang and Christian Holm, Estimate of the cutoff errors in the Ewald summation for dipolar systems, Journal of Chemical Phyics, Vol. 115, No. 14, p. 6351–9, 2001. Philippe H¨ unenberger, Optimal charge-shaping functions for the particle-particle- particle-mesh (P 3 M) method for computing electrostatic interactions in molecular simulations, Journal of Chemical Physics, Vol. 113, No. 23, p. 10464–76, 2000. Martin Neumann, Dielectric relaxation in water. Computer simulations with the TIP4P potential, Journal of Chemical Physics, Vol. 85, No. 3, p. 1567–80, 1986. William L. Jorgensen, Jayaraman Chandrasekhar, and Jeffry D. Madura, Compar- ison of simple potential functions for simulating liquid water, Journal of Chemical Physics, Vol. 79, No. 2, p. 926–935, 1983. International Association for the Properties of Water and Steam, Advisory Note No. 1, Uncertainties for Enthalpy, http://ww.iapws.org, 2003. Thanks This work was made possible by the National Science Foundation in partnership with the Uni- versity of Rochester Department of Chemistry and the Research Experience for Teachers (RET) Pro- gram. The author wishes to thank his advisor/mentor, Dr. Harry Stern for his active participation in the project. His drive and encouragement propelled things forward as various nuances of the Mac/Unix world were experienced and a familiarization with the msim program developed. The author also thanks his family for their sup- port from afar during this research experience.

Transcript of Particle-Part. Part.-Mesh Ewald Method Accuracy Investigationscalkins/uofr/exammaxi.pdf ·...

Particle-Part. Part.-Mesh Ewald Method Accuracy InvestigationsKeith G. Calkins (Department of Chemistry, University of Rochester, Summer 2006 RET Program),

(Keith teaches public high school at the Berrien County Math & Science Center at Andrews University)

Advisor/Mentor: Harry Stern (Assistant Professor, Department of Chemistry, University of Rochester)

��

��Abstract

We report the effects of mesh spacing and as-signment order on the estimated total RMS er-ror, thermodynamic properties, and timing forthe molecular dynamics software program msimused by Harry Stern’s group at the University ofRochester Department of Chemistry. This pro-gram can optionally use the Ewald-sum method tosubstantially reduce the execution time required tohandle the long-range potential. Additional sav-ings in execution time can be realized by usinghigher assignment orders with larger mesh spac-ing. Where this trade-off is valid is of great in-terest as computer runs fill available resources.

1E-07

1E-06

1E-05

0.0001

0.001

0.01

0.1

1

0.1 0.2 0.4 0.8 1.6

RMS Error Sum vs. Mesh and Assignment Order

Mesh Spacing (A)

Tot

al R

MS

Err

or (

kcal

/(m

ol A

))

Association Order and FitOrder 1

0.36*x**0.79Order 2

6.4e-2*x**1.73Order 3

1.7e-2*x**2.61Order 4

6.6e-3*x**3.48Order 5

4e-3*x**4.57

Figure 1. The Total Estimated RMS Error De-pends via Power Laws on Mesh Spacing and As-signment Order.��

��Introduction

The Stern group at the University of RochesterDepartment of Chemistry uses a molecular dynam-ics simulation package developed by Harry Sternwhich has many adjustible parameters. Some ofthese parameters were documented while learningto use the Mac/Unix/msim work environment.

Molecular dynamics uses a computer program tomathematically model a physical system, in thesecases atoms and molecules. One starts with cer-tain initial values and evolves the system in smalltime steps by applying the various forces to the in-teracting particles to calculate new positions andmomenta. The Ewald-sum method reduces com-putation by splitting any long-range force into ashort-range force done via particle-particle calcu-lations and a smoothly varying force which is ap-proximated by particle-mesh calculations. For thisreason the hybrid system is known as the Particle-Particle Particle-Mesh Ewald Method.

We examined how the total estimated RMS erroraffected the calculations of the average potentialenergy per molecule, the radial distribution func-tion, and the dielectric constant. Since the totalestimated RMS error is strongly dependent on themesh spacing and assignment order, we first deter-mined this dependence. Then various parameterranges were explored further to find where prob-lems arose. Lastly, collected timing informationwas analyzed to help optimize future runs.

The TIP4P potential model for water hasbeen well studied and has some known limitations.However, it has the advantage of fast evaluationover more complex models. This is importantwhen dealing with proteins in an aqueous environ-ment. The 58 amino acid residue Bovine Pancre-atic Trypsin Inhibitor (BPTI) is a small, simple,well-studied globular protein. Simulating it withinan aqueous environment of 2684 water moleculesis now computationally accessible without a super-computer.

��RMS Error Vs Mesh and Assignment

Figure 1 at left shows how the total estimatedRMS error (the sum of both the real and reciprocalspace estimates) varies based on assignment orderand mesh spacing.Apparently several options are available to obtain

a total estimated RMS error of 0.001 kcal/(mol A),for instance. Specifically, for an assignment orderof 2, a mesh spacing of 0.1 A is required; for anassignment order of 3, a mesh spacing of 0.4 A; foran assignment order of 4, a mesh spacing of 0.6 A;and finally, for an assignment order of 5, a meshspacing of 0.7 A. It is estimated that with an assign-ment order of 1 a mesh spacing of 0.0001 A wouldbe required to obtain that level of estimated error.However, the program encountered bus errors orsegmentation faults due to memory allocation formesh spacings below 0.05 A where the power lawequation fit reasonably well.��

��PE Vs RMS Error

-10

-9.95

-9.9

-9.85

-9.8

-9.75 0.4 0.8 1.6

P.E. and Uncertainty vs. Mesh Spacing

Mesh Spacing (A)

Ave

. Pot

entia

l Ene

rgy

(kca

l/mol

)

Run Length50 ps

2 ns10 ns

Exp. P.E.

Figure 2. The Potential Energy per Molecule is In-sensitive to the Mesh Spacing.

Figure 2 shows how potential energy permolecule varied by mesh spacing. Figure 3presents the same data as above but the x-axis is now total estimated RMS error.

-10

-9.95

-9.9

-9.85

-9.8

-9.75 1e-05 0.0001 0.001 0.01 0.1

PE and Uncertainty vs. Total Est. RMS Error

Total Est. RMS Error (kcal/(mol A))

Ave

. Pot

entia

l Ene

rgy

(kca

l/mol

)

Run Length50 ps2 ns

10 ns Exp. P.E.

Figure 3. Potential Energy per Molecules is LargelyIndependent of Estimated Total RMS Error.

The experimental potential energy per moleculehas been included for reference. The potential en-ergy is related to RT (the gas constant times thetemperature) minus the enthalpy, under certain as-sumptions. Longer runs reduced the uncertaintybut remain within 1% of the experimental value.The TIP4P model can be tuned for this. Perhapsadditional tuning is indicated. This value is gener-ally only known experimentally to within 0.1% inthis region.

��

��Radial Distribution Function

Another interesting property of water, especiallyin the liquid state, is how far apart, on average thevarious atoms are from each other. The radial dis-tribution function (rdf) for the O-O distance wasoutput and viewed over a wide range of mesh spac-ings and assignment orders. This property showedno dependence on these input parameters, even forabsurdly large mesh spacing. Since all graphs wereindistinguishable, only one typical graph is shown.

0

0.5

1

1.5

2

2.5

3

0 2 4 6 8 10

Radial Distribution Function vs Distance

Distance (A)

Rad

ial D

istr

ibut

ion

Func

tion

(O-O

)

typical

Figure 4. Typical Radial Distribution Function.��

��Dielec. Const. Vs Mesh & RMS Error

The dielectric constant for water is anomalouslyhigh. It is a measure of how a material concentratesthe electrostatic lines of flux relative to a vacuumwhere the electric field intensity (E) and electricflux density (D) are identical. It is also tempera-ture dependent. This property is very sensitive tothe model used and the length of time the dynam-ics simulation is run.

0

10

20

30

40

50

60

70

80

0.2 0.4 0.8 1.6 3.2 6.4

Dielectric Const. vs Mesh Spacing and Assign. Ord.

Mesh Spacing (A)

Die

lect

ric

Con

stan

t

assn. ord. 2assn. ord. 4

assn. ord. 5

2+ns ao=32ns ao#3

10+ns ao=3Exp. di. const. at 25C

TIP4P model: Neumann

Figure 5. Convergence of the Dielectric Constant.The dielectric constant is very sensitive to

mesh spacing above 1 A. The dielectric constantconverged to a value close to that expectedfor the TIP4P model, for mesh spacings be-low 1 A. That value is significantly less thanthe experimental value at that temperature.

Figure 6. Bovine Pancreatic Trypsin Inhibitor.

��Timing Dependencies

The original intent was to do most simulationruns on the Mac. The sources and makefilewere provided and the executable generated. Li-braries for BLAS (Basic Linear Algebra System)and LINPACK (Linear Algebra Package) as wellas FFTW3, or the Fastest Fourier Transform inthe West, version three, had to be obtained andconfigured. The Apple developer website hadhighly optimized LINPACK and BLAS versionsand FFTW3 is available via FINK Commander un-der the GNU General Public License as publishedby the Free Software Foundation.

The makefile as provided did not specifyany optimization so many runs were done be-fore it was recompiled with the -O3 option. Tomodel the dielectric constant, longer runs were re-quired so usage of the program on the 24 node,two 1.6 GHz Opteron processor per node, oc-curred. This gave the opportunity to not onlycompare a 1.6 GHz PowerPC Mac running unopti-mized and optimized versions, but also gave theopportunity to compare the POWERPC to theOpteron. Optimization provided about a 40% im-provement as did going from the PowerPC (Mac)to the Opteron (Cluster). Of course, on thecluster, multiple simultaneous runs were possible,whereas runs on the Mac were single-threaded.

10

100

1000

0.1 0.2 0.4 0.8 1.6 3.2 6.4

Run Times vs Mesh Spacing

Mesh Spacing (A)

Run

Tim

es (

Hou

rs)

Machine/Run Length1.6 GHz Opteron 2ns

1.6 GHz PowerPC 2ns1.6 GHz Opteron 50ps*40

1.6 GHz PowerPC 50ps*401.6 GHz PowerPC unopt.

Figure 7. Timing was Dependent on Mesh Spacingand Machine Type and Optimization.

Figure 7 shows the time for runs with a meshspacing above 1 A as independent of the mesh spac-ing. However, as mesh spacing decreased, the com-putation time increased. It is assumed that theEwald-sum method combined with the use of fastfourier transforms reduces this increase from be-ing proportional to N 2 to being proportional toN log N . We use N here to refer to particle num-bers not the number of grid locations. Some con-fusion in usage in the literature is apparent.

It is important to note that with higher assign-ment order the mesh spacing need not be as smallfor a given RMS error so run times can be reducedas indicated in Figure 8.

��

��Disclaimer

Multibillionaire enrichment was minimized in theexecution of this program, the writing of the finalreport, and the development of this poster. FreeSoftware Foundation and TeX products were usedas much as practical. Avoid doing business withmonopolistic companies to reduce economic dis-parancy and global warming. Long run CP-V ona SDS/Xerox Sigma 7! Σ7

��

��Conclusions

• Estimated total RMS error is dependent onmesh spacing with a power law relationship. Sig-nificantly different power laws apply for differentassociation orders.• In the TIP4P potential model for water, ra-

dial distribution fuctions were independent of meshspacing and assignment order as was the averagepotential energy which depended on run length.• The dielectric constant was dependent on mesh

spacing and required a coarseness of less than 1 Ato stablize close to a value which was significantlylower than experimental but reasonably close toprior TIP4P investigations.• Significant execution time savings can be real-

ized by utilizing the higher assignment orders withlarger mesh spacings. Figure 8 below indicatesjust how substantial some of these savings can be.

10

100

1000

1E-05 0.0001 0.001 0.01 0.1

Run Times vs RMS Error (Assignment Order=3)

RMS Error (kcal/(mol A))

Run

Tim

es (

Hou

rs)

Asgn. O

rd=5

Asgn. O

rd=4

Asgn. O

rd=2

Machine/Run Length1.6 GHz Opteron 2 ns

1.6 GHz PowerPC 2 ns1.6 GHz Opteron 50 ps*40

1.6 GHz PowerPC 50 ps*401.6 GHz PowerPC unopt.

Figure 8. Time can be Saved due to Decreased Es-timated Total RMS Error with Increased Assign-ment Order.

��

��References

Allen and Tildesley, Computer Simulation of Liquids, Clarendon Press, 1987.

R. W. Hockney and J. W. Eastwood, Computer Simulation Using Particles, In-

stitute of Physics Publishing, 1988.

Zuowei Wang and Christian Holm, Estimate of the cutoff errors in the Ewald

summation for dipolar systems, Journal of Chemical Phyics, Vol. 115, No. 14, p.

6351–9, 2001.

Philippe Hunenberger, Optimal charge-shaping functions for the particle-particle-

particle-mesh (P3M) method for computing electrostatic interactions in molecular

simulations, Journal of Chemical Physics, Vol. 113, No. 23, p. 10464–76, 2000.

Martin Neumann, Dielectric relaxation in water. Computer simulations with the

TIP4P potential, Journal of Chemical Physics, Vol. 85, No. 3, p. 1567–80, 1986.

William L. Jorgensen, Jayaraman Chandrasekhar, and Jeffry D. Madura, Compar-

ison of simple potential functions for simulating liquid water, Journal of Chemical

Physics, Vol. 79, No. 2, p. 926–935, 1983.

International Association for the Properties of Water and Steam, Advisory Note

No. 1, Uncertainties for Enthalpy, http://ww.iapws.org, 2003.

��

��Thanks

This work was made possible by the NationalScience Foundation in partnership with the Uni-versity of Rochester Department of Chemistry andthe Research Experience for Teachers (RET) Pro-gram.

The author wishes to thank his advisor/mentor,Dr. Harry Stern for his active participation in theproject. His drive and encouragement propelledthings forward as various nuances of the Mac/Unixworld were experienced and a familiarization withthe msim program developed.

The author also thanks his family for their sup-port from afar during this research experience. ♥