Ab-initio Calculations ofAb-initio Calculations ofMicroscopic Structure of Nuclei Microscopic Structure of Nuclei

James P. Vary, Iowa State University

Esmond G. Ng, Berkeley Lab

Masha Sosonkina, Ames Lab

April 17, 2008

CollaboratorsCollaborators

Nuclear Physics Iowa State University

James Vary Pieter Maris (postdoc) Alina Negoita (grad student) Andrey Shirokov (international

collaborator)

Applied Math/CS Lawrence Berkeley National

Laboratory Esmond G. Ng Chao Yang Philip Sternberg (postdoc)

Ames Laboratory Masha Sosonkina Hung Viet Le (postdoc) Anurag Sharda (grad student)

MFDnMFDn

What is the science? Determine the microscopic structure of nuclei with NN + NNN

interactions using the “ab-initio” no-core shell model and full configuration interaction methods.

What does MFDn do and what are its kernels? Evaluates the many-nucleon Hamiltonian in an m-scheme harmonic

oscillator basis that eliminates spurious center of mass motion effects. Solves an eigenvalue problem to obtain the low-lying spectra and

wavefunctions, then evaluates a suite of 1-body and 2-body observables to compare with experiment.

MFDn has external field capabilities, and, when it is run in tandem with Navratil's TRDENS code, one obtains 1-body, 2-body, etc., density matrices for comparison with energy-density functionals.

MFDnMFDn

What are the major bottlenecks? Size of eigenvalue problems.

16O: matrix dimension ~ 109; # of nonzero entries ~ 21012.

Storage restrictions. Speed of execution - including per processor performance (inner loops). I/O burden with NNN interaction files.

Future directions in improving numerical convergence. Limitations of the harmonic oscillator basis.

need more general basis spaces, resulting in more compute-intensive inner loops for generating matrix elements, which requires intelligent way of exploiting matrix kernels.

m-scheme inefficiencies. need coupled J basis capability, resulting in denser matrices.

Applied Math/Comp Sci ContributionsApplied Math/Comp Sci Contributions

Eigenvalue Calculations … Drop tolerance to reduce # of nonzero entries in rank-3 Hamiltonians.

Reduced storage requirement and execution time without significantly affecting accuracies. 15N: dim ~600,000; 60% reduction in storage; 15 mins vs 28 mins on 55

processors (Thunder) to converge to the 15 smallest eigenvalues.

Integration of an alternative eigensolver. Incorporated an implicitly restarted Lanczos (PARPACK). Reduced memory usage since fewer Lanczos vectors have to be stored. Improved termination criterion.

Improving data structure for storing the Hamiltonian. Incorporated a standard sparse matrix storage scheme. Reduced integer and logical operations, at the expense of slight increase in

memory usage.

Applied Math/Comp Sci ContributionsApplied Math/Comp Sci Contributions

In generation of Hamiltonian … Parallelization of the basis generating algorithm. Development of a hierarchical blocking scheme to compute the structure

of the Hamiltonian. Exploited combinatorial properties of the index set of the basis in determining

sparsity structure by identifying zero blocks of Hamiltonian. Improved the sorting algorithm for the basis of the Hamiltonian. Potentially improving matrix-vector multiplication kernel in eigensolver

(future work).

Reducing I/O burden with NNN interaction files. Explored approximation of NNN interaction files via singular value

decomposition.

Preliminary Comparisons - Blocking StrategiesPreliminary Comparisons - Blocking Strategies

Zero blocks

Nonzero blocks

Potentially nonzero blocks

Potentially nonzero blocks in fine partitioning

6He: no blocking: ~ 43 minutes one level: 180 seconds multiple levels: 90 seconds

coarse fine fine on top of coarse

Applied Math/Comp Sci ContributionsApplied Math/Comp Sci Contributions

Explorations of “hidden” symmetries for the Hamiltonian in MFDn. Several algebraic graph reorderings, such as Reverse-Cuthill McKee,

have been applied to reduce the matrix bandwidth and search for a (approximate) block diagonal form. Observed useful block structures, pointing the way to future optimization of

matrix-vector multiplications.

Design MFDn interfaces with search programs to automate the basis space optimization especially important for external field applications. Used the master-worker paradigm of the search code (VTDIRECT95

[Virginia Tech]) and introduced a second tier of workers for expensive function evaluations such as MFDn. Found that assigning different numbers of processors to different MFDn

executions, in accordance with the Hamiltonian size, reduces overall time.

Preliminary Comparisons: Before, Intermediate, PresentPreliminary Comparisons: Before, Intermediate, Present

Future WorkFuture Work

Year 2 – Continue to improve the many-body basis setup and Hamiltonian

evaluation phases of MFDn. Year 3 –

Improve I/O performance and reducing memory requirements in MFDn. Year 4 –

Improve performance of eigenvalue calculations and overall performance of MFDn, and integrate parallel search algorithms with MFDn.

Years 3-5 – Enhance capability of MFDn, such as alternative scalable eigenvalue

solvers, algorithmic advances to enable scattering applications and coupled J basis.

DeliverablesDeliverables

Codes Integrate improved/new math/CS tools into MFDn. Explore opportunities to incorporate such tools into other ab-initio codes.

Talks/Papers P. Sternberg, C. Yang, E.G. Ng, J.P. Vary, P. Maris, “High Performance Nuclear Structure

Computation”. SIAM Conference on Parallel Processing for Scientific Computing. Atlanta (March 12-14, 2008).

P. Sternberg, C. Yang, E.G. Ng, P. Maris, J.P. Vary, “High Performance Nuclear Structure Calculations”. The 9th International Workshop on State-of-the-Art in Scientific and Parallel Computing. Trondheim, Norway (May 13-16, 2008).

M. Sosonkina, A. Sharda, A. Negoita, J.P. Vary, “Integration of Ab Initio Nuclear Physics Calculations with Optimization Techniques”. International Conference on Computational Science. Kraków, Poland (June 23-25, 2008).

P. Sternberg, P. Maris, E.G. Ng, M. Sosonkina, H.V. Le, J.P. Vary, C. Yang, “Accelerating Full Configuration Interaction Calculations for Nuclear Structure”. SC08, Austin (November 15-21, 2008). Submitted.

Possible SIAM SISC paper. Under preparation.