Nuclear Physics at the Interfaceof 21st Century Computational Science

Michael Strayer, Assoc. Dir. Of Science for Advanced Scientific Computing Research

April 2009

Computational ScienceThe Early Years

Wilson’s Analogy:Simulation ScientistsMath/CS Researchers Computers

ExperimentalistsTheoristsExperimental Apparatus

“Perhaps the most significant applications of scientific computing come not in the solution of old problems, but in the discovery of new phenomena through numerical experimentation.”

Computational ScienceThe Early Years

Lax report on Large Scale Computing in Science and Engineering, 1982

Computational ScienceThe Early Years

Lax Report on Large Scale Computing in Science and Engineering, 1982

It is in the national interest– that access to constantly updated supercomputing facilities be provided to

scientific and engineering researchers and– that a large and imaginative user community be trained in their uses and

capabilities.

Future significant improvements may have to come from architectures embodying parallel processing elements –perhaps several thousands of processors.

Research in languages, algorithms and numerical analysis will be crucial in learning to exploit these new architectures fully.

Computational Science20 years later

Scientific Discovery through Advanced Computing (SciDAC)

““SciDAC is unique in the world. There isn't any other SciDAC is unique in the world. There isn't any other program like it anywhere else, and it has the program like it anywhere else, and it has the remarkable ability to do science by bringing remarkable ability to do science by bringing together physical scientists, mathematicians, applied together physical scientists, mathematicians, applied mathematicians, and computer scientists who mathematicians, and computer scientists who recognize that computation is not something you do recognize that computation is not something you do at the end, but rather it needs to be built into the at the end, but rather it needs to be built into the solution of the very problem that one is addressing.solution of the very problem that one is addressing.””

Dr Raymond Orbach, then Under Secretary for Science and Director, Office of Science, US Department of Energy

in SciDAC Review

Computational Science:the Third Pillar

Ken Wilson

ExperimentTheory

Advancing Science through large-scale data, modeling and simulation

– Science Application and Science Applications Partnerships: Astrophysics, Accelerator Science, Climate, Biology, Fusion, Petabyte data, Materials & Chemistry, Nuclear physics, High Energy physics, QCD, Turbulence, Groundwater

– Centers for Enabling Technology: Address mathematical and computing systems software issues

– Institutes: Assist Scientific Applications teams and foster next generation computational scientists

Scientific Discovery Scientific Discovery through Advanced Computing (SciDAC)through Advanced Computing (SciDAC)

http://www.scidac.gov

FacilitiesThen and Now

1984 2009

Nuclear Physics2.8%

Climate Research11.1%

Chemical Sciences6.3%

Astrophysics17.8%

Atomic/Molecular Physics0.1%

Combustion6.1%

Plasma Physics9.8%

Applied Mathematics3.4%

Biological Sciences11.2%

Engineering2.3%

Computer Sciences2.7%

Geosciences1.2%

Environmental Sciences0.6%

Lattice Gauge Theory9.8%

Materials Sciences12.4%

Nuclear Energy0.8%

Accelerator Physics1.5%

Fluid Turbulence0.1%

2009 INCITE projects

Innovative and Novel Computational Impact on Theory and Experiment

Approximately 890 million processors awarded in 2009

Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program started in 2004.• Small number of computationally intense, high impact projects• Open to national and international researchers, including industry• No requirement of DOE or Office of Science funding on topic area• Peer and computational reviews

INCITE TrendsRequests for allocations continue to outpace available resources

Approximately 1.3 Billion Processor hours available for INCITE in 2010

Delivering the Science

Scientific Discovery and the Role of High End Computing

Panelist Area Pete Beckman, Argonne National Laboratory

Computer Science

Jacqueline Chen, Sandia National Laboratories

Accelerator Physics, Combustion, Fluid Turbulence, Engineering Physics

Giulia Galli, University of California-Davis

Chemical Sciences, Nano and Materials Sciences, Physical Chemistry

James Hack, Oak Ridge National Laboratory

Climate Research, Environmental Sciences, Geosciences

David Keyes, Columbia Applied Mathematics Doug Kothe, Oak Ridge National Laboratory

Computational Fluid Dynamics, Nuclear Engineering

Paul Messina, Argonne National Laboratory

Computer Science

Anthony Mezzacappa, panel chair, Oak Ridge National Laboratory

Astrophysics, Solar/Space Physics

Claudio Rebbi, Boston University

High Energy and Nuclear Physics

Nagiza Samatova, North Carolina State University

Biology, Life Sciences

Tang, Princeton Plasma Physics Laboratory

Fusion, Fusion, Energy, Plasma Physics

Katherine Yelick, Lawrence Berkeley National Laboratory

Computer Science

Breakthroughs Selection Panel

Top 10 Computational Science Accomplishments

Titles in Blue – SciDAC: Titles in Black - INCITE

Via Lactea II, A Billion Particle Simulation of the Dark Matter Halo of the Milky Way (Madau)

8

Prediction and Design of Macromolecular Structures and Functions (Baker)3

Modeling the Molecular Basis of Parkinson’s Disease (Tsigelny)1

Understanding How Lifted Flame Stabilized in a Hot Coflow (Yoo)4

10

9

7

6

5

2

Rank

PETSc: Providing the Solvers for DOE High-Performance Simulations (Smith)

High Transition Temperature Superconductivity: A High-Temperature Superconducting State and a Pairing Mechanism in 2-D Hubbard Model (Scalapino)

New Insights from LCF-enable advanced kinetic simulations of global turbulence in fusion systems (Tang)

Discovery of the Standing Accretion Shock Instability and Pulsar Birth Mechanism in a Core-Collapse Supernova Evolution and Explosion (Blondin)

First Provably Scalable Maxwell Solver Enables Scalable Electromagnetic Simulations (Kovel)

Probing the properties of water through advanced computing (Galli)

Title

DCA++ Achieves 1.3 Petaflops

High Tc Superconducting in Cuprates• 2-D Hubbard Model• Study Materials with

Disorders/Impurities• First petaflop application• Spurred community debate • Inspired SNS experimentDCA++• Monte Carlo Method• 10X Speedup by Scientific Computing

Group at OLCF through:– Delaying memory intensive

operations (reorder barriers)– Mixed Precision arithmetic

(move fewer bits per flop)

2008 Gordon Bell Prize Winner

Philip AndersonDoug Scalapino

14

Petaflops to Exaflops14 years Ago

“Building a computer 10 times larger than all the networked computing capability in the USA”

2007“range of applications that would be materially transformed by the availability of exascale systems”

www.er.doe.gov/ASCR/ProgramDocuments/TownHall.pdf

Scientific Challenges Workshop Series

Enabling science communities to address scientific grand challenges through extreme scale computational science

Workshop series: • Climate Science• High-Energy Physics• Nuclear Physics• Fusion Energy Sciences• Nuclear Energy• Biology• Materials Science and Chemistry• NNSA• CS-Math & Architectures

26-28 January 2009, Washington, DC109 participants; DOE/NSF/NNSA reps

The Nuclear Physics Workshop defined Priority Research Directions in• Nuclear Astrophysics• Cold QCD and Nuclear Forces• Nuclear Structure and Reactions• Accelerator Physics• Hot and Dense QCD

Workshop chair: Dr. Glenn YoungCo-chairs: Dr. David Dean,

Dr. Martin Savage

Scientific Challenges Workshop SeriesExascale computing will unify Nuclear Physics

Nuclear Structure

QCD

Applications in astrophysics, defense, energy, and medicine

Scientific Challenges Workshop SeriesExascale computing will unify Nuclear Physics

Cold QCD andNuclear Forces

NuclearAstrophysics

Nuclear Structureand Reactions

Hot and DenseQCD

HEP neutrino

Equation of state of nuclear material?Kaon (strange meson) Condensates?Sigma (strange) Baryons?Is SN1987A a black hole or a neutron star?

Scientific Challenges Workshop SeriesExample: Nuclei

Scientific Challenges Workshop SeriesExample: Hadronic Structure

Scientific Challenges Workshop SeriesExamples: Astrophysics

“Effective use of exascale systems will require fundamental changes in how we develop and validate simulation codes for these systems and how we manage and extract knowledge from the massive amount of data produced.”

Exascale Townhall: Software – Findings

Challenges for the FuturePath to Extreme Scale

Applied Math and Computer Science have contributed even more than Moore’s Law

Hitting the Cliff Hitting the Cliff

SciDAC 1SciDAC 1

SciDAC 2SciDAC 2

SciDAC XSciDAC X

I climb the "Hill of Science,"I "view the landscape o'er;"

Such transcendental prospect,I ne'er beheld before!

-Emily Dickinson

TerascaleTerascale

PetascalePetascale

Extreme ScaleExtreme Scale

Core

Res

earc

h