Status of the Exascale Computing Project on High-Fidelity Whole Device Modeling of Magnetically Confined Fusion Plasma

A. Bhattacharjee (PI), C.-S. Chang (Co-PI), J. Dominski, S. Ethier, R. Hagar, and S. Ku,

Princeton Plasma Physics Laboratory, Princeton University

A. Siegel (Co-PI) , Argonne National Laboratory

F. Jenko and G. Merlo, University of California-Los Angeles

S. Parker and B. Sturdevant, University of Colorado-Boulder

J. Hittinger and L. Ricketson, Lawrence Livermore National Laboratory

S. Klasky, E. D’Azevedo, and E. Suchyta, Oak Ridge National Laboratory

M. Parashar, Rutgers University

www.ExascaleProject.org

2 Exascale Computing Project

ECP’s Holistic Approach

3 Exascale Computing Project

Exascale Applications Cover 6 DOE Strategic Pillars

4 Exascale Computing Project

Co-Design Centers

• Co-Design Center for Online Data Analysis and Reduction at the Exascale (CODAR)

• Block-Structured AMR Co-Design Center (AMReX)

• Center for Efficient Exascale Discretizations (CEED)

• Co-Design Center for Particle Applications (CoPA)

• Combinatorial Methods for Enabling Exascale Applications(ExaGraph)

• (FFT co-design???)

5 Exascale Computing Project

Current Set of ECP Software Projects

6 Exascale Computing Project

Fusion ECP: High-Fidelity Whole Device Modeling of Magnetically Confined Fusion Plasmas

• Develop high-fidelity Whole Device Model (WDM) of magnetically confined fusion plasmas to understand and predict the performance of ITER and future next-step facilities, validated on present tokamak (and stellarator) experiments

• Couple existing, well established extreme-scale gyrokinetic codes

– GENE continuum code for the core plasma and the

– XGC particle-in-cell (PIC) code for the edge plasma, into which a few other important (scale-separable) physics modules will be integrated at a later time for completion of the whole-device capability

• Y1: Demonstrate initial implicit coupling capability between core (GENE) and edge (XGC) on the ITG turbulence physics

• Y2: Demonstrate telescoping of the gyrokinetic turbulent transport using a multiscale time integration framework on leadership class computers

• Y3: Demonstrate and assess the experimental (transport) time scale telescoping of whole-device gyrokinetic physics

• Y4: Complete the phase I integration framework and demonstrate the capability of the WDM of multiscale gyrokinetic physics in realistic present-day tokamaks on full-scale SUMMIT, AURORA, and CORI

7 Exascale Computing Project

10-year Goal: A First-Principles-Based Whole Device Model that Covers the Full Space/Time Scales of a Reactor

• XGC full-f particle-in-cell technique with continuity across separatrix • GENE continuum delta-f capability for core

Full-f

delta-f

Integration

Applied Math + Computer Science

Framework Plasma-Material

Interaction

RF and Neutral Beam

Extended MHD

Energetic Particles

Tight / loose

coupling methods

Multi-scale time

advance PPPL (lead), ANL, LLNL,

ORNL, Rutgers, UCLA, UC-

Boulder

8 Exascale Computing Project

XGC-GENE Coupling

• XGC’s 2D unstructured triangular grid covers the whole volume and can provide a benchmarking whole-device solution

- Finite differencing in the toroidal direction

- Equation of motion is in the cylindrical coordinate system.

• GENE uses 2D structured grid in the core region

- Fourier decomposition in the 3rd direction.

- Equation of motion is in a flux coordinate system

- Cannot cross the magnetic separatrix

GENE

XGC Interface

layer

9 Exascale Computing Project

Coupling Model (L. Ricketson, J. Hittinger, LLNL, S. Parker, U. Col.)

10 Exascale Computing Project

Coupling model

11 Exascale Computing Project

Coupling

• Options 2 and B: More opportunity for concurrency, but looser coupling.

• Option 2B: ρr and ρl are completely independent - no coupling at all.

• Only consider 1A, 1B, and 2A

12 Exascale Computing Project

Coupling Model

13 Exascale Computing Project

Comparison – No Fluctuations

Option 1A Option 2A Option 1B

14 Exascale Computing Project

Adding Fluctuations to Coupling Model

15 Exascale Computing Project

Comparison with Fluctuations (Ff 0)

Option 1A Option 2A Option 1B

16 Exascale Computing Project

Benchmarking XGC and GENE: Challenge Problems

• Cyclone Base Case: Motivated by an experimentally relevant plasma on the DIII-D facility. Basis for Y1-Q1 and Y1-Q2 milestones for benchmarking GENE and XGC on linear and nonlinear dynamics of the ion-temperature gradient (ITG) instability

• DIII-D Challenge Case: ITG turbulence in DIII-D size plasma Challenge problem 2 is realistic. Translation to exascale is an issue of scale as well as algorithmic advances in code-coupling.

17 Exascale Computing Project

Cross-verification between GENE, XGC, and ORB5: linear ITG instability (S. Ku, G. Merlo, E. Lanti (SPC, EPFL))

Growth rates Real frequencies

• Codes agree within 10% for all modes considered

18 Exascale Computing Project

Cross-verification between GENE, XGC, and ORB5: linear ITG instability (S. Ku, G. Merlo, E. Lanti)

• Poloidal cross sections of the electrostatic potential associated to the

n = 24 mode obtained at the end of the simulation.

19 Exascale Computing Project

Cross-verification between GENE, XGC, and ORB5: linear ITG instability (S. Ku, G. Merlo, E. Lanti)

• Comparison of the eigenfunction associated to the mode n=24

radius Straight-field-line poloidal angle c (r=0.5)

poloidally averaged fluctuation <|f|>

20 Exascale Computing Project

On-going Cross-verification between GENE and XGC: Non-linear ITG instability (J. Dominski, S. Ku, G. Merlo, E. Lanti)

“linear stage”

21 Exascale Computing Project

On-going Cross-verification between GENE and XGC: Non-linear ITG instability (J. Dominski, S. Ku, G. Merlo, E. Lanti)

Case 2 Case 1

22 Exascale Computing Project

Final word

• ECP might grow even larger

• Our fusion project will bring in other codes and researchers if we are successful in the first 4 years