Computational Computational Astrophysics:Astrophysics:

Research to Teaching and BeyondResearch to Teaching and Beyond

Adam FrankAdam FrankUniversity of RochesterUniversity of Rochester

A Cast of ManyA Cast of Many

AstroBEAR MHD / Clumpy FlowsAstroBEAR MHD / Clumpy FlowsSorin Mitran UNCSorin Mitran UNC*Andrew Cunningham (UR, UC Berkeley, LLNL) *Andrew Cunningham (UR, UC Berkeley, LLNL) Alexei Poludnenko (UR, NRL)Alexei Poludnenko (UR, NRL)Kris Yirak (UR Grad Student)Kris Yirak (UR Grad Student)Jonathan Carroll (UR Grad Student)Jonathan Carroll (UR Grad Student)

Thanks to: NSF, DOE, NASA, UR Laboratory for Laser EnergeticsThanks to: NSF, DOE, NASA, UR Laboratory for Laser Energetics

Boundary conditions

Initial conditions

Simulations:Simulations:Numerical Experiments with Diff. Eq & PDEsNumerical Experiments with Diff. Eq & PDEs

etcx

SFSF

t

trS x ,);(),(

sp

ace

space

time

33rdrd Age of Simulations: Age of Simulations:

11stst Age: 1-D calculations with computers Age: 1-D calculations with computers– CDC 7600, etcCDC 7600, etc

22ndnd Age: Moderate rez 2-D calculations Age: Moderate rez 2-D calculations– Supercomputers: Cray, etcSupercomputers: Cray, etc

33rdrd Age: High rez Multi-D, Multi-physics calc. Age: High rez Multi-D, Multi-physics calc.– Grid Computing, Massively Parallel, Clusters Grid Computing, Massively Parallel, Clusters

We are talking about “Virtual Reality” on a We are talking about “Virtual Reality” on a scientific level.scientific level.

Sim. data sets now as “rich” as real data setsSim. data sets now as “rich” as real data sets– Petabytes/flops 10Petabytes/flops 101515 = one quadrillion bytes. = one quadrillion bytes.

Not just simulation but “Cyberscience” Not just simulation but “Cyberscience” – Integrate IT on all levels of science practiceIntegrate IT on all levels of science practice

History: Research Simulation History: Research Simulation Shocked ClumpsShocked Clumps

w

c

w

ccc V

Rt

2/12

• Astrophysical environments very heterogeneous.

• Winds, Blastwaves & ISM are all “clumpy”

• Interaction of 1-clump with passing wind is a well studied problem.

Critical Parameter tcc: Cloud Crushing Time

Woodward 1976

Frontiers of Algorithm Frontiers of Algorithm DevelopmentDevelopment Adaptive Mesh Refinement – code Adaptive Mesh Refinement – code

automatically places grid cells where automatically places grid cells where needed.needed.

Multi-physics – code simulates many Multi-physics – code simulates many physical processes simultaneouslyphysical processes simultaneously– Magnetic Fields, gravity, radiation Magnetic Fields, gravity, radiation

transport, chemistry, ionization dynamics.transport, chemistry, ionization dynamics.

UR Computational UR Computational Group:Group:

AstroBEARAstroBEAR

Multiyear, Multi grad-student effort

Began 2002

Now on 4th generation of student

Simulation in Age of Simulation in Age of CyberScienceCyberScience

How to maintain 106 line code base across

N years M revisions K Student Generations

Static documentation doesn’t work.

Our solution: Wiki, self-compiling documentation

Adaptive Mesh Refinement Adaptive Mesh Refinement AMRAMR

R. Deiterding

• Different AMR Methods:Different AMR Methods:Grid based regriddingGrid based regriddingCell based regriddingCell based regridding

• University of Rochester CodeUniversity of Rochester CodeAstroBEAR: gridAstroBEAR: grid

• Hierarchy of GridsHierarchy of Grids

• Require Require ProlongationProlongation//RestrictionRestriction Operators Operators

Carry data from one grid level Carry data from one grid level to anotherto another

• Prolongation (Corse to Fine)Prolongation (Corse to Fine)• Restriction (Fine to Corse)Restriction (Fine to Corse)

Mach 10 radiatively cooled bullet

AMR grid generation in the system

Pre-Planetary Nebula: CRL 618“Explosion” from dying Solar-type star

AMR MHD AMR MHD Div B = 0Div B = 0

Hydro: Need conservative prolongation/ restriction Hydro: Need conservative prolongation/ restriction operators. operators.

MHD: Maintain solenoidal condition. Need MHD: Maintain solenoidal condition. Need divergence free operators on “staggered mesh”.divergence free operators on “staggered mesh”.

Cunningham et al 2007

Results I MHD Shocked Clump“Standard” Test

M = 10M = 10= n= ncc/n/nw w = = == 44

Many Clumps: Radiative MHD Shocks in Many Clumps: Radiative MHD Shocks in Heterogeneous MediaHeterogeneous MediaCunningham et al 2007Cunningham et al 2007 M = 10M = 10

= n= ncc/n/nw w = = == 1010

How AMR Changes Game:How AMR Changes Game:Resolution and Convergence Resolution and Convergence

Convergence formally defined as approach to Convergence formally defined as approach to known analytic solution.known analytic solution.

For complex non-linear problem these rarely existFor complex non-linear problem these rarely exist

Define convergence as change relative to highest Define convergence as change relative to highest resolution simulation possible.resolution simulation possible.

•For adiabatic shock clump convergence appears For adiabatic shock clump convergence appears at N = 120 cells/Rat N = 120 cells/Rcc

€

f =QN −QMaxQMax

Resolution and ConvergenceResolution and ConvergenceRadiative Clumps Radiative Clumps

Radiative cooling allows post shock flows to collapse – but how far?

AMR allows us to run highest resolution radiative clump simulations to date: N =

1500/Rc

Highest resolution radiative clump simulations to date: N ~ 1500/Rc

Resolution and ConvergenceResolution and ConvergenceRadiative Clumps Radiative Clumps

Radiative cooling allows post shock flows to collapse – but how far?

Increases in resolution show qualitatively new behaviors as x < crit

Resolution and ConvergenceResolution and ConvergenceRadiative Clumps Radiative Clumps

What measures, metrics can we trust at given resolution?

http://www.pas.rocheste

r.edu/~yirak/foradam/

vorticities.png

Radiative MHD Clumps with Radiative MHD Clumps with Self-consistent FieldsSelf-consistent Fields

How many MHD-AMR CodesHow many MHD-AMR Codes

Not many

AstroBEARFlashENZOOrionAMR VACAthena

Computation in TeachingComputation in TeachingMy adventure in My adventure in E-ed-bizE-ed-biz

2000 NSF Career Award

AstroFlow – Simulation outreach tool

Planetarium asks to buy copy (?!?)

Create Truth-N-Beauty LLC with UR

E-education digital media company

Produce simulation based modules for:

McGraw-Hill, Prentice Hall etc

Second Avenue SoftwareSecond Avenue SoftwareThe e-biz goes it aloneThe e-biz goes it alone

2006 Truth-N-Beauty becomes 2nd Ave

What we builtWhat we built

Celestial Sphere

Phases of Moon

Seasons

Solar System Builder

Planetary Atmospheres

Going Beyond TeachingGoing Beyond Teaching

Outreach:

Use partners in new media to create

interactives for websites.

Discover/Astronomy/SciAmerican

“Serious Games” – twitch games

with a science theme.

Going Beyond TeachingGoing Beyond Teaching

Star Star FormationFormation

Conclusions Conclusions

ComputationComputation– Advanced AMR/multi-physics codes allow new era of Advanced AMR/multi-physics codes allow new era of

simulationsimulation– ““Weather vs. Climate”: what to do with Petaflops/bytesWeather vs. Climate”: what to do with Petaflops/bytes

Teaching by SimulationTeaching by Simulation– New opportunities if done well (graphics, pedagogy)New opportunities if done well (graphics, pedagogy)– New opportunities outside of classroom.New opportunities outside of classroom.