Secrets of the Dark Universe: Simulating the Sky on the Blue...
3
Secrets of the Dark Universe: Simulating the Sky on the Blue Gene/Q The Outer Rim Simulation Joseph A. Insley, Salman Habib, Katrin Heitmann Argonne National Laboratory {insley, habib, heitmann}@anl.gov Abstract—An astonishing 99.6% of our Universe is dark. Observations indicate that the Universe consists of 70% of a mysterious dark energy and 25% of a yet unidentified dark matter component, and only 0.4% of the remaining ordinary matter is visible. Understanding the physics of this dark sector is the foremost challenge in cosmology today. Sophisticated simulations of the evolution of the Universe play a crucial task in this endeavor. This movie shows a large simulation of the distribution of matter in the Universe, the so-called cosmic web, which evolved under the influence of dark energy. The simulation is evolving 1.1 trillion particles using HACC, a new computational framework developed to overcome the challenges posed by future supercomputing architectures. It is currently running on 32 racks of Mira, the Blue Gene/Q system at the Argonne Leadership Computing Facility. This work is described in further detail in an SC13 Gordon Bell Finalist paper [1]. The visualization was performed on Argonne’s Tukey cluster. I. VISUALIZATION This dark matter simulation evolves 1.1 trillion particles using 262,144 processes. These particles are projected onto a regular grid of 10,240 3 for visualization, resulting in a volume that is 3000 Mpc/h, or 1.3*10 10 light-years, across. This animation shows the evolution of the Universe to a redshift of z=0.7, when the Universe was 7.4 billion years old. The data from one process is used to highlight the fine-grained details. The region shown is 50 Mpc/h, or 2.3*10 8 light-years, across. This region is then compared to the data from a slab that is 10,240x10,240x800, showing the full domain in the X and Y dimensions, see Figure 1. This is followed by a fly-through of the slab at that snapshot in time, see Figure 2. The animation can be downloaded from: www.alcf.anl.gov/research/fl/flinternal/HACC/OUTER RIM/ hacc outer rim L4225 evolve scale flythru v01.mp4 ADDITIONAL AUTHORS Argonne National Laboratory: Mark Hereld, Hal Finkel, Nicholas Frontiere, Kalyan Kumaran, Vitali Morozov, Michael E. Papka, Tom Peterka, Adrian Pope, Venkatram Vishwanath, Tim Williams; Los Alamos National Laboratory: David Daniel, Patricia Fasel; Lawrence Berkeley National Labora- tory: Zarija Lukic. ACKNOWLEDGMENT This work was supported in part by the Early Science Pro- gram (ESP), a project of the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science User Facility, aimed at preparing key applications for the architecture and scale of Mira, the 10 petaflop IBM Blue Gene/Q supercomputer which was installed in the ALCF in 2012, and went into production in early 2013. This simulation and visualization were produced using resources of the ALCF at Argonne National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under contract DE-AC02-06CH11357. Fig. 1. Snapshot comparing the data from one process (right) to the data from a slab showing the full domain in the X and Y dimensions (left). Fig. 2. Snapshot showing a close-up of a fly-through of a 10,240x10,240x800 slab of the volume, at redshift z=0.7.
Transcript of Secrets of the Dark Universe: Simulating the Sky on the Blue...
Secrets of the Dark Universe:Simulating the Sky on the Blue Gene/Q
The Outer Rim SimulationJoseph A. Insley, Salman Habib, Katrin Heitmann
Argonne National Laboratory{insley, habib, heitmann}@anl.gov
Abstract—An astonishing 99.6% of our Universe is dark.Observations indicate that the Universe consists of 70% of amysterious dark energy and 25% of a yet unidentified darkmatter component, and only 0.4% of the remaining ordinarymatter is visible. Understanding the physics of this dark sectoris the foremost challenge in cosmology today. Sophisticatedsimulations of the evolution of the Universe play a crucial taskin this endeavor.
This movie shows a large simulation of the distributionof matter in the Universe, the so-called cosmic web, whichevolved under the influence of dark energy. The simulation isevolving 1.1 trillion particles using HACC, a new computationalframework developed to overcome the challenges posed by futuresupercomputing architectures. It is currently running on 32 racksof Mira, the Blue Gene/Q system at the Argonne LeadershipComputing Facility. This work is described in further detail inan SC13 Gordon Bell Finalist paper [1]. The visualization wasperformed on Argonne’s Tukey cluster.
I. VISUALIZATION
This dark matter simulation evolves 1.1 trillion particlesusing 262,144 processes. These particles are projected ontoa regular grid of 10,2403 for visualization, resulting in avolume that is 3000 Mpc/h, or 1.3*1010 light-years, across.This animation shows the evolution of the Universe to aredshift of z=0.7, when the Universe was 7.4 billion yearsold. The data from one process is used to highlight thefine-grained details. The region shown is 50 Mpc/h, or2.3*108 light-years, across. This region is then compared tothe data from a slab that is 10,240x10,240x800, showing thefull domain in the X and Y dimensions, see Figure 1. Thisis followed by a fly-through of the slab at that snapshot intime, see Figure 2. The animation can be downloaded from:www.alcf.anl.gov/research/fl/flinternal/HACC/OUTER RIM/hacc outer rim L4225 evolve scale flythru v01.mp4
ADDITIONAL AUTHORS
Argonne National Laboratory: Mark Hereld, Hal Finkel,Nicholas Frontiere, Kalyan Kumaran, Vitali Morozov, MichaelE. Papka, Tom Peterka, Adrian Pope, Venkatram Vishwanath,Tim Williams; Los Alamos National Laboratory: DavidDaniel, Patricia Fasel; Lawrence Berkeley National Labora-tory: Zarija Lukic.
ACKNOWLEDGMENT
This work was supported in part by the Early Science Pro-gram (ESP), a project of the Argonne Leadership ComputingFacility (ALCF), a DOE Office of Science User Facility, aimedat preparing key applications for the architecture and scale ofMira, the 10 petaflop IBM Blue Gene/Q supercomputer whichwas installed in the ALCF in 2012, and went into productionin early 2013.
This simulation and visualization were produced usingresources of the ALCF at Argonne National Laboratory, whichis supported by the Office of Science of the U.S. Departmentof Energy under contract DE-AC02-06CH11357.
Fig. 1. Snapshot comparing the data from one process (right) to the datafrom a slab showing the full domain in the X and Y dimensions (left).
Fig. 2. Snapshot showing a close-up of a fly-through of a 10,240x10,240x800slab of the volume, at redshift z=0.7.
REFERENCES
[1] Salman Habib, Vitali Morozov, Nicholas Frontiere, Hal Finkel, AdrianPope, Katrin Heitmann, Kalyan Kumaran, Venkat Vishwanath, TomPeterka, Joseph A. Insley, David Daniel, Patricia Fasel, and Zarija Lukic.
Hacc: Extreme scaling and performance across diverse architectures.In Proceedings of the IEEE/ACM International Conference for HighPerformance Computing, Networking, Storage and Analysis (SC 2013)[Gordon Bell Finalist], November 2013.
The submitted manuscript has been created by UChicago Argonne, LLC, Operator of Argonne NationalLaboratory (“Argonne”). Argonne, a U.S. Department of Energy Office of Science laboratory, is operatedunder Contract No. DE-AC02-06CH11357. The U.S. Government retains for itself, and others acting on itsbehalf, a paid-up nonexclusive, irrevocable worldwide license in said article to reproduce, prepare derivativeworks, distribute copies to the public, and perform publicly and display publicly, by or on behalf of theGovernment.
