SCEC Information Technology Overview for 2012

Philip J. MaechlingInformation Technology Architect

Southern California Earthquake Center

SCEC Board of Directors Meeting30 Jan 2012

SCEC3 Information Technology (IT) Work Areas

1. Administrative IT including membership, awards, contractso John McRaney, Tran Huynh, John Yu, Karen Young and others

2. Collaboration IT supports collaborative research projects, communications, meetings, creation and distribution of research results.o John Marquis, Tran Huynh, John Yu, John McRaney and others

3. Outreach IT prepares, presents, and preserves research results to collaborative partners, funding agencies, and publico Thomas Jordan, Mark Benthien, Philip Maechling, Tran Huynh, and

others

4. Computational Science IT provides access to computer hardware, scientific software, and data management.o Philip Maechling, Yifeng Cui, Ewa Deelman, Scott Callaghan, Maria

Liukis, Patrick Small, Kevin Milner, John Yu and others

SCEC Scientific Software Distributions

1. CSEP Software Framework– Last release Jan 2012

2. CVM-S Community Velocity Model– Last release Feb 2011

3. CVM-H Community Velocity Model– Last release Oct 2011

4. UCVM Software Framework– Last release Jan 2012

5. Broadband Platform– Last release Oct 2011

6. SCEC-VDO visualization of 3D geo-referenced data– Available on request

SCEC Scientific Software In Active Use

1. AWP-ODC – Finite difference dynamic ruptures and anelastic wave propagation– Simulations Jan 2012

2. Hercules – Finite element anelastic wave propagation– Simulations Jan 2012

3. R. Graves Finite Difference Wave Propagation – Simulations in Jan 2012

4. OpenSHA probabilistic seismic hazard analysis– Calculations in Jan 2012

5. Tera3D full 3d tomography processing system– California regional inversion iteration in Jan 2012

6. CyberShake physics-based PSHA hazard curve calculation– Research hazard curve in Jan 2012

7. CSEP Testing Framework– Daily use in CSEP, EEW, and TD Testing

8. Broadband Platform– Use on SCEC servers by PEER and NGA-E modelers in Jan 2012

9. SCEC-VDO– Used by UCERF Scientific Reviewers in Jan 2012

Unified Community Velocity Model Development

UCVM development included (a) CVM evaluation tools, (b) CVM integration framework, and (c) combining best of existing models into consistent state-wide CVM for use with state-wide 1Hz 3D wave propagation simulations. Standardizes query API, elevation DEM, bathymetry, Moho, GTL, and distribution format

Inversion-based Update to CVM-S4

Performed two seismic inversion iterations using different data sets1. CVM4SI1: improved CVM4 using earthquake data.2. CVM4SI2: improved CVM4SI1 using ambient-noise Green’s function data.Perturbation obtained in 2nd iteration enhances the perturbation obtained in the 1st

iteration. Waveform improvements for both earthquake recordings and ambient noise Green’s functions

SCEC CyberShake system produced hazard curves with alternative (improved) rupture generator and hazard curves based on CVM-H 11.2.

Used SCEC allocations on NSF supercomputers to calculate CyberShake PSHA hazard curves for sites of interest (San Onofre and Diablo Canyon)

SCEC Broadband Platform 11.2.2 Released

SCEC released Broadband Platform 11.2.2 which can calculate 10Hz seismograms using rupture generators, 1D wave propagations codes, and stochastic high frequency software from multiple SCEC groups.

Broadband CyberShake• Added R. Graves high-frequency stochastic modules from the

BBP into CyberShake Seismogram calculation– Stochastic high-frequency– Non-linear site response– Filtered and combined with low-frequency (<0.5 Hz)– Validate with SCSN, PBR sites

Perris (PBR)

1 sec 0.1 sec

Current Broadband CyberShake Run• 92 hazard curves

– CVM-H 11.2 and CVM-S4, Graves and Pitarka (2010)– PBR, SCSN sites– 64 SGTs, 92 post-processing– 1.5 million SUs

• Data requirements (broadband, 2-component)– SGTs: 2.5 TB– Seismograms: 3.9 TB

PBR=red, SCSN=orange, near SCSN=yellow

SCEC Computational Science Software Projects

The Quake Group at Carnegie Mellon Current Efforts and Future Plans

Jacobo Bielak, Ricardo Taborda, Haydar Karaoglu, Yigit Isbiliroglu

The Effect of Minimum S-wave VelocityVsmin 200 m/s Vsmin 500 m/s

2011 SCEC CME Meeting

Anelastic Wave Propagation (AWP)Various applicationsHerculesQuadratic ElementsNo Intrinsic Attenuationfmax 2 HzVsmin 200 m/sDomain 85 x 85 x 42.5 km3

Elements 226,670,602Cores 18,000Walltime 4 hrsSUs 72,000

Short-Period Chino Hills Simulation

fmax 4 HzVsmin 200 m/sDomain 180 x 135 x 62 km3

Elements 5 billionCores 24,000Walltime 31 hrsSUs 744,000

1.5 trillion grid points for a regular grid; 20 times the # of SUs

Blue – simulationRed – data

Source model: Chen Ji

Source model: Graves & Pitarka

SCEC Computational Science Forecast Testing Centers

1. Earthquake forecasts (CSEP)– Active SCEC CSEP Operational Testing Center

2. Transient Detection (TD)– Active Operational TD Testing Center

3. EEW CISN Testing Center– Active Operational CTC Testing Center

CSEP Development and SCEC CSEP Testing Center

CISN EEW Testing Center

SCEC Transient Detection Testing Center

SCEC Computational Science Organizational Interactions

1. National Science Foundation (NSF)2. United States Geological Survey (USGS)3. W. M. Keck Foundation4. Department of Homeland Security (DHS)5. Pacific Gas and Electric (PG and E)6. USC High Performance Computing and Communications (HPCC)7. Argonne Leadership Computing Facility (ALCF)8. Oak Ridge National Laboratory (ORNL)9. National Institute for Computational Science (NICS)10. Blue Waters NSF Track 1 Computing Facility (Univ. of Illinois)11. Extreme Science and Engineering Discovery Environment (XSEDE)12. San Diego Supercomputer Center (SDSC)

SCEC Computational Science Organizational Interactions

13. Texas Advanced Computing Center (TACC)14. Pittsburgh Supercomputing Center (PSC)15. Open Science Grid (OSG)16. National Center for Atmospheric Research (NCAR)17. NCAR-Wyoming Supercomputing Center (NWSC)18. USC Information Sciences Institute (ISI)19. Data Intensive Cyber Environments (DICE) Center at the University of

North Carolina at Chapel Hill20. Incorporated Research Institutions for Seismology (IRIS) Data

Management Center21. California Integrated Seismic Network (CISN)22. Computational Infrastructure in Geodynamics (CIG)23. Global Earthquake Model (GEM)

SCEC Computational Science In DOE Publication

SCEC CME researchers have access to three leadership class Supercomputers (NSF Blue Waters (2012), DOE Titan (2012), DOE Mira (2013)) to use for SCEC research calculations.

UCERF3.0 Extended Earthquake Rupture Forecast contains slip-time histories and rupture variations for all UCERF3.0 ruptures.

UCVMUnified California Velocity Model

UCERF 3.0 Extended Earthquake Rupture Forecast(June 2012)

UCVM – Unified California Velocity Model, a statewide 3-D velocity model assembled from existing and improved California CVM’s (Small, SCEC USR)

CME CyberShake 3.0 calculation will build on UCERF3.0 in order to calculate a “physics-based” California PSHA Map

CyberShake 3.0California Wave Propagation-based PSHA Map up to 10Hz (Fall 2012)

CyberShake 1.0Los Angeles Region Wave Propagation-based PSHA Map up to 0.5Hz (Fall 2009) will increase in range and frequencies.

CyberShake 3.0 Computational Estimates

CyberShake 3.0 Hazard Map Calculation Estimates:

Number of Sites on Map: 4240 (10km spacing)Number of Jobs: 3.6 billion Est. CPU-hours: 376.2 millionData products (seismograms, spectral acceleration): 76.5 TB Runtime on half-Jaguar: 3096 hrs (129 days) Runtime on half-Blue Waters: 770 hrs (32.1 days) Database entries: 6.95 billion

2011 Funded Projects (estimates per year)

1. CSEP Software Framework Development and Operations (200k)2. OpenSHA and SCEC-VDO in support of UCERF 3.0 (100k)3. NSF PetaSHA-3 (850k)4. NSF PRAC Blue Waters (5k)5. USGS CISN EEW Project (50k)6. Broadband Platform from PG and E (150k)7. Transient Detection Testing (60k)

2012 Funded Projects (estimates per year)

1. CSEP Software Framework Development and Operations (200k)2. OpenSHA and SCEC-VDO in support of UCERF 3.0 (80k)3. NSF SI2 SEISM Project (700k)4. NSF PetaSHA-3 (425k)5. NSF PRAC Blue Waters (5k)6. USGS Supported CISN EEW Project (25k)7. Broadband Platform from PG and E (150k)

NSF Software Infrastructure for Sustained Innovation Proposal (Submitted July 2011)

SI2-SSI: A Sustainable Community Software Framework for Petascale Earthquake Modeling PI: T. H. Jordan (USC); Co-PIs: Jacobo Bielak (CMU), Y. Cui (SDSC), and K. Olsen (SDSU)

• Year 1: – Computational milestone: simulation of Table 1 earthquake records at 0-4 Hz (deterministic) and 1-10 Hz

(stochastic) using the Broadband platform. – Validation effort: GMSV of these simulations against the earthquake records of Table 1 using linear and

nonlinear SDoF oscillators. UCVM comparison of tomographic results for Southern California. – Software releases: UCVM platform, Broadband platform; Kraken release of SEISM-IO.

• Year 2: – Computational milestone: Time-independent CSHM-LA (UCEF2-based). – Validation efforts: GMSV for geotechnical systems; comparison of CSHM-LA with NGA hazard maps and PBR

data set. Full-3D evaluation of statewide UCVM. – Software releases: UCVM platform, Broadband platform; Blue Waters release of SEISM-IO.

• Year 3: – Computational milestone: Time-dependent CSHM-CA (UCEF3-based). – Validation efforts: GMSV for nonlinear MDoF systems; comparison of CSHM-CA with NGA hazard maps and PBR

data set. – Software releases: UCVM platform, Broadband platform; Mira release of SEISM-IO.

NSF Geoinformatics Proposal (Submitted Jan 13, 2012)

Geoinformatics: Community Computational Platforms for Developing Three-Dimensional Models of Earth StructurePI: T. H. Jordan (USC); Co-PIs: P. Chen (U. Wyoming), Y. Cui (SDSC), and J. Tromp (Princeton)

Objective: Establish an interoperable set of community computational platforms that will (a) implement the techniques of full-3D waveform tomography, and (b) facilitate the development and delivery of Earth models at a variety of scales. Two tomographic platforms will be built on highly scalable codes for solving the forward problem of anelastic wave propagation (AWP):

• AWP-ODC platform. AWP-ODC is 4th-order, staggered-grid, finite-difference code developed by K. Olsen and his colleagues.

• SPECFEM3D platform. SPECFEM3D is spectral element code, developed by D. Komatitsch and J. Tromp.

• UCVM platform developed by SCEC enables users to build meshes from models with topography, hydrology, bathymetry, and standardized models of surficial (geotechnical) layers.

For More Information Please Visit:

http://www.scec.org/cme

End