Southern California Earthquake Center - SCEC

SCEC/CMETom Jordan (USC)

Bernard Minster (SIO)

Carl Kesselman (ISI)

Reagan Moore (SDSC)

Phil Maechling (USC)

SCECSCEC Member Institutions Member Institutions(October 1, 2005)(October 1, 2005)

SCEC/CME - Terashake• Kim B. Olsen (SDSU)• Bernard Minster (IGPP)• Reagan Moore (SDSC)• Steve Day (SDSU)• Phil Maechling (USC)• Tom Jordan (USC)• Marcio Faerman (SDSC)• Geoffrey Ely (IGPP)• Boris Shkoller (IGPP)• Carey Marcinkovich (EXxonMobil)• Jacobo Bielak (CMU)• David Okaya (USC)• Ralph Archuleta (UCSB)• Steve Cutchin (SDSC) • Amit Chourasia (SDSC)• George Kremenek (SDSC)• Yuanfang Hu (SDSC)

• Arun Jagatheesan (SDSC)• Nancy Wilkins-Diehr (SDSC)• Richard Moore (SDSC)• Bryan Banister (SDSC)• Leesa Brieger (SDSC)• Amit Majumdar (SDSC) • Yifeng Cui (SDSC)• Giridhar Chukkapalli (SDSC)• Qiao Xin (SDSC) • Donald Thorp (SDSC)• Patricia Kovatch (SDSC)• Larry Diegel (SDSC)• Tom Sherwin (SDSC)• Christopher Jordan (SDSC)• Marcus Thiebaux (ISI)• Julio Lopez (CMU)

SCEC/CME Digital Library

Reagan W. Moore staff - SDSC PIMarcio Faerman postdoc (now in Brazil) - dig libGeorge Kremenek staff - data managementYuanfang Hu graduate student - dig libJing Zhu graduate student - dig libPatrick Yau graduate student - visAmit Chourasia staff (contributed effort) - visStephen Cutchin staff (contributed effort) - visYifeng Cui staff (contributed effort) - applicationAmitava Majumdar staff (contributed effort) - application

Seismic Hazard Analysis

• Intensity measure: Intensity measure: peak ground peak ground acceleration (PGA)acceleration (PGA)

• Interval: 50 yearsInterval: 50 years

• Probability of Probability of exceedance: 2%exceedance: 2%

Definition:Definition: Specification of the maximum intensity of shaking Specification of the maximum intensity of shaking expected at a site during a fixed time intervalexpected at a site during a fixed time interval

Example:Example: National seismic hazard maps National seismic hazard maps

(http://geohazards.cr.usgs.gov/eq/)(http://geohazards.cr.usgs.gov/eq/)

Web Sites

• SCEC/CME– http://epicenter.usc.edu/cmeportal/index.html

• SCEC Digital Library– http://www.sdsc.edu/SCEC/

• SCEC Terashake Simulations– http://sceclib.sdsc.edu/TeraShake/

• Storage Resource Broker– http://www.sdsc.edu/srb/

• Visualizations– http://visservices.sdsc.edu/projects/scec/

IntensityMeasures

Earthquake Forecast Model

AttenuationRelationship

1

Standardized seismic hazard analysisStandardized seismic hazard analysis

Ground motion simulationGround motion simulation

Physics-based earthquake forecastingPhysics-based earthquake forecasting

Ground-motion inverse problemGround-motion inverse problem

AWMGroundMotionsSRM

Unified Structural RepresentationFaults Motions Stresses Anelastic model

2

AWP = Anelastic Wave PropagationAWP = Anelastic Wave Propagation

SRM = Site Response ModelSRM = Site Response Model

RDM

FSM

3

FSM = Fault System ModelFSM = Fault System Model

RDM = Rupture Dynamics ModelRDM = Rupture Dynamics Model

Invert

Other DataGeologyGeodesy

4

Physics-basedPhysics-basedsimulationssimulations

EmpiricalEmpiricalrelationshipsrelationships

Improvement Improvement of modelsof models

2

3

1

4

Seismic Hazard Analysis Seismic Hazard Analysis Computational PathwaysComputational Pathways

Unified Structural Representation

Tectonic models

Structural models

Community Fault Model Community Block Model

Crustal Motion Map

IMIM RupRupn,in,i

Intensity-MeasureIntensity-MeasureRelationshipRelationship

Earthquake-Earthquake-RuptureRuptureForecastForecast

€

Prob(IMT ≥ IML) =1− 1− Prob(IMT ≥ IML,Site | n,iRup ) * Prob( n,iRup )[ ]n=1

N ( i)

∑ ⎛

⎝ ⎜

⎞

⎠ ⎟

i=1

I

∏

Time SpanTime Span

Type, LevelType, Level

SourceSourceii

SiteSite

OpenSHAOpenSHAA Computational PlatformA Computational PlatformSeismic Hazard AnalysisSeismic Hazard Analysis

Pathway 1 OpenSHA Platform

Field, Jordan & Cornell (2003)

IntensityMeasures

Earthquake Forecast Model

AttenuationRelationship

1

Standardized seismic hazard analysisStandardized seismic hazard analysis

Ground motion simulationGround motion simulation

Physics-based earthquake forecastingPhysics-based earthquake forecasting

Ground-motion inverse problemGround-motion inverse problem

AWMGroundMotionsSRM

Unified Structural RepresentationFaults Motions Stresses Anelastic model

2

AWP = Anelastic Wave PropagationAWP = Anelastic Wave Propagation

SRM = Site Response ModelSRM = Site Response Model

RDM

FSM

3

FSM = Fault System ModelFSM = Fault System Model

RDM = Rupture Dynamics ModelRDM = Rupture Dynamics Model

Invert

Other DataGeologyGeodesy

4

Physics-basedPhysics-basedsimulationssimulations

EmpiricalEmpiricalrelationshipsrelationships

Improvement Improvement of modelsof models

2

3

1

4

Seismic Hazard Analysis Seismic Hazard Analysis Computational PathwaysComputational Pathways

SCEC Computations

• Terashake - simulate earthquakes in Southern California

• Cybershake - seismic hazard assessment

• Earthworks - simulate each observed earthquake

TS2.dyn.200m30x 256 procs, 12 hrs,

TG IA-64GPFS

GPFS

Datastar GPFS

Okaya200m Media

Okaya100m Media

100m Reformatting100m Transform100m Filtering

200m moment rate

HPSS

SAM-QFS

SDSC IA-64

TS2.dyn.100m10x 1024 procs, 35 hrs

Initial 200m Stress modify

Initial 100m Stress modify

TS2.wav.200m3x 1024 procs, 35 hrs

NCSA IA-64

Datastar p690

Datastar p655

Register to Digital Library

SRBVisualization

Analysis

TeraShake-2 Simulations

Network

TG IA-64GPFS-wan

NCSA-SAN

SDSC-SAN

Application Optimization

Significant Parallel Speedup of AWM Code (wave propagation mode)*

0

50

100

150

200

250

300

350

Processors

32

64

96

128

160

192

224

256

*run on IBM Power4, 32/64/96p data from 32-way 1.7GHz p690 nodes, 240p data from 8-way 1.5GHz p655 nodes

Speedup

Total Elapsed Time

Computing Time

Speedup

32 64 96 240

Source: Yifeng Cui, San Diego Supercomputer Center

Goal– Capability for physics-based probabilistic seismic hazard calculation in

Southern California, accounting for source complexity and 3D earth structure

Design– All significant sources from 2002 National Seismic Hazard Mapping Project

– Site-oriented Green functions optimized for 3D earth structure

– Full representation of source complexity by stochastic sampling

Plans– Phase I (2005): 6 sites, kinematic sources, low frequency (< 0.5 Hz)

– Phase II (2006): 625 sites, pseudo-dynamic sources

– Phase III (2007): Full map, dynamic sources

Requirements– 35,000 SU runtime per site & 10 TB data volume per site for Phases I & II

– 10 TB data volume must be accessible across grid during seismogram synthesis

CyberShake Platform

Visualization Applications and AnalysesVisualization Applications and Analyses

SCEC Portal (OGCE portlets)SCEC Portal (OGCE portlets)

Digital Library Services (seismogram extraction)Digital Library Services (seismogram extraction)

Data grid (Storage Resource Broker)Data grid (Storage Resource Broker)

Teragrid Network (TCP/IP communication)Teragrid Network (TCP/IP communication)

Hardware (distributed storage systems)Hardware (distributed storage systems)1

2

3

4

5

6

Vertical

integration

SCEC Digital LibraryA publication resource for earthquake science data

SCEC Digital Libraryhttp://www.sdsc.edu/SCEC/

SCEC Portal

• Original portal based on OGCE portlet technology

• Migrating to GridSphere portlet technology and JSR168 compliant portlets– Compatibility with GEON portlets– Compatibility with generic SRB portlets

• Interface between user access and data storage within the SRB data grid

Digital Library Components

• Organization structure– Categorized simulation data by experiment

• Metadata attributes– Query on input parameters

• Naming– Logical file names

• Security– One-time passwords for administrative access

Organization

Metadata

Metadata purposes•Support browsing

•Support queries

•Support provenance

•Support knowledge

Analysis Tools

• Seismogram extraction– 1-Terabyte data set per simulation– Data kept on disk– Backup copy on tape– Dynamically linked from portal to registered

simulations

Seismogram Extraction

Seismogram Extraction

Challenges

• Size of data– SCEC digital library holds over 130 Terabytes of

data– Replication of data needed to ensure preservation

• Planned analyses are substantially larger– Expect 600 Terabytes of simulation results

• Planned services (dynamic evaluation of each earthquake) will require automation– Dynamic creation of visualizations and access

services

Research Challenges• Data integrity

– Commodity bit error rates are designed for GB data sets– Replication, federation, automated validation

• Efficient access to large scale data– MPI-IO interface to SRB data grid (OSU)

• Efficient parsing of data structures– Integration of HDF5 technology with data grids– Integration of Data Format Description Language

• Integration with digital library and preservation standards– METS - Metadata Encoding and Transmission Standard– AIP - Archival Information Package

Storage Resource Broker

• Collection hierarchy– Schema extension– Bulk metadata loading– Query

• Distributed storage– Replication– Audit trails, quotas, checksums, versions– Access controls– Support for all types of storage systems

Conclusions• SCEC/CME is developing a digital library that holds the state-of-the-art

simulations of large earthquakes in Southern California

– Organizes simulation data, derived data products

– Integrating access to observational data as next step

• SCEC/CME collaboration and ETF have been critical to this success

– Large scale visualizations required Teragrid resources

– Data movement rates (10 TBs per day) require Teragrid networks

– Storage of results done on Teragrid storage systems

• Future opportunities

– Integration with GIS systems to composite views on data

– Interoperability between related NSF programs (GEON, NEESgrid)

– Integration of knowledge management systems on top of digital library

Visualizationshttp://visservices.sdsc.edu/projects/scec/

Visualizations

Visualizations

• Movies– Surface displacement magnitude

• Mapped on surface topography• Contoured

– Velocity magnitude– Peak velocity magnitude– Velocity components

• Images– Peak spectral acceleration

• http://visservices.sdsc.edu/projects/scec/

TeraShake

TeraShake

TeraShake