February 25, 2008 The Emerging Front Range HPC Collaboratory Dr. Rich Loft: [email protected]...
-
Upload
ethan-jennings -
Category
Documents
-
view
218 -
download
0
Transcript of February 25, 2008 The Emerging Front Range HPC Collaboratory Dr. Rich Loft: [email protected]...
February 25, 2008
The Emerging Front The Emerging Front Range HPC CollaboratoryRange HPC Collaboratory
Dr. Rich Loft: Dr. Rich Loft: [email protected]@ucar.edu
Director, Technology DevelopmentDirector, Technology Development
Computational and Information Systems LaboratoryComputational and Information Systems Laboratory
National Center for Atmospheric Research National Center for Atmospheric Research
February 25, 2008
Front Range HPC CollaboratoryComponents…
Facilities
HPC Systems
Grid Technology
Applications/Workflows
Science Collaborations
Science Drivers
Networking
February 25, 2008
The National Center for Atmospheric Research
Climate Modeling Expertise •Nobel Prize winning IPCC AR-4•Community Modeling Approach•Scalable Models
Grid Expertise•TeraGrid Resource Provider•Earth System Grid•Grid-BGC (BioGeoChemistry)
New Facility Plans (2012)•>1 PFLOPS•8 MW•>100 PB archive
High Performance Computing•40 years of experience•25 TFLOPS peak computers•5 Pbyte archive
February 25, 2008
Climate Modeling…Climate Modeling…
40 km version credit: J. Hack, NCAR
February 25, 2008
Climate of the last Millennium
Credit: Caspar AmmanNCAR
February 25, 2008
This is a Critical Moment in This is a Critical Moment in Climate Change Research…Climate Change Research…
Reproduce historical trends
Investigate climate change
Run IPCC Scenarios
Investigate solutions
Assess impacts
Simulate adaptation strategies
Work with energy industry
Before IPCCBefore IPCC AR4 AfterAR4 After2007
February 25, 2008
Mitigation: What Mitigation: What we do nextwe do next can can change the amount of warming change the amount of warming that occurs.that occurs.
Likelihood of warmingLikelihood of warming
(Knutti et al. 2005)(Knutti et al. 2005)
February 25, 2008
L. Buja Dimensions SlideL. Buja Dimensions Slide
February 25, 2008
Why High Resolution?Why High Resolution?
Ocean component of CCSM (Collins et al, 2006)
Eddy-resolving POP (Maltrud & McClean,2005)
February 25, 2008
The Case for an HPC CollaboratoryThe Case for an HPC Collaboratoryin the Western Interiorin the Western Interior
An integrated approach to An integrated approach to climate change will involveclimate change will involve– Climate modeling with Climate modeling with
regional capabilitiesregional capabilities– Mitigation efforts via, for Mitigation efforts via, for
example, example, carbon carbon sequestrationsequestration
Most sequestration reservoir Most sequestration reservoir formations in the Western formations in the Western InteriorInterior
Unconventional oil and gas Unconventional oil and gas locations in Western Interiorlocations in Western Interior
Regional climate change Regional climate change effects on the Great Plains effects on the Great Plains biomassbiomass
Half of all EPSCOR states are Half of all EPSCOR states are in the Western Interiorin the Western Interior
Power CostsPower CostsDOE FutureGenProject in Texas
February 25, 2008
Experimental Program to Stimulate Experimental Program to Stimulate Competitive Research (EPSCoR) Competitive Research (EPSCoR)
Most of the Western Interior is EPSCoR
February 25, 2008
National Security Agency - The power consumption of today's advanced computing systems is rapidly becoming the limiting factor with respect to improved/increased computational ability."
February 25, 2008
Moore’s Law = More Cores: Moore’s Law = More Cores: Quad Core “Barcelona” AMD Processor…Quad Core “Barcelona” AMD Processor…
Can 8, 16, 32 cores be far behind?
February 25, 2008
The Path to Petascale is Through The Path to Petascale is Through Massive Parallelism…Massive Parallelism…
Cray-2/8 1986 Blue Gene/L 2006130K CPU’s367 TFLOPS
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3.9 GFLOPS
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
February 25, 2008
February 25, 2008
Average Electrical Cost by State(1990 - 2003)
Data From Energy Information Administrationhttp://www.eia.doe.gov/cneaf/electricity/epa/average_price_state.xls
0
2
4
6
8
10
12
14
IDWA ORWY KY WV NE MT UT COVA IN OK WI MO MN ND SC TN IA AR NC KS SD FL AL MD TX MS DE NV GA LA DC OH
ILNM MI AZ PA NJ AK RI MA CT ME VT CA NH NY HI
(cents per kWh)
Commercial
Industrial
Cheap Power in the Western Interior
February 25, 2008
Front-Range Collaboratory Front-Range Collaboratory Resources Resources
Current SystemsCurrent Systems GECO (CSM+NREL)GECO (CSM+NREL)
– 2144, 2.66 GHz Intel CPU Dell Cluster (22.8 TFLOPS)2144, 2.66 GHz Intel CPU Dell Cluster (22.8 TFLOPS) NCARNCAR
– 2048, 700 MHz PwrPC CPU Blue Gene/L (5.7 TFLOPS)2048, 700 MHz PwrPC CPU Blue Gene/L (5.7 TFLOPS)
Planned Systems (FY2009)Planned Systems (FY2009) NCAR
– ~30 TFLOPS system upgrade – 1.5 PB HPSS-based archive
CU– ~100 TFLOPS MRI Proposal to NSF
Farther OutNCAR Supercomputing Center (NSC)
February 25, 2008
Blue Gene/L @ NCAR:Blue Gene/L @ NCAR:A platform for scalability researchA platform for scalability research
“Frost”
February 25, 2008
NCAR is part of the TeraGrid - NCAR is part of the TeraGrid - NSF’s HPC GridNSF’s HPC Grid
SDSCTACC
UC/ANL
NCSA
ORNL
PU
IU
PSC
TeraGrid is a facility that integrates computational, information, and analysis resources at the San Diego Supercomputer Center, the Texas Advanced Computing Center, the University
of Chicago / Argonne National Laboratory, the National Center for Supercomputing Applications, Purdue University, Indiana University, Oak Ridge National Laboratory, the Pittsburgh Supercomputing Center, and the National Center for Atmospheric Research.
NCAR
Caltech
USC-ISI
UtahIowa
Cornell
Buffalo
UNC-RENCI
Wisc
February 25, 2008
HPC Infrastructure End State HPC Infrastructure End State
Integrated HPC Resource& Visualization Systems
E N T E R P R I S E6 0 0 0
-
E N T E R P R I S E6 0 0 0
-
WAN Network
WAN Network
HPCStorageCluster
10Gb/s
MSSArchive
E N T E R P R I S E6 0 0 0
-
TG Shared Filesystems
E N T E R P R I S E6 0 0 0
-
Sciencegateways
E N T E R P R I S E6 0 0 0
-
Data Preservation
Switch E N T E R P R I S E
6 0 0 0
-
E N T E R P R I S E6 0 0 0
-
E N T E R P R I S E6 0 0 0
-
Data SAN
February 25, 2008
Compute cabinets
I/O nodes
Interactive nodes
Network
Switch cabinets
Future HPC Systems Are Likely to be Heterogeneous
I/O Subsystem
Front End
Switch
Compute Accelerators
Accelerator cabinets
February 25, 2008
NCAR Summer Internships in NCAR Summer Internships in Parallel Computational Science Parallel Computational Science (SIParCS)(SIParCS)
Open to:Open to:– Upper division undergradsUpper division undergrads– Graduate studentsGraduate students
In Disciplines such as: In Disciplines such as: – CS, Software EngineeringCS, Software Engineering– Applied Math, StatisticsApplied Math, Statistics– ES ScienceES Science
Support:Support:– Travel, Housing, Per diemTravel, Housing, Per diem– 10 weeks salary10 weeks salary
2008 application deadline: 2008 application deadline: – Feb 1stFeb 1st
Number of interns selected:Number of interns selected:– 8-108-10
Members of traditionally Members of traditionally underrepresented groups encouraged to underrepresented groups encouraged to apply.apply.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
http://www.cisl.ucar.edu/siparcs
http://www.cisl.ucar.edu/siparcs
February 25, 2008
Thanks! Any Questions?
February 25, 2008
Front Range HPC Collaboratory: Front Range HPC Collaboratory: A geographical viewA geographical view
Ring TopologyMultiple Network Providers
•Level3•McCleod•Platte River Power Auth.•ICG
Upgradable to multiple 10 Gb/s waves
NCAR:~30 TflopsFY2009
CU:100 TflopsFY2009
GECO:11.4 Tflops
NSC:1 Pflops2011
February 25, 2008
Over 3000 registered users make it the Over 3000 registered users make it the most used Meteorological Research model;most used Meteorological Research model;
Operational use by the National Weather Operational use by the National Weather Service, US Navy, US Army, USAF, South Service, US Navy, US Army, USAF, South Korean Meteorological Service, Indian Korean Meteorological Service, Indian Meteorological Department;Meteorological Department;
Special forecasts are made by NCAR over Special forecasts are made by NCAR over the Antarctic in support of the Antarctic in support of international operations there.international operations there.
36 h Reflectivity
Forecast
4 km WRF Model
RadarReflectivity
10 June 2003 12Z
State-of-the-art Coupled Climate System ModelState-of-the-art Coupled Climate System Model Open Source Code and freely available data Open Source Code and freely available data Significant development collaborations with:Significant development collaborations with:
– 22 US universities22 US universities– US Department of Energy (DOE)US Department of Energy (DOE)
LANL, LLNL, LANL, ORNL, ANL, LBL, NERSCLANL, LLNL, LANL, ORNL, ANL, LBL, NERSC– NASANASA
Largest contributor to the 2007 IPCC AR4Largest contributor to the 2007 IPCC AR4 – 1.4° resolution1.4° resolution– 11,000 model years simulated >100 TBytes 11,000 model years simulated >100 TBytes – of output dataof output data
WRF: Weather and Research Forecast ModelWRF: Weather and Research Forecast Model
CCSM: Community Climate System ModelCCSM: Community Climate System Model
NCAR’s Community Model ApproachNCAR’s Community Model Approachto Enabling Collaborative Researchto Enabling Collaborative Research
February 25, 2008
IPCC
February 25, 2008
Atmospheric Resolution:Number of Northern Hemisphere Cyclones
T255
ERA
T159
T95
Credit: Jung et al. 2006