Cray X1 and Black Widow at ORNL Center for Computational Sciences
description
Transcript of Cray X1 and Black Widow at ORNL Center for Computational Sciences
![Page 1: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/1.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s
Cray X1 and Black Widow at ORNL Center for Computational Sciences
Buddy BlandCCS Director of Operations
SOS7 WorkshopMarch 5, 2003
![Page 2: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/2.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Cray X1 Overview
Cray X1 is the commercial name for the “SV2” project that Cray has been building for the NSA for more than 4 years.
Combines multi-streaming vector processors with a memory interconnect system similar to the T3E.
ORNL is evaluating the X1 for the Office of Advanced Scientific Computing Research
Specific details of the configuration are on Rolf’s web page.
![Page 3: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/3.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Q1: What is unique in structure and function of your machine?
System Cray X1 Power4Red Storm
HP Long's Peak & Quadrics
Processor Performance (GF) 12.80 5.20 4.00 6.40
Memory Bandwidth (B/F) 3.00 1.061.061.33* 1.00
Interconnect Bandwidth (B/F) 1.00 0.19
0.691.6* 0.19
Balance! * Corrections by Jim Tomkins
![Page 4: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/4.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Center for Computational SciencesCray X1 System
Picture of the Cray X1 at the factory awaiting shipment to ORNL
Delivery scheduled for March 18th
32 processor, liquid-cooled cabinet
128 GB memory 8 TB disk
![Page 5: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/5.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Phase 1b – Summer 2003
256 Vector Processors
1 TB shared memory
32 TB of disk space
3.2 TeraFLOP/s
![Page 6: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/6.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Cray X1/Black Widow4 phase evaluation and deployment
1Q2003 2Q2003 3Q2003 4Q2003 1Q2004 2Q2004 3Q2004
40TF (3200 CPU)8TF (640 CPU)3TF (256 CPU)Simulator
Phase 1 Phase 2 Phase 4
Phase 1
Phase 2
Phase 3
4Q2004 4Q2005 1Q2006
Phase 3
120TF
Phase 4
BW architecture per DOE apps
8.192TF, 2.621TB, 24TB
120TF, 40TB, 400TB
40.96TF, 13.107TB, 102TB
3.2TF, 1TB, 20TB
Currently Funded
![Page 7: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/7.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Q2: What characterizes your applications?
“Every day the same thing, variety”
- Yosemite Sam
![Page 8: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/8.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s CCS is focused on capability computing for a few applications
SciDACAstrophysics
Genomesto Life
Nanophase Materials SciDAC Climate
SciDACChemistry
SciDAC Fusion
![Page 9: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/9.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Climate (CCSM) simulation resource projections
At current scientific complexity, one century simulation requires 12.5 daysSingle researcher transfers 80Gb/day and generates 30TB storage each year
Machine and Data Requirements
3
6.6
14.5
31.9
70.3
154
340.1
750
1
2.2
4.8
10.6
23.4
51.5
113.3
250
1
10
100
1000
dyn
veg
trop
chem
istry
biogeo
chem
inter
active
strat c
hem
eddy
reso
lv
cloud
reso
lv
Years
Tflops
Tbytes
Science drivers: regional detail / comprehensive model
• Blue line represents total national resource dedicated to CCSM simulations and expected future growth to meet demands of increased model complexity
• Red line shows data volume generated for each century simulated
CCSM Coupled Model Resolution Configurations: 2002/2003 2008/2009Atmosphere 230kmL26 30kmL96Land 50km 5kmOcean 100kmL40
10kmL80Sea Ice 100km 10kmModel years/day 8 8National Resource 3 750(dedicated TF)Storage (TB/century) 1 250
![Page 10: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/10.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s GTL resource projections
Biological Complexity
ComparativeGenomics
Constraint-basedflexible docking
1000 TF
100 TF
10 TF
1 TF*
Constrained rigid
docking
Genome-scale protein threading
Community metabolic regulatory, signaling simulations
Molecular machine classical simulation
Protein machineInteractions
Cell, pathway, and network
simulation
Molecule-basedcell simulation
*Teraflops
Current U.S. computing
![Page 11: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/11.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Q3: What prior experience guided you to this choice
Workshops with vendors and users IBM, Cray, HP, SGI
Long experience in evaluation of new systems
![Page 12: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/12.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s Sustained performance of 0.02TF for climate calculations on IBM Power4
The Community Climate System Model (CCSM) is fully-coupled, global climate model that provides state-of-the-art computer simulationsof Earth’s past, present, and futureclimate states
Scaling to 352 processors is expected to be possible with Federation interconnect and recent improvements in software engineering
T42 with 1 degree ocean Memory requirements: 11.6 GB 96 IBM Power4 processors Processor speed: 5.2 GF Total peak: 500 GF Sustained performance: 16.2 GF 3.2% of peak 7.9 years simulated per day
![Page 13: Cray X1 and Black Widow at ORNL Center for Computational Sciences](https://reader034.fdocuments.in/reader034/viewer/2022051622/56814f49550346895dbcee5c/html5/thumbnails/13.jpg)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Cen
ter
for
Co
mp
uta
tio
nal
Sci
ence
s
Questions ?