An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf...
-
date post
20-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf...
![Page 1: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/1.jpg)
An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster
PICASso Mini-CourseOctober 18, 2006
Curt Hillegas
![Page 2: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/2.jpg)
Introduction
• SGI Altix - Hecate• IBM Blue Gene/L – Orangena• Dell Beowulf Cluster – Della• Storage• Other resources
![Page 3: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/3.jpg)
TIGRESS High Performance Computing Center
TerascaleInfrastructure forGroundbreakingResearch inEngineering andScience
![Page 4: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/4.jpg)
Partnerships
• Princeton Institute for Computational Science and Engineering (PICSciE)
• Office of Information Technology (OIT)• School of Engineering and Applied
Science (SEAS)• Lewis-Sigler Institute for Integrative
Genomics• Astrophysical Sciences• Princeton Plasma Physics Laboratory
(PPPL)
![Page 5: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/5.jpg)
SGI Altix - Hecate
• 64 1.5 GHz Itanium2 processors• 256 GB RAM (4 GB per
processor)• NUMAlink interconnect• 5 TB local disk• 360 GFlops
![Page 6: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/6.jpg)
![Page 7: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/7.jpg)
SGI Altix – Itanium 2
• 1.5 GHz• 4 MB L3 Cache
– 256 KB L2 Cache– 32 KB L1 Cache
![Page 8: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/8.jpg)
SGI Altix - NUMAlink
• NUMAlink 4• 3.2 GB/s per direction• Physical latency – 28 ns• MPI latency – 1 s• Up to 256 processors
![Page 9: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/9.jpg)
SGI Altix - Software
• SLES 9 with SGI ProPack– 2.6.5-7.252-sn2 kernel
• Intel Fortran compilers v8.1• Intel C/C++ compilers v8.1• Intel Math Kernel Libraries v7• Intel vtune• Torque/Maui• OpenMP• MPT (SGI mpich libraries)• fftw-2.1.5, fftw-3.1.2• hdf4, hdf5• ncarg• petsc
![Page 10: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/10.jpg)
IBM Blue Gene/L - Orangena• 2048 700 MHz Power4 processors• 1024 nodes• 512 MB RAM (256 MB per
processor)• 5 Interconnects including a 3D
torus• 8 TB local disk• 4.713 TFlops
![Page 11: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/11.jpg)
![Page 12: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/12.jpg)
IBM Blue Gene/L – Full system architecture• 1024 nodes
– 2 PowerPC 440 cpus– 512 MB RAM– 1 rack– 35 kVA– 100 kBTU/hr
• 2 racks of supporting servers and disks– Service node– Front end node– 8 storage nodes– 8 TB GPFS storage– 1 Cisco switch
![Page 13: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/13.jpg)
IBM Blue Gene/L
![Page 14: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/14.jpg)
![Page 15: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/15.jpg)
IBM Blue Gene/L - networks• 3D Torus network• Collective (tree) network• Barrier network• Functional network• Service network
![Page 16: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/16.jpg)
IBM Blue Gene/L - Software• LoadLeveler (coming soon)• mpich• XL Fortran Advanced Edition V9.1
– mpxlf, mpf90, mpf95• XL C/C++ Advanced Edition V7.0
– Mpcc, mpxlc, mpCC• fftw-2.1.5 and fftw-3.0.1• hdf5-1.6.2• netcdf-3.6.0• BLAS, LAPACK, ScaLAPACK
![Page 17: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/17.jpg)
IBM Blue Gene/L – More…• http://orangena.Princeton.EDU• http://orangena-sn.Princeton.ED
U
![Page 18: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/18.jpg)
Dell Beowulf Cluster - Della• 512 3.2 GHz Xeon processors• 256 nodes• 2 TB RAM (4 GB per processor)• Gigabit Ethernet• 64 nodes connected to
Infiniband• 3 TB local disk• 1.922 TFlops
![Page 19: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/19.jpg)
![Page 20: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/20.jpg)
Dell Beowulf Cluster –Interconnects
• All nodes connected with Gigabit Ethernet– 1 Gb/s– MPI latency ~ 30 s
• 64 nodes connected with Infiniband– 10 Gb/s– MPI latency ~5 s
![Page 21: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/21.jpg)
Dell Beowulf Cluster - Software• Elders RHEL 4 based image
– 2.6.9-42.0.3.ELsmp kernel• Intel compilers• Torque/Maui• OpenMPI-1.1• fftw-2.1.5, fftw-3.1.2• R-2.1.3• MatlabR2006a
![Page 22: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/22.jpg)
Dell Beowulf Cluster – More…
• https://della.Princeton.EDU• https://della.Princeton.EDU/gang
lia
![Page 23: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/23.jpg)
Storage
• 38 TB delivered• GPFS filesystem• At least 200 MB/s• Installation at the end of this
month• Fees to recover half the cost
![Page 24: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/24.jpg)
Getting Access
• 1 – 3 page proposal• Scientific background and merit• Resource requirements
– # concurrent cpus– Total cpu hours– Memory per process/total memory– Disk space
• A few references• [email protected]
![Page 25: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/25.jpg)
Other resources
• adrOIT• Condor• Programming help
![Page 26: An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.](https://reader035.fdocuments.in/reader035/viewer/2022062714/56649d485503460f94a24596/html5/thumbnails/26.jpg)
Questions