Lightsource Lights Up The Night Sky At MahaNakhon's Official Opening
Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new...
Transcript of Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new...
![Page 1: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/1.jpg)
Mapping X-as-a-Service Model on a Petascale SupercomputingEcosystemsHPC-AI Advisory Council (HPCAIAC)Sadaf AlamChief Technology OfficerSwiss National Supercomputing CentreApril 1, 2019
![Page 2: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/2.jpg)
Interactive, On-demand and Bespoke Services on Hybrid Supercomputing and Cloud technologies (HPC Friendliness)19th International Workshop on Advanced Computing and Analysis Techniques in Physics ResearchSadaf AlamChief Technology OfficerSwiss National Supercomputing CentreMarch 13, 2019
![Page 3: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/3.jpg)
About Swiss National Supercomputing Centre (CSCS)
![Page 4: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/4.jpg)
Mission, Infrastructure and Services
§ CSCS develops and operates cutting-edge high-performance computing systems as an essentialservice facility for Swiss researchers https://www.cscs.ch
§ High Performance Computing, Networking and Data Infrastructure § Piz Daint supercomputing platform
§ 5000+ Nvidia P100 + Intel E5-2690 v3 nodes§ 1500+ dual-socket Intel E5-2695 v4 nodes§ Single network fabric (10s of Terbytes/s bandwidth)§ High bandwidth multi-Petabytes of scratch (lustre)
§ Storage Systems including SpectrumScale(10s of PetaBytes online and offline)
§ Services§ Computing services§ Data services§ Cloud services
HPCAIAC 2019 4
![Page 5: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/5.jpg)
Users and Customers§ User Lab
§ Allocation based on scientific merit
§ MeteoSwiss
§ CHIPP (Swiss Higher Energy Physics Community)
§ European research infrastructure projects
HPCAIAC 2019
Consolidation of customers and services
10s of Peta (1015) Floating-point operations/second)
100s of Peta (1015) Bytes storage
5
![Page 6: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/6.jpg)
SMC 2017
Piz Daint-–More Versatile Than Before
6
Supercomputing
Visualization
Data analysis
✙Pre-post processing
Data mover
Data Warp
Machine learning
Deep learning
2013
2016
à20
19
![Page 7: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/7.jpg)
HPCAIAC 2019 7
Current X-as-a-Service Model at CSCS (Discrete Systems)
Service Programmability
Perf
orm
ance
????
![Page 8: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/8.jpg)
HPCAIAC 2019
The largest data centre in Europe is located in a tiny village in Norway. Opening in the fourth quarter of 2018, the Kolos Data Centre will cover 6.5 mn sq. ft across four storeys, and is being billed as a hyper-scalable data centre, with plans to consume up to 1000 megawatts of power by 2027.
1 | China Telecom Data Centre, China2 | China Mobile Hohot, China3 | The Citadel, United States4 | Harbin Data Centre, China5 | Kolos Data Centre, Norway6 | Range International Data Centre, China7 | Switch SUPERNAP, United States8 | Dupont Fabros Technology9 | Lakeside Technology Centre, United States 10 | Tulip Data Centre, India
https://www.gigabitmagazine.com/top10/top-10-biggest-data-centres-world
8
![Page 9: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/9.jpg)
HPCAIAC 2019
https://newatlas.com/inside-google-data-centers/24654/
https://www.datacenterdynamics.com/news/alibaba-doubles-cloud-footprint-in-hong-kong/9
![Page 10: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/10.jpg)
HPCAIAC 2019 10
CapEx $$$ à $$$$$: Compute, storage, network, …
ssh, sbatch, scp, …IaaS, PaaS, SaaS
Data Centre Focus
Challenge to opportunities
Empowering users & customers
![Page 11: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/11.jpg)
HPCAIAC 2019
Current X-as-a-Service (Cloud & HPC Data Centre)
§ Performance & Scaling-as-a-Service§ Parallel computing§ Parallel file system technologies
(POSIX based)§ Bulk processing, scale out with fast,
integrated ecosystem§ High bandwidth networking
subsystems§ Internal and external connectivity for
high throughput data transfers§ …
§ Automation &Interactivity-as-a-Service§ IaaS (ownership infrastructure and
services)§ On-demand § High availability through service
migration§ Roles based access control§ Storage models for role based access
controls§ Isolation, security and QoS§ …
11
![Page 12: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/12.jpg)
LHC Workflow on Piz Daint à A Success Story (Bespoke Service)Input from Miguel Gila (CSCS)
![Page 13: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/13.jpg)
CERN Large Hadron Collider (LHC) Experiments
HPCAIAC 2019 13
The Large Hadron Collider (LHC) with its four detector experiments (ATLAS, CMS, LHCb and ALICE) sits in a 27-km-long (~17 miles) circular tunnel, 100m below the ground at the European Organization for Nuclear Research (Cern) on the French-Swiss border, north of Geneva.(cern.ch)
![Page 14: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/14.jpg)
HPCAIAC 2019 14
![Page 15: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/15.jpg)
Worldwide LHC Computing Grid (WLCG)
§ Tier 0 (CERN and Hungary)§ safe-keeping of the raw data (first copy=$$$$$$$$$$?) first pass reconstruction§ distribution of raw data § reconstruction output to the Tier 1s reprocessing of data during LHC down-times
§ Tier 1 (13 centers)§ safe-keeping of a proportional share of raw and reconstructed data§ large-scale reprocessing and safe-keeping of corresponding output§ distribution of data to Tier 2s § safe-keeping of a share of simulated data produced at these Tier 2s
§ Tier 2 (~ 150 centers)§ specific analysis tasks § proportional share of simulated event production and reconstruction
§ Tier 3§ Local access cluster, no formal agreement with WLCG
HPCAIAC 2019 15
http://wlcg-public.web.cern.ch/tier-centres
![Page 16: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/16.jpg)
HPCAIAC 2019 16
![Page 17: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/17.jpg)
HPCAIAC 2019 17
![Page 18: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/18.jpg)
HPCAIAC 2019 18
![Page 19: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/19.jpg)
Status (March 2019)
§ All experiments running on Piz Dainta supercomputing platform
§ No more operations on a dedicated cluster§ Service consolidation benefits for the users and customers
§ High throughput computing for HEP middleware & workflows transparently (for users) running on a Petascale system§ HPC friendliness?
§ Dedicated multi-year effort and collaboration between CSCS and our customers
HPCAIAC 2019 19
![Page 20: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/20.jpg)
HPCAIAC 2019 20
Results and Observations
CPU efficiency for various MC production campaignsthat run at CSCS and at other Grid sites.
![Page 21: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/21.jpg)
Bridging the Gap à Creating New Abstractions for Bespoke Service
§ Light-weight operating system (SLES based)§ Containers or other virtualization interfaces
§ Diskless compute nodes§ Exploit burst buffer or tiered storage hierarchies
§ Computing nodes connectivity (high speed Aries interconnect)§ Secure web services access with no address translations overhead
§ However …§ LHConCray i.e. mapping of bespoke services on Piz Daint
approach is resource (experts, time, etc.) intensive & may notscale
HPCAIAC 2019 21
![Page 22: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/22.jpg)
HPC Friendliness à Empowering Users and Customers
![Page 23: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/23.jpg)
Experiences & Expectations
§ Classical HPC users§ Data science HPC users§ Extreme data workflow /
experimental facilities
HPCAIAC 2019
#!/bin/bash -l #SBATCH --job-name=job_name#SBATCH --time=01:00:00 #SBATCH --nodes=2 #SBATCH --ntasks-per-core=2 #SBATCH --ntasks-per-node=12 #SBATCH --cpus-per-task=2 #SBATCH --partition=normal #SBATCH --constraint=gpu
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK export CRAY_CUDA_MPS=1 module load daint-gpu srun./executable.x
23
![Page 24: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/24.jpg)
Classification & Abstractions
§ Classic HPC use cases§ Performance & Scaling as a Service§ Batch processing§ Access to low level toolchains
§ Already friendly
§ Data Science HPC use cases§ Software as a Service § (Big) Data as a Service§ Interactivity§ Elasticity
§ Extreme Data Workflow use cases§ Workflow as a Service§ Automation as a Service§ Need privileged access to infrastructure services§ Need composable platform services
§ Need cloud++ delivery model or hybrid cloud & HPC to be friendly
HPCAIAC 2019
Interactive
On-demand
Bespoke
Performance
24
![Page 25: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/25.jpg)
Swissuniversities funded project with PSISELVEDAS (Services for Large Volume Experiment-Data Analysis utilizing Supercomputing and Cloud technologies at CSCS)
![Page 26: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/26.jpg)
HPCAIAC 2019
https://www.cscs.ch/publications/press-releases/2018/589/
Highlights:Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS)A total of 10 to 20 petabytes of data is produced every year A dedicated redundant network connection between PSI and CSCS, 10 GbpsCSCS tape library current storage capacity is 120 petabytes, can be extended to 2,000 petabytes By 2022, PSI will transfer around 85 petabytes of data to CSCS for archiving. Around 35 petabytes come from SwissFEL experiments, and 40 come from SLS.
https://www.psi.ch/media/overview-swissfel
26
![Page 27: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/27.jpg)
HPCAIAC 2019
Growth in Data Volume à Online & Offline Data Processing
User applies for beamtime
Prior to upgradeAfter the upgrade
27
![Page 28: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/28.jpg)
HPCAIAC 2019
Hybrid Cloud and HPC IT infrastructure
(experimental facility)
CSCS IT infrastructure (data centre)
Custom interfaces for compute & data services
Experimentalfacility users
IT infrastructure(experimental facility)
CSCS IT infrastructure (data centre)
Exp. facility community platforms
Infrastructure services
• AAI• Containers
IT infrastructure | Infrastructure services | Platforms | End users
Other IT infrastructure(experimental facility)
CSCS IT infrastructure (data centre)
Other community users
Other community platforms
Infrastructure services
• AAI• Containers
Today
Target
Future
IT infrastructure | Infrastructure services | Platforms | End users
Experimentalfacility users
Other IT infrastructure (data centre)
28
![Page 29: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/29.jpg)
Future Outlook
![Page 30: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/30.jpg)
Mapping X-as-a-Service (Hybrid Cloud and HPC)—SELVEDAS Action Items
§ On-demand§ Reservation service for HPC resources§ Batch with delegated, privileged access§ Coordination of experiments with access to
Petascale computing resources§ Bespoke
§ Offer Infrastructure-as-a-Service (IaaS), e.g. virtual machines
§ Privileged access without compromising performance and security
§ Web service access to HPC resources§ Interactive
§ Resource management and scheduling (batch and service-oriented)
§ Resource utilization metrics (policy)
HPCAIAC 2019
Making HPC ecosystems friendlier à empowering our users and customers
30
![Page 31: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/31.jpg)
HPCAIAC 2019 31
Mapping X-as-a-Service (Hybrid Cloud and HPC)
Service Programmability
Perf
orm
ance
TargetSoftware definedinfrastructure for
Performance
![Page 32: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/32.jpg)
HPCAIAC 2019 32
Invitation to SC19 Workshop (SuperCompCloud: Workshop on Interoperability of Supercomputing and Cloud Technologies)November 18, 2019 Denver, CO, USA
ssh, sbatch, scp, …IaaS, PaaS, SaaS
Data Centre FocusChallenge to opportunities
Empowering users & customers
![Page 33: Mapping X-as-a-Service Model on a Petascale Supercomputing ... · Archival storage for the new SwissFEL X-ray laser and Swiss Lightsource (SLS) A total of 10 to 20 petabytes of data](https://reader035.fdocuments.in/reader035/viewer/2022081522/5f12d3a15e5d581872174791/html5/thumbnails/33.jpg)
Thank you for your attention.