HNSciCloud update @ the World LHC Computing Grid deployment board
-
Upload
helix-nebula-the-science-cloud -
Category
Technology
-
view
86 -
download
1
Transcript of HNSciCloud update @ the World LHC Computing Grid deployment board
HNSciCloud Update
WLCG GDB
12 July 2017Bob Jones
CERNBob.Jones <at> cern.ch
Helix Nebula – The Science Cloud
Helix Nebula – The Science Cloud with Grant Agreement 687614 is a Pre-Commercial Procurement Action funded by H2020 Framework Programme
HNSciCloud Joint Pre-Commercial Procurement
Procurers: CERN, CNRS, DESY, EMBL-EBI, ESRF, IFAE, INFN, KIT, STFC, SURFSaraExperts: Trust-IT & EGI.eu
The group of procurers have committed• Procurement funds• Manpower for testing/evaluation• Use-cases with applications & data• In-house IT resources
Resulting services will be made available to end-users from many research communities
Co-funded via H2020 Grant Agreement 687614
Total procurement budget >5M€
12/07/2017
HNSciCloud project phases
Preparation
•Analysis of requirements, current market offers and relevant standards
•Build stakeholder group
•Develop tender material
Implementation and sharing
Jan’16 Dec’18
Each step is competitive - only contractors that successfully complete the previous step can bid in the next
4/5/2016 4
4 Designs3 Prototypes
2 Pilots
Call-off Feb’17
Call-off Dec’17
Tender Jul’16
Phases of the tender are defined by the Horizon 2020Pre-Commercial Procurement financial instrument
What will be procuredA hybrid cloud platform for the European research community
7/12/2017 5
HNSciCloudPCP
Sou
rce:
Clo
ud
Co
mp
uti
ng
for
Go
vies
, DLT
So
luti
on
s,D
avid
Bla
nke
nh
orn
, V
an R
ista
uan
d C
aro
n B
eesl
ey
Combining services at the IaaS level to support science workflows
The R&D services to be developed are to be integrated withResources in data centres operated by the buyers group
GEANT network
Challenges
Innovative IaaS level services integrated with procurers in-house resources and public e-infrastructure as part of a hybrid cloud to support a range of scientific workloads
Compute and Storagesupport a range of virtual machine and container configurations including HPC working with datasets in the petabyte range
Network Connectivity and Federated Identity Managementprovide high-end network capacity via GEANT for the whole platform with common identity and access management
Service Payment Modelsexplore a range of purchasing options to determine those most appropriate for the scientific application workloads to be deployed
Bob Jones, CERN 6
The Pre-Commercial Procurement process
12/07/2017
12/07/2017
12/07/2017
12/07/2017
12/07/2017
HNSciCloud Prototype Activity Report Unclassified 12
OVERVIEW OF PROTOTYPE CLOUD PLATFORM
Integrated
Realistic
Minimally Intrusive
Scalable
Managed
Sustainable
Functional components to deliver complete data analysis platform…
HNSciCloud Prototype Activity Report Unclassified 13
OVERVIEW OF PROTOTYPE SOLUTION
Functional components to deliver complete data analysis platform…
HNSciCloud Prototype Activity Report Unclassified 14
OVERVIEW OF PROTOTYPE SOLUTION
Functional components to deliver complete data analysis platform…
HNSciCloud Prototype Activity Report Unclassified 15
OVERVIEW OF PROTOTYPE SOLUTION
Functional components to deliver complete data analysis platform…
HNSciCloud Prototype Activity Report Unclassified 16
OVERVIEW OF PROTOTYPE SOLUTION
Functional components to deliver complete data analysis platform…
Martin Gasthuber | Next Steps | 03/04/2017 | Page 17
test phases and resource ramp-up
TP1 TP2 TP3 TP4
2000C
200TB
100C 10TB
3500C
350TB
Activities in Test Phase 1 (TP1): April-June 2017
Establishing network connectivity via GEANT/NRENs,federated identity management via eduGAIN andcreating accounts for tenants
Preliminary testing of the developed services by the Buyers Group has now begun
12/07/2017
Test suiteCERN
• Edugain AAI
• CERN Benchmark Suite
• S3 Endpoint
• PerfSONAR Network
• VM Provisioning and Personalisation
• Grid Jobs in Docker
• Dockerized deployment of EOS+CERNBOX+SWAN
CNRS
• IAAS access via EduGain and local accounts
DESY
• HDF5_IO
EMBL
• CLOUD_BENCHMARK
• ELIXIR_AAI
• data_transparency
ESRF
• FDMNES
IFAE
• monitoring-accounting
• image creation and contextualization
INFN
• BelleII_SIM
• COMPASS/TGEANT_IO
• INFN/GPFS_AFM
• Dynfarm
KIT
• static-batch-extension
• dynamic-batch-extension
• AFM-deployment-test
SURFsara
• BBMRi_data
• dCache_io
• kubertest
12/07/2017
More details: https://cernbox.cern.ch/index.php/s/pjy4lg3cY46vbGt
Next Steps 20172nd progress review 4-5 September
Scale-up IaaS resourcesMore testing
3rd progress review 9-10 October (DESY)Live webcast demonstrationsFeature freezeScale-up IaaS resources (totalling 10k cores 1PB storage)More testing
End of prototype phase assessment 11-13 December
Selection of 2/3 prototypes for pilot phase
12/07/2017
Pilot Phase 2018
Kick-off event 6-8 Feb 2018 (INFN Bologna)
Progressively scale-up IaaS resourcestotalling 20k cores 2PB storage
Scalability testing
Deployment of applications
Assess procurement/payment models
12/07/2017
Growing the buyers group
Private sector
Gov. agencies
Additional research orgs.
Procuring beneficiaries of
EC project
The initial group of buyers is the set of research orgs. that committed their resources at the start of the project and became beneficiaries of the EC project
Goal is to progressively grow the buyers group by including more publicly funded research organisations: Early adopter group
<see attachment to indico page>
Draft document describesbenefits for the stakeholders,procurement process,steps for implementation
CERN, DESY, EMBL-EBI, ESRF, KIT, CNRS, INFN, PIC-IFAE, SURFsara, STFC
adopter group
Summary
3 prototypes (IBM, RHEA, T-Systems) are being deployed and tested by the Buyers Group
Testing and scaling will continue during 2017
2/3 prototypes will be selected for large-scale pilots in 2018
The Early adopter group will allow more research organisations to procure the resulting services
12/07/2017