Status and Perspective of INFN scientific Computing
A. ZoccoliINFN and University of Bologna
EU-T0 Meeting Bologna February 8 2018
OUTLINE
2
Organization and current status
Future challenges and requests
Guidelines and actions for future
developments
INFN Computing Organization
1 National Computing Center (CNAF in Bologna) hosting the WLCG Tier-1, where is concentrated the major part of the personnel
9 Tier-2 centers on the Italian territory sitting mainly in University Physics Departments
C3S “Comitato di Coordinamento attività Calcolo Scientifico INFN”. Mandate: make proposals for research and developments for scientific computing, including the development of a shared computing infrastructure, to support researchers primarily from INFN, but also from different research institutions.
3
Computing for experimental physics
4
Tier1:
•200 kHS06 CPU power equivalent to
20000 cores
•19 PB disk space
•57 PB tape library
10 Tier2:
•240 kHS06 CPU power equivalent to
24000 cores
•18 PB disk space
Network provided by GARR Consortium
(10-100 Gbps), under upgrading
Major part of the computing centers are multidisciplinary (HEP, astroparticle, neutrino, etc. )
Resources distribution
5
Italian contribution 8%
In the world WLCG
In Italy Half of the resources @ Tier1
Very effective infrastructure for LHC
data analysis !
Computing for theoretical physics
6
APE100
APEmille
apeNEXT
FERMI
GALILEO
MARCONI
TH-cluster@CNAF
clusters (BA-CT-MIB-PI)
HPC@INFN
(2017)
Agreement INFN – CINECA valid for 3 years:
use of 6% MARCONI + 15 Mcorehours
GALILEO (~1.4 Pflops)
#1
#500
Top500
24
Utilizzo risorse HPC dell’INFN al CINECA da
Settembre 2012 a oggi (*)
LATTICE
GENREL
NUCL-TH
FIELDTURBBIOPHYS
COND-MATOTHERS
FERMI- GALILEO- MARCONI
(*) in unità core BG/Q FERMI
Use of INFN resources @ CINECA
from 2012
76.6%
10.2%
Currently exploiting
CINECA, the national
HPC computing center.
Costs & manpower
7
Item Cost (M€)
CPU, disk and tape 4
HPC@CINECA 0.5
Electric power 2
Network(@GARR) 5
Total/year 11.5
Personnel
Infrastructure Middleware
and software
50 FTE 30 FTE
Budget
Future needs
LHC resources for High-Lumi
9
By assuming:
•Trigger rate 5-10 kHz, factor10 wrt today
•Expected Pileup ~140-200 ⇒ complexity of events increased by a factor 10
Back on the envelope evaluation of resources needs = 100x wrt
today !!
LHC resources for High-Lumi - 2
10
Expected mitigation effects:
Technological evolution for CPU and storage: +20%/y
performances 6x in 10 years
Computing Models Optimization (CPU: exploitation of parallel
architectures) and of the trigger scheme (raw data vs AOD)
CPU: 6x (CMS) e 4x (ATLAS)
Disk space: 4x (CMS) e 7x(ATLAS)
Current estimated gap:
Future Computing Resources
11
Googlesearches98PB
LHCSciencedata
~200PBSKAPhase1–
2023~300PB/yearsciencedata
HL-LHC– 2026~600PBRawdata
HL-LHC– 2026~1EBPhysicsdata
SKAPhase2–mid-2020’s~1EBsciencedata
LHC– 201650PBrawdata
Facebookuploads180PB
GoogleInternetarchive~15EB
Yearlydatavolumes
Future Astroparticle experiments (SKA, CTA, Euclid, …) will
produce unprecedented amount of data.
Future Computing Resources
12
The needs for ExaScale systems in science
HPC is mandatory to compare
observations with theoretical models
HPC infrastructure is the theoretical
laboratory to test the physical
processes.
Let ’s talk of Basic Science...High Energy & Nuclear Physics
LQCD (again...), Dark-energy and dark matter, Fission/ Fusion react ions
(ITER)
Facility and experiments designEffective design of accelerators (also for medical Physics, GEANT...)
Astrophysics: SKA, CTA
...
Life sciencePersonal medicine: individual or genomic medicine
Brain Simulation < – HBP (Human Brain Project) flagship project
P. V icini ( IN F N Rom e) E xaN eSt st at us L N GS, M ay 22-26, 2017 2 / 50
HEP and astroparticle
physics
Human brain
Biology
Personalized medicine &
genomics
Weather predictions
Climate change studies
• Material studies
• Smart manufacturing &
Industry 4.0
• IoT
• SmartC
Impact on private sector
Huge resources demand resulting in requests of HPC and Big Data
management will come from many different research fields in the
next years:
Future developments
Guidelines & Actions
14
• Evolution of the Middleware
• Evolution of the WLCG infrastructure
• Cooperation and synergy with the European context
(ESOC + EuroHPC)
• Optimization of the INFN e-infrastructure for HEP and
other disciplines
• Integration of the Italian HTC and HPC infrastructure
• Search a new location for the Tier-1
• Test beds for the use of the INFN e-infrastructure from
private companies
Middleware evolution
15
INFN is deeply involved in the transition to CLOUD for example with the
H2020 project:
INDIGO-DataCloud: INtegrating Distributed data Infrastructures for Global
ExplOitation (11,3 M€) 26 partners PI D. Salomoni (INFN)
Computing
&
data center
GRIDComputing
&
data center
Experiments
data
Network
Computing
&
data center
Computing
&
data centerExperiments
data
Network
Today
CLOUD
TomorrowYesterday
Downloads from INDIGO Repository
•ElectricIndigo
• ElectricIndigo6(https://www.indigo$datacloud.eu/service$component):
•
•
•
•
The Horizon2020 eXtreme DataCloud – XDC
project aims at developing scalable
technologies for federating storage resources
and managing data in highly distributed
computing environments, as required by the
most demanding, data intensive research
experiments in Europe and worldwide.
D. Salomoni
16
D. CEsini
Optimization of the INFN e-infrastructure for HEP and other disciplines
17
• INFN started a survey of its computing infrastructure in order to
evaluate the status, the sustainability and the evolution
• INFN signed a MOU started a collaboration with INAF to expand and
exploit a common infrastructure. First action is hosting data and provide
computing resources to the CTA experiment
• INFN started a collaboration with ASI to host data of the Copernicus
and CosmoSkyMed satellites and to provide computing resources to the
relevant interested communities
30 e-Infrastructures
Big
(T1+T2+CS+TS)
Small
10 20
95% Resources 5% Resources
Tests on flexible use of the INFN infrastructure
18
The goal: test a flexible use of the infrastructure by using remote
resources at Tier-1, by using in a transparent way CPU at Tier-2
Recas in Bari
T1 Bari600 km
It is working…
Performed tests also on small scale with private cloud providers:
ARUBA e AZURE (microsoft)
Integration of the Italian HTC and HPC infrastructure
19
• INFN started a project with CINECA in order to integrate them to
provide services to:
• Institutional basic and applied research
• Proof of concept and innovation for private organizations and
industries
DATA
Integrated Research Data Infrastructure
Network
Search for a new location for the Tier-1
20
ECMWF center will be hosted in Bologna from
2019 in the Tecnopolo area.
Possibility to host in the area also:
INFN Tier-1
Cineca computing center
Already allocated 40 M€ for the
Italian government to refurbish the
area. Looking for extra budget for IFN
& CINECA
The goal: provide a new location for the INFN Tier-1 to take intoaccount future expansion.
IPCEI project ICT-11 call
21
Goal: build an European Data Infrastructure (with HPC and Big Data resources) based on the existing e-infrastructures for research to provide services also for Public and Private Partnership
Partners: France, Spain, Luxemburg, Italy
Test beds with involvements of private companies:
Space, Smart Manifacturing, Materials, Personalized medicine
Modular e-infrastructure to be realized at European, national or regional level, which may serve different fields
Credits
22
The presentation is based on the work performed by the
C3S Committee.
Thanks to
all the people helped in the preparation of the slides: D.
Lucchesi, T. Boccali, S. Campana, L. Cosmai, D. Salomoni,
P. Vicini …….
Conclusions
23
In the next years we have to cope with un
unprecedented amount of data coming from
many different fields (not only HEP)
INFN e-infrastructure is in a transition phase
We are actively exploring the most suitable
solutions for our future
Backup slides
External funds
25
Anno Tipo Progetto Budget
Totale (M€)
Budget
INFN (M€)
2015 H2020 INDIGO-DataCloud 11.1 2.1
2015 FISR High performance data
network
12.5 12.5
2016 H2020 ExaNeSt 8 0.7
2017 H2020 EOSCpilot 10 0.3
2017 H2020 EOSC-HUB 30.0 1.8
2017 H2020 Extreme DataCloud 3.1 0.6
2017 H2020 Deep Hybrid DataCloud 3.0 0.4
2017 H2020 EuroEXA 20 0.7
Totale 97.7 19.1
2018 MIUR PON 15 ?
Toward a Cloud based system
26
EOSC-hub Data and sertvices for EOSC
PI: EGI INFN: L. Gaido
XDCeXtreme DataCloud
Development of technologies for storage resources
federation and for data management
PI: INFN D. Cesini
DEEP HybridDataCloudDesigning and Enabling
E-Infrastructures for intensive
Processing in a Hybrid
DataCloud
Services and support for intensive computing
for different disciplines
PI: CSIC (Spain) INFN: G. Donvito
Definition of the EOSC governance
Realization of demonstrators
PI:UK INFN: D. Salomoni
INFN projects toward HPC exascale
27
European Exascale System Interconnection Network
& Storage
PI: Foundation for Research & Technology, GR
INFN: P.Vicini
ExaNeSt: FETHPC 2014
ExaNeSt: European ExascaleSystem Interconnection Network &Storage
EU Funded projectH2020-FETHPC-1-2014
Duration: 3 years (2016-2018).Overall budget about 7 MEuro.
Coordination FORTH(Foundation for Research &Technology, GR)
12 Partners in Europe (6industrial partners)
P. V icin i ( IN F N Rom e) E xaN eSt st at us L N GS, M ay 22-26, 2017 13 / 50
ExaNeSt
Co-designed Innovation and System for Resilient Exascale
Computing in Europe: From Applications to Silicon
PI: Institute of communication and computer systems, GR
INFN: P. Vicini
Acquisition and exploitation of many core "next gen” for
INFN HPC e HTC infrastructures
P. Vicini et al.
Progetto CIPE
Theoretical physics HPC requests
28
45
Stima delle necessità per la comunità HPC
risorse di calcolo
adeguato supporto per postdoc grant e per
organizzazione di workshop e meeting
4
computat ional physicists. New agreements established with the Italian Nat ional Supercomput ing Cent re (CINECA)have made larger comput ing resources available while two special projects (SUMA first , and then the so-called CIPE-project ) have helped establish a more cohesive computat ional community, offering Post -Doc posit ions at the junct ionpoint between theoret ical physics, algorithm development and code opt imizat ion, as well as a number of venues toshare ideas and expert ise in HPC comput ing. These investments have allowed INFN teams to re-establish a leadingrole in many research areas and have produced virtuous side-effects e.g., allowing improved access of Italian theoret icalcomputat ional groups to the EU supported PRACE programme [2] and have also increased the possibility to influencemajor decisions within internat ional collaborat ions.
It is crucial that this posit ive t rend be maintained and possibly further enhanced in the foreseeable future. Theseissues have been discussed in a meet ing held in Rome on February 17th, 2017 [3]. The present document is a followup of that meet ing and also of a similar paper writ ten in late 2014; it describes in full details the likely evolut ion of thetheoret ical computat ional community in the next few years and quant ifies the investments (financial and otherwise)needed in the t ime frame 2018-2020 to ensure that the current level of excellence can be retained or improved.
Computat ional theoret ical physics covers several research areas in the INFN theoret ical group:
I High Energy Physics - Lat t ice;
I I High Energy Physics - Phenomenology;
I I I General Relat ivity, Cosmology and Ast ropart icle physics;
IV Nuclear Theory;
V Fluid Dynamics;
VI Quant itat ive Biology;
VI I Disordered Systems;
VI I I Condensed mat ter;
with many research projects and more than 100 researchers involved. Research areas I to IV, clearly of st rategicinterest for INFN, have used in the last 5 years approximately 90% of the overall available comput ing resources,and research areas V to VI I I, while using only 10% of the available resources, offer an invaluable contribut ion to amult i-faceted and culturally wide research environment . Moreover, computat ional theoret ical physics in the next fewyears has the potent iality to develop synergies and collaborat ions with experimental areas of interest to INFN, bothas a theoret ical support to experiments and in the sharing of comput ing techniques.
The following sect ions of this document i) describe in details the scient ific perspect ives of these research areas inthe t ime frame of the next three to five years ii) quant ify the computat ional resources that are needed to successfullypursue the envisaged research programmes in the t ime window 2018-2020 and iii) present a number of suggest ions onthe act ion that INFN should undertake to support the computat ional community in this t ime frame. The rest of thisExecut ive Summary briefly lists our requests and suggest ions.
2018 2019 2020
LGT: hadron physics 54 108 180LGT: QGP and BSM 207 432 648LGT: flavor physics 117 234 387
Colliders Phenomenology 1 2 3General Relat ivity 142 182 227Cosmology and Ast ropart icle Physics 3 4 6
Nuclear Theory 18 27 36
Fluid Dynamics 50 80 110Quant itat ive Biology 9 18 27
Disordered systems 4 6 8
Condensed mat ter 2 4 6
G r and Tot al (M cor e-h) 607 1097 1638
G r and Tot al (Eq. P fl ops) 4.6 8.4 12.5
TABLE I: Est imate of the yearly comput ing requirements of several areas of computat ional physics; units areMcorehours. In the last line, the grand total is converted in PFlops, using as reference the core of the Intel SkyLake
processor with a nominal peak performance of 67 GFlops.
HEP-LAT74%
HEP-PH0%
GENREL14%
ASTRO-PH0%
NUCL-TH2%
FLU IDS7%
BIOPHYS2%
DISORDERED1% COND-MAT
0%
Requested resources
Possible evolution of the WLCG infrastructure
29
Commercial Clouds and
HPC centers when
available (elastic model)
1-10 Tb/s
DC
DC DCCompute
Compute
Cloud
access
for
Users
Fast network (∼Tb/s)
Data Centers (DC): host the major part of
tape and disk space with small CPU (core
of the infrastructure)
Computing Centers, withCPU and
disk space
Cooperation and synergies with the European context
30
• In 2015 launch of the European Open Science Cloud (EOSC) initiative to
provide an e-infrastructure for all European research communities.
• In 2016 signature of the EuroHPC agreement to provide a 10 years European strategy toward HPC exascale machines
EOSC
EDI
Top Related