Hosting CLARIN services in the EGI cloud · PDF fileUbercloud 19/09/17 7 The EGI Service...
Transcript of Hosting CLARIN services in the EGI cloud · PDF fileUbercloud 19/09/17 7 The EGI Service...
www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme
of the European Union under grant number 654142
[email protected] http://go.egi.eu/cloud
Hosting CLARIN services in the EGI cloud
Gergely Sipos EGI Foundation Boris Parak
CESNET Willem Elbers CLARING ERIC
2 19/09/17
EGI: Advanced Computing for Research
3 19/09/17
EGI Membership • Majornationale-Infrastructures:22NGIs• EIRO:CERN(EuropeanIntergovernmentalResearchOrganisation)• Membershipfeessustainthefederationandcommunity
– EGIFoundation– Collaboration&federation
4 19/09/17
EGI Federation Largest distributed compute e-Infrastructure of the world
300+HTCproviders23Cloudproviders
650kCPUcores
285PBonlinestorage
1.7Millionjobs/day
2.6BillionCPUhours/
year
240VirtualOrganisaKons
>48000users
EGI Foundation
(Amsterdam)
5 19/09/17
International Partnerships
AfricaandArabiaCouncilforScienKficandIndustrialResearch,SouthAfrica
IndiaCentreforDevelopmentofAdvancedComp.
ChinaInst.OfHEPChineseAcademyofSciences
La/nAmericaUniversidadeFederaldoRiodeJaneiro
UkraineUkrainianNaKonalGrid
USA
Canada
AsiaPacificRegionAcademiaSinicaatTaiwan
6 19/09/17
EGI serves researchers and innovators
ESFRIs,FETflagships
Sizeofindividualgroups
Mul;na;onalcommuni;es,(e.g.H2020projects)
‘Longtailofscience’
WLCGELICTAELIXIREPOSBBMRIINSTRUCTCLARINEISCAT_3DDARIAHLOFARLifeWatchICOSEMSOCORBELENVRIplus…
VREprojectsOpenDreamKitWeNMRDRIHMVERCEMuGAgINFRACMMSTLSGCSuperSitesExploitaKonEnvironmentalsci.neuGRID…
PeachNoteCEBAGalaxyeLabSemiconductordesignMain-beltcometsQuantumpysicsstudiesVirtualimaging(LS)BovinetuberculosisspreadConvergentevol.ingenomesGeographyevoluKonSeafloorseismicwaves3DlivermapswithMRIMetabolicratemodellingGenomealignmentTapewormsinfecKononfish…
Industry,SMEs
AgroknowCloudEOCloudSMEEcohydrosgnubilaSinergiseSixSqTEISSTerradueUbercloud…
7 19/09/17
The EGI Service Catalogue www.egi.eu/services
8 19/09/17
Introduction to EGI cloud
9 19/09/17
EGI Cloud • Gridofclouds• Uniformeduserinterfaces• Harmonisedoperational
behaviour• Basedonopenstandardsand
opentechnologies• PrimarilyanIaaScloud,but
withcommunity-hostedPaaSandSaaSontop
• InfrastructureàAccessANDTechnologystackàDeploy
10 19/09/17
The current infrastructure Today:• 23providersfrom14NGIs
• 15OpenStack• 7OpenNebula• 1Synnefo
11 19/09/17
Typical usage models
• Computeanddataintensiveworkloads– Batchandinteractive(e.g.Jupyternotebooks)withscalableand
customizedenvironments
• ServiceHosting– Long-runningservices(e.g.webserver,database,applicationserver)
• Datasetsrepository– Storeandmanagelargedatasets(inastoragevolume)
• Disposableandtestingenvironments– Hosttrainingenvironments,testapplications
12 19/09/17
Computing models
HTCGridservice(e.g.clusters;HTC)
Commercialcloudservice(e.g.Azure)
EGIcloudservice(federatedIaaScloud)
• ForbatchcompuKng• Forembarrassingly
parallelapplicaKons• UsingmulKplesitesat
aKme
• ForinteracKveandbatchcompuKng
• ForservicehosKng• FlexibleOSandapplicaKon
use• Typicallyonesiteisused• Pay-as-you-go• Nousersupport(orcentral
paidservice)
• ForinteracKveandbatchcompuKng
• ForservicehosKng• FlexibleOSandapplicaKon
use• UsesingleormulKplesites
ataKme• VirtualOrganisaKonbased
access• Freeatthepointofuse(for
research)• Withlocalusersupport
13 19/09/17
Main resource about EGI Cloud
EGI Federated Cloud User Guides: • https://wiki.egi.eu/wiki/Federated_Cloud_user_support
14 19/09/17
CLARIN services in the EGI Cloud
15 19/09/17
Brokering resources for a community
Project/Communityrepresen;ngtheVO
Nego;ator Gridprovider
Cloudprovider
OperaKonLevel
Agreement
ServiceLevelAgreement
SaKsfacKonreview(every3/6/12months)
Storageprovider
Servicerequirements Condi;ons
Applic.provider
Performancereports
SupportTraining
Type,number,size,cost,availability,etc.
TriggertheprocesswithaservicerequestontheEGIwebsite
16 19/09/17
CLARIN cloud provider:
• National grid & cloud infrastructure including storage
• Multiple production IaaS clouds operated by CESNET
– MetaCloud (OpenNebula-based) • 2,900 CPU, ≈ 30 TB RAM1
– EGI Federated Cloud (also OpenNebula-based) • ≈300 CPU + 1,600GB RAM
– Full-fledged & certified FedCloud site – Numerous testbed deployments (also OpenNebula-based) – ownCloud personal cloud-based storage
17 19/09/17
CLARIN services in the CESNET cloud
1. CLARIN component registry (http://clarin.eu/cmdi), the backbone of the Component-based MetaData Infrastructure.
2. The CLARIN docker registry (http://docker.clarin.eu), which plays an important role in the CLARIN deployment workflow.
3. The CLARIN maven repository (http://nexus.clarin.eu), intensively used during CLARIN software development projects.
4. A grafana instance (http://metrics.clarin.eu), to visualize server status and metrics.
5. Document archive for the CLARIN ERIC central office.
18 19/09/17
Future outlook – The European perspective European Open Science Cloud
“The European Open Science Cloud (EOSC) is a vision for a federated, globally accessible,
multidisciplinary environment where researchers, innovators, companies and citizens can publish,
find, use and reuse each other's data, tools, publications and other outputs for research,
innovation and educational purposes.”
Credits: Open Science Policy Platform EOSC wg
à EOSCpilotH2020project:2017.Jan1–2018Dec31.à EOSC-hubH2020project:2018.Jan1–2020Dec31.
19 19/09/17
Next Community Event Digital Infrastructures for Research (DI4R)
hlps://www.digitalinfrastructures.euCo-locatedwithEOSCpilot1stStakeholderEngagementEvent(28-29Nov)
CALLFORCONTRIBUTIONISNOWOPEN!
20 19/09/17
Summary
• EGI Foundation: Cloud service broker • CESNET: Cloud service provider • CLARIN ERIC: Cloud customer • CLARIN community: Cloud user