Www.eu-eela.eu E-science grid facility for Europe and Latin America EELAs:the seed LGI / NGIs:the...
-
Upload
aubrie-dean -
Category
Documents
-
view
215 -
download
0
Transcript of Www.eu-eela.eu E-science grid facility for Europe and Latin America EELAs:the seed LGI / NGIs:the...
www.eu-eela.eu
E-science grid facility forEurope and Latin America
EELAs: the seedLGI / NGIs: the goal
Bernard M. Marechal (EELA-2 Project Coordinator)CETA-CIEMAT & UFRJ (Spain & Brazil)
CONAIS 2009 Villahermosa (Mexico), 23.09.2009
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 2www.eu-eela.eu
Outline
• Short EELA & EELA-2 story
• The EELA-2 model for long-term sustainability
• Proposal for future activities
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 3www.eu-eela.eu
Computationally intensive research
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 4www.eu-eela.eu
Computationally intensive research
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 5www.eu-eela.eu
Computationally intensive research
Virtual Organisations
E-Infrastructure, i.e.
DCI (Grid, Clouds,...)
Applications
Data
Instruments
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 6www.eu-eela.eu
The beginning
• Several meetings (Rio de Janeiro, CERN, Itacuruca) in 2004, foreseeing a EU–LA Grid Project supported by EC
• EELA proposal prepared, submitted and approved in 2005
• EELA worked hard and well during 2 years (2006 – 2007)
• EELA-2 prepared & submitted in 2007 and approved in 2008
• EELA-2 will end on 31 March 2010
• EC support: fade out in the future
• LGI / NGIs: fade in mandatory to support e-Science in LA on the long term
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 7www.eu-eela.eu
Projects in numbers
EELA-2 (CP-CSA under EU FP7) EELA (SSA under EU FP6)
E-infrastructure shared between Europe and Latin America
• EC support: 1.7 M€
• CIEMAT extra support: 0.4 M€
• 10 Countries (3 in Europe)
• 2 International Organisations
• 20 Members (7 in Europe)
E-science grid facility for Europe and Latin America
• EC support: 2.1 M€
• CETA-CIEMAT extra funds: 0.3 M€
• Currently 16 Countries (11 in LA)
• 2 new countries (Panama & Uruguay)
• 1 International Organisation (CLARA)
• Currently 78 Members (62 in LA)
• 32 Institutions joining (31 in LA)
• Currently 13 JRUs (9 in LA)
• 4 new JRUs (3 in LA)
At the final review EELA was awarded the highest EC rank:
“Good to excellent project”
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 8www.eu-eela.eu
Countries and Resources Centres
France
Ireland
Italy
Portugal
Spain
ArgentinaBrazilChileColombia CLARA (International)CubaEcuadorMexicoPeruVenezuela
New
• Panama• Uruguay
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 9www.eu-eela.eu
Joint Research Units in EELA-2• A Joint Research Unit (JRU) is a partnership for e-Science Collaboration, with no no
legal statuslegal status as such, formed between entities of the same nationality and meeting the following conditions:
– scientific and economic unity (internal MoU in place);– lasting a certain length of time (not linked to a particular project);– recognised by a public authority (minister, secretary of state, etc.).
One of the major
successes o
f
EELA-2
Country # of JRU members in total # of JRU members in EELA-2
Argentina 7 7
Brazil 18 14
Chile 5 5
Colombia 9 5
Ecuador 5 5
France 33 3
Ireland 18 1
Italy 14 2
Mexico 9 8Panama 3 3
Peru 7 7
Portugal 5 3
Spain >50 7
Venezuela 4 4
No JRUs were established in LA
during EELA
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 10www.eu-eela.eu
EELA & EELA-2 objectives
EELA
• Build a bridge between consolidated e-Infrastructure initiatives in Europe and emerging ones in Latin American
• Create a collaboration network to deploy a large portfolio of scientific applications on a well supported Pilot Test-bed
• Care in parallel of the training in grid technologies and of the knowledge dissemination and outreach
EELA-2
• Provide an empowered Grid Facility with versatile services fulfilling application requirements
• Ensure production quality services
• Ensure the long term sustainability of the e-Infrastructure beyond the term of the project
• Expand the current EELA e-Infrastructure
• Look for new communities outside academia (Industry and Business)
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 11www.eu-eela.eu
More Numbers
EELA-2 EELA
At the end of the Project
• # Sites: 19 (13 in Production)
• # Cores: 2700
• Storage: 100 TB
• # Jobs: 430,000
• Scientific domains: 4
• # Applications: 18- 12 in production- 6 gridified
• Conferences: 3
• Workshops: 8
• Tutorials: 18
• Trained people: 733
• Grid Schools: 2
After 1 year
• # Sites: 22
• # Cores: 5800
• Storage: 190 TB
• # Jobs : 1,000,000
• Scientific domains: 9
• # Applications: 55- 12 in production- 15 Gridified
• Conferences: 1
• Workshops: 4
• Tutorials: 8
• Trained people: 158
• Grid Schools: 1
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 12www.eu-eela.eu
810 participant · day (as of end on April 2009)
Participant · day
600 by year 2
Project goals DoW
300 by year 1
EELA-2 achievements - NA2
NA2 = Dissemination & Training
Much more training effort delivered than pledged
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 13www.eu-eela.eu
EELA-2 achievements - NA3
GILDA training-infrastructure
TutorialsGrid Schools
Support system Wiki pages /
EOC website
Gridification
Weeks
Gridification team
NA3 = Application SupportThe EELA-2 “gridification universe”
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 14www.eu-eela.eu 14
2
Applications distribution per country, continent and scientific domain (Y1)
EELA-2 achievements - NA3
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 15www.eu-eela.eu
Certification Authority
Registration Authority
CA Applicant
Catch All Certification Authority
EELA-2 achievements - SA1
SA1 = Infrastructure ServicesEvery EELA-2 country has access to the LA PKI
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 16www.eu-eela.eu
Legacy Sitesprod.vo SitesOurGrid Sites
Resource Centres Map (Y1)
EELA-2 achievements - SA1
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 17www.eu-eela.eu
About 1 million jobs executed
About 1 million jobs executed
About 1 million jobs executed
About 1 million jobs executed
ResourceCentres
EELA-2 achievements - SA1
Infrastructure Usage - all VOs (Y1)
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 18www.eu-eela.eu
Deployment of a sustainable perfSONAR-based multi-domain network monitoring service – 3 measurement points on EELA-2 RCs (Overlay network monitoring) already installed– Integration with available monitoring services in the domains (LA and EU-NRENs,
RedCLARA, GÉANT2, EGEE-III, etc.)
SA2 = Network Resource ProvisionEELA-2 Network Monitoring
LA-NREN A - MPs
RedCLARA MPs
EELA-2 Measurement Points (MPs)
LA-NREN B - MPs
LA-NREN
A
RedCLARA
LA-NREN
B
EELA-2MP A
EELA-2MP B
Visualization tool
Overlay network monitoring
Multi-domain monitoring
EELA-2 achievements - SA2
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 19www.eu-eela.eu
EELA-2 achievements - JRA1
JRA1 = Development of Services for Applications and Infrastructure
Activity Goals
• Making the e-Infrastructure more useful and wide spread to increase its reach …
Execution of grid middleware on top of different platforms, including Microsoft Windows Scavenging of idle resources, particularly in shared desktops
… and its usability New application-oriented grid services Support for the integration of a lightweight opportunistic grid middleware (OurGrid)
whose development is led by one of the consortium members
• With the effect of helping in fostering the sustainability of the e-Infrastructure
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 20www.eu-eela.eu
• Interoperability between OurGrid and gLite, planned for M16, just available (see “Hot News” in the EELA-2 Website homepage):
– MoU signed with other projects working on similar goals (EDGeS and SEE-GRID-SCI)
– Initial assessment of the possibility of using EDGeS G3 indicates the feasibility of using technology that has already been successfully used in a similar setting
– Initial evaluation of the innovative co-existence approach reveals advantages that had not been foreseen at the time the project was proposed
• Possibility of draining BoT applications out of the service grid, improving the performance of tightly-coupled applications
EELA-2 achievements - JRA1
Highlights of the 1st Year
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 21www.eu-eela.eu
The “global” Grid coverage
How to take care of the future of e-infrastructures?
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 22www.eu-eela.eu
Grid “Ecosystem” evolution in Europe
Testbeds Utility ServiceRoutine Usage
National
Global
Ref.: D
. Kra
nzlm
ülle
r pre
sem
tatio
n a
t EG
EE’0
8
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 23www.eu-eela.eu
…and in Latin America ?
EELA
EELA-2
?
4/2008-3/20101/2006-12/2007
4/2010 →
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 24www.eu-eela.eu
Some considerations…
24
• EELA-2 will end up on 01/04/2010
• Will 6-year of successful efforts to: – Set up a production quality Latin American e-Infrastructure at the level of
existing European Grids
– Acquire the various Grid technology skills to autonomously support the Latin American e-Infrastructure on the long term (about 1000 LA people trained and 10’s of local tutor formed)
– Allow LA scientific communities to: Strengthen their active participation in EU – LA collaborations by
satisfying the required contributions to computing resources
Enter new international collaborations to take part in new research field or attract new partners in their own field of research
– Altogether “contribute to amplify the relevance of e-Infrastructures, blazing the trail towards Latin American e-Science Initiatives via National Grid Initiatives (NGIs) created in cooperation with existing NRENs and RedCLARA” (EELA-2 proposal)
just be wasted ???
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 25www.eu-eela.eu
The EELA-2 proposal
25
• A 3-level organisation is proposed: – Local (RCs ↔ LAN/MAN)– National (NGIs ↔ NRENs)– Continental (LGI ↔ CLARA)
• Multi-middleware (gLite, OurGrid, OSG, …) structures: NGIs and LGI must not be middleware dependent
• Close synergy with EGI that foresees to support three middleware (ARC, gLite and UNICORE)
• Important role of both the LA NRENs and CLARA: fully justified by their participation in EELA and EELA-2
• Even more: e-Infrastructures (Grid) should be part of the NRENs and CLARA structure (cost effective, technically correct, politically easier)
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 26www.eu-eela.eu
The Latin American Grid Initiative (LGI)(http://documents.eu-eela.org/record/1119/files/)
RC: Resource Centre, NGI: National Grid Initiative, GOC: Grid Operation Centre, NREN: National Research & Education Network, NOC: Network Operation Centre, LGI: Latin American Grid Initiative, CLARA: Cooperación Latino Americana de Redes Avanzadas, GSC: Grid Support Centre, NSC: Network Support Centre
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 27www.eu-eela.eu
The Latin American Grid Infrastructure - GridLGI
• GridLGI is composed, from bottom to top of three layers, each of them including Grid and Network components:
– At the lower Institute level: The Resource Centre (RC) is the actual site of the Grid computing resources; The backbone LAN or MAN network connects the Computing and Storage Elements (CE &
SE) at the RC site;
– At the intermediate national level: The National Grid Initiative (NGI) integrates all Grid Institutions in the country and federates
under its Grid Operation Centre (GOC) all their RCs in a National Grid (GridNGI); The underlying National Research & Education Network (NREN) links country-wide the
Research and Education Institutes and optimizes / monitors their network activity via its associated Network Operation Centre (NOC);
– At the upper international layer: The Latin American Grid Initiative (LGI) which brings together all NGIs and supervises the
various GridNGIs operation from its Grid Support Centre (GSC); CLARA: it is the WAN network which integrates the Latin American NRENs and supervises
the overall RedCLARA network activity by means of its Network Support Centre (NSC).
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 28www.eu-eela.eu
RC tasks
RC
StorageStorageElement(s)Element(s)
ComputingComputingElement(s)Element(s)
1st Level1st LevelUser Supp.User Supp.
Reg. Auth.Reg. Auth.
MonitoringMonitoring
AccountingAccounting
SecuritySecurityinc. resp.inc. resp.
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 30www.eu-eela.eu
GOC tasks
GOC
MonitoringMonitoring
AccountingAccounting
M/W M/W Repos.Repos.
Cert. Auth.Cert. Auth.
VOMSVOMS
2nd Level2nd Leveluser supp.user supp.
DataDataCatalogsCatalogs
Info. Syst.Info. Syst.
L&BL&BWMSWMS
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 32www.eu-eela.eu
GSC tasks
GSC
M/W releasesM/W releases
MonitoringMonitoring
AccountingAccounting
Catch-all CACatch-all CA
Global VOMSGlobal VOMS
3rd Level3rd Leveluser supp.user supp.
Gen. Info. Gen. Info. SystemSystem
Knowledge repo.Knowledge repo.
High levelHigh levelGrid servicesGrid services
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 35www.eu-eela.eu
NSC tasks
Single point Single point of accessof access
NSC
Network quality Network quality indicatorsindicators
User ticket GSC TicketSystem
Ticket Processing
Network
Operations
Training and Dissemination
Grid Site Deployment
Middleware
...
NSC Team
RC 1
Grid Site 1
RC 2
Grid Site 2
LA-NREN A EU-NREN B
RedCLARA
Operated byNOC of NREN A
Operated byNOC of NREN B
Operated byCLARA
Operated byNOC of
Campus Network
Operated byNOC of
Campus Network
NSC
GÉANT2
Operated byDANTE
E2E servicesE2E services
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 37www.eu-eela.eu
Training & dissemination tasks
T&D
ConferencesConferences
DisseminationDisseminationMaterialMaterial
ConsultancyConsultancy
Event calendarEvent calendar
Training eventsTraining events
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 40www.eu-eela.eu
User support tasks
UserSupp.
GuidelinesGuidelines
NewNewcommunitiescommunities
Tech. requirementsTech. requirementsgatheringgathering
Local/remote supportLocal/remote support
Apps. Apps. RegistryRegistry
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 42www.eu-eela.eu
LGI Management / Governance
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 43www.eu-eela.eu
Possible structure of a Latin American NGI
A strong liaison with the NREN is strongly suggested/recommended.A strong liaison with the NREN is strongly suggested/recommended.Local conditions permitting, the NGI should Local conditions permitting, the NGI should
be incorporated into the NREN.be incorporated into the NREN.
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 44www.eu-eela.eu
Long-term sustainability status (1/2)
• Long-term sustainability of e-Infrastructures world wide is key for a continued support of scientific communities
• EELA-2 is definitely contributing to the establishment of JRUs and NGIs in Latin America, in close synergy with NRENs and CLARA, also in countries not initially involved in the project
• EELA-2 has defined a detailed model and architecture of the Latin American Grid Initiative
• Contacts with CLARA have formally been established and a Joint Committee has been formed at the Bogotá Conference (with Spanish participation) to analyse, agree on, and start implementing the proposed model
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 45www.eu-eela.eu
Clear support of the former CLARA President (C. Casasus)
Long-term sustainability status (2/2)
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 46www.eu-eela.eu
What’s round the corner? GISELA?
• No EELA-3 foreseen then …
• Grid Initiatives for e-Science virtual communities in Europe and Latin
America (GISELA)
• Objective of the proposal: Following about 4 years of successful activities in
EELA and EELA-2, the objective of the GISELA proposal is to guarantee the long-
term sustainability of the EU-LA e-Infrastructure to ensure the continuity and the
future enhancement of the European – Latin American Virtual Research
Communities. Building on the EELA-2 e-Infrastructure, its associated Services and
the numerous User Communities, the project will focus on two inter-related goals:
– Implement the NGI / LGI sustainability model, as specified in DSA1.3 (http://documents.eu-eela.org/record/1119/files/), in association with CLARA and collaborating with EGI.
– Provide the Virtual Research Communities with the suited e-Infrastructure and Application-related Services required to improve the effectiveness of their research.
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 47www.eu-eela.eu
GISELA summary proposal 1/3
• Call Identifier: FP7-INFRASTRUCTURES-2010-2
• URL: http://cordis.europa.eu/fp7/dc/index.cfm?fuseaction=UserSite.CapacitiesDetailsCallPage&call_id=263#infopack
• Funding Scheme: CP - CSA
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 48www.eu-eela.eu
GISELA summary proposal 2/3
• Work Packages and their objectives (short summary)
– NA1: Administrative and Technical Management of the Project EELA-2 type of Administrative and Technical management International Cooperation
– NA2: Dissemination and Outreach Organise Dissemination events focusing on Decision Makers interaction to get their support for
the LGI / NGI implementation Promote outreach actions (e.g. Conference talks, seminars, exhibitions, Virtual Days,..) to
widely spread the usefulness of e-Infrastructures for different research communities
– NA3: Virtual Communities Support Identify and support Virtual Research Communities (VRC) by fostering e-Science collaboration
among multidisciplinary groups Create and maintain support tools to ease the utilisation of the e-Infrastructure (Application´s
portals and database, self-training material, User’s and developer’s Guide, Help Desk support etc.)
Organise training activities – tailored to each VCR needs – to transfer knowledge concerning the e-Infrastructure utilisation
Validate the services provided by SA1, SA2 and JRA1 by performing tests with pilot applications
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 49www.eu-eela.eu
GISELA summary proposal 3/3
– SA1: NGI / LGI Infrastructure Services (as in DSA1.3) Deploy the catch-all Grid Operation Centre (GOC) as defined in DSA1.3 to serve as
model for future ROCs Implement the Grid Support Centre (GSC) as defined in DSA1.3 Support Virtual Research Environment (VRE) tools, systems and processes
– SA2: NGI / LGI Network Services (as in DSA1.3) Keep managing Network resources via the existing national Network Operation
Centres (NOC) Implement the Network Support Centre (NSC) as defined in EELA-2 DSA1.3 Proceed with the deployment of performing Network monitoring in collaboration with
EGI
– JRA1: Infrastructure & Application-oriented Services for Virtual Communities Increase the reach, the attractiveness, the usability and the effectiveness of the EU-LA
e-Infrastructure for well-established and new communities. For this goal, collaborate with EGI to divulge the Services developed in the EELA-2 context and devise those new Services required by the User Communities
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 50www.eu-eela.eu
Why is it possible?
• The EELAs – Trained users, tutors and administrators in LA– Disseminated the Grid culture– “Approached” Decision Makers– Proposed a model for long-term sustainability of Grid
infrastructures in LA– Is currently looking for funds in EU (EC & CIEMAT) to help
implementing the model during the next 2 years
ConclusionIt should work!!!!!!!!!!!!!!!!!!!!!
Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 51www.eu-eela.eu
Thank you for your attention