WP6 - Infrastructure and Application-oriented Services for User Communities – Achievements

www.gisela-grid.eu

Grid Initiatives for e-Science virtual communities in Europe and Latin America

WP6 - Infrastructure and Application-oriented Services for User Communities – Achievements

Francisco Brasileiro (WP6 Manager)

UFCG (Brazil)

Final Project Review

Brussels, 11.10.2012

www.gisela-grid.eu GISELA - Final Project Review - Brussels - 11/10/2012 2

Outline

• Work Package goals

• Work planned for the second year

• Implementation of reviewers’ recommendations

• Highlights of the results achieved in the second year

• Conclusions


• In the DoW the WP6 goals are stated as:

“The objective of the work package WP6 is to increase the usability of DCIs in general, and GISELA’s infrastructure in particular, by:

• Offering a portfolio of application-oriented and infrastructure-oriented services, including both services already available and new services to be developed during the course of the project;

• Customising these services to attend particular requirements of the VRCs supported by GISELA;

• Providing, together with WP3, comprehensive support to the users of the portfolio of services offered; and

• Helping WP2 to disseminate the GISELA’s portfolio of services, so that the benefits yield from their use is as broad as possible.”

Work Package goals


• TWP6.1: Coordination of the Activity

– Standard coordination activities Weekly status report Monthly conf-call meetings

• TWP6.2: Support and Customization of Available Services

– Continue supporting the GISELA portfolio of services

– Improvements on the OurGrid middleware to support the GISELA Science Gateway

• TWP6.3: Development of New Services

– Efficient opportunistic execution of data-intensive applications based on MapReduce

– Seamless execution of applications in hybrid e-Infrastructures (cloudburst)

Work Planned for the 2nd year

www.gisela-grid.eu

Implementation of reviewers’ recommendations

• “Establish a process for collecting feedback from the users as well as partners supporting the sustainability in order to understand what are missing services and features of the science gateway...”

– We have established a close connexion with both WP3 and WP4, so that user’s requests could be accurately and promptly made available to WP6

– The main outcome of this procedure were:

Request for the implementation of a JSAGA adaptor for the OurGrid middleware

Request for investigating alternatives to improve efficiency of data-intensive applications

Request for assessing the use of hybrid infrastructures based on grid and cloud technologies

GISELA - Final Project Review - Brussels - 11/10/2012 5


• Achievements in the context of TWP6.2

–Improvements have been added to DIRAC’s core software

The DIRAC File Catalog

Numerous improvements in DIRAC Web Portal• user proxy upload, support of parametric jobs, pilot submission to

BOINC Desktop resources, and improved MPI support

–JSAGA adaptors for the OurGrid middleware

Allows seamless execution of applications through the GISELA Science Gateway, both on gLite and OurGrid resources

Was recently showcased in the EGI Technical Forum in Prague, as part of the CHAIN worldwide grid interoperability demo

Will be integrated in the official JSAGA release as a GISELA contribution

Highlights of the 2nd year (1/3)


• Achievements in the context of TWP6.3

– Support for the efficient opportunistic execution of data-intensive applications based on the MapReduce paradigm

Built atop BeeFS base Storage Elements

– Seamless execution of applications in hybrid e-Infrastructures

Allows cloudbursting OurGrid infrastructures with resources provided by EC2 based cloud providers

• Working prototype for private clouds based on Eucalyptus and the AWS public cloud



• Several scientific publications were produced, from which we highlight:

– Paulo Ditarso Maciel Jr., Francisco Vilar Brasileiro, Ricardo Araújo Santos, David Candeia, Raquel Vigolvino Lopes, Marcus Carvalho, Renato Miceli, Nazareno Andrade, Miranda Mowbray: Business-driven short-term management of a hybrid IT infrastructure. J. Parallel Distrib. Comput. 72(2): 106-119 (2012)

– Juan D. Osorio, Harold Castro, Francisco Vilar Brasileiro: Perspectives of UnaCloud: An Opportunistic Cloud Computing Solution for Facilitating Research. CCGRID 2012: 717-718

– Tsaregorodtsev, R. Graciani Diaz, A. Casajus Ramo, F. Stagni, V. Hamar, M. Sapunov. Status of the DIRAC Project, Computing in High Energy and Nuclear Physics (CHEP), USA 2012

– Tsaregorodtsev, V. Hamar. MPI support in the DIRAC Pilot Job Workload Management System, Computing in High Energy and Nuclear Physics (CHEP), USA 2012

– Paulo Ditarso Maciel Jr., Francisco Vilar Brasileiro, Raquel Vigolvino Lopes, Marcus Carvalho, Miranda Mowbray: Evaluating the impact of planning long-term contracts on the management of a hybrid IT infrastructure. Integrated Network Management 2011: 89-96


www.gisela-grid.eu

WP6 overcame its metrics

Metric Threshold for 2nd year

Achieved in 2nd year

Number of applications using at least one service from the WP6 portfolio

≥ 80% 100%

Number of scientific publications

≥ 4 13

GISELA - Final Project Review - Brussels - 11/10/2012 9

WP6 metrics


• Overall, the outcomes from the second year of execution of the activity have been very good– We were able to upgrade in many ways the services already available in

the GISELA portfolio and enhance the portfolio with the addition of two new services

– Most importantly, the services supported are already fulfilling their aim of facilitating the implementation and deployment of applications

• We were able to cooperate with both WP3 and WP4 teams to understand the users’ requirements and cater to their needs– The best example of this integrated effort is the successful deployment

of the GISELA Science Gateway

Conclusions : WP6 Achievements


• Partners that lead the development of most available services are keen to continue providing support for their users’ base– This is already in place in most partners through a number of different funding

schemes, including community-based open source model

– Partners expect their users’ base to grow, thanks to CLARA’s and its associated NRENs involvement in promoting the available services to its users

• Important outcomes were also achieved in strengthening the Latin American network of research in the area of Distributed Computing Infrastructures (DCIs)– The development of the GISELA project enabled new collaborations to flourish,

resulting in actions to allow mobility of students and researchers among Latin American institutions

– This is an indication that the research activity in the area of DCIs will continue to happen in a cooperative way beyond GISELA’s lifetime

Conclusions : WP6 Beyond GISELA

WP6 - Infrastructure and Application-oriented Services for User Communities – Achievements

Documents

Transcript of WP6 - Infrastructure and Application-oriented Services for User Communities – Achievements