1 The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) GRID Science Gateway...
-
Upload
earl-mcdonald -
Category
Documents
-
view
216 -
download
0
Transcript of 1 The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) GRID Science Gateway...
1
The EPIKH Project(Exchange Programme to advance e-Infrastructure Know-How)
GRID Science Gateway
Riccardo Rotondo ([email protected])
Consortium GARR
Tutorial for Grid Application Porting on Grid Science GatewayBeijing, 11.04.2012
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 1
2
Outline
• Grid Usage
• Science Gateway
• The e-Collaboration environment– The old paradigm– Science Gateway architecture– The new “brick” approach
• Use Cases:– The DECIDE Science Gateway– The INDICATE e-Culture Science Gateway– The CHAIN Science Gateway– The Gilda t-Infrastructure Portal– The RICeVI Portal
• Conclusions
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 2
3
GRID at world scale
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
>340 sites in 57 countries~337,000 CPU cores>220 PB of storage~1 million jobs/day~20,000 users in ~280 VOs
3
4
But…using Grids is not straightforward
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
Type = "Job";Type = "Job";
JobType = "JobType = "MPICHMPICH";";
MPIType = "MPIType = "MVAPICH2_PGI706MVAPICH2_PGI706";";
CpuNumber = 16;CpuNumber = 16;
MPIGranularity = 4;MPIGranularity = 4;
Executable = "flash2";Executable = "flash2";
StdOutput = "mpi.out";StdOutput = "mpi.out";
StdError = "mpi.err";StdError = "mpi.err";
InputSandbox = InputSandbox = {"{"watchdog.shwatchdog.sh","","mpi.pre.shmpi.pre.sh","","mpi.post.shmpi.post.sh","flash.par","summers_den","flash.par","summers_den_1e0.rates","flash2"};_1e0.rates","flash2"};
OutputSandbox = OutputSandbox = {"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"};{"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"};
Requirements = (other.GlueCEUniqueId == Requirements = (other.GlueCEUniqueId == "unipa-ce-01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc");"unipa-ce-01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc");
RetryCount = 0;RetryCount = 0;
$ voms-proxy-info --all subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy/CN=proxy issuer : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy identity : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy type : proxy strength : 1024 bits path : /tmp/x509up_u512 timeleft : 500:38:44 === VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/idl/Role=NULL/Capability=NULL timeleft : 23:59:33 uri : voms.ct.infn.it:15001
$ glite-wms-job-status https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN-XTAIGAg ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN-XTAIGAg Current Status: Done (Success) Logged Reason(s): - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda Submitted: Tue Jun 29 15:34:40 2010 CEST *************************************************************
CLI
ScriptsJDL
echo Staging Input Data \(Courtesy of European Space Agency\);#edg-rm --vo=gilda copyFile lfn:$1.N1 file://$PWD/$1.N1;lcg-cp --vo=gilda lfn:$1.N1 file://$PWD/$1.N1;echo Staging Application;gunzip beam20.tar.gz;tar xvf beam20.tar;cd beam-2.0/bin;echo Starting Application;./pds2jpg-ASAR-run.sh $1;mv $1-b*.jpg ../..cd ../..rm -fr beam-2.0;rm -fr $PWD/$1.N1;rm -fr $PWD/beam20.tar;echo Input ENVISAT Product courtesy of European Space Agencytouch ENVISAT_Product_courtesy_of_European_Space_Agencyecho No Output Packaging;echo Done!;
GSI
Users have to cope with complex security procedures,
execution scripts, job description languages, command
line based interfaces and lack of standards.
This makes the learning curve very steep and keeps
non IT-experts away.
4
5
Grid Interface Evolution
• The way users access Grid resources has continuously evolved towards simplicity and transparency:
• Command Line– Globus and gLite CLI– Used by the enthusiastic and early adopter scientists
• GUI applications– gEclipse, Grid2Win– Good to expand the communities but difficult to maintain
• Web Interface– GENIUS, P-GRADE– Easier for new users but monolithic
• Science Gateways
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
“robot” certificate
s on “e-tokens”
5
6
Science Gateways
A framework of tools that allows scientists to run applications with little concern for where the computation actually takes place. This is similar to cloud computing in which applications run as Web services on remote resources in a manner that is not visible to the end user. However, a science gateway is usually more than a collection of applications. Gateways often let users store, manage, catalogue, and share large data collections or rapidly evolving novel applications they cannot find anywhere else. Training and education are also a significant part of some Science Gateways
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 6
7
Reference Model
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
Science G
ateway
Science G
ateway
Appl 1Appl 1 Appl 2Appl 2 Appl NAppl N
Grid ServicesGrid Services
Embedded ApplicationsAdministratorPower UserBasic User
Users from different organisations having different roles and
privileges
Users from different organisations having different roles and
privileges
Other MiddlewareOther Middleware
7
8
Federated Grid User
Riccardo Rotondo 8
Science GatewayScience Gateway
Beijing, Asia 4, 11.04.2012
9
Identity provided federated
Riccardo Rotondo 9
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idPN }
{ idp1, idp2, … idPN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
Beijing, Asia 4, 11.04.2012
10
Federated Grid User
Riccardo Rotondo 10
Science GatewayScience Gateway
Beijing, Asia 4, 11.04.2012
12
AuthN & AuthZ Schema
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 12
Science GatewayScience Gateway
13
Identity Provider Federated
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 13
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idPN }
{ idp1, idp2, … idPN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
{ idp1, idp2, … idpN }
14
eduGAIN(www.edugain.org)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
All the Science Gateways developed at Catania are
Service Providers of the eduGAIN inter-federation!
1414
15
Federation Supported
• 7 Federations supported: • GrIDP (“catch-all”, Italy); • GARR-IDEM (Italy);• CARSI (Chinese);• GRNET-AAI (Greece);• RCTSaai (Portugal);• SIR (Spain);• eduGAIN (Europe).
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 15
16
Federation supported by DECIDE Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 16
17
Other federation supported (1/2)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 17
18
Other federation supported (2/2)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 18
19
The GrIDP Identity Provider (1/2)• 4 Identity Providers are available in GrIDP:
• A “catch-all” IdP created at Catania;
• The maat-G (enterprise) IdP;
• INFN-AAI IdP (all INFN researchers and associates);
• An idp that enables Social Networks credentials.
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 19
20
The GrIDP Identity Provider (2/2)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 20
21
Liferay (www.liferay.com)
• Highly-configurable, scalable, open source portal framework;
• Compatible with JSR 168/286 standards and based on modern web 2.0 technologies;
• Liferay services planned to be used:– Portal;– CMS & WCM;– Collaboration and “social” software
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 21
22
Grid Access enable: Portlets as bricks• Portlets can interact with the Grid e-
Infrastructure
• Different approaches are available:– Execute the Command Line behind the portal;– Using API where available:
• Must be in Java or other languages supported by Liferay;
– Call REST services from Javascript code in the browser;
• Additional layers between Liferay and the Grid can be necessary for some services;
• Each portlet can follow its own communication method.
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 22
23
Usage Workflow
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
1. Sign in
eTokenServer
User Track-ing DB
3. Proxy
request
4. Proxy
transfer
5. Tracking
23
2. Grid Request
6. Getting Results
23
24
Functionalities
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 24
25
Science Gateway Developed
• DECIDE Science Gateway
• EUMDEDGRID SUPPORT Science Gateway
• INDICATE Science Gateway
• VIRALGRID Science Gateway
• GISELA Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
• CHAIN Science Gateway• The Gilda t-Infrastructure Portal
• The RICeVI Portal
25
26
Liferay AT INFN-CT
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 26
27
GRID Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
GRID SCIENCE
GATEWAY
27
28
The DECIDE Science Gateway (1/5)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 28
29
The DECIDE Science Gateway (2/5)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 29
30
The DECIDE Science Gateway (3/5)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 30
31
The EUMEDGRID-Support Science Gateway (1/2)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 31
32
The EUMEDGRID-Support Science Gateway (2/2)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 32
33
The INDICATE e-Culture Science Gateway (5/6)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 33
34
The INDICATE e-Culture Science Gateway (6/6)
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 34
35
CMS Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
CMS SCIENCE
GATEWAY
35
36
The CHAIN Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 36
37
The CHAIN Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 37
38
The CHAIN Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 38
39
The Gilda t-Infrastructure Portal
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 39
40
E-learning Science Gateway
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
E-learning SCIENCE
GATEWAY
40
41
RICeVI Portal
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
Collaboration Sharing
Dissemination Training
41
42
Current Status
• 16 Liferay-based Science Gateways (hosted in 2 servers) are currently powered by Shibboleth at INFN Catania;
• 7 Federations supported;
• 4 instances are registered as official IDEM Service Provider;
• 4 Identity Providers are available in GrIDP.
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 42
43
Credits & Acknowledgments
• Valeria Ardizzone (GARR);• Roberto Barbera (UNICT & INFN)• Riccardo Bruno (COMETA);• Antonio Calanducci (COMETA);• Marco Fargetta (COMETA)• Elisa Ingrà (GARR);• Giuseppe La Rocca (INFN)• Salvatore Monforte (INFN);• Fabrizio Pistagna (INFN);• Rita Ricceri (INFN);• Diego Scardaci (INFN);
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
Credits Acknowledgments• Vincenzo Ciaschini (INFN);
• Enrico Fasanelli (INFN);
• Maria Laura Mantovani (GARR);
• Barbara Monticini (GARR);
• Simona Venuti (GARR)
43
44
References
• Science Gateway Url:–http://applications.eu-decide.eu/
–http://applications.eumedgrid.eu/
–http://gilda.ct.infn.it
–http://www.chain-project.it
–http://indicate-gw.consorzio-cometa.it
–https://gridp.ct.infn.it/
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo 44
45
Questions?
Beijing, Asia 4, 11.04.2012 Riccardo Rotondo
Thank you for your attention
45