DataTag is a project funded by the European UnionCHEP 2003 – 24-28 March 2003 - The WorldGRID...
-
Upload
jasper-lyons -
Category
Documents
-
view
216 -
download
1
Transcript of DataTag is a project funded by the European UnionCHEP 2003 – 24-28 March 2003 - The WorldGRID...
DataTag is a project funded by the European UnionDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 -CHEP 2003 – 24-28 March 2003 -
The WorldGRID transatlantic testbedA successful example of Grid interoperabilityacross EU and US domains
Flavia Donno (Formerly of DataTAG WP4, LCG)
http://chep03.ucsd.edu/files/249.ppt
CHEP 2003 – 24-28 March – nCHEP 2003 – 24-28 March – noo. (1). (1)
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 22
Talk Outline
• Motivation
• Participants
• Interoperability issues
• Solutions
• Architecture
• Monitoring/Support
• Spin off
F. DonnoCERN/IT and INFN
R. GardnerUniversity of Chicago
Applications
CMS
ATLAS
Monitoring with Nagios
Monitoring with Ganglia
Conclusions
Next Steps
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 33
ParticipantsParticipants
DataTAG:F. Donno, CERN - INFNS. Andreozzi, R. Barbera, V. Ciaschini, S. Fantinel, A. Ghiselli, M. Mazzucato, D. Rebatto, G. Tortone, L. Vaccarossa, M. Verlato, C. Vistoli – INFNM. Draoli, CNR-Rome
CrossGrid:A. Garcia, M. Hardt, FZK - GermanyJ. Marco, UC - Spain M.David, J. Gomes, LIP - PortugalO. Maroney, U.Bristol, UK
Trillium/iVDGL:P. Avery, J. Rodriguez - U. FloridaE. Deelman, N. Olomu - USC/ISIJ. Gieraltowski, S. Gose, E. May, J. Schopf – ArgonneAfaq, J. Annis, R. Glossum, R. Pordes, V. Sekrhi – FermilabW. Deng, J. Smith, D. Yu - BNL A. DeSmit, A. Roy - Wisconsin C. Dumitrescu, I. Foster, R. Gardner, U. ChicagoL. Grundhoefer ,J. Hicks, F. Luehring, L. Meehan - U. IndianaS. Youssef, Boston UniversityB. Moe - MilwaukeeD. Olson – LBNLS. Singh - Caltech
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 44
MotivationsMotivations
Basic collaboration between European and US Grid projects
Interoperability between Grid domains for applications submitted by users from different virtual organizations Controlled use of shared resources subject to agreed policy Integrated use of heterogeneous resources from iVDGL and DataGrid/CrossGrid testbed domains
Goal:Goal:
Build a “transatlantic grid” based on the existent European Build a “transatlantic grid” based on the existent European and American Grids with the goal of offering transparent and American Grids with the goal of offering transparent access to the distributed computing infrastructure necessary access to the distributed computing infrastructure necessary to the “data-intensive” modern applicationsto the “data-intensive” modern applications
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 55
Interoperability IssuesInteroperability Issues Many grids with several OSseveral OS (RH 6.2, RH 7.x, Fermi Linux, CERN Linux,…), several compilers and software components.Different Grid ArchitecturesGrid Architectures (VDT server/client vs. Computing Elements, Storage Elements, User Interfaces, …)Need to identify minimum set of core servicesminimum set of core services and define collective/optional servicesoptional services Common protocols/Same or compatible versions of the softwareAuthentication and AuthorizationAuthentication and Authorization mechanism: authority trusting, user authentication/authorization via LDAP VO Servers.Grid resource descriptionGrid resource description/status: Globus schema vs. EDG schema vs. GLUE schema Several Grid Data managementGrid Data management ToolsSoftware distribution and configurationSoftware distribution and configuration : rpm based vs. PACMAN
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 66
SolutionsSolutions
Many grids with several OSseveral OS (RH 6.2, RH 7.x, Fermi Linux, CERN Linux,…), several compilers and software components.
Partition WorldGrid in subdomains with uniform or compatible set of basic services. Such resources will advertise themselves with specific targets to the applications (such as RH6.2).Try to keep the subdomains as large as possible.
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 77
SolutionsSolutions
Different Grid ArchitecturesGrid Architectures (VDT server/client vs. Computing Elements, Storage Elements, User Interfaces, …)
CECE
UIUI
SESERBRB
VDT Client
VDT Server
RCRC
ISIS
RCRC
ISIS
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 88
SolutionsSolutions
Need to identify minimum set of core servicesminimum set of core services and define collective/optional servicesoptional services Common protocols/Same or compatible versions of the software
Globus and Condor core services (GRAM, GSI, MDS, GridFTP, …)
Resource Broker, User Interface and JDL, Data Management high level tools (edg-replica-manager, MAGDA, Globus Replica Catalog, …) collective optional services not installed universally
User Grid Portals (Genius, GRAPPA, …): a variety available not to change the User Interface to the GRID
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 99
SolutionsSolutions
Authentication and AuthorizationAuthentication and Authorization mechanism: authority trusting, user authentication/authorization via LDAP VO Servers.
DOE and EDG certificates universally accepted DataTAG and iVDGL VO LDAP servers trusted mkgridmap tool universally installed Local security policy sites agreed to allow
access to grid demonstration users (kerberos, …)
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 1010
SolutionsSolutions
Grid resource descriptionGrid resource description/status: Globus schema vs. EDG schema vs. GLUE schema
three coexistent schemas in place (Globus, EDG, GLUE) installed on all resources
Some tool (monitoring) working with all of them
EDG middleware using both EDG and GLUE US tools using none or Globus
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 1111
SolutionsSolutions
Software distribution and configurationSoftware distribution and configuration : rpm based vs. PACMAN
Created WorldGrid distribution (rpm/LCFGng and PACMAN)
Effort to ensure coherency and automatic configuration
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 1212
Final ArchitectureFinal Architecture
CECE
UIUI
SESERBRB
VDT Client
VDT Server
RCRC
ISIS
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 1313
Monitoring and SupportMonitoring and Support
Two monitoring tools VO based in place: Two monitoring tools VO based in place: edt-monitor based edt-monitor based on Nagios and iVDGL based on Ganglia (see talk from R. on Nagios and iVDGL based on Ganglia (see talk from R. Gardner)Gardner)
Support infrastructure: Support infrastructure: to support to support
site site administrators administrators during the during the installation and installation and configuration configuration procedure. Also procedure. Also for problem for problem fixing fixing during during normal normal operationoperation
DataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – nDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 – noo 1414
Spin-offSpin-off GLUE schema:GLUE schema: WorldGrid has allowed to prove the validity of WorldGrid has allowed to prove the validity of the GLUE schema and encouraged EDG to deploy itthe GLUE schema and encouraged EDG to deploy it VOMS:VOMS: The authentication/authorization problems were The authentication/authorization problems were identified and parallel research activities started, like the one on identified and parallel research activities started, like the one on Virtual Organization Manager ServiceVirtual Organization Manager Service GLUE Packaging:GLUE Packaging: A working group is trying to find a solution A working group is trying to find a solution for a standardization of the packaging, distribution and for a standardization of the packaging, distribution and configuration problem for a software releaseconfiguration problem for a software release GLUE Testing:GLUE Testing: The problem of verifying an installation and The problem of verifying an installation and validate a site for joining the Grid has been addressed and a validate a site for joining the Grid has been addressed and a working group has startedworking group has started Support:Support: A first operation/monitoring center has started A first operation/monitoring center has started in US taking advantage of the in US taking advantage of the monitoring toolsmonitoring tools. Other . Other centers in EUcenters in EU LCG-0:LCG-0: After the demonstration at IST2002 and SC2002, After the demonstration at IST2002 and SC2002, LCG has based his LCG has based his first middleware distribution onfirst middleware distribution on the the WorldGridWorldGrid experience experience
DataTag is a project funded by the European UnionDataTag is a project funded by the European Union CHEP 2003 – 24-28 March 2003 -CHEP 2003 – 24-28 March 2003 -
Rob Gardner University of Chicago
on behalf of the WG group
The WorldGRID transatlantic testbed, Part 2A successful example of Grid interoperabilityacross EU and US domains
16 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 1616
Talk Outline
Motivation
Participants
Interoperability issues
Solutions
Architecture
Monitoring/Support
Spin off
F. DonnoCERN/IT and INFN
Part 2
R. GardnerUniversity of Chicago
Applications
CMS
ATLAS
Monitoring with Nagios
Monitoring with Ganglia
Conclusions
Next Steps
17 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 1717
Installing Apps on 2 Grids We needed a way to get applications from three
experiments (VO’s) setup on the execution sites On DataTAG resources, selected CE’s were loaded
with CMS or ATLAS rpms On iVDGL resources, we Pacmanized binaries
(rpms and tarballs) of bundled applications– %pacman –get iVDGL:ScienceGrid
> Atlas-kit, Atlas-ATLFAST
> CMS-MOP, EDG-CMS
> SDSS Astrotools
– binaries, and run time environments
3 experiments
18 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 1818
ATLSIMJob
Write to Grid
Storage Element
Read from Grid
Storage Element Input Data
Output ZEBRA
Grid Storage
Apache
EnginFrame
GENIUS
https+java/xml+rfb
WEB Browser
EDGUI
Local
WS
the GridEDG+GSI
ATLAS and CMS with GENIUS
see R. Barbera’s Genius talk this conference
19 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 1919
RB/JSS II
SE
input data location
Replica Catalog TOP
GIIS
. . .CE
Executable = "/usr/bin/env";Arguments = "zsh prod.dc1_wrc 00001";
VirtualOrganization="datatag";Requirements=Member(other.GlueHostApplicationSoftwareRunTimeEnvironment,"ATLAS-3.2.1" );Rank = other.GlueCEStateFreeCPUs;InputSandbox={"prod.dc1_wrc",“rc.conf","plot.kumac"};OutputSandbox={"dc1.002000.test.00001.hlt.pythia_jet_17.log","dc1.002000.test.00001.hlt.pythia_jet_17.his","dc1.002000.test.00001.hlt.pythia_jet_17.err","plot.kumac"};ReplicaCatalog="ldap://dell04.cnaf.infn.it:9211/lc=ATLAS,rc=GLUE,dc=dell04,dc=cnaf,dc=infn,dc=it";InputData = {"LF:dc1.002000.evgen.0001.hlt.pythia_jet_17.root"};StdOutput = " dc1.002000.test.00001.hlt.pythia_jet_17.log";StdError = "dc1.002000.test.00001.hlt.pythia_jet_17.err";DataAccessProtocol = "file";
JDL GLUE-aware files
WNATLAS sw
data
registration
GLUE-Schema basedInformation System
WorldGridTestbed
JDL
Job
GENIUS
UI
see WorldGrid Poster this conf.
20 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2020
CMS Applications Monte Carlo Production chain on Grid
– CMKIN: generation physics events with PYTHIA
– CMSIM: simulation of the detector with GEANT3 CMS production software installed in the WN’s Job workflow and data management
– CMKIN jobs sent by the RB to WN with CMS software, store the output at nearby SE
> register LFN to the RC
– CMSIM jobs sent by the RB to WN nearby SE> Register LFN to the RC
21 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2121
ATLAS Applications Grappa and Genius
submissions ATLAS Detector
Simulations– Simulation of the detector
response using ATLSIM (GEANT3)
– Based on DC1 Grid script
ATLAS production software installed in the WN’s
22 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2222
Grappa Portal Engine
Web browser interface
https
Cactus framework
Scriptinterfac
e
Resource A Resource Z. . . MAGDA: replica and metadataCompute Elements
Java CoGsubmission,monitoring
Storage Elements: - Disk/HPSS
inputfiles
Grappa and ATLASsee D. Enghthis conf.
23 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2323
Job Submission Animation
24 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2424
VO Monitoring Initial Requirements:
– Grid-level resource activity, utilization, and performance monitoring;
– VO-level resource activity and resource utilization monitoring;
– Customized views: > Hardware resources (clusters, sites, grids);
> VO usages, jobs, work-types;
Design Goals: – Scalability over large number of resources and networks;
– Simplicity and distributed architecture; Two approaches
– iVDGL: built on popular Ganglia resource monitoring package from UC Berkeley
– DataTAG: built on popular Nagios package http://www.nagios.org/
25 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2525
Web php client
Grid Aggregation
Round Robin DB Tool
Logging &
Bookeeping
UI
RB JSS CE
RRDB Tool
gmond gmond gmond gmond
Site a
RRDB Tool
gmond gmond gmond gmond
Site b
DataTAG
VO Ganglia
iVDGL
26 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2626
Site Level VO Usage and Policy
27 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2727
VO Nagios Monitoring based on Nagios (a host and service monitoring engine)
[detailed information on: http://www.nagios.org]
host local plug-ins – collect info from OS- CPU load- RAM- disk- jobs
MDS plug-ins - collect aggregate info from GRIS- number of running/waiting jobs- number of total/free CPUs
history graphs for all monitoring metrics
aggregate info/graphs per Site and Virtual Organization
28 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2828
Status and Summary Map
grid-aggregate monitors
3-levelstatus map
29 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 2929
VO Usage Graphs
MDS collected
site andaggregated montiors
see G. Tortone et. al., this conference
30 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 3030
WorldGrid Next Steps New developments in DataTAG:
– Test/experiment with SRM solutionSRM solution for Storage Element access (multiple implementations of the protocol)
– Test/experiment with advanced Data Management toolsData Management tools such as Globus-EDG/RLS
– Propose alternative Grid Resource DiscoveryGrid Resource Discovery mechanisms based on WEB services
– Improve the monitoringmonitoring tools taking advantage of OGSAOGSA
– Develop a WorldGrid GOCWorldGrid GOC, coordinated operations centers
Continue themes in iVDGL:– site-friendly installations, untouched by humans
– multi-VO (controlled use of shared resources)
– pursue concept of ‘projects’
31 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 3131
A project consists of
• A (typically small) list of distinguished names or VO(s).
• Email and phone contact.
• A software environment expressed as a Pacman package.
• Local disk space requirements.
• A url describing the project.
Basic site management operations:
• Join a project
• Leave a project
• Pause a project
Site manager commands
Projects as unit of access
32 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 3232
% worldgrid –info
-join <project>
-leave <project>
-pause <project>
-kill <project>
-update <project>
-getCA <CA>
-setForum <URL>
Example Site Manager Commands
33 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 3333
iVDGL WorldGrid FAQ Forum Help
Joined projects
Demo
ATLASDC2-higgs
ChimeraTest8
Projects Certified
Demo
CMS-DC2-SUSY
ChimeraTest8
ChimeraTest9
ATLASDC2-higgs
SDSC-scan45
Installed Software
WorldGrid
ScienceGrid
ProjectAccess
CAs
Batch jobs
Performance
10/150 G used in WorkSpace
History
34 R. Gardner CHEP 2003 – 24-28 March 2003 – nR. Gardner CHEP 2003 – 24-28 March 2003 – noo 3434
Conclusions Lessons from WorldGrid 2002
– Grid building> Packaging and configuration key
> GLUE meta-packaging study launched, report available
> Testing and site validation
– Interoperability> Configuration of common MDS schema allowed joint use of VDT
and EDG middleware installations
> good experience for LCG
– Integrate two very different grids> “Top down” EDG-style of Grids with high level services
> “Bottoms up” VDT-style grids providing core services with
– Transatlantic cooperation can be fun!