Post on 21-Feb-2016
description
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
EMI 1 and WLCG
Francesco Giacomini (INFN) - SA1Morris Riedel (JSC) – JRA1
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• EMI 1 Release Plan– Francesco Giacomini (INFN)
• EMI 1 Developments– Morris Riedel (JSC)
2011-02-09 WLCG GDB, CERN 2
Outline
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 3
Release Plan
Start EMI 1 EMI 2 EMI 3
Support & Maintenance
Support & Maintenance
Support & Maintenance
Supp. & Maint.Major releases
01/05/2010 30/04/2011 30/04/2012 28/02/2013
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
WLCG GDB, CERN 4
• Periodic major releases of the distribution– Tentatively once per year– Right place to introduce new features and
backwards-incompatible changes– Minimize number of changes within a major release
• Clear separation between maintaining what is in production and introducing new features– SA1 vs JRA1– People (developers, testers, etc.) are the same, but
the rules are different in the two contexts
2011-02-09
Stability vs Innovation
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 5
Release Plan
Start EMI 1 EMI 2 EMI 3
Support & Maintenance
Support & Maintenance
Support & Maintenance
Supp. & Maint.Major releases
01/05/2010 30/04/2011 30/04/2012 28/02/2013
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Two EMI major releases supported at any time→Lifetime of an EMI major release is two years– Full support: 18 months– Security only: another 6 months– Applies to previous mw distributions as well• e.g. gLite 3.2 and arc 0.8.x would be fully
supported until October 2011, with security updates until April 2012
2011-02-09 WLCG GDB, CERN 6
Support
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Guiding principles: simplification and adoption of established best practices
• EMI 1 reference platform: SL5/64 + EPEL– SL6 and Debian available in EMI 2, maybe sooner already during EMI 1 for
some components• One EMI repository (per major release) for all services
– base + updates + security• Following EPEL packaging policies
– e.g. installation in /usr, not in /opt/glite– Full compliance may be achieved later
• Metapackages just for installation– Cannot be used to version the installation of a certain product
• In general, no upgrade path between major releases– Including from gLite 3.2 to EMI 1
2011-02-09 WLCG GDB, CERN 7
Deployment Highlights
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 8
EMI 1 Developments At a Glance
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Development impacts: Integrated Release– EPEL-compliant component installation (/usr)– Change of Globus from VDT to EPEL
(IGE project officially supports the EPEL version)• Key Developments– VOMS using standard SSL (instead of GSI)– Started migration towards GLUE2 • Improved information validation checkings
– Improvements for Data components• NFS4.1, WebDAV, better performance, etc.
2011-02-09 WLCG GDB, CERN 9
EMI 1 Developments At a Glance
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 10
EMI 1 Developments – Security
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• VOMS– Drop of the dependencies on Globus Toolkit• Will simplify the installation (+ smaller size)
– Completed integration of VOMRS functionalities in VOMS-Admin• (VOMRS itself is going to end support)
– Usability improvements: Better error messages, cleaner handling of invalid inputs
– REST interface to obtain VOMS ACs• No major features in security components 2011-02-09 WLCG GDB, CERN 11
EMI 1 Developments – gLite Security
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 12
EMI 1 Developments – Compute
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• CREAM-CE (CREAM, BLAH, CEMON)– Experimental GLUE 2 support • Working on solutions with information providers• GLUE1.3 still supported
– ARGUS Integration for both CREAM and CEMon• Config: Using ARGUS or existing authZ with gJAF
– Better support for multi core allocations • (i.e. JDL attributes WholeNodes, HostNumber, etc.)
2011-02-09 WLCG GDB, CERN 13
EMI 1 Developments – CREAM
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Workload Management System (WMS)– Work in progress: Adoption of GLUE2, but in
WMS needs to be further investigated– Manage GLUE1.3 and GLUE2 in JDL unclear– Support for 'feedback‘• Jobs 'stuck' in a queue for too long can be
migrated on another Computing Element• At configuration time: feature must be enabled to
work as described
2011-02-09 WLCG GDB, CERN 14
EMI 1 Developments – WMS
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• EMI 1 A-REX based ARC CE– End of life of
Grid Manager– The same interfaces are kept• In addition: New Web service interface
– Existing features kept– New Features: plugins, improved input/output
data handling, GLUE2 information publishing, new security framework
– ARGUS support (experimental)2011-02-09 WLCG GDB, CERN 15
EMI 1 Developments – ARC CE
http://www.nordugrid.org/arc/gm-eol.html
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 16
EMI 1 Developments – Data
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• dCache – Initial GLUE2 support for exposing storage information
in parallel to GLUE1.3– NFS4.1 Support (client comes free most likely in SL6)– WebDAV Support – Support for ARGUS blacklisting– Significantly improved configuration system• Already tested at selected Tier 1's and Tier 2's.
– More details, tutorials and hands-on will be provided during the "5. dCache Workshop" in Goettingen (16/17 March)
2011-02-09 WLCG GDB, CERN 17
EMI 1 Developments – dCache
http://indico.desy.de/conferenceDisplay.py?confId=3959
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• File Transfer Service (FTS)– Support for resuming gridftp transfers where the
endpoints allow it
• Disk Pool Manager (DPM)– GLUE2 Support exposing storage information
(gfal/lcg_util support expected shortly after EMI 1)– Improved performance in dpm-drain
(parallel file replication)
2011-02-09 WLCG GDB, CERN 18
EMI 1 Developments – FTS & DPM
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 19
EMI 1 Developments – Infrastructure
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• No major new features in BDII– Information flow largely based on GLUE1.3
moving towards GLUE2• But Information content improvement– Low confidence level that the values for
some attributes are correct– Better information providers are needed in the
overall set of relevant components– Better test tools available that we work on• GLUE2 validation, GStat tests
2011-02-09 WLCG GDB, CERN 20
EMI 1 Developments – Information
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
2011-02-09 WLCG GDB, CERN 21
EMI 2 Outlook
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Solution for storage elements and catalogue synchronization
• Common Storage Accounting Records– Input to standardization activities
• Common EMI Execution Service Interface– Preserve backwards-compatibility
• Common Security Profiles– SAML&XACML: Common Attribute Profiles
• Experimental adoption of messaging2011-02-09 WLCG GDB, CERN 22
EMI 2 Outlook
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Thank you
23
EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611
2011-02-09 WLCG GDB, CERN