EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
SA1 – Maintenance and Support
Francesco Giacomini (INFN)SA1 Leader
1st EMI Periodic ReviewBrussels, 22 June 2011
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Outline
22/06/2011 1st EMI Periodic Review 2
• SA1 Objectives and Tasks in Y1• Key Performance Indicators• Lessons learned• Y2 Outlook
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611SA1 Objectives and Tasks
22/06/2011 1st EMI Periodic Review 3
• Software Maintenance: Ensure the efficient and transparent management of the maintenance process providing software problems analysis and resolutions
• Release Management: Ensure customers receive certified software releases of middleware services and components according to agreed release policies and quality of service attributes
• Quality Control: Increase the quality levels of the EMI software by contributing to the implementation of the Quality Control procedures defined by the project QA activity
• User Support: Establish the EMI User Support function and integrate it with the overall EGI, PRACE and VRCs user support channels
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
SA1 in EMI
22/06/2011 1st EMI Periodic Review 4
Project Management
NA1, JRA1
JRA1
JRA1SA1
SA1
SA2
Software & Services
Requirements
Defines
ImplementsCertifies
Release Candidate
Process definition
Process monitoring
NA2NA2
NA1, NA2
Collaborations, exploitation, sustainability
Training
Dissemination & Out-Reach
In-Reach
NA1
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
EMI Release and Software Timeline
22/06/2011 1st EMI Periodic Review 5
01/05/2010
Start EMI 1 EMI 2 EMI 3
Support & Maintenance
Support & Maintenance
Support & Maintenance
Supp. & Maint.
30/04/2011 30/04/2012 28/02/2013
Major releases
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Software Maintenance
22/06/2011 1st EMI Periodic Review 6
• Keep the stability of the software currently in production, introducing only the changes that are needed to keep the infrastructure running– Bug fixes or requested improvements– No backwards-incompatible changes
• Waiting for EMI 1 the Software Maintenance plan was applied as much as possible to existing software distributions– Following procedures inherited
from previous projects• Constant flow of updates
with no discontinuity betweenprojects
• Security vulnerabilities addressedfollowing the recommendationsof the Risk Assessment Team ofthe EGI Security VulnerabilityGroup
0
5000000
10000000
15000000
20000000
25000000
30000000
35000000
40000000
May
06
Aug
06N
ov 0
6Fe
b 07
May
07
Aug
07N
ov 0
7Fe
b 08
May
08
Aug
08N
ov 0
8Fe
b 09
May
09
Aug
09N
ov 0
9Fe
b 10
May
10
Aug
10N
ov 1
0Fe
b 11
May
11
# of jobs on EGI
Ensure the efficient and transparent management of the maintenance process providing software problems analysis and resolutions. [DoW] SO1.4, SO3.2Ensure the efficient and transparent management of the maintenance process providing software problems analysis and resolutions. [DoW] SO1.4, SO3.2
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Release Management
22/06/2011 1st EMI Periodic Review 7
EMI 1 – KebnekaiseReleased on May 12th, 2011
•54 products from ARC, dCache, gLite, UNICORE•Scientific Linux 5 (augmented with EPEL5), 64 bit, fully supported platform
– Selected products for SL5, 32 bit, available– Porting to SL6 starting soon
•333 binary packages– 86% of source packages– Digitally signed
•192 external dependencies– 165 from SL5/EPEL5, 27 managed by EMI
Ensure customers receive certified software releases of middleware services and components according to agreed release policies and quality of service attributes. [DoW] SO1.4, SO1.5, SO3.4
Ensure customers receive certified software releases of middleware services and components according to agreed release policies and quality of service attributes. [DoW] SO1.4, SO1.5, SO3.4
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
EMI 1 Preparation
22/06/2011 1st EMI Periodic Review 8
• Required a huge integration effort to put together in a coherent way all the existing software components– Common tools (ETICS, Savannah)– Common procedures
• Tools and procedures provided by SA2– Common external dependencies, same versions
• Globus libraries by IGE, taken from EPEL• Coordination of many people from different
experiences– Engineering Management Team (EMT)– Weekly phone conferences + e-mail discussions
• EMI 0 and four EMI 1 Release Candidates, with increasingly strict criteria, towards the final release– Integration builds run multiple times a day
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
User Support
22/06/2011 1st EMI Periodic Review 9
• Organization based on the EGEE experience, following reviewers' recommendations, and addressing ESFRI requirements
• Automatic notifications and reminders to SUs.• No discontinuity between projects
Establish the EMI User Support function and integrate it with the overall EGI, PRACE and VRCs user support channels. [DoW] SO1.4, SO3.1Establish the EMI User Support function and integrate it with the overall EGI, PRACE and VRCs user support channels. [DoW] SO1.4, SO3.1
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Quality Control
22/06/2011 1st EMI Periodic Review 10
• The activity performed by the Software Maintenance, Release Management and User Support tasks was reviewed by the Quality Control task– Following policies and check-lists contained in the
Software Quality Assurance Plan by SA2– Feedback was returned to SA2 to improve the SQAP
itself, about:• Alignment between SA2-defined metrics and KPIs• Tools to use to measure metrics and KPIs• Introduction of thresholds for metrics• Introduction of tolerance ranges for the checklists
• Coordination of the EMI 1 QC Verification– Joint activity with JRA1 and SA2
Increase the quality levels of the EMI software by contributing to the implementation of the Quality Control procedures defined by the project QA activity. [DoW] SO1.4
Increase the quality levels of the EMI software by contributing to the implementation of the Quality Control procedures defined by the project QA activity. [DoW] SO1.4
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611Security Assessment Plan
22/06/2011 1st EMI Periodic Review 11
• Under the responsibility of the SA1 QC task• Novelty wrt previous projects• Plan available and being implemented
starting from security-related software components– Based on the “First principles vulnerability
assessment” methodology• Collaboration with the EGI Software
Vulnerability Group and the University of Wisconsin
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611Service Level Agreements
22/06/2011 1st EMI Periodic Review 12
• One SLA signed so far, with EGI, in April 2011• It covers Software Development and Support
services needed for EGI– Provide clear description of roles and responsibilities– Provide a measurable description of the offered services
• E.g. target response time for GGUS tickets, depending on their priority
• Covered services– Software Releases– Requirements Analysis– Requirements and Software testing reports– Web-based support
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611Key Performance Indicators
22/06/2011 1st EMI Periodic Review 13
User Support•KSA1.1 – Number of incidents (i.e. tickets)•KSA1.2 – Incident resolution time
Software Maintenance•KSA1.3 – Number of problems (i.e. bugs)•KSA1.4 – Number of urgent changes•KSA1.5 – Change application time
Release Management•KSA1.6 – Number of releases•KSA1.7 – Number of release rollbacks
•Difficult to collect raw data in a uniform way for existing software– Some KPIs were postponed to after EMI 1 release
from GGUS
from RfC trackers
from Release tracker
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
KSA1.1 – Number of Incidents
22/06/2011 1st EMI Periodic Review 14
0
5
10
15
20
25
30
35
May 2010 - April 2011
Less Urgent Urgent Very Urgent Top Priority
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
KSA1.2 – Incident Resolution Time
22/06/2011 1st EMI Periodic Review 15
0
50
100
150
200
250
300
350
400
450
days
May 2010 - April 2011
Less Urgent Urgent Very Urgent Top Priority
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
KSA1.3 – Number of Problems
22/06/2011 1st EMI Periodic Review 16
0
1
2
3
4
5
6
7
Since EMI 1 Release
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Issues, Mitigation, Lessons Learned
• Putting together software developed in very different contexts is hard– EMI 1 represents the starting point
• Putting together the people who developed the above software is harder– Build consensus among project members before
approving plans and policies– It took a lot of time and patience– Agreed plans and policies are now available from
SA2• Application of Quality Assurance methods is
not straightforward– Improve communication
22/06/2011 1st EMI Periodic Review 17
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Issues, Mitigation, Lessons Learned /2
• Difficult/impossible to measure the performance of heterogeneous procedures in a uniform way– No KPIs for existing software– It is now easier, after the EMI 1 release
• Relationship with EGI Unified Middleware Distribution– We find their procedures overly complex but we’ll
do our best to comply with them• Not much effort/experience available within
the project for security assessments– Internal training– External collaborations
22/06/2011 1st EMI Periodic Review 18
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Outlook
22/06/2011 1st EMI Periodic Review 19
• Contribute EMI software to EGI UMD– According to the agreed procedures
• Support and maintain EMI 1• Continue support and maintenance of pre-
existing distributions– With diminishing effort and according to the service
end-of-life policy• Establish support relationships with other DCIs
(e.g. PRACE)• Work towards EMI 2 - Matterhorn
EMI 1
Apr 2011 Apr 2012Jun 2011 Feb 2012Jan 2012
Nov 2011Mar 2012
RC1
RC2
RC3
RC4
EMI 2
Development plans readyETICS infrastructure ready
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611
22/06/2011 1st EMI Periodic Review 20
Thank you
Top Related