Download - Informatica certification

Transcript
Page 1: Informatica certification

Barcelona, 2nd June 2008

OGF 23Quality for Grid & Grid for Quality: sharing experiences and perspectives of Grid and Open Source communities Engineering Ingegneria Informatica S.p.A.

INFSOM-RI-1234567

Adriano Rippa

Grid-QCMAutomation in Quality Assurance

Page 2: Informatica certification

2Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Summary

• The project

• Introduction to QA concepts

• The starting point of the study

• The Grid Quality Certification Model (Grid-QCM)

• The Grid-QCM dissemination timeline

• Some FAQs

• Grid-QCM adoption in a CMMi lev.3 organization

• Conclusions

Page 3: Informatica certification

3Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

The ETICS2 project

Page 4: Informatica certification

4Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

ETICS2 The ETICS second phase

Started in March 2008Duration: 2 yearsFunded by the European CommissionExtended name:

• “eInfrastructure for Testing Integration and Configuration of Software 2”

Purpose: • Automate the way a software is built, tested and certified.

It provides• Fully automatic build, test and certification tools

• A repository for the artefacts

• A Grid based solution optimal performances and multiple platform and test options

Page 5: Informatica certification

5Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Introduction to QA concepts

Page 6: Informatica certification

6Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-QCM focusProduct VS Process

Quality refers to several concepts like:

• Quality of the Implementation process

• Quality of the requirements management

• Quality of the service

• Quality of the software

Page 7: Informatica certification

7Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

TerminologyProduct VS Process

According to ISO 9126 documentation we can define:

• Measure: the number or category assigned to an attribute of an entity by making a measurement (sometimes used as synonymous of metric)

• Metric: The defined method to measure an attribute and the scale

• Measurement: The use of a metric to assign a value (which may be a number or category) from a scale to an attribute of an entity)

Page 8: Informatica certification

8Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

The starting point

Page 9: Informatica certification

9Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

State of the ArtWe have tons of stuff in the SoA!

State of the art provides • Hundreds of metrics

• Approaches to define user’s own metrics

• Many standards to asses the quality of processes and products

So, what’s the matter?!?

Page 10: Informatica certification

10Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

The matterEffort and resources

QA means initial investments and

Managing QA means devote resources to it

Many companies (e.g. SMEs) cannot afford the initial effort and do not recognise the promised increase of value.

• Only ~70 companies in the world are CMMi lev.5 certified • 50 of them are in India

• source: Gartner

• Just 25% of the companies in the world are CMMi lev.2 or above • source: Kulik, Weber: “Software Metrics Best Practices – 2001”

and “Software Metrics State of the Art – 2000”

Page 11: Informatica certification

11Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Current standards issuesWhy studying for one more QA certification model

Current quality assurance standards are useful but

• They need lot of time to be applied. • People need training

• Certification needs inspections to be achieved

• What for the short-lived consortia?

• They provide only theoretical guidelines which need to be adapted and implemented. What for homogeneity and comparability of results?

• It’s hard to systematically verify goodness of results: managing tools needed

Page 12: Informatica certification

12Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

The Grid Quality

Certification Model

Page 13: Informatica certification

13Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-Quality Certification ModelOur proposal to help developers

Grid-QCM is

• fully automatable in measuring and verifying activities to reduce investments and management effort

• not subjective, to certify the object not the process nor the organization

• product oriented, not process oriented

• easily adoptable within short-lived Projects

Page 14: Informatica certification

14Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-Quality Certification Modelpreliminary remarks

Grid-QCM has been developed according to

• The feedback received from expert people and potential users

• ISO standards guidelines:• Grid-QCM has been described according to ISO 25000, 14598

• Grid-QCM has been structured according to ISO 25041

• Quality attributes has been named according to ISO 9126

Page 15: Informatica certification

15Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-Quality Certification ModelThe ETICS vision

Grid-QCMGrid-QCM

ETICSETICS

SoftwareSoftware

ETICS grid ETICS grid

infrastructureinfrastructure

Defines Defines

metricsmetrics

RunsRuns

measuresmeasures

Allows Allows

TestTest

Allows Allows

automationautomation

Page 16: Informatica certification

16Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

The structureEvaluation modules

Grid-QCM is structured in Evaluation Modules (EM).

The set of evaluation techniques are grouped in families. Every family is an Evaluation Module

5 Evaluation Modules:• Static analysis

• Coding style

• Structural testing

• Functional testing

• Standards compliance

Page 17: Informatica certification

17Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Evaluation Modules 1/2

Static analysis• Quality characteristics:

• Reliability – maturity

• Maintainability – analysability

• Maintainability – changeability

• Maintainability – testability

• Static analysis of classes. Statistics on measures are used as predictor of quality characteristics.

Coding style• Quality characteristics:

• Maintainability – analysability

• Static analysis of the source code.

Page 18: Informatica certification

18Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Evaluation Modules 2/2

Structural testing• Quality characteristics:

• Functionality – accuracy• Reliability – maturity

• Structural testing to classes identified more likely to have many errors.

Functional testing• Quality characteristics:

• Functionality – accuracy• Functionality – interoperability• Reliability – maturity• Portability – adaptability• Portability - installability

• Platform compliance and to functional abilities of the software

Standards compliance• Quality characteristics:

• Functionality – standards compliance

Page 19: Informatica certification

19Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-QCM: Final Score

Final score will consist in:• A table summarizing the results with several possible views

• A list of passed and failed tests

• All the important information as:• Software product

• Platform

• Quality characteristics

• Standard compliance

• Identification of evaluation report

• Identification of certification body (organization, contact information)

• Certification data (dates, certification number)

• Electronic signature of certification record

Page 20: Informatica certification

20Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

The Grid-QCM dissemination timeline

Page 21: Informatica certification

21Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-QCM worldwide Since 2007

OCTOCT

DECDEC

ECHOGRID/EUChinagridECHOGRID/EUChinagrid

Conference 24-25 AprilConference 24-25 April

Beijing (CHINA)Beijing (CHINA)

MAYMAY

FEB ‘07FEB ‘07

OGF 20/EGEE UFOGF 20/EGEE UF

7-11 May7-11 May

Manchester (UK)Manchester (UK)

Belief ConferenceBelief Conference

25-28 June25-28 June

Rio de Janeiro (BRA)Rio de Janeiro (BRA)

OGF 21OGF 21

15-19 Oct15-19 Oct

Seattle (USA)Seattle (USA)

QUALIPSO QUALIPSO ConferenceConference

16-17 Jan 200816-17 Jan 2008

Rome (IT)Rome (IT)

NowNow

EGEE ’07EGEE ’07

1-5 Oct1-5 Oct

Budapest (Hun)Budapest (Hun)

EELA 3° Conference EELA 3° Conference 3-5 Dec 2007 3-5 Dec 2007

Catania (IT)Catania (IT)

ESA 3rd GRID & e-ESA 3rd GRID & e-Collaboration Workshop Collaboration Workshop

16-17 Jan 200816-17 Jan 2008

Frascati (IT)Frascati (IT)

JAN ‘08JAN ‘08

JUNEJUNE

OGF 23OGF 23

2-6 June2-6 June

Barcelona (SPA)Barcelona (SPA)

ETICS review ETICS review

15 Feb15 Feb

Geneva (CH)Geneva (CH)

Page 22: Informatica certification

22Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Some FAQs

Page 23: Informatica certification

23Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Some Questions and Answers Yes, but…

What’s good about Grid-QCM for my project?• This model (and the capability of automation) reduces the effort

in performing continuous build and test activities (e.g. coverage tests) on different SW releases.

What are the costs?• Using the ETICS tool people can have the model integrated in

the build and test application free of charge

My organisation is certified ISO/CMMI can I use Grid-QCM?• The model is a standalone quality certification model.• However it can be easily integrated in yet ISO/CMMI certified

organisations.

Page 24: Informatica certification

24Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-QCM in a

CMMi lev.3 organisation

Page 25: Informatica certification

25Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Grid-QCM in a CMMi lev.3 certified company The Engineering experience

Engineering Ingegneria Informatica S.p.A is certified as follows:• CMMi lev. 3

• For all production divisions

• ISO 9001• For all the ICT sector in which the organization is involved

• Since 1994 ( ISO9001:2000 from December 2002)

• NATO AQAP 2110/160• Since December 1996 (from 1996 to 2005 there were different

rules)

The whole Grid-QCM work is born from internal needs of• The research area

• The production area

Page 26: Informatica certification

26Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Engineering QA toolsKeeping the certification

Engineering tools to reach and maintain the CMMi lev. 3 certification:

Spago4Q (www.spago4q.org) • measures processes, performances and bugs through the

integration with other tools• Internally developed and available for free• Examples of calculated metrics:

• Test Coverage• Anomalies distribution• Effort used to solve anomalies• Cost of not founded anomalies• Metrics about requirements• Metrics about risk management• …

Page 27: Informatica certification

27Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

It is not enough…What ETICS and Grid-QCM can do for ENG

…Some quality aspects are not covered

Grid-QCM + the ETICS suite • Measures other quality aspects such as:

• Quality of the code

• Compliance with standards

• Compliance with OS

• …

• Can be integrated with the Spago4Q suite

So, Engineering is managing the activities to make Grid-QCM + ETICS a tool to keep the CMMi lev.3 certification

Page 28: Informatica certification

28Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Conclusions

Page 29: Informatica certification

29Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Conclusions And further steps

Grid-QCM is a certification model• Automatable

• Implemented by default in the ETICS tool

• Ready to interact with classical standards

Adopting Grid-QCM requires little human effort

During the ETICS 2 project • Grid-QCM is going to be validated on-the-field with at first two

(Dec. ‘08) and then four projects (Dec. ’09)

• Grid-QCM will be proposed for the ISO standardisation (2009)

Page 30: Informatica certification

30Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

E-mail addresses

We wish Grid-QCM and ETICS to be as simple and effective as possible to use. For this we need your feedback

[email protected]

Page 31: Informatica certification

31Grid-QCMAutomation in Quality Assurance

INFSOM-RI-1234567

Open Grid Forum 23Barcellona, 2° June 2008

Q&A Thank you!

http://www.eticsproject.eu