20150620 Meetup U-Qasar - Obtaining an integrated and objective overview of the SQ in the new...

Obtaining an integrated and objective overview of the SQ in the new changing

environments Manu García (INNOPOLE) 2015

Software Craftsmanship

June 20th

2015

Software-CraftsmanshipJune 20th, 2015. Innopole (Spain)

FP7-ICT2011-8. Project #: 318082

10:30 Presentation and Coffee10:35 U-QASAR Project11:00 SaaS Market Overview11:20 Demo12:10 Open Source discussion12:30 Conclusion

PROGRAM


FP7-ICT2011-8. Project #: 318082

About Us. (1)

• Spanish consultancy and innovation center located in Toledo.

• Typically networked SME. Several partners for different purposes around Europe.


FP7-ICT2011-8. Project #: 318082

About Us. (2)

• Our research projects:

www.uqasar.eu Platform and methodology for objectively measuring software quality in rapidly changing environments.

www.extremefactories.euPlatform for the management of innovation campaigns. Radical innovation: configuration of the campaign’s scope (Open or with different levels of restriction).


FP7-ICT2011-8. Project #: 318082

The U-QASAR Project (1)

• U-QASAR EU funded research project under FP7

• 8 organizations from Finland, Norway, Germany, Greece and Spain: Experts in SW Quality, Ontology modeling and implementation, SW Engineering methods, and SW Measurement.


FP7-ICT2011-8. Project #: 318082

The U-QASAR Project (2)

• Objective: • Create a solution to provide project managers,

software engineers, testers and other actors with an objective measurement of quality for: Software Development:

–Processes–Products


FP7-ICT2011-8. Project #: 318082

The U-QASAR Concept (1)

• Two elements in U-QASAR:

– A methodology for gathering and exploiting data about the progress and quality of software development projects and products.

– A platform with semantic capabilities to obtain an objective value of the software development process quality and the resulting product.


FP7-ICT2011-8. Project #: 318082


• A Model-based Methodology:–How to identify Quality Objectives.–How to plan a Measurement Program: definition of Quality

Indicators and Measurement.–How to select the metrics.–How to collect data (auto/manual).–How to assess the validity of the measurement.–How to provide valuable feedback out of the

measurements.


FP7-ICT2011-8. Project #: 318082

• The U-QASAR Services (i):

U-QASAR monitoring services, which will monitor the contexts of the development life cycle and the product to be aware of any potential change that may have an effect on the Quality Requirements.

U-QASAR enhancement services, that will propose specific updates and new assessments to verify its fulfilment depending on the changes detected in the context by the U-QASAR monitoring services.



FP7-ICT2011-8. Project #: 318082

• The U-QASAR Services (ii):

U-QASAR analytical services, which will analyse the development history data of the company in order to propose specific values for weights and thresholds of each of the concepts that will be evaluated during the assessment activities.

U-QASAR reporting services, which will produce reports offering a quantified view of the quality of the process and product. The reports will offer different abstraction levels depending on the targeted audience (development team, project management, customers).



FP7-ICT2011-8. Project #: 318082


• The Platform:


FP7-ICT2011-8. Project #: 318082

Support to SW development LC (1)

• U-QASAR solution is:

–Paradigm agnostic! (Agile or traditional LCs - V, waterfall, RUP…).

–Applicable to any type of software product or process.

–Adaptable to the company’s Quality Model.


FP7-ICT2011-8. Project #: 318082


Requirements

Implem. Unit Testing

Validation

Design Integration Testing

# Redundant reqs# Incomplete reqs# Format errors

% Blocking cr viol.% Critical cr violComplexity…

% Successful Uts% Coverage

# Redundant specs# Incomplete specs# Format errors

% Successful TCs% Coverage

# Misconceptions

% Successful TCs% Coverage

# Misconceptions

• What to measure?


FP7-ICT2011-8. Project #: 318082


• What to measure?• % Incomplete stories, i.e. Stories lacking story

points, BV estimations, categorization or classification.

• # Scheduled vs. Solved issues: Ratio of solved/closed issues per sprint.

• Technical debt: Indicator aggregating data about pending bugs, known issues, etc after each sprint.

• % of stories accepted per sprint.• Perception of the customer & Team

enthusiasm by means of surveys.• # Issues identified in retrospective meetings.


FP7-ICT2011-8. Project #: 318082


Requirements, Stories, Issues

Implementation & Unit Testing

Validation, Acceptance or Integration Tests

Design

• Where to look:

Wikis, Confluence, JIRA, Greenhopper, Trac, Doors …

SpiraTest, TestRobot, HP QA Center, Selenium, JIRA …

Confluence, JIRA, SONAR

SVN, SONAR

Deployment Bamboo, Jenkins, Continuum…


FP7-ICT2011-8. Project #: 318082

How does it work? (1)

• Quality Model• Create a Q-model (or Import an existing one)

– It can be based on existing standards or models.– It can be an organization’s self-made model.– It can deal with process, products, or both.

Creation of a model consists of identifying expected values, weights and thresholds for:

• Quality Objectives, or aggregation of indicators to evaluate the achievement of an objective.

• Quality Indicators, or aggregation of metrics used to evaluate• Quality Metrics, or raw measurements obtained automatically or manually


FP7-ICT2011-8. Project #: 318082


Adapters• Link Quality Metrics to existing adapters (or Introduce

values manually).

Main connectors:

• Developed: JIRA, SONARQube, GitLab, Cubes, • To be decided: Confluence, Greenhopper, SpiraTest, TestRobot,

Jenkins, SVN …


FP7-ICT2011-8. Project #: 318082


Quality Project• Create Quality Objectives (QOs) or Select existing

ones.• Select or create Quality Indicators (QIs) by aggregating

metrics (MEs).

CONTEXT AWARENESS• QOs’ and QIs’ weights are adapted depending on the LC stage

(e.g. In the requirements stage a bigger relevance is given to completeness of specifications).


FP7-ICT2011-8. Project #: 318082


Business Intelligence & Reporting


FP7-ICT2011-8. Project #: 318082


• Support in the implementation and follow-up of certifications or standards (e.g. CMMi, TMMi, SPICE, ISO…)

• How? –Requirements of the certification model ->Quality

Objectives of the model–QIs and metrics of the QOs will indicate the degree

of fulfillment of the standard/certification model.


FP7-ICT2011-8. Project #: 318082

A TMMi Example

• Objective 1 - Establish a Test Policy. A test policy, aligned with the business (quality) policy, is established and agreed upon by the stakeholders. –Indicator 1.1Define test goals. Define and maintain

test goals based upon business needs and objectives. • Mission statements• Business and user needs regarding the products• Business drivers • Main goals of a quality program


FP7-ICT2011-8. Project #: 318082

U-QASAR Tag

U-QASAR QO U-QASAR QO

U-QASAR QI U-QASAR QI

U-QASAR Metric

U-QASAR Metric


FP7-ICT2011-8. Project #: 318082

SaaS MarketOverview

Competition & trends


FP7-ICT2011-8. Project #: 318082

SQA Tools


FP7-ICT2011-8. Project #: 318082

Saa

S A

LM

ove

rvie

w


FP7-ICT2011-8. Project #: 318082

VE

RS

ION

ON

EProject Burndown

Test Trend

Work Item TreeMap


FP7-ICT2011-8. Project #: 318082

Visual S

tudio Online


FP7-ICT2011-8. Project #: 318082

Saa

STe

st M

anag

emen

tTool Agile

fitFlexibility Tool Agile

fit Flex.

✔ ✔✔ ✔ ✔

✔ ✔ ✔ ✔

✔ ✔ ✔ ✔

✔

✔

✔ ✔ ✔✔

✔ ✔✔ ✔ ✘✘

✔ ✘ ✔✔ ✔✔

✔

✔

✔ ✔✔✔ ✔✔


FP7-ICT2011-8. Project #: 318082

QM

ET

RY


FP7-ICT2011-8. Project #: 318082

Zep

hyr

-Jir

a


FP7-ICT2011-8. Project #: 318082

SaaS Software metrics & Test

Mobile and web app testing (load and testing)

Static Code analysis


FP7-ICT2011-8. Project #: 318082

Asian rack Keynote rack


FP7-ICT2011-8. Project #: 318082

Gluing all together


FP7-ICT2011-8. Project #: 318082


FP7-ICT2011-8. Project #: 318082

How does it work?


FP7-ICT2011-8. Project #: 318082

Open Source discussion!


FP7-ICT2011-8. Project #: 318082

I’ll be happy to hear from you!

Manu [email protected]


FP7-ICT2011-8. Project #: 318082

.com/uqasaruqasar.eu /uqasar

Where to find us

Many Thanks!!

20150620 Meetup U-Qasar - Obtaining an integrated and objective overview of the SQ in the new...

Technology

Transcript of 20150620 Meetup U-Qasar - Obtaining an integrated and objective overview of the SQ in the new...