Done by,B.Shruthi

(11109A067)

Software Quality Metrics. Types of Software Quality Metrics. Three groups of Software Quality Metrics. Difference Between Errors, Defects, Faults,

and Failures. Lines of code. Function Points. Customer Satisfaction Metrics

The subset of metrics that focus on quality Software quality metrics can be divided

into:End-product quality metrics In-process quality metrics

The essence of software quality engineering is to investigate the relationships among in-process metric, project characteristics , and end-product quality, and, based on the findings, engineer improvements in quality to both the process and the product.

Product metrics – e.g., size, complexity, design features, performance, quality level

Process metrics – e.g., effectiveness of defect removal, response time of the fix process

Project metrics – e.g., number of software developers, cost, schedule, productivity

Product quality In-process quality Maintenance quality Product Quality Metrices

Intrinsic product quality Mean time to failureDefect density

Customer relatedCustomer problemsCustomer satisfaction

Intrinsic product quality is usually measured by: the number of “bugs” (functional defects) in

the software (defect density), orhow long the software can run before

“crashing” (MTTF – mean time to failure) The two metrics are correlated but

different

An error is a human mistake that results in incorrect software.

The resulting fault is an accidental condition that causes a unit of the system to fail to function as required.

A defect is an anomaly in a product. A failure occurs when a functional unit of a

software-related system can no longer perform its required function or cannot perform it within specified limits

This metric is the number of defects over the opportunities for error (OPE) during some specified time frame.

We can use the number of unique causes of observed failures (failures are just defects materialized) to approximate the number of defects.

The size of the software in either lines of code or function points is used to approximate OPE.

Possible variationsCount only executable linesCount executable lines plus data definitionsCount executable lines, data definitions, and

commentsCount executable lines, data definitions,

comments, and job control languageCount lines as physical lines on an input screenCount lines as terminated by logical delimiters

Other difficultiesLOC measures are language dependentCan’t make comparisons when different

languages are used or different operational definitions of LOC are used

For productivity studies the problems in using LOC are greater since LOC is negatively correlated with design efficiency

Code enhancements and revisions complicates the situation – must calculate defect rate of new and changed lines of code only

Depends on the availability on having LOC counts for both the entire produce as well as the new and changed code

Depends on tracking defects to the release origin (the portion of code that contains the defects) and at what release that code was added, changed, or enhanced

A function can be defined as a collection of executable statements that performs a certain task, together with declarations of the formal parameters and local variables manipulated by those statements.

In practice functions are measured indirectly.

Many of the problems associated with LOC counts are addressed.

The number of function points is a weighted total of five major components that comprise an application.Number of external inputs x 4Number of external outputs x 5Number of logical internal files x10Number of external interface files x 7Number of external inquiries x 4

The function count (FC) is a weighted total of five major components that comprise an application.Number of external inputs x (3 to 6)Number of external outputs x (4 to 7)Number of logical internal files x (7 to 15)Number of external interface files x (5 to 10)Number of external inquiries x (3 to 6)

the weighting factor depends on complexity

Each number is multiplied by the weighting factor and then they are summed.

This weighted sum (FC) is further refined by multiplying it by the Value Adjustment Factor (VAF).

Each of 14 general system characteristics are assessed on a scale of 0 to 5 as to their impact on (importance to) the application.

1. Data Communications2. Distributed functions3. Performance4. Heavily used configuration5. Transaction rate6. Online data entry7. End-user efficiency

8. Online update9. Complex processing10. Reusability11. Installation ease12. Operational ease13. Multiple sites14. Facilitation of change

VAF is the sum of these 14 characteristics divided by 100 plus 0.65.

Notice that if an average rating is given each of the 14 factors, their sum is 35 and therefore VAF =1

The final function point total is then the function count multiplied by VAF

FP = FC x VAF

Customer problems are all the difficulties customers encounter when using the product.

They include: Valid defects Usability problems Unclear documentation or information Duplicates of valid defects (problems already

fixed but not known to customer) User errors

The problem metric is usually expressed in terms of problems per user month (PUM)

PUM = Total problems that customers reported for a time period <divided by> Total number of license-months of the software during the periodwhere

Number of license-months = Number of the install licenses of the software x Number of months in the calculation period

Improve the development process and reduce the product defects.

Reduce the non-defect-oriented problems by improving all aspects of the products (e.g., usability, documentation), customer education, and support.

Increase the sale (number of installed licenses) of the product.

Defect Rate Problems per User-Month (PUM)

Numerator Valid and unique product defects

All customer problems (defects and nondefects, first time and repeated)

Denominator Size of product (KLOC or function point)

Customer usage of the product (user-months)

Measurement perspective

Producer—software development organization

Customer

Scope Intrinsic product quality Intrinsic product quality plus other factors

CustomerSatisfactionIssues

Customer Problems

Defects

Customer satisfaction is often measured by customer survey data via the five-point scale:Very satisfiedSatisfiedNeutralDissatisfiedVery dissatisfied

CUPRIMDSO Capability (functionality) Usability Performance Reliability Installability Maintainability Documentation Service Overall

FURPSFunctionalityUsabilityReliabilityPerformanceService

1. Percent of completely satisfied customers2. Percent of satisfied customers (satisfied and

completely satisfied)3. Percent of dissatisfied customers

(dissatisfied and completely dissatisfied)4. Percent of nonsatisfied customers (neutral,

dissatisfied, and completely dissatisfied)

Defect density during machine testing Defect arrival pattern during machine

testing Phase-based defect removal pattern Defect removal effectiveness

Defect rate during formal machine testing (testing after code is integrated into the system library) is usually positively correlated with the defect rate in the field.

The simple metric of defects per KLOC or function point is a good indicator of quality while the product is being tested.

Scenarios for judging release quality: If the defect rate during testing is the same

or lower than that of the previous release, then ask: Does the testing for the current release deteriorate? If the answer is no, the quality perspective is

positive. If the answer is yes, you need to do extra

testing.

Scenarios for judging release quality (cont’d): If the defect rate during testing is

substantially higher than that of the previous release, then ask: Did we plan for and actually improve testing effectiveness? If the answer is no, the quality perspective is

negative. If the answer is yes, then the quality perspective

is the same or positive.

The pattern of defect arrivals gives more information than defect density during testing.

The objective is to look for defect arrivals that stabilize at a very low level, or times between failures that are far apart before ending the testing effort and releasing the software.

The defect arrivals during the testing phase by time interval (e.g., week). These are raw arrivals, not all of which are valid.

The pattern of valid defect arrivals – when problem determination is done on the reported problems. This is the true defect pattern.

The pattern of defect backlog over time. This is needed because development organizations cannot investigate and fix all reported problems immediately. This metric is a workload statement as well as a quality statement.

This is an extension of the test defect density metric.

It requires tracking defects in all phases of the development cycle.

The pattern of phase-based defect removal reflects the overall defect removal ability of the development process.

DRE = (Defects removed during a development phase <divided by> Defects latent in the product) x 100%

The denominator can only be approximated.

It is usually estimated as:Defects removed during the phase

+Defects found later

When done for the front end of the process (before code integration), it is called early defect removal effectiveness.

When done for a specific phase, it is called phase effectiveness.

The goal during maintenance is to fix the defects as soon as possible with excellent fix quality

The following metrics are important:Fix backlog and backlog management indexFix response time and fix responsivenessPercent delinquent fixesFix quality