Cmpe 589

Spring 2008

Software Quality Metrics

• Product product attributes– Size, complexity, design features, performance,

quality level

• Process Used to improve development and maintenance processes

– Effectiveness of defect removal, the response time of the fix process

• Project Describe project attributes/characteristics and execution

– The number of developers, the staffing pattern, cost, schedule and productivity

Software Quality Engineering

Seeks relationships among in-process metrics, project characteristics, and end-product quality (then use this to improve)

 

Product Quality Metrics

• Mean time to failure (MTTF)

• Defect density

• Customer problems

• Customer satisfaction

Product Quality Metrics

• Measured by – the number of “bugs” (functional defects)– How long the software can run before it

“crashes”

• MTTF is used in safety critical systems– Measures the time between failures

• Defect density is used in commercial apps– Defects relative to the software size

Defect vs. Failures

• Human mistakes: resulting in incorrect software operation

• Failure- software module no longer carries out intended function or performance level

Defect Density Metrics

• Time frame – L.O.P – Life of Product – Four years (?)

• LOC – Lines of Code

• KLOC – Thousand lines of code

• Time Frame/ LOC or KLOC

Lines of Code

• Count only executable lines• Count executable lines plus data definitions• Count executable lines, data definitions, and

comments• Count executable lines, data definitions,

comments, and job control language• Count lines as physical lines on an input screen• Count lines as terminated by logical delimiters

Lines of Code

• In the context of defect rate calculation

• Productivity studies– The amount of LOC is negatively correlated

with design efficiency

• Enhancements and new versions– LOC count for the entire product and the

changed code– Defect tracking

Maintenance Process

• Answer: compute defect rates for new and old code

• Change flagging- (comments) – ID number linked to specific requirement, version release number

Function Points

• A collection of executable statements that performs a certain task together with declarations of the formal parameters and local variables manipulated by those statements

• Originated by Albrecht at IBM in mid 70s

Function Points

• Weighted total of 5 major components:– Number of external inputs (transaction types)X4– Number of external outputs (report types)X5– Number of logical internal files (the ones that user

may concieve, not the physical files)X10– Number of external interface files (files accessed by

the apps but not maintained by it)X7– Number of external inquiries (types of online inquiries

supported) X4

• These are average weighting factors

Function Points

• There are low and high weighting factors depending on the complexity assessment of the app.:– External input: low complexity,3; high complexity, 6– External output: low complexity, 4; high complexity, 7– Logical internal file:low complexity, 7; high complexity,

15– External interface file: low complexity, 5; high

complexity, 10– External inquiry: low complexity, 3; high complexity, 6

Function Points

• Complexity is defined as set of standards according to objective guidelines– e.g. For the external output: if the no of data

element types is 20 or more and the no of file types referenced is 2 or more, than complexity is high.

• Calculate Function Counts (FC):FC = Σ Σ w x

i=1

5

j=1

3

ij ij

Function Points• Scale from 0 to 5 to assess the impact of 14 general system

characteristics:– Data communications– Distributed functions– Performance– Heavily used configuration– Transaction rate– Online data entry– End-user efficiency– Online update– Complex processing– Reusability– Installation ease– Operational ease– Multiple sites– Facilitation of change

Function Points

• Sum up the scores and calculate Value Adjustment Factor (VAF)

VAF = 0.65 + 0.01Σc¡c¡ = the score for general system characteristic i.

• The number of function points is obtained:

FP = FC x VAFInternational Function Point User’s Group

Standard (IFPUG, 1999)

i=1

14

Function Points

• Issues related to the function point metric:– More research is needed on: meaning of FP

and the derivation algorithm– Various other standards than IFPUG– Time consuming and expensive– Accurate calculation need certified FP

specialists

Function Points: example

• From Jones (2000). Software Assessments, Benchmarks, and Best Practices– The average no of software defects in US is approx. 5

per function point during the entire life cycle– Defect removal efficiency is calculated by the level of

CMM– The estimated defect rates per function point:

• Level 1: 0.75• Level 2: 0.44• Level 3: 0.27• Level 4: 0.14• Level 5: 0.05

Customer Satisfaction

• Customer problems metric- problems using product • Problems per user month (PUM) = (number of valid

defects/time period)• Compute this every month if you want to lower PUM • Ex. Number of installed licenses times the number

sold per month to reduce PUM– Improve development process and reduce product defects – Reduce non-defects oriented problems- improve, support,

usability, documentation, communication, training – Increase sales of installed licenses

Scopes of Three Quality Metrics

Defects

CustomerSatisfaction

CustomerProblems