Cmpe 589
Spring 2008
Software Quality Metrics
• Product product attributes– Size, complexity, design features, performance,
quality level
• Process Used to improve development and maintenance processes
– Effectiveness of defect removal, the response time of the fix process
• Project Describe project attributes/characteristics and execution
– The number of developers, the staffing pattern, cost, schedule and productivity
Software Quality Engineering
Seeks relationships among in-process metrics, project characteristics, and end-product quality (then use this to improve)
Product Quality Metrics
• Mean time to failure (MTTF)
• Defect density
• Customer problems
• Customer satisfaction
Product Quality Metrics
• Measured by – the number of “bugs” (functional defects)– How long the software can run before it
“crashes”
• MTTF is used in safety critical systems– Measures the time between failures
• Defect density is used in commercial apps– Defects relative to the software size
Defect vs. Failures
• Human mistakes: resulting in incorrect software operation
• Failure- software module no longer carries out intended function or performance level
Defect Density Metrics
• Time frame – L.O.P – Life of Product – Four years (?)
• LOC – Lines of Code
• KLOC – Thousand lines of code
• Time Frame/ LOC or KLOC
Lines of Code
• Count only executable lines• Count executable lines plus data definitions• Count executable lines, data definitions, and
comments• Count executable lines, data definitions,
comments, and job control language• Count lines as physical lines on an input screen• Count lines as terminated by logical delimiters
Lines of Code
• In the context of defect rate calculation
• Productivity studies– The amount of LOC is negatively correlated
with design efficiency
• Enhancements and new versions– LOC count for the entire product and the
changed code– Defect tracking
Maintenance Process
• Answer: compute defect rates for new and old code
• Change flagging- (comments) – ID number linked to specific requirement, version release number
Function Points
• A collection of executable statements that performs a certain task together with declarations of the formal parameters and local variables manipulated by those statements
• Originated by Albrecht at IBM in mid 70s
Function Points
• Weighted total of 5 major components:– Number of external inputs (transaction types)X4– Number of external outputs (report types)X5– Number of logical internal files (the ones that user
may concieve, not the physical files)X10– Number of external interface files (files accessed by
the apps but not maintained by it)X7– Number of external inquiries (types of online inquiries
supported) X4
• These are average weighting factors
Function Points
• There are low and high weighting factors depending on the complexity assessment of the app.:– External input: low complexity,3; high complexity, 6– External output: low complexity, 4; high complexity, 7– Logical internal file:low complexity, 7; high complexity,
15– External interface file: low complexity, 5; high
complexity, 10– External inquiry: low complexity, 3; high complexity, 6
Function Points
• Complexity is defined as set of standards according to objective guidelines– e.g. For the external output: if the no of data
element types is 20 or more and the no of file types referenced is 2 or more, than complexity is high.
• Calculate Function Counts (FC):FC = Σ Σ w x
i=1
5
j=1
3
ij ij
Function Points• Scale from 0 to 5 to assess the impact of 14 general system
characteristics:– Data communications– Distributed functions– Performance– Heavily used configuration– Transaction rate– Online data entry– End-user efficiency– Online update– Complex processing– Reusability– Installation ease– Operational ease– Multiple sites– Facilitation of change
Function Points
• Sum up the scores and calculate Value Adjustment Factor (VAF)
VAF = 0.65 + 0.01Σc¡c¡ = the score for general system characteristic i.
• The number of function points is obtained:
FP = FC x VAFInternational Function Point User’s Group
Standard (IFPUG, 1999)
i=1
14
Function Points
• Issues related to the function point metric:– More research is needed on: meaning of FP
and the derivation algorithm– Various other standards than IFPUG– Time consuming and expensive– Accurate calculation need certified FP
specialists
Function Points: example
• From Jones (2000). Software Assessments, Benchmarks, and Best Practices– The average no of software defects in US is approx. 5
per function point during the entire life cycle– Defect removal efficiency is calculated by the level of
CMM– The estimated defect rates per function point:
• Level 1: 0.75• Level 2: 0.44• Level 3: 0.27• Level 4: 0.14• Level 5: 0.05
Customer Satisfaction
• Customer problems metric- problems using product • Problems per user month (PUM) = (number of valid
defects/time period)• Compute this every month if you want to lower PUM • Ex. Number of installed licenses times the number
sold per month to reduce PUM– Improve development process and reduce product defects – Reduce non-defects oriented problems- improve, support,
usability, documentation, communication, training – Increase sales of installed licenses
Scopes of Three Quality Metrics
Defects
CustomerSatisfaction
CustomerProblems
Top Related