3. Software product quality metrics The quality of a product: -the “totality of characteristics...
-
Upload
stephanie-cummings -
Category
Documents
-
view
215 -
download
3
Transcript of 3. Software product quality metrics The quality of a product: -the “totality of characteristics...
![Page 1: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/1.jpg)
3. Software product quality metrics3. Software product quality metrics
The quality of a product:
-the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.
Metrics of the external quality attributes
producer’s perspective: “conformance to requirements”
customer’s perspective: “fitness for use” - customer’s expectations
![Page 2: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/2.jpg)
Some of the external attributes cannot be measured: usability,
maintainability, installability.
Two levels of software product quality metrics:
Intrinsic product quality
Customer oriented metrics:
Overall satisfaction
Satisfaction to specific attributes:
capability (functionality), usability,
performance, reliability, maintainability, others.
![Page 3: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/3.jpg)
Intrinsic product quality metrics:
Reliability: number of hours the software can run before a
failure
Defect density (rate):
number of defects contained in software, relative to its size.
Customer oriented metrics:
Customer problems
Customer satisfaction
![Page 4: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/4.jpg)
3.1. Intrinsic product quality metrics
Reliability --- Defect density
• Correlated but different!
• Both are predicted values.
• Estimated using static and dynamic models
Defect: an anomaly in the product (“bug”)
Software failure: an execution whose effect is not conform to software specification
![Page 5: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/5.jpg)
3.1.1. Reliability3.1.1. Reliability
Software Reliability:
The probability that a program will perform its specified function,
for a stated time period, under specified conditions.
Usually estimated during system tests, using statistical tests,
based on the software usage profile
Reliability metrics:
MTBF (Mean Time Between Failures)
MTTF (Man Time To Failure)
![Page 6: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/6.jpg)
MTBF (Mean Time Between Failures):
the expected time between two successive failures of a system
expressed in hours
a key reliability metric for systems that can be repaired or restored (repairable systems)
applicable when several system failures are expected.
For a hardware product, MTBF decreases with the its age.
![Page 7: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/7.jpg)
For a software product, MTBF it’s not a function of time!
It depends on the debugging quality.
![Page 8: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/8.jpg)
MTTF (Man Time To Failure):
the expected time to failure of a system
in reliability engineering metric for non-repairable systems
non-repairable systems can fail only once; example, a satellite is not repairable.
Mean Time To Repair (MTTR): average time to restore a system after a failure
When there are no delays in repair: MTBF = MTTF + MTTR
Software products are repairable systems!
Reliability models neglect the time needed to restore the system after a failure.
with MTTR =0 MTBF = MTTF
Availability = MTTF / MTBF = MTTF / (MTTF + MTTR)
![Page 9: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/9.jpg)
Is software reliability important?
• company's reputation
• warranty (maintenance) costs
• future business
• contract requirements
• custom satisfaction
![Page 10: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/10.jpg)
3.1.2. Defect rate (density)3.1.2. Defect rate (density)
Number of defects per KLOC or per Number of Function Point,
in a given time unit
Example:
“The latent defect rate for this product, during next four years, is 2.0 defects per KLOC”.
Crude metric: a defect may involve one or more lines of code
Lines Of Code
-Different counting tools
-Defect rate metric has to be completed with the counting method for LOC!
-Not recommended to compare defect rates of two products written in different languages
![Page 11: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/11.jpg)
Defect rate for subsequent versions (releases) of a software product:
Example:
LOC: instruction statements (executable + data definitions – not comments).
Size metrics:
Shipped Source Instructions (SSI)
New and Changed Source Instructions (CSI)
SSI (current release) = SSI (previous release)
+ CSI (for the current version)
- deleted code
- changed code (to avoid double count in both SSI and CSI)
![Page 12: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/12.jpg)
Defects after the release of a product:
field defects – found by the customer (reported problems that required bug fixing)
internal defects – found internally
Post release defect rate metrics:
Total defects per KSSI
Field defects per KSSI
Release – origin defects (field + internal) per KCSI
Released – origin field defects per KCSI
Defect rate using number of Function Points:
If defects per unit of function is low, then the software should have better quality
even though the defects per KLOC value could be higher –
when the functions were implemented by fewer lines of code.
![Page 13: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/13.jpg)
Reliability or Defect Rate ?
Reliability:
often used with safety-critical systems such as: airline traffic control systems, avionics, weapons.
(usage profile and scenarios are better defined)
Defect density:
in many commercial systems (systems for commercial use)
• there is no typical user profile
• development organizations use defect rate for maintenance cost and resource estimates
• MTTF is more difficult to implement and may not be representative of all customers.
![Page 14: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/14.jpg)
3.2. Customer Oriented Metrics3.2. Customer Oriented Metrics
3.2.1. Customer Problems Metric3.2.1. Customer Problems Metric
Customer problems when using the product:
valid defects, usability problems, unclear documentation, user errors.
Problems per user month (PUM) metric:
PUM = TNP/ TNM
TNP: Total number of problems reported by customers for a time period
TNM: Total number of license-months of the software during the period
= number of install licenses of the software x number of months in the period
![Page 15: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/15.jpg)
3.2.2. Customer satisfaction metrics3.2.2. Customer satisfaction metrics
Often measured on the five-point scale:
1. Very satisfied
2. Satisfied
3. Neutral
4. Dissatisfied
5. Very dissatisfied
IBM: CUPRIMDSO
(capability/functionality, usability, performance, reliability, installability, maintainability,
documentation /information, service and overall)Hewlett-Packard: FURPS (functionality, usability, reliability, performance and service)
![Page 16: 3. Software product quality metrics The quality of a product: -the “totality of characteristics that bear on its ability to satisfy stated or implied needs”.](https://reader035.fdocuments.in/reader035/viewer/2022072009/56649d8b5503460f94a7197f/html5/thumbnails/16.jpg)
Metrics:
– Percent of completely satisfied customers
– Percent of satisfied customers
– Percent of dissatisfied customers
– Percent of no satisfied customers
Net Satisfaction Index (NSI)
Completely satisfied = 100% (all customers are completely satisfied)
Satisfied = 75% (all customers are satisfied or
50% are completely satisfied and 50% are neutral)!
Neutral = 50%
Dissatisfied= 25%
Completely dissatisfied=0%(all customers are completely dissatisfied)