2University of Sunderland CIFM03 Lecture 4
Where does Software Measurement fit in?
Look at software measurement from the perspective of software quality assurance…..
SQA
REVIEWS TESTING
MEASURE-MENT
STANDARDS&
PROCEDURES
3University of Sunderland CIFM03 Lecture 4
Software Measurement is . . .
“the objective assignment of numbers or symbols to attributes of a software process or product in such a way as to describe them according to clearly defined rules...”
see ‘Software Metrics, A Rigorous & Practical Approach’, Fenton & Pfleeger
4University of Sunderland CIFM03 Lecture 4
Why should we measure?
what are our goals?
what information is required?
what can we collect?
How do we act upon the data?
5University of Sunderland CIFM03 Lecture 4
DON’T!
Don’t define what you won’t collect..
Don’t collect what you won’t analyse..
Don’t analyse what you won’t use..
6University of Sunderland CIFM03 Lecture 4
Goal / Question / Metric
• Goal - achieve a 10-fold improvement in
post-release defects.
• Question - how many existing faults? How
effective are inspection techniques? How
effective are tests?
• Metric - no. & type of faults, no. of defects
found, % paths tested, complexity of code.
7University of Sunderland CIFM03 Lecture 4
Hard & Soft Data
HARD - objective figures - no.of people on a project, duration of tasks, size of documentation, etc.
SOFT - subjective - level of experience, balance of teams, etc.
8University of Sunderland CIFM03 Lecture 4
Obstacles to Effective Measurement
no agreement over standard measures
patchy tool support
initial set-up expense
9University of Sunderland CIFM03 Lecture 4
Basic Measurement Theory
Nominal - names / categories
Ordinal - ‘cold, warm, hot’ ‘small, med, large’
Interval - temperature ‘20oC, 40oC’ - scale
Ratio - absolute temperature ‘20oK, 40oK’, time
Absolute - counting no. of occurrences
10University of Sunderland CIFM03 Lecture 4
Analysis..
Nominal - equivalence
Ordinal - as above + greater than
Interval - as above + ratio of any intervals
Ratio - as above + ratio of any 2 scale values
Absolute - as above + geometric mean,
coeff. of variation
11University of Sunderland CIFM03 Lecture 4
Function Points & Cyclomatic Complexity
Function Points:
Measure of functionality for ‘size’
- no. of external inputs- no. of external outputs- no. of enquiries- no. of files- no. external interfaces
Cyclomatic Complexity:
Module complexity based on a ‘graph’ of the module
- nodes
- predicate nodes- paths
12University of Sunderland CIFM03 Lecture 4
Measurement and the CMM
Maturity Level Characteristics Type of Metrics to Use
5. Optimizing Improvement fed- back to the process
Process plus feedback for changing the process
4. Managed Measured process Process plus feedback for control
3. Defined Process defined andinstitutionalised
Product
2. Repeatable Process dependent on individuals
Project management
1. Initial Ad hoc Baseline
13University of Sunderland CIFM03 Lecture 4
Software Reliability is…
The probability that the software will;
operate as required (i.e., without fail),
for a specified time,
in a specified environment.
14University of Sunderland CIFM03 Lecture 4
Software Reliability - features• failures in software are design faults,
• reliability during test changes continually(new problems are found as old ones are fixed / new code is never perfect)
• phenomenon of software reliability growth
• environment is important (platform/inputs) - new envt. may require s/w retest
15University of Sunderland CIFM03 Lecture 4
Reliability - General (i)
• Of all software quality characteristics, reliability is consistently identified as most important.
• Reliability, for both hardware and software, is a probability - SYSTEM reliability can be derived from combining the hardware and software reliabilities.
• Reliability is directly related to the number of failures. [ p(Reliable) =1 - p(Fail) ]
16University of Sunderland CIFM03 Lecture 4
Reliability - General (ii)
• Reliability measures are much more useful than fault measures - from the user perspective.
• Measures of faults/defects are developer oriented.
• Change of environment (operating platform, operational data profile) changes reliability.
17University of Sunderland CIFM03 Lecture 4
Fault, Error, Failure
(defined by BS5760 part8: 1998)
• Faulta defect in a software component which, when
triggered, causes one or many errors
• Errora discrepancy between its actual state and the
‘correct state’ which, if it propagates, means that the component ceases to perform as required
• Failure the component ceases to perform as required
18University of Sunderland CIFM03 Lecture 4
Fault, Error, Failure
FAULTS
ERRORS
FAILURE
ENVIRONMENT
OPERATOR
INPUT
OR
REVEALINGMECHANISMAND
LEADS TO ZERO OR MANY
LEADS TO ZERO OR MANY
POTENTIALLYLEADS TO ZERO OR MANY
MISTAKES
(PERSON MAKES)
CAN BE ATTRIBUTED TO ONE OR MANY
CAN BE ATTRIBUTED TO ONE OR MANY
CAN BE ATTRIBUTED TO ONE OR MANY
(This slide from BS5760 Part 8: 1998)
19University of Sunderland CIFM03 Lecture 4
Reliability Measures
When we talk of reliability measures the irony is that we invariably talk about failure measures. There are four general ways of measuring failures against time;
time of failure, interval between failures, cumulative failures experienced up to a given time, failures experienced in a time interval.
20University of Sunderland CIFM03 Lecture 4
Availability & Maintainability
• Availability – is the probability that the software is operating at
a given time. It is equal to the mean time to failure divided by the mean time between failures.
• Maintainability – is the mean time to repair the software and
restore to working order.
Top Related