SENG 521 Software Reliability & Testing

SENG521 (Fall 2002) [email protected] 1

SENG 521SENG 521Software Reliability & Software Reliability & TestingTesting

Software Reliability ModelsSoftware Reliability Models

(Part 2b)(Part 2b)

Department of Electrical & Computer Engineering, University of Calgary

B.H. Far （ [email protected] ）http://www.enel.ucalgary.ca/~far/Lectures/SENG521/02b/


ContentsContents Basic Features of the Software Reliability ModelsBasic Features of the Software Reliability Models Single Failure ModelSingle Failure Model Reliability Growth ModelReliability Growth Model Exponential Failure Class ModelsExponential Failure Class Models Weibull and Gamma Failure Class ModelsWeibull and Gamma Failure Class Models Infinite Failure Category ModelsInfinite Failure Category Models Bayesian ModelsBayesian Models Early Life-Cycle Prediction ModelsEarly Life-Cycle Prediction Models


GoalGoal It is important to be able to

Predict probability of failure of a component or system

Estimate the mean time to the next failure Predict number of (remaining) failures

during the development. Such tasks are the target of the reliability

management models.


BackgroundBackgroundRandomness & Randomness & ProbabilityProbability


Randomness /1Randomness /1 Random actions:

Introduction of defects into the code Execution of the test-cases

We should define some random processes random processes to represent the randomness

How to handle randomness? Collect failure data through testing Find a distribution function that is a best-fit for the

collected data Make assumptions about the presence of errors and

reliability


Randomness /2Randomness /2 What is a random variable?

A random variable x on a sample space S is a rule that assigns a numerical value to each outcome of S (a function of S into a set of real numbers)

In reliability modeling what can be represented by random variable? Time of failure Number of failures


Probability Distribution /1Probability Distribution /1 Suppose that a random variable X assigns a

finite number of values to a sample space S Then X induces a distribution function f that

assigns probabilities to the points in Rx

Rx ={x1, x2, x3, …, xn}

f(xk) = P( X=xk)


Probability Distribution /2Probability Distribution /2 The set of ordered pairs [xk, f(xk)] is usually

represented by a table or a graph (histogram)

The expected value of X, denoted by E(X) is defined by

E(X) = x1 f(x1) + x2 f(x2) + …+ xn f(xn)

0

5

10

15

20

1 1.5 2 2.5 3 3.5 4 4.5 5

Time (days)

Fre

qu

en

cy

(Fa

ilu

re/t

ime

un

it)


Probability Distribution /3Probability Distribution /3 Let M(t) be a random process representing the

number of failures at time t The mean function μ(t) represents the expected

number of failures at time t μ(t) = E(M(t))

Failure intensity is the rate of change of the expected number of failures with respect to time

λ(t) = d μ(t) / dt (t) is the number of failures per unit time (t) is an instantaneous value


Probability Distribution /4Probability Distribution /4 Discrete distributions:

Binomial distribution Poisson distribution

Continuous distributions: Normal / Gaussian distribution Lognormal distribution Weibull distribution Rayleigh distribution Exponential distribution Gamma distribution 2 distribution


Binomial Distribution /1Binomial Distribution /1 Gives probability of exact number

of successes in n independent trials, when probability of success p on single trial is a constant.

Situations with only 2 outcomes (success or failure)

Probability remains the same for all independent trials (Bernoulli trials)

Jakob Bernoulli (1645-1705)


Binomial Distribution /1Binomial Distribution /1 Probability of exactly x successes:

n : number of trials f(x) : probability of x successes in n trials p : probability of success q : probability of failure

p + q =1

xnxxnx qpx

nqp

xnx

nxf

!!

!


Binomial Distribution /2Binomial Distribution /2 Probability of at least r successes:

n : number of trials f(x) : probability of x successes in n trials p : probability of success q : probability of failure p + q =1 F(r) : probability of obtaining r or fewer successes in n trials

xnxr

x

qpx

nrF

0

Calculating F(r) becomes increasingly difficult as n (sample set) gets larger

It is possible to find an approximate solution by means of a normal distribution


Example: Binomial Example: Binomial DistributionDistribution

Acceptance sampling:Acceptance sampling: A lot is accepted if not more than 2 defectives are

found in a sample of 6. The defect probability is 25%. Probability of having exactly 2 defects in the lot is:

Probability of having more than 4 defects in the lot is:

0046.025.075.025.05

6654

297.075.025.02

62

615

42

FFrF

f


Poisson Distribution /1Poisson Distribution /1 Some events are rather rare, they don’t happen

that often. Still, over a period of time, we want to say something about the nature of rare events.

Poisson distribution is special case of binomial distribution (either p or q is very small and n very large)

Conditions under which a Poisson distribution holds

counts of rare events all events are independent average rate does not change over the period of

interest

Simeon Poisson (1781-1840)


Poisson Distribution /2Poisson Distribution /2 Poisson distribution is a special case of

binomial distribution (either p or q is very small and n very large): =np

: mean rate of occurrence (in statistics literature is usually denoted by )

x : observed number of failures

0,1,2,... xex!μ

f(x) μx


Example: Poisson Example: Poisson DistributionDistribution

Suppose that the defect rate is only 2%

find the probability that there are 3 defective items in a sample of 100 items.

18.03

23

!

23

e

xf


Exponential Failure Exponential Failure Time ModelsTime Models


Exponential Failure Time Exponential Failure Time ModelsModels

Assumes finite failure models Functional form of failure intensity function

is exponential Binominal type:

All faults have a constant hazard rate z(t) = Hazard rate for the i-th fault is a function of the

remaining number of faults Failure intensity is teNt


Various Reliability ModelsVarious Reliability Models Exponential Failure Time ModelsExponential Failure Time Models

Jelinski-Moranda model (JM) Nonhomogeneous Poisson Process model

(NHPP) Schneidewind model Musa’s Basic Execution Time model (BET) Hyperexponential model (HE) Others


Jelinski-Moranda Model /1Jelinski-Moranda Model /1 Jelinski-Moranda model follows exponential

distribution but differs from the Geometric model in that the parameter used is proportional to the remaining number of faults rather than a constant.

All faults equally contribute to the reliability of the system

Hazard Rate z(t): Probability density function divided by reliability function.

is a constant


Jelinski-Moranda Model /1Jelinski-Moranda Model /1 Assumptions:Assumptions:

The rate of fault detection is proportional to the current fault content of the software.

The fault detection rate remains constant over the intervals between faults.

A fault is corrected instantaneously without introducing new faults into the software.

Every fault has the same chance of being encountered within a severity class as any other fault in that class.

The failures are independent. Data requirement:Data requirement:

Either actual failure times: t1, t2, …, tn Or elapsed time between failures: xi = ti – ti-1


Jelinski-Moranda Model /2Jelinski-Moranda Model /2 Meantime between failures at time t

MTTF = 1/ (N-(i-1)) Mean failures experienced

Failure intensity function

t

t

eNt

eNt

1 N total number of faults (constant)

Parameters to estimate using collected data are: N and

N total number of faults (constant)

Parameters to estimate using collected data are: N and


Non Homogeneous Non Homogeneous Poisson Process Model /1Poisson Process Model /1

Assumes that the cumulative number of failures detected at any time follows a Poisson distribution.

Time periods can be unequal Data requirement:Data requirement:

Fault counts on each testing interval: f1, f2, …, fn

Completion time of each period: t1, t2, …, tn


Non Homogeneous Poisson Non Homogeneous Poisson Process Model /2Process Model /2

Assumptions:Assumptions: Every failure has the same chance of being detected as

any other failure. The cumulative number of failures detected at any time

follows a Poisson distribution with mean (t). That mean is such that the expected number of failures in

any small time interval about t is proportional to the number of undetected failures at time t.

The mean is assumed to be a bounded non-decreasing function with (t) approaching in the limit, as the length of testing goes to infinity.

Perfect debugging is assumed.


Non Homogeneous Poisson Non Homogeneous Poisson Process Model /3Process Model /3

Mean failures experienced


btNbet

bteNt 1

Parameters to estimate using collected data are: N and b

Parameters to estimate using collected data are: N and b


Schneidewind ModelSchneidewind Model Assumes that the current fault rate might be a better

predictor of the future behaviour than the observed rate in the distant past

Suppose that there are n units of time all of fixed length

Has 3 variations: Utilize all individual fault counts from all n time periods Ignore fault counts from 1 to s-1 periods (forget the

distant past) Use cumulative fault count for 1 to s-1 periods and

individual fault counts for s to n


Musa Basic Model /1Musa Basic Model /1 One of the most widely used software reliability

models. Uses execution time rather than calendar time.

0 and 1 are parameters. 0 is equal to the number of faults in the system and 1 is a fault reduction factor.


Musa Basic Model /2Musa Basic Model /2 Assumptions:Assumptions:

The detections of failures are independent of one another.

The software failures are observed (i.e. the total number of failures has an upper bound).

The execution times (measured in CPU time) between failures are piecewise exponentially distributed.

The hazard rate is proportional to the number of faults remaining in the program.

The fault correction rate is proportional to the failure occurrence rate.

Perfect debugging is assumed.


Musa Basic Model /3Musa Basic Model /3 Meantime between failures at time t

Mean failures experienced


tet 110

tet 110

t

MTTF nC

exp MTTFMTTF(t)00

0

n0 = total number of errors in the program

MTTF0=MTTF at the start of testing

C= test compression factor


Musa Basic Model /4Musa Basic Model /4 Number of errors at time t

Reliabilityn0 = total number of errors in the program

MTTF0=MTTF at the start of testing

C= test compression factor

MTTFt

expR(t)


Hyper-exponential Hyper-exponential Model /1Model /1

The basic idea is that the different sections (or classes) of the software experience an exponential failure rate; however, the rates vary over these sections to reflect their different natures. This could be due to different programming groups doing the different parts, old versus new code, sections written in different languages, etc.

We thus reflect the sum of these different exponential growth curves, not by another exponential, but by a hyper-exponential growth curve.


Hyper-exponential Hyper-exponential Model /2Model /2

Data requirement:Data requirement: Fault counts on each testing interval: f1, f2, …, fn

Completion time of each period: t1, t2, …, tn


101101

1

i

K

iii

K

i

tii

pp

epNt i

SENG 521 Software Reliability & Testing

Documents

Transcript of SENG 521 Software Reliability & Testing