01 Why Testing is Necessary (v2.5)

27
© SIM Group Ltd., SQS Group AG, 2002 ISEB Foundation Certificate in Software Testing Why Testing is Necessary Why Testing is Necessary

Transcript of 01 Why Testing is Necessary (v2.5)

Page 1: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

ISEB Foundation Certificate in Software Testing

Why Testing is NecessaryWhy Testing is Necessary

Page 2: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

In this session we willIn this session we will

Understand what testing is and why it is necessaryUnderstand what testing is and why it is necessary

Define some of the words used in software testingDefine some of the words used in software testing

Look at the cause and cost of errorsLook at the cause and cost of errors

Page 3: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Why Test?Why Test?

To find bugsTo find bugs

What is a bug?What is a bug?

Page 4: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

What is Testing?What is Testing?

“…“…the process of executing a program with the intent to certify its the process of executing a program with the intent to certify its Quality”Quality”

MillsMills

“…“…the process of executing a program with the intent of finding the process of executing a program with the intent of finding failures / faults” failures / faults”

MyersMyers

“…“…the process of exercising software to detect bugs & to verify that the process of exercising software to detect bugs & to verify that it satisfies specified functional & non-functional requirements”it satisfies specified functional & non-functional requirements”

Page 5: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Why is Testing becoming more important?Why is Testing becoming more important?

The Y2K problemThe Y2K problem

EMUEMU

e Commercee Commerce

Increased user baseIncreased user base

Increased complexityIncreased complexity

Speed to MarketSpeed to Market

Page 6: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

Page 7: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

ERRORERROR““A human action that produces an incorrect result”A human action that produces an incorrect result”

FAULTFAULT““A manifestation of an ERROR in software”A manifestation of an ERROR in software”

Page 8: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

FAULTFAULTA Fault, if encountered, may cause a failureA Fault, if encountered, may cause a failure

FAILUREFAILURE““A deviation of the software from its expected delivery or A deviation of the software from its expected delivery or

service” service”

Page 9: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

DEFECTDEFECT““The departure of a quality characteristic from its specified value The departure of a quality characteristic from its specified value

that results in a product or service not satisfying its normal that results in a product or service not satisfying its normal usage requirements”usage requirements”

Page 10: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

RELIABILITYRELIABILITY““The probability that software will not cause the failure of a The probability that software will not cause the failure of a

system for a specified period of time under specified system for a specified period of time under specified conditions” conditions”

Page 11: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

QUALITYITY““The totality of the characteristics of an entity that bear on its The totality of the characteristics of an entity that bear on its

ability to satisfy stated or implied needs”ability to satisfy stated or implied needs”

Page 12: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Testing & Quality

Does testing improve quality?Does testing improve quality?

Testing does not build Quality into the softwareTesting does not build Quality into the software

Testing is a means of determining the Quality of the Software Testing is a means of determining the Quality of the Software under Testunder Test

Page 13: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Definitions

QUALITY ASSURANCEQUALITY ASSURANCE

““All those planned actions used to fulfil the requirements for All those planned actions used to fulfil the requirements for quality”quality”

Page 14: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why do errors occur?

Page 15: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why do errors occur?

How Errors Occur How Errors Occur

No one is perfect! No one is perfect! We all make mistakes or omissionsWe all make mistakes or omissions

The more pressure we are under the more likely we are to The more pressure we are under the more likely we are to make mistakesmake mistakes

In IT development we have time and budgetary deadlines to In IT development we have time and budgetary deadlines to meetmeet

Poor TrainingPoor Training

Page 16: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why do errors occur?

How Errors Occur How Errors Occur

Poor CommunicationPoor Communication

Requirements not clearly definedRequirements not clearly defined

Requirements change & are not properly documentedRequirements change & are not properly documented

Data specifications not completeData specifications not complete

ASSUMPTIONS!ASSUMPTIONS!

Page 17: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why do errors occur?

The Cost Of Errors The Cost Of Errors

A single failure may incur little cost - or millionsA single failure may incur little cost - or millions

Report layouts may be wrong - little true costReport layouts may be wrong - little true cost

Or a significant error may cost millions… Or a significant error may cost millions… Ariane V, Venus Probe, Mars Explorer and Polar LanderAriane V, Venus Probe, Mars Explorer and Polar Lander

Page 18: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why do errors occur?

The Cost Of ErrorsThe Cost Of Errors

In extreme cases a software or systems error may cost LIVESIn extreme cases a software or systems error may cost LIVES

Usually safety critical systems are tested exhaustively Usually safety critical systems are tested exhaustively Aeroplane, railway, nuclear power systems etcAeroplane, railway, nuclear power systems etc

Unfortunately there are exceptions to thisUnfortunately there are exceptions to thisLondon Ambulance ServiceLondon Ambulance Service

Page 19: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why do errors occur?

The Cost Of ErrorsThe Cost Of Errors

The cost of defects increases proportionately (tenfold?) with The cost of defects increases proportionately (tenfold?) with the passing of each successive stage in the system the passing of each successive stage in the system development process before they are detecteddevelopment process before they are detected

To correct a problem at requirements stage may cost £1To correct a problem at requirements stage may cost £1

To correct the problem post-implementation may cost £000’sTo correct the problem post-implementation may cost £000’s

Page 20: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Page 21: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Why Test? Why Test?

Identifies faultsIdentifies faultsReduces live defectsReduces live defects

Improves the quality of the users applicationImproves the quality of the users application

Increases reliabilityIncreases reliability

To help ensure live failures don’t impact costs & profitabilityTo help ensure live failures don’t impact costs & profitability

To help ensure requirements are satisfiedTo help ensure requirements are satisfied

To ensure legal requirements are metTo ensure legal requirements are met

To help maintain the organisation’s reputationTo help maintain the organisation’s reputation

Provides a measure of qualityProvides a measure of quality

Page 22: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Quality MeasurementsQuality Measurements

Quality can be measured by testing forQuality can be measured by testing for

correctnesscorrectness

reliabilityreliability

usabilityusability

maintainabilitymaintainability

testabilitytestability

reusabilityreusability

Page 23: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

How much is enough?How much is enough?

How do we know when to stop? How do we know when to stop? There are many factors to considerThere are many factors to consider

Page 24: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Exhaustive TestingExhaustive Testing

Find all faults by testing everything?Find all faults by testing everything?

To test everything is rarely possibleTo test everything is rarely possible

The time required makes it impracticalThe time required makes it impractical

The resource commitment required makes it impracticalThe resource commitment required makes it impractical

Page 25: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

If we can’t test everything, what can we do?If we can’t test everything, what can we do?

Managing and reducing RiskManaging and reducing Risk

Carry out a Risk Analysis of the applicationCarry out a Risk Analysis of the application

Prioritise tests to focus on the main areas of riskPrioritise tests to focus on the main areas of risk

Apportion time relative to the degree of risk involvedApportion time relative to the degree of risk involved

Understand the risk to the business of the software not Understand the risk to the business of the software not functioning correctlyfunctioning correctly

Page 26: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

Should you stop testing when the product goes live?Should you stop testing when the product goes live?

Continuing testing beyond the implementation date should be Continuing testing beyond the implementation date should be consideredconsidered

Better that you find the errors than the usersBetter that you find the errors than the users

Page 27: 01 Why Testing is Necessary (v2.5)

© SIM Group Ltd., SQS Group AG, 2002

Why Testing is Necessary

SummarySummary

The purpose of testing is to find faultsThe purpose of testing is to find faultsFaults can be fixed, making better softwareFaults can be fixed, making better software

Better software is more reliable, less prone to failuresBetter software is more reliable, less prone to failures

Establish the relationship between the software and its’ Establish the relationship between the software and its’ specificationspecification

Testing enables us to measure the quality of the softwareTesting enables us to measure the quality of the software

This enables us to understand & manage the risk to the This enables us to understand & manage the risk to the businessbusiness