Kaspar Loog

kaspar.loog@knowit.ee

”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979

“Testing is measuring quality” - William Hetzel, 1988

Testing reduces our risks

2008-10-07 2

o

”Measure” quality (”Testing is to knowing everything”)

Release the right product in timeNOT to lose moneyStay in the market

2008-10-07 3

o

Enhance, and help the development of, the productVerifiy requirements/functionalityMeasure the Quality of the Product

2008-10-07 4

2008-10-07 5

Many defects found Few defects found

Few defects found Few defects found

High

HighLow

Low

Test Quality

SW Quality

You think you are here

..but you are here

o

”Testing is expensive” ◦ Manhours for test◦ Test environmentCompared to:◦ What does it cost to recall a release?◦ What does it cost to get unhappy customers?◦ What does it cost to lose market volume because

the quality is too poor? ◦ etc...

2008-10-07 6

o

Economical

Communication

Ethical

2008-10-07 7

Exhaustive-testning

What is Exhaustive-testning?◦ We test all combinations of in parameters, prerequisites and

environments.

How long time does it take? (Yes/No)◦ Infinitely long time◦ Short time◦ Impractically long time

2008-10-07 8

o

It depends of the risk of:◦ Miss serious defects◦ The defects we will find are expensive to correct.◦ Need to recall a release.◦ Releasing bad code◦ Losing market volume◦ Not be first on the market◦ Execute effective testing

2008-10-07 9

How is risk normally defined?Probability * ImpactRisk = S * K

o

2008-10-07 10

Prioritise the testing so that whenever you stop testing you have done the best possible testing on the given time.

o

ISO/IEC 12119◦ Information technology - Software packages -

Quality requirements and testingISO/IEC 14598◦ Information technology - Software product

evaluationIEEE Std 829◦ Standard for test documentationSIGIST◦ Specialist Interest Group in Software TestingISTQB◦ International Software Testing Qualifications Board

2008-10-07 11

o

VerificationValidationTesting

BugError, Defect, FailureReliability

2008-10-07 12

o

”The process of evaluating a system or component to determin whether the products of the given development phase satisfy the conditions imposed at the start of the phase”[ BS 7925-1]

2008-10-07 13

That the system is correct according to the written requirements.

o

”Determination of the correctnesss of the products of software development with respect to the user needs and requirements”[ BS 7925-1]

2008-10-07 14

That the system is correct to the real requirements.

o

”The process of exercising software to verify that it satisfies specifed requirements and to detect defects”[ BS 7925-1]

2008-10-07 15

o

2008-10-07 16

o

2008-10-07 17

A person does aprogramming error.ERROR

The Error createsa defect in the code.DEFECT

The defect can create aunwanted reaction.FAILURE

2008-10-07 18

Test closure activities

Evaluating exit criteria and reporting

Implementation and execution

Analisys and Design

Planning and control

Delivery

START

o

Unit test: The lowest level of testing, often executed by a developer.Functionality test: Test of the functionality of a controlled part of the system.System test: Tests the system.Acceptance test: Customer/user/supplier/3:e partAlpha- & Beta-test: executed by user/possible customer to obtain feedback and for information.Test in the maintenance level, mostly executed when changes are made to the product.

2008-10-07 19

2008-10-07 20

Static testing

Formal / informal review Staticanalysis

o

2008-10-07 21

Effective -Finds defects

Well constructed -represents other TC

Maintanable -easy to maintain

Economical -easy to use

o

System with 2 input fields◦ Choose a method for black-box testing◦ How many test cases would you write?

2008-10-07 22

o

Age

Shoe size

Statical (does not execute any code)Whitebox (glassbox)BlackboxOther (ex. experience based)

2008-10-07 23

Goto 10for i=1 to 100

o

Black-box methods:◦ Equivalence partitioning (EP)◦ Boundary value analysis (BVA)◦ Random input◦ etc...

White box methods◦ Statements, decisions, test path coverage, dataflow,

branch coverage etc...

Experience based◦ Error guessing◦ Exploratory testing

System with 2 input fields◦ Choose a method for black-box testing◦ How many test cases would you write?

2008-10-07 25

o

Age

Shoe size

2008-10-07 26

UC1-1UC2-1....

Requirement spec.

TEST CASE 1UC1-1

TEST CASE2UC2-1

Test spec..

TestCase1 UC1-1 OK

TestCase2 UC2-1NOK

Test Protocol

UC2-1 NOKTestcase2, Step X

Incident report

o

2008-10-07 27

Examples of risk criterias:◦ Test where the defect has the most impact◦ Test where the defects are most visible◦ Test where defects are most probable◦ Ask the customer to prioritise◦ Test where many changes have been done◦ Test where many defects have been found◦ Test where most complexity or complicated functions exist

2008-10-07 28

Evil messsenger, ”your baby is ugly”

29

A creative workLearn more and deeper than most peopleIn the middle of the actionWide technical skillsDeep technical skillsHard, a challengeMay sometimes pretend that he is a customer

30

ISTQB – www.istqb.org -- downloadsTMap & TMap NextTPI (Test Process Improvement)

www.stickyminds.com

2008-10-07 31

Creating a test plan

Management doc not a techincal docWhat (tested items, deliverables)Why? – give reasoning for decisionsHow? – give indications for methodsWho? – how many people, what kind ofpeople?When? – at what stage, when to stopWhere? – which environment, technicalrequirements etc.

Define the purpose of testingDefine test items◦ Funcional◦ Non-functional

PerformanceResponse-timeScalabilitySecurity and authenticationRobustness, handling errors, wrong intput etc.

Tested or not testedGive reasoning for each!

Approach – white-box, black-boxAcceptance criteriaDeliverables – test spec, instructions, protocols, reports, scripts, programsEnvirionmental needs – OS, browser etc.Testing tasks – writing spec, install, test datacreation, execution, bug reports, reportingRoles/responsibilities - creator-reviewerStaff & trainingScheduleRisks

1. Write a test plan with your group1. Purpose & exit criteria2. Test items3. Environment needs4. Tasks5. Risks (project/product risks not testing risks!)

2. Write a review of your neighbours test plan3. Submit when done◦ Put your names on!!!

4. ASK QUESTIONS!

An application ordocumentation?

http://www.cs.gordon.edu/courses/cs211/AddressBookExample/Executable.html