Prof. Mohamed Batouche

batouche@ksu.edu.sa

Software Testing

Software testing is a popular risk management strategy. It is used to verify that functional requirements were met.

The limitation of this approach, however, is that by the time testing occurs, it is too late to build quality into the product

3

Risks are potential problems that may affect successful completion of a software project.

Risks involve uncertainty and potential losses.

Risk analysis and management are intended to help a software team understand and manage uncertainty during the development process.

4

Reactive strategies very common, also known as fire fighting. project team sets resources aside to deal with

problems. team does nothing until a risk becomes a

problem.

Proactive strategies risk management begins long before technical

work starts, risks are identified and prioritized by importance.

team builds a plan to avoid risks if they can or to minimize risks if they turn into problems.

5

Component testing Testing of individual program components; Usually the responsibility of the component developer

(except sometimes for critical systems); Tests are derived from the developer’s experience.

System testing Testing of groups of components integrated to create a

system or sub-system; The responsibility of an independent testing team; Tests are based on a system specification.

6

7

Validation testing To demonstrate to the developer and the system

customer that the software meets its requirements; A successful test shows that the system operates as

intended.

Defect testing To discover faults or defects in the software where its

behavior is incorrect or not in conformance with its specification;

A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.

8

Software Testing Process

9

Involves integrating components to create a system or sub-system.

May involve testing an increment to be delivered to the customer.

Two phases: Integration testing - the test team have access

to the system source code. The system is tested as components are integrated.

Release testing - the test team test the complete system to be delivered as a black-box.

10

There are two groups of software integration strategies:

- Non Incremental software integration- Incremental software integration

Non Incremental software integration:

Big bang integration approach

Incremental software integration:

Top- down software integration Bottom-up software integration Sandwich integration

11

Involves building a system from its components and testing it for problems that arise from component interactions.

Top-down integration Develop the skeleton of the system and populate it with

components. Use stubs to replace real components.

Bottom-up integration Integrate infrastructure components then add

functional components. Use drivers to test components

To simplify error localisation, systems should be incrementally integrated.

12

The process of testing a release of a system that will be distributed to customers.

Primary goal is to increase the supplier’s confidence that the system meets its requirements.

Release testing is usually black-box or functional testing Based on the system specification only; Testers do not have knowledge of the system

implementation.

13

Only exhaustive testing can show a program is free from defects. However, exhaustive testing is impossible,

Testing policies define the approach to be used in selecting system tests: All functions accessed through menus should

be tested; Combinations of functions accessed through

the same menu should be tested; Where user input is required, all functions

must be tested with correct and incorrect input.

14

Software Testing Process

15

Black-box or behavioral testing knowing the specified function a product is to

perform and demonstrating correct operation based solely on its specification without regard for its internal logic

White-box or glass-box testing knowing the internal workings of a product,

tests are performed to check the workings of all possible logic paths

16

Black-Box TestingFunctional Testing

Input data and output results often fall into different classes where all members of a class are related.

Each of these classes is an equivalence partition or domain where the program behaves in an equivalent way for each class member.

Test cases should be chosen from each partition.

18

Black-box technique divides the input domain into classes of data from which test cases can be derived.

An ideal test case uncovers a class of errors that might require many arbitrary test cases to be executed before a general error is observed.

19

Black-box technique focuses on classes and also on the boundaries

of the input domain.

Guidelines:1. If input condition specifies a range bounded by

values a and b, test cases should include a and b, values just above and just below a and b

2. If an input condition specifies a number of values, test cases should exercise the minimum and maximum numbers, as well as values just above and just below the minimum and maximum values

20

21

Factorialn n!

Equivalence partitioning – break the input domain into different classes:

Class1: n<0 Class2: n>0 and n! doesn’t cause an overflow Class3: n>0 and n! causes an overflow

Boundary Value Analysis:

n=0 (between class1 and class2)

22

Test case = ( ins, expected outs)

Equivalence partitioning – break the input domain into different classes:

1. From Class1: ((n = -1), “ function not defined for n negative”)

2. From Class2: ((n = 3), 6)3. From Class3: ((n=100), “ input value too big”)

Boundary Value Analysis:

4. ((n=0), 1)23

White-Box TestingStructural Testing

The objective of path testing is to ensure that the set of test cases is such that each path through the program is executed at least once.

The starting point for path testing is a program flow graph that shows nodes representing program decisions and arcs representing the flow of control.

Statements with conditions are therefore nodes in the flow graph.

25

26

2

3

4

5

1

6

27

1

2

A sequence:X = 1;Y = X * 10;

1

4

23

If condition:If … Then …Else …End if

2

3

4

5

1

While loop:While … do …statements …End while

28

29

30

elemArray [mid] != key

elemArray [mid] > key elemArray [mid] < key

1

2

3

4

5

6

7

8

9

14 10

11

12 13

bottom > top while bottom <= topbottom > top

elemArray[mid] = key

1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14 1, 2, 3, 4, 5, 14 1, 2, 3, 4, 5, 6, 7, 11, 12, 5, … 1, 2, 3, 4, 6, 7, 2, 11, 13, 5, …

Test cases should be derived so that all of these paths are executed

31

McCabe’s cyclomatic number, introduced in 1976, is, after lines of code, one of the most commonly used metrics in software development.

The cyclomatic complexity of the program is computed from its control flow graph (CFG) using the formula:

V(G) = Edges – Nodes + 2

or by counting the conditional statements and adding 1

This measure determines the basis set of linearly independent paths and tries to measure the complexity of a program

32

How to evaluate quality?

Fix a Threshold value

An important aspect of a metric is guidance about when the values are reasonable and when the values are not reasonable.

McCabe analyzed a large project and discovered that for modules with cyclomatic number over 10, the modules had histories of many more errors and many more difficulties in maintenance.

Thus, 10 has been accepted as the threshold value for the cyclomatic number in a module. If the cyclomatic number is greater than 10, efforts should be made to reduce the value or to split the module.

33

V(G) = Edges – Nodes + 2V(G) = 6 – 6 + 2 = 2

V(G) = conditional statements + 1= 1 + 1 = 2

Two linearly independent paths:

1, 2, 5, 61, 2, 3, 4, 2, 5, 6

Complexity = 2 < 10 => good quality

34

2

3

4

5

1

6

V(G) = Edges – Nodes + 2V(G) = 16 – 14 + 2 = 4

V(G) = conditional statements + 1= 3 + 1 = 4

four linearly independent paths

Complexity = 4 < 10 => good quality

35

elemArray [mid] != key

elemArray [mid] > key elemArray [mid] < key

1

2

3

4

5

6

7

8

9

14 10

11

12 13

bottom > top while bottom <= topbottom > top

elemArray[mid] = key

K. NAIK AND P. TRIPATHY: “SOFTWARE TESTING AND QUALITY ASSURANCE”, WILEY, 2008.

IAN SOMMERVILLE, SOFTWARE ENGINEERING, 8TH EDITION, 2006.

ADITYA P. MATHUR,“FOUNDATIONS OF SOFTWARE TESTING”, PEARSON EDUCATION, 2009.

D. GALIN, “SOFTWARE QUALITY ASSURANCE: FROM THEORY TO IMPLEMENTATION”, PEARSON EDUCATION, 2004

DAVID GUSTAFSON, “THEORY AND PROBLEMS OF SOFTWARE ENGINEERING”, Schaum’s Outline Series, McGRAW-HILL, 2002.

36