Testing

Prepared by :1.Mohammed Siddig Ahmed.2.Mohammed Zeinelabdeen.3.Omer Salih Dawood.

Introduction to Testing. Verification and validation. Black & White-box testing. Types of testing that involve

both black & white-box techniques.

Strategies for writing test cases. Using a template for writing

repeatable, defined test cases.

2

Software testing is the process of analyzing a software item to detect the differences between existing and required conditions (that is, bugs) and to evaluate the features of the software item.

Software testing is an activity that should be done throughout the whole development process .

Software testing is one of the V&V, software practices.

3

Verification is the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase.

Validation is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements.

Verification: Are we building the product right?

Validation : Are we building the right product?

4

Standard or specification to measure or identify correct results from incorrect results must define some terms:•Mistake .• Fault [or Defect] .• Failure .• Error . •Specification. 5

In software development, there are costs associated with testing our programs.

Quality is much more important for safety- or mission critical software e.g. aviation software.

Goal of testing covering many defects as possible with a little testing.

6

There are two basic classes of software testing definitions below:

• Black box testing (also called functional testing) is testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions.• White box testing (also called

structural testing and glass box testing) is testing that takes into account the internal mechanism of a system or component.

7

1. Unit Testing Opacity: White box testing. Specification: Low-level design and/or code structure. (Unit testing is the testing of individual hardware or

software units or groups of related units).

2. Integration testing Opacity: Black- and white-box testing. Specification: Low- and high-level design. (Integration test is testing in which software

components, hardware components, or both are combined and tested to evaluate the interaction between them).

8

3. Functional and system testing Opacity: Black-box testing

Specification: high-level design, requirements specification.

( Functional testing involves ensuring that the functionality specified in the requirement specification works,& System testing is testing conducted on a complete, integrated system to evaluate the system compliance with its specified requirements).• Stress testing• Performance testing • Usability testing

9

4. Acceptance testing Opacity: Black-box testing

Specification: requirements specification. (Acceptance testing is formal testing

conducted to determine whether or not a system satisfies its acceptance criteria (the criteria the system must satisfy to be accepted by a customer) and to enable the customer to determine whether or not to accept the system).

10

5. Regression testing Opacity: Black- and white-box testing Specification: Any changed documentation,

high-level design

(Regression testing is selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified Requirements).– A subset of the regression test cases can

be set aside as smoke tests.

11

6. Beta testing Opacity: Black-box testing. Specification: None. The advantages of beta test:

• Identification of unexpected errors• A wider population search for errors• Low costs

The disadvantages• Lack of systematic testing• Low quality error reports• Much effort is necessary to examine error

reports 12

( Document describing the scope, approach, resources, and schedule of intended test activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency plans).

Test case ( set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement).

14

The earlier testing is planned at all levels, the better.

Very important to consider test planning and test execution as iterative processes.

It is best to begin to write functional and system test cases.

When requirements change, revise the test cases. When code changes, run the test cases again. All in all, testing should be considered an iterative

and essential part of the entire development process.

15

Black-Box Testing

Called :– functional testing – behavioral testing

focuses on:– whether or not a program does what

it is supposed to do based on its functional requirements.

19

1. incorrect or missing functionality.2. interface errors .3. errors in data structures used by

interfaces.4. behavior or performance errors.5. initialization and termination errors.

But: no amount of testing can demonstrate the absence of errors.

20

blackbox tester is not the programmer of

the code (it is best). Programmer:– If the program does what they

programmed it to do?. Needed:– If The program does what the

customer wants it to do?.

21

Focus on input and output of the software without regard to the internal code of the program

22

1. test case a unique identifier.2. describe the set of steps and/or

input for the particular condition.3. the expected results for an

input/output oracle.4. the actual results.

23

Why test Strategies?:-– test every possible thing (Cost).– find many defects in few test

cases .– avoid writing redundant test cases.– design the simplest test cases .

25

1. Tests of Customer Requirements.2. Equivalence Partitioning.3. Boundary Value Analysis.4. Decision Table Testing.5. Failure (“Dirty”) Test Cases.

26

Black box test cases are based on customer requirements

Have two paths:– success path.– failure path.

risky requirements should tested first.

27

don’t want to write several test cases that test the same aspect of our program(cost).

A good test case uncovers a different class of errors.

Equivalence partitioning is used to reduce the number of test cases .

divides the input of a program into classes

28

For each of these classes :-– the set of data should be treated the

same by the module under test .– And should produce the same

answer.

29

When creating BVA test cases, consider the following : – If input conditions have a range from a

to b (such as a=100 to b=300), create test cases:

– immediately below a (99) – at a (100) – immediately above a (101) – immediately below b (299) – at b (300) – immediately above b (301)

31

If input conditions specify a number of values that are allowed, test these limits.

32

record complex business rules that must be implemented in the program, and therefore tested.

conditions represent possible input. actions are the events that should

trigger. Each column in the table is a unique

combination of input conditions that result in triggering the action(s) associated with the rule.

33

every possible thing a user could possibly do with your system to demolish the software.

35

White-Box Testing

White-box testing is a verification technique software engineers can use to examine if their code works as expected.

White-box testing is testing that takes into account the internal mechanism of a system or

component . It also known as structural testing, clear box testing, and glass box testing (Beizer, 1995).

• types of testing:o There are six basic types of testing:

1. Unit Testing.2. Integration Testing.3. Regression Testing.4. function/system Testing.5. Acceptance Testing. 6. Beta Testing.

White-box testing is used for 1,2,and 3 respectively.

white-box testing techniques :1. White-Box Testing by Stubs and

Drivers.2. Deriving Test Cases.3. Control-flow/Coverage Testing.4. Data Flow Testing.5. Failure (“Dirty”) Test Cases.6. Flow Graphs Revisited.

• With white-box testing, you must run the code with predetermined input and check to make

sure that the code produces predetermined outputs. Often programmers write stubs and

drivers for white-box testing.• A driver: is a software module used to

invoke a module under test and, often, provide test inputs, control and monitor execution, and report test results

• A stub: is a computer program statement substituting for the body of a software module that is or will be defined elsewhere.

A driver: movePlayer(Player1, 2); A stub: public void movePlayer(Player player, int

diceValue) { player.setPosition(1); }

statistics show that the act of careful, complete, systematic test design will catch as many bugs as the act of testing .

The test design process, at all levels, is at least as effective at catching bugs as is running the test case designed by that process.

Basis Path Testing : A testing mechanism proposed by

McCabe. is a means for ensuring that all

independent paths through a code module have been tested.

An independent path is any path through the code that introduces at least one new set of processing statements or a new condition.

• cyclomatic number :• Number of conditions (diamonds) and add 1.

1-2-3-4-5-10 (property owned by others, no money for rent)

1-2-3-4-6-10 (property owned by others, pay rent)

1-2-3-10 (property owned by the player)1-2-7-10 (property available, don’t have

enough money)1-2-7-8-10 (property available, have money,

don’t want to buy it)1-2-7-8-9-10 (property available, have money,

and buy it)

• Equivalence Partitioning/Boundary Value Analysis :

• Equivalence partitioning (EP) and boundary value analysis (BVA) provide a strategy for writing white-box test cases. Undoubtedly, whenever you encounter any kind of number or limit in a requirement, you should be alert for EP/BVA issues.

• then we will check the object by giving boundary values as :min,min-1,min+1,max,max-1,max+1

Equivalence Partitioning/Boundary Value Analysis :

1. property costs $100, have $200 (equivalence class “have enough money”).

2. property costs $100, have $50 (equivalence class, “don’t have enough money”).

3. property costs $100, have $100 (boundary value).



Coverage: is a measure of the completeness of the set of test cases

Statement Coverage Decision/Branch Coverage Condition Coverage Decision/Condition Coverage Path Coverage

• If ((A>1) & (B=0)) then Do;

• X=X/A;• END;• If ((A==2) | (X>1))

then Do;• X=X+1;• END;• END;

Source: The Art of Software Testing – Glenford Myers

Statement Coverage

• Exercise all statements at least once

• How many test cases?A=2 and B=0 (ace)


Decision/Branch Coverage

• Each decision has a true and a false outcome at least once

• How many test cases?A=2 and B=0 (ace)A=1 and X=1 (abd)


Condition Coverage• Each condition in a

decision takes on all possible outcomes at least once

• Conditions: A>1, B=0, A=2, X>1

• How many test cases?A=2, B=0, and X=4

(ace)A=1, B=1, and X=1

(abd)Source: The Art of Software Testing – Glenford Myers

Decision/Condition Coverage

• Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once

• How many test cases? A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

• What about these? A=1, B=0, and X=3 A=2, B=1, and X=1

(abe)

(abe)

Multiple Condition Coverage

• Exercise all possible combinations of condition outcomes in each decision

• Conditions:


A>1, B=0A>1, B<>0A<=1, B=0A<=1, B<>0

A=2, X>1A=2, X<=1A<>2, X>1A<>2, X<=1

Multiple Condition Coverage

• How many test cases?A=2, B=0, X=4A=2, B=1, X=1A=1, B=0, X=2A=1, B=1, X=1


(ace)(abe)(abe)(abd)

Path Coverage

• Every unique path through the program is executed at least once

• How many test cases?A=2, B=0, X=4 (ace)A=2, B=1, X=1 (abe)A=3, B=0, X=1 (acd)A=1, B=1, X=1 (abd)


the control flowgraph is annotated with information about how the program variables are defined and used.

Look at the structure of your code and think about every possible way a user might break it.

Some suggestions follow:- Look at every input into the code

you are testing.- Look at every calculation

When you encounter a compound predicate, such as in, you must break the expression up so that each Boolean sub-expression is evaluated on its own.

Why Do Both?

Black-box• Impossible to write a

test case for every possible set of inputs and outputs

• Some of the code may not be reachable without extraordinary measures

• Specifications are not always complete

White-box• Does not address the

question of whether or not the program matches the specification

• Does not tell you if all of the functionality has been implemented

• Does not discover missing program logic

References : Cem Kaner, Testing Computer

Software. http://www.win.tue.nl/~mousavi/testi

ng/7.pdf.

www.sisqa.com/Training/White_Box_Testing-200709.ppt

http://www.win.tue.nl/~mousavi/testing/7.pdf

http://www.win.tue.nl/~mousavi/testing/7.pdf

Questions ?

64

Testing

Technology

Transcript of Testing