Download - Software Testing and Quality Assurance Testing Interactions.

Software Testing and Quality Assurance

Testing Interactions

2

Reading Assignment

John McGregor and David A. Sykes, A Practical Guide to Testing Object-Oriented Software, Addison-Wesley, 2001, ISBN: 0-201-325640. Chapter 6: Testing Interactions

3

Objectives

To understand different types of interactions. Learn techniques on how to select test cases

to run.

4

Topics covered

Introduction Object interactions Testing object interactions Sampling test cases Testing off-the-shelf components Protocol testing Test patterns Testing exceptions

5

Object Interactions

An instance of a trusted primitive class, for example, may contain no faults, but if the services of that instance are not used correctly by other program components, then the program contains faults.

Thus the correct collaboration—or interaction—of objects in a program is critical to the correctness of the program.

6

Introduction Object-oriented program comprises a collection

of objects that collaborate to solve some problem.

Most classes have collaborators: The methods in the class interact with instances of

other classes. The interactions being tested are between

objects at runtime. The focus of interaction testing is ensuring that

messaging occurs correctly with objects whose classes have already been tested separately.

7

Object interactions An object interaction is simply a request by one object

(the sender) to another (the receiver) to perform one of the receiver’s operations and all of the processing performed by the receiver to complete the request.

Object interactions include: Messages between an object and its components Messages between an object and other objects with which it is

associated. We want to consider the impact of these interactions

both on the internal state of the receiving object and on those objects with which it has an association

8

Partial Class Testing Partial class testing is a result of the iterative,

incremental development approach. In general,

It is not possible to sequence the development of a class so that all the classes it needs to interact with are totally developed and tested.

A project's schedule is usually based on delivery of end-user functionality (use cases).

9

Partial Class Testing (cont.) Lower level— Primitive classes are more

likely to be completely developed at one time and tested as a complete unit.

Other classes are therefore developed and tested incrementally.

10

Object Interactions: Identifying Interactions A primitive class is one that can be used

without any collaboration with other objects.

A nonprimitive class is one that requires collaboration with other objects in order to accomplish its objectives.

11

Object Interactions: Identifying Interactions (cont...) Possible collaborations:

Parameter: public operation collaborates with an object passed in as an argument (formal parameter).

Return: public operation returns an object with which it collaborates.

Internal: a method of a class creates an instance of another as part of its implementation.

Global: reference to a global instance of some class (undesired).

12

Object Interactions: Identifying Interactions (cont...) Categories of nonprimitive classes based on

degree of interaction:

Collection classes: maintain references to instances of other classes, but do not interact with those instances.

Collaborating classes: interacts directly with its associated classes classes that are not collection classes.

13

Object Interactions: Identifying Interactions— Collection Classes Collection classes: Use objects in their

specifications but never actually collaborate with any of them.

Instead they do one of the following: Store references Create instances Delete instances

Examples: lists, stacks, and queues.

14

Object Interactions: Identifying Interactions— Collaborating Classes Collaborating Classes

Non-primitive classes that are not collection classes. They are classes that use other objects in one or

more operations as part of their implementation. When a post condition of an operation in a class's

interface refers to the state of an instance of an object and/or specifies that some attribute of that object is used or

modified,

then that class is a collaborating class.

15

Object Interactions: Identifying Interactions— Collaborating Classes (cont...) The number of potential collaborations can

become impossibly large quickly .

Often the bugs that are most serious do not arise from the interaction of two objects, but from the interactions between a complete set of objects.

16

Object Interactions: Identifying Interactions— Collaborating Classes (cont...) Factors that determine the size for testing:

Composition and association: The interaction of the composing object with its composed

attributes is tested.

The interaction between an object and its associated objects are tested as successive layers of aggregation are integrated.

The number of layers of aggregations created between interaction tests is closely related to the visibility of defects.

17

Object Interactions: Identifying Interactions— Collaborating Classes (cont...) Object complexity: the more complex the objects,

the fewer that should be integrated prior to a round of testing.

number of parameters for each method, number of methods, number of attributes in each object.

Trying to test a chunk that is too complex often results in defects that successfully hide from the tests.

18

Testing Object Interactions: Testing Collection Classes

Collection classes are tested using techniques for primitive classes: A test driver will create instances that are passed as

parameters in messages to a collection being tested.

Test cases center around ensuring that those instances are correctly incorporated into and removed from the collection class

If defensive design has been used then include negative test cases in the test suite.

19

Specifying Interactions

Assume that operations defined by a class are specified by preconditions, post conditions, and class invariants.

It is important to know whether defensive design or design by contract has been used in creating the specification of the particular interface to be tested.

20

Specifying Interactions (Cont.) Defensive design assumes that little or no

checking of parameter boundaries occurs prior to a message being sent. reduces the number of clauses in preconditions, requires checks internally for violations of attribute

constraints, increases the number of clauses in post

conditions.

21

Specifying Interactions (Cont.) Design by contract assumes that appropriate preconditions are checked prior to a message being sent and that the message is not sent if any of the parameters are outside acceptable limits. increases the number of clauses in

preconditions, requires no checking internally for violations of

attribute constraints, and reduces the number of clauses in the post

condition clause.

22

Testing Object Interactions: Testing Collaborator Classes The complexity of testing a collaborating

class is greater than that of testing a collection class or a primitive class.

For a collaborator class under test: Identify the collaborating classes Determine the sequence of testing

Collaborating classes, Next level of collaboration Then the next level of collaboration and so on

23

The Interaction between Testing and Design Approach Contract design places more responsibility on the

human designer than on error-checking code. This reduces the amount of class-level testing since

there are fewer paths due to a smaller amount of error-checking code.

At the interaction level, there is more testing required in order to be certain that the human designer has complied with the client side of the contract using precondition constraints.

A focus of interaction testing is whether the preconditions of methods in a receiving object are being met by the sending object.

24

The Interaction between Testing and Design Approach (Cont.) It is not legitimate to build test cases that

violate these preconditions. The intention is to determine that the sending

object checks the preconditions of the receiving object before sending the message inappropriately.

The test should also check whether the sending object aborts correctly, probably by throwing an appropriate exception.

25

Sampling Test Cases

When the number of test cases is too large to handle reasonably

A systematic technique is needed for determining which ones to use.

We are interested with a testing approach in which the level of coverage can be increased systematically.

26

Sampling Test Cases Population: all possible test cases that can be executed.

It includes all preconditions and all possible combinations of input values.

A sample is a subset of a population that has been selected based on some probability distribution.

Possibilities for determining which test cases to select Based on probability distribution

A probability distribution defines, for each data value in a population, a set of allowable values and the probability that value will be selected.

Uniform probability distribution Each value in the population is assigned the same selection probability

27

Sampling Test Cases (cont...) A stratified sample is a set of samples in which

each sample represents a specific subpopulation Stratified sample tests are selected from a series

of categories. A population of tests is divided into subsets so

that a subset contains all of the tests that exercise a specific component. Sampling occurs on each subset independent of the others

Example: select a sample of test cases from the use cases of each actor

28

Sampling Test Cases (cont...)

Stratifying the test case samples by each actor provides an effective means of increasing the reliability of the system.

Running the selected tests uses the system the way that it will be used in typical situations and finds those defects that are most likely to be found in typical use.

Removing these defects produces the largest possible increases in reliability with the smallest effort.

29


Use of random number generator in sampling Advantage: all values have equal probability

Disadvantage: test cases cannot be reproduced.

Example Velocity class, set direction: values between 0 and 359, inclusive.int (random() * 360).

30


The first dimension in sampling is considering the class family Class family: a set of classes related by

inheritance. Example, Sprite is the base class in a very large class

family, which is a set of classes related by inheritance. An object from any one of the classes in the family can

be substituted for the sprite parameter.

31


The second dimension for sampling Consider that each member of the family may

have different states that can cause two objects from the same class to behave differently.

There will be the same number of states or more states in the derived class as there are in the base class.

Cover the states defined for each class with special emphasis on the new states added at that level in the inheritance hierarchy.

32

Sampling Test Cases: Orthogonal Array Testing (OATS) Orthogonal arrays provide a specific sampling

technique that seeks to limit the explosion by defining pair-wise combinations of a set of interacting objects.

An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).

It represents a specific class family (a class and its children)

E.g. 3 factors with 3 levels each (27 possibilities), with pair-wise only 9 possibilities

33

Sampling Test Cases: Orthogonal Array Testing (OATS)

• This is a systematic way of reducing the number of test cases.

• Most of the errors that are encountered are between pairs of objects rather than among several objects.

34

Sampling Test Cases: Orthogonal Array Testing (OATS)

Decide how many independent variables will be tested for interaction - Factors of the array.

Decide the maximum number of values that each independent variable will take on - the Levels of the array.

Find a suitable orthogonal array - has at least as many Factors as needed - has at least as many levels for each of those factors

Map the Factors and values onto the array. Transcribe the Runs into test cases.

35

Sampling Test Cases: OATS Example 1

Number of states: Class A: 2 Class B: 3 Class P: 2 Class C: 2 Class D: 3 Class E: 3

* State transition diagrams are not shown

36

Sampling Test Cases: OATS Example 1 Step 1: Identify all factors: sending hierarchy, receiving hierarchy, parameter position in the message

six factors in the example: class A hierarchy, class P hierarchy, class C hierarchy and factors for the states associated with each class hierarchy

Step 2: Determine levels for each factor by considering the set of possible values: One factor has one level: the parameter class family only has one

member: P Two factors have a maximum of two levels; A and C; the maximum

number of states for a class in the P family is two. Three factors have a maximum of three levels: B, D and E.

37


Step 3: Locate a standard orthogonal array that fits the problem.

6 factors 3 levels

38

Sampling Test Cases: OATS Example 1 Step 4: Establish a mapping from each factor onto the integers in the array so that standard array can be interpreted The first column in L18 can be used to represent the sender class

family which has two classes: A and B (1 corresponds to A class and 2 corresponds to class B)

The second column in L18 when there is a difference in the number of levels (class A has 2 states and class B has 3 states).

The third column in L18 represents the parameter hierarchy that only has one class, P. (any value in the third column represents P).

The fourth column represents the state of P (which there are two). The fifth column represents the class C hierarchy, which has three

members. The sixth column represents the states of the C, D, and E classes.

39


2

40

Sampling Test Cases: OATS Example 1 Step 5: Construct test cases based on the

mapping and the rows in the table.

Each row in the orthogonal array specifies one specific test case.

41

Sampling Test Cases: OATS Example 1— Result Interpretation

The 10th column represents: An instance of B in

state 1 Is to send a

message by passing an instance of class P in state 2

To an instance of class E in state 2.

42

Sampling Test Cases: OATS Example 2 Consider a web page with three distinct

sections (Top, Middle, and Bottom) that can be individually shown or hidden by the user. You wish to test the interactions of the different sections.

43

Sampling Test Cases: OATS Example 2 There are three independent variables (the

sections of the page).

Each variable can take on two values (hidden or visible).

An L4(23) orthogonal array will do the job — two

levels for the values and three factors for the variables.

Mapping the values onto the array where Hidden=0 and Visible=1

44


OA before mapping factors

Factor 1 Factor 2 Factor 3Run 1 0 0 0

Run 2 0 1 1

Run 3 1 0 1

Run 4 1 1 0

OA after mapping factors

Top Middle BottomTest case 1 Hidden Hidden Hidden

Test case 2 Hidden Visible Visible

Test case 3 Visible Hidden Visible

Test case 4 Visible Visible Hidden

45

Sampling test cases: Adequacy Criteria for OATS Exhaustive: all possible combinations of all factors

are considered. Minimal: only the interactions between the base

classes from each hierarchy are tested. Random: the tester haphazardly selects cases from

several of the classes. Representative: a uniform sample that ensures that

every class is tested to some level. Weighted Representative: Add cases to the

representative approach based on relative importance or risk associated with the class.

46

Testing off-the-shelf components Functionality can be added to an application by

purchasing “Components”. Quality of components varies from vendor to another. Acceptance testing for the component in the context

in which it will be used is necessary. Stress tests: extreme, may be correct or incorrect

values e.g., moving mouse a lot to generate a large number of mouse move events, making multiple menu selections, etc.

Tests are normally based on the main class.

47

Testing off-the-shelf components: example ―a component test plan for Grid

48

Protocol testing Protocol testing investigates whether the

implementation of a class satisfies its specification Protocol: sequence of method invocations by

combining a method whose postconditions enable the precondition of another method

Each protocol corresponds to a sequence of states beginning with initial states of the two objects, a sequence of states for each object, and ending with terminal states (denoted in the state diagram)

A test case takes the two objects through one complete sequence of methods

49

Protocol testing: Example

Timer maintains list of observers to notify when a timer event occurs.

When it is enabled, it notifies the observers

50

Protocol testing: Example

A protocol can be found by tracing through the state diagram. One protocol would be to create the object then

sending one or more attach(...) messages followed by the enable() message, the disable message, and finally, the delete() message.

This provides a life-cycle test case. This provides an effective test of the object in the

ways that it will interact with its client objects.

51

Test patterns Test patterns are design patterns for test

software. A design pattern: captures and reuses design

knowledge that has gained widespread application in the OO software development community, e.g., Grid component is based on the Listener design pattern

A pattern is a specific configuration of interactions among a set of objects that form some cluster in the overall design.

For a design pattern there can be a test pattern

52

Testing exceptions

An exception: provides an alternative route for returning from a method that does not necessary move to the next statement after the method is invoked.

Exceptions are powerful: The exceptional return value is an object and can

be arbitrary complex.

The points at which an exception is thrown varies based on the depth of the aggregation hierarchy.

53

Testing exceptions (cont...) Testing exceptions provides two perspectives:

At the class level, whether each of the methods in that class does throw the exceptions that it claims to in the appropriate circumstances. The coverage criteria requires that a class throws every exception

contained in the specifications of the methods. There would be at least one test case per exception.

During integration, interaction testing will determine whether the exceptions that are being thrown at correct time are being caught at correct place. The coverage criteria for this level of testing is to be certain that

every exception thrown is caught in the correct location.

Both of these points of view can be tested very early in the development.

54

Testing exceptions: testing interactions at the system level When the components become complex it is

easier to test them in the context of the application itself instead of in the environment provided by a test driver.

The interactions that can be tested at the system level are only those that can be verified at the system level (e.g. user interface):

55

Key points

Primitive and nonprimitive classes. Possible collaborations: parameter, return,

internal, global. Categories of nonprimitive classes: collection

and collaborating classes Factors that determine the size for testing

Composition and association: aggregations Object complexity

56

Key points (cont...) Possibilities for determining which test cases to select:

based on probability distribution, uniform probability distribution and probability distribution based on the use profile.

A stratified sample is a set of samples in which each sample represents a specific subpopulation

An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).

Adequacy criteria for OATS: exhaustive, minimal, random, representative, and weighted representative.

57

Key points (cont...) Stress tests: extreme, may be correct or incorrect

values Protocol: sequence of method invocations by

combining a method whose postconditions enable the precondition of another method

A design pattern: captures and reuses design knowledge that has gained widespread application in the OO software development community

An exception: provides an alternative route for returning from a method that does not necessary move to the next statement after the method is invoked.