1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)
-
date post
21-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)
1
Software Testing and Quality Assurance
Lecture 25 – Testing Interactions
(Chapter 6)
2
Lecture Outline Sampling Test Cases Orthogonal array testing
3
Sampling test cases Population: all possible test cases that
can be executed A sample is a subset of a population
that has been selected based on some probability distribution.
4
Sampling test cases Possibilities for determining which test cases
to select Based on probability distribution
A probability distribution defines, for each data value in a population, a set of allowable values and the probability that value will be selected.
Uniform probability distribution Each value in the population is assigned the same
selection probability For example, Probability distribution based on the
use profile
5
Sampling test cases (cont...) A stratified sample is a set of samples
in which each sample represents a specific subpopulation Example: select a sample of test cases
from the use cases of each actor Stratified sample tests are selected from
a series of categories.
6
Sampling test cases (cont...) Use of random number generator in
sampling Advantage: all values have equal
probability Disadvantage: test cases cannot be
reproduced (you may let the test driver record the generated values)
Class family: a set of classes related by inheritance.
7
Sampling test cases: Orthogonal array testing (OATS)
Orthogonal arrays provide a specific sampling technique that seeks to limit the explosion by defining pair-wise combinations of a set of interacting objects.
An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).
8
Sampling test cases: Orthogonal array testing (OATS)
It represents a specific class family (a class and its children)
E.g. 3 factors with 3 levels each (27 possibilities), with pair-wise only 9 possibilities
9
Sampling test cases: OATS Example Decide how many independent variables will
be tested for interaction. This will map to the Factors of the array.
Decide the maximum number of values that each independent variable will take on. This will map to the Levels of the array.
10
Sampling test cases: OATS Example Find a suitable orthogonal array. A
suitable array is one that has at least as many Factors as needed from Step 1 and has at least as many levels for each of those factors as decided in Step 2.
Map the Factors and values onto the array.
Transcribe the Runs into test cases.
11
Sampling test cases: OATS example Number of states:
Class A: 2 Class B: 3 Class P: 1 Class C: 2 Class D: 3 Class E: 3
* State transition diagrams are not shown
12
Sampling test cases: OATS example (cont...)
Step 1: Identify all factors: sending hierarchy, receiving hierarchy, parameter position in the message (six factors in the example: class A hierarchy, class P hierarchy, class C hierarchy and factors associated with each class hierarchy)
13
Sampling test cases: OATS example (cont...)
Step 2: Determine levels for each factor by considering the set of possible values: One factor has one level: the parameter
class family only has one member: P Two factors have a maximum of two levels;
B and C. Three factors have a maximum of three
levels: A, D and E.
14
Sampling test cases: OATS example (cont...)
Step 3: Locate a standard orthogonal array that fits the problem.
6 factors 3 levels
15
Sampling test cases: OATS example (cont...)
16
Sampling test cases: OATS example (cont...)
Step 4: Establish a mapping from each factor onto the integers in the array so that standard array can be interpreted The first column in L18 can be used to represent
the sender class family which has two classes: A and B (1 corresponds to A class and 2 corresponds to class B)
The second column in L18 when there is a difference in the number of levels (class A has 2 states and class B has 3 states).
17
Sampling test cases: OATS example (cont...) The third column in L18 represents the
parameter hierarchy that only has one class, P. (any value in the third column represents P).
The fourth column represents the state of P (which there are two).
The fifth column represents the class C hierarchy, which has three members.
The sixth column represents the states of the C, D, and E classes.
18
Sampling test cases: OATS example (cont...)
Step 5: Construct test cases based on the mapping and the rows in the table. Each row in the orthogonal array specifies
one specific test case.
19
Sampling test cases: OATS example— result interpretation
The 10th column represents: An instance of B in
state 1 Is to send a
message by passing an instance of class P in state 3
To an instance of class E in state 2.
20
Sampling test cases: OATS another example
Consider a web page with three distinct sections (Top, Middle, and Bottom) that can be individually shown or hidden by the user. You wish to test the interactions of the different sections.
21
Sampling test cases: OATS another example (cont...)
There are three independent variables (the sections of the page).
Each variable can take on two values (hidden or visible).
An L4 orthogonal array will do the job — two
levels for the values and three factors for the variables.
Mapping the values onto the array where Hidden=0 and Visible=1
22
Sampling test cases: OATS another example (cont...)
OA before mapping factors
Factor 1 Factor 2 Factor 3Run 1 0 0 0
Run 2 0 1 1
Run 3 1 0 1
Run 4 1 1 0
OA after mapping factors
Top Middle BottomTest case 1 Hidden Hidden Hidden
Test case 2 Hidden Visible Visible
Test case 3 Visible Hidden Visible
Test case 4 Visible Visible Hidden
23
Sampling test cases: adequacy criteria for OATS
Exhaustive: all possible combinations of all factors are considered.
Minimal: only the interactions between the base classes from each hierarchy are tested.
Random: the tester haphazardly selects cases from several of the classes.
24
Sampling test cases: adequacy criteria for OATS
Representative: a uniform sample that ensures that every class is tested to some level.
Weighted representative: Add cases to the representative approach based on relative importance or risk associated with the class.
25
Key points Possibilities for determining which test cases to
select: based on probability distribution, uniform probability distribution and probability distribution based on the use profile.
A stratified sample is a set of samples in which each sample represents a specific subpopulation
An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).
Adequacy criteria for OATS: exhaustive, minimal, random, representative, and weighted representative.