Testing Techniques
description
Transcript of Testing Techniques
Testing Techniques
Equivalence class analysis Cause effect graphs All pair testing
Equivalence classes
Equivalence classes
Equivalence classes
Triangle Problem
• Read 3 integer values in range [1..200]. • These 3 values represent the length of the
sides of a triangle. The programme displays a message which establishes that the triangle is isosceles, equilateral or scalene.
Triangle Problem• we are interested in 4 questions:
– Is it a triangle?– Is it an isosceles?– Is it a scalene?– Is it an equilateral?
• We may define the input test data by defining the equivalence class through “viewing” the 4 output groups:– input sides <a, b, c> do not form a triangle– input sides <a, b ,c> form an isosceles triangle– input sides <a, b, c> form a scalene triangle– input sides <a, b, c> form an equilateral triangle
Valid Classes
Non Valid Classes
Non Valid Classes
Could you find other non valid classes ?
Cause effect graph
Methodology
Triangle example
Triangle Problem
Cause Effect Graph
Decision Table
Test cases
All pairs testing
• Based on orthogonal Array• Drastically reduces number of test cases• A combination of worst cases and equivalence
class testing• Well supported in the agile methods
community• Supported by commercial and non-
commercial products
What Does All Pairs Do?
• Input is a set of equivalence classes for each variable.• Sometimes the equivalence classes are those used in
Boundary Value testing (min, min+, nominal, max-, max)• Output is a set of (partial) test cases that approximate an
orthogonal array, plus some pairing information.• Why partial? No expected outputs are provided. natural
enough, but a tester still needs to do this step.
All Pairs Assumptions
• Variables have clear equivalence classes.
• Variables are independent.
• Failures are the result of the interaction of a pair of variable values.
Example
• Suppose that we have three parameters.• For each parameter, there are two possible values.
– Values are :• A, B for parameter 1.• J, K for parameter 2.• Y, Z for parameter 3.
• Degree of interaction coverage is 2.– We want to cover all potential 2-way interactions
among parameter values.
Set of all possible test configurations
A J Y
A K Z
A K Y
A J Z
K YB
B ZJ
B YJ
K ZB
There are 23 = 8 possible test configurations.
Three parameters, twovalues for each.
Set of all possible degree 2interaction elements
• Coverage measure:– Percentage of interaction elements covered.
There are ( ) 2 2 = 12 possible interaction elements. 32
A J
B K
B J
A K
A Y
A Z
B Z
B Y
J Y
K Z
K Y
J Z
Test configurations assets of interactions
A J Y
A J
A Y
J Y
One test configuration...
… covers 3 possibleinteraction elements.
…using a subset ofall test configurations.
Interaction test coverage goal
Goal: cover all interactionelements…
A J Y
K YB
A K ZA K YA J Z
B ZJB YJ
K ZB
A J
B KB JA K
A YA Z
B ZB Y
J Y
K ZK YJ Z
A J Y
K YB
A K ZA K YA J Z
B ZJB YJ
K ZB
A J
B KB JA K
A YA Z
B ZB Y
J Y
K ZK YJ Z
Selection of test configurations forcoverage of interaction elements
Interaction elements Test configurations
A J A Y J YA J Y
Degree 2 coverage: 3 / 12 = 25%
Degree 3 coverage: 1 / 8 = 12.5%
A KA K
A J Y
K YB
A K ZA K YA J Z
B ZJB YJ
K ZB
A J
B KB J
A YA Z
B ZB Y
J Y
K ZK YJ Z
Selection of test configurations forcoverage of interaction elements
Interaction elements Test configurations
A K Z
A Z
K Z
Degree 2 coverage: 6 / 12 = 50%
Degree 3 coverage: 2 / 8 = 25%
A J Y
K YB
A K ZA K YA J Z
B ZJB YJ
K ZB
A J Y
A K Z
A J
B KB JA K
A YA Z
B ZB Y
J Y
K ZK YJ Z
Selection of test configurations forcoverage of interaction elements
Interaction elements Test configurations
B ZJ
B JB Z
J Z
Degree 2 coverage: 9 / 12 = 75%
Degree 3 coverage: 3 / 8 = 37.5%
A J Y
K YB
A K ZA K YA J Z
B ZJB YJ
K ZB
A J Y
A K Z
B ZJ
A J
B KB JA K
A YA Z
B ZB Y
J Y
K ZK YJ Z
Selection of test configurations forcoverage of interaction elements
Interaction elements Test configurations
B KB Y K Y
K YBDegree 2 coverage: 12 / 12 = 100%
Degree 3 coverage: 4 / 8 = 50%
Choosing the degree of coverage• In one experiment, covering 2 way interactions resulted in the following
average code coverage:– 93% block coverage.– 83% decision coverage.– 76% c-use coverage.– 73% p-use coverage.– Source: Cohen, et al, “The combinatorial design approach to
automatic test generation”, IEEE Software, Sept. 1996.• Another experience report investigating interactions among 2-4
components:– Dunietz, et al, “Applying design of experiments to software testing”,
Proc. Of ICSE ‘97.
Online Mortgage Application(Thanks, Bernie Berger, STAREast)
NY L 1 fam A+ Pri 80% Yes Yes Yes Cust Yes Yes
NJ M 2 fam A Vac 90% No No No Bank No No
FL H 3 fam A- Inv 100%
TX H+1 4 fam B
CA H+2 Coop <B
DC H+3 Condo
Other
Regi
on
tier
Prop
erty
Cred
it
Resid
ence
LTV
NIV
NAV
Refin
ance
Clos
ing
Cost
Intr
o Ra
te
Bank
Em
p
Twelve variables, with varying numbers of values, have7 x 6 x 6 x 5 x 3 x 3 x 2 x 2 x 2 x 2 x 2 x 2 = 725,760combinations of values.
“All Pairs does it in 50.” (Bernie Berger, STAREast2003)
Total number of combinations
1234
47484950
NYNYNJNJNY
OtherOtherOther
LMLM
H+1H
H+2H+1
1 fam2 fam2 fam1 fam
~Coop2 fam3 fam
Condo
A+A
A-BB
~B~A-<B
PriVacInvPri
~Inv~Vac~Pri
~Vac
80%90%
100%90%
~80%~100%~80%~90%
YesNo
YesNo
~No~No
~Yes~No
YesNoNo
Yes~No~No~No~No
YesNo
YesNo
~Yes~Yes~No
~Yes
CustBankBankCust
~Cust~Cust~Cust~Cust
YesNo
YesNo
~Yes~No~No~No
YesNoNo
Yes~Yes~No~No~No
666654461112
Selected All Pairs Test Cases (note fault resolution!)
Mortgage Application Observations
• The example uses values, not equivalence classes.• Values are independent,• System level testing (apparently).• The efficiency comes at the expense of fault isolation.
There are 2C12 = 12!/(2! x (12 - 2)!) = 66 pairs in the first row of the table. If that test case fails, which of the 66 pairs caused the failure?
• NB: excellent candidate for regression testing
What if the fault involves more than just a pair of variables?
pCn1
12
å =2^12 - 1=4095
pCn =n!
p! (n - p)!
Number of combinations of n things taken p at a time is
Worst case (for first test case):