Chapter 8: Path Testing CSCI 565. Objectives Program Graphs Decision-to-decision path (DD-Paths) ...
-
date post
20-Dec-2015 -
Category
Documents
-
view
235 -
download
3
Transcript of Chapter 8: Path Testing CSCI 565. Objectives Program Graphs Decision-to-decision path (DD-Paths) ...
Chapter 8: Path TestingCSCI 565
Objectives
Program Graphs
Decision-to-decision path (DD-Paths)
Test Coverage Metrics
Basis Path Testing
Observation on McCabe Basis Path Method
Essential Complexity
Structural Coverage Analysis
The specification based testing can be used to verify the intended functionality
The techniques cannot be used with respect to unintended functionality
Structural testing (white box) is complementary solution
Shows reachability
Program Graph
Program Graph (PG) A directed graph G= (V, E) Where
V = a set of nodes corresponding to program statements in an imperative programming languages (e.g., C)
E = a set of edges corresponding to flow of control
Resilient properties of PG can be nested, which is the equivalent of a subroutine call
can be sequenced, which is the equivalent of sequential execution of two instructions
Program Graphs for elements of Programming langauges
ignore
t f
t
ff
f
f
Test Selection Criteria?
Test Selection coverage Criteria
Help to measure the adequacy of a test suite
E.g., if we use statement coverage to test 40% of the code, it means 60% of code was never executed
Help to decide when to STOP testing
E.g., use other criteria to cover 85%
Coverage?
refers to the extent by which a given verification activity has satisfied its objectives
a measure, not a method or a test
expressed as the percentage of an activity that is
accomplished.
Types of Structural Coverages
Typically structural coverage criteria are divided into two types: Data flow Control flow
Data flow criteria? measure the flow of data between variable
assignments and references to the variables Control flow criteria?
measure the flow of control between statements and sequences of statements
E.F. Millers’s Coverage Metrics
Overview of the families of test selection criteria Families of test selection criteria include
Structural model coverage criteria All statements (C0)
All DD-path (C1) (chain)
All conditions (predicates) to each outcome (T orF) (C1p)
All paths (C)
C C1p C0
Multiple condition coverage (CMCC)
A test set achieves CMCC if it exercises all possible combinations of condition outcomes in each decision For example, for N conditions, it requires 2N tests
(truth table) Dependent pairs Cd (Define/Usage paths)
Example of MCC: a compound condition of three variables of a, b, c.
Statement testing (C0)
Let T be a test suite for a Program P, then T satisfies the statement criterion for P, iff, for each statement S of P, there exists at least one test case in T that causes the execution of S.
In terms of Flow graph model of program P, it is the same as visiting each single node on some execution path exercised by a test case in T
C0 = number of executed statements/ total number of statements
Statement testing: Small Java Program
Statement testing: Flow Graph
C0: path aceTest case: A=2, B=0, X=3
C1p: Every predicate
Let T be a test suite for a program P. T satisfies the decision adequacy criterion for P, iff, for each branch B (or predicate P), there exists at
least one test case in T that causes execution of B.
This is the same as stating that every edge in the flow graph model of program P belongs to some execution path exercised by a test case in T
C1p = number of executed branches/ total number of branches
T satisfies the branch adequacy criterion if C1p =1
predicate Testing: Flow Graph
C1: p1: acd and p2: abeT1: A=3, B=0, X=1T2: A=2, B=1, X=1
P1:a,c,d
predicateTesting: Flow Graph
C1: p1: acd T1: A=3, B=0, X=1
predicate Testing: Flow Graph
C1: p2: abeT2: A=2, B=1, X=1
Predicate Testing: Flow Graph
Four conditions:A>1, B=0, and A=2, X>1Need test cases to force when 1) A>1, and A<=12) B=0, and B<>13) A=2, and A<> 24) X>1, and X<=1
C2 metric : Simple Loop Coverage + loop coverage
C2 metric demands the coverage of C1 and loop testing
Simple loop testingRequires that every loop decision
to be tested for two possible outcomes:One to force traversing the loop
bodyThe other one to exist
Loop boundary adequacy criterion (LC)
A test suite T for a program P satisfies the LC criterion iff for each loop l in P:
There is at least one execution in which control reaches the loop, and then the loop control condition evaluates to False the first time it is evaluated (zero time)
There is at least on e execution in which control reaches the loop and then the body of the loop is executed exactly once before control leaves the loop (one time)
There is at least one execution in which the body of the loop is repeated more than once (many times)
Dd Metric: C1 coverage + every define/usage
Refers to the most common dependency in DD-paths known as define/usages (covered in chapter 9: dataflow testing)
The dependency issues closely related to infeasible path problem associated with the control flow testing
E.g., C and H, and D and H pair in figure 8.5 The H branch can be traversed s when C = true The H branch can not be traversed when D=
false The path is syntactically feasible but
semantically is impossible
CMCC: Multiple Condition Coverage
A more complete extension that includes both the basic condition and branch adequacy criteria is called the multiple condition coverage CMCC
CMCC requires a test case for each possible evaluation of compound conditions. For N basic conditions, we need 2N
combinations of N basic conditionsShort-circuit evaluation is effective in
reducing the above number to a more manageable number.
CMCC: Multiple Condition Coverage
Four conditions:A>1, B=0, and A=2, X>1Need test cases to force when 1) A>1, and A<=12) B=0, and B<>13) A=2, and A<> 24) X>1, and X<=1
Test cases:T1: A=2, B=0, X=4T2: A=1, B=1, X=1
Truth table for (A>1 AND B =0)
conditions
T1 T2 T3 T4
A true true false false
B true false true falseA>1 AND B=0
true false false false
ACTIONS
X=X/A x - - -
Truth table for (A=2 or X>1)
conditions
T5 T6 T7 T8
A true true false false
X true false true falseA=2 OR x>1
true true true false
ACTIONS
X=X+1 x - - -
Test Data and Test cases for CMCC
General Form for Compound Condition
Example 2: Compound Condition
Modified Condition/Decision Coverage (MC/DC)
MC/DC Requires that each basic condition be shown to
independently affect the outcome of each decision a condition has independent effect when that condition
alone determines the outcome of the decision (AKA unique-cause approach)
For each basic condition C, select two test cases such that the truth values of all
evaluated conditions except C are the same, Also,the compound condition as a whole evaluates to
True for one of those test cases and False for the otherMC/DC is mandated by RTCA/Do-178B
Terminologies: Condition, Decision
Condition? A Boolean expression containing no Boolean
operators. E.g., A>B
Decision (compound Boolean expressions)? A Boolean expression composed of conditions and
zero or more Boolean operators. (AND, OR) (A>B OR (C=B AND D=H))
A decision without a Boolean operator is a condition. Also, If a condition appears more than once in a
decision, each occurrence is a distinct condition.
Understanding Logical operator
Minimum testing to meet MC/DC for AND
Minimum testing to meet MC/DC for OR
Minimum testing to meet MC/DC for OR
MC/DC for Z= (A or B) AND (C OR D)
Test case 2 is coupled by test case 10
for A
Test cases from 1-16
Path testing C (or P)
A test suite T for a program P satisfies the path adequacy criterion iff
for each path pi in P, there exists at least one test case in T that causes the execution of pi
C = # of executed paths/total number of paths
Linearly independent paths
ex1= p2+p3-p1
ex1= (1,0,1,2,0,0,0,0,1,0) +(1,0,0,0,1,0,0,1,0,1)-(1,0,0,1,0,0,0,0,1,0)
ex1 = (1,0,1,1,1,0,0,1,0,1)=A,B,C,B,E,F,G
ex2=2p2-p1
ex2=(2,0,2,4,0,0,0,0,2,0) – (1,0,0,1,0,0,0,0,1,0)= (1,0,2,3,0,0,0,0,1,0)=A,B,C,B,C,B,C,G
McCabe’s Algorithmic Procedure (Baseline method)
McCable’s Algorithmic Procedure can be used to identify a set of basis paths Select a path with highest number of decision nodes Retrace each decision in baseline path (i.e., normal path
that can be arbitrary path) Flip each decision to create a new path
E.g. p1: A,B,C,B,E,F,G (baseline, flip A)
P2: A,D,E,F,G (flip p1 at D)
P3:A,D,F,G (take D)
P4:A,B,E,F,G (flip p1 at B)
P5:A,B,C,G (flip p1 at C)
-The basis paths is different from the one in table 2
-A basis path is not required to be unique
t f
t
ff
f
f
Example: Applying McCabe Procedure
p1
p2
p3
p4
p5
Observations on McCabe’s Basis Path
Infeasible paths
Essential Complexity
V(G)=1wellStr(P)
V(G) >=10 requires substantial testing
Use the characteristics of code to select appropriate criteria
More on White-box testing
The purpose of structural coverage analysis with the associated structural coverage analysis resolution is to complement requirements-based testing as follows:
Provide evidence that the code structure was verified to the degree required for the applicable software level;
Provide a means to support demonstration of absence of unintended functions;
no information about whether the code is doing what it is supposed to be doing as specified in the requirements
Establish the thoroughness of requirements-based testing.
References
Material in this presentation is taken from the following resources: Paul C. Jorgensen. Software Testing: A
Craftsman’s Approach, 3rd Edition. Auerbach Publication, 2008
Mauro Pezze and Michal Young. Software Testing And Analysis. Wiley 2008
Kelly j. Hayhurst et al. A Practical Tutoring on Modified Condition/Decision Coverage NASA/TM-2001-210876, May 2001.