Path selection criteria Tor Stålhane & ‘Wande Daramola.
-
Upload
lorena-logan -
Category
Documents
-
view
229 -
download
3
Transcript of Path selection criteria Tor Stålhane & ‘Wande Daramola.
![Page 1: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/1.jpg)
Path selection criteria
Tor Stålhane & ‘Wande Daramola
![Page 2: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/2.jpg)
Why path selection criteria
Doing white box testing (Control flow testing, data flow testing, coverage testing) using the test-all-paths strategy can be a tedious and expensive affair. The strategies discussed here are alternative ways to reduce the number of paths to be tested.
As with all white box tests, it should only be used for small chunks of code – less than say 200 lines.
![Page 3: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/3.jpg)
• Data flow testing is a powerful tool to detect improper use of data values due to coding errors.
main() {
int x;
if (x==42){ ...}
}
Data flow testing -1
![Page 4: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/4.jpg)
• Variables that contain data values have a defined life cycle. They are created, they are used, and they are killed (destroyed) - Scope{ // begin outer block
int x; // x is defined as an integer within this outer block …; // x can be accessed here { // begin inner block
int y; // y is defined within this inner block ...; // both x and y can be accessed here
} // y is automatically destroyed at the end of this block ...; …; // x can still be accessed, but y is gone
} // x is automatically destroyed
Data flow testing -2
![Page 5: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/5.jpg)
• Variables can be used – in computation – in conditionals
• Possibilities for the first occurrence of a variable through a program path – ~d the variable does not exist, then it is defined (d)– ~u the variable does not exist, then it is used (u)– ~k the variable does not exist, then it is killed or
destroyed (k)
Static data flow testing
![Page 6: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/6.jpg)
define, use, kill (duk) – 1
We define three usages of a variable:
• d – define the variable
• u – use the variable
• k – kill the variable.
A large part of those who use this approach will only use define and use – du.
Based on the usages we can define a set of patterns potential problems.
![Page 7: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/7.jpg)
duk – 2 We have the following nine patterns:• dd: define and then define again – error• dk: define and then kill – error• ku: kill and then used – error • kk: kill and then kill again – error • du: define and then use – OK • kd: kill and then redefine – OK • ud: use and then redefine – OK• uk: use and then kill – OK • uu: use and then use – OK
![Page 8: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/8.jpg)
Example: Static data flow testing
For each variable within the module we will examine define-use-kill patterns along the control flow paths
![Page 9: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/9.jpg)
Example cont’d: Consider variable x as we traverse the left and then the right path
~define correct, the normal casedefine-define suspicious, perhaps a programming errordefine-use correct, the normal case
ddu du
![Page 10: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/10.jpg)
duk examples (x) – 1
Define x
Define xUse x
Define x
Define xUse x
Use x Use x
ddudu
![Page 11: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/11.jpg)
~use major blunderuse-define acceptabledefine-use correct, the normal caseuse-kill acceptable
Example Cont’d: Consider variable y
uduk udk
![Page 12: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/12.jpg)
duk examples (y)- 2
Define y
Use y
Use y
Define y
Use y
Kill y
Use y
Kill y
udukudk
![Page 13: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/13.jpg)
~kill programming error
kill-use major blunder
use-use correct, the normal case
use-define acceptable
kill-kill probably a programming error
kill-define acceptable
define-use correct, the normal case
kuuud kkduud
Example Cont’d: Consider variable z
![Page 14: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/14.jpg)
duk examples (z) - 3
Kill z
Use zUse z Kill z
Define z Kill zDefine z
Define zDefine z
Kill z
Use zUse z
kuuudkkduud
Use zUse z
![Page 15: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/15.jpg)
Dynamic data flow testingTest strategy – 1
Based on the three usages we can define a total of seven testing strategies. We will have a quick look at each
• All definitions (AD): test cases cover each definition of each variable for at least one use of the variable.
• All predicate-uses (APU): there is at least one path of each definition to p-use of the variable
![Page 16: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/16.jpg)
Test strategy – 2 • All computational uses (ACU): there is at
least one path of each variable to each c-use of the variable
• All p-use/some c-use (APU+C): there is at least one path of each variable to each c-use of the variable. If there are any variable definitions that are not covered then cover a c-use
![Page 17: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/17.jpg)
Test strategy – 3
• All c-uses/some p-uses (ACU+P): there is at least one path of each variable to each c-use of the variable. If there are any variable definitions that are not covered then cover a p-use.
• All uses (AU): there is at least one path of each variable to each c-use and each p-use of the variable.
![Page 18: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/18.jpg)
Test strategy – 4
• All du paths (ADUP): test cases cover every simple sub-path from each variable definition to every p-use and c-use of that variable.
Note that the “kill” usage is not included in any of the test strategies.
![Page 19: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/19.jpg)
Application of test strategies – 1
Define x
Define xc-use xc-use z
Kill zc-use xDefine z
p-use yKill z
Define yp-use z
c-use c-use z
Kill yDefine z
All definitionsAll c-use
Define x
Define xc-use xc-use z
Kill zc-use xDefine z
p-use yKill z
Define yp-use z
c-use c-use z
Kill yDefine z
All p-use
![Page 20: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/20.jpg)
Application of test strategies – 2
Define x
Define xc-use xc-use z
Kill zc-use xDefine z
p-use yKill z
Define yp-use z
c-use c-use z
Kill yDefine z
ACUAPU+C
![Page 21: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/21.jpg)
Relationship between strategies - 1All paths
All du-paths
All uses
All c/some p All p/some c
All c-uses
All p-uses
All defs
Branch
Statement The higher up in the hierarchy, the better is thetest strategy
![Page 22: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/22.jpg)
Relationship between strategies - 2
To make the relationships clear:
The main testing method here is path-testing.
However, since full path testing often will require us to execute a large number of paths, the duk-strategy helps us to reduce the number of paths necessary.
The diagram on the preceding slide helps us to make the necessary decisions.
![Page 23: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/23.jpg)
Acknowledgement
The material on the duk patterns and testing strategies are taken from a presentation made by L. Williams at the North Carolina State University.
Available at: http://agile.csc.ncsu.edu/testing/DataFlowTesting.pdf
Further Reading:
An Introduction to data flow testing – Janvi Badlaney et al., 2006
Available at: ftp://ftp.ncsu.edu/pub/tech/2006/TR-2006-22.pdf
![Page 24: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/24.jpg)
Use of coverage measures
Tor Stålhane
![Page 25: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/25.jpg)
Model – 1
We will use the following notation:• c: a coverage measure• r(c): reliability• 1 – r(c): failure rate• r(c) = 1 – k*exp(-b*c)
Thus, we also have that ln[1 – r(c)] = ln(k) – b*c
![Page 26: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/26.jpg)
Model – 2The equation ln[1 – r(c)] = ln(k) – b*c is of
the same type as Y = α*X + β.
We can thus use linear regression to estimate the parameters k and b by doing as follows:
1.Use linear regression to estimate α and β
2.We then have– k = exp(α)– b = - β
![Page 27: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/27.jpg)
Coverage measures considered
We have studied the following coverage measures:
• Statement coverage: percentage of statements executed.
• Branch coverage:percentage of branches executed
• LCSAJLinear Code Sequence And Jump
![Page 28: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/28.jpg)
Statement coverage
Statment
-ln(F
5)
1,00,90,80,70,60,50,40,3
13
12
11
10
9
8
7
Scatterplot of -ln(F5) vs Statment
![Page 29: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/29.jpg)
Graph summary-ln(F
5)
0,80,60,40,20,0
12
10
8
0,80,60,40,20,0
12
10
8
Statment Branch
LCSAJ
Scatterplot of -ln(F5) vs Statment; Branch; LCSAJ
![Page 30: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/30.jpg)
Equation summary
Statements:
-ln(F) = 6.5 + 6.4 Cstatement, R2(adj) = 85.3
Branches:
-ln(F) = 7.5 + 6.2 Cbranches, R2(adj) = 82.6
LCSAJ
-ln(F) = 6.5 + 6.4 CLCSAJ, R2(adj) = 77.8
![Page 31: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/31.jpg)
Usage patterns – 1
Not all parts of the code are used equally often. When it comes to reliability, we will get the greatest effect if we have a high coverage for the code that is used most often.
This also explains why companies or user groups disagrees so much when discussing the reliability of a software product.
![Page 32: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/32.jpg)
Usage patterns – 2
input domain
X
X
X
X
X
X
X
X
X
XInput space A
Corrected
![Page 33: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/33.jpg)
Usage patterns – 3
As long as we do not change our input space – usage pattern – we will experience no further errors.
New user groups with new ways to use the system will experience new errors.
![Page 34: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/34.jpg)
Usage patterns – 4
input domain
X
X
XX
X
X
Input space A
Input space B
![Page 35: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/35.jpg)
Extended model – 1 We will use the following notation:• c: coverage measure• r(c): reliability• 1 – r(c): failure rate• r(c) = 1 – k*exp(-a*p*c)• p: the strength of the relationship between
c and r. p will depend the coupling between coverage and faults.
• a: scaling constant
![Page 36: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/36.jpg)
Extended model – 2
C
1 - k
R(C)
1
Large p
Small p
1.00.0
Residual unreliability
![Page 37: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/37.jpg)
Extended model - commentsThe following relation holds:ln[1 – r(c)] = ln(k) – a*p*c
• Strong coupling between coverage and faults will increase the effect of test coverage on the reliability.
• Weak coupling will create a residual gap for reliability that cannot be fixed by more testing, only by increasing the coupling factor p – thus changing the usage pattern.
![Page 38: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/38.jpg)
Bishop’s coverage model – 1
Bishop’s model for predicting remaining errors is different from the models we have looked at earlier. It has a
• Simpler relationship between number of remaining errors and coverage
• More complex relationship between number of tests and achieved coverage
![Page 39: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/39.jpg)
Bishop’s coverage model – 2 We will use f = P(executed code fails). Thus,
the number of observed errors will depend on three factors
• Whether the code– Is executed – C– Fails during execution – f
• Coupling between coverage and faults - p
N0 – N(n) = F(f, C(n, p))
C(n) = 1 – 1/(1 + knp)
![Page 40: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/40.jpg)
Bishop’s coverage model – 3
Based on the assumptions and expression previously presented , we find that
If we use the expression on the previous slide to eliminate C(n) we get
)1)]((1[
)(11
)(
0
0pp fnCf
nC
N
nNN
1)(
11
)(
0
0
pnfkN
nNN
![Page 41: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/41.jpg)
A limit resultIt is possible to show that the following
relation holds under a rather wide set of conditions:
The initial number of defects – N0 – must be estimated e.g. based on experience from earlier projects as number of defects per KLOC.
tN
eMTTFt
0ˆ
![Page 42: Path selection criteria Tor Stålhane & ‘Wande Daramola.](https://reader034.fdocuments.in/reader034/viewer/2022052308/56649e4d5503460f94b438e7/html5/thumbnails/42.jpg)
An example from telecom