1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.
-
Upload
augustus-tate -
Category
Documents
-
view
214 -
download
1
Transcript of 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.
![Page 1: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/1.jpg)
1
Software Testing & Quality Assurance
Lecture 14
Created by: Paulo AlencarModified by: Frank Xu
![Page 2: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/2.jpg)
2
OverviewStructural Testing
Introduction – General ConceptsFlow Graph TestingData Flow Testing
DefinitionsSome Basic Data Flow Analysis AlgorithmsDefine/use TestingSlice Based TestingGuidelines and Observations
Hybrid MethodsRetrospective on Structural Testing
![Page 3: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/3.jpg)
3
Issues on Structural Testing
• An important question relates to the point where we stop to test and release the system for beta testing. Some possible answers could be: – When all faults have been removed!– When we run out of time– When continued testing produces no new failures– When continued testing reveals no new faults– When we have exhausted the testing criteria and the test cases we
have designed– When we reach a point of diminishing returns– When the mandated coverage has been achieved– When the failure intensity (failures / CPU hour) level drops below
a certain point
![Page 4: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/4.jpg)
4
Metrics for Testing Method Comparison (1)
• Functional testing methods may (eventually) provide a high level of path coverage, but at the cost of high redundancy
• The question is to define metrics that relate the effectiveness of a functional technique with that of a structural technique (path coverage)
• To start addressing this question we first observe that functional testing methods are measured according to the number of test cases they produce, while structural testing methods are measured according to the level of the corresponding path coverage they achieve.
![Page 5: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/5.jpg)
5
Metrics for Testing Method Comparison (2)
• Let’s assume a functional testing method M, and a structural metric S that identifies s coverage elements in the unit under test
• Furthermore, let us assume that the functional testing method M produces m test cases, and these m test cases traverse n of the s structural coverage elements. We then define the following:
– Coverage of methodology M with respect to metric S is defined as C(M,S) = n/s
– Redundancy of methodology M with respect to metric S is defined as R(M,S) = m/s
– Net redundancy of methodology M with respect to metric S is defined as NR(M,S) = m/n
![Page 6: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/6.jpg)
6
Sample Comparison of Functional Testing Methods with respect to a Structural Metric
Method m n s C(M,S) R(M,S) NR(M,S)
nominal 15 7 11 0.64 1.36 2.14
Worst case 125 11 11 1.00 11.36 11.36
Method m n s C(M,S) R(M,S) NR(M,S)
Output bva 25 11 11 1.00 2.27 2.27
Decision Table
3 11 11 1.00 0.27 0.27
S: Program Pathss: total number of pathsn: number of paths traversed using method Mm: number of test cases generated by method MM: Functional method
Ref: “Software testing: A Craftsman’s Approach”, P. Jorgensen
Triangle Problem
Commission Problem
![Page 7: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/7.jpg)
7
Sample Comparison of a Functional Testing Method (Output BVA) with respect to Structural Metrics (DD-Path, DU-Path,
Slices)
Method m n s C(M,S) R(M,S) NR(M,S)
DD-Path 25 11 11 1.00 2.27 2.27
DU-Path 25 33 33 1.00 0.76 0.76
Slices 25 40 40 1.00 0.63 0.63
S: Structural Metrics: total number of paths in structural metric Sn: number of paths traversed using method Mm: number of test cases generated by method MM: Functional method
Ref: “Software testing: A Craftsman’s Approach”, P. Jorgensen
Commission Problem
![Page 8: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/8.jpg)
8
Testing Sophistication vs. Number of Test Coverage Items
DD-Path Basis Path DU-Path Slice
low
high
Sophistication
Number of Test Coverage Items
Ref: “Software testing: A Craftsman’s Approach”, P. Jorgensen
![Page 9: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/9.jpg)
9
Test Coverage Items
DD-Path Basis Path DU-Path Slice
11
40
Number of Test Coverage Items
5
33
Ref: “Software testing: A Craftsman’s Approach”, P. Jorgensen
![Page 10: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/10.jpg)
10
Test Case Sophistication vs. Test Case Identification Effort
DD-Path Basis Path DU-Path Slice
low
high
Sophistication
Effort to Identify Test Coverage Items
Ref: “Software testing: A Craftsman’s Approach”, P. Jorgensen
![Page 11: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/11.jpg)
11
Remarks
• Functional testing is prone to gaps and redundancies
• Structural path-coverage testing considers infeasible paths
• Data flow testing considers program semantics on Control Flow Graph paths
• Hybrid approaches that combine functional-structural methods are useful
• Structural testing techniques are best suited for unit testing
![Page 12: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/12.jpg)
12
Subsumption
• A test criterion C1 subsumes a test criterion C2 if every test suite that satisfies C1 also satisfies C2
• Two test criteria C1 and C2 are equivalent if C1 subsumes C2 and vice-versa
• What is the subsumption relation among coverage criteria?
![Page 13: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/13.jpg)
13
Subsumption Hierarchy – Control Flow
![Page 14: 1 Software Testing & Quality Assurance Lecture 14 Created by: Paulo Alencar Modified by: Frank Xu.](https://reader036.fdocuments.in/reader036/viewer/2022082818/56649ed25503460f94be206a/html5/thumbnails/14.jpg)
14
Subsumption Hierarchy – Data Flow