Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating...
Transcript of Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating...
![Page 1: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/1.jpg)
Efficient and Effective Mutation Testing:Supporting the Implementation of Quality
Software by Purposefully Inserting Defects
Gregory M. Kapfhammer†
Department of Computer ScienceAllegheny College
http://www.cs.allegheny.edu/∼gkapfham/
University of Delhi – May 2, 2012†Joint with Rene Just and Franz Schweiggert (University of Ulm) and Jonathan Miller Kauffman (Allegheny College)
![Page 2: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/2.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Important Points
Presenter Introduction: Gregory M. Kapfhammer
test
testing
softwaresuites
prioritization
components
coverageem
piric
allyregressionsuiteapplications
evaluating
algo
rithm
analysis
data
effectiveness
empirical
genetic
perf
orm
ance
understanding
using
approachcommercialofftheshelf
comparison
comprehensive
constrainedcreation
databasecentric envi
ronm
ents
execution
findi
ng
framework
identifying
interactive
javamethods
multiplots
mutation
party
prioritized
reduction
relational
study
third
timeaware
towards
105
adequacy
appr
oach
es
array
automatically
building
callchallenges
chapter
communication
compare
compressing
com
pute
r
conditional
cost
cots
covering
creating
criteria
database
databaseaware
data
base
driv
en
databases
declarativedependable
detection
devices
distributed
distributing
duringsearchbased
dynamic
efficiency
efficient
engi
neer
ing
environment
evaluate
examination
executing
experimental
family
flow
forward
frameworks
free
generation
greedy
gui
hamiltonian
handbook
heaps
implementation
impr
ove
incorporating
incr
ease
information
initial
intranode
invariant
javaspace
javaspacebased
kernel
knapsack
linux
measurement
memory
method
monitoring
operators
paths
poster
potential
preliminary
primitives
prio
ritiz
atio
ns
prio
ritiz
ers
prioritizing
problematic
receive
remote
reports
resource
resourceconstrained
results
role
runtime
science
searchbased
selectionsolutions
solvers
space
studies
stud
yingsupported
synthetic
techniques
transmission
transparently
trees
tuple
unstructured
wrappers
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 3: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/3.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Important Points
Inspiration and Motivation
The magic of myth and legend has come true inour time. One types the correct incantation ona keyboard, and a display screen comes to life,showing things that never were nor could be.
Frederick P. Brooks, Jr.
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 4: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/4.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Important Points
Inspiration and Motivation
The magic of myth and legend has come true inour time. One types the correct incantation ona keyboard, and a display screen comes to life,showing things that never were nor could be.
Frederick P. Brooks, Jr.
In reference to software!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 5: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/5.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Important Points
Inspiration and Motivation
I believe the hard part of building software to bethe specification, design, and testing of this con-ceptual construct, not the labor of representingit and testing the fidelity of the representation.
Frederick P. Brooks, Jr.
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 6: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/6.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Important Points
Inspiration and Motivation
I believe the hard part of building software to bethe specification, design, and testing of this con-ceptual construct, not the labor of representingit and testing the fidelity of the representation.
Frederick P. Brooks, Jr.
What happens if the “incantation” is incorrect?
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 7: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/7.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Important Points
Inspiration and Motivation
I believe the hard part of building software to bethe specification, design, and testing of this con-ceptual construct, not the labor of representingit and testing the fidelity of the representation.
Frederick P. Brooks, Jr.
How do we efficiently and effectively test software?
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 8: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/8.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 9: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/9.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
Input
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 10: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/10.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
Input Output
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 11: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/11.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 12: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/12.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 13: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/13.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 14: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/14.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 15: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/15.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 16: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/16.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
ExpectedOutput
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 17: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/17.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
ExpectedOutput
TestVerdict
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 18: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/18.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
ExpectedOutput
TestVerdict
ExpectedOutput
Output
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 19: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/19.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
ExpectedOutput
TestVerdict
ExpectedOutput
Output
TestVerdict
The test case passes and the code is correct!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 20: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/20.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
ExpectedOutput
TestVerdict
ExpectedOutput
Output
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 21: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/21.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Case?
MethodUnder Test
TestSet Up
Input Output
TestClean Up
TestOracle
ExpectedOutput
TestVerdict
ExpectedOutput
Output
TestVerdict
The test case fails and a defect is found!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 22: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/22.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 23: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/23.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 24: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/24.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 25: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/25.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 26: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/26.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 27: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/27.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 28: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/28.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 29: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/29.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 30: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/30.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 31: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/31.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 32: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/32.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 33: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/33.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 34: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/34.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 35: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/35.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
B2B2B2B2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 36: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/36.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
B2B2B2B2B2B2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 37: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/37.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
B2B2B2B2B2B2B2B2B2B2
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 38: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/38.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 39: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/39.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
How Good is Test Suite T ?
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 40: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/40.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
How Good is Test Suite T ?
Coverage Analysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 41: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/41.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Software Testing
What is a Test Suite?
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
Test Suite T = 〈T1,T2, . . . ,T9,T10〉
R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2
Requirements R = {R1, . . . ,R6}, Features F = {F1, . . . ,F4}, Bug Fixes B = {B1,B2}
How Good is Test Suite T ?
Coverage Analysis Mutation Analysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 42: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/42.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10)
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 43: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/43.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 44: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/44.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
Implemented
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 45: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/45.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
Implemented Potential Fault
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 46: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/46.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 47: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/47.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 48: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/48.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 0
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 49: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/49.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 0
false false
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 50: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/50.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 15
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 51: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/51.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 15
true true
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 52: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/52.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 10
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 53: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/53.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 10
false true
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 54: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/54.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 10
false true
Can the tests differentiate between implemented and potential fault?
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 55: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/55.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 10
false true
If yes, then the tests are adequate!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 56: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/56.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 10
false true
If no, then the tests must be improved!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 57: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/57.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Conceptual Faults
if(a > 10) if(a >= 10)
a
true false
a
true false
a = 10
false true
Purposefully insert faults in order to implement quality software!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 58: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/58.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 59: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/59.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 60: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/60.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 61: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/61.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 62: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/62.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Methodicallyinject smallsyntacticalfaults into
the programunder test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 63: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/63.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Methodicallyinject smallsyntacticalfaults into
the programunder test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 64: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/64.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Methodicallyinject smallsyntacticalfaults into
the programunder test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 65: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/65.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Methodicallyinject smallsyntacticalfaults into
the programunder test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 66: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/66.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
MutationOperator
MutationOperator
MutationOperator
MutationOperator
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 67: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/67.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 68: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/68.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 69: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/69.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 70: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/70.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 71: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/71.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 72: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/72.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 73: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/73.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 74: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/74.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 75: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/75.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 76: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/76.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
The test suitecannot kill themutant – either
a test suiteweakness oran equivalent
mutant!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 77: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/77.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Overview of Mutation Analysis
Test Case T1 Test Case T2 Test Case T3 Test Case T4
Execute thetest suite after
enabling asingle mutantin the program
under test
Repeat thisprocess forall of the
test casesand mutants– calculate
mutation scorewhen finished
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 78: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/78.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 79: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/79.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 80: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/80.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 81: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/81.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Conditional Mutation
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 82: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/82.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Conditional Mutation
Syntax TreeTransformation
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 83: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/83.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Conditional Mutation
Syntax TreeTransformation
Expressionsand Statements
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 84: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/84.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Conditional Mutation
Syntax TreeTransformation
Expressionsand Statements
CompilerIntegrated
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 85: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/85.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Conditional Mutation
Syntax TreeTransformation
Expressionsand Statements
CompilerIntegrated
ComprehensiveEmpirical Study
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 86: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/86.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Contributions of this Presentation
EfficientMutationAnalysis
Challenges
Solutions
Conditional Mutation
Syntax TreeTransformation
Expressionsand Statements
CompilerIntegrated
ComprehensiveEmpirical Study
Efficient Technique - Fully Integrated into the Java 6 SE Compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 87: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/87.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Understanding Mutation Analysispublic int eval(int x){
int a=3, b=1, y;
y = a * x;
y += b;return y;
}
public int max(int a, int b){int max = a;
if(b>a){
max=b;}
return max;}
=⇒
=⇒
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 88: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/88.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Understanding Mutation Analysispublic int eval(int x){
int a=3, b=1, y;
y = a * x;
y += b;return y;
}
public int max(int a, int b){int max = a;
if(b>a){
max=b;}
return max;}
=⇒
=⇒
Methodicallyinject smallsyntacticalfaults into
the programunder test
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 89: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/89.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Understanding Mutation Analysispublic int eval(int x){
int a=3, b=1, y;
y = a * x;
y += b;return y;
}
public int max(int a, int b){int max = a;
if(b>a){
max=b;}
return max;}
=⇒
=⇒
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 90: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/90.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Understanding Mutation Analysispublic int eval(int x){
int a=3, b=1, y;
y = a * x;
y += b;return y;
}
public int max(int a, int b){int max = a;
if(b>a){
max=b;}
return max;}
=⇒
=⇒
• y = a - x;
• y = a + x;
• y = a / x;
• if(b < a)
• if(b != a)
• if(b == a)
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 91: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/91.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Understanding Mutation Analysispublic int eval(int x){
int a=3, b=1, y;
y = a * x;
y += b;return y;
}
public int max(int a, int b){int max = a;
if(b>a){
max=b;}
return max;}
=⇒
=⇒
Unbiasedand powerfulmethod forassessing
oracles andinput values
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 92: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/92.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Understanding Mutation Analysispublic int eval(int x){
int a=3, b=1, y;
y = a * x;
y += b;return y;
}
public int max(int a, int b){int max = a;
if(b>a){
max=b;}
return max;}
=⇒
=⇒
Unbiasedand powerfulmethod forassessing
oracles andinput values
Useful methodfor fault seeding
during theempirical study
of testingtechniques
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 93: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/93.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 94: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/94.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 95: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/95.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 96: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/96.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 97: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/97.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 98: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/98.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
High Time Over-head for Generation
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 99: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/99.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
High Time Over-head for Generation
MutationAnalysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 100: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/100.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
High Time Over-head for Generation
MutationAnalysisTests
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 101: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/101.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
High Time Over-head for Generation
MutationAnalysisTests Results
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 102: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/102.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
High Time Over-head for Generation
MutationAnalysisTests Results
Individually Executing theMutants is Too Expensive
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 103: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/103.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Mutation Analysis Challenges
MutantGeneration
MutationOperators
Program
Mutants
Often Yields aSubstantial Num-
ber of Mutants
High Time Over-head for Generation
MutationAnalysisTests Results
Individually Executing theMutants is Too Expensive
PriorSolutions?
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 104: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/104.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 105: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/105.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 106: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/106.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 107: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/107.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer
Sampling Selection
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 108: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/108.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer Do Smarter
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 109: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/109.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer Do Smarter
Distributed Weak Mutation
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 110: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/110.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer Do Smarter
Do Faster
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 111: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/111.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer Do Smarter
Do Faster
CompilerIntegrated
BytecodeTransformation
MutantSchemata
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 112: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/112.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Fundamental Concepts
Prior Work in Mutation Analysis
Improving Mutation Analysis Offutt andUntch
Do Fewer Do Smarter
Do Faster
Higher OrderMutation
Jia andHarman
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 113: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/113.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Conditional Mutation
Conditional Mutation
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 114: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/114.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Conditional Mutation
Conditional Mutation
Encapsulates allmutants withinthe same block
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 115: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/115.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Conditional Mutation
Conditional Mutation
Encapsulates allmutants withinthe same block
Can be inte-grated withinthe compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 116: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/116.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Conditional Mutation
Conditional Mutation
Encapsulates allmutants withinthe same block
Transforms theabstract syntax
tree (AST)
Can be inte-grated withinthe compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 117: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/117.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Conditional Mutation
Conditional Mutation
Encapsulates allmutants withinthe same block
Transforms theabstract syntax
tree (AST)
Stmt → Conditional Stmt(if-then-else, switch)
Expr → Conditional Expr(conditional operator ?:)
Can be inte-grated withinthe compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 118: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/118.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Transforming the AST
public int eval(int x){int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
⇓
=⇒
=⇒
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 119: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/119.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Transforming the AST
public int eval(int x){int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
⇓
ASSIGN
IDENT
y
BINARY
∗
a x
=⇒
=⇒
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 120: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/120.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Transforming the AST
public int eval(int x){int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
⇓
ASSIGN
IDENT
y
BINARY
∗
a x
=⇒
=⇒
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 121: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/121.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Transforming the AST
public int eval(int x){int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
⇓
ASSIGN
IDENT
y
BINARY
∗
a x
=⇒
=⇒
ASSIGN
IDENT
y
COND-EXPR
THEN
BINARY
+
a x
COND
(M NO ==2)
ELSE
COND-EXPR
THEN
BINARY
-
a x
COND
(M NO ==1)
ELSE
BINARY
∗
a x
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 122: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/122.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 123: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/123.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 124: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/124.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = a * x ;
y += b;return y;
}
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 125: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/125.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==1)? a - x :a * x ;
y += b;return y;
}
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 126: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/126.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==2)? a + x :(M_NO==1)? a - x :
a * x ;
y += b;return y;
}
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 127: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/127.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==3)? a / x :(M_NO==2)? a + x :(M_NO==1)? a - x :
a * x ;
y += b;return y;
}
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 128: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/128.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Source Code View of Inserting Mutantspublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==3)? a / x :(M_NO==2)? a + x :(M_NO==1)? a - x :
a * x ;
y += b;return y;
}
Mutants that are not ex-ecuted cannot be killed
1 Define mutation operators MOP(x ∗ y) = {x − y , x + y , x/y}
2 Determine whether current expression or statement isaffected by mutation
3 Apply mutation operators
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 129: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/129.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Collecting and Using Mutation Coveragepublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==3)? a / x :(M_NO==2)? a + x :(M_NO==1)? a - x :
a * x ;
y += b;return y;
}
Mutants that are not ex-ecuted cannot be killed
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 130: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/130.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Collecting and Using Mutation Coveragepublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==3)? a / x :(M_NO==2)? a + x :(M_NO==1)? a - x :
(M_NO==0 &&
COVERED(1,3))?
a * x : a * x ;
y += b;
return y;}
Mutants that are not ex-ecuted cannot be killed
Determine coveredmutants with addi-
tional instrumentation
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 131: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/131.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Collecting and Using Mutation Coveragepublic int eval(int x){
int a=3, b=1, y;
y = (M_NO==3)? a / x :(M_NO==2)? a + x :(M_NO==1)? a - x :
(M_NO==0 &&
COVERED(1,3))?
a * x : a * x ;
y += b;
return y;}
Mutants that are not ex-ecuted cannot be killed
Determine coveredmutants with addi-
tional instrumentation
Only execute and investi-gate the covered mutants
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 132: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/132.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Compiler
MAJOR’sCompiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 133: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/133.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Compiler
MAJOR’sCompiler
Enhanced the StandardJava Compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 134: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/134.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Compiler
MAJOR’sCompiler
Enhanced the StandardJava Compiler
Source Files
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 135: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/135.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Compiler
MAJOR’sCompiler
Enhanced the StandardJava Compiler
Source Files
CommonCompiler Options
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 136: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/136.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Compiler
MAJOR’sCompiler
Enhanced the StandardJava Compiler
Source Files
CommonCompiler Options
Domain SpecificLanguage
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 137: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/137.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Compiler
MAJOR’sCompiler
Enhanced the StandardJava Compiler
Source Files
CommonCompiler Options
Domain SpecificLanguage
Bytecode withEmbedded
Mutants
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 138: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/138.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Integration into the Java Compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 139: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/139.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Integration into the Java Compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 140: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/140.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Integration into the Java Compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 141: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/141.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Integration into the Java Compiler
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 142: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/142.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Domain Specific Language// variable declarationlistCOR={&&, ||, ==, !=};
// Define replacement list
BIN(+)<"org"> -> {-,*};
BIN(*)<"org"> -> {/,%};
// Define own operator
myOp{
BIN(&&) -> listCOR;
BIN(||) -> listCOR;
COR;
LVR;
}
// Enable built-in operator AOR
AOR<"org">;
// Enable operator myOp
myOp<"java.lang.System@println">;
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 143: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/143.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Domain Specific Language// variable declarationlistCOR={&&, ||, ==, !=};
// Define replacement list
BIN(+)<"org"> -> {-,*};
BIN(*)<"org"> -> {/,%};
// Define own operator
myOp{
BIN(&&) -> listCOR;
BIN(||) -> listCOR;
COR;
LVR;
}
// Enable built-in operator AOR
AOR<"org">;
// Enable operator myOp
myOp<"java.lang.System@println">;
Specify mutationoperators in detail
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 144: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/144.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Domain Specific Language// variable declarationlistCOR={&&, ||, ==, !=};
// Define replacement list
BIN(+)<"org"> -> {-,*};
BIN(*)<"org"> -> {/,%};
// Define own operator
myOp{
BIN(&&) -> listCOR;
BIN(||) -> listCOR;
COR;
LVR;
}
// Enable built-in operator AOR
AOR<"org">;
// Enable operator myOp
myOp<"java.lang.System@println">;
Specify mutationoperators in detail
Define own mutationoperator groups
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 145: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/145.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
MAJOR’s Domain Specific Language// variable declarationlistCOR={&&, ||, ==, !=};
// Define replacement list
BIN(+)<"org"> -> {-,*};
BIN(*)<"org"> -> {/,%};
// Define own operator
myOp{
BIN(&&) -> listCOR;
BIN(||) -> listCOR;
COR;
LVR;
}
// Enable built-in operator AOR
AOR<"org">;
// Enable operator myOp
myOp<"java.lang.System@println">;
Specify mutationoperators in detail
Define own mutationoperator groups
Enable operators fora specific package,
class, or method
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 146: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/146.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Optimized Mutation Analysis Process
1 Embed and compile all mutants2 Run test suite on instrumented program3 Sort tests according to their runtime4 Perform mutation analysis with reordered test suite
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 147: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/147.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Optimized Mutation Analysis Process
1 Embed and compile all mutants2 Run test suite on instrumented program3 Sort tests according to their runtime4 Perform mutation analysis with reordered test suite
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 148: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/148.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Optimized Mutation Analysis Process
1 Embed and compile all mutants2 Run test suite on instrumented program3 Sort tests according to their runtime4 Perform mutation analysis with reordered test suite
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 149: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/149.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis with MAJOR
Optimized Mutation Analysis Process
1 Embed and compile all mutants2 Run test suite on instrumented program3 Sort tests according to their runtime4 Perform mutation analysis with reordered test suite
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 150: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/150.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Mutant Generation and Compilation
1
2
3
4
5
6
7
8
9
10
11
12
0 20000 40000 60000 80000 100000 120000 140000
Co
mp
iler
run
tim
e in
se
co
nd
s
Number of mutants
apache antjfreechart
itextjava pathfindercommons mathcommons lang
numerics4j
Overhead for generating and compiling mutants is negligible
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 151: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/151.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Mutant Generation and Compilation
1
2
3
4
5
6
7
8
9
10
11
12
0 20000 40000 60000 80000 100000 120000 140000
Co
mp
iler
run
tim
e in
se
co
nd
s
Number of mutants
apache antjfreechart
itextjava pathfindercommons mathcommons lang
numerics4j
Overhead for generating and compiling mutants is negligible
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 152: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/152.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Time and Space OverheadApplication Mutants Runtime of test suite Memory consumption
original instrumented original instrumentedwcs wcs+cov
aspectj 406,382 4.3 4.8 5.0 559 813apache ant 60,258 331.0 335.0 346.0 237 293jfreechart 68,782 15.0 18.0 23.0 220 303
itext 124,184 5.1 5.6 6.3 217 325java pathfinder 37,331 17.0 22.0 29.0 182 217commons math 67,895 67.0 83.0 98.0 153 225commons lang 25,783 10.3 11.8 14.8 104 149
numerics4j 5,869 1.2 1.3 1.6 73 90
• Runtime overhead is application dependent• Larger for CPU-bound applications
• Small for I/O-bound applications
• Even for large projects, applicable on commodity workstations
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 153: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/153.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Time and Space OverheadApplication Mutants Runtime of test suite Memory consumption
original instrumented original instrumentedwcs wcs+cov
aspectj 406,382 4.3 4.8 5.0 559 813apache ant 60,258 331.0 335.0 346.0 237 293jfreechart 68,782 15.0 18.0 23.0 220 303
itext 124,184 5.1 5.6 6.3 217 325java pathfinder 37,331 17.0 22.0 29.0 182 217commons math 67,895 67.0 83.0 98.0 153 225commons lang 25,783 10.3 11.8 14.8 104 149
numerics4j 5,869 1.2 1.3 1.6 73 90
• Runtime overhead is application dependent• Larger for CPU-bound applications
• Small for I/O-bound applications
• Even for large projects, applicable on commodity workstations
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 154: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/154.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Time and Space OverheadApplication Mutants Runtime of test suite Memory consumption
original instrumented original instrumentedwcs wcs+cov
aspectj 406,382 4.3 4.8 5.0 559 813apache ant 60,258 331.0 335.0 346.0 237 293jfreechart 68,782 15.0 18.0 23.0 220 303
itext 124,184 5.1 5.6 6.3 217 325java pathfinder 37,331 17.0 22.0 29.0 182 217commons math 67,895 67.0 83.0 98.0 153 225commons lang 25,783 10.3 11.8 14.8 104 149
numerics4j 5,869 1.2 1.3 1.6 73 90
• Runtime overhead is application dependent• Larger for CPU-bound applications
• Small for I/O-bound applications
• Even for large projects, applicable on commodity workstations
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 155: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/155.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Time and Space OverheadApplication Mutants Runtime of test suite Memory consumption
original instrumented original instrumentedwcs wcs+cov
aspectj 406,382 4.3 4.8 5.0 559 813apache ant 60,258 331.0 335.0 346.0 237 293jfreechart 68,782 15.0 18.0 23.0 220 303
itext 124,184 5.1 5.6 6.3 217 325java pathfinder 37,331 17.0 22.0 29.0 182 217commons math 67,895 67.0 83.0 98.0 153 225commons lang 25,783 10.3 11.8 14.8 104 149
numerics4j 5,869 1.2 1.3 1.6 73 90
• Runtime overhead is application dependent• Larger for CPU-bound applications
• Small for I/O-bound applications
• Even for large projects, applicable on commodity workstations
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 156: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/156.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Compilation Efficiency
Time and Space OverheadApplication Mutants Runtime of test suite Memory consumption
original instrumented original instrumentedwcs wcs+cov
aspectj 406,382 4.3 4.8 5.0 559 813apache ant 60,258 331.0 335.0 346.0 237 293jfreechart 68,782 15.0 18.0 23.0 220 303
itext 124,184 5.1 5.6 6.3 217 325java pathfinder 37,331 17.0 22.0 29.0 182 217commons math 67,895 67.0 83.0 98.0 153 225commons lang 25,783 10.3 11.8 14.8 104 149
numerics4j 5,869 1.2 1.3 1.6 73 90
• Runtime overhead is application dependent• Larger for CPU-bound applications
• Small for I/O-bound applications
• Even for large projects, applicable on commodity workstations
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 157: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/157.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis Efficiency
Evaluating and Improving Mutation Analysis
0
20
40
60
80
100
120
140
160
180
0 20 40 60 80 100
Nu
mb
er
of
kill
ed
mu
tan
ts
Runtime in seconds
optimized order (using coverage information)
random order (using coverage information)
original order (using coverage information)
optimized order (without coverage information)
random order (without coverage information)
original order (without coverage information)
• Mutation analysis is not feasible without coverage information• Reordering the test suite significantly speeds up the process,
especially if runtimes of tests differ by orders of magnitudeKapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 158: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/158.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis Efficiency
Evaluating and Improving Mutation Analysis
0
20
40
60
80
100
120
140
160
180
0 20 40 60 80 100
Nu
mb
er
of
kill
ed
mu
tan
ts
Runtime in seconds
optimized order (using coverage information)
random order (using coverage information)
original order (using coverage information)
optimized order (without coverage information)
random order (without coverage information)
original order (without coverage information)
• Mutation analysis is not feasible without coverage information• Reordering the test suite significantly speeds up the process,
especially if runtimes of tests differ by orders of magnitudeKapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 159: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/159.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Mutation Analysis Efficiency
Evaluating and Improving Mutation Analysis
0
20
40
60
80
100
120
140
160
180
0 20 40 60 80 100
Nu
mb
er
of
kill
ed
mu
tan
ts
Runtime in seconds
optimized order (using coverage information)
random order (using coverage information)
original order (using coverage information)
optimized order (without coverage information)
random order (without coverage information)
original order (without coverage information)
• Mutation analysis is not feasible without coverage information• Reordering the test suite significantly speeds up the process,
especially if runtimes of tests differ by orders of magnitudeKapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 160: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/160.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 161: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/161.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Program Test Suite
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 162: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/162.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Program Test Suite
Mutation Score
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 163: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/163.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Program Test Suite
Mutation Score
Improve Tests
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 164: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/164.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Program Test Suite
Mutation Score
Improve Tests
Automated Manual
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 165: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/165.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Program Test Suite
Mutation Score
Improve Tests Use Tests
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 166: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/166.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Improving Test Suite Quality
MutationAnalysis
Program Test Suite
Mutation Score
Improve Tests Use Tests
Test improvement is only effective if mutation analysis is efficient!
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 167: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/167.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Reviewing MAJOR’s Contributions
MutationAnalysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 168: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/168.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Reviewing MAJOR’s Contributions
MutationAnalysis
Efficiency: MAJOR has ac-ceptable time and space over-heads and scales to large,real-world programs
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 169: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/169.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Reviewing MAJOR’s Contributions
MutationAnalysis
Efficiency: MAJOR has ac-ceptable time and space over-heads and scales to large,real-world programs
Usability: MAJOR’s inte-gration into the Java SEcompiler makes it a no-hassle, drop-in tool
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 170: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/170.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Retrospective
Reviewing MAJOR’s Contributions
MutationAnalysis
Efficiency: MAJOR has ac-ceptable time and space over-heads and scales to large,real-world programs
Usability: MAJOR’s inte-gration into the Java SEcompiler makes it a no-hassle, drop-in tool
We will release MAJOR as free and open source software
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 171: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/171.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Conclusions and Future Work
Conclusion
Key Concepts and Features:
• Compiler-integrated solution• Conditional mutation with the abstract syntax tree• Furnishes its own domain specific language• Collects and leverages mutation coverage information
Characteristics of MAJOR:
• Fast and scalable technique• Configurable and extensible mutation tool• Enables an optimized workflow for mutation analysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 172: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/172.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Conclusions and Future Work
Conclusion
Key Concepts and Features:
• Compiler-integrated solution• Conditional mutation with the abstract syntax tree• Furnishes its own domain specific language• Collects and leverages mutation coverage information
Characteristics of MAJOR:
• Fast and scalable technique• Configurable and extensible mutation tool• Enables an optimized workflow for mutation analysis
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 173: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/173.jpg)
Introduction Mutation Analysis Empirical Evaluation Conclusion
Conclusions and Future Work
Recently Published Papers
• Rene Just, Gregory M. Kapfhammer, and FranzSchweiggert. Using conditional mutation to increase theefficiency of mutation analysis. In Proceedings of the 6thInternational Workshop on the Automation of Software Test,Honolulu, Hawaii, May 2011.
• Rene Just, Franz Schweiggert, and Gregory M.Kapfhammer. MAJOR: An efficient and extensible tool formutation analysis in a Java compiler. In Proceedings of the26th IEEE/ACM International Conference on AutomatedSoftware Engineering (Tool Paper), Lawrence, Kansas,November 2011.
Kapfhammer Allegheny College
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects
![Page 174: Efficient and Effective Mutation Testing: Supporting the ...€¦ · cost cots covering creating criteria database databaseaware databasedriven ... Software Testing What is a Test](https://reader036.fdocuments.in/reader036/viewer/2022071213/6028c47dbbad51624934b263/html5/thumbnails/174.jpg)
Efficient and Effective Mutation Testing:Supporting the Implementation of Quality
Software by Purposefully Inserting Defects
Gregory M. Kapfhammer
Department of Computer ScienceAllegheny College
http://www.cs.allegheny.edu/∼gkapfham/
Thank you for your attention!I welcome your questions and comments.