ICSE 2017 - Evocrash
-
Upload
annibale-panichella -
Category
Presentations & Public Speaking
-
view
90 -
download
0
Transcript of ICSE 2017 - Evocrash
![Page 1: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/1.jpg)
A Guided Genetic Algorithm for Automated Crash Reproduction
MozhanSoltani AnnibalePanichella Arie vanDeursen
![Page 2: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/2.jpg)
Software systems fail!
2
![Page 3: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/3.jpg)
When Debugging, Crash Reproducible Tests are Required!
3
!
![Page 4: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/4.jpg)
4
STAR
Automated Crash Reproduction
JCHARMING MuCrash
Nayrolles etal.,J.Softw.Evol.andProc.,2016
Chen,andKim,TSE2015 Xuanetal.,ESEC/FSE2015
![Page 5: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/5.jpg)
5
STARJCHARMING MuCrash
Automated Crash Reproduction
ASampleStackTrace
![Page 6: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/6.jpg)
6
Automated Crash Reproduction via JCHARMING
JCHARMINGApproach:1) Deriveapartialmodelofthesystem2) Identifythestatementsthattriggerthecrash
Limitation:Requiresexpensivecomputation!
Nayrolles etal.,J.Softw.Evol.andProc.,2016
![Page 7: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/7.jpg)
7
Automated Crash Reproduction via STAR
Approach:1) Computethecrashpreconditions,usingBSE2) Generateatestthatsatisfiesthepreconditions
Limitation:Maynotscaledueto:• Pathexplosion• LimitationsofSMTsolvers
STAR
Chen,andKim,TSE2015
![Page 8: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/8.jpg)
8
Automated Crash Reproduction via MuCrash
Approach:Mutateexistingtestsuntilonereproducesthecrash
Limitation:Lackofeffectivemutationoperators
MuCrash
Xuanetal.,ESEC/FSE2015
![Page 9: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/9.jpg)
9
Crash Reproduction via Search-Based Software Testing!
![Page 10: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/10.jpg)
10
ASampleStackTraceSearchproblem:Findingacrashreproducible unittest!
Metaheuristicoptimization:AGuided GeneticAlgorithm
LeveragingEVSUITE
Crash Reproduction via EvoCrash!
![Page 11: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/11.jpg)
Guided Genetic Algorithm - GGA
11
EvaluateFitness
Selection GuidedCrossover
InitializePopulation
F=0|| T
GuidedMutation
CreatetheNextGeneration
Re-insertion
![Page 12: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/12.jpg)
Guided Genetic Algorithm - GGA
12
EvaluateFitness
Selection GuidedCrossover
InitializePopulation
F=0|| T
GuidedMutation
CreatetheNextGeneration
Re-insertion
Howisthisaguided geneticalgorithm?
![Page 13: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/13.jpg)
Guided Initialization
13
Eachgeneratedtestshallincludeatleastonecalltothetargetfailingmethod!
ASampleStackTrace
Thetargetfailingmethod
![Page 14: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/14.jpg)
Anidealtestcaseleadsto:1. Theexecutionofline1902. TheIllegalArgumentException beingthrown3. Similarstacktracebeinggenerated
14
Fitness Evaluation
ASampleStackTrace
Targetfailingmethod
![Page 15: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/15.jpg)
Guided Genetic Algorithm - GGA
15
EvaluateFitness
Selection Crossover
InitializePopulation
F=0|| T
Mutation
CreatetheNextGeneration
Whatifthereisnocrashreproducibletestinthepopulation?
![Page 16: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/16.jpg)
16
Single-Point Crossover
Parent#1
Parent#2
Offspring#1
Offspring#2
Thetargetcallislost!
![Page 17: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/17.jpg)
17
Parent#1
Parent#2
Offspring#1
Offspring#2
Thetargetcallislost!
What if the target call is lost?
Callincluded?GuidedCrossover
Taketheparents
Takethechildren
![Page 18: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/18.jpg)
18
Mutation
BeforeMutation
Assignanewobject/value
Removeastatement
Addastatement
![Page 19: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/19.jpg)
19
BeforeMutation
Assignanewstatement(constructor)
Removeastatement
Addastatement
What if the target call is lost?
Callincluded?GuidedMutation
MutationAccepted
![Page 20: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/20.jpg)
Is the test useful for debugging?
20
UsefulnessCriterion:
1
34
2
Ausefultestreveals thebug!
5
TheTargetCrashTraceAUsefulCrashReproducibleTest
Chen,N.,&Kim,S.(2015).Star:Stacktracebasedautomaticcrashreproductionviasymbolicexecution.IEEETransactionsonSoftwareEngineering.
![Page 21: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/21.jpg)
Guided Genetic Algorithm - GGA
21
EvaluateFitness
Selection Crossover
InitializePopulation
F=0|| T
Mutation
CreatetheNextGeneration
HowmanycrashescanEvoCrash successfully reproduce?
HowdoesEvoCrash performcomparedtothestate-of-the-artapproachestocrashreproduction?
![Page 22: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/22.jpg)
Empirical Study
Project NumberofCrashes
ApacheCommonsCollections 12
ApacheAnt 20
ApacheLog4j 18
22
ExceptionTypes
NullPointerException
IllegalArgumentException
ArrayIndexOutOfBoundsException
StringIndexOutOfBoundsException
UnsupportedOperationException
IllegalStateException
ConcurrentModificationException
![Page 23: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/23.jpg)
UsefulReproduction
Reproduced
How many crashes can EvoCrashsuccessfully reproduce?
230
5
10
15
20
ACC ANT LOG
• 82%werereproduced,and• ~80%ofthemwereuseful(inaverage)
Results:
20%
28%
34%
90% 71%
88%
![Page 24: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/24.jpg)
24
An Example: ACC-104
Reproductionrate:4/50Challenge: settingboundedFifoBuffer0intherightstate
ThetestbyEvoCrash forACC-104
ThecrashstacktraceforACC-104
![Page 25: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/25.jpg)
How does EvoCrash perform compared to the state-of-the-art approaches?
250
10
20
30
40
50
Total,JCHARMING,EvoCrash Total,STAR,EvoCrash Total,MuCrash,EvoCrash
EvoCrash out-performedthestate-of-the-art!
75% 100%8
50
58%
82%
1266% 83%
ReproductionrateforothertoolsReproductionrateforEvoCrash
Totalnumberofcases
![Page 26: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/26.jpg)
26
EvoCrash Addresses the Limitations!
BackwardSymbolicExecution:• Pathexplosion• LimitationsofSMT solvers
DirectedModelChecking:Expensivecomputations
TestCaseMutation:Limitedmutationoperators
![Page 27: ICSE 2017 - Evocrash](https://reader030.fdocuments.in/reader030/viewer/2022021508/5a6592a57f8b9a9f2f8b461b/html5/thumbnails/27.jpg)
EvoCrash is Available!
27
www.evocrash.org