From resilient to antifragile - Chaos Engineering Primer DevSecCon
Antifragile Software and Genetic Improvement Martin...
Transcript of Antifragile Software and Genetic Improvement Martin...
![Page 1: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/1.jpg)
1Martin Monperrus, University of Lille & Inria, France
Martin MonperrusUniversity of Lille & Inria, France
Antifragile Software and Genetic Improvement
CREST Open Workshop on Genetic ImprovementJan 2016
![Page 2: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/2.jpg)
![Page 3: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/3.jpg)
3Martin Monperrus, University of Lille & Inria, France
Exception Handling Analysis and Transformation Using Fault Injection: Study of Resilience Against Unanticipated Exceptions (IST 2014)
try { prepare_meringue() add_cream() make_balls() add_chocolate_shavings()} catch (MissingChocolateEx e) { use_nutella();}
« Anticipated errors »
![Page 4: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/4.jpg)
4Martin Monperrus, University of Lille & Inria, France
● Test colors
– Pink– Blue- White
Expected errors in test suites
Blue and white tests specify anticipated errorsWhite tests specify resilience
![Page 5: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/5.jpg)
5Martin Monperrus, University of Lille & Inria, France
Empirical results
3950 2046 2031 481 406 339 191 135 1000%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
white
blue
pink
![Page 6: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/6.jpg)
6Martin Monperrus, University of Lille & Inria, France
Our intuition: simulating
unforeseen errors by perturbing test case execution
![Page 7: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/7.jpg)
7Martin Monperrus, University of Lille & Inria, France
Why ?
![Page 8: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/8.jpg)
8Martin Monperrus, University of Lille & Inria, France
Short circuit testing: Exception injection in test suites
+ =White test throw new X()
![Page 9: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/9.jpg)
9Martin Monperrus, University of Lille & Inria, France
The try-catch blocks that keep the tests passing under exception injection are capable to handle unanticipated errors.
We call them ''source-independent''.
Source-Independence
We call them ''source-independent''.
Empirical assessment of resilience against unforeseen errors
![Page 10: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/10.jpg)
10Martin Monperrus, University of Lille & Inria, France
shindig-java-gadgetsCommons-lang
commons-codecsonar core
shindig-commonjbehave core
joda timesonar plugin api
spojo core
0
5
10
15
20
25
30
35
40
45
non agnostique
agnostique
Empirical Evaluation
dependent independent
![Page 11: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/11.jpg)
11Martin Monperrus, University of Lille & Inria, France
Test driver
try{ // instrumentation code if(Controller.isCurrentTryCatchWithInjection()) throw new Exception(); }
... //normal try body...} catch ( Exception e ) {... //normal catch body}
IO
IO
// Genetic Improvement
![Page 12: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/12.jpg)
12Martin Monperrus, University of Lille & Inria, France
Limit cases
Stack of try-catch at runtime
// originalCatch (OufofMemory) { Catch (NullPointer) { Catch (IOException) { } }}
// stretchedCatch (OufofMemory) { Catch (NullPointer) { Catch (Exception) { } }}
throw new NullPointerException()
![Page 13: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/13.jpg)
13Martin Monperrus, University of Lille & Inria, France
Empirical Evaluation
dependent independent
commons-lang
commons-codec
joda time
spojo core
sonar core
sonar plugin
jbehave core
shindig-java-gadgets
shindig-common
0 5 10 15 20 25 30 35
unstretchable
stretchable
![Page 14: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/14.jpg)
14Martin Monperrus, University of Lille & Inria, France
Take-away
“We inject exceptions during test suite execution to assess and improve
resilience”References:
Exception Handling Analysis and Transformation Using Fault Injection: Study of Resilience Against Unanticipated Exceptions (Benoit Cornu, Lionel Seinturier, Martin Monperrus), In Information and Software Technology, Elsevier, 2014.
![Page 15: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/15.jpg)
15Martin Monperrus, University of Lille & Inria, France
Definition
A thing is antifragile if it becomes
better with stressors, shocks, volatility, noise, mistakes, faults, attacks, failures . . .(Antifragile, N. N. Taleb, 2013)
![Page 16: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/16.jpg)
16Martin Monperrus, University of Lille & Inria, France
A software system is antifragile if it becomes
better with failures, attacks, failures, misconfigurations, weird usages . . .
(Principles of Antifragile Software, M. Monperrus, 2014)
![Page 17: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/17.jpg)
17Martin Monperrus, University of Lille & Inria, France
Short circuit testing is antifragile:● Better with exception injection
●Does not try to anticipate or predict
![Page 18: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/18.jpg)
18Martin Monperrus, University of Lille & Inria, France
Chaos Monkey
// Chaos Engineering// principlesofchaos.org, 2015
hypothesis ← perturbation, measurewhile (true) { perturb if (measure unacceptable) { report failure }}
![Page 19: Antifragile Software and Genetic Improvement Martin ...crest.cs.ucl.ac.uk/cow/45/slides/cow45_Monperrus.pdf · Martin Monperrus, University of Lille & Inria, France 16 A software](https://reader033.fdocuments.in/reader033/viewer/2022042402/5f11395e57dfbb0325744050/html5/thumbnails/19.jpg)
19Martin Monperrus, University of Lille & Inria, France
Antifragile software engineering
Core model:● perturbation model● perturbation controller● perturbation cost● perturbation gain
http://www.monperrus.net/martin/antifragile-software