Successful Unit testing for large applications
31
-
Upload
erik-lebel -
Category
Technology
-
view
855 -
download
2
Transcript of Successful Unit testing for large applications
© P
yxis
Te
chnolo
gie
s in
c.
PYXIS STUDIO
SETTING THE SCENE
© P
yxis
Te
chnolo
gie
s in
c.
You are trying to reduce frequency with
which regressions get pushed into
production
You are trying to speed up validation for
quicker release cycles
You are cool
You‟ve been told to do so by someone
higher up in your organization
IF YOU ARE USING AUTOMATED
TESTING, THEN…
© P
yxis
Te
chnolo
gie
s in
c.
Your project has tests, but they aren't
helping you deliver stable software
Your project has tests, but they‟re a
nightmare to maintain
Your project has tests, but they are a
haphazard mess in the making
You like tests
YOU ARE INTERESTED IN THIS
PRESENTATION BECAUSE…
© P
yxis
Te
chnolo
gie
s in
c.
We will be covering:
practices and concerns to help make
automated tests an enabling force
We are not discussing:
why we automate testing
© P
yxis
Te
chnolo
gie
s in
c.
SO, WHAT‟S THIS ABOUT DEBILITATING
TESTS?
© P
yxis
Te
chnolo
gie
s in
c.
Tests:
Are hard to understand
Break frequently
Are slow to execute
Are hard to fix
Let bugs through
Don‟t validate anything
useful
But we want them:
To be clear to
understand
To break legitimately
To be fast to execute
To be concise
THIS IS OFTEN BECAUSE…
MAKE TESTS
READABLE
© P
yxis
Te
chnolo
gie
s in
c.
Use OO practices to:
Make setup code easy to read
Unclutter
Use factory methods
Use fluent builders
Can build complex actions as well as
objects
CLEAN UP TEST SETUP
© P
yxis
Te
chnolo
gie
s in
c.
Simplify result validation
Can easily handle unordered cases
Write your own if your test framework
doesn‟t have any
COLLECTION ASSERTS
© P
yxis
Te
chnolo
gie
s in
c.
They are easier to read
USE LITERALS
© P
yxis
Te
chnolo
gie
s in
c.
Use extension methods for concise type
assertions
Build utility methods for complex
validations
BUILD INTUITIVE ASSERTIONS
© P
yxis
Te
chnolo
gie
s in
c.
Domain Specific Languages (DSLS)
Test API wrappers
Model wrappers
USE TEST DSLS FOR INTEGRATION
TESTS
© P
yxis
Te
chnolo
gie
s in
c.
Capture large expected value sets in
common file formats such as XML, JSON,
YAML
Write assertion utilities that highlight
multiple differences between expected and
actual values
(eg XmlUnit)
USE READABLE DATA FORMATS
© P
yxis
Te
chnolo
gie
s in
c.
You will more easily spot redundant or
useless tests
Tests should start to match business
requirements
Its should start to be clearer where there are
gaps
The factories, builders and helpers that you
develop for testing will make it easier to add
new tests scenarios
AS TESTS GET CLEANER
© P
yxis
Te
chnolo
gie
s in
c.
„If‟ statements in tests
Loops in tests
Exception handling
AT ALL COST, AVOID…
BALANCE
© P
yxis
Te
chnolo
gie
s in
c.
THE NUMBERS SHOW THAT…
System tests
Integration tests
Unit tests
Time to execute
Fragility
Lines of test code
Effort to fix
Overall cost of use
$$$
$
© P
yxis
Te
chnolo
gie
s in
c.
Module1.dll
Module1.Unit.Tests.dll
Module1.Integration.Tests.dll
Module2.dll
Module2.Unit.Tests.dll
Module2.Integration.Tests.dll
BUILD YOUR TESTS AROUND YOUR
MODULES
© P
yxis
Te
chnolo
gie
s in
c.
Configurations
Databases
File systems
External services
Sandboxing technologies
Flashy, shiny widgets
Use them… don‟t let it be the other way
around
TECHNOLOGIES THAT HINDER TESTING
© P
yxis
Te
chnolo
gie
s in
c.
Don‟t expect to stop delivering software while fixing things
Don‟t be afraid to experiment
When a pattern emerges, make sure it‟s communicated to the whole team
Don‟t get hung up on the “green field” vs “brown field” over time even the greenest fields get muddy
This is and will continue to be an organic process
GETTING THERE IN BABY STEPS
BUILDING TRUST
© P
yxis
Te
chnolo
gie
s in
c.
Commit to keeping tests green all the time.
If it‟s broken, fix it, or delete it.
DO OR DO NOT
© P
yxis
Te
chnolo
gie
s in
c.
Use a Continuous Integration tool:
TFS, CruiseControl.NET, Hudson/Jenkins,
roll your own
Don‟t allow the build to stay broken
Make sure your integration build and tests
are run as often as possible
Split long-running test suites to run as part
of an alternate set of build jobs
AUTOMATE TEST EXECUTION
© P
yxis
Te
chnolo
gie
s in
c.
Work to improve test coverage
Beware of high test coverage and low
software quality
And remember it‟s just an indicator
WATCH CODE COVERAGE
© P
yxis
Te
chnolo
gie
s in
c.
Monitor test and production failures
Use production issues to infer how deep
the iceberg is
When something fails, find the cause, and
improve your test and application design
INSPECT AND ADAPT
© P
yxis
Te
chnolo
gie
s in
c.
Test the reliability of your tests by seeding
bugs and checking the detection rate
Automate the activity with a mutation
testing framework:
See http://www.mutation-testing.net
FAULT SEEDING
CONCLUSION
© P
yxis
Te
chnolo
gie
s in
c.
Treat your tests as first order citizens
Work to improve your feedback cycle
Commit or get out, but don‟t go in halfway
TAKE AWAY
