xUnit Test Patterns (Some) xUnit Test Patterns (in practice) by Adam Czepil.
-
Upload
heriberto-newhall -
Category
Documents
-
view
221 -
download
2
Transcript of xUnit Test Patterns (Some) xUnit Test Patterns (in practice) by Adam Czepil.
xUnit Test Patterns
(Some)xUnit Test Patterns
(in practice)by Adam Czepil
Who needs these?
What kind of problems xUnit test patterns may help to solve? Who will benefit the most?
Goals of Test Automation
Code Smells
Principles of Test Automation
Behavior Smells
Projects Smells
Test Patterns
Is it a fish?
Or is it a mammal?
Revenons a nos moutons!
What is Defect Localization and why developers in good projects (almost) don’t use debugger?
Debugging is an art
The problem
The problem
We run the test (possibly manual test)
The problem
We run the test (possibly manual test)The test fails
The problem
We run the test (possibly manual test)The test failsWe have to figure out why it fails?
The problem
We run the test (possibly manual test)The test failsWe have to figure out why it fails?
The problem is that this can be very time consuming...
Fortunatelly we have extra-skilled developers ...
... who can find the problem very quickly ;-)
What we expect from very good tests is that
What we expect from very good tests is thatif the tests fail they provide information
What we expect from very good tests is thatif the tests fail they provide information detailed enoughso we can find the source of the failure quickly.
Defect Localization
ability to very quickly find the actual cause of test failure
Defect Localization is
Defect Localization is
one of the major goals of test automation!
Defect Localization is
Defect Localization is
one of the major advantages of unit tests over customer (functional) tests!
What to do to have good Defect Localization?
Frequent Debugging
When we deal with Frequent Debugging?
When we deal with Frequent Debugging?
When manual debugging is required to determine the cause of most test failures.
When we deal with Frequent Debugging?
When manual debugging is required to determine the cause of most test failures.
This is also true when instead of debugging we use extensive logging during testing.
Frequent Debugging...
Frequent Debugging...
reduces productivity!
Frequent Debugging...
reduces productivity! andmakes development schedules much less predictable!
Frequent Debugging...
reduces productivity! andmakes development schedules much less predictable!
because a single manual debugging session could extend the time required to develop the software by half a day or more.
What are the causes of Frequent Debugging?
What are the causes of Frequent Debugging?
Frequent Debugging is caused by
What are the causes of Frequent Debugging?
Frequent Debugging is caused by missing tests or
What are the causes of Frequent Debugging?
Frequent Debugging is caused by missing tests or lack of Defect Localization in our suite of automated tests.
We should be able to quickly determine the root cause of test failure based on
We should be able to quickly determine the root cause of test failure based on Assertion Message
We should be able to quickly determine the root cause of test failure based on Assertion Message or through the pattern of test failures
We should be able to quickly determine the root cause of test failure based on Assertion Message or through the pattern of test failures andknowledge about changes being made.
What are the main causes for no Defect Localization?
What are the main causes for no Defect Localization?
• not enough detailed unit tests compared to functional tests,
What are the main causes for no Defect Localization?
• not enough detailed unit tests compared to functional tests,• missing component tests (for a cluster of classes) that would
point out an integration error between classes.
What are the main causes for no Defect Localization?
• not enough detailed unit tests compared to functional tests,• missing component tests (for a cluster of classes) that would
point out an integration error between classes, • Infrequently Run Tests, because it causes that it is difficult
to point out the change that caused the tests to fail.
Very good solution for Frequent Debugging problem is...
Very good solution for Frequent Debugging problem is...
TDD – Test Driven Development
Very good solution for Frequent Debugging problem is...
TDD – Test Driven Development (or at least Test First)
What I want you to remember?
What I want you to remember?
Defect Localization is about finding quickly bug causing the tests to fail.
What I want you to remember?
Defect Localization is about finding quickly bug causing the tests to fail.
Unit tests are usually much better in Defect Localization than customer tests.
What I want you to remember?
Defect Localization is about finding quickly bug causing the tests to fail.
Unit tests are usually much better in Defect Localization than customer tests.
Frequent Debugging usually means that project is either missing tests or has poor Defect Localization.
What I want you to remember?
Defect Localization is about finding quickly bug causing the tests to fail.
Unit tests are usually much better in Defect Localization than customer tests.
Frequent Debugging usually means that project is either missing tests or has poor Defect Localization.
Use TDD as often as possible
WARNING! Too much debugging may broke your eyes! ;-)
Questions?
How NOT to automate manual tests?What are some differences between
manual and automated test-case? How to create automated test
comparing to manual test?
Manual vs. Automated Tests
Manual vs. Automated Tests
Let’s compare some aspects...
Manual vs. Automated Tests
Let’s compare some aspects...-time
Manual vs. Automated Tests
Let’s compare some aspects...-time,-fragility.
Time
Executing manual tests requires A LOT OF time from HUMAN
Time
Executing automated tests requires almost NO time from HUMAN
Since HUMAN time is precious and COMPUTER time is cheap we should care much about the first anddon’t have to worry too much about the latter.
Conclusions
Opposite to manual tests we can create unlimited number of regression test cases.
Fragility
When the test fails/stops in some non-final point HUMAN usually finds a way (work around) to continue its execution.
Fragility
When the test fails in some non-final point test execution STOPS.
HUMAN, although much slower than COMPUTER, is smarter test executor, especially when the test fails before all test conditions have been checked.
Conclusions
Opposite to manual tests we have to care about assertions order andfind a way of handling not executed assertions.
Verify One Condition per Test
Verify One Condition per Test principle is not the same asOne Assertion Per Test principle
Let’s see an example...
example-refactored.zipExample.zip
example example refactored
When (not) to use setup (@Before) and teardown (@After) in xUnit?
Verify One Condition per Test helps to give us good Defect Localization to pinpoint the source of the failure.
Another reason we Verify One Condition per Test is to Minimize Test Overlap
Another reason we Verify One Condition per Test is to Minimize Test Overlap, so that we have fewer tests to modify if we later modify the behavior of the SUT.
So? How NOT to automate manual tests?
So? How NOT to automate manual tests?
Don’t write long, complex test scenarios
So? How NOT to automate manual tests?
Don’t write long, complex test scenarios,instead Verify One Condition per Test
So? How NOT to automate manual tests?
Don’t write long, complex test scenarios,instead Verify One Condition per Test,but not necesserily one assertion per test.
Questions?
Hey! Test code is code too!
Is test code different than “regular” code? How to organize test cases into classes/files/directories? What about test utils/helpers? Where to keep them?
Questions?
There is a lot of „patterns” described in the book. Here are „maps” of them...
The Maps
Why it’s worthwile to study xUnit patterns?
Questions?
Thank you!Go there: http://xunitpatterns.com/Really! You don’t have to read everything there...