LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS...
Transcript of LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS...
![Page 1: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/1.jpg)
LESSONS LEARNED IN
PROGRAMMER TESTINGPATTERNS AND IDIOMS
James Newkirk and Brad Wilson
![Page 2: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/2.jpg)
What is Programmer Testing?
Brian Marick http://www.testing.com
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Customer
Technology
Support C
ritique
![Page 3: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/3.jpg)
Do you do programmer testing?
How many of you have been doing
programmer testing for 5 years or more?
4 years
3 years
2 years
1 year
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 4: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/4.jpg)
Why do programmer testing?
“There is no such thing as done. Much more
investment will be spent modifying programs
than developing them initially” [Beck]
“Programs are read more often than they are
written” [Beck]
“Readers need to understand programs in
detail and concept” [Beck]
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 5: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/5.jpg)
Total Development Cost
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Extend/MaintainDevelop
![Page 6: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/6.jpg)
Understand
Deploy
Test
Change
Extend/Maintain Cost [Beck]
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 7: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/7.jpg)
I might break
something
![Page 8: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/8.jpg)
Where do I start?
![Page 9: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/9.jpg)
Just Do It!
![Page 10: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/10.jpg)
Write tests using the 3A pattern
Lesson #1
![Page 11: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/11.jpg)
3A Pattern
Attributed to Bill Wake (http://xp123.com)
Arrange – Setup the test harness
Act – Run the test
Assert – Check the results
Let’s look at an example!
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 12: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/12.jpg)
[Fact]public void TopDoesNotChangeTheStateOfTheStack(){
Stack<string> stack = new Stack<string>();stack.Push("42");
string element = stack.Top;
Assert.False(stack.IsEmpty);}
A Typical Test
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Assert
![Page 13: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/13.jpg)
[Fact]public void TopDoesNotChangeTheStateOfTheStack(){
Stack<string> stack = new Stack<string>();stack.Push("42");
string element = stack.Top;
Assert.False(stack.IsEmpty);}
3A Pattern
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Assert
Arrange
![Page 14: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/14.jpg)
[Fact]public void TopDoesNotChangeTheStateOfTheStack(){
Stack<string> stack = new Stack<string>();stack.Push("42");
string element = stack.Top;
Assert.False(stack.IsEmpty);}
3A Pattern
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Assert
Act
![Page 15: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/15.jpg)
[Fact]public void TopDoesNotChangeTheStateOfTheStack(){
Stack<string> stack = new Stack<string>();stack.Push("42");
string element = stack.Top;
Assert.False(stack.IsEmpty);}
3A Pattern
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Assert
Assert
![Page 16: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/16.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
3A Summary
Benefits
Readability
Consistency
Liabilities
More Verbose
Might need to introduce local variables
Related Issues
One Assert per Test?
![Page 17: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/17.jpg)
Keep Your
Tests Close
Lesson
#2
![Page 18: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/18.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Keep Your Tests Close
Benefits
Tests are equivalent to production code
Solves visibility problems
Liabilities
Should you ship your tests?
If No, how do you separate the tests from the
code when you release?
![Page 19: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/19.jpg)
ExpectedException leads to uncertainty
Lesson #3
![Page 20: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/20.jpg)
ExpectedException Violates 3A
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
[Test][ExpectedException(typeof(InvalidOperationException))]public void PopEmptyStack(){
Stack<string> stack = new Stack<string>();
stack.Pop();}
![Page 21: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/21.jpg)
Record the Exception instead
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
[Fact]public void PopEmptyStack(){
Stack<string> stack = new Stack<string>();
Exception ex = Record.Exception(() => stack.Pop());
Assert.IsType<InvalidOperationException>(ex);}
![Page 22: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/22.jpg)
Use Assert.Throws - .NET 2.0
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
[Fact]public void PopEmptyStack(){
Stack<string> stack = new Stack<string>();
Assert.Throws<InvalidOperationException>(delegate{
stack.Pop();});
}
![Page 23: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/23.jpg)
Use Assert.Throws - .NET 3.5
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
[Fact]public void PopEmptyStack(){
Stack<string> stack = new Stack<string>();
Assert.Throws<InvalidOperationException>(() => stack.Pop());
}
![Page 24: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/24.jpg)
More ExpectedException
Problems
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
public CheckingAccount(double balance){
if (balance == 0) throw new ArgumentException("...");}
public void Deposit(double amount){
if(amount == 0) throw new ArgumentException("...");}
[Test, ExpectedException(typeof(ArgumentException))]public void DepositThrowsArgumentExceptionWhenZero(){
CheckingAccount account = new CheckingAccount(0.00);
account.Deposit(0.00);}
![Page 25: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/25.jpg)
Use Assert.Throws
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
public void Deposit(Decimal amount){
if(amount == 0) throw new ArgumentException("...");
// the rest of the implementation}
[Fact]public void DepositThrowsArgumentExceptionWhenZero(){
CheckingAccount account = new CheckingAccount(150.00);
Assert.Throws<ArgumentException>( () => account.Deposit(0));
}
![Page 26: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/26.jpg)
Improved Control Flow
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
[Fact]public void PopEmptyStack(){
Stack<string> stack = new Stack<string>();
Exception ex = Record.Exception(() => stack.Pop());
Assert.IsType<InvalidOperationException>(ex);Assert.Equal("Stack empty.", ex.Message);
}
![Page 27: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/27.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Use Alternatives to ExpectedException
Benefits
Readability (these tests look like all the rest)
Identify and isolate the code where you are
expecting the exception
Improved control flow
Liabilities
Act and Assert are together in Assert.Throws
Anonymous delegate syntax in .NET Framework
2.0 is not great for readability
![Page 28: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/28.jpg)
Small Fixtures
Lesson #4
![Page 29: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/29.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Small Fixtures
Benefits
Smaller more focused test classes
Class contains nested classes
Liabilities
Potential code duplication
Issues with test runners
Related Issues
Do you need SetUp and TearDown?
![Page 30: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/30.jpg)
Don’t Use
SetUp or TearDownLesson #5
![Page 31: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/31.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Don’t Use SetUp orTearDown
Benefits
Readability
Test isolation
Liabilities
Duplicated initialization code
Related Issues
Small Fixtures
![Page 32: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/32.jpg)
ImproveTestability
withInversion
of Control
Lesson
#6
![Page 33: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/33.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Inversion of Control (IoC)
Pattern
Article: http://shrinkster.com/wkm
Dependency Injection
Constructor Injection
Setter Injection
Let’s look at an example from the article!
![Page 34: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/34.jpg)
Before
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Movie FileBasedRespository
MovieLister
![Page 35: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/35.jpg)
After
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Movie
FileBasedRespository
MovieLister
«interface»
IMovieRepository
![Page 36: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/36.jpg)
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
Improve Testability with IoC
Benefits
Better test isolation
Decoupled class implementation
Liabilities
Decreases encapsulation
Interface explosion
Related Issues
Dependency injection frameworks are overkill for
most applications
![Page 37: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/37.jpg)
Summary
Just Do It!
Lesson #1 – Write Tests using the 3A Pattern
Lesson #2 – Keep your tests Close
Lesson #3 – Use Alternatives to
ExpectedException
Lesson #4 – Small Fixtures
Lesson #5 – Don’t use SetUp or TearDown
Lesson #6 – Improve Testability with
Dependency InjectionUnless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 38: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/38.jpg)
Tools
xUnit.net – http://codeplex.com/xunit
Nunit – http://nunit.org
MbUnit – http://mbunit.com
Visual Studio 2008 -
http://msdn2.microsoft.com/enus/vstudio/default.aspx
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 39: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/39.jpg)
Blogs
Brian Button
http://www.agileprogrammer.com/oneagilecoder
Brian Marick http://www.testing.com/cgi-bin/blog
Peter Provost http://peterprovost.org
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 40: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/40.jpg)
Books
[Beck] Implementation Patterns by Kent Beck,
Addison-Wesley, 2008
xUnit Test Patterns by Gerard Meszaros,
Addison-Wesley, 2007
Refactoring to Patterns by Joshua Kerievsky,
Addison-Wesley, 2005
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 41: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/41.jpg)
Contact Information
James Newkirk
Email: [email protected]
Blog: http://jamesnewkirk.typepad.com
Brad Wilson
Email: [email protected]
Blog: http://bradwilson.typepad.com
Twitter: http://twitter.com/bradwilson
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved
![Page 42: LESSONS LEARNED IN PROGRAMMER TESTING · LESSONS LEARNED IN PROGRAMMER TESTING PATTERNS AND IDIOMS James Newkirk and Brad Wilson](https://reader035.fdocuments.in/reader035/viewer/2022062605/5fc6dc49894be75522293249/html5/thumbnails/42.jpg)
Questions
Unless Noted Otherwise Copyright © 2008 James Newkirk. All Rights Reserved