www.classfive.com Date
Jim Moore, ClassFive LLC, CTO
Unit Testing and Test Automation
NOVATaig May 13, 2009
2
Topics
•Concepts and terminology of Unit Testing•Benefits and Limitations of Unit Test•See sample unit tests and test fixtures•Related technologies and methodologies•Discuss where Unit Test fits into overall testing methodology•Share knowledge
3
About Jim & ClassFive
Jim – Software Engineer – Multimedia, Office Automation, Point of Sale, Training, Consulting – 20 years exp
ClassFive - Focused on small to medium associations for IT consulting and providing Association Management systems and collabortaion tools
4
Why Unit Testing for ClassFive?
“Begin with the end in mind” -- Stephen Covey (The 7 Habits of Highly Effective People)
Helpful toolset/methodology to help creating a new product codebase that is highly stable, changeable and usable.
Something we can confidently sell and still sleep at night.
5
Challenges starting Unit Test
•We don’t have time to do this!•It won’t make meaningful difference.•My code doesn’t need tests!•Just make it work now!•How many unit tests do we need?•What toolset to use?•How far to go? (e.g. Mock objects, CI)•Deadlines vs process improvement•Can we reduce our testing staff if we unit test?
6
Challenges starting Unit Test – Part 2
Hey!
What is this Unit Testing thing – and what does it do?
9
This is a Unit Test
public void MyAddTest() { MyMath m = new MyMath(); Assert.AreEqual(m.Add(2, 3), 5); }
10
Definition: Unit Test
A unit test is a piece of code (usually a method) that invokes another piece of code and checks the correctness of some assumptions afterward. If the assumptions turn out to be untrue, the unit test has failed.
11
Unit Tests – Independence
•Units are tested independent of each other and their environment.•Tests should only test one thing.•Try to keep a traceable connection between potential bugs and test code.
12
Why Unit Test?
• Implicit documentation• Simplifies Integration• Improves code design – code becomes highly decoupled due to 2 users : application and test harness – code resists transition from 1 user to 2 users, the rest are easy• Facilitates Change• Allows programmer to refactor code at a later date and show individual parts are correct
• Continuous unit tests support sustained maintenance
13
Limitations of Unit Testing
•Don’t expect Unit Testing to solve all problems or catch all defects – not silver bullet•Will not catch integration errors – focuses on individual units•Only 1 pare of an effective testing methodology – cover all bases•Requires rigorous discipline•Requires sustainable process of construction and remediation
16
More about Quality
•What is quality software?•Who decides?•Pretty front vs. termites in the framing.•Technical quality vs. Business value
17
Elements of a Good Unit Test Fixture
•Smell test (first reaction)•Unit test hierarchy should parallel units•Naming convention clear and consistent•Good coverage of units in module•Low to no % of unintended failures•Test intended failures, not just expected behavior
18
Unit Test Quality Types (Good, Bad, Ugly)
Good – effective, well designed testsBad – ineffective (wrong) tests (who cares about design?)Ugly – effective, poorly designed tests
19
Elements of a Good Unit Test
•Automatic • Invocation
• Checking
•Thorough• Depends on needs of project
•Repeatable•Independent•Professional
• Test code is “real code”(Source: Pragmatic Unit Testing with C#)
20
Software Development Testing Stages
Unit Testing – testing of individual componentsIntegration Testing – testing to expose problems from combination of componentsSystem Testing – test complete system prior to deliveryAcceptance Testing – by users to check system from satisfaction of requirements
21
Unit Test vs. Acceptance Test
•Unit Test – satisfy programmer that software does what programmer thinks it does
•Acceptance Test – satisfy customer that software provides business value and makes them willing to pay for software
23
What is a Unit Test Framework
•Automatically run a set of Unit Tests
•Collects Unit Tests into cohesive groups (e.g. by module)
•Collects Unit Test results into a consolidated report
•Provides setup and teardown support to unit tests
24
What is a Test Harness?
Program that executes set of Test Fixtures, collects results and produces combined report.
25
What is a Test Fixture?
•Code that executes tests
•Phases:• Setup Environment
• Execute Test
• Teardown Environment
26
Sample Unit code
using System;namespace MyApp{ public class MyMath { public int Add(int i, int j) { return i + j; } }}
27
•Sample Test Fixture
using System;using NUnit.Framework;using MyApp; namespace MyAppTest{ [TestFixture] public class Class1 { [Test] public void MyAddTest( ) { MyMath m = new MyMath( ); Assert.AreEqual(m.Add(2, 3), 5); } }}
28
Unit Test Frameworks for All
•Java – JUnit, Cactus, EasyMock•Ruby – Test:Unit, RSpec•.Net - Nunit, TestDriven.Net, NMock•Javacript – JSUnit, JSpec, •Python – PyUnit
List of frameworks can be found at : http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
29
StringTemplate Unit Tests
StringTemplate is template library (e.g. merge data into template file)•Substantial set of unit tests
•Originally written in Java (and ANTLR)
•Ported to C# and Python
30
StringTemplate Unit Test Example – C#
C# version
[Test] public virtual void testSingleExprTemplateArgument() { string templates = "" + "group test;" + NL
+ "test(name) ::= \"<bold(name)>\"" + NL+ "bold(item) ::= \"*<item>*\"" + NL;
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates), typeof(AngleBracketTemplateLexer)); StringTemplate e = group.GetInstanceOf("test"); e.SetAttribute("name", "Ter"); string expecting = "*Ter*"; string result = e.ToString();
Assert.AreEqual(expecting, result);}
31
StringTemplate Unit Test Example - Java
public void testSingleExprTemplateArgument() throws Exception {String templates =
"group test;" +newline+"test(name) ::= \"<bold(name)>\""+newline+"bold(item) ::= \"*<item>*\""+newline;
StringTemplateGroup group =new StringTemplateGroup(new StringReader(templates),
AngleBracketTemplateLexer.class);StringTemplate e = group.getInstanceOf("test");e.setAttribute("name", "Ter");String expecting = "*Ter*";String result = e.toString();assertEquals(expecting, result);
}
32
Common Unit Test Patterns
•Correct results•Boundary conditions correct•Inverse relationships checked (e.g. use result to check itself)•Cross check results by other means•Force error condition to occur•Was performance within bounds (e.g. impact of data set size, network latency)
33
Common Unit Test Patterns
Boundary Conditions:
•Format conformance•Result ordering•Input data range•External references required state•Resource existence (e.g database, file)•Correct number items (e.g. 0, 1, more than 1)•Time (e.g. timeout length, tolerance of delay)
37
Refactoring
The process of changing module’s internal structure without modifying its external functional behavior or existing functionality.
This is done to: • Improve internal non-functional properties of the software (e.g. readability)• Simplify code structure (e.g. adhere to a given programming paradigm), • Improve maintainability• Improve performance• Improve extensibility
39
Test Driven Development (TDD)
A software development technique that uses short development iterations based on pre-written test cases that define desired improvements or new functions.
Each iteration produces code necessary to pass that iteration's tests. Finally, the programmer or team refactors the code to accommodate changes.
A key TDD concept is that preparing tests before coding facilitates rapid feedback changes.
40
TDD Steps
•Write tests that fail•Implement modules to make test pass.•Run all tests to ensure nothing broken•Refactor to keep code clean•Continue development
42
Continuous Integration
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day.
Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.
43
Continuous Integration Practices
* Maintain a Single Source Repository. * Automate the Build * Make Your Build Self-Testing * Everyone Commits Every Day * Every Commit Should Build the Mainline on an Integration Machine * Keep the Build Fast * Test in a Clone of the Production Environment * Make it Easy for Anyone to Get the Latest Executable * Everyone can see what's happening * Automate Deployment
(Souce: http://martinfowler.com/articles/continuousIntegration.html)
44
Continuous Integration Tools
Version Control: Subversion, CVS, Git, SourceSafe
Build Tools: Ant, NAnt, MSBuild, Make
CI Server: CruiseControl, CruisControl.Net, TeamCity
Unit Test: NUnit, JUnit, NMock, EasyMock, etc
45
Unit Test As Software Test Automation
•Unit test execution best when part of an automated build system. •Unit Test practice can directly affect the quality of the code produced.•Good set of automated tests allow for incremental creation of code quality.•Unit Test starts with immature codebase while other automation requires mature codebase and design
46
Summary
•Unit Tests are independent tests of small chunks of code created during development.
•There are good Unit Test frameworks and harnesses available for many languages and environments.
•Unit Testing can directly affect the quality of code during construction.
•Unit Testing is only one part of an effective test automation methodology.
www.classfive.com
Thanks for coming!
Jim Moore
CTO
ClassFive, LCC
http://www.classfive.com/
Top Related