Automated Testing

Ted Driggs (tdriggs)

2

What

• Verify program behavior without human interaction

• Programmatically load and run test code on a wide array of systems

3

Why

4

Why

5

Why

1. Bugs are bad2. Programs are big3. Tests are boring

6

METHODSTypes of automated testing

7

Test Harnesses

• A test harness is a program which executes test cases.

• Most unit-testing frameworks are test harnesses.

8

Test HarnessesThe average test harness follows a pattern like the one at right. There is a single method which is called when test execution starts: This might be opening a connection to a database.

For each test, setup and cleanup methods are run. These methods ensure that each test runs from a consistent state and that test failures are atomic.

Once the tests are all run, the class cleanup method is run. This method ensures the class does not leave the system in an inconsistent state.

9

Data-Driven Tests

• Use configuration files to pass data to test methods.

• Frequently XML, but not always

10

Data-Driven Tests <TestRow>

<Int32>4</Int32><String>This will be read</String><String>by a factory class</String><String>and transformed</String>

</TestRow><TestRow>

<Int32>5</Int32><String>into an object</String><String>and will höpefully</String><String>find lots of bugs.</String>

</TestRow>

At left is the XML for a data-driven test using TAEF (the Test Authoring and Execution Framework). Each test row is treated like a test from the previous test harnesses.

11

TOOLSExisting software

12

JavaThe premiere unit testing frame-work for Java is jUnit. Supported by a very good Eclipse plugin, jUnit was one of the first successful test harnesses.

13

C# and .NETC# is a newer language, but the similarity to Java means similar test capabilities.

There are two options: nUnit, which is a port of jUnit for .NET, and the newer Microsoft Visual Studio Unit Test Framework. The former was widely used until recently, but the expanded feature set of the latter means it is likely to become dominant.

14

JavascriptThe most widely-used option for a unit-tests is JsUnitTest, which is based on Prototype.js.

15

PythonIn keeping with Python’s “batteries included” philosophy, the language comes with the unittest module. It behaves very similarly to the ones shown previously.

16

RECOMMENDED USESource Control Integration

17

Source Control

• Tests are only good as long as they are actually being used.

• Source control can verify that new code passes existing automation.

18

Source Control

Write Submit Test Comm

it

19

Source Control

Write Submit Test Comm

it

20

Bad Ideas

• Test-driven development• Opt-in testing (devs will not

remember)

21

SUMMARYFinal Remarks

22

Benefits

• Prevents feature regressions• Limits risk of refactoring• Increases tests’ code coverage• Makes life easier for developer and

tester