Finding the Right Testing Tool for the Job

Sebastian Bergmann and Ciaran McNulty

Who are these guys?

3 Dimensions of Testing

Goal - why we are writing the testScope - how much of the system is involved in the testForm - how we express the test

3 4 Dimensions of Testing

Goal - why we are writing the testScope - how much of the system is involved in the testForm - how we express the testTime - when we write the test

What we will talk about

— Characterisation Tests

— Acceptance Tests

— Integration Tests

— Unit Tests

Characterisation Tests

Characterisation TestsGoals:

— Capture existing behaviour

— Find out when behaviour changes

Characterisation TestsScopes:

— Often at UI level

— Sometimes at object/service level

Characterisation TestsTimes:

— Always after implementation

Characterisation TestsBest practice:

— Treat these tests as a temporary measure

— Use a tool that makes it easy to create tests

— Expect pain and suffering

Characterisation TestsForm: Behat + MinkExtension builtin steps

Scenario: Product search returns results Given I am on "/" And I fill in "search" with "Blue jeans" When I press "go" Then I should see "100 Results Found"

Characterisation TestsForm: PHPUnit + phpunit-mink-trait

class SearchTest extends PHPUnit_Framework_TestCase{ use phpunit\mink\TestCaseTrait;

public function testProductSearchReturnsResult() { $page = $this->visit('http://example.com/');

$page->fillField('search', 'Blue Jeans'); $page->pressButton('go');

$this->assertContains('100 Results Found', $page->getText()); }}

Characterisation TestsForm: Selenium IDE

Characterisation TestsForm: Ghost Inspector

Characterisation TestsForm: PHPUnit + de-legacy-fy

See docs at on GitHub at sebastianbergmann/de-legacy-fy

Acceptance Tests

Acceptance TestsGoals:

— Match system behaviour to business requirements

— Get feedback on proposed implementations

— Understand business better

— Document behaviour for the future

Acceptance TestsScopes:

— At a UI layer

— At an API layer

— At the service layer

— Lower level may be too disconnected

Acceptance TestsTimes:

— Before implementation

— Before commitment (as long as it's not expensive?)

— Hard to write in retrospect

Acceptance TestsBest practices:

— Get feedback on tests early

— Make them readable, or have readable output, for the intended audience

— Apply for the smallest scope first

— Apply to core domain model first

— Minimise end-to-end tests

Acceptance TestsForm: Behat at service level

Scenario: Sales tax is applied to basket Given "Blue Jeans" are priced as €100 in the catalogue When I add "Blue Jeans" to my shopping basket Then the basket total should be €120

class BasketContext implements Context{ public function __construct() { $this->catalogue = new InMemory\Catalogue(); $this->basket = new Basket($catalogue); }

/** * @Given :productName is/are priced as :cost in the catalogue */ public function priceProduct(ProductName $productName, Cost $cost) { $this->catalogue->price($productName, $cost); } //...}

class BasketContext implements Context{ //... /** * @When I add :productName to my shopping basket */ public function addProductToBasket(ProductName $productName) { $this->basket->add($productName); }

/** * @Then the basket total should be :cost */ public function checkBasketTotal(Cost $cost) { assert($this->basket->total == $cost->asInt()); }}

Acceptance TestsForm: PHPUnit at service level

class BasketTest extends PHPUnit_Framework_TestCase{ public function testSalesTaxIsApplied() { $catalogue = new InMemory\Catalogue(); $basket = new Basket($catalogue);

$productName = new ProductName('Blue Jeans'); $catalogue->price($productName, new Cost('100')); $basket->add($productName);

$this->assertEquals(new Cost('120'), $basket->calculateTotal()); }}

Acceptance TestsForm: PHPUnit at service level

Acceptance TestsForm: Behat at UI level

Scenario: Sales tax is applied to basket Given "Blue Jeans" are priced as €100 in the catalogue When I add "Blue Jeans" to my shopping basket Then the basket total should be €120

class BasketUiContext extends MinkContext{ public function __construct() { $this->catalogue = new Catalogue(/* ... */); }

/** * @Given :productName is/are priced as :cost in the catalogue */ public function priceProduct(ProductName $productName, Cost $cost) { $this->catalogue->price($productName, $cost); } //...}

class BasketUiContext extends MinkContext{ //... /** * @When I add :productName to my shopping basket */ public function addProductToBasket(ProductName $productName) { $this->visitPath('/products/'.urlencode($productName)); $this->getSession()->getPage()->pressButton('Add to Basket'); }

/** * @Then the basket total should be :cost */ public function checkBasketTotal(Cost $cost) { $this->assertElementContains('#basket .total', '€120'); }}

Acceptance TestsForm: PHPUnit at UI level

class BasketUiTest extends PHPUnit_Framework_TestCase{ use phpunit\mink\TestCaseTrait;

public function testSalesTaxIsApplied() { $catalogue = new Catalogue(/* ... */); $catalogue->price(new ProductName('Blue Jeans'), new Cost(120));

$page = $this->visit('http://example.com/products/'.urlencode($productName)); $this->getSession()->getPage()->pressButton('Add to Basket');

$this->assertContains( '€120', $page->find('css', '#basket .total')->getText() ); }}

Integration Tests

Integration TestsGoals:

— Test cross-boundary communication

— Test integration with concrete infrastructure

Integration TestsScopes:

— Large parts of the system

— Focus on the edges (not core domain)

— Areas where your code interacts with third-party code

Integration TestsTimes:

— After the feature is implemented in core / contracts are established

— During integration with real infrastructure

— When you want to get more confidence in integration

— When cases are not covered by End-to-End acceptance test

Integration TestsBest Practices:

— Use tools with existing convenient integrations

— Focus on testing through your API

— Make sure your core domain has an interface

Integration TestsForm: PHPUnit + DbUnit

class CatalogueTest extends PHPUnit_Extensions_Database_TestCase{ public function getConnection() { $pdo = new PDO(/* ... */); return $this->createDefaultDBConnection($pdo, 'myDatabase'); }

public function getDataSet() { return $this->createFlatXMLDataSet(dirname(__FILE__) . '/_files/catalogue-seed.xml'); } // ...}

class CatalogueTest extends PHPUnit_Extensions_Database_TestCase{ // ...

public function testProductCanBePriced() { $catalogue = new Catalogue(/* ... */);

$catalogue->price( new ProductName('Blue Jeans'), new Cost('100') );

$this->assertEquals( new Cost('100'), $catalogue->lookUp(new ProductName('Blue Jeans') ); }}

Unit Tests

Unit TestsGoals:

— Test components individually

— Catch errors earlier

— Drive internal design quality

— Document units for other developers

Unit Tests:Scopes:

— Single classes

— Single classes + value objects?

— Extremely small units of code

Unit Tests:Times:

— Just before you implement

Unit Tests:Times:

— Just before you implement

OK, maybe...

— Just after you implement

— If you want to learn more about a class

— But always before you share the code!

Unit TestsBest Practices

— Write in a descriptive style

— Describe interactions using Test Doubles

— Don't get too hung up on isolation

— Don't touch infrastructure

— Don't test other people's code

— Don't double other people's code?

Unit TestsForm: PHPUnit

class BasketTest extends PHPUnit_Framework_TestCase{ public function testSalesTaxIsApplied() { $catalogue = $this->getMock(Catalogue::class); $catalogue->method('lookUp')->with(new ProductName('Blue Jeans')) ->willReturn(new Cost('100'));

$basket = new Basket($catalogue); $basket->add(new ProductName('Blue Jeans'));

$this->assertSame(new Cost('120'), $basket->calculateTotal()); }

}

Unit TestsForm: PhpSpec

class BasketSpec extends ObjectBehavior{ function it_applies_sales_tax(Catalogue $catalogue) { $catalogue->lookUp(new ProductName('Blue Jeans'))->willReturn(new Cost('100')); $this->beConstructedWith($catalogue);

$this->add(new ProductName('Blue Jeans'));

$basket->calculateTotal()->shouldBeLike(new Cost('120')); }

}

5th Dimension - Who?

— Choose the right approaches for your context

— What mix of languages can the team use?

— What styles of testing will add the most value?

— What formats make the most sense to the team?

— How will tests fit into the development process?

There is no right answer, there are many right answers!

Photo Credits

— "tools" by velacreations (CC) - https://flic.kr/p/8ZSb3r

— "Components" by Jeff Keyzer (CC) - https://flic.kr/p/4ZNZp1

— Doctor Who stolen from BBC.co.uk

— Other images used under license

Thank You & Questions?

@s_bergmann@ciaranmcnulty

https://joind.in/talk/80dbd