Tea camp #5Automated software

testing

David DurantGreater London Authority

@cholten99

Intro

Why do we test? It's about trust

Ensuring quality everyone can see, making updates faster and safer.

It’s part of the whole development narrative from business needs, coding,

testing, deployment, and iteration - part of modern fast Agile development cycles

The art of possible - GDS makes multiple zero downtime fully tested

deployments every day

The testing pyramid

We test at lots of different levels - often defined as the following.

Unit tests

Regression tests - ensure bugs have been fixed

Integration tests

System tests

User acceptance tests (UAT)

Some history

Testing has been around (almost) as long as software

Early testing was all manual

Later this included manual repeatable testing made up of lists of tests and test

steps in a spreadsheet - this kind of testing is slow, potentially accident prone

and takes up lots of people-time

Automated testing - driven by business needs (“happy path”), expected errors

and exploratory testing

Early automation

Unit tests are small / fast pieces of test code that test specific blocks of system

component code - there can be 100s of 1000s of them in a big system

At this stage teams are still doing manual integration, system and UAT testing

Modern code delivery pipeline

Automated testing is part of a full managed development pipeline using software

such an Jenkins

Team has an agreed branching / merging strategy for code

Zero downtime deployments

Jenkins

Testing stages

Start by developer running tests on their own machine before committing code

to integration branch

Test machine - VM or Docker clone of production

Staging - at most one update behind production

Production - could be a cluster

Types of tests, numbers & when they are run

Unit tests - 100,000s - own machine

Integration / system tests - 1000s - own machine / test / staging

UAT - 100s - own machine / test / staging

Smoke - 10s - production

How tests are done (1)

Integration testing - API specific ‘black box’ testing

System testing

Browser compatibility - often cloud based

Load testing (Gatling)

Security (OWASP, NCSC, etc - constant not just periodic PEN testing)

Others

How tests are done (2)

User acceptance testing

End-to-end user journey testing - sometimes call Behaviour Driven Development

Selenium “headless” testing

Re-entrant - consistent test data

BDD test tool - Cucumber - based on acceptance criteria for user stories agreed

with the business as part of the Agile process

Based on “given / when / then” format

Cucumber example

Scenario: Duplicate email

When someone tries to create an account for an email address that

already exists

Given I have chosen to sign up

When I enter an email address that has already been registered

Then I should be told that the email is already registered

And I should be offered an option to recover my password

Same test, multiple data

Cucumber supports the idea of tables of data

The same test is executed with data from each row in the table

This can included testing for expected errors

The next frontier - Test Driven Development

Test Driven Development is writing your tests for each sprint before you write

your code

Initially all your tests will fail!

But as your implement the required functionality they will start to pass

Any regressions are immediately obvious during development

Online testing communities

Cross-government Slack : http://tiny.cc/uk-cross-gov-testers

Local government Slack : http://tiny.cc/local-gov-testers

Any questions?

London Digital Peer Group Slack : http://tiny.cc/london-digital-peer-group

David Durant

Senior Business Analyst

Greater London Authority