AGILE TEST AUTOM. AT SCALEPaper presentation at PNSQC 2013

Mark FinkQuality Engineer, Tabane Limited

(c) 2013 Mark Fink

Consulting-Services & Customers

Consulting-Services:

Customers:

AGILE TEST AUTOMATION AT SCALE

Performance Testing

Test Automation

Exploratory Testing

·

·

·

3/18

AGILE TEST AUTOMATION AT SCALE 4/18

Agenda

AGILE TEST AUTOMATION AT SCALE

Context & Problems

How to approach this situation

Activity Testing Pattern

Emphasis of Scope over Form

·

·

·

·

5/18

What is meant by "Agile Test Autom. at Scale"?

AGILE TEST AUTOMATION AT SCALE

Agile -> no big design up front, release often

Modern Web-App -> Single page app (SPA): HTML, CSS, Javascript (Ajax)

Test Automation -> using continuous integration

Scale -> scope >10 man years, more than 5 developers, high velocity

·

·

·

·

6/18

Continuous Integration

AGILE TEST AUTOMATION AT SCALE 7/18

Evolving with Continuous Integration

AGILE TEST AUTOMATION AT SCALE 8/18

Jenkins Wall Display

AGILE TEST AUTOMATION AT SCALE 9/18

Test Automation Methodology

Common Goals:

AGILE TEST AUTOMATION AT SCALE

Release the application often e.g. every two to four weeks

Go-Live decision support for each release

Instant feedback regarding regressions to developers

Means to identify gaps in the test infrastructure

·

·

·

·

10/18

Method: Activity-Testing Pattern

An excellent model when designing GUI level functional tests is to think aboutdescribing the test and the automation at the following three abstraction levels:

AGILE TEST AUTOMATION AT SCALE 11/18

Method: Activity-Testing Pattern

Level Description Example

BusinessRule /Functionality

What is the intention of the test, what is itdemonstrating or exercising? A good start whenimplementing tests is using examples which havebeen gained during specification of the storyregarding the feature in question.

Free delivery is offered to customers whoorder two or more books.

GUIWorkflow

What does a user have to do in order to exercisethe functionality through the GUI, on a higheractivity level.

Put two books in a shopping cart, enteraddress details, verify that delivery optionsinclude free delivery.

TechnicalActivity

What are the technical steps required to exercisethe functionality. On this level the API of theautomation tool (in this case Selenium) is used.There is no need to wrap the features of theautomation tool!

Open the shop homepage, log in with'testuser' and 'password', go to the '/book'page, click on the first image with the 'book'id, wait for page to load, click on the 'Buy-now' link and so on.

by Gojko Adzic

AGILE TEST AUTOMATION AT SCALE 12/18

Emphasizing Scope over Form

James Whittaker, 2012, "How we test at Google"

AGILE TEST AUTOMATION AT SCALE 13/18

AGILE TEST AUTOMATION AT SCALE 14/18

Hitchhiker's Guide to Test Automation

Tutorial Style:

AGILE TEST AUTOMATION AT SCALE

Working single page application (SPA)

Test-Suites to demonstrate different test scopes

Jenkins job configurations

Section on performance testing

Section on exploratory testing

Samples on Nosetests, FitNesse, Selenium, Jasmine BDD, Jenkins, JMeter, ...

·

·

·

·

·

·

15/18

and... it has supercars!

AGILE TEST AUTOMATION AT SCALE 16/18

</ Thank You! >

twitter @markfinkwww www.testing-software.orggithub github.com/markfink