Making Test Automation Work in Agile Projects 2012

1

Making Test Automation Work in

Agile Projects

ØREDEV 2012 Lisa Crispin

With Material from Janet Gregory

2

Introductions: Experience, Goals

§ Form groups: § New to automation § One to two years experience w/ automation § More than two years experience

§ Talk with at least two other people § Tell each other your learning goals for today § Note the most interesting one you hear

Copyright 2012: Lisa Crispin 3

About me…

Programmer, tester, agile team member


Introduction - You


Vive la Rebellion!

Successful test automation is an innovation! •  Let go of traditional roles, reach across them •  Visualize how tests should look •  Be smart about when to automate •  Experiment to overcome obstacles •  Find/build the right tools

6

Exercise: Your Learning Goals

§ Write one interesting learning goal you heard from another participant at the start of the class on a sticky note.

§ Write your number one learning goal for today on a sticky note.

§ Put the sticky notes on your “learning goals” sheet

§ Group similar ones together


Let’s start by defining “agile”

Agile teams: q  Deliver business value

frequently q  at a sustainable pace q  while adapting to the changing

needs of the business Source: Elisabeth Hendrickson


The key is “sustainable pace”

Technical debt slows us down

Copyright 2012: Lisa Crispin

High technical debt + insufficient automation = even less time


Barriers to Test Automation What’s holding you back?


Exercise: Your barriers Individually and silently – write one barrier

hindering your team per sticky note Put these on the “impediments” wall chart

for your table group Talk to your fellow group members – do you

see any patterns?


Questions?


The Whole-Team Approach §  Automated tests are code §  Respecting the tests §  Collaborating §  Commitment to quality §  Return on investment


Automated tests are code public class CalculatorFixture extends

ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget; private Calculator calculator = new

Calculator(); private Double irr;

Source: Gojko Adzic, StarEast 2011 keynote


Testers are especially good at… § Eliciting examples § Turning them into tests § Ensuring the right testing gets done § Exploratory testing

18

Give testers time to do what we do best

How?

19 The Whole-Team Approach


Experiment: Iteration 1 §  Pair up: one will be tester, one programmer §  Sit back to back so you face away from each

other §  Tester gets a drawing which needs to be

replicated §  Tester tells the programmer what to draw, all at

one time. §  Programmer draws the shapes based on what

the tester explained. §  No talking during ‘coding’!

§  Tester “tests” the drawing, reports “bugs” on index cards

§  Programmer fixes the “bugs” §  How long did it take? Will the customer be

happy?


Experiment: Iteration 2 §  Collaborate! §  Tester tells programmer what to draw, watches

the programmer draw, answers questions, points out ‘defects’ for programmer to fix immediately §  (Don’t show the programmer the drawing, that

makes it too easy, we’re trying to simulate real coding)

§  How long did it take? Will the customer be happy?

Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game


Ways to collaborate

Pair: Tester-Coder Tester-Tester Any role – any role


Team responsibility

Automate all regression tests


Learn

Experiment

Meaningful Commitment


Under-commit

Plan less work than you think you can do Including all test automation


Learn to write maintainable tests §  Get over the “hump of pain”

From Gerard Meszaros’ XUnit Test Patterns

Whole-team software development

28

Create a user story

Write Customer (Q2) Tests

Start thinking how to code

Expand tests – Story Tests

Automate Q2 Tests

Exploratory testing

TDD

Customer User Acceptance

Product owner Product owner/ Tester Tester Tester/Programmer Programmer

Pair, “Show Me”


Exercise

• Think of an experiment to get your whole team engaged in automating tests • Share with your group


Getting Over the Hump §  The test automation pyramid §  The agile testing quadrants §  What should be automated §  What shouldn't §  Difficult areas


Test Automation Pyramid


Test Automation Volcano

From Matt Barcomb


Agile Testing Quadrants


Shared Understanding § Start with tests § Collaborate


What Should We Automate? §  Quadrant 1 tests

§  Unit, component, TDD §  Quadrant 2 tests

§  API, service-level §  Quadrant 4 tests

§  Load, performance, stress §  Quadrant 3 tests?

§  Leverage automation where useful


What Shouldn’t We Automate?

§  Quadrant 2 tests §  Wizard of Oz, prototyping

§  Quadrant 3 tests §  Usability, UAT, ET

§  Tests that will never fail? §  Assess risk

§  ROI not enough §  One-off tests


Where Should We Be Careful?

§  GUI tests §  Watch ROI

§  End-to-End tests §  Push testing down to lowest level

§  Remember the Pyramid


Hard to Automate?

§  Legacy code §  Hard to automate, or just lack of skill? §  “Working Effectively with Legacy Code” –

Feathers §  “Strangling” – Fowler, Thomas


Exercise: Low-Hanging Fruit


Agile Automation Strategy

§  What hurts the most §  Layered approach §  Applying agile principles

§  Small chunks/thin slices §  Smart test design

§  Choosing the right tools


What Hurts the Most §  Use retrospectives §  Keep an impediment backlog


Multi-Layered Approach Example:

§  Learn TDD at unit level §  Automate GUI smoke tests


Simplicity

§  Make the biggest problem 20% better §  Understand the problem first §  Try simplest approach first §  Work in small chunks, thin slices §  Incremental & iterative


Automate a Slice at a Time

Example: 4-step UI to validate, upload profit sharing contribution data

•  Thread 1: All four pages with navigation •  Thread 2: Select year, enter description on page

1, display on page 2, browse and upload file on page 2

•  Thread 3: Validate data in file, display on page 3 •  Thread 4: Persist data, display ‘success’

message on page 4


Thin Slice Example Thin Slice Example


Exercise: Thin Slices

Here’s our user story: As an internet shopper, I want to create an account so that I do not have to enter my address and billing information each time I make a purchase Draw a mind map for this feature on a big sheet of paper Identify a basic end-to-end slice of functionality that can be coded, tested, and automated. If you have time, identify additional slices.

47

Test Design Patterns/Principles §  Patterns, eg, Page

Object §  One clear purpose §  Don’t Repeat Yourself


Page Object Pattern A “page object” is a test object that holds the details of all the elements on a web page that might be involved in an automated test.


Page Object Pattern

§  Test object holds all details of one page §  Hide details and eliminate duplication

Go To ePS Admin Page Click Link eps admin Wait Until Page Contains plan administration Location Should Be ${EPSADMIN URL}


Facilitating Exploratory Testing

§  Templates to set up tests §  Consoles to facilitate visual inspection


Demo


Iterative Feedback

§  Spike two different approaches §  Pick one to try for N # of iterations §  Use retrospectives to evaluate


Learn by Doing §  Courage – don’t be afraid to fail §  Production code practices for test code §  Incremental, thin slices §  Experiment


Questions About Automation Strategy?


Choosing Tools

§  Team effort §  Time §  Requirements §  Focus on goals, problems, not tools. §  Experiment


Understand the Purpose

§  Who’s using the tests? What for? §  What’s being automated? §  Existing tools, environment §  Who’s doing what for automating?


What Fits Your Situation •  Existing skills •  Language of application under test •  Collaboration needs •  What’s being automated •  Life span, future use of tests


Test Drivers/Frameworks §  Divide into sub-teams and prototype

solutions §  Have a “bake-off” showing each prototype


Where To Find Tools

§  http://bit.ly/AgileTestTools §  www.softwareqatest.com/qattls1.html §  www.testingfaqs.org §  www.opensourcetesting.org §  groups.yahoo.com/group/agile-testing §  Grow your own!


Example: One Team’s Tool Choices

•  IntelliJ Idea •  Jenkins, ant, Maven •  JUnit •  FitNesse •  Canoo WebTest •  Watir •  Robot Framework •  JMeter •  Selenium 2.0 / WebDriver with Geb framework


Exercise: Test Frameworks

•  What format might work for your team’s tests? •  Write examples on sticky notes

•  What patterns, frameworks and drivers do you want to learn more about?

•  Place sticky notes next to impediment they might help overcome

•  Discuss how this would help solve problem


Making Test Automation Work

§  Time to do it right §  Experiments, bake-offs §  Testable architecture §  Test data §  Managing tests


Time To Do It Right

§  Limit scope, don’t over-commit §  Write automation task cards §  Quality must be team goal §  Long-term, will let you go faster


Bake-offs

§  Sub-teams/pairs experiment §  Time box §  Pros & cons


Testable Architecture •  Layered architecture

•  eg. UI, business logic, data access •  Ports and Adapters pattern

•  App can work without UI or database •  Ports accept outside events •  Adapters convert for human or automated

users


Test Data

§  Avoid database access when possible §  Setup/Teardown

§  Independent, rerunnable tests §  Canonical data

§  Refresh before each test run §  Customizable data for ET §  Production-like data

§  Get customers to provide example data


Managing Automated Tests

§  Tests as Living Documentation §  Continuous Integration §  Reporting results §  Metrics


Tests as Living Documentation §  Understandable §  Who will really use them? §  Once passing, must always pass


Any Example Can Become a Test


Given/Then/When Example

Scenario: Valid name search returns results GIVEN that Kant is a supervisor with employees AND Kant has an employee named Smith WHEN Kant navigates to the employee name

search page AND enters the value “S” THEN Kant will see a search result that includes

Smith


Continuous integration/testing…

…short feedback loop


Keep tests passing

Stop the line to fix problems


Test Management Tools §  Manage tests, code together §  Some tools have own management §  What problem are you trying to solve?


Exercise: Tests as Documentation

•  Pair up •  Walk around the room and talk about

what approach may work for your team •  BDD? •  Tabular style? •  Examples? •  Something else? •  How will people in different roles

use them?

75

Key Success Factors

Whole Team Approach

Thin Slices

Short Feedback Loops

Experiments

Specification by Example


Exercise: Breaking Barriers


Remember §  It’s a team problem! §  Tackle automation problems with diversity §  Try small experiments §  Start the test automation rebellion with baby

steps


Questions? “Aha” Moments?


Agile Testing: A Practical Guide for Testers and Agile Teams

By Lisa Crispin and Janet Gregory www.agiletester.ca


Experiences of Test Automation

Dorothy Graham and Mark Fewster


Beautiful Testing: Leading Professionals Reveal How They Improve Software

Edited by Tim Riley, Adam Goucher

Includes chapter by yours truly


Test Patterns

Xunit Test Patterns: Refactoring Test Code

By Gerard Meszaros

Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire

83

Specification by Example How successful teams deliver the right software

Gojko Adzic

Case studies from > 50 teams

Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire

84

ATDD by Example A practical guide to Acceptance Test-Driven development

Markus Gaertner


Agile Test Automation Resources

§ dhemery.com/pdf/writing_maintainable_automated_acceptance_tests.pdf § lisacrispin.com § janetgregory.ca § gokjo.net § exampler.com § [email protected] § testobsessed.com § testingreflections.com § www.beer30.org

Making Test Automation Work in Agile Projects 2012

Technology

Transcript of Making Test Automation Work in Agile Projects 2012