Test automation and Agile

software development

Bas Dijkstra

bas@ontestautomation.com

www.ontestautomation.com

@_basdijkstra

Who are you?

Who am I?

I help teams and organizations to implement automation

Blogs (ontestautomation.com, TechBeacon, StickyMinds)

Trainer (test automation, service virtualization)

Test automation

experience?

Test automation

experiences?

What whould you like

to have learned after

tonight?

What are we going to

do?

Role of and realism about test automation

Principles for automation

Demos, examples, case studies, requests

Test automation strategy

How?

Why?

What?

Why test automation?

Why test automation?

Does not replace testers and testing

Does not save you money (at least not from the start)

Shorten feedback loop

Free time for other activities

Support testing activities

Can you do without

test automation?

What kind of tests

are you going to

automate?

Highly automatable

Verifications (does A equal B?)

Correctness of algorithms and calculations

Deterministic tests

Machinal activities

Not automatable

Validations (is this the behavior we want?)

User experience

Tests that have an open answer

Human activities: experimentation, evaluation, …

There is no “manual testing”; there is

testing

There are no “manual testers”; there

are testers

Checking —an element of testing, a

tactic of testing— can be automated

http://www.developsense.com/blog/2017/11/the-end-of-manual-testing/

--- Michael Bolton ---

“You should automate

100% of the tests

that should be

automated”

--- Alan Page ---

“Don’t become the

world’s best

automator of useless

checks”

--- Bas Dijkstra ---

How do you automate

your tests?

Scope

Unit tests

Integration tests

End to end tests

Nonfunctionals (performance, security, …)

Test automation pyramid

Unit

Integratie

E2E

_ Customer journeys

_ (Often) user interface-driven

_ ‘Ultimate integration test’

_ Layer / component integration

_ Data / business logic exposure

_ (Often) API-driven

_ ‘The forgotten layer’

_ Component level

_ Code driven

_ Mocks and stubs

Tools and tool selection

In the end, the tool

is not important

How does automation

fit into Agile?

Testing often > short feedback loop required

It all starts at the basis (unit tests)

Too many E2E tests too soon > lots of maintenance

Avoid the user interface whenever possible

Test automation

in the DoD

Do all features delivered need to be tested?

Do they all require a full set of automated tests?

100% code coverage?

Do you need to include automation in the DoD in the

first place?

https://leanpub.com/testingindevops

Case

Online banking application

Reasonably mature

Testing everything every sprint takes too long

Proposal: use test automation

Demo

ParaBank(Application under test)

Case

_Craft an automation implementation strategy

_What do you need? Which questions do you have?

_What would you do in the first sprint?

_What would you do in the second sprint?

_Present your strategy to your coworkers

And what about…

Test data?

Test environments?

Repeatability?

Reusability?

Planning?

Who?

Features or tests?

How do you become a

good automation

engineer?

A good automation engineer…

Has experience in software testing

Has experience in software development

Asks ‘why?’ before asking ‘what?’ or ‘how?’

Creates realistic expectations

Why do things still go wrong?

Testers are expected to do automation ‘on the side’

Features get precedence over tests

Inefficient automation (automate it all,

automate everything through the UI, …)

Lack of knowledge and experience (craftsmanship)

Automated Testing

Continuous Testing

Automated tests

Focused

Informative

Trustworthy

Repeatable

Continuous Testing

Focused

Write tests at the right level

As close to the implementation as possible

Lower level > narrower scope

Lower level > faster to write, faster to run

Informative

Your automation should provide useful feedback

Know who the recipients of this feedback are

Reporting (for humans, for systems)

Clean, readable and maintainable code

Clear and unambiguous messages

Trustworthy

Decisions are made based on test results

You should therefore be able to trust those results

Stable test execution

Eliminate false positives

Eliminate false negatives

Repeatable

Test data management (this is hard!)

Adopt the right strategy for your tests

Test environments and dependencies

Mocks, stubs, service virtualization

Test automation

… is a craft in and of itself, done by craftspeople

… is not something you do ‘on the side’

… can shorten the feedback loop in Agile development

… supports testers, does not replace testers

Contact

_Email: bas@ontestautomation.com

_Blog: https://www.ontestautomation.com

_LinkedIn: https://www.linkedin.com/in/basdijkstra

_Twitter: @_basdijkstra