20141113 devoxx2014 jochim van dorpe testing in agile

42
@ Smals_ICT #DV14 # AgileContextTesting @Smals_ICT #DV14 #AgileContextTesting Beyond the urban legend Testing in an agile context Jochim Van Dorpe

description

Testing in an agile context - Beyond the urban legend Talk from #Devoxx 2014 on November 13 by @jochimvandorpe on software testing in an agile context? What does it mean for testing methodology when switching from waterfall to agile? Have testers become redundant or rather an essential part of a software development team? How do you keep the pace when functional requirements are changing every two weeks? This presentation is giving no-nonsense advice from hands-on testing experience in a large ICT organisation. #agile # testing #Java #Devoxx

Transcript of 20141113 devoxx2014 jochim van dorpe testing in agile

Page 1: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting @Smals_ICT#DV14 #AgileContextTesting

Beyond the urban legend

Testing in an agile contextJochim Van Dorpe

Page 2: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

In-house ICT shared services for e-government

• Focus on social security and e-health services

• For federal, regional, local & European institutions

• Based in Brussels

• Software development, ICT-operations & staffing

• >1700 people

Introduction: What is Smals?

Page 3: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Introduction: Who Am I?

(QA) ( (lead) (technical) test(er)

analyst/coördinator/manager) (automator/engineer)

I am a tester

Page 4: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Introduction: What do I have to tell?

Page 5: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

What is testing?

It's not about proving that the software works, it's about finding bugs

Page 6: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Why is testing necessary?

To prevent:

• Death

• (Serious) injury

• Loss of business

• Loss of reputation

Page 7: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Why do we test?

Page 8: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

The test project in waterfall

Page 9: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Agility

Agility is the continuous

delivery of prosperity (value) to

stakeholders of a system in a

sustainable and balanced

manner -Schalk Cronjé (@ysb33r)-

Page 10: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Agile fundamentalists are like …

Did you even read

our manifesto?

Page 11: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Agile manifestoOf course …

• Individuals & interactions over

processes and tools

• Working software over

comprehensive documentation

• Customer collaboration over

contract negotiation

• Responding to change over

following a plan

Page 12: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Urban legends

• Agile doesn’t need testers

• Agile = de facto good code, built-in quality

• Testing is dead

• Agile = TDD, ATDD, BDD

• Testers should be able to code

• Agile hasn’t time for testing

• 100% automation

• Faster, shorter, better!

• Developers and Testers are like oil and water

• You only need to unit test

• User Acceptance Testing is no longer necessary

Page 13: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

So…

• The Agile manifesto shouldn’t stand between you &

good quality

• No documentation, few documentation or poor

documentation is no excuse for “not testing”

• Don’t put an Agile stamp on your team, just to avoid

the things you dislike, but find a way of doing them

so you like them …

Page 14: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

No difference at all?

Page 15: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Many ways to be agile …

Page 16: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

… but I’m no salesman

Page 17: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Principles of testing in an agile

context

-Elisabeth Hendrickson (@testobsessed)-

Page 18: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

1. Testing moves the project forward

Shifting focus:

No longer the last line of defense

Page 19: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

2. Testing is not a phase

Integrated & continuous:

develop & test together

Page 20: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

3. QA = responsability of the team

Page 21: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

4. Reduce feedback latency

No long gaps between development

& testing feedback

Page 22: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

5. Don’t let bugs crawl around

Deal with bugs as soon as possible

Page 23: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

6. Test should represent requirements

Don’t ASSUME, it makes an ASS of

U & ME

SCHOOLBUS

Page 24: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

7. Reduce overhead

Think high level: don’t waste time on

too much detail

Page 25: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

8. Don’t believe in fairy tales

If something

seems to good to be true, it probably is

Page 26: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

9. Done means done

Done means implemented and tested

Page 27: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Concrete practices

Page 28: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

A. Automate More

• The testing triangle: This will drown your testers:

Manual tests

Acceptance tests

System &

integration tests

Comp.

tests

Page 29: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

A. Automate More

• The testing triangle: Make sure your testers can

cope with the pace

Component tests

System &

Integration tests

Acc.

Tests

Manual

testsManual tests

Acceptance tests

System &

integration tests

Comp.

tests

Page 30: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

A. Automate Testing, not only tests

Test

execution

ch

ec

kin

g

Load

Data-

sets

Reset

DB

Continuous integration

Logging

resultsGenerate

reports

Live

dashboards

Calculating

‘metrics’

Use the tools that fit your purpose

AND NOT THE OTHER WAY AROUND!

Page 31: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

B. Explore

Leave some room for

exploratory testing

Page 32: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

C. Diversify

• …in the test levels

• Unit-tests only won’t find

everything

• System test only, and you’ll

drown

• … in the way of testing

• Automation is good, automation

is fun, but in the end, an

automated test isn’t better than a

human based test (manual test)

Page 33: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

D. Test the tests

• GREEN == GOOD ?

• DO: Review tests

• DONT: No test coverage fairy tales

• DONT: Mock lasagna

• DONT: I should adapt the test cases so it’s green again

Page 34: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

E. Crowdsource …

• Crowdsource in your team to find

the good & passionate tester

• Accept that there are learning curves

• Testing is a craftmanship

• A passionate tester will tend to be more

effective than the chinese-volunteer-

tester

• Encourage collaborative testing

Page 35: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

… or ask a testjumper

• Ask a ‘test jumper’ to join the

team

• May spend some hours as a consultant

• Or months as an ordinary tester

• Coaches or contributes himself

• Encourages testability

• Help the developers think

productively about testing

Page 36: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

The test project in agile

Page 37: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

What should the PL not expect?

• Metrics such as:

• # of bugs found

• Bugs per functionality

• Bugs per 1000 lines of code

• DDP (defect detection percentage)

• We can however show our QA-efforts

• QA-survey

• Test case progress

• Fixed test cases

Page 38: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

What should the PL not expect?

• Qualification (good / bad) of the devs

• 100% green charts

• Upfront fixed number of test cases

• Automated self-generating auto-code-correcting

tests

Page 39: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

conclusion

There are no agile testers,

but we need testers who

adapted their mindset to an

agile context

Page 40: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Read & experience more 1/2• Books:

• Lisa Crispin, Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and

Agile Teams

• Lisa Crispin, Janet Gregory (2014), More Agile Testing: Learning Journeys for the

Whole Team

• Blogs:

• Huibschoots.nl

• LisaCrispin.com

• http://testobsessed.com : Agile testing overview

• Http://www.satisfice.com : Test jumpers: one vision of Agile testing

• pascaldufour.wordpress.com

• agile-and-testing.chriss-baumann.de

Page 41: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Read & experience more 2/2• Presentations:

• 40 agile methods in 40 minutes by Craig Smith

• Agile testing quadrants explained by @RubyTester

• Certification:

• CAT-training: Certified Agile Tester

• ISTQB foundation level add-on: Agile tester

• Conferences:

• Agile testing day(s)

• More references:

• Word clouds generated on wordle.net

• Images found with google image search

Page 42: 20141113 devoxx2014 jochim van dorpe testing in agile

@ Smals_ICT#DV14 # AgileContextTesting

Questions