QA in an Agile World

Tom Churchwell

TChurchwell@GMail.com

Agenda

2

● Role Setup (5 Minutes)

● Exercise 1 (2 Minutes)

● Agile QA In 3 Easy Steps (20 Minutes)

● Exercise 2 (15 Minutes)

● Q&A/Retro (5 Minutes)

Role Setup

3

● Product Owner

● Architect

● QA

● Everyone Else is Developers

Exercise 1

4

Stand Up

Stretch your arms above your head

Fold your arms

Are you left arm over right or right arm over

left?

Exercise 1

5

Stretch your arms above your head

Fold your arms the opposite way

1. Could you do it?

2. Is it comfortable?

Change is uncomfortable. Be compassionate.

Team/Organization Prerequisites

1. Teamful Practices

a. Customer is the locus of concern

b. Teams Survive Beyond Projects

c. Self-Organizing, Autonomous, Craftspeople

d. Coordination, Communication, Collaboration

i. Co-Located - High Trust

2. Craftsmanship/Continuous Improvement Practicesa. Retro’s

b. Exploratory Testing

c. Code Kata’s/WISE/Lunch’N’Learns/etc...

Getting To Agile QA in 3 Easy Steps

1. Establish Quality Standards and a Definition of

Done

a. DOD for each phase of delivery

i. Story Done

ii. DEV Done

iii. Quality (Not Necessarily QA) Verified

b. Empower Team Ownership of Quality

c. Move Escalations/Discovered Defect to the top

of the Queue

Getting To Agile QA in 3 Easy Steps

2. Gather Metrics and make them prominent

a. Code Quality

b. Delivery Velocity

c. Defects

d. Performance

e. Others….

Not A Bludgeon!

Make it easy for people to do the right things

Getting To Agile QA in 3 Easy Steps

3. Automate Everything

a. Testing

i. All Possible Layers

b. Builds

c. Releases

What Is Different?

10

● Big Definition Up Front vs Emergent Design

● Adaption vs Prediction

● Short vs Long Iterations

● Short vs Long Customer Feedback Loops

● Working Software Baked In vs Tested In

● Whole Team Quality Ownership vs QA

● Automated vs Manual (Test, Build, Release)

● And so much more….

BDUF - Big Definition Up

Front

11

Adaption vs Prediction

12

● Tradition tries to understand how things will work and

defines architecture up front

● Tradition makes change hard to accommodate.

● Agile embraces emergent design and adapts as a system

evolves.

● Agile embraces change.

Last Responsible Moment

13

Long vs Short Iterations &

Customer Feedback Loops

14

● Agile anticipates releases to PROD more and more

often…sometimes even multiple times a day

● Testing starts earlier, happens more often and relies almost

entirely on automation to include build automation

● Agile wants feedback weekly at the BV Demo

Otto The Autopilot

15

Working Software & Quality Ownership

16

● Tradition uses QA to verify functionality after development

● Traditional relies on QA to test and verify and “Own”

quality and as a result hands responsibility for quality

over to QA

● Test Drive

● Automate Testing

● Verify working software during and after development

● Bake Quality In

Bake Quality In…

We Cannot Improve Our Recipe’s By Eating More

Cookies…

We Must Bake, Rather Than Try To Test Quality In!

Testing Pyramid

19

Agile Does Not Move At Manual Testing

Speed

Automation (Test, Build, Release)

20

● Is anyone still testing manually?

● The slow manual testing practices of traditional QA will not

be sufficient to meet the new pace of iterative development

● If a full manual testing cycle is more than a week, then a

new testing cycle is needed as soon as the last deployment is

complete

Getting To Agile QA in 3 Easy Steps

Culture of Quality

Organic Not Mechanical

Team Confidence is the Goal

Swagger is Good!

1. Establish Quality Standards and a Definition of

Done

2. Track and Make Metrics Prominent

3. Automate Everything

Team Confidence

22

● Confidence in the codebase is the key to:

● Generating

momentum

● Having choices

● Velocity

● Autonomy

● Adaptability

● Competitive

advantage

● And the inherent

propensity for

innovation and

luck

Traditional Team Ownership of Quality

● Historically QA has been:◦ An event that occurs after development is done

⚫ Point in time validation rather than an ongoing standard for quality

⚫ Verification before production launch

◦ Held by QA ⚫ Not part of “Definition of Done” for the team

⚫ Not part of what every member of the team was committed to uphold.

◦ Part of a rigid sequential process⚫ At the end of development

⚫ An event rather than an ongoing concern

Whole Team Ownership

● Quality as a fundamental ongoing concern for the team◦ Not just at the end of a release

◦ Meeting quality standards has become a part of the “Definition of Done” for the whole team

● An Integrated Team Focus◦ Teams take ownership

⚫ Definition of Done

⚫ Quality Standards

⚫ Level of quality being produced every day

● Business Verification◦ Weekly verification demo

◦ Product Owner Drives the Demo

Exercise 2

25

Review Designs for Feasibility (2 Min)

Iteratively:

1. Estimate Production (1 Min)

2. Develop/Quality Check Plane(s) (4 Min)

3. Retro & Metrics ( 2 Min)

2 Iterations

Report outs

Exercise 2 Success Criteria

26

● 1 Plane must gently fly 18 feet

● Plane must have a 5 point star on the

underside of each of the wings

● Plane cannot have any markings on the

topside of the wings

● Team must capture:

◦ Estimated Production vs Actual Production

◦ Defects Found (Pre-Dev, Dev, Testing)

Questions● Did you feel constrained to use a design

from the packet?

● Did the team own quality or did the QA

role?

● Did you capture accurate metrics?

● Did you make more than one plane? (Over-

achievers)