Practical Agile QA

A practical guide to implementing Agile QA process on Agile Projects

Agile QA

Syed RayhanCo-founder, Code71, Inc.Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com

http://blog.syedrayhan.com/

www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.

2

My Background

Expertise

Career

Iterative incremental development

Technology planning and architecture

On-shore/Off-shore software development using Agile/Scrum

Interests

Co-founder, Code71, Inc. 14+ years of total experience Co-author of “Enterprise Java with UML”

Cultural aspect of self-organizing team Scrum for projects delivered remotely Agile engineering practices

http://www.code71.com/

http://www.scrumpad.com/


3

Agenda

Recap

Individual Practices

Section 2

Section 4

Section 3

A Case Study

Holistic View of QA

Section 5

IntroductionSection 1

Q&ASection 6




4

What to Expect

Focus

Context

Build common base of understanding

Develop a set of guidelines- process, roles, and team composition

Address typical questions asked

K ey Takeaways

How to perform QA on an Agile/Scrum project

Agile/QA best practices

Teams and organizations are adopting Agile/Scrum

Teams struggle with making the transition from waterfall to

Agile/Scrum




5

Agenda

Recap


Section 2

Section 4

Section 3

A Case Study

Holistic View of QA

Section 5


Q&ASection 6




6

The challenges?

Is QA part of the development team?

Can we fit QA in the same iteration as development?

How can we scale Agile QA?

Who does QA?

Does QA costs more in Agile as product seems to change from sprint to sprint?

Do we need “test plan”?

Are story acceptance tests enough? When do we know testing is done?

Who defines test cases?

Do we need to track bugs?




7

What is QA (Quality Assurance)?

To ensure Software is working right

How?

Test, Test, Test

We will primarily focus on single-team model for our discussion




8

Types of Testing?

Unit Testing

Integration Testing

Functional Testing

System Testing

Regression Testing

Acceptance Testing

Load Testing

Smoke Testing

Black Box

White Box




9

Who Performs What?

Unit Testing

Integration Testing

System Testing

Regression Testing

Acceptance Testing

Load Testing

Smoke Testing

What? Who? When? Automation?

Developer

Developer

Tester

Developer/Tester

Client/Users

Tester/Support Engineer

Performance Engineer

Coding

Coding

Test

Build/Test

Deployment/Delivery

Deployment

Deployment

Always

Always

Possible

Possible

Possible

Possible

Always




10

Right tools for right tests?

Unit Testing

Integration Testing

System Testing

Regression Testing

Acceptance Testing

Load Testing

Smoke Testing

Test Tool NUnit, JUnit, Mock, DBUnit

Unit test tools, HttpUnit, SoapUI, RESTClient

JMeter, Httperf

FIT, FitNesse

Selenium, Fit, WET, Watir, WatiN

Unit test tools, System test tools

Regression test tools




11

What is missing?

Assumptions

1. Right Spec

2. Right Design

Reality?

Measures

Spec review

Design review

Spec

Design

Code reviewCode

3. Right amount of Tests

Test CoverageTest

Assumptions are farther from truth

4. Right Tests




12

Team Composition?

1Dev

Test

2 3

1 2 3

iterationregion

Prod 1 2 …

1Dev

Test

2 3

1 2 3

iterationregion

Prod 1 2 …

Out-of-Cycle

In-Cycle

2

1

Integrated Team

Separate Team

Developer to tester ratio?




13

Quality Funnel

QG#4QG#3QG#2QG#1 QG#5

bugs

bugs

BacklogReview

DesignReview

Unit & Int.Test,

CI

CodeReview

SystemTest




14

Agenda

Recap


Section 2

Section 4

Section 3

A Case Study

Holistic View of QA

Section 5


Q&ASection 6




15

Test Coverage

Tests coverage metrics can tell you what code is not tested

“A measure of the proportion of a program exercised by a test suite, usually expressed as a percentage.”

• Function coverage

• Path coverage

• Statement coverage

Types of coverage

Definition

Measure Usually expressed as a percentage




16

Continuous Integration as the Glue

Monitor Source Code

AutomatedBuild

RegressionTest

Test Coverage

Report

Continuous Integration is Continuous QA




17

Test Case Prioritization

high

high

low

low

1

2

Frequency of use

Risk of having bugs

3

4

hot

warm

normal

cold




18

“Inspect and Adapt” through QA Lens

Five “whys” of root cause analysis

Prioritize bugs over stories

Log bugs found by testersDiscover

Triage & fix

Prevent




19

Definition of “Done”

Is QA part of your definition of “Done?”




20

Tracking Quality

Quality Metrics

Defect Rate Bug count per iteration

Defect Density Bug count per module Bug count per function point

What else should we track?




21

Technical Debt & QA

“Aged” open bugs can contribute to increasing “technical debt”

“Reoccurring” bugs may indicate hidden “technical debt”




22

Agenda

Recap


Section 2

Section 4

Section 3

A Case Study

Holistic View of QA

Section 5


Q&ASection 6




23

A Case Study

ProjectA large enterprise system that includes technologies like ASP.Net, BizTalk, Workflow, Scanning, SQL Server, Data Warehouse, and Mainframe

Team2 product owners, 1 scrum master, 1 architect, 5 developers, 1 QA tester

Sprint 2 weeks

QA Process Refine scope, acceptance test cases

Identify UI elements, test dataQA schedule

Write test scriptstest data

Day 1, 2 Day 3, 4 Day 5-8 Day 9 Day 10

FinalTest&Fix

Demo&Acceptance Test




24

Agenda

Recap


Section 2

Section 4

Section 3

A Case Study

Holistic View of QA

Section 5


Q&ASection 6




25

Recap “In-cycle QA” or “Integrated Team” is critical to the success of a project

System testing is not the only “quality gate,” it includes all types of testing and

reviews

Test automation is critical to “in-cycle” QA

Target at least 90% test coverage

“Continuous Integration” is “Continuous QA”

Prioritize test cases based on risk and frequency of usage




26

Recap contd.

All known bugs should be fixed first

Right size story with well-thought out acceptance tests improves quality

Include all “Quality Gates” as part of definition of “Done”

Analyze each bug to understand where (Quality Gate) it should have been

caught and improve (Inspect and adapt)

QA is not a designated person’s responsibility, it is a team’s responsibility

(self-organizing team)




27

Q&A

“QA is not an act, but a habit”

“QA is making sure right software works right”

Please contact for on-site training or Webinar:

Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com



Practical Agile QA

Technology

Transcript of Practical Agile QA