Testing Does Not Equal Quality
-
date post
22-Oct-2014 -
Category
Technology
-
view
1.448 -
download
4
description
Transcript of Testing Does Not Equal Quality
![Page 1: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/1.jpg)
Testing != QualityRethinking quality and how to achieve it in an agile world
Bob HartmanRichard Lawrence
![Page 2: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/2.jpg)
“How does QA fit into all of this?”
![Page 3: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/3.jpg)
are conflicting goals
Speed/value
quality
and
![Page 4: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/4.jpg)
Or are they?
![Page 5: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/5.jpg)
How can quality enable speed and value?
What is quality?
How do we ensure our software has it?
![Page 6: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/6.jpg)
What is quality?
![Page 7: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/7.jpg)
The goal of a software org:Produce valuable software now and in the future.
![Page 8: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/8.jpg)
A feature can be modeled with a cumulative cash flow chart
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09
Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
-$300,000
-$200,000
-$100,000
$0
$100,000
$200,000
$300,000
$400,000
$500,000
$600,000
![Page 9: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/9.jpg)
Or, we can remove the investment and just look at value flow
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 10: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/10.jpg)
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
Value flow for a feature that never works right
![Page 11: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/11.jpg)
Value flow for a feature that fails for some users some of the time
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 12: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/12.jpg)
Value flow for a feature with downtimes
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 13: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/13.jpg)
Value flow for a feature that doesn’t scale
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 14: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/14.jpg)
Quality is the foundation of value
![Page 15: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/15.jpg)
A team’s productivity can be measured with a velocity chart
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 16: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/16.jpg)
Velocity of a team with a well-designed, high quality system
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 17: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/17.jpg)
Velocity of a team with a poorly-designed, low quality system
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09 Aug-09
Sep-09
Oct-09
Nov-09
Dec-09
![Page 18: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/18.jpg)
Quality is the foundation of speed
![Page 19: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/19.jpg)
Photo credit: woodsy on sxc.hu
Quality is what ensures: Value now and in the future
![Page 20: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/20.jpg)
How do we ensure our software has it?
![Page 21: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/21.jpg)
Testing at the end doesn’t workIteration Start Iteration End
The Plan
The Reality
Plan Dev Test
![Page 22: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/22.jpg)
What’s really going on?
The Plan
The Reality
Dev Test
Dev Test
Dev expands to
fill the available
time
Dev to test is really a loop
![Page 23: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/23.jpg)
Build quality in vs. inspecting for it at end
Inspect for quality Dev TestPlan
Build quality in
![Page 24: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/24.jpg)
Manual testing doesn’t scale
Sprint 1 Sprint 2 Sprint 3 Sprint 4
New feature testing
Sprint 5
Testing that doesn’t happen,
but should
(or
happens
in
overtime)
Regression testing
Testing capacity
![Page 25: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/25.jpg)
Automate and get regression tests for free
Sprint 1 Sprint 2 Sprint 3 Sprint 4
New feature testing
Sprint 5
Automated tests that are now
regression tests
Testing capacity
![Page 26: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/26.jpg)
What is really being tested?
1 2 3 4 5 Total1 5
2 4
3 3
4 2
5 1
IterationsFe
atur
es
![Page 27: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/27.jpg)
Which features need to work best?
Never Used45%
Rarely19%
Sometimes16%
Often13%
Always7%
Source: The Standish Group
Usage of features in a typical software product
![Page 28: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/28.jpg)
But without regression it is different
1 2 3 4 5 Total1 1
2 1
3 1
4 1
5 1
IterationsFe
atur
es
![Page 29: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/29.jpg)
Minimal regression is a little better
1 2 3 4 5 Total1 3
2 3
3 3
4 2
5 1
IterationsFe
atur
es
![Page 30: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/30.jpg)
But what will happen where we don’t have regression tests running?
1 2 3 4 5 Total1 3
2 3
3 3
4 2
5 1
= Missed tests, which Murphy’s Law says will lead to defects being missed!
IterationsFe
atur
es
![Page 31: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/31.jpg)
Build and deploy processes can kill speed
Define acceptance criteria and automate
tests
Develop Package Deploy to test
env
Test
DONEThe Plan
The Reality Loops, waits, errors…more time
![Page 32: Testing Does Not Equal Quality](https://reader033.fdocuments.in/reader033/viewer/2022061109/54478f35afaf9f16708b45bd/html5/thumbnails/32.jpg)
To ensure quality in an agile world:
• Move testing to the front of the process• Collaborate• Automate tests (using agile-friendly tools)• Automate/accelerate build and deploy• Work on user stories one at a time, all the way
to DONE