Acceptance Test-driven Development: Mastering Agile Testing
25
MN PM Tutorial 4/29/13 1:00PM Acceptance Test-driven Development: Mastering Agile Testing Presented by: Nate Oster CodeSquads, LLC Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
-
Upload
techwellpresentations -
Category
Technology
-
view
170 -
download
1
description
On agile teams, testers often struggle to “keep up” with the pace of development if they continue employing a waterfall-based verification process—finding bugs after development. Nate Oster challenges you to question waterfall assumptions and replace this legacy verification testing with Acceptance Test-driven Development (ATDD). With ATDD, you “test first” by writing executable specifications for a new feature before development begins. Learn to switch from “tests as verification” to “tests as specification” and to guide development with acceptance tests written in the language of your business. Get started by joining a team for a simulation and experience how ATDD helps build in quality instead of trying to test out defects. Then progress to increasingly more realistic scenarios and practice the art of specifying intent with plain-language and table-based formats. These paper-based simulations give you meaningful practice with how ATDD changes the way you think about tests and collaborate as a team. Leave empowered with a kit of exercises to advocate ATDD with your own teams!
Transcript of Acceptance Test-driven Development: Mastering Agile Testing
MN PM Tutorial
4/29/13 1:00PM
Acceptance Test-driven
Development: Mastering Agile Testing
Presented by:
Nate Oster
CodeSquads, LLC
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Nate Oster
An agile player-coach and founder of CodeSquads, Nate Oster helps clients adopt lean and agile methods. Nate builds high-performance teams that adapt to change, embrace a pragmatic philosophy of continuous improvement, measure progress with new features, and deliver high-quality software that delights customers. As a coach, he inspires adopters with hands-on mentoring and simulations that provide a safe learning environment for new ideas. Nate promotes testing as a serious technical discipline. While Nate maintains a speaking schedule at software development conferences, he spends most of his time as a hands-on advisor to software product teams from startups to multinationals. Nate is easily defeated at all card games. Contact him at [email protected].
ACCEPTANCE TEST-DRIVEN DEVELOPMENT: MASTERING AGILE TESTING
Nate Oster
NATE OSTER
Agile Player-Coach & Founder, CodeSquads
Focused on adopting Lean & Agile methods
Mail: [email protected]
Twitter: nateoster
Voice: 703.930.4100
LEARNING OBJECTIVES
Stop finding bugs after development!
Shift from tests as verification to tests as specification by learning to test first.
Simulate the feel of ATDD in a non-threatening way
Gain meaningful practice with table-based and plain-language formats for specifying intent.
Leave empowered to advocate ATDD with your own teams and managers!
AGENDA
Agile Testing has limits
Simulation: Tests as Verification
Intro to ATDD
Simulation: Tests as Specification
AGENDA
Specification by Example
Simulation: Examples using Tables
Break!
Simulation: Examples using Given-When-Then
Wrap-up: Action Planning, Q&A
THE PROBLEM
Finding defects is expensive
Rework, Scrap, Delays
Feedback speed is limited
SIMULATION: TESTS AS VERIFICATION
SIMULATION: TESTS AS VERIFICATION
Teams of 3
“Develop” an image by punching graph paper
“Test” your solution by fitting it over pegs
Satisfy your Customer!
RETROSPECTIVE: TESTS AS VERIFICATION
ACCEPTANCE TEST-DRIVEN DEVELOPMENT
Stop testing bugs out. Start building quality in.
Technology-facing Tests
Business-facing Tests
Test
s th
at s
uppo
rt t
he T
eam
Tests that critique the product
“Agile Testing,” Lisa Crispin & Janet Gregory, used with permission (originally Brian Marick, 2006)
Exploratory Tests Usability Testing
UAT
Unit & Component Tests Performance Testing Security analysis
“-ility” tests
Acceptance Tests
Agile&Tes)ng&Quadrants&Mostly
Automated Mostly Manual
Automated Frameworks
Specialized Tools
The$Test$Automa,on$Pyramid$
More business facing, realistic
Lower-cost Easier maintenance Faster feedback Acceptance Tests
(API Layer)
GUI
Unit & Component Tests
Exploratory /Manual
xUnit
FitNesse/ Cuke
Adapted from Mike Cohn (Automated Test Pyramid) & “Agile Testing”, Lisa Crispin & Janet Gregory..
Sn
THE TEST AUTOMATION PYRAMID
Push tests as low as possible in the pyramid
Have tests focus on intent, not implementation
Team decides appropriate level for each test
Anatomy(of(an(Itera-on(
Define(
Build( Test(
2(–(4(week(-mebox(
Define(
Build( Test(
Define(
Build( Test(
Define(
Build( Test(
Define(
Build( Test(
Define(
Build( Test(
Define(
Build( Test(
Define(
Build( Test(
Itera-on(Planning(2@4(hours(
Retrospec-ve(Itera-on(Dem
o(2@4(hours(2(hours(
a(few(days(
a(few(hours(
Itera-on(Planning…(
next(day…(
Feature(Team(
Nate%Oster,%“The%Agile%Challenge”%Workshop,%CodeSquads%LLC.%
SIMULATION: TESTS AS SPECIFICATION
SIMULATION: TESTS AS SPECIFICATION
Start each story by outlining the acceptance criteria
Specify acceptance tests as concrete examples
If we don’t know when we’re done, then why are we starting?
Hypothesize the top 3 effects of this change
RETROSPECTIVE: TESTS AS SPECIFICATION
Reflect:
Where your hypotheses correct?
Why or why not?
Prepare to Share:
Appoint a scribe
Share your results at your tables
List the top improvements
SPECIFICATION BY EXAMPLE
“An example would be helpful right about now.” - Brian Marick
Essential Incidental
Hat tip: Dale Emery
Technology-facing Tests
Business-facing Tests
Test
s th
at s
uppo
rt t
he T
eam
Tests that critique the product
“Agile Testing,” Lisa Crispin & Janet Gregory, used with permission (originally Brian Marick, 2006)
Exploratory Tests Usability Testing
UAT
Unit & Component Tests Performance Testing Security analysis
“-ility” tests
Acceptance Tests
Agile&Tes)ng&Quadrants&Mostly
Automated Mostly Manual
Automated Frameworks
Specialized Tools
Current'Itera*on'
Itera*ons'
Current''Release'
Future'Releases'
+/2'?'
?'
We'refine'the'Product'Backlog'just2in2*me'for'immediate'goals.'
Next'Release'(forecast)'
Future'Roadmap…'
+/2'+/2'+/2'
Vision'&'Roadmap'Release'Planning'
Itera*on'Planning'
Es*mate'Accuracy'
Near'Term:'Fine2Grained' Long2Term:'Coarse2Grained'
Nate%Oster,%“The%Agile%Challenge”%Workshop,%CodeSquads%LLC.%
The$Test$Automa,on$Pyramid$
More business facing, realistic
Lower-cost Easier maintenance Faster feedback Acceptance Tests
(API Layer)
GUI
Unit & Component Tests
Exploratory /Manual
xUnit
FitNesse/ Cuke
Adapted from Mike Cohn (Automated Test Pyramid) & “Agile Testing”, Lisa Crispin & Janet Gregory..
Sn
SOME TOOLS FOR EXECUTABLE SPECIFICATIONS
TOOL TECHNOLOGY SWEET SPOT
FitNesseJava (native)
.NET (plugin)
CucumberRuby (native)
Java (CucumberJVM).NET (plugin)
SpecFlow .NET (native)
SIMULATION: SPECIFICATION BY EXAMPLE
USING TABLES
TABLE-BASED SPECIFICATIONS
(Re)-build a brokerage pricing engine
Start with the simplest scenarios using realistic examples
Be customer-focused!
Trade Source Ticker Buy/Sell Order Type Commission?
Internet LXK Buy Limit $9.99
Internet LXK Sell Limit $9.99
IVR
Broker
Pricing.Stocks
Trade Source Ticker Buy/Sell Order Type Commission?
Internet LXK Buy Limit $9.99
Internet LXK Sell Limit $9.99
IVR LXK Buy Market $34.99
Pricing.Stocks
[expected “$34.99”, actual “$9.99”]
Pricing.MutualFunds.Buy
Test FundsTicker Fund Type Load %
MNTF NTF N/A
MNLD No-Load N/A
MFEE Load 4% on Buy
Pricing. MutualFunds. NTF. ShortTermRedemption
No Transaction Fee Mutual Funds
180 days or less
No commission
$49.99 fee
Ticker Fund Type Load %
MNTF NTF N/A
MNLD No-Load N/A
MFEE Load 4% on Buy
WHAT IF I BUY TWICE IN 180 DAYS?
TABLES MAY NOT BE EXPRESSIVE ENOUGH FOR SOME SCENARIOS
What if I buy the same fund more than once in 180 days?
“An example would be helpful right about now.” - Brian Marick
What if I sell when only part of the balance is 180 days old?
SIMULATION: SPECIFICATION BY EXAMPLE
USING TEST LINGOS**Credit to Patrick Wilson-Welsh and Dawn Cannon
for coining the term!
LINGO-BASEDSPECIFICATIONS
Given - When - Then
GIVEN - Preconditions
WHEN - Action
THEN - Result / Postcondition
THINKING WITH GIVEN-WHEN-THEN
Pricing. MutualFunds. Buy
Scenario: Buy a no-load fund.
Given I do not own fund MNLDWhen I buy $10,000 in fund MNLDthen the commission is $49.99 and the fee is $0.00.
When I buy fund MNLD,then the commission is $49.99.
Test FundsTicker Fund Type Load %
MNTF NTF N/A
MNLD No-Load N/A
MFEE Load 4% on Buy
Pricing. MutualFunds.Buy
Scenario: Buy a mutual fund.
GIVEN-WHEN-THEN CAN BE MORE EXPRESSIVE THAN TABLES
What if I buy the same fund more than once in 180 days?
“An example would be helpful right about now.” - Brian Marick
What if I sell when only part of the balance is 180 days old?
Pricing. MutualFunds. NTF. ShortTermRedemption
No Transaction Fee Mutual Funds
180 days or less
No commission
$49.99 fee
Ticker Fund Type Load %
MNTF NTF N/A
MNLD No-Load N/A
MFEE Load 4% on Buy
Mutual Fund Short Term Redemptions
No transaction fee (NTF) funds held 180 days or less are subject to a Short Term Redemption Fee of $49.99.
Scenario: Part of sale held less than 180 days charges short term redemption fee.
Given I own 100 shares of MNTF for 181 days and I own 100 shares of MNTF for 179 daysWhen I sell 101 shares of MNTFThen I am charged a short term redemption fee of $49.99.
Scenario: ...
Scenario: Part of sale held less than 180 days charges short term redemption fee.
Given I own MNTF |shares| days held | | 100 | 181 | | 100 | 179 |
When I sell 101 sharesThen I am charged a short term redemption fee.
NEW FEATURE: CONCIERGE LEVEL PRICING
Concierge Level pricing is a trial promotion.
It offers selected customers free internet trades if their total assets exceed $250,000 or if a trade is for more than $10,000.
Brainstorm acceptance tests for this feature using tables, given-when-then, or both. Your choice!
Feature: Concierge pricing promotion.
Background:Given the concierge pricing promotion is active.
Scenario Outline: Buying in a selected account.Given a selected account with a balance of <total assets>,when I buy <trade amount> of <ticker> using source <trade source>, then the commission is <fee>.
Examples:| total assets | trade amount | ticker | trade source | fee || | | | | || | | | | || | | | | |
REVIEW
WRAP-UP: CREATE AN ACTION PLAN
What outcomes could ATDD improve on your Feature Team?
What impediments to adopting ATDD do you anticipate on your team?
Is there evidence these impediments can be overcome?
What small experiments will confirm or disprove these opportunities with the least possible risk?
Learn More:
Specification by Example (Gojko Adzic)
The Cucumber Book(Aslak Hellesoy)
Agile Testing(Lisa Crispin & Janet Gregory)
EXPLORING FURTHER
Experiment:
“Pull” in support through your Team’s retrospectives
Feedback improvement ideas
Experiment:
“Pull” in support through your Team’s retrospectives
People & Interactionsover Processes & Tools!
NATE OSTER
Agile Player-Coach & Founder, CodeSquads
Focused on adopting Lean & Agile methods
Mail: [email protected]
Twitter: nateoster
Voice: 703.930.4100
