Functional testing patterns
-
Upload
premanand-chandrasekaran -
Category
Technology
-
view
2.140 -
download
4
description
Transcript of Functional testing patterns
![Page 2: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/2.jpg)
Types of Tests Business Facing
Technology Facing
Supp
ort P
rogram
ming C
ritique Product
Showcases Exploratory Tests Usability Tests
Performance Tests Stress Tests Security Tests
Functional Tests Acceptance Tests
Unit Tests Component Tests System Tests
2
h0p://www.exampler.com/old-‐‑blog/2003/08/21/#agile-‐‑testing-‐‑project-‐‑1
© 2012 ThoughtWorks, Inc.
![Page 3: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/3.jpg)
Functional Tests -‐‑ The Pitch • What Are Functional Tests?
o Verification that business requirements are met
o Black Box o Automated
• Why Functional Tests? o Maintain (high) external quality o Allow team to be bolder o Allow team to go faster
3 © 2012 ThoughtWorks, Inc.
![Page 4: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/4.jpg)
Functional Tests – The Reality • Flaky • Take long time to run • Cumbersome to maintain • Don’t catch many bugs • Some things are hard to automate • Don’t quite give you the feedback
you were hoping for 4 © 2012 ThoughtWorks, Inc.
![Page 5: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/5.jpg)
Consequences • Low confidence in automation • Over-reliance on manual testing
• Slipped Deadlines AND/OR • Lesser Testing AND/OR • Testing in Production :-)
5 © 2012 ThoughtWorks, Inc.
![Page 6: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/6.jpg)
How To Get Be0er?
6
People
Process
Technology
© 2012 ThoughtWorks, Inc.
![Page 7: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/7.jpg)
Process
7 © 2012 ThoughtWorks, Inc.
![Page 8: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/8.jpg)
Functional Test – Example
© 2012 ThoughtWorks, Inc. 8
![Page 9: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/9.jpg)
Example -‐‑ Specification
9 © 2012 ThoughtWorks, Inc.
![Page 10: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/10.jpg)
Example – Implementation
10 © 2012 ThoughtWorks, Inc.
![Page 11: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/11.jpg)
Implementation Problems? • No correlation between specification
and implementation • Intent lost in translation • Easy to write, hard to read, maintain • Too monolithic, lacks abstraction
• Causes non-technical domain experts to tune out of the process
11 © 2012 ThoughtWorks, Inc.
![Page 12: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/12.jpg)
Executable Specifications
12
Specification Step
Implementation Step Tight Correlation
© 2012 ThoughtWorks, Inc.
![Page 13: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/13.jpg)
Functional Test -‐‑ Specification
13 © 2012 ThoughtWorks, Inc.
![Page 14: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/14.jpg)
14 © 2012 ThoughtWorks, Inc.
More useful information in failure reports
![Page 15: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/15.jpg)
Acceptance Criteria DSL • Given some initial context • When an event occurs • Then ensure outcome • Example –
15 © 2012 ThoughtWorks, Inc.
![Page 16: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/16.jpg)
Functional Test – New Implementation
16 © 2012 ThoughtWorks, Inc.
![Page 17: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/17.jpg)
Good Acceptance Criteria
17
Scenario: Advanced Search for a hat !!Given I am searching on Etsy.com!And I am not logged in !And I specify the Knitting sub category !And I search for hat !Then there are search results !!
Scenario: Advanced Search for a hat !!Given I am searching on "http://www.etsy.com" !When I click on the select box with css class "select.handmade" !And I select "Knitting" !And I click on the text box with id "#search_query" !And type in "hat" !Then there are search results !
Focus on WHAT, not HOW
© 2012 ThoughtWorks, Inc.
Be@er
![Page 18: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/18.jpg)
How Many Tests?
Unit Tests
Integration Tests
Functional Tests
Manual Testing
18
10%
20%
70%
h0p://blog.mountaingoatsoftware.com/tag/agile-‐‑testing © 2012 ThoughtWorks, Inc.
![Page 19: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/19.jpg)
What About Legacy Systems? • Start with functional tests • Cover with unit tests for new
functionality • Be on the lookout to replace
functional tests more fine-grained tests at every given opportunity
19 © 2012 ThoughtWorks, Inc.
![Page 20: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/20.jpg)
What To Automate? • The MOST important scenarios
o End-To-End system level scenarios o Happy paths
• Avoid automating individual feature-level acceptance criteria
• Make a distinction between “ephemeral” and “long-standing” tests
20 © 2012 ThoughtWorks, Inc.
![Page 21: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/21.jpg)
When do these tests run? • After every check-in
• As part of CI build pipeline o Unit à Smoke à Regression à Deploy
• Incubator Test Suites
21 © 2012 ThoughtWorks, Inc.
![Page 22: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/22.jpg)
Workflow • Sprint [N -1]
o Feature elaboration with entry level acceptance criteria
• Sprint [N] o Feature kick-off meeting o Developers implement feature o Testers (optionally) add additional acceptance criteria
• Sprint[N, N + 1] o Testers verify all acceptance criteria to be satisfied o Testers refactor functional test suite o Feature marked complete
22 © 2012 ThoughtWorks, Inc.
![Page 23: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/23.jpg)
23
Technology
© 2012 ThoughtWorks, Inc.
![Page 24: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/24.jpg)
Test Characteristics • Be idempotent
o At least re-runnable o Keep tests isolated from one another
• Be runnable in parallel o No inter-dependencies between tests
• Be deterministic o Quarantine/eradicate non-deterministic tests o Don’t “sleep”
• Prefer callbacks or at least poll for asynchronous responses o Consider mocking remote third-party services
• Have separate tests to verify interaction with remote services o Consider using relative dates/times or mock the system clock
o http://martinfowler.com/articles/nonDeterminism.html
24 © 2012 ThoughtWorks, Inc.
![Page 25: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/25.jpg)
Test Sources • Don’t have any dependencies on
production sources • Treat as first-class citizens
o Keep DRY o Run static analysis metrics
• Leverage design patterns like o Page Objects o Compound Steps
• Consider the use of terser languages o Groovy, Ruby etc.
25 © 2012 ThoughtWorks, Inc.
![Page 26: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/26.jpg)
Test Data • Use application to create test data • Use test data creation steps
o Frameworks like DBUnit
• Use anonymized production data subsets o Requires lot of time and effort
• Use anonymized production data copies o Huge data volumes may make it prohibitive
• Avoid the shared database anti-pattern • Consider mocking remote third-party
services 26 © 2012 ThoughtWorks, Inc.
![Page 27: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/27.jpg)
Test Infrastructure • Invest in fast build hardware
o CI servers support multiple remote agents
• Use same software configuration as production in all environments
• Consider running tests on the cloud o http://www.saucelabs.com
27 © 2012 ThoughtWorks, Inc.
![Page 28: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/28.jpg)
28
People
© 2012 ThoughtWorks, Inc.
![Page 29: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/29.jpg)
Who Writes/Owns The Tests? • The specifications – acceptance
criteria? o Domain Experts
• Step implementations? o Developers & Testers
• The functional suite?
o The Team o Day-to-Day – Testers
29 © 2012 ThoughtWorks, Inc.
![Page 30: Functional testing patterns](https://reader034.fdocuments.in/reader034/viewer/2022052310/554f5b1cb4c905c8088b462a/html5/thumbnails/30.jpg)
Team Dynamics • Fix broken builds and tests immediately
o OK to occasionally break the build o Not OK to leave the build broken for long
• Do not allow check-ins over a broken build o Unless check-in is being made to fix the build
• Co-locate teams as much as possible o At least create fully formed remote teams
30 © 2012 ThoughtWorks, Inc.