The Lone Tester in an Agile World: STP Online Summit
-
Upload
michael-larsen -
Category
Technology
-
view
105 -
download
0
description
Transcript of The Lone Tester in an Agile World: STP Online Summit
![Page 1: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/1.jpg)
Session 7:The Lone Tester in an Agile World
![Page 2: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/2.jpg)
Michael Larsen
Michael Larsen,Senior Tester
Senior Tester, Sidereel.com@mkltesthead
Producer & Commentator: TWiST PodcastSpeaker: PNSQC, CAST, STAREASTFacilitator: Weekend Testing AmericasContributor: How to Reduce the Cost of Software TestingBlog: TESTHEAD (http://mkltesthead.com)Articles: ST/QA, StickyMinds, The Testing Planet, Tea Time with Testers
![Page 3: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/3.jpg)
About Me (cont.)
– 18 Years Software Testing Experience– Mostly with Traditional Development
Approaches– Usually Sole Tester for Projects or Companies– Made the switch to Agile development in
January, 2011– Close enough to the Pain to Remember It!
![Page 4: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/4.jpg)
Traditional Test WorkflowReview requirements document
Review a software specification
Write out test cases based on software spec
Create detailed scripted test procedures
Wait for a build to arrive so you can test
Execute your scripted tests
Re-run tests that were added to cover bugs that were found in previous builds
Determine after enough testing or the specific release date if software is ready to ship
![Page 5: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/5.jpg)
The Lone Tester in the Traditional World
– Finding issues in requirements and design documents
– Verifying/validating software meets the spec
– Running tests at given milestone points to see if the product break
![Page 6: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/6.jpg)
Problems With Traditional Testing Approach
– Requirements are never complete – Spec often misses the mark– Product "meets the spec”, but isn't really what the
customer wants– Separate development and test cycles– Software thrown "over the wall" to be tested and
then thrown back to be reworked– Waiting for the product to be viable again to test
![Page 7: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/7.jpg)
Is There Another Way?!
![Page 8: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/8.jpg)
What is Agile?
– Build system in smaller chunks
– Business requirements discussed along with product development
– Build what is actually wanted
– Develop and test smaller bits of functionality more frequently
– Delivering product that meets real business needs
![Page 9: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/9.jpg)
The Agile Manifesto
![Page 10: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/10.jpg)
The Agile Approach• Small slices (Stories)
• Testing from the beginning of the project
• The whole team responsible for quality
• Test Driven Development used to help design code
• Tests fail first, build code to pass tests, refactor code to be more efficient, integrate with other code
• Continuous build and integration each time code is checked in
![Page 11: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/11.jpg)
Is Your Team "agile"?
– is the team using a test-driven approach to development?
– are stakeholders actively involved in the development process?
– does the team produce high-quality software at regular intervals?
– is the team highly collaborative and self-organizing?
– is the team actively improving the process of how they work?
![Page 12: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/12.jpg)
A Typical Story
– Developed in coordination with the product owner
– Develop the story's acceptance criteria along with it
![Page 13: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/13.jpg)
Test Driven Development
– Developers often working in paired teams• Focus on Red,
Green, Refactor• Repeat the process
if necessary• When the story has
reached the state of DONE, move on to the next story
![Page 14: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/14.jpg)
TDD is Not Testing
– ???
– TDD is a design process for development
– Helps to cover regression tests and unit tests on all code segments
– No build goes out that is not "all green" (all unit and regression tests pass)
![Page 15: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/15.jpg)
The Whole Team is Responsible for Quality
– TDD
– Acceptance testing
– Automated testing throughout the entire cycle.
– Provide continuous feedback to the development team
![Page 16: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/16.jpg)
???
Lots of "Testing"...
But where are the "Testers"?
![Page 17: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/17.jpg)
Are Testers Needed?
– Some view Agile as a "post tester world”
– Testing is done by development, product owners, and customers
– Is there a need for dedicated testers?
![Page 18: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/18.jpg)
Testers Needed and Valued
– Testing is integrated at all levels
– Quality is seen as a team effort
– Traditional Testing role (Quality Police) not required
– Testing are being done at many different levels
– Model of dedicated tester doing all the testing has changed
![Page 19: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/19.jpg)
Agile Testing Requires Variety of Skills
– Domain knowledge about the system under test
– Technical competency to interact effectively with development team
![Page 20: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/20.jpg)
Testing in Thin Slices
– Testing focused on if story meets customer requirements
– Exploration around stories to guide and provide feedback for team
– Automating acceptance tests to run in the future along with new features being developed
![Page 21: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/21.jpg)
“Agile Testing is…”• According to Bret Pettichord:
• “Headlights of the project – where are you now? Where are you headed?”
• “Provide information to the team – allowing the team to make informed decisions”
• “Find and inform team of bugs”– A “bug” is anything that could bug a user –
testers don’t make the final call
• “Testers do not "assure" quality – the team does (or doesn’t)”
• “Testing not a game of “gotcha” – set goals, rather then focus on mistakes”
![Page 22: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/22.jpg)
Challenges for Agile Testers
– Requirements and Acceptance Criteria specific to single story
– Testing as early as possible– Acceptance Tests developed before the software is
developed– Automated tests run against code every time a
build is performed– Regression testing requirements significantly
increased
![Page 23: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/23.jpg)
Single Tester in an Agile Development Team
– Testers have to adapt to a different expectation and needs
– Plenty of room for "testing" but we need to broaden our toolkit
![Page 24: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/24.jpg)
My Story: Sidereel
![Page 25: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/25.jpg)
Sidereel: Pre-Tester
– Prior to 2011, no dedicated tester– All developers actively use TDD– 4 Stage deployment method put into place• developer machines / demo / staging / production
– Each step had additional people looking at the product
– Testing done at each level by product owners, content developers, support staff and execs
– Worked well, but still plenty of issues discovered after the fact
![Page 26: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/26.jpg)
Sidereel: Post Tester
– In 2010, they decided that having a dedicated tester would be a benefit• Testers can provide many more directed efforts against
the code because of training and focus
• Product owners and other staff members can apply some time to testing, but they have other jobs
• Testers can bring all of their time to testing an application and continue to apply numerous approaches
![Page 27: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/27.jpg)
How Can We Use Testers? Step Outside of the Obvious
• Tester verifies bug fixes
• Tester confirms user stories
Good start, but there is a lot more we can do!
![Page 28: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/28.jpg)
Augment The Testing Already Happening
– TDD is meant to prevent defects
– TDD will help keep mistakes out, but not all of them
– Testers operate from a different mindset• Use Exploratory testing techniques• Aspects development team might not have considered• Look for missing story criteria
![Page 29: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/29.jpg)
Acceptance Testing: Great Place for Testers
– Have testers implement Active and Automated Acceptance Testing
– Not essential for testers to know how to code, but helpful
– Developing acceptance test requirements and potential automated tests at the start of the iteration
![Page 30: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/30.jpg)
Where to Look for Acceptance Tests
• Backlog• Current user stories• Current bugs
![Page 31: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/31.jpg)
Automating Acceptance Tests
– Variety of Tools to Help This Process• Fitnesse• Selenium/
WebDriver• Capybara• Cucumber• Other testing
frameworks
![Page 32: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/32.jpg)
Benefits
– Share results with the development team
– Help provide information about implementation
– Help the team get to DONE • DONE from a development perspective• DONE from a testing perspective
![Page 33: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/33.jpg)
Disadvantages
– Passive "checking" as opposed to Active Testing• Inquiring about the results we get• Learning based on results and exploring new avenues
– Tests can quickly become stale
– Consistent upkeep required
![Page 34: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/34.jpg)
Tester as Anthropologist
– Observe the development team
– Look for feedback of actual users
– Work with content developers
– Examine workflows– Discover the
underlying and unspoken "culture”
![Page 35: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/35.jpg)
Other Benefits
– Testers help with planning and future work
– Testers determine testability requirements
– Testers clarify stories and validate them early
– Testers can help identify when a story is "too big"
![Page 36: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/36.jpg)
Advocate for the Customer
– Understand the customer needs
– Review and relate to the business needs
– Review and relate to development practices
– The tester can straddle all of these
![Page 37: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/37.jpg)
Being The Lone Tester
– Requires communication– Willingness to stretch– Pair with programmers to help make automation a
reality– Participate with planning and standups with
development team– Become a domain expert about the customers
needs
![Page 38: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/38.jpg)
Tips to Thrive as a Lone Tester
– Focus on Communication – Learn the Expectations of Your Development Team – Stop Thinking in Terms of Us vs. Them – Learn How to Work with the Development Tools of Your
Organization – Plan for Pairing Sessions with Developers and Domain Experts
• Tester/Developer• Tester/Support• Tester/Designer• Tester/Customer
– Cultivate Relationships with Mentors – Develop Your Craft
![Page 39: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/39.jpg)
Use Context-driven Principles– The value of any practice depends on its context. – There are good practices in context, but there are no best– practices.– People, working together, are the most important part of any
project’s context.– Projects unfold over time in ways that are often not predictable.– The product is a solution. If the problem isn’t solved, the product
doesn’t work.– Good software testing is a challenging intellectual process.– Only through judgment and skill, exercised cooperatively
throughout the entire project, are we able to do the right things at the right times to effectively test our products.
![Page 40: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/40.jpg)
Secret Memo for Lone Testers: You're Not Really ALONE
– You have many allies– Developers are actively testing
as they develop code– Customer actively informs you
and guides you about features– Support team gives feedback
about pain points– Designers give feedback about
usability and user experience– Every one of these people
does testing, it's not ALL ON YOU!
![Page 41: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/41.jpg)
References• Ambler, Scott (2010). "Agile Testing and Quality Strategies: Discipline over Rhetoric".
(http://www.ambysoft.com/essays/agileTesting.html)
• Crispin, Lisa (2003). "XP Testing Without XP: Taking Advantage of Agile Testing Practices" (http://www.methodsandtools.com/archive/archive.php?id=2)
• Hendrickson, Elizabeth (2008). "Driving Development with Tests: ATDD and TDD" (http://testobsessed.com/wp-content/uploads/2011/04/atddexample.pdf)
• Larsen, Michael (2011). "The Challenges of the Lone Tester: Learn to Thrive" (http://www.softwaretestpro.com/Item/5174/The-Challenges-of-the-Lone-Tester-Learn-to-Thrive)
• Parkinson, Shane (2008). "Agile Methods and Software Testing" (http://agiletesting.com.au/agile-methodology/agile-methods-and-software-testing/)
![Page 42: The Lone Tester in an Agile World: STP Online Summit](https://reader034.fdocuments.in/reader034/viewer/2022051613/54c920334a795905448b45a7/html5/thumbnails/42.jpg)
Questions???