Session-Based Test Management
-
Upload
caltonhill -
Category
Software
-
view
850 -
download
0
Transcript of Session-Based Test Management
Session-Based Test Management
An approach to testing with– No test cases– No expected results– No pass/fail metrics
Testing on a project is divided up into focussed “sessions” of exploratory testing.
Before we start…
• I’m no expert• A lot to cover. I will circulate links with more detail• Definitions/opinions are as per “Context-Driven
Testing”– No best practices, depends on context– Very influenced by social sciences and related fields– All testing is “sapient” and exploratory, or it’s not testing– “Good software testing is a challenging intellectual process”
What problem is this solving?
• Exploratory testing the most valuable and powerful approach – especially in Agile
BUT• Perceived as vague, unstructured, unmanageable– Hard to gauge coverage– The tester themselves may not be able to clearly explain
what testing they did – Managers want metrics, estimates and line-graphs (even if
these can be meaningless)• “If you can’t measure it, you can’t manage it”
Exploratory Testing – the CDT view
• Defined as “Simultaneous learning, test design and test execution”
• An experimental, scientific approach• Fits the natural human approach to learning or
problem-solving• Exploratory testing is not random• Exploratory testing is not a luxury• Exploratory testing is not what you do if you have
spare time after executing all the test cases
Session-Based Test Management
• Formulated by Jonathan and James Bach• Manages and tracks Exploratory Testing• Testing tasks on a project divided into sessions• A test session:– A dedicated and uninterrupted period of exploratory
testing – Focussed on set objectives defined in a “Charter”– Generates notes and lightweight reporting on the testing
carried out• The testing task is expected to change and evolve
Session-Based Test Management: Planning
PLANNING: The Charter
• Each test session is described by a Charter• A single sentence that sets the scope of the testing– What you’ll test– How you plan to test it– The information you hope to provide
• That scope should be achievable in around 90 mins• Charters are basis of estimation and coverage
PLANNING: Drawing up Charters
• Charters can come from many sources:– What are the elements to be tested? What are the
requirements? What are the risks?– Backlog/sprint planning meetings
• Elisabeth Hendrickson’s “Explorer” format:“Explore [target]with [resources, techniques]to discover [information valuable to the project]”Eg. “Explore unexpected failures with power and network interruptions to discover if the product recovers gracefully”
PLANNING: Drawing up Charters
……Read Chapter 2 of “Explore It!” By Elisabeth Hendrickson.
(It’s only 13 pages)
Examples….
Session-Based Test Management: Testing
TESTING: Focus
• Charter as scope– Some flexibility possible
• A dedicated and uninterrupted period of testing– No meetings, no email, no drive-bys
• Session length will vary but perhaps 90 mins
TESTING: Test Ideas
• Use testing experience or instinct; use your domain knowledge
• Test scenarios could be ok, but not pre-scripted stepsRemember it should be a fluid process …• Use the software, observe it and be alert to new
questions or test ideas – Follow those new thoughts now?– Note them as ideas for follow-up?
TESTING: Test Idea Frameworks
• Personas• Tours• Heuristics(Could be with the “with” part of your Charter)
•Encourage thinking beyond specific features or tasks in isolation. – how parts of the software work together– user flows
TESTING: Idea Frameworks - Personas
• Fully imagine and describe users of the software. Be that person while you test the product– Role– Age– Education – What devices do they use?– What environment do they use your product in?– What are their goals in using the product?– What motivates them, What annoys them? – What websites would they use? What TV programmes would they
watch?
TESTING: Idea Frameworks - Tours
• Mike Kelly: touring an application– Claims tour– Configuration tour
• James Whittaker: exploratory testing tours– The Museum tour– The Supermodel tour– The Bad Neighbourhood tour
TESTING: Idea Frameworks - Heuristics
• Shortcut approaches to complex questions or problems which are usually based on experience. They’re usually effective, but not guaranteed to be.– Eg. “Trial and error”
• In Testing:“Test at the boundaries” ; “CRUD” ; “Some, none, all” ; “OFAT (One Factor At a Time)” ; “Break the rules” ; “The Iceberg Heuristic”You will have your own ….
• Test Heuristic Cheat Sheet
TESTING: Note Taking
• Notes on your testing are crucial• Not just a Pass/Fail count– What you did. Why you did it. What you saw. What
questions and ideas you had. Any frustrations/blocks.• Style of note taking can be personal to you
– I tend to use Rapid Reporter or just pen/paper
• You need to be able to summarise and discuss the notes from your session
Session-Based Test Management: Reporting
REPORTING: Session De-brief
• Tester reviews session findings with Test Manager/Test Lead/Other Tester– A bit like reporting in a stand-up– Is more testing needed in this area?
• May lead to ideas for more charters– Are there issues or blocks affecting the testing?– Are there questions which need to be referred to a
product owner?• Raise bugs as normal• In my case: lightweight reporting
REPORTING: Project Status
• Managers can have metrics on ET, including:– Charters/Sessions completed and remaining– Sessions completed per day or week (velocity)– Product areas/functionality coverage– Bugs raised– Time spent testing vs setup, admin etc
• See links to articles by Jonathan Bach and Rob Lambert
The End
To summarise - in SBTM:• Testing described in session charters, not test cases• Exploratory Testing is not random• Exploratory Testing is creative and requires intelligent
thought. A number of techniques and approaches are available.
• More information in the links I’ll distribute• For anyone interested, I can demo the Rapid Reporter
note-taking tool