Rapid Software Testing: Strategy
22
MH Half-day Tutorials 5/5/2014 8:30:00 AM Rapid Software Testing: Strategy Presented by: James Bach Satisfice, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
-
Upload
techwellpresentations -
Category
Technology
-
view
128 -
download
2
description
A test strategy is the set of ideas that guides your test design. It's what explains why you test this instead of that, and why you test this way instead of that way. Strategic thinking matters because testers must make quick decisions about what needs testing right now and what can be left alone. You must be able to work through major threads without being overwhelmed by tiny details. James Bach describes how test strategy is organized around risk but is not defined before testing begins. Rather, it evolves alongside testing as we learn more about the product. We start with a vague idea of our strategy, organize it quickly, and document as needed in a concise way. In the end, the strategy can be as formal and detailed as you want it to be. In the beginning, though, we start small. If you want to focus on testing and not paperwork, this approach is for you.
Transcript of Rapid Software Testing: Strategy
MH Half-day Tutorials
5/5/2014 8:30:00 AM
Rapid Software Testing:
Strategy
Presented by:
James Bach
Satisfice, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
James Bach Satisfice, Inc.
James Bach is founder and principal consultant of Satisfice, Inc., a software testing and quality assurance company. In the eighties, James cut his teeth as a programmer, tester, and SQA manager in Silicon Valley in the world of market-driven software development. For nearly ten years, he has traveled the world teaching rapid software testing skills and serving as an expert witness on court cases involving software testing. James is the author of Lessons Learned in Software Testing and Secrets of a Buccaneer-Scholar: How Self-Education and the Pursuit of Passion Can Lead to a Lifetime of Success.
What is Rapid Testing?
Rapid testing is a mind-set and a skill-set of testing focused on how to do testing more quickly, less expensively, with excellent results.
This is a general testing methodology. It adapts to any kind of project or product.
The Premises of Rapid Testing
1. Software projects and products are relationships between people, who are creatures both of emotion and rational thought.
2. Each project occurs under conditions of uncertainty and time pressure.
3. Despite our best hopes and intentions, some degree of inexperience, carelessness, and incompetence is normal.
4. A test is an activity; it is performance, not artifacts.
5. Testing’s purpose is to discover the status of the product and any threats to its value, so that our clients can make informed decisions about it.
6. We commit to performing credible, cost-effective testing, and we will inform our clients of anything that threatens that commitment.
7. We will not knowingly or negligently mislead our clients and colleagues.
8. Testers accept responsibility for the quality of their work, although they cannot control the quality of the product.
What is a test strategy?
Test strategy is the set of ideas that guide your choice of tests.
A set of ideas does not necessarily mean a document. The test strategy may be entirely in your head. Or it may be in several heads, and emerge through discussion, over time. It may be documented partially on a whiteboard or Post-Its or in a mindmap. Or it could be in a formal document all dressed like Cinderella at the royal ball.
What is a test strategy?
Test strategy is the set of ideas that guide your choice of tests.
To guide is to influence but not necessarily to determine. Testing is shaped by many factors in addition to strategy, including opportunities, skills, mistakes, time pressures, limitations of tools, testability, and unconscious biases.
What is a test strategy?
Test strategy is the set of ideas that guide your choice of tests.
I mean choice in the most expansive sense of the word, not simply the selection of existing test cases. Choice of tests includes choices of what tests to design and how to design them and all decisions made during test design. It includes choices made during test execution, too, including how to perform tests and what mix of tests to perform in response to which perceived risks.
Why have a test strategy?
If you test, then you already have a test strategy, so that’s not a meaningful question… Here are some better questions:
Why have an explicit test strategy? Why worry about your test strategy? Why explain it? Why document it?
Why not let the tests “speak for themselves?”
1. To get more credibility, control, agility, and accountability, for less time and effort.
2. Tests don’t talk.
(Test Strategy is a Middle Level Tool)
Strategic Thinking Begins with the Context
Test Team
Expertise
Loading
Cohesion
Motivation
Leadership
Project Integration
Test Lab
Test Platforms
Test Tools
Test Library
Problem Tracking System
Office Facilities
Product Mission
Stakeholders
Quality Criteria
Reference Material
Requirements
Product
Project Lifecycle
Project Management
Configuration Management
Defect Prevention
Development Team
Development
Mission
Find Important Problems
Assess Quality
Certify to Standard
Fulfill Process Mandates
Satisfy Stakeholders
Assure Accountability
Advise about QA
Advise about Testing
Advise about Quality
Maximize Efficiency
Minimize Cost
Minimize Time
Test
Process
Strategy
Logistics
Work-products
Ask yourself
What testing is easy (even if not very important)?
What testing is important (to find big bugs)?
What testing is expected (by people who matter)?
How can tools help?
Have we advocated for testability?
Are we minimizing administrative costs?
Happy Path
Tour the Product – Sample Data
– Variables
– Files
– Complexity
– Menus & Windows
– Keyboard & Mouse
A quick test is a cheap test that has some value but requires little preparation, knowledge,
or time to perform.
Interruptions
Undermining
Adjustments
Dog Piling
Continuous Use
Feature Interactions
Click on Help
Cost as a Simplifying Factor Try quick tests as well as careful tests
Input Constraint Attack
Click Frenzy
Shoe Test
Blink Test
Error Message Hangover
A quick test is a cheap test that has some value but requires little preparation, knowledge,
or time to perform.
Resource Starvation
Multiple Instances
Crazy Configs
Cheap Tools
Cost as a Simplifying Factor Try quick tests as well as careful tests
Value (or Risk) as a Simplifying Factor Find problems that matter
Instead of thinking pass vs. fail, consider thinking problem vs. no problem.
In general it can vastly simplify testing if we focus on whether the product has a problem that matters, rather than whether the product merely satisfies all relevant standards.
Effective testing requires that we understand standards as they relate to how our clients value the product.
One way to make a strategy…
1. Learn the product.
2. Think of important potential problems.
3. Think of how to search the product for those problems.
4. Think of how to search the product, in general.
Think of ways that: will take advantage of the resources you have.
comprise a mix of different techniques.
comprise something that you really can actually do.
serve the specific mission you are expected to fulfill.
Test Strategy for DecideRight
Test Strategy Text
Risk-Based Test Project Cycle: Testing itself is risk analysis.
Experience Problems
In the Field
Analyze Potential
Risks
Analyze Actual Risks
Short loop
Long loop
New Project
ship
Exploratory vs. Scripted Perform
Appropriate Testing
Experience Problems &
Potentialities
18
Tests
Project Environment
Product Elements
Quality Criteria
Perceived Quality
A Heuristic Test Strategy Model
19
Project Environment
Tests
Product Elements
Quality Criteria
Perceived Quality
A Heuristic Test Strategy Model
Example: Medical Device
