Breaking Illusions: Testing is your most valuable

asset!Maaret Pyhäjärvi

Email: <maaret@iki.fi> | Twitter: maaretp

Maaret PyhäjärviNimeä | Attribution (Finland)http://creativecommons.org/licenses/by/1.0/fi/http://creativecommons.org/licenses/by/1.0/fi/deed.en

Testing is about Feedback

Checking• Machines can do with

algorithmic decision rules

• Focuses on specific facts

• Cmp. “compiling”, “generating”

Exploring• Skilled humans can do• Open-ended search for

information• Cmp. “programming”

Testing not testers – but becoming a skilled tester requires time and focus

Context Matters – Expectations Differ

• Testing in Production?– Rolling releases: limit the extent of impact– Product lock-in: limit the options

• Testing in the delivery chain?– Development teams skilled in exploring (as

well as checking)– Internal releases – Beta releases – Releases

• Available skills?– Intentional vs. accidental production

feedback

Testers don’t break your code, they

break your illusions about the code.

-- adapted from James Bach

Dispelling Illusions: Empirical Evidence over Speculation

VALUE

ILLUSIONS

•Code doing what it’s supposed to•Product doing what it would need to•Your process is able to deliver with change in mind•Business growing with uninformed risks on the product and the business model around it.

The World Has Already Changed

COMMODITY TESTERS• Manual checkers• Tests are an artifact

SKILLED TESTERS• Explorers of products

and businesses• Testing is a

performance

• DEVELOPERS IN TEST• All development is

automating, testing is just a domain of development

Exploration SkillsSource: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory

Testing Dynamics. v.2.2. 2009

Self-manageme

nt

Developing ideas

Examining product

DoneTo DoIssuesCoverage

All sources available Best use of time – effective and efficient work

Making modelsTool support – creative solutionsRisk-based testing – scientific approachKeeping one’s eyes open

Things Can Look Different from Different Perspectives

Culture of Feedback and Experiments

• Illusions are meant to be broken• Teams over individuals– Diverse teams (personalities, skills)

deliver better results– Social dissonance gets the team talking

and voices out uncertainties to address on time

• Experiment: try something different

Summary

1. TESTING is FEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better.

2. CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers

3. BUGS not defects: anything this might bug a user, there’s many types of illusions we have

4. COMMODITY testers and SKILLED testers are two completely different breeds

5. SKILL to break DIVERSE illusions – skill in team, not in individual

6. DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development