Exploratory Testing Live - WordPress.com · exploring, just for the sake of curiosity. What is...
57
Transcript of Exploratory Testing Live - WordPress.com · exploring, just for the sake of curiosity. What is...
Who Am I• Simon ‘Peter' Schrijver
• Age : 51
• Coming from The Netherlands
• Married with children
• Independent Tester
• Since 2005
• Founding Member of DEWT
• Current : Mobile App Tester
Just an Exploratory Test
What was I thinking?• Is there a maximum size?
What is the value?
• Is there a minimum size? What is the value?
• Can I use decimals? How many behind the decimal point?Any rounding?
Observations• Max size = 56 inch
• Min size = 1 inch
• Two digits -> 12,34
• Rounds 3rd digit
QuestionWhat do you do with this requirement?
In this text field, no more 100 characters are allowed
Send a string with 101 characters and see what happens
Skilled testers are always exploring, just for the
sake of curiosity.
What is testing about?• In search for qualitative
information
• Questioning the product to evaluate it, to learn from it
• Gathering information to make an informative decision
Name 3 keywords within these statements
What is testing about?• In search for qualitative
information
• Questioning the product to evaluate it, to learn from it
• Gathering information to make an informative decision
The 3 keywords within these statements
3 key words• Information
• Question
• Learn
By asking questions, you collect information and learn more about the application you are testing
While learning, you create more questions to reveal more information
Question
ExploratoryTesting
InformationLearn
Another Exploratory Test“Enter your birthdate on your account in your favorite retailer's website so that it can be shown on your account on the mobile App of your favorite retailer”
date format is in ‘dd-mm-yyyy’ notation and will be translated into UNIX timestamp
Examples?? (i.e questions to your application)
15-07-1992; 21-01-1997; 27-08-1965; 18-02-1934
Birthday Test - Observation
https://simonsaysnomore.wordpress.com/2016/03/08/the-summertime-bug/
Next Year - Observation
Example of Change Daily Limit
Example of Change Daily Limit
To make Exploratory Testing successful for your organization, you must provide tangible output
To make Exploratory Testing successful for your organization, you must provide tangible output
To make Exploratory Testing successful for your organization, you must provide tangible output
Why? To provide your stakeholders with information to help them make decisions
How (do I do that)? By asking questions so you can collect and reveal that information
What (do I need)? An approach like Session Based Exploratory Testing to document that information
–E. L. Doctorow
“Writing is an exploration. You start from nothing and learn as you go.”
Author of “The Book of Daniel”
Target your exploration based on the needs of
your team and stakeholder
But be aware of the following …
KNOWN KNOWNS KNOWN UNKNOWNS UNKNOWN UNKNOWNS
TESTING
INFORMATIONCHECKING
https://findingdeefex.com/2016/05/20/the-testing-checking-synergy/
enables
confirms
informs
uncovers
What do we know when we see this screen?
What do we not know when we see this screen?
What do we expect?
Do your exploration
Test and find out.
Write down what you observe
I see the screen "Transfer"
I enter "1.10" in the "amount field"
I enter "<name>" in the "beneficiary field"
I enter "<account number>" in the "accountnumber field"
I tap on button "Next"
And I tap on button "Transfer in mutations" And I see the screen "Transfer" And I enter "1.10" in the "amount field" And I enter "<name>" in the "beneficiary field" And I enter "<account number>" in the "accountnumber field" And I tap on button "Next" And I see the screen "Overview"
Gathering information
http://swtester.blogspot.nl/2012/05/what-is-exploratory-testing.html
Change the screen to add a new button to introduce a new feature
What do we check?
What do we test?
I see the screen "Transfer"
I enter "1.10" in the "amount field"
I enter "<name>" in the "beneficiary field"
I enter "<account number>" in the "accountnumber field"
I tap on button "Next"I tap on button "Another payment"
And I enter "<description>" in the "description field"
And I tap on button "Transfer in mutations" And I see the screen "Transfer" And I enter "1.10" in the "amount field" And I enter "<name>" in the "beneficiary field" And I enter "<account number>" in the "accountnumber field" And I enter “<description>” in the “description field” And I tap on button “Another payment” And I see the screen ”Transfer"
Exploratory Testing will help you with answering
the risk question
Risk is an uncertain event that can happen at some moment
in time affecting people
RiskIn software testing, we think of risk on three dimensions:
• How the program could fail
• How likely it is that the program could fail in that way
• What the consequences of that failure could be
BBST - Test Design
Exploratory Testing from risk point of view.
An approach to drive the discovery of unwanted, unexpected and unpredicted
behavior
Look for uncertain events.
Where are the vulnerabilities?
Which parts of the software are new or have been changed.
Which parts have not changed
Check out the main flow(s) of your application
e.g. in Mobile banking • Payment flow • Signing flow
• Imagine how the product can fail • Design tests to expose these
(potential) failures.
Design your tests with the risk in mind
BBST - Test Design
Gather as much information as you can about the application. Then choose what is relevant for you to proceed.
A CEO asks always the following two questions: • When can we ship? • Which risks do we have (when we ship)?
All Testing Starts Exploratory, because …
Exploratory Testing is about …
Questioning
Exploratory Testing is about …
Information
Exploratory Testing is about …Learning
Exploratory Testing is about …Thinking
Exploratory Testing is about …
Risk
Exploratory Testing is about …
Experiments
Exploratory Testing is about …
Observation
Exploratory Testing is about …Validation
Exploratory Testing is about …
Note taking
Exploratory Testing is about …
Communication
Exploratory Testing is about …Structure
All Testing Is Exploratory
Todays definition of TestAccording to James Bach and Michael Bolton, the definition of testing is now this:
“Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.”
http://www.satisfice.com/blog/archives/1509
• Session Based Exploratory Test URL : https://simonsaysnomore.wordpress.com/
• Testopsyhttps://www.improveqs.nl/media/1382/testopsy-etc-2017-huib-schoots.pdf
Where to start
