Software Testing (back to the basics)
Critical activities to make your testing
efforts more successful!
June 3, 2014
DOES YOUR TEST STRATEGY FAIL MISERABLY?
3©Alliance Global Services 2014
a story from the past
At the conclusion of 3 years of development and QA efforts
Development built an application
QA tested an application
QA tested and signed-off project completion
Application deployed out to the field
End-users could not conduct their business
End-users rejected the applicationo Lost (Time, Money, Reputation)
Application was pulled from production
QA was blamed for the failure
An assessment of the overall project determined that many issues contributed to the failure
When asked what was the cause: Your QA Strategy sucks!
“QA has the responsibility of maintaining the quality level of a company and its products.”
4©Alliance Global Services 2014
responsibility of QA
“QA provides process and a system to ensure that company
developed products work and that they meet or exceed
customer requirements and expectations!”
5©Alliance Global Services 2014
testing basics
Description of what is going to be built
o Test acceptance criteria
Test cases
o Traceability
Functional test coverage
o Ad-hoc Testing
Positive and negative testing
Non-functional testing
o Performance / Security / Usability
Regression testing
o Risk based testing and test case priority
Defect management
Test metrics
User Acceptance Testing
UNDERSTANDING WHAT YOU ARE BUILDING
7©Alliance Global Services 2014
what are you building
8©Alliance Global Services 2014
requirements
user stories
goalsroles
reason
epic
themes
requirements
9©Alliance Global Services 2014
written requirements / documentation
Tree Swing
FR: 001
10©Alliance Global Services 2014
waterfall and agile
Waterfall SDLC CyclesPlanning Phase Design Phase Construction Phase QA Phase
Planning Design Construction Testing
Agile SDLC CyclesFeature 1 Feature 2 Feature 3 Feature 3
Planning
Design
Construction
Testing
Planning
Design
Construction
Testing
Planning
Design
Construction
Testing
Planning
Design
Construction
Testing
Time
11©Alliance Global Services 2014
test acceptance criteria
pass
fail
=
<>
12©Alliance Global Services 2014
summary
Requirements
Written Documentation
Location to store and maintain and search the documentation
Process to review requirements
Requirements that are testable
13©Alliance Global Services 2014
TEST CASES
14©Alliance Global Services 2014
test cases
What is a test case?
How to determine the level of effort required for your test cases…
15©Alliance Global Services 2014
no formal test case
Complete faith in testers
QA in-depth knowledge of system
Access to business
Access to development
Testers remain on the project for
the duration
Example #1:
Testing CompleteX
16©Alliance Global Services 2014
Minimal test case
Complete faith in testers
QA in-depth knowledge of system
Access to business
Access to development
Testers remain on the project for
the duration
Example #2:
Test the functionality of the login screen to be sure that the user can login.
17©Alliance Global Services 2014
formal test case
Inexperienced test team
Have experienced lack of quality
in the past
Anyone can execute the test case
Required for automated testing
Example #3:
Test the functionality of the login screen:1: Navigate to the login
screen2: Enter a valid user name3: Enter a valid password4: Click on the login button5: Run again pressing enter
on the login button
Expected results: should be left on the home screen
18©Alliance Global Services 2014
risk based testing and test case priority
RBT helps project managers to balance Risk
with project constraints like Schedule,
Resource and Cost. This is achieved by
prioritizing the system to identify the areas
that carry a higher business risk. Hence
Projects can perform intelligent testing over
complete testing
*About 80% of the Defects come from 20% of the Modules
Identification of critical defect at an early stage of testing
Considerable Decrease in QA cost
Faster Time-to-Market
Increase involvement between Business and Testing
Collaborative working model between SME’s and testers
Business can make informed decision on the Risk v/s Testing Level forrelease acceptability
Over Testing the application
Large amount of effort is spent of areas of the applicationwhich carry very low risk
Minimal involvement of business in Quality Control
High Cost
Conventional Testing Risk Based Testing
Conventional Testing V/S RBT
19©Alliance Global Services 2014
simple risk and priority calculator
Scenario Probability of Failure
Complexity of Module
Size of Change
Frequencyof Use
Impact of Failure
Cost of Failure
Priority
Test Case 2 1 1 1 1 1 7
Test Case 1 2 3 4 4 2 16
Test Case 4 4 1 1 2 2 14
Test Case 4 3 3 3 4 4 21
1=High, 2=Medium, 3=Low, 4=None1=Big, 2=Medium, 3-Small, 4=None
20©Alliance Global Services 2014
traceability
Original Documentation of what you are
Building
Dev
elo
pm
en
tQuality
Assurance and Testing Efforts
FinishedProduct
21©Alliance Global Services 2014
FUNCTIONAL TEST COVERAGE
22©Alliance Global Services 2014
test coverage reporting
23©Alliance Global Services 2014
positive and negative testing
Positive Testing – System works as expected
Negative Testing – system can handle unexpected input
Expected result when certain data is used.
What happens when unexpected data is used?
24©Alliance Global Services 2014
non-functional testing
Performance Testing – meet SLAs
Security Testing – protected data and property
Usability Testing – end user can successfully use product
25©Alliance Global Services 2014
ad-hoc testing
Let the testers and SME’s play with the system!
Chase down that rabbit hole
The Grandma test
Test engineers are unbound by formal practices
26©Alliance Global Services 2014
regression testing
Verifies that previously tested features are not impacted by
new development or defect fixes
Generally a mixture of positive and negative test cases
Can be designed for complete coverage or partial coverage
through risk-based test planning
27©Alliance Global Services 2014
user acceptance testing
User Acceptance Testing (UAT) is a final test of the application
by the users of the application before release.
o Last chance to get a new perspective outside of Engineering
o Testing can be formal scripts or ad-hoc “play with it” testing
It is not based on detailed requirements and specs
o High level contractual agreements and
o General look and feel, and usability from the end-users point of view
not the engineers.
SDLC Differences
o Waterfall: this is last step before release
o Agile: this is done within the iteration by story during the Product
Owner Acceptance and often as waterfall type release UAT testing
of system as a whole
28©Alliance Global Services 2014
defect and metrics management
Defect management
o Best done with a tool
o Memory
o Excel
Test Metrics
o Track the progress and output of testing
o Defects found per cycle
o Cases created / executed
o Pass / fail ratios
o Overall test coverage
29©Alliance Global Services 2014
Thank you
Top Related