[Srijan Wednesday Webinars] Creating an Awesome Organization by Weaving Great Experiences
[Srijan Wednesday Webinars] Why Adopt Analytics Driven Testing
-
Upload
srijan-technologies -
Category
Software
-
view
93 -
download
4
Transcript of [Srijan Wednesday Webinars] Why Adopt Analytics Driven Testing
Why Adopt
Analytics Driven Testing
Ori Bendet | Inbound Product Manager, HPE Software |
@bendet_ori
#SrijanWW | @srijan
7y in HPE Softwarein various managerial positionsStarted in testing
Today:Inbound Product ManagerStormRunner Functional
ABOUT ME
AGENDA • Applications Development Overview• Today’s Testing Challenges• What are the risks? • Real World Examples • How can you empower your testing by
using Analytics
The World of Software is changingMega-trends create demands on modern applications
MODERN APPLICATION DEVELOPMENT
Reduce costs
Increase customer attraction/retention
Increase the value of your brand
Get to market faster
300,000tweets
200 million+emails sent
220,000new photos posted
50,000 appsdownloaded
$80,000in online sales
72 hoursof new video content uploaded
2.5 millionpieces of content shared
Agile
Every minute*…
* Source: “The Data Explosion in 2014 Minute-by-Minute” by ACI Information Group.http://aci.info/2014/07/12/the-data-explosion-in-2014-minute-by-minute-infographic/
Mobile
Cloud
Digital
DevOps
How to mitigate the risk?
• Have more testers• Crowdsource• Don’t test• Let your users test for you
• Or… use Analytics!
Example • My product: 30 features• Supported Matrix: 10 environments
(OS/Browser)• My testing team: 10 testers
• Testing a feature on an environment = 1 day
• Target regression cycle: <3 days!
Google Analytics • A freemium web analytics service offered by Google
that tracks and reports website traffic.
• Google launched the service in November 2005
after acquiring Urchin.
• Google Analytics is now the most widely used web
analytics service on the Internet.
• Note: I’m not affiliated with Google Analytics in
anyway * Source: https://w3techs.com/technologies/overview/traffic_analysis/all
Environment Optimization • Focus on the majority of combinations (OS x
Browser)
• Reduce your regression risk to a minimum
• Wait for customer feedback on the other areas
User Behavior • Focus on the majority of the functionality
• Target the most active areas
• Wait for customer feedback on other areas (% of escaped
defects)
• Calculated risk(!)
30 12 features X 10 2 combinations* = 24 days
24 days / 10 testers = 2.4 days
(*) with 76% confidence
Building a risk calculator • Test everything (100%) = 300 days
• All functionality, top combinations = 60 days
• Main features, top combinations = 24 days
• Add additional levels based on your needs and
time
Building a risk calculator
Top flows
1 OS + Browser
Main Functionalities
top combinations
All Functionalities
top combinations
All Functionalities
Additional combinations
Test Everything
50%
75%
90%
100%
95%
Don’t have analytics? • Use market analysis and statistics
• https://netmarketshare.com/
• https://www.w3schools.com/browsers/
• Many more!
• Conduct annual user survey
• Customers Validations – before & after
Re-use calculator for Automation
Top flows
1 OS + Browser
Main Functionalities
top combinations
All Functionalities
top combinations
All Functionalities
Additional combinations
Test Everything
50%
75%
90%
100%
95%
CI
Nightly
Full regression
Summary • Don’t test everything!
• Get to know your users
• Calculate your risk
• Measure your un-tested areas
• % of Escaped defects
• Customer Satisfaction