Using Analytics to Drive Your Agile Testing Strategy Forward
-
Upload
qasymphony -
Category
Sports
-
view
89 -
download
1
Transcript of Using Analytics to Drive Your Agile Testing Strategy Forward
September 25, 2015
Introduction to QASymphony for [INSERT COMPANY NAME]
#AgileTransformation
Agile Transformation: People, Process and Tools to Make
Your Transformation Successful
The audio for this webinar is delivered through your computer. There is no dial-in number. Make sure your speakers are turned up or use a pair of headphones.
#testmetrics
Agile Testing Analytics
HOW TO WATCH THIS WEBINAR
• Audio for this webinar is delivered through your computer. Make sure your speakers are turned up or use a set of headphones.
• If your audio quality is choppy, it could be your internet connection.
• You can customize your webinar viewing experience by increasing, decreasing or minimizing the size of the widgets on your screen.
• If you have questions, enter them in the widget on the right.
#testmetrics
MAXIMIZING THE PRESENTATION
#RiskBasedTesting
Click the icon in the top right of the presentation to make the slides full screen.
#testmetrics
• This webinar will be recorded and available on-demand tomorrow. You will get an email when it is available.
• Join the conversation on Twitter using the hashtag #testmetrics
• Use the Q&A widgets to ask questions during the webinar.
• At the end of the webinar, you will be asked to take a short survey.
HOUSEKEEPING
#RiskBasedTesting#testmetrics
Live Demo of QASymphony Products: Agile Testing and Tools
NEXT WEBINAR
#RiskBasedTesting
WHAT?
Join us for a 30-minue live demo of the entire qTest Platform including, Test Management, Test Analytics,
Exploratory Testing and more
WHY?
Learn how to easily track, manage, organize and report on software testing with the qTest Platform
WHEN?
January 11th, 2-2:30 PM EST
Ryan YackelSenior Product
EngineerQASymphony
@ryanyackattack
April 18-19, 2017 | qualityjam.comEmail [email protected] to be entered to win a free
ticket.
MARK YOUR CALENDARS!
#RiskBasedTesting#TestAutomation#testmetrics
Test Case Management | Exploratory Testing | Test Analytics | BDD and more...
ABOUT QASYMPHONYThe #1 Agile Enterprise Testing Platform
POWERFUL, EASY-TO-USE TEST MANAGEMENT
• Central test repository to organize and manage test cases quickly and efficiently
• Seamless integration with JIRA for requirements and issue coverage
• Integrates with your CI/CD processes through tools like Bamboo and Jenkins
• Flexible options to organize their tests and powerful reporting to track their results across projects
• Customization abilities allows for flexible project deployments and integration settings
#testmetrics
ACTIONABLE, REAL-TIME TEST METRICS
• Your data, your way. Arrange and display charts and graphs exactly the way you want. Tabs let you create multiple dashboards for different projects
• Customized reporting. Zoom in and dynamically adjust the charts. Click on a chart to view and download the data
• Visualize your data with interactive heat maps that help quickly identify where the issues are in the application. Then drill down to get more insights
#testmetrics
In This Session
Analytics - What and Why
Quality Analytics
Coverage and Risk Analytics
Velocity Analytics
Test Case Optimization
How to Get Started
#testmetrics
Analytics - Data Sources
Need to create links: dev tickets -> code check-ins, test cases, defects, support cases
#testmetrics
Many potential data sources
The Goal: Leverage Data for Improvement
● Use analytics to improve:
○ Test coverage
○ Forecasting completion dates
○ Efficiency and most effective use of resources
○ Test case quality
○ Productivity
● Think of analytics as an objective input to the planning process
#testmetrics
Quality Analytics
● Core:○ Test result %s by project/release
■ separate out latest runs○ Defect priority and status %s
● Extra:○ Test results by day or week○ Defect status/priority crosstab○ Defects per test run○ Defect leakage (found by users)
● Tips:○ Use color-coding to identify potential issues○ Put manual and automated results side-by-side
#testmetrics
Coverage Analytics● Core:
○ Test cases by requirement■ Latest run results■ Breakdown by type
○ Defects by requirement
● Extra:○ Test case complexity○ Test time per requirement○ Last date of test run(s)
● Tip:○ Use data visualization to spot risks
#testmetrics
Velocity Analytics● Core:
○ Requirements inflow rate○ Test case creation rate○ Test run rate (cases & steps)○ % tests complete and blocked○ Defects opened and closed
● Extra:○ Avg. and total testing time spent○ Mean time to test(s) created, run, passed○ Forecast time and defects remaining
● Tip:○ Breakdown analytics by tester
#testmetrics
Test Case Optimization● Start to think of test cases like source code
● Track manual and automated test cases, exploratory scripts
● Track analytics that will help optimize your test case library○ Days since last run
■ Archive test cases that are not used anymore○ Flapping (# of times consecutive runs have different results)
■ Examine these tests and code/functional areas, might indicate need to refactor one or the other
○ Percentile complexity (steps and time spent per test)■ Refactor highly complex tests for greater efficiency and
more pinpoint understanding of results○ Cumulative execution time
■ Automate the manual tests that are taking the most time■ Refactor automated tests that are running the longest
#testmetrics
How To Do This Yourself● Setup a Test Analytics Reporting Server
○ Use an open source or 3rd party BI tool (such as qTest Insights)○ Keep it simple
● Identify team members that will Work on Test Analytics○ Depending on tool(s) may need technical and “analyst”○ Commit to a certain # of hours per week or per month
● Start with Requirements, Test Results, and Defects data○ For most companies data size will be very manageable○ Don’t tackle big data problems (partition data if necessary)
● Start with Quality analytics, then add Coverage, then Velocity○ Focus on weekly project reports
● Add More Detail and More Data Sources Over Time
#testmetrics