DRIVING SUCCESS WITH CONTINUOUS TESTINGqaiquest.org › 2019 › wp-content › uploads › 2019 ›...

QUEST Webinar:

Driving Business Success with Continuous Testing

FeaturingAlissa Lydon from Sauce Labs

WEBINA SERIES

presents the

WEBINAR SERIESwww.qaiQUEST.org/2019

4/9/19

DRIVING SUCCESS WITH CONTINUOUS TESTINGA L O O K A T T H R E E I N D U S T R I E S

A L I S S A L Y D O N , S A U C E L A B S

HIGHLIGHTS

4/9/19 © Sauce Labs, Inc.

You are wondering what Continuous Testing is

• I will provide a precise definition of what I believe it is

You’re not sure Continuous Testing is a real thing!

• I will provide concrete examples of 3 industries that are changing thanks to testing paradigm shifts

• I will also provide real world stories of companies that have made quality a strategic enabler

You want to achieve continuous testing, but don’t know how to get started

• I will provide advice on how to start affecting change

2

W H Y A M I H E R E ?

INTRO

4/9/19 © Sauce Labs, Inc.

Product Marketing Manager at Sauce Labs

• Focused specifically on community enablement

I LOVE the testing community

• Conferences, meetups, customer visits

• Meet new people, learn new trends, understand problems

I used to play a lot of softball

• Stanford University, NPF, Team USA

• Big believer in team mentality, building CULTure

3

W H O I S T H I S L A D Y ?

CONTINUOUS TESTING IS MISSION CRITICAL

4/9/19 © Sauce Labs, Inc. 4

By 2020, DevOps initiatives will cause50% of enterprises to implement continuous testing using frameworksand open-source tools.

CATCHING BUGS BEFORE IT ’S TOO LATE


• Launching new app or major update

• Press release issued

• Social media blitz is on

• Customers are (hopefully) excited

• In final stages of QA and testing, major glitch is identified

• Consequences

• Lost users > revenue

• Damaged company reputation

• Angry leadership

TESTING IS #1 BOTTLENECK


WHERE IN THE DEVELOPMENT PROCESS DO YOU ENCOUNTER THE MOST DELAYS?

Source:GitLab2018DeveloperSurvey

THE TESTING PARADIGM SHIFT

I T ’ S A L L A B O U T U S E R E X P E R I E N C E


• Testing is now a key consideration for executives

• Buggy apps can cause lost business/revenue

• 79% of users will retry an app once or twice if it fails to work the first time (Comscore)

• Quality problems can damage a company’s reputation

• An app with a 4.5 star rating will be downloaded 3.7x more than one with 3.5 stars (Comscore)

• 59% of 24-34-year-olds share poor customer experiences online (New Voice Media)

TESTING MUST TRANSFORM TO KEEP UP


L E G A C Y T E S T I N G I S S L O W , E X P E N S I V E A N D L A G S D E V

C O N T I N U O U S T E S T I N GL E G A C Y

T E S T I N G

WHAT IS CONTINUOUS TESTING?


T e s t . a l l T h e T h i n g s ( )

C O N T I N U O U S T E S T I N G

“Best practice approach of executing automated tests throughout the software development lifecycle”

BENEFITS OF CONTINUOUS TESTING

REMOVE TESTING AS A BOTTLENECK TO DIGITAL EXCELLENCE

DevelopAppsFaster

ImproveProductivity

DelightUsers

10

NO INDUSTRY IS SAFE

C O N T I N U O U S T E S T I N G C A N H E L P Y O U R B U S I N E S S S T A N D O U T


FinTech

Retail

Media

NO INDUSTRY IS SAFE



FinTech

FINANCIAL TECH

C R O W D E D A N D C O M P E T I T I V E


• Over 1,000 companies in the space

• $25 billion investment increase from 2013-2015

• The race for loyal customers

• 18% of millennials changed banks in the last 12 months (Accenture, Consumer Digital Banking Survey 2015)

• Key to success – deliver customers quality products, quickly

• Agile workflows ensure success

• Continuous Testing a requirement

SNAPSHOT – N26

B O O S T I N G C O N F I D E N C E I N M O B I L E B A N K I N G


• N26 is revolutionizing the mobile banking landscape with their apps

• Problem: Teams were experiencing slowed releases due to manual testing on physical devices

• Solution: automate their tests using Appium, and run them in parallel to increase release velocity

• Results: Running tests across 30 different devices

• Greater confidence in results, at less cost

• Faster releases > more time for creative work

• Happier customers

NO INDUSTRY IS SAFE



Retail

RETAIL

C U S T O M I Z E D E X P E R I E N C E S A R E K E Y


• Digital disruption is no longer a threat, it is a part of the landscape

• Must be able to innovate products quickly in response to user demand

• Omnichannel experiences set companies apart

• Flawless personalized experience, no mater where customers are engaging

• Amazon releases feature updates, on average, every 11 seconds

• Continuous Testing can help meet the need for speed without compromising quality

SNAPSHOT – WALMART

C O M M I T M E N T T O T E S T I N G A N D O P E N S O U R C E


• Walmart needed to invest in digital to keep up in the new economy

• Problem: Siloed teams, manual testing, and slow releases bogged down their teams

• Solution: embrace testing from the top down, standardize practices, and adopt automated testing at scale

• Results: Now running 100,000 tests a day, and releasing on every front end build

• Improved customer experience

• $750,000 saved per year in labor costs

NO INDUSTRY IS SAFE



Media

MEDIA

E N G A G E T H R O U G H E V E R Y C H A N N E L


• Must be able to provide a seamless experience, no matter where your users are coming from

• Mobile-first media consumption

• More than 50% of the world’s traffic comes though mobile devices (Digital in 2017)

• 48% of millennials now view videos solely through mobile devices (Business of Apps)

• Continuous Testing can help ensure cross device compatibility

SNAPSHOT – BLEACHER REPORT

P U T T I N G M O B I L E F I R S T


• Bleacher Report is a leading publisher of sports content – over 16 million unique users

• From 2012, 50% of their traffic coming from mobile devices

• Relied on manual testing until then

• Not compatible with their accelerated release cycles (20 deploys every 3-4 days)

• Solution: Automate with Appium, test early and often with scalable infrastructure

• Results: Now full suite of tests, on multiple device types, in minutes

• Allows team to focus on bigger picture

PARTING THOUGHTS

21

D E V O P S

CONTINUOUS TESTING

ACHIEVING CONTINUOUS TESTING

S T A R T W I T H T H E S E K E Y E L E M E N T S

Visibility and Analytics

Comprehensive Coverage

Use Expertise to Automate

Commitment to Quality

Scale On Demand


Test at Every Stage

T H A N K Y O Ua l y d o n @ s a u c e l a b s . c o m

@ h a b e s 0 6

B O N E Y A R D

DRIVING SUCCESS WITH CONTINUOUS TESTINGqaiquest.org › 2019 › wp-content › uploads › 2019 ›...

Documents

Transcript of DRIVING SUCCESS WITH CONTINUOUS TESTINGqaiquest.org › 2019 › wp-content › uploads › 2019 ›...