3 Free Tools That Will Help You Create the Right Mobile & Web Test Strategy

05/02/2023 1© 2016, Perfecto Mobile Ltd. All Rights Reserved.

How to Create the Right Mobile & Web Test StrategyMay 2016

Eran KinsbrunerDirector, Tech. Evangelist

Perfecto

Patrick McCartneySenior Sales Engineer

Perfecto

Some housekeeping before we start


Submit questions using the Q&A panel.

Participate in the polls.

Watch your email for a copy of the

recording and slides.

Complete the survey following the presentation.

Agenda for Today


Web CoverageAppendixBest

PracticesThe

OptimizerIndex Intro

Mobile Coverage Method

The Challenges

The Test Coverage Conversation – What To Answer?

Which Devices & Browsers?

Which OS Versions?

How Many platforms are

enough?

When should I refresh my test

lab?

Who is in charge of coverage?

How to size my lab across

projects and teams?


Introducing The – Digital Test Coverage Toolkit


Test Coverage Optimization Process

2. Lab Sizing

Teams Projects

SDLC Cycle

Requirements

How many?

v

Me

Analytics

Market

My Space

Popular & emerging

1. Coverage Mix

Industry trends & insights

Which?

Test Coverage Optimization Process

Me

Analytics

Market

My Space

Popular & emerging

Industry

Current state

Internal

External

Future state

The Index

The Optimizer

What does it take to cover the REAL end user experience?

v

Device & Platform

Model

OS Version

Screen Size

browserv

Environment

Location

Network

Phone events

Other apps

v

Conditions

Signal

Load

CPU

What’s new?• Brazil, China, Netherlands• Usage vs. Purchase Model

Digital Test Coverage Index 4th Edition

2016 Calendar and Forecast


Market Events Which Impact Your Test Coverage Decisions

11© 2015, Perfecto Mobile Ltd. All Rights Reserved.

&

Mobile Test Coverage Method

Mobile Device Coverage - Attributes

• Device and OS popularity (market share)• Screen sizes, resolution and other screen attributes such as pixel per inch (PPI)• Device age (launch date)• New and trending devices and platforms• Operating system version update rate (e.g. reference devices like Android Nexus

get a higher score)• Unique device properties important for testing purposes – chipset, CPU,

memory• Audience demographics


Reference Devices/OS

Popular Devices/OS

Legacy Devices/OS

Emerging Devices/OS

Reading The IndexRecommended

OS version

Pixels per Inch

Trend

Types

52%

35.5%

Recommended OS – Usage & Quality Rationale

Usage & Quality Issues

The Optimizer


Optimizer Live DEMO


LIVE DEMO

Test Coverage Recommendations

Leverage market calendars and stay updated Refresh/Review your test lab once a quarter Beta versions are no longer an option

Chrome, FireFox, iOS Leverage analytics but validate and calibrate using market trends Real user conditions extends your test coverage Assign an owner for digital test coverage but keep the conversation open to all

teams Mix your mobile test lab with the 4 device types


Digital Test Coverage Guide


Lab Sizing


QUALITYVELOCITY

16 DEVICESREQUIRED COVERAGE 2 WEEK

SPRINTSGOAL: 400 REGRESSION TESTS TO RUN IN 48 HOURS

ARE 16 DEVICES ENOUGH TO COMPLETEFULL REGRESSION IN 48 HOURS?

Quality vs. Velocity

COVERAGECAPACITY

EACH DEVICE EXECUTING 400 TEST CASES

48 HOURSDESIRED

REGRESSION TIME

32 DEVICES REQUIRED FOR FULL COVERAGE IN 48 HOURS

ACTUAL REGRESSION TIME

16 X

Meeting Velocity Goals

10 MIN * 400 TEST CASES = 67 HOURS (2.8 DAYS) 67 HOURS – 48 HOURS = 19 HOURS

19 HOURS / 48 HOURS = 0.4 (1 ADDITIONAL DEVICE PER DEVICE TYPE)2X DEVICES REQUIRED

2.8 DAYS

Balancing Velocity and Coverage

Apple iPhone 6Apple iPhone 6 PlusApple iPad Air 2Apple iPhone 5SApple iPhone 6SApple iPad miniApple iPad 2Apple iPhone 6S Plus

Primary {{Secondary

Samsung Galaxy S6Samsung Galaxy S5Samsung Galaxy S4Samsung Galaxy Note 5Google Nexus 5XHTC One (M9)LG G4Samsung Galaxy Tab S2

Primary

{Secondary

{iOS Android

75 High Priority Tests

200 Medium Priority Tests

125 Low Priority Tests{ }RUN ON PRIMARYDEVICES (400 TESTS)

RUN ON SECONDARY DEVICES (275 TESTS)

PRIMARY

25 TOTAL DEVICES FOR OPTIMIZED COVERAGE

PARALLELCAPACITY

10 MIN * 400 TEST CASES = 67 HOURS (2.8 DAYS) 67 HOURS – 48 HOURS = 19 HOURS

19 / 48 = 0.4 (1 ADDITIONAL DEVICE PER DEVICE TYPE)2X DEVICES REQUIRED FOR HIGH PRIORITY TESTS

10 MIN * 275 TEST CASES = 46 HOURS (1.9 DAYS) 7 SECONDARY DEVICESSECONDARY

Optimized Coverage

18 X

7 X

←

Q & A


Practical Toolkit for Webinar Attendees


perfectomobile.com/coveragetoolkit

Thank You

3 Free Tools That Will Help You Create the Right Mobile & Web Test Strategy

Mobile

Transcript of 3 Free Tools That Will Help You Create the Right Mobile & Web Test Strategy