Mobile Testing Tools 101

T22 Mobile Testing

5/2/2013 3:00:00 PM

Mobile Testing Tools 101

Presented by:

David Dang

Zenergy Technologies, Inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

David Dang

With more than fifteen years in IT, David Dang has focused on test automation, quality assurance, and test process improvement. David has extensive experience in designing, developing, and implementing advanced test automation frameworks that provide higher ROI by significantly reducing maintenance while increasing reuseability. David has worked with more than 100 companies to assist and drive their test automation and QA projects. He works with all the major license-based test automation tools and open source alternatives such as Selenium. David is a mobile automation expert; an HP Certified Instructor (CI) for QuickTest Professional, WinRunner, and QualityCenter; and frequent speaker at QA and testing conferences.

4/16/2013

1

Slide 1 Software Peace of Mind in Action

Presented By: David Dang

Mobile Testing 101

STAREAST 2013


???

� Introduction

� Challenges of Mobile Testing

� Factors to consider when establishing a manual testing strategy to test mobile websites and mobile apps

� Conclusion

4/16/2013

2


???

Companies use mobile websites and applications to harness the power of mobile devices to increase sales, market products, create collaboration with internal resources, offer convenience to their customers, etc… With exploding usage, companies must perform more QA/testing on mobile websites and apps to minimize defect risks.

Introduction


???

� Why test web applications and apps on mobile devices?

� What are the challenges?

� How do you think mobile websites and apps affect the way we do testing?

Discussion

4/16/2013

3


???

� In addition to “traditional” testing challenges such as:

� Verify functionalities

� Verify business process flow

� Verify look and feel

� Verify data

� Verify user experience

� Mobile has other factors that impact testing.

Challenges of Mobile Testing


???

� Carriers – Verizon, AT&T, T-Mobile, Sprint, etc.

� The two common local network standards are CDMA and GSM. Each network has a different way of tunnelling the data into the mobile device. This tunnelling can have an impact on the speed and access to the web proxy of mobile application servers

� Devices – Smart Phones, Tablets, iTV

� New devices with more features and functionality are released often. Most of these devices must be tested to ensure the mobile application works correctly. The challenge is to keep up with the changes and ensure quality in the existing and new devices


4/16/2013

4


???

� OS and OS version – iOS, Andriod, Blackberry, Windows

� With the mobile Operating System (OS) segment being so volatile , it is difficult to determine the trend within the marketplace. To add to the complexity, many devices run on different versions of the same OS. The challenge is to pick the “right” combination to test, seemingly an impossible task

� Location Based Service

� Location based service is a tracking service based on a mobile device’s current location. It can be used for target marketing of customers in a certain location. This can take the form of coupons, special offerings, or incentives. In addition, location based services can be used to analyze consumer participation based on a region of the country



???

� Portability of test cases for mobile web applications and applications

� With the number of platforms, operating systems, and devices, it is critical that test cases can be ported from one device combination to the next. The challenge is to create a methodology that encourages reusability and portability.

� Network speed

� The network speed (Extended Coverage, 3G, and 4G) has an impact on the user experience. A slow responding mobile application will be interpreted by the user as poorly designed and low quality. The challenge is the ability to test across multiple network speeds to ensure the mobile application is functioning similarly across variations.


4/16/2013

5


???

� Agile vs. Waterfall development methodologies

� Agile – A software development methodology based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams

� Waterfall – A software development methodology based on a downward progression flow (like a waterfall) from Requirements � Design � Code � Testing � Production

Establish Manual Testing Strategies


???

� Agile Impact

� Testing will occur earlier

� Timeline will be shortened

� Requires more “technical” testers

� Must determine how to do integration and regression testing

� Waterfall Impact

� Time to market; testing takes too long

� Too formal for small mobile websites and mobile apps


4/16/2013

6


???

� Mobile testing variations compared to “normal” application testing

� Mobile device functionality: portrait/landscape, swipe, pinch, external button on the device, etc.

� For mobile web application, mobile browser translation of html, dhtml, javascript, etc.

� For mobile application, different OS translations of mobile applications

� Mobile features: location based, network speed, on/off network, etc.



???

� Mobile content vs. mobile website and app functionality

� Testing content look and feel

• How do websites look on a Smart Phone, Tablet, and iTV based on screen compression and expansion?

• Are there overflow of pages or objects?

• How does the scrolling feature work on portrait and landscape?

� Test mobile website and app functionality

• Reference data such as, all fifty states available on the drop down

• Functionality such as login, search for item, shopping cart, etc.

• Business process flow such as, create an order with an out of stock item


4/16/2013

7


???

� Is it better to test on real devices or simulators?

Discussion


???

� Real Device Testing – The use of physical devices to test mobile websites and apps. The tester physically interacts with the device to determine the validity of the website or app. There are solutions on the market that offer real devices on the cloud; Device AnyWhere and Perfecto Mobile

� Mobile Simulator Testing – Mobile simulators/emulators are typically web-based or a computer application that mimics the behavior of a mobile device. It is used by QA to test how the website and/or application will appear and function in a mobile environment


4/16/2013

8


???

� Real Device Considerations

� Determine devices and OS combinations

� Use analytics to determine necessary coverage

� Develop a migration plan to move mobile apps onto the device

� Develop a device maintenance plan

� Consider real device alternatives; DeviceAnywhere or Perfecto Mobile



???

Establish Manual Testing Strategies –Analytics to Consider

4/16/2013

9


???

Establish Manual Testing Strategies -Analytic to Consider


???

� DeviceAnywhere - DeviceAnywhere is a cloud-based Software-as-a-Service platform for testing and monitoring the usability, functionality, performance and availability of mobile apps and websites


4/16/2013

10


???

� Perfecto Mobile - The MobileCloud is a web-based Software-as-a-Service platform that allows testers and developers to access a multitude of real mobile devices that are spread across different geo-locations



???

� Mobile Simulator Considerations

� How closely the simulator mimics real device, e.g., OS, hardware, browser, etc.

� Purchase vs. open source simulator

� Will it work with both, mobile websites and mobile apps?

� Does it support multiple devices

� What functionality can be tested using the simulator; what are the gaps that must be covered by real devices


4/16/2013

11


???

� Some common simulation tools

� Opera Mini Simulator

� iOS Simulator

� Android Emulator

� MobiOne

� TestiPhone



???


4/16/2013

12


???

� Real Device Pros

� Test what the end-user will experience

� Able to test mobile functionality, e.g., portrait/landscape, pinch, swipe

� Able to test mobile hardware, e.g., location based, network speed, etc.

� Able to test mobile websites and app’s look and feel of the device

� Experience the performance of the websites or apps

� Real Device Cons

� Expensive – device cost, phone plan, data plan, etc.

� Limited by number of same device and OS, e.g. two iphone 4s running iOS 5

� Difficult to keep up-to-date on new devices

� Maintenance on all the devices



???

� Simulator Pros

� Cheaper than real devices

� Easier to setup for testing (most of the simulators are web-based or standalone applications)

� Can test multiple instances (run simulators on multiple PCs)

� Simulator Cons

� Not the real device

� Hard to determine real user experience

� Cannot accurately test mobile functionality


4/16/2013

13


???

� Identify various levels of testing for mobile

� Unit testing

• Test a unit of code

• White box testing

• Performed by development team

� Functional testing

• Layout and content are correct

• Mobile functionality such as pinching, swiping, and scrolling are working properly

• Each function of the mobile website or mobile app is working properly

• Perform both positive and negative tests



???

� Identify various levels of testing for mobile

� System testing

• Business process driven testing

• Test multiple functions at the same time

• Ensure data is correct from one function to the next

� User acceptance testing

• Verify the look and feel are consistent throughout the entire mobile website or app

• Ensure all the functionalities are working properly

• Ensure that the mobile functionality such as, pinching, swiping, and finger scrolling is working properly


4/16/2013

14


???

� Tips for manual testing strategies

� Understand the challenges of mobile testing

� Identify the device portfolio you must cover

� Understand the impact of the development methodology

� Determine if simulator, real device, or both, are needed

� Determine logical breakdown of activities including who does what, and at what time for QA/testing



???

It is necessary and critical to test mobile websites and mobile apps to reduce the risk of defects and failures:

� Understand the challenges of testing on mobile devices

� Understand the factor that impacts manual testing strategy

Conclusion

Mobile Testing Tools 101

Technology

Transcript of Mobile Testing Tools 101