Michael BurnsideTwitter: @SQAEvangel istBlog: http:/ /sqaevangel ist.blogspot.com/Website: http:/ /sqaevangel ist.com/

Software Qual i ty Assurance, Qua l i ty Engineer ing , and Web and Mobi le Test Automat ion Expert

02 Ma y 2014

EFFECTIVE TEST AUTOMATION DEBT STRATEGY AND PAGE/SCREEN DESIGN PATTERN USING DATA-DRIVEN TESTING PARADIGM FOR AUTOMATED WEB/MOBILE APP TESTING

Effective Test Automation Debt Strategy and Page/Screen Design Pattern Table of Contents:1. Mission Statement2. Automated testing innovations and pros/cons3. Decisions, Decisions….Which Technology to choose for

automated testing investment?4. Example of simple mobile app testing for iOS/Android

apps.5. Architecture6. Test Technology Stack7. Class diagrams8. Run the tests demo9. Q & A, Feedback

Effective Test Automation Debt Strategy and Page/Screen Design Pattern

Mission Statement

The purpose of this presentation to identify test automation concepts, technology and design options while offering a learning example using a mobile application.

Effective Test Automation Debt Strategy and Page/Screen Design Pattern Automated Testing Innovations: Pros/Cons (mobile)

1. Native language: ie, using the platform’s recommended programming language and test harness.

• Pros: less dependency on third party libraries and support for each API release immediately. Minimal refactoring.

• Cons: less flexibility and options possibly for test result reporting, trend analysis, and usually needs more “testability” from the actual app to be interacted with from the test app., requires test automation developers to proficient with code used (how hard would it be to find a QA person with Objective-C skills for iOS apps testing?)

Effective Test Automation Debt Strategy and Page/Screen Design Pattern Automated Testing Innovations: Pros/Cons (mobile) – Part 2

2. Expressive language/keyword-driven: ie, using a proprietary expressive language to send directives to a mechanism that can interact with cross-platform UI’s and simulate human interaction use cases and use assertions.• Pros: Usually a small amount of training to learn and use

language for testing. Possible contribution of non-technical resources to add tests. Some ability to test cross-platform. No testability changes on the app needed.

• Cons: dependency of technology on software development API’s and OS’s which change rapidly and can break existing test automation. Example: Calabash

Effective Test Automation Debt Strategy and Page/Screen Design Pattern Automated Testing Innovations: Pros/Cons (mobile) – Part 3

3. Data-Driven: ie, using many popular programming languages to interact with apps rendered on multiple platforms with lightweight software objects that interact with single mobile app screens and can test using simple workflows from files like Excel, java properties (.ini files), CSV.• Pros: tests. Ability to test cross-platform using the same code with

framework supporting simple abstractions. Minimal effort to test another app on a different platform. Non-technical resources can create their own test testes for their test environments with their own accounts. No testability changes needed.

• Cons: dependency of technology on software development API’s and OS’s which change rapidly and can break existing test automation. Usually need at least one skilled programmer on the QA team to facilitate the testing and have some object-oriented design knowledge.

• Example: Appium

Effective Test Automation Debt Strategy and Page/Screen Design Pattern Decisions, Decisions….Which Technology to choose for automated testing investment?

What are the best decisions for my company organization considering the following information:

1) Do I want to pay for a product that allows for test automation for my app?2) What are the investment risks?3) When is my timeframe for X amount of code coverage?4) What is the acceptable time/money investment for maintenance?5) Am I comfortable with the dependencies on OS/API/Test harness dependencies?6) Do I have enough technically capable resources on staff/available to invest a strategy for

long-term maintenance investment?

There are no right or wrong answers. Make the decision based on what you want to accomplish for short/long terms goals, or both. Regardless of what you choose, there is ALWAYS time/money investment in test automation.

Effective Test Automation Debt Strategy and Page/Screen Design Pattern Example of simple mobile app testing for iOS/Android apps

Let’s investigate a simple login application that is identical for iOS and Android mobile devices using the screen object design pattern using an entirely data-driven set of tests (no hard-coded tests).

Desired:1) Design and implement a set of automated tests using simple

object-oriented design patterns that are directed and validated for acceptance criteria using data-driven files.

2) Easy capability to extend capability of testing the same app on a different mobile device OS.

3) Easy maintainability for lowest future cost and time effort.

Effective Test Automation Debt Strategy and Page/Screen Design Pattern

Architecture:There are 4 basic components to the automated testing solution:1) Screen interfaces and implementation classes – interaction with the actual

mobile app happens here. 3rd party api calls happen here.2) Workflow classes – navigate from point A to point B in the application,

execute a consistent set of operations3) Base Screen and Test classes – utility encapsulating many widely-used

methods in the language of choice and 3rd party method simplification.4) Base functional test and screen classes per feature. Allows for

customization of testing using separate accounts/users/data environment setup.

5) Test implementation classes using data-driven framework to parse files in CSV, Pipe-delimited, CSV

Effective Test Automation Debt Strategy and Page/Screen Design Pattern

Test Automation Technology Stack:

1. Java2. Junit3. TestNG4. Appium5. Web/Mobile Test automation framework (can be downloaded from

SQAEvangelist.com – open source)6. Xcode (iOS)7. Android Studio ADT

Effective Test Automation Debt Strategy and Page/Screen Design Pattern

1) Create a single interface class for each screen that can be used for both iOS app and Android applications.

2) Create separate implementation classes (one for ios and other for android)

3) Create a single workflow that takes a parameter that indicates which screen implementation class to use. Both testing classes use the same workflow.

4) Data files drive the tests and can test positive and negative test cases.

Effective Test Automation Debt Strategy and Page/Screen Design Pattern

Example/Demonstration

Effective Test Automation Debt Strategy and Page/Screen Design Pattern

Thanks!

I sincerely hope this presentation was useful to you and that your company or organization implement the information and processes in this presentation to the fullest extent.

If you have any feedback, please feel free to get in touch with me (via Twitter) at @SQAEvangelist. I would love to hear very constructive comments on how to improve this presentation and make it more effective.