Test Automation

An Approach to Automated Software Regression Testing

Presented by Adnet, IncFeb 2015

Introduction• Automates manual repeatable test processes using formal testing procedure• Controls test execution, comparison of actual outcomes to predicted outcome

with test preconditions• Saves time and deduces cost by automating repeated manual testing

Why Test Automation• Repeated manual test prone to human errors (overlooking)• Learning cost of new test teams or new members• Regression test cycles are increasing with distributed development• Wider acceptance of Agile model (needs frequent and more tests)• Manual test data setup with multiple data-sets increases complexity

2

Test Automation – Introduction

• Scope and Approach (each software, not enterprise)• What to Automate (often repeatable tests- regression tests)

• Smoke Tests (undocumented exploratory tests)• Individual often repeated test cases• More useful for key business function where accuracy required• Easy accommodates variation in test datasets

• What not to automate • Stress and Load Test

• Need different tools• Different Strategy

3

Test Automation – Scope and Approach

44

• Reduce Cost • Reduce time in testing • Reduce cost from automation• ROI: Gain time in new function test• Reduce investment by using Freeware Tools rather than Vendor based

(Options)

• Automation strategy• Incrementally add more of the new functions for the next cycle of

regression testing • First automate smoke tests and then individual test cases• Baseline scripts and reuse each time.• Develop a library for commonly used routines like login for reuse.

Test Automation Strategy

5

Test Automation Strategy

6

• Record-Playback• Record once playback each time.

• Data-driven testing• One script runs with multiple sets of data.

• Modularity-driven testing• Common libraries for common functions like..

• Login() • Check_VA_Enroll()• Check VA Eligibility()• Exit()

• Keyword-driven testing• Keywords can make up a script..

• Initialization():Launch browser and login• Test1() :Test for a Requirement• Test2() :Test for GUI like labels, links• Exit() :Exit from the browser

Type of Frameworks

77

Test Automation Framework HP ALM Test Resources Module

HP ALM Test Plan /Lab Module

Configuration Files Test Data Files

Object Repository

Generic Functions library

Business Process Library

Documentation

Folders by Release

Automated Tests (QTP)

Test Set

Test Set

TestRunZ

TestRun

TestRun

TestRun

Test Machine / HEC App

Database client / drivers Run Scripts

Results

Detailed logs

Test Execution initiated from Lab

Results get uploaded back to Test Lab

Any supporting docs / user guides for the project

Scripts are organized into folder named after releases

Database drivers required in order query the database

Application runs in a browser

Results, logs and screenshots are stored locally for easy access

Screenshots

8

Results Generated After ExecutionA snapshot of test result file in excel format.

For Illustration Only

A detailed log file will also be created. Screenshots stored locally as well as in ALM.

9

Manual vs Automated Test Efforts

CYCLE 1 CYCLE 2 CYCLE 3 CYCLE 4 CYCLE 5 CYCLE 60.00

100.00

200.00

300.00

400.00

500.00

600.00

270.00 270.00

522.33 522.33 522.33 522.33

70.50

5.50 7.50 7.50 7.50 7.50

MANUAL HOURS PER CYCLE

TOTAL AUTOMATION HOURS PER CYCLE

10

Cumulative Savings

CYCLE 1 CYCLE 2 CYCLE 3 CYCLE 4 CYCLE 5 CYCLE 60.00

500.00

1000.00

1500.00

2000.00

2500.00

3000.00

CUMULATIVE MANUAL EFFORT

CUMULATIVE AUTOMATED HRS

CUMULATIVE SAVINGS

11

• Automated results generated in Excel or HTML- not dependence on the test software and format easy to analyze

• Manual tester released for new and on key features.• Automating regression test allow frequent small changes

requests.• Automation framework provides flexibility to change existing

SQL queries (for test data pull).• Easy to customize test data case for changes in test scenario• Huge savings in manual effort• Screenshots captured traced sources of errors and easy to

debug software

Benefits of Automation

12

Challenges for Automation• Higher initial cost• Higher skill set needed • More initial investment in infrastructure set-up• Availability of test data could be limited• Inconsistent application behavior• Application response times

13

Tools • Type of Tools

• Commercial Tools: • Support and maintenance advantage• High cost

• Freeware: • No support and depending on support groups• More labor intensive-higher implementation cost

• Adnet-HEC In-house Expertise in• QTP• Rational Functional Tester• Test complete• Selenium

14

Contact Information

Shri SinhaPresident

Adnet, Inc. Shrisinha@Adnetcmm.com

703-598-4485