Centralized Test Automation Framework Web viewAutomation Framework is a process of Structuring and...

Dharmendra BhangaleLead Test Architect

Rave Technologies Pvt. Ltd.

Centralized Test Automation Framework For Web Application

Centralized Test Automation Framework for Web Application

Abstract:

This document is related to the challenges faced during functional and regression automation testing for web applications.Test Automation frameworks are complex to develop and maintain. In this white paper we provide the several guideline and strategies to creating successful Test Automation Frameworks.The purpose of this document is to define a common set of Test Automation Development guidelines for Test Automation Group (TAG).The guidelines are the product of a set of mutually accepted standards and best practices to be implemented by all members of testing team in TAG. While this document covers many aspects of the test automation processes as they apply to automation, the primary focus is on automated test scripts.

Automation Testing of 18


Contents

1. Agenda............................................................................................................................................... 4

2. Factors Deciding Automation Testing..........................................................................................4

3. Challenges in Test Automation......................................................................................................4

4. Benefits of Automation.................................................................................................................... 5

5. Components of Automation Testing.............................................................................................5

6. Reason Why Automation Testing Might Fail?..............................................................................6

7. Successful Automation Testing......................................................................................................6

8. Centralized Test Automation at Web Based Projects................................................................7

9. Automation Guidelines for Testers................................................................................................7

10. Guidelines to identify Test Case to automate.............................................................................8

11. About Centralized Automation Framework...................................................................................8

12. Centralized Automation Framework..............................................................................................8

14. Features of Centralized Automation Framework.........................................................................9

15. Architecture of centralized Automation Framework.................................................................10

16. Centralized Framework - Work Flow Design...............................................................................10

17. High Level Centralized Framework Design.................................................................................11

18. Centralized Framework Components..........................................................................................12

19. Centralized Database..................................................................................................................... 12

20. Centralized Global Environment...................................................................................................12

21. Centralized Function Libraries......................................................................................................13

22. Centralized Object Repositories...................................................................................................13

23. Tools, Scripts, Recovery scenario & Results..............................................................................14

24. Centralized Result Log Creation...................................................................................................14

25. Benefits of Centralized Framework over other frameworks:...................................................14

26. Conclusion:....................................................................................................................................... 15

27. Bibliography:.................................................................................................................................... 16



1. Agenda

Automation Decision Making Challenges of Automation Benefits of automation Components of Automation testing Reasons for Automation failure Successful Automation Testing Automation Decision making Process Automation Guidelines for Testers Identify Test Cases to automate Hybrid Framework Approach Elements of a Framework

2. Factors Deciding Automation Testing

3. Challenges in Test Automation


Test

Au

tom

atio

n

Project Size

Minimizing the Rework

Driving the value

Functional complexity

Application consistency

The Core Object of Automation is to provide 100% test Coverage

Automation Suites should be accurate. Customers expects the reliable results from automation testing on every run.

High Quality Testing

Automation process should support future enhancements Automation May fail due to critical functionality /New technology

New Technology

The Inconsistency state of application will prevent the Automation to score the Estimated ROI Achieve the Planned ROI

ROI

The Core Object of Automation is to provide 100% test Coverage

Complete Test Coverage


4. Benefits of Automation

5. Components of Automation Testing

Automation Frame Work :Automation Framework is a process of Structuring and Organizing Scripts to increase the level of reusability and readability of scripts.

Tool :

Is a mechanism used to simplify and speed the effort required for testing?

Regression Suite :

A collection of Regression scenarios in the form of automated build.


Benefits of Automation Framework

Standard Process in Production

Free From Dependencies

Future Enhancements

Support

High Quality of Automation with low

cost

Reusability & Repeatability of

scripts

Regression SuiteToo

l Framework


6. Reason Why Automation Testing Might Fail?

While talking about automation decision making, we have to list out the reasons for automation failure. This activity will help you to create a quality in automation testing.

Reason for Failure Best Practice

Inaccurate Expected Results Define in scope and out of scope functional scenarios

Unclear Functional Requirement Create Guidelines to testers to analyze functional scenarios

Inconsistent Application Behavior Study Application before you start automation.

Dependent Automation Scripts Define Automation Framework and Process

7. Successful Automation Testing


Decide what Test Cases to Automate

Test Early and Test Often

Select the Right Automated Testing Tool & Framework

Divide your Automated Testing Efforts

Create Good, Quality Test Data

Create Automated Tests that are Resistant to Changes


8. Centralized Test Automation at Web Based Projects

9. Automation Guidelines for Testers



10. Guidelines to identify Test Case to automate

Validation Yes/NoAccurate Expected Results YesClear Functional Requirement YesConsistent Functional Behavior YesDependence on other systems YesCore functional component YesTest Data Availability and Connectivity Yes

11. About Centralized Automation Framework

About Automation Framework :

A framework can be a wrapper around complex internal architecture which makes end user to interact with system easily. It also defines guidelines and set standards for all phases of ADLC (automation development life cycle).

Centralized Automation Framework :

The centralized Test automation framework is what most frameworks evolve into overtime and multiple projects. The most successful centralized frameworks generally accommodate both Keyword driven as well as Data driven This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture. The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been

12. Centralized Automation Framework

Automation Script encapsulation:Function Libraries, Scripts all are encapsulated components, Users will be able to access only the Scripts related to their testing user get prevented from accessing the entire framework scripts.

Systematic Approach:Automation Testing of 18

Centralized Hybrid Automation Framework - Diagram

Centralized Driver Script

Project 1

Project 2

Project 3

Centralized Automation Framework

Centralized Input Data


Defines a standard procedures, processes, best practices and guidelines for test automation implementation. A tester can’t create a script on this own style or way for the project test automation.

Quality of automation: The Framework will be evaluated in different levels to object quality in automation.

Framework Design:Hybrid model has been used to design the centralized automation framework.

13. Extendable to new technology using Centralized Framework

Note: Assume that web technology is implemented in framework and “sap” is a new technology that’s need to be implemented. So Project 1 and project2 can be implemented immediately and project 3 needs an up-gradation in the framework.

14. Features of Centralized Automation Framework


Automation

Framewor

k

Defining

processes to

obtain

quality of auto

mation

Centralized Automatio

n implementation

Automatio

n Script encapsulati

on

Systematic Maintenance of auto

mation

scripts

New Project 3

Other Technology

(SAP,Siebel,CRM)

Application Function Library

Utilities Function Library

Generic Function Library

Centralized Data SystemCentralized Result System Global

Configuration

Centralized Framework

Start Up

ScriptsFramework ready to implement test automation

Framework ready to implement test automation

Project 1

Technology –

Web(.NET)

Project 2

Technology –

Web(Java)

Framework ready to implement test automation


Design of the Framework is independent of “Functionality & Technology” Minimum level of maintenance and framework change requirement Framework should provide user friendly Regression Suite Systematic Approach in automation implementation On the shelf Product – Ready for the current market requirement.

15. Architecture of centralized Automation Framework

16. Centralized Framework - Work Flow Design

Framework design will be independent of functionality and technology

Object type and properties is checked up to the 7th level

Application type, Object Class, Object Name, Operation and Operation Type are the deciding factor of the automation flow of execution

Your Data Drives the Flow not the framework


Regression Suite

Centralized Framework

Application Under Test

Application

Driver Sheet

Main DataSheet

ResultSheet

Sub Script

IF Flag Y/N

No

YesPerform Validation

Driver Script

IF Flag Y/N

Yes

No

Main Script

IF Flag Y/N

No

Yes

Perform Validation

SubSuit

e YesNo

Sub Suite Folder

Regression Suite

Automation Framework Start


Framework will process the inputs and perform the operation in the application, update the result back to the suite,

If your data is incorrect then the suite will fail


Web

App Type

SAP

Object Class

Web Button

Web Edit Box

Operation

Checkpoint

Function

Operation Type

Std.CheckPoint

TxtCheckPoint

User Defined Function Call


17. High Level Centralized Framework Design

Driver Sheet :The Starting Point of Framework and will contain all the information about Regression Suite( like (Path, Suite Name, Module Name and Flag, etc )

Driver Script : Will check the Flag status of All the Regression Suite in the Driver sheet Flag = Yes ( Suite need to be executed) Flag = No ( Suite need not to execute)

Main Script : It will locate the regression suite and get data from the Main datasheet It will read all the rows in the datasheet, 1 to n rows, before process a row, it will check the Flag Status( Yes/No)

Main Datasheet : Each Row will contain technical data, test data and a flagOne line of Script will be converted into one row in Datasheet

Sub Script and Sub Data Sheet :We can call Reusable Suites inside your Reg. Suites

Function Library : Set of Reusable Functions

Result Sheet : Customized results for user s better understanding


N ex t

N ex t

Driver Script Driver Sheet

Main Script Main Data Sheet

Sub Script Sub Data Sheet

n

n

Frame work Automation Suite

Driver SheetDriver ScriptMain Script Sub ScriptFunction LibraryRecovery scenarios

Main Data Sheet

Object Repository

Reusable Suite

Result Sheet


18. Centralized Framework Components

Below are the components of the Automation

Framework:

• Data Tables • Environment• Libraries • Object repository• Results• Runtime• Scripts• Recovery scenario• Documents• Tools

19. Centralized Database

• Data table leads to Parameterization• Repository for Test Data• Designed for User Interaction• Data table can be an Excel Document, Ms-

Access, Database• Framework will Design the Data table Structure

and mapping

Username Password

User1 pwd1


Tools

Data Tables

Environment

Object Reposit

ory

Scripts

Librarie

s

Recovery scenario

Runtime

Docume

nts

Framework

Results


User2 pwd2

User3 pwd3

20. Centralized Global Environment

21. Centralized Function Libraries

Function library

Collection of Reusable operations in the form of Predefined Structure

Time and resource can be saved using reusable functions. Decomposing the scripts into functions, will increase the level of script understandably. Creating functions will reduce the dependencies within the scripts. Scripts maintenance cost will considerably will come down. Support Future requirements.

22. Centralized Object Repositories


Environment component is the initial setup for script execution

s Environment variables are the test environment credentials

Accessible to users to set prerequisitesApplication

Driver ScriptsFunction Library


The Storage band for Application Objects It Provides the interaction between scripts

and application

The object repository is shared and module-specific.

While Execution the Object repository will help the script to Identify the Objects in the application

Repository will have list of objects along with configured properties and their values

Object Repository will have enough no. of properties to uniquely identify an Object

Shared object repository should have the extn : ABC.tsr (Test Shared Repository)

Module-specific repository should have a extn : ABV.mtr (Module Test Repository)

23. Tools, Scripts, Recovery scenario & Results

Tools: This folder contains macro files. Using these files will reduce time for writing scripts, converting files, and shortcut to generate code.

Scripts: This folder contains local test scripts; this folder will be used while development and maintenance stage. Once script gets delivered it will upload to centralize location Test Director or Quality Centre.

Recovery scenario: This folder can be used as recovery scenario function library file. Also, through QTP particular scenario, converted file is kept under this folder and associates the test script.

Results: The current execution status will be updated after every run. All the Script failures, test status, recovery actions will be logged in Result.

24. Centralized Result Log Creation

Centralized Automation framework will create automatic result log in below format

PDF HTML EXCEL VIDEO RECORDED FILE WORD NOTEPAD AUTOMATIC EMAIL NOTIFICATION


Global Object Repository

Project 1

Object 1 Name

Object 2 Name

Object 1 Property

Project 2

Object 1 Name

Object 2 Name

Object 1 Property

Object 2 Property

Centralized Object Repository Structure

Object 2 Property


25. Benefits of Centralized Framework over other frameworks:

Benefits Other Automation frameworks

Automation Framework(Hybrid)

Centralized Automation Framework

Cost of Implementation High High LowExtendable to New technology

No No Yes

Functional Independence

Yes Yes Yes

Technology independence

Yes Yes No

Script Maintenance Required

High High Low

User Friendly No Yes YesEncapsulated Scripts No Yes YesReliable and Stable No Yes YesROI Low Medium HighReusability Low Limited High

26. Conclusion:

ROI is Approximately increase up to the 70 to 75 %

Minimum level of Script maintenance.

If there is change to any functional test case then only the functional test case file needs to be updated and the Driver Script and Startup script will remain the same. There's no need to update the scripts in case of changes to the application

Testing effort saved around 70% of overall efforts.

New resource can easily start his work with this centralised automation framework within 10-15 days.

Business point of view this is very economical, feasible and user friendly.



27. Bibliography:

1. HP-UFT User Manual

2. Automation Process Wikipedia (http://en.wikipedia.org/wiki/Test_automation)

3. http://www.automatedtestinginstitute.com/

4. http://www.knowledgeinbox.com/

5. www. automation repository.com

6. Dustin, Elfriede, Rashka, Jeff, and Paul, John (1999) Automated Software Testing: Introduction, Management and Performance. Addison-Wesley Longman, Inc. Reading, Massachusetts

7. Fewster, Mark and Graham, Dorothy (1999) Software Test Automation: Effective use of test execution tools. Addison-Wesley: ACM Press, New York

8. Kaner, Cem, Falk, Jack, and Nguyen, Hung Quoc (1999) Testing Computer Software. Wiley Computer Publishing, New York


http://www.automationrepository.com/

http://www.knowledgeinbox.com/

http://www.automatedtestinginstitute.com/

http://en.wikipedia.org/wiki/Test_automation


Author Profile:10+ Years of experience in software development, Oracle implementation, automation testing and performance engineering. Experience and varied knowledge of Telecom, BFSI, protocol testing, retail and banking, and finance Domain.In-depth knowledge of multiple automation testing tools like UFT 12.01, Silk Test, Rational Robot, Test Complete Open-STA, Strong Knowledge of Oracle, C++, Java and Seibel 8.0.


Centralized Test Automation Framework Web viewAutomation Framework is a process of Structuring and...

Documents

Transcript of Centralized Test Automation Framework Web viewAutomation Framework is a process of Structuring and...