1

vegaTest® - A Robust, technology

agnostic, Business User Friendly Test

Automation Framework

- Author

Suresh Garimella, General Manager – IT Services

2

ABSTRACT

DST vegaTest® Automation Framework is a leading-edge tool-agnostic and

business user friendly solution architected, designed and developed for

automating functional and regression testing. vegaTest® ensures:

� Tool-agnostic framework guarantees risk-proof transition to any

automation tool.

� Business User and Business Analyst friendly:

o Keyword-driven approach to develop test scripts and input test

data using Microsoft® Excel

o Abstraction from tool level complexities

� Cross-cutting capability ensures lesser time spent on test scripts during

Application re-engineering and modernization

� Standardization of test automation across different Business Units in

the company

� Higher Return on Investments resulting from reduced effort in Script

development, maintenance and porting from one automation tool to

another

� Test Reports and Metrics in Multiple output formats

3

KEY TAKEAWAYS

vegaTest® Automation Framework is an innovative test automation framework.

It delivers several business benefits to DST and its customers.

TOOL AGNOSTIC

There were several projects in DST that were migrated from QTP to Selenium

for business reasons. vegaTest® ensured:

� Proven 65 - 80% reduction in automation script migration effort

� Reduced tool dependency as it can be integrated with any new

functional or regression automation tool

Figure 1: Ease of converting scripts to a new tool using vegaTest®

BUSINESS USER AND BUSINESS ANALYST FRIENDLY

� Test data and scripts configured using keywords in MS Excel by

Business Users and Business Analysts

� Only 1,000 hours of effort were required to understand and set up

2,000 scripts, saving more than 19,000 hours of development effort.

� Vibrant User Community of over 150+ users

� Add-on tools further simplify the test script and test data creation

4

CROSS CUTTING CAPABILITY

� $40K saved by a Healthcare domain client (1800 person hours for initial

500 test scripts) during an application modernization / re-engineering

from COBOL to Java technology.

� vegaTest®ensures the re-use of test scripts generated on old

application code base.

STANDARDIZATION

� Over 90% of DST adopted vegaTest®and they internally use several

automation tools at the backend

� The ownership for test Scripts development and maintenance is with

Business Users and Analysts. It ensures documentation of domain

knowledge.

SAVINGS AND RETURN ON INVESTMENT

Figure 2: Savings and RoI – vegaTest® vs. Conventional Approach

� Proven reduction of effort

o 30% reduction in Script development

o 60% reduction in Script maintenance

� Over $800K licensed revenue

� Savings of over $990K across several DST Business Units (@an average

of 2 hours per script X 22,000 scripts)

� Over $40K saved in Application reengineering and migration projects

5

TEST REPORTS AND METRICS IN MULTIPLE OUTPUT FORMATS

� Extensive test reports generated for every run

� Test Reports include screen shots of error

� Test reports in multiple formats including: Excel, HTML, etc.

6

BACKGROUND

INSTANCE 1 – KNOWLEDGE MANAGEMENT

At the enterprise level, DST has commanded deep knowledge and

experience handling IT software products for over 40 years. Our domain

knowledge cuts across business sectors including financial services,

healthcare, insurance, retirement planning, and business process

management. However, this knowledge has not been systematically

documented and integrated for future use of our teams.

The criticality of effective knowledge management came to the fore as

DST undertook to migration, modernization and re-engineering of its

legacy applications. As re-engineering got underway, DST found it

increasingly difficult to execute complete regression testing due to lack of

adequate domain expert resources.

INSTANCE 2 – RE-ENGINEERING

A large technology vendor for the health insurance field needed an

automated testing solution to offer customers as part of their software

product package. The vendor’s customers were spending six months on

testing with each new product release, with the effort to execute one full

regression test cycle (comprised of 2,000 scripts) taking an average of

4,000 hours. As this testing was not a continuous business need for the

customers, they were not staffed to perform dedicated testing and quality

assurance on this scale. Additionally, they were using manual test

processes supported by limited automated scripts and testing tools, and

lacked the resource capacity to create test scripts and support test

activities, while still satisfying everyday business requirements.

ANALYSIS

Against this backdrop, DST delved into the issue to analyze the

challenges before the organization.

7

� The domain knowledge is with the Business users and Business

Analysts. It is vital to enable them to trigger the test scripts and

test data.

� Converting manual test cases into automated test cases for

several clients involved engineers having domain knowledge

and Business Users having tool knowledge.

DST needed a solution to mitigate the risks and enable Business

Users, Business Analysts and Subject Matter Experts to develop the

automation scripts without having to understand the technicalities

involved in writing automation scripts.

HIGH LEVEL SOLUTION

DST vegaTest® Automation Framework is a leading-edge framework

designed and developed for automating functional and regression testing in

system environments. vegaTest® was developed by using industry-standard

process frameworks to provide clients with comprehensive end-to-end

automation capabilities. All while addressing, the key challenges of traditional

test automation approaches.

vegaTest® automation framework is enriched by its comprehensive reporting

capabilities and can support multiple sets of data. It supports repeatability at

decreased costs. This results in faster time-to–market, allowing subject matter

experts to spend more time in testing complex system functionality.

Here is a high level overview of the vegaTest® framework compared against

the traditional automation tool approach:

Business Benefit vegaTest® Approach Traditional Approach

Resource Skills Automation engineers,

business users, manual

testers and processing

engineers

Automation Engineers

8

Business Benefit vegaTest® Approach Traditional Approach

Script

Development

Faster, Using Keywords

and vegaTest® build-in

UI

Standard, Scripting

Coding

Standards

Applied Automatically Manually

Code Review Minimum Requires additional

effort

Data Access Rational, Flexible Multiple data files

Automation Tool

Licenses

TAF saves one license in a

set of 5

No Saving

Script Portability QTP, Selenium, Silk Test Supports only one tool

New Resource

Ramp Up

In days In weeks

Table 1: Business benefits vegaTest® Vs. Traditional Approach

INSTANCE – 3 TOOL AGNOSTIC

One of the important clients in Brokerage domain was using QTP

automation tool along with vegaTest®Automation Framework for

automating the testing requirements of a legacy Web Application. Its

business needs demanded to move the test automation to Selenium for

the speed of test execution and multi-browser support. It required moving

approximately 200 test cases from QTP to Selenium.

ANALYSIS

The challenge before DST was to have their test cases tool agnostic.

While vegaTest®Automation Framework was initially developed to

9

work with a QTP automation tool, its robust architecture ensured

that with very minimal effort it would also work with Selenium as

the back end automation tool.

TOOL AND TECHNOLOGY AGNOSTIC CAPABILITY

A quick enhancement to the vegaTest® Automation Framework ensured that

it is tool agnostic and compatible with:

� HP-Quick Test Professional (commercial tool)

� Selenium (open source tool)

� Quality Center (commercial tool)

� Zephyr (commercial tool)

It works seamlessly in automating test cases across technologies - Web

Applications, BPMS Applications, Mainframe Applications, Client-Server Applications and

Desktop Applications.

10

ARCHITECTURE

DST undertook research on defining an automation framework that can act as a

catalyst to standardize the test automation approach across the DST Enterprise.

The product landscape of DST included several legacy applications, BPMS

applications, Web based applications, each of which were using different

automation tools. Standardizing the automation approach across all these

complexities involved developing the vegaTest® Automation Framework to

take inputs from a tool most used by the Business Users.

At the very outset, our research revealed that the business knowledge that

resides with individuals within our organization is a critical part of our solution.

Hence, the active participation of our business users was critical to the success

of our framework. Our survey showed that the majority of DST users had been

using MS Excel for more than 25 years, so we developed the input interface

using MS Excel.

Let’s have a look into the vegaTest® Automation Framework architecture.

Figure 3: vegaTest® Automation Framework Architecture

vegaTest® AUTOMATION FRAMEWORK: It is a packaged solution that provides all the

components required to build and execute scripts with several automation tools

11

over multiple environments. It provides:

� ability to configure multiple test data sheets

� tools to perform global data edits

� flexibility to share the scripts and test reports

AUTOMATION TOOLS: The current support includes automation tools like

QTP, Selenium, Quality Center and Zephyr. The internal architecture of

vegaTest® Automation Framework makes it easy for future-guarding it

against any new automation tools or upgrades in the current tools.

DATA: The test data can be configured in excel sheets by business users. It

provides for the flexibility to configure multiple sets of data.

SCRIPTS: End-to-end application test scripts can be developed in excel

sheets by business users using keywords. A jump start of 1,000 hours of

effort was required to understand and setup 2,000 scripts, saving more

than 19,000 hours of development effort.

CONTROLLERS: Business users can configure in Excel sheets the location

from which data is picked up and the test scripts are executed.

RESULTS: The test results can be downloaded from the framework in

multiple formats including Excel, HTML and PDFs.

APPLICATIONS: vegaTest® guarantees seamless connection with

applications developed on several technologies including Web

technologies, Mainframe, BPMS, Client-Server and desktop.

12

TOOLKIT

vegaTest® also has add-on tools that help in ready access of test objects and

controls from a repository, data consolidation from multiple excel files and

optimal utilization of test machines.

AUTO-GENERATE USER INTERFACE

The Auto-Generate User Interface (UI) tool minimizes the script

development effort by simplifying the test case creation process with a

plug-in for Microsoft® Excel that readily accesses test objects and controls

from a repository.

TEST DATA MANAGEMENT - CDF

The Composite Data File (CDF) tool manages test script data access from

multiple files. Our tool consolidates data from multiple test Microsoft®

Excel files to a single data Microsoft® Excel file. Using CDF, clients can

add, update and clean-up test data files from a single data source.

TEST LAB VIRTUALIZATION - TLO

Test Lab Optimization (TLO) is an intellectual tool that controls and

minimizes manual intervention and optimizes the utilization of test

machines.

CONCLUSION

vegaTest® Automation Framework ensures:

� Future proof and Tool-agnostic framework

� Keyword driven framework for business users

� Componentization and Reusable test scripts

� Detailed test reports and metric

� Higher return on investment

13

ABOUT THE AUTHOR

Suresh Garimella works as General Manager, IT Services at DST Worldwide

Services. With 16+ years proven track record, he is the end-responsible

and business owner for Testing COE, AWD BPM Practice and program

manager for several projects across BFSI and Healthcare domains.

Previously, he worked with TCS as Engagement Manager and Delivery

Manager. He is a B.E from R. V. College of Engineering - Bangalore,

student at XLRI, certified HP Quick Test Pro SME and Open Source -

Selenium SME, author of white paper - Estimation Models in Testing, an

avid reader, cricket player and Wii gamer.

Connect with Suresh Garimella on LinkedIn.

ABOUT DST WORLDWIDE SERVICES

DST Worldwide Services India Private Ltd is a leader in delivering world-

class information technology and back office processing solutions to the

financial services, healthcare and insurance industries. With more than

2,000 associates across the United States, India, and Thailand, we provide

application development and support, testing, data management,

migration, infrastructure management, back office and consulting services

to customers globally. DST Worldwide Services is a wholly owned

subsidiary of DST Systems Inc., which is located in Kansas City, MO.

COMMITTED TO EXCELLENCE

We are committed to delivering excellence in test automation and

assurance services. Contact us today to discuss your project or to learn

more about how the DST Worldwide Services Testing CoE can become an

asset to your business.

For more information please contact one of our Client Relations Managers

at info@dstworldwideservices.com, call 1.855.809.5505, or visit our website

www.dstworldwideservices.com.

14

Bibliography

� DST Internal vegaTest® Documentation

� DST internal vegaTest® presentations

15

THANK YOU!