Universal test solutions customer testimonial 10192013-v2.2
-
Upload
testmagic -
Category
Technology
-
view
255 -
download
1
description
Transcript of Universal test solutions customer testimonial 10192013-v2.2
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
1
Redefining Test Automation by Innovating Possibilities
Universal Test Solutions 01 Apr 2013
Test Optimization to Align Business with IT
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
2
and what did we do to “ make the it happen”
Which has helped in Business Value Realization of our client
and what has changed with it
How we accomplished it….
1
2
3
4
5
and who we are6
What got us thinking …
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
3
Over past three decades, there has been an explosion of test automation tools and heightened customer expectations…
1994-2001 2001-2005 2005-2008 2008-2012
Variety of tools over generations
Frameworks for automation
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
4
but test automation is perceived as risky and failing to meet ROI expectations….
Cost of Failed Implementations 2004 2012 (Estimated)
Industry: Test Automation (net worth) $ 1 billion $ 6 billion
Automation Projects (failure cost) $ 1/2 billion $ 3 billion
*Ovum Study Projects the testing tool market to be worth 56 billion US dollars by 2013 growing at an rate of 9.5 %
Source A: ROI on Test Automation – http://www.keane.comSource B: HP pays 4.5 billion to acquire mercury - http://www.mytimes.com/2006/07/26/technology/26hewlett.html
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
5
Test automation is not as easy as it seems to be: just buy one of the popular test execution tools, record the manual tests, and then play them back whenever you want to.
despite steep investment, high cost skill deployment and uncertainty of costs
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
6
and what did we do to “ make the it happen”
Which has helped in Business Value Realization of our client
and what has changed with it
How we accomplished it….
1
2
3
4
5
and who we are6
What got us thinking …
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
7
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
7
Gen 3rd
Keyword Driven
Scripting
Gen 1st
Modular Scripting
Gen Zero
Record &
Play
Gen 4th
Hybrid Data + Keyword Scripting
Gen 2nd
Data Driven
Scripting
A Step towards innovating possibilities …..
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
8
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
8
Gen 3rd
Keyword Driven
Scripting
Gen 1st
Modular Scripting
Gen Zero
Record &
Play
Gen 4th
Hybrid Data + Keyword Scripting
Gen 5th
Introducing Script less Automation for UI and Non
UI (Web Services)
Gen 2nd
Data Driven
Scripting
by redefining test automation…..
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
9
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
9
XPATH Check Regular Expressions Descriptive Programming Fuzzy Logic
Artificial Intelligence Social Intelligence
Gen 3rd
Keyword Driven
Scripting
Gen 1st
Modular Scripting
Gen Zero
Record &
Play
Gen 4th
Hybrid Data + Keyword Scripting
Gen 5th
Introducing Script less Automation for UI and Non
UI (Web Services)
Gen 2nd
Data Driven
Scripting
We have redefined test automation…..
Artificial Intelligence
New Era in Testing- introducing revolution in test automation…
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
10
by eliminating challenges to Align business with IT
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
11
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
11
with drastically improved test automation creation and execution metrics
11
Key Parameters Other Tools
Test Magic
Remarks
Automation Script Creation % Effort of Overall Automation Effort
Automation Framework Approach & Architecture Finalization & Creation/Customization of the Automation Framework
15% 0% It’s Inbuilt in eZscript and can be implemented to any web based product directly.
Creation Of the object Repository (GUI/Objects Learning)
10% 0% eZscript has intelligence to identify object directly from the web page.
Creation of Reusable Function Library 10% 0% eZscript is having action based keywords hence grouping of actions creates a functions/Business Component.
Creation of Keyword driven (KWD) automated test scripts and converting it to data driven
40% 25% eZscript has the inbuilt capability and creates the KWD script during manual testing.
KWD Review & Dry run 15% 10% Debug Mode Execution enable to test script with different set of data.
Fixing and tuning of Object Repository/ Automated Framework
10% 5% Artificial Intelligence; No Object Repository.
Automation Script Execution
Automation Test Management 10% 5% Test Magic has UI based test management.
Automation Execution 70% 40% Execution is faster because of the Artificial Intelligence approach to identify object
Reporting the issues/defects from team and merging/publishing reports of a release
20 % 15% Test Magic has huge web based Report Dashboard which can be emailed or printed .
Approx. 60 % Effort Saving Approx. 60 % Effort Saving
Approx. 40% Productivity ImprovementApprox. 40% Productivity Improvement
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
12
and improving product quality, increasing test infrastructure utilization and enhancing operational efficiency …
Product QualityProduct Quality
Operational Efficiency
Operational Efficiency
Automate more regression testing, achieve greater test coverage
Automate more regression testing, achieve greater test coverage
Automate testing both UI and non-UI including SOA architectures
Automate testing both UI and non-UI including SOA architectures
Collaboration(manual & automation)
Collaboration(manual & automation)
Eliminate repetitive manual tasks, minimize errors and
troubleshooting
Eliminate repetitive manual tasks, minimize errors and
troubleshooting
Share expensive assets across multiple projects and user groups
Share expensive assets across multiple projects and user groups
Max Resource Utilization- Manage and automate allocation of
automation test lab assets, Test Sets, and execution of automated
test scripts.
Max Resource Utilization- Manage and automate allocation of
automation test lab assets, Test Sets, and execution of automated
test scripts.
Reduce capital and operating costs by improving scalability
Reduce capital and operating costs by improving scalability
Easy to use by all levels of testing staff
Easy to use by all levels of testing staff
Test Infrastructure
Utilization
Test Infrastructure
Utilization
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
13
and what did we do to “ make the it happen”
Which has helped in Business Value Realization of our clients
and what has changed with it
How we accomplished it….
1
2
3
4
5
and who we are6
What got us thinking …
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
14
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
14
Background and client situation
Background
Client is a software product MNC in USA having complete solution for Cards and Payments. Their
product interfaces with multiple applications such as
–Visa/MasterCard - Analytics system
–General Ledger - Payment Gateway
–ACH - Embossing Vendor
–IVR - Credit Bureaus
–Dialer - Statement & Letter Production Vendor
Client Situation & Challenges
The Client faced the following challenges with respect to cards testing across their global delivery centers
- Real-time processing - Online, real-time access to data
- Complex authorization parameters and rules testing - Effective dating of all transactions and rules
– Flexible, user-defined billing cycles - Flexible pricing and payment terms
– Generic model for the organization – encompassing all applications - Multiple languages
– Multiple team support (over multiple locations) - Multiple currency capabilities
– Multiple environnent support (for multiple applications)
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
15
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
15
TestMagicTM Empowered
~10 % improvement in
Schedule
~ 75 %
Automation coverage
~ 23 % improvement in
defect verification of Showstopper
bugs
Overall 27% reduction in cycle
time
Before TestMagicTM Implementation
•Resource-intensive and time-consuming Most of the Testing exceed 50% of development cost
•Automation Test Script Creation, Execution and Result Analysis was time consuming and need SME for all.
•Low Test Coverage 50% resulted in High Defect Slippage in Production
•High Learning Curve
After TestMagicTM Implementation
•Test execution cost came down from 25 USD to 10 USD in first release
•40% reduction in cycle time
•100% Test Coverage
•Accelerated Time To Production
•Increase in 10% Adhoc Bugs
•Reduces the learning effort by 82%
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
16
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
16
with acknowledgement and recognition from the client …….
Test Magic is comprehensive testing automation software that we have been using for more than 2 years in Production. We are fully satisfied with Test Magic’s software and the services they have provided us.
Test Magic is a highly flexible automation solution designed for complete web application testing including Rule based File testing, validation of data in Database. TM’s team has delivered numerous customizations we asked for with short turnaround time helping us meet our stringent delivery schedule. The software is capable of providing high throughput and is very scalable. Its processing speed is limited only by the number of machines (processors) that are used. This is very critical for regression testing of complex financial software such as ours. The product is easy to use and can be mastered in as less a couple of days. TM’s support team has also provided us prompt and excellent technical support.
I believe that the product derives its value from its use of latest technologies which are coherent with most current Operating Systems and web browsers. Test Magic continuously evolves itself by adding new features such as SOAP based Web Service testing and can meet the challenges of the highly integrated and automated systems of today’s enterprise"
Service Category: IT ConsultantYear first hired: 2010Top Qualities: Great Results, Expert, Good Value
Liliya BurshteynDirector CoreCard Software, Inc
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
17
and the “delight of being able to do what was thought…..
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
18
and what did we do to “ make the it happen”
Which has helped in Business Value Realization of our client
and what has changed with it
How we accomplished it….
1
2
3
4
5
and who we are6
What got us thinking …
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
19
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
19
Typical ROI Mantra
ROI (%) = (Savings from automation) / (Costs of automation)
• Identification of the automation tool / Methodology.
• Test automation framework/solution creation based on best suited automation methodology as per need such as
o Record & Play or o Scripting/Functional Decomposition or o Keyword/Data Driven or o Hybrid approach
o Customization of automation existing solution
• Define the automation test strategy & approach
• Framework/Solution validation to confirm its correctness that It is rightly build
• Automation of the manual test cases selected as automation candidates
• Automation Test Execution
• Reporting the issues/defects from team and merging/publishing reports of a release
Hmmm…… ROI needs significant investment of effort and
money
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
20
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
20
Our ROI Mantra
• Identification the automation tool / Methodology.
• Test automation framework/solution creation based on best suited automation methodology as per need such as
o Record & Play or o Scripting/Functional Decomposition or o Keyword/Data Driven or o Hybrid approach
o Customization of automation existing solution
• Define the automation test strategy & approach
• Framework/Solution validation to confirm its correctness that It is rightly build
• Automation of the manual test cases selected as automation candidates
• Automation Test Execution
• Reporting the issues/defects from team and merging/publishing reports of a release
Automatic
ROI Simplified By
Innovating Possibilities
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
21
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
21
Typical automation breakeven cycle
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
22
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
22
Our breakeven is achieved soon as you start and increases exponentially through various levers…
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
23
and what did we do to “ make the it happen”
Which has helped in Business Value Realization of our client
and what has changed with it
How we have accomplished it….
1
2
3
4
5
and who we are6
What got us thinking …
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
24
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
24
• TestMagic (SaaS Compliant)– Automation Lifecycle Management (ALM)
• eZscript– Scripting & Execution Engine
• MagicSchedular– Distributed and Remote Scheduling (Execution Load Balancing)
Test any applicationOn any Windows OS
Test any applicationsWritten in any language
VB. NET Win32 ActiveX Delphi PHP C# ASP
JavaScript Java Python C++ AJAX HTML Perl
TestMagicTM Enterprise Solution: A Snapshot
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
25
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
25
File Parsing/Testing Database Testing ApplicationUnder Test
SOAP Services
API
CSV XMLText
25
TestMagicTM in a nutshell
Unique Features:
•Automatic Script Generation (No-programming )
•Automatic Step Description Generation
•Automatic Load Distribution – Group Scheduling
•Multi Country Support
•Multi Lingual Support
•Robust Automation Life Cycle Management
•Test Data Creation and Data Quality Checks
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
26
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
26
TestMagicTM Architectural View
26
TEST LAB CONFIGURATION
MANAGEMENT
Initialization of Parameters
Configuration settings
Test Lab Management
Super UserCreate Projects
Backup & Recovery
Project Admin
Tester(s)
AssignProject Admin
Create/ManageTester(s)
Manage Test Suite
Create Scripts
Schedule RunExecute Test Suite View My Report
SERVER
DB
DISTRIBUTED EXECUTION
AUT 1 / AUT under Environment 1
REPORT DASHBOARD
TEST RESULTS
E-MAIL TEST RESULTS
User Status Report
Execution Status Report
Test Verification Report Blocked Test CaseReport
Module Stability Report
Manage Test Environment
View All Reports
AUT n / AUT under Environment n
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
27
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
27
eZscript Business Component Snapshot
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
28
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
28
eZscript Business Process Management Snapshot
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
29
Evolution of TestMagicTM @ Cloud
• TestMagicTM not part of emerging or core technologies/application supported by
corporate IT of client.
• Well-suited for incubation as Server and Storage requirements not completely
known
• Scalability as per demand
• No extra configuration /cost required by TestMagicTM.
• Test Automation Development and Execution can be also done on a private cloud
Client Management
TestMagicTM @ Cloud : data-intensive, compute-intensive,
storage-intensive
TestMagicTM @ Cloud : data-intensive, compute-intensive,
storage-intensiveIndia Office
Romania Office
Atlanta OfficeCentralized IT Support
Client’s Client
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
30
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
30
TestMagicTM @ Cloud/WAN and eZ Virtualization
MS SQL/MY SQL Automation Life Cycle Management eZscript DB Server ALM Server Script Engine
eZ Virtualization
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
31
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
31
TestMagicTM - Strategic Benefits
Test Magicis designed in a way that it reduces the need of highly skilled automation experts for normal
day to day automation activities like defining of business flow,
scripting of test cases etc in less time.
Test MagicArchitecture is designed for
distributed processing and highly scalable to support large set of test cases
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
32
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
32
TestMagicTM Demo Strategy
Today’s TestMagicTM
Demonstration focus
• eZscript Overview
• Business Component and Business
Process Workflow Creation overview
• Data-driven creation demonstration
• Multi-Country support demonstration
• Multilingual support demonstration
• TestMagic (SaaS-ALM) overview
• Q&A
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
33
and what did we do to “ make the it happen”
Which has helped in Business Value Realization of our client
and what has changed with it
How we have accomplished it….
1
2
3
4
5
and who we are6
What got us thinking …
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
34
Introduction – Universal Test SolutionsAbout Universal Test Solutions
• MNC operating from India and UAE.
• Formally Started in 2009 (work started in 2007)
• Specialize in Testing and Test Automation
Solutions to provide Sustainable, Scalable and
Adoptable Solutions
Our Strategy
– Simple and Complete Solution
– Innovation by implementing concept of
Artificial intelligence in Test Automation
– Simplified approach to achieve Quick ROI
– Automation Testing For Everyone
– Continuous Improvement
Our focus
•Business Value Realization by improving
– Product Quality
– Test Infrastructure Utilization and
– Operational Efficiency
•Test Automation to achieve:
– Reduction in Total Cost of Ownership
(TCO) by 45% to 60%
– Reduction in the total time to
automate and execute tests by 50%
to 70%
– Get ROI from Next Cycle
– Negligible maintenance costs
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
35
Introduction – Universal Test Solutions
Our Vision
To provide comprehensive solutions to meet various needs of Testing.
Our Values and Beliefs
Our values of unflinching innovation for the client and the world
Our belief in our ability to constantly generate ideas that add new dimension to test all the
areas of your business application
What drives us…..
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
36
Banking, Financial
Services and Insurance
Manufacturing and
Logistics
HealthCare
Telecom
Retail
36
Service Spectrum for YOU
• Perform Feasibility Analysis• Assess test automation requirements; • Develop POCs; • Define/Review Automation objective, approach /
strategy• ROI Analysis• Define Short term and Long term
automation roadmap• Mentor/Manage implementation of test automation
Test Automation ConsultancyTest Automation Consultancy
• Technical support (L1, L2 and L3 including
Query resolution and Training )• Continuous Research and Strategic TestMagicTM
Development including bug fixes and
customization as per client need.• Seminar, Workshops,
Training and Certification
TestMagicTestMagicTM TM Licensing and SupportLicensing and Support• Test Automation Management- Estimate, Plan and
manage test automation assignments • Establish or support to implement Automation
Test Lab• Develop and Maintain test automation using
TestMagicTM • Define Automation methodology • Formulates Scripting Standards,• Develop Automation Templates and Traceability
Matrix
Test Automation ServiceTest Automation Service
• Business Coverage Analysiso Business Criticalityo Risk Analysis
• Test Program Management• Identification of End-to-End functionality • Test Scenario Identification and Preparation• Test Planning, Test Design and Execution• Test Reporting and Defect Management• Optimized Test Execution• Test Metrics Management
Independent Functional TestingIndependent Functional TestingSystem Integration
testing
Regression Testing
Risk Based Testing
E2E Testing
Agile Testing
Mobile Testing
UAT Testing
Web Services Testing
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
37
Milestones.….
2007- 2008
2009
2010
2011
2012
Approach, initial design finalized for a generic automation solution.
Development started
Release of TestMagiceZscript
mSchedulerwith Excel as
backend database
Release of TestMagic version with SQL
Server as backend database
Implemented TestMagic Solution in
3 locations (India, Atlanta and Romania)
Release of
MultilingualMulti Browser
Multi Windows OSMultiCountry
Enhanced eZscript UI for improving User
experience
Rule based File Operation/
ParsingFunctionality
SOAP based Web Service Testing
Functionality for Report reconciliation
Distributed Centralized Scheduling Mechanism
Quick Script Creation
Creation of Manual Steps while navigating
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
38
Q & AAverage 25 % increase in Automation Health Index
©2012 UTS Global, Confidential and proprietary. All rights reserved worldwide.
39
Thank youAverage 25 % increase in Automation Health Index