SOLUTION TESTING - AN APPROACH FOR

TESTING MULTIPLE ENTERPRISE INTEGRATED

PRODUCTS

STC-2012 Conference Author:Rashmi Mahajan, Saroj Patnaik

Confidentiality Statement

This document contains confidential and proprietary information for Fiserv

and may not be disclosed to others than those to whom it was originally

distributed. It must not be duplicated, published, or used for any other purpose than originally intended without the prior written permission of

Fiserv, Inc.

DisclaimerInformation described in this document is believed to be accurate and

© 2012 Fiserv, Inc. or its affiliates.

Information described in this document is believed to be accurate and reliable, and much care has been taken in its preparation. However, no

responsibility, financial or otherwise, is accepted for any consequences

arising out of the use or misuse of this material.

Fiserv is an equal opportunity employer.Fiserv is a trademark of Fiserv, Inc.Other brands and names are the property of their respective owners. Information subject to change.© Copyright 2010 Fiserv, Inc.All rights reserved.

Author Information

Rashmi MahajanTesting Manager – Testing Center of Excellence

Fiserv India

Mrs. Rashmi Mahajan is Manager (Testing) at Fiserv Global Services, Noida.

She is a PMP certified professional having more than 12 years of IT experience

in development and testing projects. She is currently working as a Testing

© 2012 Fiserv, Inc. or its affiliates.

3

in development and testing projects. She is currently working as a Testing process consultant in Solution Delivery Process group.

Co-Author Information

Saroj PatnaikGeneral Manager – Testing Center of Excellence

Fiserv India

Mr. Saroj is General Manager (Testing) at Fiserv Global Services, Noida. He is a PMI®,

Prince2® and CSQP® certified professional having more than 18 years of IT experience

in consulting Testing Projects and having extensive experience in implementing Test

Automation Framework using HP and IBM suite of product. On his recent engagement as

Testing Practice Manager with Fiserv Global Services, he has been developing many

© 2012 Fiserv, Inc. or its affiliates.

4

turnkey automated solutions for various customers of Fiserv. Some of his key

implementation includes ALM implementation using SpiraTeam/HP ALM Quality Center

11.0, End-to-End Service testing using HP STM/Soap UI Pro, Performance Testing best

practices for a Batch Process System of a large financial product, Applying Risk Based

Testing using HP Quality Center and many more.

Abstract

Enterprise Solution Testing

becoming extremely important

Enterprise Solution Testing

becoming extremely important

Lack of a defined process

for testing the entire Solution before delivery

Lack of a defined process

for testing the entire Solution before delivery

© 2012 Fiserv, Inc. or its affiliates.

Need an approach to Solution Delivery that can helps better turnover and engagement of implementations team from sales team

Need an approach to Solution Delivery that can helps better turnover and engagement of implementations team from sales team

The individual product teams are aligned towards the product delivery

The individual product teams are aligned towards the product delivery

Problems at go live

We never have time to plan in the beginning, but we always have time to fix

our mistakes at the end.

- Author Unknown

© 2012 Fiserv, Inc. or its affiliates.

Outcome

Penalties

Over Budget

© 2012 Fiserv, Inc. or its affiliates.

Not Fully Functional

Defective

Approach

A defined Solution Testing process within Solution Delivery

Process can help address this problem.

Solution Testing a consistent cohesive approach to test Multiple

Enterprise Integrated Products before delivering to the Client.

Solution Testing includes set of services, standards, processes,

© 2012 Fiserv, Inc. or its affiliates.

Solution Testing includes set of services, standards, processes,and tools for planning and executing Testing for engagements to

ensure high-quality testing of integrated products.

Solution Delivery Process

Initiation

PlanningTransition

Engagement Delivery Team begin to get

engaged

Deal shifts from Sales process to

being driven by the Delivery process

Leadership shifts from delivery to the

operational support team

© 2012 Fiserv, Inc. or its affiliates.

Execution

Verification

Production

Go

Live

�Configuration�Smoke Testing�System Integration Testing�Limited Regression �Performance Testing�Readiness Assessment�Data Conversion(if applicable)

�Data Verification (if applicable)�Implementation�Smoke Testing�User Acceptance Testing (UAT)�Mock Testing�Performance Testing�Readiness Assessment

Planning and Requirement Analysis

StrategiesProduct Knowledge

Test Design

Test DataProduct Configuration

Test Execution

Test Manageme

nt

Solution Testing

Planning and Strategy

• Process and methodology for Integration Testing

• Studying requirement from Solution Definition Document

• Identifying Integration Testing scope

• High Level Integration Testing Schedule

Strategies

• High Level Integration Testing Strategy and Plan

• Entry/Exit Criteria for Integration Testing

• High Level Environment Strategy including touch point’s environment details

• Defect Management Strategy

• Approach for sharing Integration defects with Product Teams

• High Level Regression

Product Knowledge

• Implementation Specific Knowledge from Product Teams to Testers

Test Design

• Creation of High Level Test Scenarios

• Creation of Integration test cases

• Including Batch transmission and Real Time Testing (Both) under Integration Test Cases

• Validation of Integration Test Cases with Product Teams

© 2012 Fiserv, Inc. or its affiliates.

• High Level Regression Testing Strategy

Test Data

• Creating Test Data Requirement in Sync with Other Product Teams

• What Data for Testing (Production/ Dummy/ Client)?

• Data Tweaking?

Product Configuration

• Implementation Specific Configuration Knowledge from Product Teams to Testers

Test Execution

• Smoke Testing • Integration Testing

• Defect Sharing with Product Teams

• Joint Defect Triage Meetings with Product Teams

• Prioritization of Integration Testing defects

• Defect Retesting• Checking of Exit Criteria for Integration Testing (before UAT)

• Regression Testing with Final Build before GO-LIVE

Test Management

• Test Management throughout the Solution Testing in Execution Phase

Engagement Delivery Team Structure

Delivery Liaison

Enterprise Business Analyst

Solution Architect

Engagement Training Manager

Engagement Deployment

Manager

Engagement Testing

Manager

© 2012 Fiserv, Inc. or its affiliates.

Engagement Director

ArchitectManager

Solution Definition

Focus

Engagement Implementation

Focus

Solution Delivery Team Structure

Solution Testing Tester

Solution Testing Business Analyst

Solution Testing Lead

Solution Testing Manager

© 2012 Fiserv, Inc. or its affiliates.

Engagement Testing Manager

Solution Testing Focus

Metrics

S. No. Metrics Name

1 Planned Vs Actual - Test Case Execution

2 Test Case Execution - Distribution by Status

3 Test Case Distribution by Integration Point

4 Weighted Defect Distribution by Product

5 Weighted Defect Distribution by Integration Points

© 2012 Fiserv, Inc. or its affiliates.

5 Weighted Defect Distribution by Integration Points

6 Defect Distribution by Severity/Priority*

7 Defects by State

8 Defect Aging

9 Pre-Delivery Defect Distribution by Severity

10 Post Delivery Defect Distribution by Severity

11 Defect Removal Efficiency

Level of Prescription

Unifyingframeworks

Alignment in design (What)

� Language� Definitions� Roles

Unity

QualityPerformance, Protection

Unifyingframeworks

Alignment in design (What)

� High-level processes� Deliverables� Performance assessment

� Language� Definitions� Roles

Clarity in execution (How)� Procedures / SOP� Templates

� Tools� Metrics

Unity

Uniformity

© 2012 Fiserv, Inc. or its affiliates.

QualityPerformance, Protection

Alignment in design (What)

� High-level processes� Deliverables� Performance assessment

Clarity in execution (How)

� Procedures / SOP� Templates� Tools� Metrics

Uniformity

Other Considerations

• End-to-end Testing

• Dedicated Environment for Solution Testing

• Performance Testing

• Competency building for future engagements

© 2012 Fiserv, Inc. or its affiliates.

Benefits of the Approach

Solution Testing helps deliver a defect free Products or Solutions

and attain Client Satisfaction

Setting up a standard process for testing enterprise level integrated

product helps execute the Solution Testing effectively and efficiently

A centralized team executing the Solution Testing can help organization

© 2012 Fiserv, Inc. or its affiliates.

A centralized team executing the Solution Testing can help organizationto avoid repeating mistakes and in saving cost of testing bysharing resources over engagements

A satisfied Client will help promote the products to other institutions

References

• Presentations from the Solution Delivery Process Team at Fiserv.

• Other resources and images on internet.

© 2012 Fiserv, Inc. or its affiliates.

Thank you

Rashmi MahajanTesting Manager – Testing Center of Excellence

Fiserv India

Saroj PatnaikGeneral Manager – Testing Center of ExcellenceFiserv India

© 2012 Fiserv, Inc. or its affiliates.