Testing Framework

Controlled copy

Release ID: QFTS-FW.doc / 1.01 / 05.03.2004 C3: Protected

Framework

for

Testing

Controlled copy Testing Framework

Release ID: QFTS-FW.doc / 1.01 / 05.03.2004 C3: Protected of 22

Table of Contents

1.0 Introduction 4

2.0 Definitions and Acronyms 5

2.1 Acronyms 5

2.2 Definitions 5

3.0 Purpose 6

What is not the intended use for the Framework? 6What then defines the process for the projects? 6How should the Framework be used? 6

4.0 Organization of the Framework 7

4.1 Section A – Process Disciplines 7

4.2 Section B – Standard PSPH 7

5.0 Section A - Process Disciplines 9

5.1 Test Requirements Study 9

5.1.1 Test Requirements Study (AWR) 9Scoping 9Requirements Gathering 9Identification of Estimation Methodology 9Estimation 9Business Process Understanding 10Application Understanding 10Tools Understanding 10Customer Testing Process Understanding 11Infrastructure requirements Understanding 11Customer Performance Baseline Collection 11Traceability matrix for Test Requirements 11Knowledge Transition to Offshore 12

5.2 Test Strategy 12

5.2.1 Test Strategy (AWR) 12Tool Evaluation and Selection (in case of automated testing) 12Testing Tools Installation (in case of automated testing) 12Proof Of Concept (in case of automated testing) 12Test Strategy formulation 13

5.2.2 Test Planning (AWR) 13Test Planning 13Receive requests from Customer 14Revisit Test Strategy 14

5.3 Test Design 15

5.3.1 Test Environment Setup (AWR) 15Test Environment Setup 15

5.3.2 Test Design (AWR) 15Test Design 15Test Data Preparation / Customization / Sampling / Manipulation 16



5.4 Test Execution 17

5.4.1 Test Execution (AWR) 17Pilot Execution of Test cases 17Test Execution 17

5.5 Delivery 18

5.5.1 Delivery (AWR) 18Delivery 18Verification by Onsite team 18

5.6 Acceptance Testing 19

5.6.1 Acceptance Testing (AWR) 19Acceptance Test Plan 19Acceptance Test Procedure 19Assist in UAT Data Preparation 19Acceptance Testing 19Acceptance Sign – Off 20

6.0 Section B – Process Handbook (ETVX) 21

7.0 Section C – Measurements Applicable 22



1.0 Introduction

This handbook describes the Cognizant Testing framework and applies to Projects executed by Cognizant delivery team as part of

1. Standalone projects taken up for any new/existing customer.

2. ODC engagement with existing customers,

This framework is organized as follows:

SECTION A – TESTING WORKFLOW

SECTION B – STAGES OF TESTING

ANNEXURES

Section A – Testing Workflow

The testing workflow refers to the logical group of tasks performed during the testing process. Logical grouping aims at grouping tasks that fulfill a common objective like System testing, Delivery etc.

A similar workflow exists for the Project Management process, which happens in parallel / overlaps with the testing workflow. (Refer to the Delivery Management Framework for more details)

Section B –Testing Process (ETVX Model/ OSSP)

Tasks in Testing workflow and Project Management workflow are fitted into the different stages of Testing operational model



2.0 Definitions and Acronyms

2.1 Acronyms

NFR Non functional requirement

RFP Request For Proposal

SOW Statement of work

AWR Activity, Work Product, Responsibility

ODC Offshore Development Center

POC Proof of Concept

2.2 Definitions

ROLE The roles depicted here identify the skill background of the people doing the tasks. A Project team member could be playing more than one role. It is suggested that projects do a mapping of the roles to the team members accordingly



3.0 Purpose

The primary purpose of this framework is to describe the testing related practices that are applicable to projects executed by Cognizant.

Where special guidelines are available, they have been referenced in the framework. It is recommended that such guidelines be read and understood entirely to have a complete understanding of the Testing Framework.

The practices that are mandatory for all projects and the practices that are optional for the projects are listed in separate boxes throughout the framework. An explanation of the legends is provided below:

What is not the intended use for the Framework?

The Framework cannot be considered to be the PSPH for the project. Understanding this is important, as the framework is not organized by Stages. It is rather organized by process disciplines. The Section B of the Framework contains references to PSPH, which defines the process to be adopted by the projects.

What then defines the process for the projects?

The Section B of the Framework contains references to PSPH, which defines the process to be adopted by the projects.

How should the Framework be used?

1. The Framework provides the basis on which all Operational life-cycle models are defined.

2. Additional operational life-cycle models needs to be arrived at using the Framework as the basis.

3. The Framework splits each of the tasks in the PSPH into further activities thus explaining how a task needs to be performed. Further references are provided to Testing guidelines that describe the Inputs, Outputs, and Tools & Techniques to be performed for each of these tasks.

Mandatory / Optional Discussions

Guidelines

Notes



4.0 Organization of the Framework

This framework is organized as follows:

SECTION A – Standard Cognizant Process Disciplines

SECTION B – Standard PSPH for QSM defined Operational Life-cycle models

SECTION C – Measurements Applicable. This provides a reference to the DM Framework where Metrics Plan and Guidelines have been consolidated across Solution Offerings

4.1 Section A – Process Disciplines

This refers to the logical groups of tasks performed during the testing process. Tasks from a logical group could be split across several stages and the process discipline provides an approach to view information from a logical view.

If the project wished to understand the total effort spent on Acceptance Testing (for all the activities done as part of Acceptance Testing), it may not be possible only using stages as the plan for Acceptance Testing is done upfront in the project.

During the start of the project, the tasks are mapped to the chronological stages in the projects while the Process Discipline mapping is fixed, abstracted but available for reporting purposes.

4.2 Section B – Standard PSPH

Here the stages of Application development life cycle are depicted.

Framework

Standard Cognizant Process

Disciplines

Standard Cognizant PSPH

Tasks

Activities

Stages

Tasks



Tasks from the various Process Disciplines are fitted into the different stages depending on the life-cycle model chosen for the project

Advantage of distinguishing Process Disciplines And Stages

1. Tasks can be looked upon as components, which can be plugged and played into different Stages of the project. The work products are tied to the tasks and the entry criteria and exit criteria of the stages would vary depending on how the tasks are interweaved.

2. Organizing tasks by process discipline and stages also helps in slicing and dicing project metrics and help in comparing one versus the other.



5.0 Section A - Process Disciplines

5.1 Test Requirements Study

5.1.1 Test Requirements Study (AWR)

Scoping

Activity Work Products Responsibility

Define scope of testing (In Scope and Out of Scope)

Test Requirements Document

Account Manager / Senior Business Analysts

Identification of Estimation Methodology


Evaluate Estimation methodologies

Estimation SheetProject Manager / Project Leader

Define Estimation methodology for testing

Estimation SheetProject Manager / Project Leader

Estimation


Estimate for the total volume of work

Estimation Sheet (If Test Case Estimation: Test Case Estimation Document)

Project Leader

Requirements Gathering

Activity Work products Responsibility

Requirement gathering discussions and clarifications

Discussion notes /

CRAD

Business Analysts, Project Manager / Project Leader

Study of system Requirements Test Requirements

Document


Documentation of software requirements and sign-off




Business Process Understanding


Understand and document the Application process

Testing KT Document Project Leader

Understand and document Technical aspects of the application


Identify and define the customer performance baselines of the existing system

Customer Performance Baseline

Project Leader

Application Understanding


Understand the applications to be tested

Testing KT document Project Leader

Understand and document the interfaces to the application, if any


Understand the functionality impacts on other external interfaces


Tools Understanding


Understand if there are any tools used for testing in client’s place


Understand the test script coding guidelines of the tool, if any


Obtain Test Script Coding Guidelines and User Manual if any




Customer Testing Process Understanding


Understand and document the testing strategies used


Understand and document the testing techniques used


Understand and document the various outputs of the testing process (like Test Results, Test Cases etc.)


Understand the gaps in the existing Test Process


Infrastructure requirements Understanding


Understand and document the hardware / software environment that the application is running


Identify if there are any connectivity to be established


Customer Performance Baseline Collection


Identify if there are any Customer Performance Baselines set

Customer Performance Baseline

Project Leader

Traceability matrix for Test Requirements


List down the test requirements to trace it throughout the testing life cycle

Requirements Traceability matrix

Project Leader



Knowledge Transition to Offshore


KT of application systems, existing test process, infrastructure setup etc. to Offshore

- Project Leader

5.2 Test Strategy

5.2.1 Test Strategy (AWR)

Tool Evaluation and Selection (in case of automated testing)


Identify tools to be evaluated Test Tools Evaluation and Selection Report

Team Leads

Identify various evaluation parameters

Team Leads

Evaluate tools on the various parameters

Team members / Team Leads

Report findings Team members

Review findings Test Tools Evaluation and Selection review report

Team Leads

Finalize tool to be used Test Tools Evaluation and Selection Report

Team Leads

Testing Tools Installation (in case of automated testing)


Installation, Setup and customization of tools that are finalized for testing

Tools ready to use Project Leader / Team Leads

Proof Of Concept (in case of automated testing)


Refer Black-Box testing questionnaire and Performance testing questionnaire for evaluating various tools on Black-Box testing and Performance testing



Proof of Concept in case of automated testing

Proof Of Concept Document Team Leads

Test Strategy formulation

Activity Work Product Responsibility

Determine the types of testing to be performed (Eg: Unit / Integration / System testing, Load Testing, Link Testing, Performance Testing etc.)

Test Strategy / Approach Document

Project Leader / Team Leads

Define testing methodology (Eg: Top-Down / Bottom-Up)

Define the risks involved and assumptions made in the project

Identify testing tools, if applicable (This section can be filled once the testing tool is finalized)

Define the testing approach / methodology to be followed in case of automated testing

5.2.2 Test Planning (AWR)

Test Planning


Types of Testing to be done are identified

Test StrategyProject Leader / Team Leads

The tasks for Test Planning are identified

Test Plan Project Leader / Team Leads

Test schedule is defined Project Leader / Team

1. Refer Test Strategy Guidelines for various types of testing that could be performed and various testing techniques

2. Refer Manual Testing Guidelines and Automated Testing Guidelines for the difference in the testing process



Leads

Specific responsibilities are assigned to team members along with clear-cut deadlines


Preparatory activities are identified (Eg: acquiring special testing utility, hiring and training engineers etc.)


Test items are identifiedProject Leader / Team Leads

Additional needs, if any, are identified


Review Test Plan Test Plan review report Team Leads

Log Test Requests

Test Request Sheet and Tracking Sheet

Team members

Identify the Deadlines (Start Date and End Date) for the Test Request

Update Requirements Traceability matrix

Updated Requirements Traceability matrix

Team members

Receive requests from Customer


Receive requests from Customer

Test Request and Tracking Sheet

Team Leads / Team members

Revisit Test Strategy


Identify if Test Planning has brought in any changes to

Updated Test Strategy Project Leader

Requirements could come in two ways

1. Upfront Defined requirements

2. Adhoc requests which needs to be tracked



the Test Strategy

5.3 Test Design

5.3.1 Test Environment Setup (AWR)

Test Environment Setup


Manual Testing

Setup System / Integration / Unit Test environment as applicable

Updated Test Strategy document

Team members

Setup System / Integration databases

Team Leads / Test DBA

Run scripts and populate the databases

Populated System / Integration database

Team Leads

Review System / Integration / Unit test environment

Test Environment review report

Team Leads

Automated Testing

Setting up the automated testing environment

Automated test bed Team Leads / Team members

5.3.2 Test Design (AWR)

Test Design


Manual Testing

Identify the type of testing Test Procedure with test cases

Team Leads/ Team members

Identify the scope of testing Team Leads/ Team members

Identify the pre-requisites for testing



Identify the test scenarios for generating the test cases

Generate test cases based on the type and scope of testing

Define execution and verification procedures

Finalize environment data requirements

Build test suites, test drivers as needed for testing

Test Suites / Test Drivers

Team members

Review Test DesignTest Procedure review report

Team Leads

Update Requirements Traceability Matrix to include test specifications

Updated Requirements Traceability Matrix

Team members

Automated Testing

Prepare Test Scripts Test Scripts / Test Procedure

Team Leads / Team members

Review Test Scripts Reviewed Test Scripts Team Leads / Team members

Test Data Preparation / Customization / Sampling / Manipulation


Manual Testing

Prepare necessary test data for System / Integration / Unit Testing

Test environment ready with Test Data

Team members

Validate test dataTest environment ready with validated Test Data

Team Leads



5.4 Test Execution

5.4.1 Test Execution (AWR)

Pilot Execution of Test cases


Manual Testing

Perform Pilot RunTest Logs Team members

Log Results

Analyze Results Pilot Run Results Report Project Leader / Client

Define Corrective action, if necessary

- Project Leader

Revisit Test Strategy, if necessary

Updated Test Strategy document

Project Leader

Test Execution


Identify the type of testing to be done -

Identify number of test runs from the test plan. -Team members / Team Leads

In every test run, Run test cases as per the test procedure and records results.

Test logs /As run logs (in case of Automated Testing)

If a test run fails altogether, because of an abortive functionality, then record the failure of the run as a whole (E.g. RUN N: ABORT M)

Record of disruption of Cycle completion in the test logs /As run logs (in case of Automated Testing)

Team members

Once the abortive failure is corrected, restart the Test Run and repeat for all the planned Test Runs

Test logs /As run logs (in case of Automated Testing)

Team members

Update Requirements Traceability Matrix to include test results

Updated Requirements Traceability Matrix

Team members



5.5 Delivery

5.5.1 Delivery (AWR)

Delivery

Activity Work products ResponsibilityZip / Pack the testing documents (typically Test Plans, Test Procedures / Test Cases and Test Logs) for the tested module / requirements

Zipped file containing Test Plans / Test Procedures / Test Logs for the tested module


Prepare summary report for testing Summary ReportProject Leader / Team Leads

Verification by Onsite team


Deliverables verification by Onsite team

Verification mail from Onsite co-ordinator

Onsite Co-ordinator



5.6 Acceptance Testing

5.6.1 Acceptance Testing (AWR)

Acceptance Test Plan


Plan for Acceptance Testing Acceptance Test Plan Customer and Cognizant

Acceptance Test Procedure


Determine Acceptance CriteriaTest Requirements Document

Customer and Cognizant

Identify Acceptance test scenarios Acceptance Test

Procedure


Define Acceptance Test Cases Customer and Cognizant

Assist in UAT Data Preparation


Prepare Acceptance Test DataAcceptance testing environment ready with test data


Acceptance Testing


Review and communicate any changes to the acceptance test plan

Updated Acceptance test plan

Client / Cognizant (S)

Get the acceptance test cases and review the same

Acceptance test cases Cognizant Delivery Manager



Support during acceptance testing

Issue Resolution, Test logs from the client

Onsite coordinator / Delivery manager

Acceptance Sign – Off


Get Acceptance sign-off from customer

Formal mail from Client Customer



6.0 Section B – Process Handbook (ETVX)

Testing Project Software Process HandBook



7.0 Section C – Measurements Applicable

Metrics Plan for Testing

http://cognizantonline/qview/Docs/DeliveryMgmt/QTDM-MBTE.xls

Testing Framework

Documents

Transcript of Testing Framework