GuidewireClaimCenter Performance TestPlan

Guidewire Claim Center

Guidewire ClaimCenter Performance Test Plan

Prepared By : L&T Infotech ltdDate : Aug 02, 2013Version : 1.0

Performance Testing Strategy 1 HSB


Revision History

Date Version Description Author Aug 02, 2013 1.0 Initial Version Venkateswarara

o Dodda



GW Claim Center Performance TestingAPPROVAL SIGN OFF

Name(s) Date

Business Analyst

IS Lead

Project Manager

Integration Development Lead

Database Lead

QA Lead

Performance Engineer

If not approving, note your issues and explain your position below:


The below referenced deliverable has been reviewed and accepted by:


Index

GW Claim Center Performance Testing................................................................3APPROVAL SIGN OFF....................................................................................................3Index..................................................................................................................................41. Introduction................................................................................................................5

1.1 Description...........................................................................................................51.2 Purpose..................................................................................................................51.3 Scope......................................................................................................................5

2. Performance Testing Approach..........................................................................63. Workload Distribution...........................................................................................10

3.1 Workload Distribution for Guidewire ClaimCenter..............................103.2 Details of Application under Test..............................................................103.2.1 Guidewire ClaimCenter..............................................................................10

4. Module Details.........................................................................................................106. Schedule Tests........................................................................................................117. Deliverables.............................................................................................................148. Project Closeout......................................................................................................159. RISKS...........................................................................................................................1510. Assumptions..........................................................................................................1611. Milestones..............................................................................................................1712. Stakeholders..........................................................................................................1713. System Architecture......................................................................................17

13.1 Test Environment..........................................................................................1713.2 System Architecture (Production)..........................................................18



1. Introduction

1.1 Description

The goal of this project is to implement Guidewire ClaimCenter System, to integrate it with other existing HSB IT Systems, so as to seamlessly migrate users of existing claim system onto the new system and provide continuity of business processes.

Performance Testing is considered to carry out a sanity performance testing to understand that the application is performing as required under defined user load.

1.2 Purpose

Performance tests will be conducted to validate the performance of Guidewire ClaimCenter application and verify if the performance requirements are met.

1.3 ScopeIn ScopeGuidewire ClaimCenter application will be tested in Production and Stage environment and will be limited to the following in scope:

Application study

Requirements Gathering & Analysis (for performance tests to be carried out)

Preparation of Performance test design with all scenarios & workload model

Aid in Test Data Preparation (For accelerating the process)

Stress & Load Test Execution

High-level analysis of results obtained after the test execution

Test Scripting – considering 10 test script for the current scope (10 for each Production & Stage Environment)

o Each Test Script will have minimum of 15 steps (step mean mouse or keyboard event) and maximum of 50 steps (step mean mouse or keyboard event)

o Each Test Script will have minimum of 1 page & maximum of 13 pageso Each Test Script will have maximum of 25 transactions & 13

parameterso Each Test Script will have maximum of 7 correlationso Each Test Script will have maximum of 3 unique test-data parameters

Out of Scope



Following are the requirements that would be out of scope for L&T Infotech team:

Hardware capacity management Network capacity management Hardware maintenance Network maintenance Baseline server/system parameters Any other activity which is not mentioned in the scope of work Performance Testing of web services or any other third party

interfaces File/Image comparison Performance Testing of any other application (dependent,

independent, interfaced, or otherwise) other than the one mentioned in the scope

Performance testing of pages involving :- RIA (Rich internet application) objects Java/Flash objects

2. Performance Testing ApproachL&T Infotech will conduct sanity & performance testing of Guidewire application having various integrations. L&T Infotech will initially do application study to finalize the scope. Depending on the need, if required, L&T Infotech will change the scope (add/remove) and/or ramp up resources by mutual discussion with HSB.

We assume that there will be load of 500 concurrent user for Stress Test and 250 users for Load Test. Post application understanding & discussion with HSB, we would rework the concurrent user load & hence the necessary part of approach if required.

Activity Sub-Activities Primary Owner

Secondary Owner

Training & Orientation of Resources on HSB Applications, Account and Project Overview HSB L&T

InfotechFloor Access, ID Creation, etc HSB L&T

InfotechPerformance

Test Readiness

Performance Test Objective Finalization HSB L&T Infotech

Dedicated Performance Test Environment Setup HSB L&T Infotech

Test & Finalize Performance Test Environment - with all required Integrations in place HSB L&T

InfotechCreation of 1500 user ids to access the environment thro virtual users HSB L&T

InfotechIdentifying teams that would monitor the performance of the servers during execution HSB L&T

InfotechFinalize counters & acceptable threshold values and SLAs for response time HSB L&T

InfotechIdentify & Finalize Test Cases to be considered for performance testing - Max 10 in number - for both

HSB L&T Infotech



Stage & Perf Environment

Initial Setup & Access

HP License agreement & processing L&T Infotech HSB

Hand Over of Load Generator & Controller Machines HSB L&T Infotech

Installation of Load Runner tool L&T Infotech HSB

Installation of licenses L&T Infotech HSB

Run a dummy script to check the connectivity between controller & LoadRunner

L&T Infotech HSB

Access to Environment - for scripting purpose HSB L&T Infotech

Protocol finalization L&T Infotech HSB

Preparation & Script Preparation

Test Step Documentation - Max 10 L&T Infotech HSB

Test Script preparation - Max 10 L&T Infotech HSB

Validate Test Scripts L&T Infotech HSB

Test Data preparation, if required HSB L&T Infotech

Smoke Test Execution Preparation L&T Infotech HSB

Smoke Test & address Modifications in scripts or data or application, if required

L&T Infotech HSB

Performance Test Execution preparation L&T Infotech HSB

Execution

Stress Test Execution # 1 (To find out breaking point of the system - Considering max 500 VUsers)

L&T Infotech HSB

Stress Test Execution # 2 (For averaging purpose) L&T Infotech HSB

Re-Stress Test Execution # 1 (To find out breaking point of the system - Considering max 500 Vusers - post fixes)

L&T Infotech HSB

Re-Stress Test Execution # 2 (For averaging purpose - post fixes)

L&T Infotech HSB

Load Test Execution # 1 (To find performance of the application at user load of 230)

L&T Infotech HSB

Load Test Execution # 2 (For Averaging Purpose) L&T Infotech HSB

Fixing issues in Environment HSB L&T Infotech

Load Re-Test Execution # 1 (To find performance of the application at user load of 230 - post fixes)

L&T Infotech HSB

Load Re-Test Execution # 2 (For Averaging Purpose - post fixes)

L&T Infotech HSB

Fixing issues in Environment HSB L&T Infotech

Load Re-Test Execution # 1 (To find performance of the application at user load of 230 - post more fixes)

L&T Infotech HSB

Load Re-Test Execution # 2 (For Averaging Purpose - post more fixes)

L&T Infotech HSB



The following table summarizes the overall interaction with multiple stakeholders in order to execute this engagement smoothly and effectively.

R – Responsible A - Accountable C – Consulted I – Informed NA – Not Applicable

Sr. No

Activity Sub Activity

L&T Infotech

Perf. Test Team

Development Team

L&T Infotech Program Manager

HSB Project Owner

DBA HSB Network/Infra

Team-HSB

Architects HSB

1 Onboarding Responsible Consulted

Responsible Accountable NA NA NA

2 Application Study

Detailed Requirement Gathering

Responsible

Responsible

Responsible Accountable Consulted NA Consulted

3 Test Tool Set up Responsible NA

Accountable Responsible NA

Responsible NA

4Finalization of Test Cases

Responsible

Responsible Informed Accountable NA NA Consulted

5Test Step Documentation

Responsible

Responsible

Accountable Informed NA NA Consulted

6Test Scripting & Standardization

Responsible NA

Accountable Informed NA NA Consulted

7 Test Data Preparation Responsible Informed

Accountable Informed Consulted NA Consulted

8 Test Execution Setup Responsible Informed

Accountable Informed Responsible Consulted Informed

9 Dry run Responsible Informed

Accountable Informed Responsible Consulted Consulted

10

Test Execution

Run1Responsible Informed


Analysis Report 1

Responsible

Responsible

Accountable Informed Responsible

Responsible

Responsible

Run2Responsible Informed


Analysis Report 1

Responsible

Responsible

Accountable Informed Responsible

Responsible

Responsible

11 Project Closure Consulted InformedResponsible Accountable Consulted Informed Informed



The following table summarizes roles & Responsibilities of each stakeholder:

Role Responsibilities

Engagement Manager (L&T Infotech)

Serves as an escalation point for Project Delivery Tracks Project delivery & status Instrumental in ramp up/ramp down of resources, as required SPOC from L&T Infotech for additional project requirements and for

commercial queries

Delivery Manager(L&T Infotech)

Participate in governance meeting Overall Delivery Management of projects Ensuring smooth operations Resource Management & Query/Issue Management

Performance Test Lead(L&T Infotech)

Understand & finalize the performance requirements with help of SPOC

Prepare execution approach for Performance Testing Coordinate with Functional Team for Test Case Finalization Coordinate with DBA Team for monitoring counters, Report Analysis,

Database Backup/Restore Setup of Load Generators & Monitoring tools with help of

Infrastructure Team Prepare Test Scripts & Execution Plans and Perform Test Execution

& Reporting along with Performance Tester(s)

Performance Tester(L&T Infotech)

Document Test Steps for all the Test Cases identified with help of Functional Team

Prepare Test Scripts for Performance Testing and Test Execution Plan

Schedule Performance Test Execution, Perform Test Execution & prepare report

Perform re-executions as per requirement & prepare final performance report, if planned

Setup monitoring for the servers

DBA Team/Development Team(HSB)

Identify major pain areas in the application - Performance Contribute towards analyzing the Performance Test Results Provide recommendations on the application issues identified during

Performance Testing Facilitate Database Backup/Restore during the Performance Test

Execution cycles Monitor the logs of the application during Performance Test

Execution Tune the application for potential performance issues identified

Business / IT Team(HSB)

Finalize requirements, Test Cases to be considered, approach to be followed

Provide final signoffs on the deliverables Facilitate the project execution

Networking/Infrastructure Team(HSB)

Help Setup the 'Test Environment‘, 'Load Generators' & 'Development Machines'

Contribute towards analyzing the Performance Test Results Provide recommendations on the infrastructure/network issues

identified during Performance Testing Facilitate Ramp-up/down of Infrastructure - for Performance Test

Purpose



Functional Team(HSB)

Prepare Test Cases flows Help Testing Team with the navigation of Test Cases Prepare Test Data for Test Script Preparation & Execution



3. Workload DistributionA Workload Distribution is a representation of the functions performed by a user community on a system. A Workload Distribution is based on a percentage of users performing a specific function over a given period of time.

3.1 Workload Distribution for Guidewire ClaimCenter

The sections below describe, in detail, the workload distributions to be used for testing the Guidewire ClaimCenter. The Workload Distribution includes all of the functions to be executed during the performance testing effort.

Order/faults/DS Forecast volumes/hr

Test Volumes/hr TPS

Guidewire ClaimCenter 150 claims/hr 75 Claims/hr TBD

3.2 Details of Application under Test

3.2.1 Guidewire ClaimCenter

4.

Module Details

4.1 Guidewire ClaimCenter


Name (h/w, s/w, other) Type CPU RAM (GB)

cvgs080092 App Server (Prod) TBD TBD

cvgs080093 CM Server (Prod)

TBD TBD

cvgs080094Batch Server (Prod)

TBD TBD

cvgs080018 DB Server (Prod)

TBD TBD

cvgs080019 DB Server (Prod)

TBD TBD


6. Schedule Tests Schedule tests would include those activities that are mandatory to validate the performance of the system. These tests need to be executed even if no performance issues are detected and no tuning is required. There are only two activities that can be conducted in this aspect. They are

• Stress Test• Load Test

Following are the various scenarios that can be considered for performance testing:-

Scenario 1 – Guidewire ClaimCreation Low Load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies.

Test Scenario Number Scenario 1 – Low Load – 20 usersTest Scenario Name Claim Creation - Low LoadScript Name Claims/Executions per

1 HourTC02_IntakeQueueNewClaimForCRTAssumedPolicy 40TC03_IntakeQueueNewClaimForDirectPolicy 30TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 30Total 100

Scenario 2 – Guidewire ClaimCreation Medium Load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies.

Test Scenario Number Scenario 2– Medium Load – 30 usersTest Scenario Name Claim Creation - Medium LoadScript Name Claims/Executions per

1 HourTC02_IntakeQueueNewClaimForCRTAssumedPolicy 50TC03_IntakeQueueNewClaimForDirectPolicy 50TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 50Total 150

Scenario 3 – Guidewire ClaimCreation High Load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies.

Test Scenario Number Scenario 3 – High Load – 40 usersTest Scenario Name Claim Creation - High LoadScript Name Claims/Executions per

1 HourTC02_IntakeQueueNewClaimForCRTAssumedPolicy 80TC03_IntakeQueueNewClaimForDirectPolicy 60TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 60



Total 200

Scenario 4 – Guidewire ClaimCenter All Scripts – Low load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search, attaching documents to existing claims and search for existing claims.

Test Scenario Number Scenario 4– All Functions – 100 usersTest Scenario Name Claim Center - All Functions – Low LoadScript Name Claims/Executions per

1 HourTC01_PolicySearch 50TC02_IntakeQueueNewClaimForCRTAssumedPolicy 100TC03_IntakeQueueNewClaimForDirectPolicy 100TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 100TC05_AttachDocumentsForExistingClaim 100TC06_SearchExistingClaims 50Total 500

Scenario 5– Guidewire ClaimCenter All Scripts – Medium Load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search, attaching documents to existing claims and search for existing claims.

Test Scenario Number Scenario 5– All Functions – 150 usersTest Scenario Name Claim Center - All Functions – Medium LoadScript Name Claims/Executions per


Scenario 6– Guidewire ClaimCenter All Scripts – High Load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search, attaching documents to existing claims and search for existing claims.

Test Scenario Number Scenario 6– All Functions – 200 usersTest Scenario Name Claim Center - All Functions – High LoadScript Name Claims/Executions per

1 HourTC01_PolicySearch 125TC02_IntakeQueueNewClaimForCRTAssumedPolicy 200TC03_IntakeQueueNewClaimForDirectPolicy 200TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 200TC05_AttachDocumentsForExistingClaim 150



TC06_SearchExistingClaims 125Total 1000

Scenario 7– Guidewire ClaimCenter – Peak Load: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search, attaching documents to existing claims and search for existing claims.

Test Scenario Number Scenario 7– All Functions – 300 usersTest Scenario Name Claim Center - All Functions – Peak LoadScript Name Claims/Executions per


Scenario 8– Guidewire ClaimCenter – Stress Test: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search, attaching documents to existing claims and search for existing claims.

Test Scenario Number Scenario 8– All Functions – 500 usersTest Scenario Name Claim Center - All Functions – Stress TestScript Name Claims/Executions per


Scenario 9– Guidewire ClaimCenter – Soak Test: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search and search for existing claims. This test will be executed with minimum load for 4 hrs.

Test Scenario Number Scenario 9– All Functions – 100 usersTest Scenario Name Claim Center - All Functions – Soak TestScript Name Claims/Executions per

1 HourTC01_PolicySearch 100TC02_IntakeQueueNewClaimForCRTAssumedPolicy 100TC03_IntakeQueueNewClaimForDirectPolicy 100TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 100



TC06_SearchExistingClaims 100Total 500

Scenario 10– Guidewire ClaimCenter – Longevity Test: This scenario comprises of test script pertaining to CRT and Phoenix policies in which Claim Center will be able to create claims for both Assumed and Direct policies along with policy search and search for existing claims. This test will be executed with minimum load for 12 hrs.

Test Scenario Number Scenario 10– All Functions – 50 usersTest Scenario Name Claim Center - All Functions – Longevity TestScript Name Claims/Executions per

1 HourTC01_PolicySearch 50TC02_IntakeQueueNewClaimForCRTAssumedPolicy 50TC03_IntakeQueueNewClaimForDirectPolicy 50TC04_IntakeQueueNewClaimForPhoenixAssumedPolicy 50TC06_SearchExistingClaims 50Total 250

7. Deliverables a. From L&T Infotech

L&T Infotech would deliver the following during the course of the project:

Performance Test plan and strategy document

Validated Test Scripts

Test scenarios

Execution Summary Report containing following information:

o Average Transaction Response Timeo Throughput detailso CPU & RAM Utilizationo Bottleneck detailso Closure Report

b. From HSB

L&T Infotech expects following deliverables from HSB during the course of the project:

Finalized list of test cases to be considered for performance testing



8. Project Closeout

The Project Closeout aspect only has one activity, but that activity is important and independent enough to deserve an aspect of it‘s own. That activity is:

Document Results

Results document would include: Spreadsheets of recorded times for each user load Spreadsheet of average times grouped by user load Test execution reports for each test executed

9. RISKSL&T Infotech has a well-defined risk management process to identify the potential risks in advance and focus on preventing them in order to ensure proper execution and completion of project.

A Risk Management process is followed, wherein all the risks for a project are identified and maintained in a Risk Monitoring Log (RML). For each of the risk the probability of occurrence, impact and action plan for managing or mitigating the risk is identified. The risks listed in the RML are assessed and monitored during the weekly project status review meetings. Any new risk that is identified during the execution of the project is added to the RML and monitored closely.

L&T Infotech has identified the following risk factors along with the possible mitigation plan. L&T Infotech together with HSB will chalk out the risk mitigation plan at the start of the project. These risks would be revisited/ monitored periodically to ensure smooth project delivery.

S. No

Risk Impact Mitigation/ Contingencies

1.

Clarity and stability of requirements (changes in requirements, scope creep, etc during project execution)

HighAny changes to the scope of the project would be addressed through the standard Change Control procedures followed by L&T Infotech

2. Timely approval/ sign-off for key documents High

Careful planning, effective communication and efficient escalation process would mitigate this risk.

3. Non availability of required test data

Very High

SPOC to facilitate Test Data generation & test execution plan need to be reworked upon

4. Delay in resolution of queries and clarifications regarding the project

High

Queries/issues to be resolved at the earliest. An escalation process should also be in place to escalate unresolved issues to higher level in Project Management chain

5. Test environment instability due to activities like

High SPOC to ensure that only a functionally stable build is used for performance testing



upgrades or code fixes activity – any instability would require rework on test execution plan

6. Test environment is not same as production environment

Very High

SPOC to ensure that the performance test environment should be replica of production or mathematically scaled down version of production environment

7. Non-dedicated Environment for performance testing Medium

To have a dedicated performance testing environment where the application could be accessed only for performance testing activities

8.

Unavailability of DBA Team, Development Team, or Infrastructure/Networking Team for performance isolation & fix

Very High

Ensure the support/availability of DBA Team, Development Team or Infrastructure/Networking Team

10. Assumptions Requirements are base lined or approved prior to testing. Environment will be dedicated to performance testing during test execution

and has been properly configured to support performance testing. Support (application development, database, deployment, and System

Administrator teams) will be available during test runs to troubleshoot potential problems that may arise.

Performance fixes are delivered according to defect management guidelines or agreed upon timeline

HP LoadRunner is recommended for Performance Testing Any issue related to the application/network would have to be solved by

respective application / network owners. L&T Infotech would only be responsible for LoadRunner related issues during the tenure of the engagement

The resource loading is based on our current knowledge of the scope of work and understanding of requirements. Resource ramp up or change in scope (addition/removal) will be done after mutual agreement between L&T Infotech and HSB

Performance Testing Lead & Performance Testers would travel to Hartford, CT office of HSB & work from onshore location for the duration of the project

Any delay in availability of HSB resources, sign offs, access to environments, on boarding activities would potentially affect the project timelines and cost

Deliverables to be signed off in 2 working days Stress Test would be executed with maximum of 500 concurrent users with

failing criteria as more than 10% errors observed at client side Load Test would be executed with concurrent user load of 250 with failing

criteria as more than 5 seconds of response time of a transaction (Transaction is a submit event) & CPU/RAM utilization of less than 50%

Any delay in availability of the machines at specified locations would necessitate a revision of the project schedule Cincinanti, Ohio Hartfort, CT



11. Milestones

S. No. Activities Start Date End Date Owned By

1. Completion of Performance Test Plan

07/29/2013 08/09/2013 Performance Test Team

3. Conduct Shake out Test 07/29/2013 08/16/2013 Performance Test Team

4. Completion of Performance Test Data

07/29/2013 08/16/2013 TBD

5. Completion of Performance Test Scripts and Scenarios


6. Conduct Performance Tests on Production


12. Stakeholders

Name RoleKathleen Kelting Product OwnerRandy Wallace ISD LeadAlok Tewari Project ManagerJerold Adair Business AnalystDonna Senchesak Business AnalystKaren James Business AnalystCarolyn Clyde Lofton Business AnalystWalter Swiatlowski Development LeadRanjana Choudhary Test LeadVenkateswara Rao Dodda Performance Test LeadManav Garg Integration Development LeadWebb Douglas Database LeadRonald Foz Configuration Development Lead

13.System Architecture 13.1 Test Environment

The test environment consists of: Web Server - cvgs080055App Server - cvgs080055Batch Server - cvgs080091DB server - cvgs070004

13.2 System Architecture



(Production) The Production environment is a shared environment consisting of:Load Balancer(s) - TBDWeb Server(s) - TBDApp Server(s) - cvgs080092, cvgs080093Batch Server - cvgs080094Database - cvgs080018, cvgs080019


GuidewireClaimCenter Performance TestPlan

Documents

Transcript of GuidewireClaimCenter Performance TestPlan