Performance Test Strategy for Applications Not Supported by Performance Tools

Yakub Reddy Gurijala Sr. Technical Architect

Sujatha Badri Narayanan Group Project Manager

Infosys Limited

1

In today's world, applications, systems not meeting the performance expectations of the customers are branded as failures even if they are rich in functionality. Efforts spent in developing such applications go nowhere if their performance is poor. Hence more and more organizations are adapting to Performance Testing and even Early Performance Validation these days. There are various performance testing tools available in the market to support this cause.

Being in a service oriented company, we evaluate performance of various client applications that are developed using different software and hardware environments and communicate with users/servers using different protocols. Moreover, the client application’s environments and protocols are dynamic in nature and change frequently and newer versions get released whenever there is a change. We generally use commercially available performance testing tools or freeware tools that are available in the market to test these applications. However, we have seen instances where none of these performance test tools can be used to create the required scripts.

For example, we cannot record desktop applications because of custom communication or proxy issues, though the protocol may be supported by various performance testing tools.

In these situations, to ensure successful evaluation, we either contact the performance tool vendors or development team. Tool vendors generally take long time to provide a solution. But the performance testing team will not have the luxury of waiting until the tool vendor get back with support/solution.

Hence PT Team need to come up with alternative methods to continue the testing of those applications which are currently not supported by the tools. This paper is an attempt to address such issues faced by performance test team. Strategies proposed are given in the following sections.

2

Performance Test Strategies Proposed

As the first step, analyze the application behavior and how the communication is happening between client and servers, protocols used to communicate, different components in the application and technologies used to build the application. If none of the available tools in the market can be used to create scripts for your application under test, then go for any of the following approaches.

1.1 Approach1 - Creating the scripts from Logs

In this approach we have explained the process to create test scripts from client logs.

Collect the client side log files and understand them. Sample log file Snapshot is given below:

Identify all requests generated by client application. For example, in the above log, request is xml data and pattern is “Data out”

Identification of all requests can also be automated using a script

3

Once all the requests are identified create an automation script to generate the performance test script. Snapshot of automation script that can be used to generate Jmeter script:

Jmeter Script Created

4

Create the custom code for specific communications. (If required) Integrate the custom code with performance test script. Convert single user script to multiuser by applying the

parametrization and correlation. Automate above procedure using any of the scripting language to

create the script from log files.

1.1.1 Scenarios Tested Desktop applications TCP Socket based applications FTP Based applications

1.1.2 Benefits We can mimic the real user behavior Test the applications with expected number of users without any

extra infrastructure. Can be extended to any application that will have client side logs. Supports custom communication. Applicable to all performance tools and protocols.

1.1.3 Limitations Availability of client side logs with all requests. Need to wait till client side application is developed.

5

1.2 Approach 2 - TCP/IP level recording

Though applications are built on different hardware’s, technologies and protocols finally they use TCP as the underlying protocol.

We can record the scripts for the desired scenarios using TCP/IP protocol by capturing the communication between client and server instead of actual protocol. Once scripts are created enhancements such as Parametrization, Correlation can be done on these scripts.

1.2.1 Scenarios Tested Win sockets of Load Runner TCP/IP Recording of Silk Performer Network Traffic Simulation Tools

1.2.2 Benefits We can mimic the real user behavior

Physical LayerEIA/TIA-232 ·EIA/TIA-449 ·ITU-T V-Series ·I.430 ·I.431 ·POTS ·PDH ·SONET/SDH ·PON ·OTN ·DSL ·IEEE 802.3 ·IEEE 802.11 ·IEEE 802.15 ·IEEE 802.16 ·More

DataLink LayerATM ·SDLC ·HDLC ·ARP ·CSLIP ·SLIP ·GFP ·PLIP ·IEEE 802.2 ·LLC ·L2TP ·IEEE 802.3 ·Frame Relay ·ITU-T G.hn DLL ·PPP ·X.25 ·Network switch

Network LayerIP (IPv4, IPv6) ·ICMP ·IPsec ·IGMP ·IPX ·AppleTalk

Transport LayerTCP ·UDP ·SCTP ·DCCP ·SPX

Application LayerNNTP ·SIP ·SSI ·DNS ·FTP ·Gopher ·HTTP ·NFS ·NTP ·SMPP ·SMTP ·SNMP ·Telnet ·DHCP ·Netconf ·RTP ·SPDY ·(more)

6

Test the applications with expected number of users without any extra infrastructure.

Collect all performance metrics under different load conditions

1.2.3 Limitations It is very difficult to understand the data flowing between the client

and server as they will be in Hexadecimal and Binary formats Correlation and Parametrization will be very difficult as data flows in

Hexadecimal format and it will take very long time to correlate the values

Sync issues will come-up. May land up writing custom code based on patterns

Verification of each response will be extremely difficult as data can be hexadecimal or binary. May land up writing wrapper codes to verify

Error handling will be very difficult and not possible in all cases

1.3 Approach 3 - By Using SQL Trace

If application under test is built using 2-tier architecture and only Client (Presentation Layer) and DB Server are present in the application. In these scenarios if we are not able to record scripts for application using any tool then we can use the following approach. Transactions / Scenarios here are nothing but executing SQL statements.

Collect the SQL commands executed on DB Server using profiler while executing the workflows in the client application

Pick up commands related to the transactions, workflows under test Create a workable SQL script using the SQL commands collected in

above steps Use performance tools to connect to the DB and create the scripts

using the above SQL commands. Parameterize the scripts with available options in the tools. Add the transactions to scripts to collect the transaction response

times. Create the different load scenarios using above scripts to achieve the

expected load on the server.

7

1.3.1 Scenarios Tested Applications with two-tier architecture ODBC of Load Runner SQL Recording of Silk Performer Unit Tests with VSTS

1.3.2 Benefits We can mimic the real user behavior as if user working on the DB. Able to test the applications with expected number of users without

any extra infrastructure. Able to measure the performance of databases independent of other

components.

1.3.3 Limitations It requires developers help to prepare the SQL scripts or

Performance resources should have very good understanding of SQL.

DB Connection/Login times are very small when compared to Client Application.

Response times of workflows are very less when compared to response times using Client application because there is no presentation layer involved.

We can only get response time of entire workflow or one SQL statement. If we need response times of any particular action then we need to have workable SQL scripts for each action. It will require very good understanding of the data workflow.

8

1.4 Approach 4 - With RDP Server as MiddlewareIf application under test is built using N-tier architecture and multiple servers(Web, App and DB Servers) are involved in application. In these scenarios if we are not able to record scripts for application using any tool then we can use following approach. To record the scripts, we have to setup RDP Server, which will be primary starting point for application performance test. We can invoke the application from RDP server and record the user action using performance tools by performing the USER actions on application inside RDP session.

Build RDP Server and install the pre-requisites to invoke the client application from RDP Server. (If Needed)

Install Client software of the application under test in RDP server. (If Needed)

Record the scripts using performance tools by performing the USER actions on application by opening the session to RDP Server.

Parameterize the scripts with available options in the tools. Add the transactions to scripts to collect the transaction response

times. Create the different load scenarios using above scripts to achieve the

expected load on the server.1.4.1 Scenarios Tested

Any Application RDP protocol of Load Runner GUI Level execution of Silk Performer

9

1.4.2 Benefits Applicable to any application, any protocol

1.4.3 Limitations It requires setting up a RDP Server and using the RDP server to

perform the tests. It requires extra H/W and S/W costs to perform the performance

tests. Response times of scenarios/transactions are high when compared

to response times using Client application because there is a delay in getting the data displayed in Client application and getting it recognized by the performance tool.

If we have to test the application with very high number of users then cost of H/W and S/W will be very high and we need to setup multiple RDP server to test the performance of the application.

1.5 Approach 5 - With Citrix Server as MiddlewareIf application under test is built using N-tier architecture and multiple servers(Web, App and DB Servers) are involved in application. In these scenarios if we are not able to record scripts for application using any tool then we can use following approach. To record the scripts, we have to setup Citrix Server, which will be primary starting point for application performance test. We will invoke the application from Citrix server and record the user action using performance tools by performing the USER actions on application inside Citrix ICA session.

Build Citrix Server and install the pre-requisites to invoke the client application from Citrix Server. (If Needed)

Install Client software of the application under test in Citrix Server. (If Needed)

10

Record the scripts using performance tools by performing the USER actions on application by opening the ICA connection to Citrix Server.

Parameterize the scripts with available options in the tools. Add the transactions to scripts to collect the transaction response

times. Create the different load scenarios using above scripts to achieve the

expected load on the server.

1.5.1 Scenarios Tested Any application ICA protocol of Load Runner Citrix ICA protocol of Silk Performer Citrix protocol of RPT

1.5.2 Benefits We can mimic the real user behavior as if user working real client

application. Able to collect all other performance metrics under different load tests

and results will be very useful.

1.5.3 Limitations It requires setting up a Citrix Server and using the RDP server to

perform the tests.

11

It requires extra H/W and S/W costs to perform the performance tests.

Response times of scenarios/transactions are high when compared to response times using Client application because there is a delay in getting the data displayed in Client application and get it recognized by performance tool.

If we have to test the application with very high number of users then cost of H/W and S/W will be very high and we need to setup multiple Citrix servers to test the performance of the application.

Conclusion

In this paper we tried to explain the strategies that can be used to performance test applications that are not supported by any of the available performance test tool. Every approach has benefits and limitations associated. Detailed Analysis need to be done before selecting a specific approach.

12

Author Biography

Yakub Reddy Gurijala, Senior Technology Architect has 13+ years of experience in IT Industry. He has in-depth knowledge in Performance testing and Engineering. He has experience in MFG, Hi-Tech, Automobile, Retail and Healthcare domains. As an architect, he will design and implement various Performance Testing solutions and works on new service offerings. He is currently handling and mentoring team for a large Retail Manufacturing client.

Sujatha Badri Narayanan has around 20 years of experience and is currently working as a Group Project Manager in Infosys Limited, Bangalore (NASDAQ: Infy www.infosys.com ). She has expertise in both software development and testing field. In testing, her expertise lies in White box testing, Performance testing, Automation, SOA and Middleware testing. She has worked on various domains such as Embedded, Avionics, Multimedia, Storage and Banking. She is PMP certified and Six Sigma Green Belt Certified. Her current research areas are Performance Engineering, Cloud and IOT Performance testing. Sujatha can be contacted at sujatha_b03@infosys.com

13

THANK YOU!

14