Introductions, Objectives and Overview

Reason for performance test

Slow applications produce user frustration

Any feature can be source of performance frustration

Inappropriate resource use can be disaster like failure.

Perception Load time ? transaction measure? (how is it different

from time measure) Resource utilization? Number of users load? Ability to run without interruptions?

Black-box perspective Response time Resource utilization Reliability

White-box perspective Code profiling Resource leakage Code coverage ( branch and statement )

Performance Perpetrator

Response Time is total time from once request made till completion.

Response time can be calculated based on queuing model M/M/1.

Formula is R= S/1-U. where R is response time S is No of server U is utilization

Response time and Resource utilization

• Example: The service time for a single component increases as the utilization increases beyond 60 percent. – If a transaction requires 1 second of processing by a

given component, at 45 percent utilization, it could be expected to take (1/(1-0.45)=) 1.66 seconds

– At 80 percent utilization time would be (1/(1-0.8)=) 5 seconds.

– When utilization for the resource reaches 90 percent, the transaction could be expected to take 10 seconds to make its way through that component.

CPU Utilization Trade offs - High Utilization graph of the CPU does not always indicate a performance issue.CPU performs all calculations that are

needed to process transactions. The more transaction-related calculations

that it performs within a given period, the higher the throughput will achieved for that period.

As long as transaction throughput is high and proportional to CPU utilization, this indicate that computer is being used to the fullest advantage.

If the response time for transactions increases to such level that it become unacceptable, this could be an indication that The processor might be swampedThe transaction process load is too high for

the System to manage. The CPU is processing transactions

inefficiently CPU cycles may be diverted

Network Bandwidth Utilization - Network bandwidth utilization should be captured to understand the possible bottlenecks because of large volumes of data is transfer

Database size and System performance - Performance degrades as the database size increases which do not take care of the increasing size.

Role of Hardware in Streaming applications - Same bit rate video shows different behavior when played from a USB 2.0 and 3.0 devices.

Performance of APIs - Performance of system is sum total of API responses + services

Other Factors

Consistent hardware Consistent software Consistent measurement method Benchmarks

Scholarly Perpetrator

Loading of unnecessary components Deadlocks/sleeps Memory leaks/Corruption Continuous increasing data size Unnecessary encryption /decryption Algorithms with out optimization Heavy Graphics Too much CPU or I/O bound

Common perception

Performance testing requires clearly identified performance goals like page load time of less than 4 seconds, page

fetch and rendering time of less than 2 seconds, saving file of certain size/complexity in certain time

Identify ranges for these goals with the operating conditions

For other scenarios(not in performance test plan ) identify typical use conditions and a subjective feel of acceptable parameters

Performance Destination

Reduce global variables Optimized string sizes Removal of extra strings Late initialization

Resolvent

Identification of features to be tested for performance

Set of performance goals for each feature –( response time, reliability and indicative ranges for resource utilization, )

Specification of Test environment creation of tests and scenarios Test execution and data collection Analysis Iterations of performance tests after

changes

Testing Process

All features are not equal priority Before performance testing begins

prioritize featuresMost used featuresCritical featuresNon-feature actions

Portent Prioritization

Performance Testing atUnit Level?? Integration Level??System Level??System Integration Level??

Which is better? First performance test cycle??Last performance test cycle??

Why?

Time for performance testing

Performance testing process should start from the stage which is determined by the risk associated with performance of the system.

If risk based testing method should adopted, performance risks should identified and prioritized at the start of project. If performance is deemed to be a high

priority risk area, performance testing is typically started earlier.

Client hardware benchmarking is also important in Performance testing. The hardware incapability may provide faulty

results, which leads confusion of performance issue of server.

Memory , CPU (cycles must be taken care when dealing with multi-thread applications), network bandwidth etc. should be benchmarked prior to start the performance test.

Benchmarking of client machine helps in getting a better idea of how much throughput could be attainable, and this could be useful in scaling process.

Benchmarking of client hardware

dynamic analysis performed on the code to monitor the code as the program executes.

Profiling is the collection of performance analysis data of a software at execution.

Profiling helps improve the software’s performance through code optimization.

Code profiling

Why Code Profiling?• Profiling is made to optimize the code with

issues• Profiling:

– Identifies Bottlenecks in Performance of the Code (program)

– Utilization Catches Memory and Resource Leaks

– Determines the subroutines/ functions consuming the longest time to execute

• Profiling can improve the overall performance of the code, and thereby the whole software.

What to look for while Profiling?Profiling tools basically look into the code

for specific areas of concern like: Resource/Memory Leaks Library Calls (Static as well as Dynamic) Functions execution time mostly etc.

A leak occurs when a program uses resources on a computer but never releases the resources.

The resources can include handles, physical RAM, the paging file, system resources, etc.

normally it is the result of a defect in a program that prevents it from freeing memory that it no longer needs

Symptoms of a memory leak include – system hangs, slow system performance, and program memory error messages.

Leaks

Leaks: Why an issue? It can diminish the performance of the

computer - By reducing the amount of available memory.

OR Too much of the available memory may become

allocated and the system/application may stop working correctly, fail, or slow down unacceptably.

• Memory Leak Scenarios – – The following code sample depicts a

Memory Leak

Memory Leak: “x” has not been freed

Conclusion it can demonstrate that the system

meets performance criteria It can compare two systems to find

which performs better It can measure what parts of the system

or workload cause the system to perform badly