Download - Reengineering-PERFORMANCE TESTING

7/31/2019 Reengineering-PERFORMANCE TESTING

1/14

Load Runner Test ing / Th ink T im e Set t ing Cons iderat ions

Version 1.0 / 11.05.2004

Ar ik L i t v inPubl ishing CoordinatorPer formance Lab Israel


2/14

SAP Portals Israel 2003, Title of Presentation, Speaker Name / 2

What is a Stress Test

Explain what is Think Time in scope ofperformance tests

Detail possible Think Time Setting Manipulations

Explain the two approaches considered withregards to the Think-Time setting

Recomend on the right approach to adopt, inscope of the SAP Netweaver Stack PerformanceQualification activities

Agenda


3/14


Stress testing, which is a specialized form of performancetesting, is aimed at discovering under what conditions anapplication's performance becomes unacceptable

This is done by changing the application inputs to place a

changed load on the application, measuring how performance

changes with the variation in those inputs This activity is also called load testing, though load testing

usually describes a specific type of stress testing - increasing

the number of users to stress test your application

If there are many inputs, or a large range of values over which

to vary those inputs, or in cases where an accurate re-tests are

needed, an automated tool is required, such as Mercury Load

Runner, especially in cases of increased number of users

What Is a St ress Test ?


4/14


Actual users do not commit transactions, such as portal

navigations, immediately after another. There are generally delays

between them

Think time is the standard term to express adding a delay into a

script to more realistically simulate user behavior. It is the length of

this waiting period that is perceived as the performance of thesystem. In other words, it is the elapsed time between the receipt of

a reply to one request and the submission of the next one

Most load testers support both static & randomly generating think-

times - based on a statistical distribution. Think Time can also be

used to calculate server throughput:

Server Throughput = # of concurrent users / ( Average Response Time + Think Time )

What i s Th ink T im e in Sc ope of Per formanc e Tests ?


5/14


How t o Manipu late Tes ts us ing Th ink T ime Set t ing ?

Often, Customers have only limited Load Runner licenses touse for the load tests. Think time manipulation can help in suchsituations but one must be aware of the limitations this imposeson the load

The number of VUs required to achieve the desired loadcan be reduced by decreasing the think time proportionally

e.g. 2,000 VUs clicking once every 4 minutes produce thesame amount of requests as 1,000 VUs clicking every 2minutes

It is always recommended to state transactions think-times

outside transactions script themselves

It is also recommended to avoid scenarios with no think timessetting completely since this expects to create a non-realisticload on the server


6/14


What a re The St a t i c & t he Random Approaches ?

Approach One Fixed Think Time Setting

Think time runtime setting should be manually set to a fixeddesired time since using a random percentage of think time:

Prevent an exact repeatable reproduction of the test

Simplicity of tests.

Can be applied with other test applications

Approach Two Randomized Think Time Setting

Think time runtime setting should be manually set to arandomized value, within a pre-defined range, in order to:

Avoid repeated transaction response-time peaks, once user re-starttheir activity after a Full GC (see graphs..)

Simulate more realistic load performed by real users.

Not getting synchronized with application settings (e.g sessiontermination and cache settings)


7/14


What happens w i t h s ta t ic Th ink T im e a f te r Fu l l GCs ?

New VUser Login

Transaction Per Second during the test

Transaction Response Time during the test

Test Duration

FullGC

FullGC

VirtualUsers

When using fixed Think Time, full GC cycles cause VUs to align theirIterations once the cycle is done, causing unnatural peaks of load onthe server, increase-trend of response time and decrease trend of TPS


8/14


What happens w i t h random Th ink T im e a f te r Ful l GCs ?

VirtualUsers Full

GCFullGC

Test Duration

New VUser Login

Transaction Per Second

Transaction Response Time

When using randomized Think Time, the difference in iteration lengthhelps measurements recover to their average values as before the full GC


9/14


Act ual Present at ion of TPS us ing St at ic t h ink T im e


10/14


Using fixed Think Time, causes all users to make a progress

at the same rate, performing the script at the same time,

which causes notable peaks in server performance (High

Response times and Low TPS levels)

Using randomized Think-time, causes users to perform thescript randomly along the test duration, avoiding high-

performance peaks as mentioned above and provide linearly

performance behavior

What do t he Graphs Prove ?


11/14


In long-term tests (more than 10 minutes)

Using random Think Time values, from a pre-defined scale, within,

will create a greater number of random Think Time values. This large

set of values, will increase the probability that the average of all these

values, will be more closely to the static, required Think Time value

In short-term tests (less than 10 minutes)

When using random Think Time values, from a pre-defined scale, in

short-term tests, the average Think Time might be different on each

run, due to a small number of iterations, causing the results to be

irreproducible

What is t he di f fe rence betw een shor t and long-t erm t ests ?


12/14


Conclus ions

In long termed scenarios, with idle time between iterations, it is

recommended to use random think time (50% -150% of recorded time)

In short termed scenarios, fixed think time can be used in order to enable

repeatable tests. However, in such cases, response time levels must be

carefully observed to avoid peaks


13/14


No part of this publication may be reproduced or transmitted in any form or for any purpose without the expresspermission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of othersoftware vendors.

Microsoft, WINDOWS, NT, EXCEL, Word, PowerPointand SQL Serverare registered trademarks ofMicrosoft Corporation.

IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390,OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, Informixand InformixDynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries.

ORACLE is a registered trademark of ORACLE Corporation.

UNIX, X/Open, OSF/1, and Motifare registered trademarks of the Open Group.

Citrix, the Citrix logo, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, MultiWinandother Citrix product names referenced herein are trademarks of Citrix Systems, Inc.

HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C, World Wide Web Consortium,Massachusetts Institute of Technology.

JAVAis a registered trademark of Sun Microsystems, Inc.

JAVASCRIPT is a registered trademark of Sun Microsystems, Inc., used under license for technology invented

and implemented by Netscape.

MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One.

SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com, xApps, mySAP Business Suite, and other SAP products andservices mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG inGermany and in several other countries all over the world. All other product and service names mentioned are thetrademarks of their respective companies.

Copyr ight 2003 SAP. Al l R ights Reserved


14/14


Corporat e Colors for PPT

SAP Blue SAP Gold SAP Gray

You can use these colors with gradations

Removepageforslide

show