Reengineering-PERFORMANCE TESTING

download Reengineering-PERFORMANCE TESTING

of 14

Embed Size (px)

Transcript of Reengineering-PERFORMANCE TESTING

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    1/14

    Load Runner Test ing / Th ink T im e Set t ing Cons iderat ions

    Version 1.0 / 11.05.2004

    Ar ik L i t v inPubl ishing CoordinatorPer formance Lab Israel

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    2/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 2

    What is a Stress Test

    Explain what is Think Time in scope ofperformance tests

    Detail possible Think Time Setting Manipulations

    Explain the two approaches considered withregards to the Think-Time setting

    Recomend on the right approach to adopt, inscope of the SAP Netweaver Stack PerformanceQualification activities

    Agenda

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    3/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 3

    Stress testing, which is a specialized form of performancetesting, is aimed at discovering under what conditions anapplication's performance becomes unacceptable

    This is done by changing the application inputs to place a

    changed load on the application, measuring how performance

    changes with the variation in those inputs This activity is also called load testing, though load testing

    usually describes a specific type of stress testing - increasing

    the number of users to stress test your application

    If there are many inputs, or a large range of values over which

    to vary those inputs, or in cases where an accurate re-tests are

    needed, an automated tool is required, such as Mercury Load

    Runner, especially in cases of increased number of users

    What Is a St ress Test ?

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    4/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 4

    Actual users do not commit transactions, such as portal

    navigations, immediately after another. There are generally delays

    between them

    Think time is the standard term to express adding a delay into a

    script to more realistically simulate user behavior. It is the length of

    this waiting period that is perceived as the performance of thesystem. In other words, it is the elapsed time between the receipt of

    a reply to one request and the submission of the next one

    Most load testers support both static & randomly generating think-

    times - based on a statistical distribution. Think Time can also be

    used to calculate server throughput:

    Server Throughput = # of concurrent users / ( Average Response Time + Think Time )

    What i s Th ink T im e in Sc ope of Per formanc e Tests ?

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    5/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 5

    How t o Manipu late Tes ts us ing Th ink T ime Set t ing ?

    Often, Customers have only limited Load Runner licenses touse for the load tests. Think time manipulation can help in suchsituations but one must be aware of the limitations this imposeson the load

    The number of VUs required to achieve the desired loadcan be reduced by decreasing the think time proportionally

    e.g. 2,000 VUs clicking once every 4 minutes produce thesame amount of requests as 1,000 VUs clicking every 2minutes

    It is always recommended to state transactions think-times

    outside transactions script themselves

    It is also recommended to avoid scenarios with no think timessetting completely since this expects to create a non-realisticload on the server

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    6/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 6

    What a re The St a t i c & t he Random Approaches ?

    Approach One Fixed Think Time Setting

    Think time runtime setting should be manually set to a fixeddesired time since using a random percentage of think time:

    Prevent an exact repeatable reproduction of the test

    Simplicity of tests.

    Can be applied with other test applications

    Approach Two Randomized Think Time Setting

    Think time runtime setting should be manually set to arandomized value, within a pre-defined range, in order to:

    Avoid repeated transaction response-time peaks, once user re-starttheir activity after a Full GC (see graphs..)

    Simulate more realistic load performed by real users.

    Not getting synchronized with application settings (e.g sessiontermination and cache settings)

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    7/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 7

    What happens w i t h s ta t ic Th ink T im e a f te r Fu l l GCs ?

    New VUser Login

    Transaction Per Second during the test

    Transaction Response Time during the test

    Test Duration

    FullGC

    FullGC

    VirtualUsers

    When using fixed Think Time, full GC cycles cause VUs to align theirIterations once the cycle is done, causing unnatural peaks of load onthe server, increase-trend of response time and decrease trend of TPS

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    8/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 8

    What happens w i t h random Th ink T im e a f te r Ful l GCs ?

    VirtualUsers Full

    GCFullGC

    Test Duration

    New VUser Login

    Transaction Per Second

    Transaction Response Time

    When using randomized Think Time, the difference in iteration lengthhelps measurements recover to their average values as before the full GC

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    9/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 9

    Act ual Present at ion of TPS us ing St at ic t h ink T im e

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    10/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 10

    Using fixed Think Time, causes all users to make a progress

    at the same rate, performing the script at the same time,

    which causes notable peaks in server performance (High

    Response times and Low TPS levels)

    Using randomized Think-time, causes users to perform thescript randomly along the test duration, avoiding high-

    performance peaks as mentioned above and provide linearly

    performance behavior

    What do t he Graphs Prove ?

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    11/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 11

    In long-term tests (more than 10 minutes)

    Using random Think Time values, from a pre-defined scale, within,

    will create a greater number of random Think Time values. This large

    set of values, will increase the probability that the average of all these

    values, will be more closely to the static, required Think Time value

    In short-term tests (less than 10 minutes)

    When using random Think Time values, from a pre-defined scale, in

    short-term tests, the average Think Time might be different on each

    run, due to a small number of iterations, causing the results to be

    irreproducible

    What is t he di f fe rence betw een shor t and long-t erm t ests ?

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    12/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 12

    Conclus ions

    In long termed scenarios, with idle time between iterations, it is

    recommended to use random think time (50% -150% of recorded time)

    In short termed scenarios, fixed think time can be used in order to enable

    repeatable tests. However, in such cases, response time levels must be

    carefully observed to avoid peaks

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    13/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 13

    No part of this publication may be reproduced or transmitted in any form or for any purpose without the expresspermission of SAP AG. The information contained herein may be changed without prior notice.

    Some software products marketed by SAP AG and its distributors contain proprietary software components of othersoftware vendors.

    Microsoft, WINDOWS, NT, EXCEL, Word, PowerPointand SQL Serverare registered trademarks ofMicrosoft Corporation.

    IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390,OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, Informixand InformixDynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries.

    ORACLE is a registered trademark of ORACLE Corporation.

    UNIX, X/Open, OSF/1, and Motifare registered trademarks of the Open Group.

    Citrix, the Citrix logo, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, MultiWinandother Citrix product names referenced herein are trademarks of Citrix Systems, Inc.

    HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C, World Wide Web Consortium,Massachusetts Institute of Technology.

    JAVAis a registered trademark of Sun Microsystems, Inc.

    JAVASCRIPT is a registered trademark of Sun Microsystems, Inc., used under license for technology invented

    and implemented by Netscape.

    MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One.

    SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com, xApps, mySAP Business Suite, and other SAP products andservices mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG inGermany and in several other countries all over the world. All other product and service names mentioned are thetrademarks of their respective companies.

    Copyr ight 2003 SAP. Al l R ights Reserved

  • 7/31/2019 Reengineering-PERFORMANCE TESTING

    14/14

    SAP Portals Israel 2003, Title of Presentation, Speaker Name / 14

    Corporat e Colors for PPT

    SAP Blue SAP Gold SAP Gray

    You can use these colors with gradations

    Removepageforslide

    show