IBM Software Group

Rational Performance Tester Introduction

Rational Performance Tester: OverviewMulti-user automated performance testing tool for Web-based applicationsHosted in the IBM Rational Software Development Platform (RSDP), a version of the Eclipse shellRPTs tests and execution engine based on the Java programming languageHighly scalablePlatform supportRecord and test execution on Windows and LinuxAgents (test execution) on some additional Windows Server platformsEclipse An open-source initiative that provides engineers and other tool developers with a standard platform for developing plug-compatible application development tools.

Please see the RPT Release Notes for the latest updates on the various supported platforms.

Supported platforms for Rational Performance Tester: Windows 2000 Professional SP3, SP4 Performance Tester and Agent Controller Windows XP Professional SP1, SP2 Performance Tester and Agent Controller Red Hat Enterprise Linux Workstation 3 Performance Tester and Agent Controller SUSE Linux Enterprise Server 9 SP1 Performance Tester and Agent Controller Windows Server 2003 Enterprise Edition, Windows Server 2003 Standard Edition Agent Controller only Windows 2000 Server, Windows 2000 Advanced Server SP3, SP4 Agent Controller only

Rational Performance Tester: ArchitectureBuilt on top of the IBM Rational Software Development Platform (RSDP), TPTP, and custom components

IBM Rational Software Development Platform (RSDP)

Eclipse Test and Performance Tools Platform (TPTP)

IBM Rational Performance Tester (RPT)Synonyms for Eclipse:IDEWorkbenchShellPlatformA version of Eclipse was used to develop the IBM Rational Software Development Platform (RSDP).For more information on Eclipse, visit www.eclipse.org.TPTP is the framework that RPT is built on. This is where the agent architecture comes from.

IBM Rational Software Development Platform (RSDP)Workbench - The user interface and integrated development environment (IDE) in Eclipse and in the Rational Developer products, which are based on Eclipse. Workspace - In Eclipse, the collection of projects and other resources that the user is currently developing in the workbench. Metadata about these resources resides in a directory on the file system; the resources might reside in the same directory.

Powered by Eclipse TechnologyThe workbench comprises a development environment and toolset. It is a shell for plug-in applications (like RPT) and integrations. Perspectives and views allows you to access RPT functionality and files.The workspace is RPTs directory structure where projects, folders, and files are located. You use the workbench to navigate and manipulate the workspace. You can specify the workspaces location anywhere (on any disc drive, for example).

Relationship of the Workbench and Workspace

WORKSPACEProjectSupportFilesTest AssetsFolders

Workbench

Specify a Workspace to run the Workbench

Workbench

Eclipse Test and Performance Tools Platform (TPTP)Test Perspective - Common set of views and editorsHTTP Proxy Recorder - Test capture and generation toolsetExecution framework - Test suites, deployments and locationsAgent Controller - A daemon process that resides on each deployment host and provides the mechanism by which client applications either launch new host processes or attach to agents that coexist within existing processes on the hostReporting frameworkIBM Rational Agent Controller runs as a service called RAService.exe.View In Eclipse-based user interfaces, any pane in the workbench that is outside the editor area and can be stacked (dragged and dropped) on top of other views. Views provide different ways to look at or work with the resources in the workbench.Editor In Eclipse and Eclipse-based products, the area in the workbench window where files are opened for editing.HTTP Proxy Recorder Captures and generates tests in a two-step process.Test suite A collection of test cases that define test behavior and control test execution and deployment. Agent Controller Captures tests, manages running users, and data collection.

Test Perspective: Default Layout

Default Views

Perspective Indicators

Hierarchical project browser

Display area for:Test EditorSchedule EditorReal-time test run statsTest Execution HistoryReport ViewerThe Test Perspective is shown in the slide as an entire window; each of the views are contained in this perspective and are accessed through the tabs.The Test Navigator view displays files, schedules, datapools, and other test assets.The Performance Test Runs view displays test results.The Recorder Control view displays information about tests that are recorded.The Protocol Data view displays information about when tests are inspected.

Rational Performance Tester: FeaturesHTTP and HTTPS protocol support and captureGraphical test editing and workload modelingBuilt-in Verification Points (VPs)Data substitution with datapoolsReal-time monitoring and reportingReport customization and export capabilityProgramming extensibility with Java custom codeHTTP and HTTPS capture is through the HTTP Proxy Recorder.Verification point Verifies whether or not an expected behavior occurred; when a test containing a verification point runs, an error is reported if an expected behavior did not occur.Data correlation When data is mapped through the different parts of a transaction to ensure its accuracy. Web sites typically have dynamic data embedded that is not always visible.Datapool Framework used to vary user data in a transaction during test execution. They provide variable data to your tests during execution and is used for user emulation during test playback.

RPT and the Performance Testing Framework (PTF)

Set up Test Environment

Test Development and Validation

Execute Tests and Gather Measurements

Test Planning and Design

Performance Test PlanWorkload Analysis

Record TransactionsModify TestsDesign WorkloadsValidate and Debug Tests

Test Lab HardwareSystem SoftwareData Set Test and Monitoring Tools

Reset System and Data StateRun Monitoring ToolsRun TestsEvaluate Results

IBM Rational Performance Tester

Rational Performance Tester: Test Development

Project and file browser

Test editor

Protocol Data view

The Test Navigator view is similar to the Windows Explorer directory structure.This test development perspective is where you will spend most of your time in RPT.

Rational Performance Tester: Workload Design

Schedule editor

Schedule Run Settings

ScheduleA performance schedule enables you to accurately emulate the actions of individual users.The left side of the Schedule editor contains the workload.

Rational Performance Tester: Test Execution

Change Log LevelAdd Virtual UsersStop Test Run

Run-Time Test ControlThe Performance Test Runs view displays test results and is where you access reports. This view also controls the following during test execution:Log level the indicator of the amount of logging to be done. The higher the level, the more information to be logged. Virtual users add or subtract the number of virtual users during test execution.Stop test run halt the test.

Rational Performance Tester: Test Monitoring

Default Real-Time Reports

Real-time reports are available during test execution (when you monitor the test). These reports are identical to the default reports you can access after test execution.You can monitor a test during its execution by clicking the tabs to access various reports.

Rational Performance Tester: Results Analysis

Reports

Execution History

Log of Events

The Performance Report is the default report.Component (1) In Eclipse, one or more plug-ins that work together to deliver a discrete set of functions; (2) A reusable object or program that performs a specific function and is designed to work with other components and applications. JavaBeans is a well-known architect for creating components.JavaBean A portable, platform-independent, reusable component model, as defined for Java by Sun Microsystems.JavaBeans events Events that are signaled when an activity occurs, such as a button being pressed or a window being closed. The visual editor for Java shows events in the JavaBeans view. You can use this view to add and remove events.

Key TermsView - In Eclipse-based user interfaces, any pane in the workbench that is outside the editor area and can be stacked (dragged and dropped) on top of other views. Views provide different ways to look at or work with the resources in the workbench.Perspective - A group of views that show various aspects of the resources in the workbench. The workbench user can switch perspectives, depending on the task at hand, and customize the layout of views and editors within the perspective.Projects - In Eclipse, a unit of organization used to group folders or packages. Projects are used for building, version management, sharing, and organizing resources.

Views Contained within the Test PerspectiveTest-related ViewsTest NavigatorRecorder ControlProtocol DataPerformance Test RunsOther ViewsPropertiesTasks (rarely used)Outline (not used)

Customize your perspective close views to minimize clutter

Editors and ViewersTest and Schedule EditorsReport and History Viewers

Editor and Viewer areaViews are not the same as viewers. Editors and viewers are components of RPT.

Other Perspectives and ViewsPerspectivesJavaResourceViewsNavigatorHelpful to add this view to the Test Perspective

Package Explorer

Navigator View exposes file extensions and hidden files

Window > Show View > Other > Basic > NavigatorThe Navigator view differs from the Test Navigator view in that it provides a hierarchical view of the resources in the Workbench. From here, you can open files for editing or select resources for operations such as exporting. The Test Navigator view is the main navigation view for browsing and editing test suites and viewing test results. It is displayed by default in the Test perspective and shows an execution-oriented view of test projects. The code-oriented elements of test projects are shown in the Package Explorer view.

Working with PerspectivesWindow > Open PerspectiveSwitch between open Perspectives using the shortcut bar

Managing Perspectives

Manage PerspectivesReset to restore

Perspective Shortcut Bar

Or click Other

Working with ViewsWindow > Show ViewCustomize the layout of your viewsGroup, dock, and layer using drag and dropMinimize, maximize, and resizeShow, close, move to Fast View

Resize

Min/Max

Or click OtherFast views are hidden views that can be made visible quickly. They work the same way as normal views except when they are hidden, they do not take up screen space on the Workbench window.

Fast View ToolbarAn alternative to closing views is to move them to the Fast View toolbar.Click an icon in the Fast View toolbar to open the View.Click anything else in the Perspective to close the View.

Fast View toolbar

To create a Fast View, click a view (tab), drag the view to the toolbar.

Rational Performance Tester: Data StorageWorkspaceDefault location: C:\Documents and Settings\user\IBM\rationalsdp6.0\workspaceMaintains perspectives and view settings (.metadata)ProjectStorage location for:Recordings (trace files)Test Suites (Performance Tests and Schedules)Test logs and resultsDatapools.metadata is a directory.Customized reports are saved in the workspace and are persistent in the .view file.Trace files Files that show the time and sequence of methods called by run-time base classes. Use these files to pinpoint the failure. Test suite A collection of tests that define test behavior and control test execution and deployment.Test A set of tasks, scripts, or routines that automate the task of testing software. Test log A file containing the execution history.

Performance Test ProjectSpecialized Java projectSpecific directory structure for housing and running RPT testsTemplate provided out-of-the-box for manual creationAuto-created on first record if no project existsDefault name: testprojProjects can reside inside the Eclipse Workspace or in another locationIf others will access your test project, consider not locating it on your local drive (C:\Documents and Settings\user\IBM\rationalsdp6.0\workspace).

Create a Performance Test Project ManuallyFile > New > Performance Test Project

Create a Performance Test Project Manually (cont.)Creates a Java project customized for performance testingBy default, it is created in the Eclipse workspaceProject name cannot include/ \ * : < > | ?Names ending with a period

Click No in the Create Performance Test dialog box if your workload analysis document has not been completed. Resist the temptation to start recording tests until you are ready to begin.

Test Navigator View: Project and File ManagementProject browser toolCreate and browse objectsOpen or close projectsDelete objectsFunctions enabled using shortcut menuRPT specificWorkbench toolsFiltered view

Default project structure

The Test Navigator view is a filtered view. It displays a subset of the files that are in the project.

Test Navigator View: File FilteringShows only commonly edited and viewed files such as:Performance Tests and SchedulesDatapoolsLocationsSupporting files not shownSome directories appear empty

Tip: File Management Using ViewsTest Navigator view filters many file typesPerformance Test Runs only show test results files

Navigator View exposes additional project files

Navigator View: Alternate File BrowserNo filteringTest filesRecordings (trace files)Test and SchedulesSource and compiled filesReport source filesRaw test run dataProject files.classpath, and so on

Use the Performance Test Runs view to run reports.

Testproj Default Project: Directory Structure

Default location for all projects is the workspace directory

Testproj Default Project: Directory Structure (cont.)DefaultbinsrcSuggestedTestsSchedulesTest ResultsDatapoolsDeployments or Locations

Suggested Directory Structure

DefaultDirectory Structure

Creating FoldersTest Navigator view; shortcut menuFile > New > FolderWorkbenchmenuFile > New> FolderSpecifyLocationName

Default Directory Structure and Test Execution

Test Navigator does not display test source or compiled filesBefore Test Execution\bin

\src

After Test Execution\bin\test.class files\src\test.java filesBecause the Test Navigator view employs file filtering, you will not see .class or .java files after test execution. Use the Java perspective to view these files.

Overall PageIt contains a progress indicator and bar chartsThe bar charts showOverall success of run% Page element status code success% Page status code successOptionally, % Verification Points passed (Title, Page or Size)

Summary ReportIt contains Run Summary, Page Summary and Page Element summaryEach summary explains Run StatusDurationActive/completed/total userMax/Min/Avg response timeTotal Hits% VP passed

Page PerformanceIt contains two section i.e. graph and tableAvg. response time for each pageThe table containsMin response timeAvg. response timeMax response timeNo of attempts/Sec No. of total attempts

Response Vs Time Summary ReportIt shows avg. response time through timeIt contains two line graphs and related summary tableOne line graph shows the avg. response time for all page during the runOther line graph shows the avg. response time for all page element during the runThe tables shows total avg. response time

Response Vs Time DetailsIt shows the response trend through timeIt contains a graph and a tableGraph shows the avg. response of each page through timeMin. response timeMax response timeAvg. response timeNo of page attempts per intervalRate of page attempts per interval

Page ThroughputIt gives the overview of the frequency of the requests being transferred per intervalPage attempts rate and page hit rate per intervalSummary tables shows total hit rates and counts for each pageActive and completed users over the course of runSummary table Sample interval

Server Health Summary ReportOverall indication about the server response to the user loadThe bar charts (left)Total Page attemptsPage hitsStatus code successThe bar chart (right)Total Page elements attemptsPage element hitsPage element status code successThe tables gives the same info as the charts

Server Health DetailsProvides specific details about 10 least successful pagesThe table containsNo of attempts HitsSuccess Attempts per second

Transaction Report OverallContains a line graph and summary tableLine graph shows avg. execution time for all transaction within intervalThe table showsAvg. execution time Std deviation of the avg. response timeLongest transactionShortest transaction

Transaction Report Duration Vs TimeAvg. response time of each transaction during an intervalThe table containsMin execution timeAvg. execution timeStd deviation of avg. response timeMax execution timeRate per secondNo. of attempts

Transaction Report Transaction Throughput PageOverview of the transaction frequency and the no of user adding load over the runGraph (left) overall rates for starting and completing transaction during intervalTable (left) showsTransaction rate per secNo of transactionGraph (right) no of active and completed user over the courseTable contains Active userCompleted userTotal users

Verification Point ReportSummary (Graph and Table)% verification point passed per interval Page verification Point (Table)No of VP passedNo of VP failed% of VP passedPage Element Verification Point (Table)No of page and elementNo of VP passedNo of VP failed% of VP passed

Resource MonitoringUsed to capture data such as CPU usage, Memory etc.Comprehensive view of a system to aid in problem determinationCan be done by Test agentTivoli Monitoring agentWindows Performance MonitorUNIX rstatd monitorHow to performEnabling resource monitoringAdding resource monitoring data source

Resource Monitoring Enabling Enabling resource monitoringEnable resource monitoring from scheduler tabAdding resource monitoring data sourceConfigure the time intervalConfigure the countersView reportA separate tab generated with the details log

Performance SchedulerGeneralRun DurationThink TimeResource MonitoringStatisticsTest LogResponse time Break downProblem Determination

Response Time BreakDownResponse time breakdown lets you drill down into the response time statistics on any HTTP page element. The statistics show how much time was spent in each part of the system under test.Which application on which server is the performance bottleneck, and then drill down further to determine exactly which package, class, or method is causing the problem.