Post on 24-Dec-2014
description
ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution
PLANNING LOAD TESTINGPLANNING LOAD TESTING
LESSON 1LESSON 1
CONFIDENTIAL © Copyright IBM Corporation 2006
Why Plan?Why Plan?
So that you will be able to:
Build test scenarios that accurately emulate your system
Know the required resources for testing
Define the load test objective
Define success criteria (in measurable terms)
CONFIDENTIAL © Copyright IBM Corporation 2006
LoadRunner WorkflowLoadRunner Workflow
Plan Load TestingPlan Load Testing11
Create Scenario3Run Scenario4Analyze Results5
Create Vuser Scripts2
Debug and Run Scenario Again6
Define load testing objectives
Analyze system under test
Plan LoadRunner Implementation
CONFIDENTIAL © Copyright IBM Corporation 2006
Planning Load Testing Planning Load Testing
In this lesson you will learn how to:
1 - Define load testing objectives
Objectives
2 - Analyze the system under test
3 - Plan LoadRunner implementation
Vuser Vuser
VuserVuser
Controller
CONFIDENTIAL © Copyright IBM Corporation 2006
1 - Define Load Testing Objectives1 - Define Load Testing Objectives
Why test objectives?
“Successful projects are those that have a defined test objective… and provide actionable results and conclusion…”
Source: “The Art of Testing Network Systems” Robert W. Buchanan, Jr.
Limit your objectives
“...Projects should have one or, at the most, two testobjectives… More than two objectives dilutes the effortand makes [ testing ] virtually impossible.”
State objectives in measurable terms
“...There is nothing worse than completing [ a test ]and then disagreeing on whether or not the system passed.”
CONFIDENTIAL © Copyright IBM Corporation 2006
Common Test ObjectivesCommon Test Objectives
Application response timeHow long does it take to complete a task?
Configuration sizingWhich configuration provides the best performance level?
AcceptanceIs the system stable enough to go into production?
RegressionDoes the new version of the software adversely affect response time?
ReliabilityHow stable is the system under a heavy work load?
Capacity planningAt what point does degradation in performance occur?
Bottleneck identificationWhat is the cause of degradation in performance?
Product evaluationWhat is the best server for 100 users?
CONFIDENTIAL © Copyright IBM Corporation 2006
State Objective in Measurable TermsState Objective in Measurable Terms
Measurable objectives provide baselines for evaluating results
Examples
Examples
Response timeunder a load of 50 sales representatives, the maximum response time for any transaction will not exceed 1 minute
Reliabilitythe system will run for 72 hours at 50% load(as measured by CPU utilization) without a crash, or any corrupted data
CONFIDENTIAL © Copyright IBM Corporation 2006
2 - Analyze The System Under Test2 - Analyze The System Under Test
Identify system componentsschematic diagram of system
Describe configuration of each componenthardware/CPU speed
operating system
memory
Describe user profiletypes of users
common tasks for each user
task distribution during the work day
CONFIDENTIAL © Copyright IBM Corporation 2006
Identify System ComponentsIdentify System Components
Example
ExampleClients
586/ Windows 98
OCI
Pentium 400MHZ
NT 4.0
Oracle 8.0
Server
CONFIDENTIAL © Copyright IBM Corporation 2006
Describe ConfigurationDescribe Configuration
Front End ClientNo of clients 120Hardware/Memory 586 PC / 32 MbOperating system Windows 95Development tool Visual Basic
Example
Example
Middleware/App Server (if exists)Server typeCommunication with serverServer operating system
Back End ServerServer type Database- Oracle 8.0Communication with server ODBCServer operating system NT
Other Network ComponentsCommunication device handles 50 calls max.
Provide details about the configuration of each component:
CONFIDENTIAL © Copyright IBM Corporation 2006
User ProfileUser Profile
Who are the users?
How many users are there?
What tasks do the users perform?
Business Process
Type and number of users
Task frequency (throughput per hour)
ReservationAgent (170)
Freq. FlyerAgent (50)
Manager (30)
1- Enter Orders
3 - Calc. F.F. Miles
4 - Calc. Sales
2 - Update Orders
100
50
70
8
B.P.Users
Example
Example
25
10
5
How often do users perform their tasks?
CONFIDENTIAL © Copyright IBM Corporation 2006
Transaction ProfileTransaction Profile
What tasks do the users perform?
2- Enter Orders
4 - Calc. F.F. Miles
5 - Calc. Sales
3 - Update Orders
125
60
75
8
Business ProcessTransactions Per Hour
1- Logon 10
Example
Example
Tasks Frequency
How often do users perform their tasks?
CONFIDENTIAL © Copyright IBM Corporation 2006
3 - Plan the LoadRunner Implementation3 - Plan the LoadRunner Implementation
1 - Set up test environment
2 - Define type and number of Vusers to be created
3 - Determine hardware required to run Vusers
CONFIDENTIAL © Copyright IBM Corporation 2006
Set Up Test EnvironmentSet Up Test Environment
All testing should be done in a test environment
Allocate and configure machines to real-world conditionsone for each type of GUI Vuser
one for the database server
any other middleware server(s)
Allocate machines to run necessary number of DB Vusers
Allocate and configure one machine for Controller
Seed test database with appropriate amounts of data
Verify that business processes function correctly
CONFIDENTIAL © Copyright IBM Corporation 2006
What Is a GUI Vuser?What Is a GUI Vuser?
A WinRunner script, performing actions on the client application
WinRunner
Allows you to measure the response time a real user will experience end-to-end
One GUI Vuser per host machine
Host
GUI Vuser
Client application
CONFIDENTIAL © Copyright IBM Corporation 2006
What Is a DB Vuser?What Is a DB Vuser?
Multiple DB Virtual UsersMultiple DB Virtual Users ServerServer
API Calls
C program that sends API calls to the server
Allows you to generate load on the server using few machines
Excludes application’s GUI processing time
Many DB Vusers per host machine
C Program
C Program
C Program
C Program
CONFIDENTIAL © Copyright IBM Corporation 2006
GUI vs. DB VuserGUI vs. DB Vuser
Database Server
Application Server
Application Server
Application Server
WAN and/or
Internet
Example
Example
GUI VuserGUI Vuser: End-to-end (including GUI)
GUI VuserGUI Vuser: End-to-end (including GUI)
DB VuserDB Vuser: Network and Server (No GUI)
DB VuserDB Vuser: Network and Server (No GUI)
CONFIDENTIAL © Copyright IBM Corporation 2006
Type and Number of VusersType and Number of Vusers
How many Vusers of each type (DB/GUI) do you need to run?
Guidelines:one or few GUI Vusers for each type of user connection
multiple DB Vusers generate remaining load for that user
100 customer service users(LAN connection) 2 98
50 Tokyo users (dial-in ISDN connection) 1 49
5 background batch processes — 5
20 managers(15 with 486; 5 with 586 machines)
1 (486 PC)1 (586 PC) 18
UserExample
Example
DBGUI
CONFIDENTIAL © Copyright IBM Corporation 2006
Which Hardware/Configuration?Which Hardware/Configuration?
Based on previous steps, which hardware and software configuration is needed for testing?
How many machines (PC/UNIX)?
How much memory?
Processor speed?
How much disk space?
How much swap?
CONFIDENTIAL © Copyright IBM Corporation 2006
Hardware/Configuration GuidelinesHardware/Configuration Guidelines
General:
One machine for Controller
One machine per GUI Vuser
GUI Vuser machine configuration should be similar to ‘real-world’ machine
Test server machine should be configured to real world
DB Vuser machines:
Can use a combination of NT and UNIX
Machine configuration depends on number of Vusers and activity of user
Exact configuration requirements in Appendix A
CONFIDENTIAL © Copyright IBM Corporation 2006
Which Hardware/Configuration?Which Hardware/Configuration?
Controller
NT machine Pentium 200 MHz 64 MB memory
Database Server
1 GUI Vuser(Accounting)
• NT machine • Pentium 200 MHz• 32 MB memory
29 DB Vuser(Accounting)
• NT machine•Pentium 166
MHz• 64 MB memory
UNIX machine 256 MB memory
Example
Example
450 DB Vusers(Bank tellers)
CONFIDENTIAL © Copyright IBM Corporation 2006
Summary Summary
Define load testing objectiveslimit your objectives
state objectives in measurable terms
Analyze the system under testsystem components
configuration of each component
user profile (users, tasks, distribution)
Plan LoadRunner implementationset up test environment
decide type and number of Vusers to be created
hardware required to run Vusers
CONFIDENTIAL 23 © Copyright IBM Corporation 2006
Lab ExerciseLab Exercise
ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution
LOADRUNNER SCENARIOS
LESSON 2
LR/NTIntro6.0/02Copyright 2000Mercury Interactive Corp.
CONFIDENTIAL © Copyright IBM Corporation 2006
LoadRunner Workflow
Plan Load TestingPlan Load Testing11
Create ScenarioCreate Scenario33Run ScenarioRun Scenario44Analyze ResultsAnalyze Results55
Create Vuser ScriptsCreate Vuser Scripts22
Debug and Run Scenario AgainDebug and Run Scenario Again66
• Define and connect hosts
• Define Vusers and scripts
• Group and schedule Vusers
• Set rendezvous policy
CONFIDENTIAL © Copyright IBM Corporation 2006
Lesson Objectives
In this lesson you will learn: LoadRunner elements How to set up a scenario How to run a scenario
CONFIDENTIAL © Copyright IBM Corporation 2006
LoadRunner Environment
LoadRunnerController
Vusers
Host 1 Host 2
VusersDatabaseServer
CONFIDENTIAL © Copyright IBM Corporation 2006
The LoadRunner Controller
Runs scenarios Controls and communicates with each Vuser via the Agent Displays output messages Displays execution status Collects performance data
CONFIDENTIAL 29 © Copyright IBM Corporation 2006
Controller Main Window
Initialize /Run Vuser
Vusergroups
Vuser details
Sub-window tabs
CONFIDENTIAL © Copyright IBM Corporation 2006
Controller Sub-Window Tabs
Hosts window- all available hosts and their connection statuses
Scripts window- paths to all available Vuser scripts
Vusers window- create and select Vusers and groups
Online window- view real-time system resources during scenario execution
CONFIDENTIAL © Copyright IBM Corporation 2006
How Does Controller Communicate with Vusers ?
Start the Remote Command Launcher on each host machine
Controller
Machine A
Machine B
RemoteCommandLauncher
• RCL starts the Agent and initializes the Vusers
INITIALIZE
Agent
Vusers
• Controller Runs Vusers via the Agent
RUN
CONFIDENTIAL © Copyright IBM Corporation 2006
The Scenario
An environment that describes all the conditions under which you want to test your system
virtualusers
loadconditions
hostmachines
resultslocation
testscripts
CONFIDENTIAL © Copyright IBM Corporation 2006
Setting Up a Scenario
1 - Define Hosts
2 - Connect Hosts
3 - Define Vuser scripts
4 - Add Vusers
5 - Set rendezvous policy (optional)
6 - Set up Vuser Scheduling (optional)
CONFIDENTIAL 34 © Copyright IBM Corporation 2006
Machine name orIP address
Host Status, Ready/Active
1 - Define Hosts
Results handling
information
UNIX information
for DB Vusers
Initialization Quota
Limits per host for each Vuser
type
WinRunner information
for GUI Vusers
CONFIDENTIAL 35 © Copyright IBM Corporation 2006
2 - Connect Hosts
Connect /disconnect host
Connection statusHost name
CONFIDENTIAL 36 © Copyright IBM Corporation 2006
3 - Define Vuser Scripts
A scenario typically contains multiple instances of each script
Vusers assignedto this script
Transactions and Rendezvous in script
Path to script
Vuser type depends on script type
CONFIDENTIAL 37 © Copyright IBM Corporation 2006
4 - Add Vusers
Quantity Hostname
Script name
Click the Add Vusers toolbar button
CONFIDENTIAL 38 © Copyright IBM Corporation 2006
4 - Add Vusers (cont.)
NOTE: Run-Time Settings are discussed in the VuGen course
Host machine
Script name
1. Select an existing Vuser from the grid2. Select Vuser > Details from the menu
CONFIDENTIAL 39 © Copyright IBM Corporation 2006
5 - Define Rendezvous Policy(optional)
Enable/disablerendezvous
Specify when torelease Vusers
Specify maximumtime Vusers wait
to be released
Release Vusersthat arrived
CONFIDENTIAL 40 © Copyright IBM Corporation 2006
• LoadRunner Program group – select SchedulerDouble-click to schedule a new group
Properties button to select details
Specify duration settingsSpecify Incremental Initialization PaceSpecify Incremental Run PaceSpecify Termination Pace – only available when a group duration has been set
Visual Group Settings
6 - Scheduling Vusers(optional)
CONFIDENTIAL © Copyright IBM Corporation 2006
Scenario Wizard
Use theScenario Wizard to simplifyscenario creation
CONFIDENTIAL © Copyright IBM Corporation 2006
LoadRunner Workflow
Plan Load TestingPlan Load Testing11
Create ScenarioCreate Scenario33Run ScenarioRun Scenario44Analyze ResultsAnalyze Results55
Create Vuser ScriptsCreate Vuser Scripts22
Debug and Run Scenario AgainDebug and Run Scenario Again66
• Check Remote Command Launcher
• Specify results directory
• Initialize Vusers
• Run Vusers
CONFIDENTIAL © Copyright IBM Corporation 2006
Running a Scenario
1 - Make sure Remote Command Launcher is active on all hosts
2 - Specify the directory for storing execution results- By default, the runtime files are stored locally on each host. After the scenario, the
results are collated and processed on the Controller machine.- Alternatively, instruct LoadRunner to use a shared network drive- Use Mercury’s TestDirector to store results to a project
3 - Initialize Vusers
4 - Run Vusers
CONFIDENTIAL © Copyright IBM Corporation 2006
The Vuser Statuses
Initializing Ready Running
Rendezvous
Exiting
Done
CONFIDENTIAL © Copyright IBM Corporation 2006
Tips for Smooth Scenario Execution
Make sure Remote Command Launcher is active on each host. Verify successful execution of each Vuser script before running with
LoadRunner. Make sure all scripts have the proper path and run-time settings. Disconnect from the host and refresh after editing a script, to ensure that
the new version of the script is transferred to the host.
CONFIDENTIAL © Copyright IBM Corporation 2006
Managing the ScenarioGood Practices:
Run the scenario gradually for easy problem solving:- individual Vusers- a group of Vusers- entire scenario
Initialize Vusers before running- initializing process is resource intensive- ensures that all Vusers start from same point
CONFIDENTIAL © Copyright IBM Corporation 2006
Summary
Set up a scenario1 - define Hosts
2 - connect Hosts
3 - define Vuser scripts
4 - add Vusers
5 - set rendezvous policy (optional)
6 - schedule Vusers (optional)
Run a scenario1 - check Remote Command Launcher
2 - specify results directory
3 - initialize Vusers
4 - run Vusers
ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution
SCENARIOS
DEMO
CONFIDENTIAL 49 © Copyright IBM Corporation 2006
Lab Exercise