SAP best practices for performance testing
-
Upload
hp-software-solutions -
Category
Documents
-
view
1.703 -
download
12
description
Transcript of SAP best practices for performance testing
1 ©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
SAP Best practices for performance testing
Andreas Gloege, QRM Solutions, SAP Americas
“Quality & Testing” - Biggest Pain Point of Upgrades
83%
17%
% Members
In the
Process of
upgrading
Not
upgrading
Major Cost and Time Factors during Upgrade or Implementation,
% Customers
QA & Testing
Change Mgmt.
Planning
Training & Doc.
Resource Mgmt.
Complexity
Project schedule
n=52
0%
5%
7%
12%
16%
30%
75%
Testing and coverage is the #1
stated pain point for all
customers
Over 86% of customers are
concerned about risks due
to lack of comprehensive
testing
Complexity and project
schedule are rated much
lower compared to the rest
Source: ASUG Test influence council member survey
© SAP 2007 / Page 3
Why Performance Test?
To ensure implementation complies to non-functional requirements at both
individual business processes and a system wide level
Identify and remove bottlenecks in individual
transactions
Improve codeChange hardware configurations
Modify the business process
Determine system scalability under load
Measure KPIs at reasonably expected peak volumes (100% load scenario)
at increased volumes (100% + n% load scenario)
Understand the impact of peak processing times by simulating the exact processes in a non-production
system before go-live
© SAP 2007 / Page 4
Performance Testing Objectives
Note: Proper testing methodology is required to identify *real* bottlenecks. Improper testing will lead to bottlenecks that will never occur in real world conditions
What are the Risks ?
The primary objectives of System Performance Tests are to reduce risks associated with performance issues before go-live
Data processing/transformation not able to complete in required time
User productivity is decreased due to poor system response time
User perception of the system is poor due to transactional response time
How To Reach The Objectives ?
Testing against Service Level Agreements (SLA) SLAs may be a function of a business requirement Define a strategic approach to testing based on Best
Practice and follow the plan Eliminate Bottlenecks occur in infrastructure
(hardware) and/or the application layers test approach must provide coverage in both areas
© SAP 2008 / Page 5
Project Phase Overview
Preparation In the first phase we will work in conjunction with the project managers, process teams and technical teams to perform such preparation tasks as planning, designing, and configuring the environment.
Development In the second phase test scripts and scenarios will be developed using LoadRunner following the test plans prepared in phase one.
Execution/Analysis The third phase includes running the scenarios, gathering data, analyzing system performance, identifying bottlenecks and tuning. This phase is cyclical requires multiple iterations to meet load test goals.
Reporting In the final phase reports are developed and reviewed with project stakeholders.
© SAP 2008 / Page 6
What to Test - Conceptual
What uses system resources?
Online Users
Data processing online/background
Goal:
Identify peak usage times
Simulate peak load (users and data processing)
SAP System
Users executing online
transactions
Incoming Interfaces
Outgoing Interfaces
Background Processes
(batch)
© SAP 2008 / Page 7
What to Test - Modeling
The first step in setting up the test scenario is choosing the business processes to be investigated.
Work out a 24-hour time-grid showing the planned execution time-windows and associated processing volumes for each business step involved in the test. Identify all load scenarios that may differ from this model. The test scenarios that are non-critical or already included in another scenario can be neglected.
Example:
Similar modeling required for business activities happening• Weekly, Monthly• Quarterly, Yearly
© SAP 2008 / Page 8
What To Test: Online Transactions
Core business processes as well as other processes known to be time-critical or business critical.
The processes chosen for testing online transactions are selected based on the following criteria:
Transaction CriticalCore transactions which must be run in order to establish conditions for running other processes (Example: Login, Search Invoices, Create Billing Docs).
Heavy ThroughputHigh volume with respect to concurrent users and high transactional throughput (Example: Period End Reporting).
Dynamic ContentPrograms that require high level of interaction with the database (Example: Create orders w/ multiple schedule lines, ).
Customized ComponentsRICEFW Development Objects / Z programs / User exits have higher risk of failure than standard (“out of the box”) functionality.
© SAP 2008 / Page 9
What To Test: Batch/Interface
Custom interfaces and batch jobs from each process being implemented are selected for testing based on the following criteria:
Heavy Throughput during business hours with expected heavy user load
Complex logic on SAP or middleware tier
Time sensitive batch processes. (Example: Starts at 3AM and must be complete by 6AM)
Business Critical
Is *NOT* A One Time Data Load
Interfaces can be considered as critical if they have one or more of the following requirements:
High throughput requirements (Rules of thumb)
– Maximum data volume per hour > 1,000 or
– Average data volume per day > 10,000 or
– Maximum number of documents per hour X Response time [sec] > 3600
– High data volume >10.000 data records or documents x line items per day
Requirements for good response time are (rule of thumb):
– < 2 seconds for synchronous interfaces (sender waits for response of receiver)
– < 1 Minute for asynchronous interfaces (sender does not wait)
When Is Performance Testing Required?
Test performance of your projects:
New SAP installations/ deployments
Upgrades/Updates
Application of Support Packages and Legal Change Packages
In-house developments
Consolidations
Capacity planning and sizing
SAP Application Lifecycle
Capacity
planning for
hardware
Building
performance
scripts
Load
testing Stress testing
Diagnostics
for quick
problem
resolution
Project PrepContinuous
ImprovementRealizationBlueprint
Final
Preparation
Go Live
and Support
Manual or Automated Performance Testing
What Is Automated Load Testing?
Replaces real users with thousands of “virtual” users
Generates accurate, measurable and repeatable load on the
system from a single point of control
Pinpoints bottlenecks in the system
Web Server App. Server DatabaseInternet/
WANSimulated
UsersController
Emulates production workloads on an IT system
What Tools Are Recommended by SAP ?
“The only load testing vendor to increase market share every year since 2000 …..”
Source: Yankee Group 2005 Source: Gartner Research (March 2005)
HP77%
Other 1%
Empirix
2%
Spirent
2%
Rational
9%
Other
6.7%
Compuware
3% Segue
4%
SAP LoadRunner by HP and SAP Performance Center by HPThe industry standard
Manual
Automation
Standardization
Centralization
Performance Authority
Level 0
Project based manual testing
Project-by-project
automation
Standard common testing
infrastructure standardization
Center of Excellence,
best practice processes
shared across projects and
teams
Centralized testing
services, standardized performance
lifecycle for all projects
Level 1
Level 2
Level 3
Level 4
SAP Performance Center by HP
SAP LoadRunner by HP
Performance Testing - Maturity Model
Why SAP LoadRunner by HP ?
Ease of use reduces the learning curve and script
development time
Breadth of environment support helps assure that all
enterprise applications are covered
Depth of analytical data allows fast triage of
problems and the information needed to fix them
quickly
SAP Internal Usage
Industry Standard
Market-leading capabilities mitigate
performance risk and accelerate application delivery
LoadRunner Components
© SAP 2007 / Page 15
LoadRunner ControllerRun the Load and Performance Testing Scenarios.
LoadRunner Virtual User GeneratorCreate Virtual User scripts by using recording on a specified single or multi-protocol.
LoadRunner Virtual UsersOne Virtual User script can be executed several hundreds or thousands of times in parallel to simulate hundreds or thousands of real users. While this is happening the response times for specified end-user steps will be measured for every single user.
LoadRunner Monitors and DiagnosticsWhile Load Test is running the LoadRunner Controller can use Monitors and Diagnostics capabilities to monitor the different components of the infrastructure under load. Using the Diagnostics capabilities LoadRunner provides the insight in different applications and infrastructure to identify directly where the response times was lost
LoadRunner AnalysisThe LoadRunner Analysis module is now providing the offline analysis where the response times from the virtual users, plus the monitoring information and the diagnostics details come together to enable the performance team for full offline results and route cause analysis.
How Does LoadRunner Work?
Record an application business process into an automated scripte.g. “Create New Order” in an ecommerce web application
Add end-user transactions, parameterize input data, add verification checkpoints
Apply user behaviors such as network connectivity, pacing etc…
Application
Scripts
“CREATE NEW ORDER”
VUGen Recorder
LoadRunner VuGen
SAPGUI or
Browser
How Does LoadRunner Work?
Design and execute Load Scenarios that mimic production conditions
Emulate hundreds or thousands of concurrent users from one or more locations
Gather end-to-end transaction response times
e.g. “Logon”, “Select Product”, “Process Credit Card”, and total “Create Order” process
Collect application, network and system data across all architectural tiers and components
DIAGNOSTICS
MONITORS
LOADBALANCER
WEBSERVER
APPLICATION SERVERS
DATABASENETWORKCONTROLLER LOADGENERATORS
LoadRunner Controller
VUsers
US
Europe
How Does LoadRunner Work?
© SAP 2007 / Page 18
SAP Load Runner by HP
Analyze Online &
Offline Results
Test SAP R/3
Diagnostics
Isolate/pinpoint bottlenecks in SAP R/3
Comprehensive alerts for proactive analysis
Delegate problem resolution to the right team
Performance Testing
Load test SAP GUI, SAP R/3 transactions and Enterprise Portal based Applications
Capture the results
Analyze results, fix issues & rerun tests
LoadRunner SAP Diagnostics
SAP LoadRunner by HP Summary
Industry Standard solution for application performance testing
Single platform that enables a consistent, repeatable process for:
• Performance testing critical business processes
• Testing across a wide breadth of applications and protocols
• Monitoring system and application data during tests
• Result analysis and problem isolation
Support a broad set of technologies including performance testing for web 2.0 applications
Isolate performance bottlenecks using analysis and diagnostics
Shorten performance testing cycle time and reduce total cost of ownership
As Customers Mature…
…the Next Set of Problems
Multiple projects• Scheduling
• Auditing
• Reporting
• Security
Multiple people • Global access
• 24x7 access
• Privilege management
• Asset sharing
Growing Testing Needs
Apps
Expertise
Infra
……
Localized testing teams with individual tools have to
buy more for new projects
Apps
Expertise
Infra
Apps
Expertise
Infra
Testing needs
increase for major
initiatives:
• ERP Upgrades
• Web 2.0
• SOA
• Virtualization
Need for Centralization
Team B
Team DTeam C
Applications
Expertise
Infrastructure
Team A
Improve efficiency by moving to Center of
Excellence
Central testing team with
resources to support multiple
projects
Reasons for Center of Excellence
Eliminate inefficienciesCan’t utilize or reach available resources to maximize application testing
Drive best practicesLack processes required to reach performance goals
Test more applications more completelyApplication complexity and number quickly overtake available resources
Minimize IT riskPerformance varies greatly from project to project and from LOB to LOB
Forces driving centralization
SAP Performance Center by HP
Summary
SAP Performance Center by HP is the de-facto standard for enterprise level performance validation
24x7, global access
Pooled licenses and vusers
Shared and reusable testing assets
Central and standard testing
Schedule / Auto Start tests
IncreasedProductivity
• Standardized best practices (less duplication)
• Increased expertise due to knowledge sharing
• 24x7 concurrent usage model (follow the sun)
Why Moving from SAP LoadRunner by HP to
SAP Performance Center by HP
Increased Efficiency, Better Application Quality
IncreasedUtilization
• Load testing available to all projects
• Visibility promotes ubiquitous load testing
• More flexible access to assets promotes use
ReducedCosts
• Lower acquisition costs (pooled model)
• Lower maintenance costs (central infrastructure)
• Lower staff costs (training)
© SAP 2010 / Page 26
ASAP Methodology v7.0
Defines How to Use SAP LoadRunner by HP and SAP Performance Center by HP
Contact Information
Andreas Gloege
QRM Solutions
SAP America
+1-650-690-2167
Thank
you!
Q&A
29 ©2010 Hewlett-Packard Development Company, L.P.
To learn more on this topic, and to connect with your peers after
the conference, visit the HP Software Solutions Community:
www.hp.com/go/swcommunity
30