Monitoring for SAP BI Reporting

Best Practice for Solution Management

Version Date: July 2008

SAP NetWeaver 7.0 BIThe newest version of this Best Practice can always be obtained through

the SAP Solution Manager or the SAP Service Marketplace.

Contents

Applicability, Goals, and Requirements................................................................................................2

Best Practice Procedure and Verification.............................................................................................4

Preliminary Tasks..........................................................................................................................4

Procedure .....................................................................................................................................4

Monitor and Tuning Query Performance ..................................................................................4

BI Statistics .............................................................................................................................4

BI Query Monitor RSRT...........................................................................................................6

BI Workload Monitor ST03 ......................................................................................................7

BI Administration Cockpit.........................................................................................................9

Monitoring the BI Accelerator................................................................................................. 11

Prefilling Cache..................................................................................................................... 11

Create Aggregates ................................................................................................................12

Delete Aggregates.................................................................................................................13

Delete BI Statistic Tables.......................................................................................................13

Further Information ...........................................................................................................................15

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

2

Applicability, Goals, and RequirementsTo ensure that this Best Practice is the one you need, consider the following goals and requirements.

Goal of Using this ServiceThis service provides recommendations about jobs and tasks that have to be performed on a regularbasis to ensure stable reporting performance in a BI system.

Alternative PracticesYou can get SAP experts to deliver this Best Practice if you order the “SAP Business ProcessPerformance Optimization for BI Reporting” Service. For further details concerning this service, pleaserefer to SAP Service Marketplace http://service.sap.com/bppo . The BPPO service can also bedelivered as empowering service where experienced support consultants will transfer knowledge toyour operations team. The service is called “SAP REPORTING PERFORMANCE WORKSHOP FORSAP NETWEAVER BUSINESS INTELLIGENCE (BI)". Please contact your SAP accountrepresentative to get more information from the following WikiPagehttps://wiki.wdf.sap.corp/display/K3S/SAP+Empowering+Service

Staff and Skills RequirementsFor optimal benefit you should make sure that you have experience in the following areas:

The SAP Solution Manager 7.0 must be installed

Practical experience with the SAP BI Reporting and Data Warehousing Workbench

Root Cause Analysis must be configured in the SAP Solution Manager. This can be done bythe SAP Solution Manager Starter Package.

Experience with monitoring of SAP BI queries

System RequirementsThis document refers to SAP NetWeaver 7.0 BI systems. Please be aware that the screenshotsprovided in this document refer to SAP NetWeaver 7.0 BI; some of them might differ slightly in the“look & feel” from SAP BW 3.x but the menu path is the same for both systems.

In order for this document to be of relevance and benefit to all BI administrators, it will not coverfeatures or restrictions that belong to any particular database or operating system.

Duration and TimingThe monitoring tasks in BI 7.0 have to be done on a regular basis. The time required depends on theobjects used and the complexity of the reporting process.

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

3

Changed TerminologyAs of SAP NetWeaver 7.0, some existing BI terms have been changed to better reflect the enhancedfunctions of the tools and objects.

Old term New term

ODS object DataStore object

Transactional ODS object DataStore object for direct update

Transactional InfoCube Real-time InfoCube

RemoteCube,SAP RemoteCube,virtual InfoCube with services

VirtualProvider

Administrator Workbench Data Warehousing Workbench

Monitor Extraction monitor

Reporting authorizations Analysis authorizations

OLAP Engine Analytic Engine

OLAP statistics BI query runtime statistics

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

4

Best Practice Procedure and Verification

Preliminary TasksIt is recommended that this Best Practice be first read completely before any given recommendationsare implemented. We strongly advise against direct changes in the production environment, andrecommend first testing any changes in a test system.

Procedure

Monitor and Tuning Query PerformanceFrequency: Weekly

To improve end-user satisfaction, SLRs (service level requirements) should be defined for all newqueries and a pro-active query performance monitoring should be set up. It should not be the standardprocedure that long running queries are only identified by the end users in the functional departmentand reported to the technical department.

Classify and prioritize the queries. Based on the outcome of this categorization, ensure thatappropriate monitoring and tuning procedures are in place to guarantee the maximum performanceand availability of these queries. This should include:

Having a minimal system load when the query runs or when the associated data is uploaded.

Having the responsible people on call during a critical load and ensuring that the correctprocedures are in place to contact that person.

SLRs should be defined by the functional department for all new queries. The technical departmentshould check if it is possible to meet these requirements by testing the queries in the developmentsystem, therefore it is necessary to have a comparable data volume in the development andproduction systems respectively. Queries with a high runtime on the database can be tuned byaggregates or in SAP NetWeaver 7.0 BI by using BI Accelerator. If aggregates are not suitable or donot improve the performance of these queries to the level required, consider if queries could be tunedby pre-filling the OLAP cache with Information Broadcasting. Note that usually only a restrictednumber of queries can be pre-calculated, therefore a prioritization is necessary and the performanceof these important queries should be monitored.

As the amount of data being read by the query and the reporting behaviour of the end-users typicallychanges over the time, it is important to also monitor the query performance regularly, e. g. weekly orwhen a high volume of data is loaded to the InfoProvider. In SAP NetWeaver 7.0 BI you can use theBI Administration Cockpit to monitor query performance; in SAP BW 3.x you can use the TechnicalContent and/or transaction ST03 (the workload monitor). A prerequisite for ST03 and theAdministration Cockpit is that the technical content needs to be activated. Please find a detaileddescription in SAP Note 934848.

BI StatisticsThere are three main processing areas during query execution; front-end, OLAP processor, and dataselection (which includes Database and BI Accelerator). When analyzing query performance, first

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

5

identify where the performance problem is located and focus on the tuning possibilities for thatprocessing area.

Statistics identification: The two central tables for analyzing query performance areRSDDSTAT_OLAP and RSDDSTAT_DM. The first one is used to find out where the time is spent.You can distinguish between data selection, OLAP or front-end. The second table provides moredetails about the data selection. With BI 7.0 it is now possible to check which BasisProviders andaggregates are involved in a MultiProvider query.

Events are the basis for end-to-end runtime monitoring of processing in SAP NetWeaver BI 7.0.Events can have two types

calculated by time (start time / end time)

calculated by a counter (e.g. number of records selected / transferred...etc)

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

6

The detail explanation for the events is available in the online help:http://help.sap.com/saphelp_nw70/helpdata/en/45/f0488a1aa03115e10000000a1553f7/frameset.htm

Transaction RSDDSTAT: This transaction is used to switch on statistical recording for single objectslike queries, web templates, workbooks or whole InfoProviders. Only when statistics are switched onruntime information in table RSDDSTAT* will be recorded. The level of detail can be influenced by aparameter. It is possible to define a default value for all objects.

You can also find the find Transaction RSDDSTAT from RSA1 via menu navigation -> Tools ->Settings for BI statistics:

For further information, please check the following link:

http://help.sap.com/saphelp_nw70/helpdata/en/43/e37f8a6df402d3e10000000a1553f7/frameset.htm

BI Query Monitor RSRTRSRT is the most important tool for query analysis. It should be used to investigate individual queriesand their executions/navigations in detail and to check whether query executions spend their time onthe database, in the OLAP processor or in the front-end. RSRT can be used to get a general technicalimpression of a query definition and the usage of performance relevant features.

The important features in RSRT are:

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

7

Execute + Debug: This is used among others to do query debugging, set predefined break-points,show statistical entries for query execution, get aggregate proposals, and to prevent cache or BIAccelerator access.

Properties: Here query specific properties like read mode and cache mode can be defined. There isnow a new Delta Cache Mode available. For more details please follow this link:

http://help.sap.com/saphelp_nw70/helpdata/EN/57/b10022e849774f9961aa179e8763b6/frameset.htm

Technical Information: From the Technical Information screen, you get an overview of importanttechnical reference numbers for a query. Please find details under following link:

http://help.sap.com/saphelp_nw2004s/helpdata/en/f2/c9bae61cca904284aebbecc84ed01b/frameset.htm

BI Workload Monitor ST03You should use the Workload Analysis to proactively monitor the performance of queries andaggregate maintenance.

Identify and analyze InfoProviders and queries that have performance tuning potential

Identify InfoProviders and aggregates that consume most of the aggregate maintenance timewindow

Identify the focus area for performance improvement i.e. database, OLAP, front-end

The BI Content for statistics has to be activated before using transaction ST03, as the information withST03 comes via the DataSources: 0TCT_DS01 (OLAP aggregated data), 0TCT_DS02 (OLAP detaileddata) and 0TCT_DS03 (DataManager detailed data).The data will be loaded to the InfoCubes 0TCT_C01 (Front-End and OLAP Statistics (Aggregated))and 0TCT_C02 (Front-End and OLAP Statistics (Details)) from the Technical Content. Please checkthe SAP Note 934848 and online help for more details:http://help.sap.com/saphelp_nw70/helpdata/en/80/1a66d5e07211d2acb80000e829fbfe/frameset.htm

For current analysis, for example of today, the information will be read from the tables directly as thedata is not in the InfoProviders yet.

Enter the workload monitor with transaction ST03 and switch to “Expert Mode”. Under the “BIWorkload” tree you can specify the timeframe for you analysis. In the “Analysis View” tree you select

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

8

the different areas where your analysis will focus on.

For BI performance monitoring and analysis, we can focus on 4 areas:

1. ReportingTo analyze the performance for Queries, Workbooks and Web Templates

2. AggregatesInvestigate the performance for initial filling, rollup or delta update of aggregates

3. Load DataAnalyze the performance for data loading during the extraction and DTP

4. Process ChainsAnalyze the performance of process chain

Workload Overview:

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

9

Query Runtime overview on tab “Share of runtime”:

If you want to learn more about transaction ST03 and the general BI Reporting Monitoring werecommend the Empowering Service “SAP REPORTING PERFORMANCE WORKSHOP FOR SAPNETWEAVER BUSINESS INTELLIGENCE (BI)".https://wiki.wdf.sap.corp/display/K3S/SAP+Empowering+Service

BI Administration CockpitThe BI Administration Cockpit is a central point of entry for monitoring complex BI landscapesproviding information about data warehousing, reporting and query analysis and planning services. Itfocuses solely on monitoring the ABAP Stack. Java Stack monitoring like heap memory consumptionon the Java server nodes is done with Solution Manager Diagnostics (SMD).

Admin Cockpit Overview:

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

10

Admin Cockpit Architecture:

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

11

Monitoring the BI AcceleratorThe central point of access to the monitoring and administration functions for the BI Accelerator is theBI Accelerator monitor (transaction RSDDBIAMON2). This transaction provides an overview of thecurrent status of the BI Accelerator and offers a detailed, technical overview of the hardware, BIAccelerator services, any trace files that exist, and the BI Accelerator indexes.

For more information about the BIA technology and usage please note this linkhttps://www.sdn.sap.com/irj/sdn/bia

Prefilling CacheFrequency: Daily

By using the OLAP cache, the load on the database server is reduced. Cached queries, especiallythose that are cached in main memory and in flat files, do not cause any database access (at least thedatabase accesses are reduced significantly). Because of this reduction, there are fewerdisplacements in the database buffer and therefore the cache has a positive impact on the overalldatabase performance and the network load. With query execution, the runtime for reading the data isreduced by the cache especially for queries reading a huge data volume.

To meet the business requirements in reporting performance, pre-calculation of some importantqueries could be necessary. This means you can "warm up" the OLAP cache for these queries directlyinstead of it being filled "indirectly" by the reporting in question. Then the result can be read out of theOLAP cache and this is much faster than from the database.

This function is not explicitly provided with BW 3.x. However, you can use the method that is describedin "How to ...Performance Tuning with the OLAP Cache" with certain restrictions.

In SAP NetWeaver 7.0 BI, you can use the BEx Broadcaster to fill the OLAP cache. Call thebroadcaster from the BEx Query Designer:

Choose “Create New Setting” Choose “Fill OLAP Cache”. Select the filter navigation. When thequery uses variables, also a variant has to be maintained; otherwise the variable could not be filled inthe background job.

Save and schedule the job: You can either schedule the pre-filling of the OLAP Cache to run atpredefined times or with each data change in the InfoProvider.

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

12

When the pre-filling of the OLAP cache is scheduled to run with each data change in the InfoProvider,an event has to be raised in the process chain which loads the data to this InfoProvider. When theprocess chain executes the process “Trigger Event Data Change (for Broadcaster)”, an event is raisedto inform the Broadcaster that the query can be filled in the OLAP cache.

Create AggregatesFrequency: When the service level requirements of the queries are not met, when youencounter increasing I/O or bad data hit ratio on the database server.

When you are using a SAP NetWeaver 7.0 BI system, consider the usage of the new BI Acceleratorinstead of aggregates. The BI Accelerator does not only speed up queries which have a high databasetime, it also helps avoiding the ongoing and time-consuming finding and creation of aggregates.Further information about the BI Accelerator is available on the SAP Service Marketplace:http://service.sap.com/bifaq BI Accelerator.

If the query runtime is mainly spent on the database, suitable aggregates should be created. The ratiobetween rows selected and rows transferred indicates potential performance improvements withaggregates. It should be investigated which queries are performing badly; and then, which of them canbe tuned by aggregates.

Define responsibilities for aggregate creation in order to ensure that the queries are performing well.

Further information is available on the SAP Service Marketplace:

http://service.sap.com/bi Performance Aggregates

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

13

We also recommend ordering the TEWA50 “Empowering Workshop: SAP BW - Query Tuning withAggregates. “ This workshop contains information about:

Concept of aggregates

Find queries that can be tuned with aggregates

Create suitable aggregates

Maintain aggregates efficiently

Delete AggregatesFrequency: Quarterly

There are several reasons, why an aggregate might be unnecessary:

There can be very similar aggregates that might be combined into one new aggregate.

There can be aggregates which are never used and which are no basis aggregates.

There can be aggregates with an insufficient reduction factor compared to the InfoCube.

As aggregates contain redundant data and are especially created for performance reasons,unnecessary aggregates need disk space and have to be regularly maintained via roll-up and/orchange run.

Check regularly the usage of your aggregates. It can be the case that the merging of similaraggregates is possible. Some aggregates may not be used at all, or some aggregates could be similarin size to the InfoCubes and lead to no performance improvement.

The EarlyWatch Report can also be checked for aggregates to be deleted.

Delete BI Statistic TablesFrequency: Quarterly

The BI statistic tables RSDDSTAT* and the BPS statistic tables UPC_STATISTIC* have to be deletedregularly.

Please follow the SAP Notes 211940, 195157, 179046, 366869 before deleting or archiving.

For the BI statistic tables RSDDSTAT*, the deletion of records older than x days can be done intransaction RSA1. To delete data, call transaction RSA1, choose 'Tools' 'Settings for BI Statistics’and select 'Delete Statistical Data':

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

14

The time period for data, which should be deleted, can now be entered. Please read SAP Note309955 for information on usage and errors in the BI Statistics.

For the BPS statistic tables UPC_STATISTIC*, the deletion of records older than x days can be donein transaction BPS_STAT0:

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

15

Further Information

Alternative PracticesYou can get SAP experts to deliver this Best Practice if you order the SAP Business ProcessPerformance Optimization for BI Reporting. For further details concerning this service, please refer toSAP Service Marketplace http://service.sap.com/bppo . The BPPO service can also be delivered asempowering service where experienced support consultants will transfer knowledge to your operationsteam. Please contact your SAP account representative to get more information from the following WikiPage https://wiki.wdf.sap.corp/display/K3S/SAP+Empowering+Service

Best Practice: Monitoring for SAP BI Reporting

© 2008 SAP AG

16

Feedback and QuestionsSend any feedback by formulating an SAP customer message to component SV-SMG athttp://service.sap.com/message in the SAP Service Marketplace.

© Copyright 2008 SAP AG. All rights reserved.No part of this publication may be reproduced or transmitted in any form or for any purpose without the expresspermission of SAP AG. The information contained herein may be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components ofother software vendors.Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks ofMicrosoft Corporation.IBM®, DB2®, OS/2®, DB2/6000®, Parallel Sysplex®, MVS/ESA®, RS/6000®, AIX®, S/390®, AS/400®, OS/390®, andOS/400® are registered trademarks of IBM Corporation.ORACLE® is a registered trademark of ORACLE Corporation.INFORMIX®-OnLine for SAP and Informix® Dynamic Server

TM are registered trademarks of Informix Software

Incorporated.UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group.HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium,Massachusetts Institute of Technology.JAVA® is a registered trademark of Sun Microsystems, Inc. JAVASCRIPT® is a registered trademark of SunMicrosystems, Inc., used under license for technology invented and implemented by Netscape.SAP, SAP Logo, R/2, RIVA, R/3, ABAP, SAP ArchiveLink, SAP Business Workflow, WebFlow, SAP EarlyWatch,BAPI, SAPPHIRE, Management Cockpit, mySAP.com Logo and mySAP.com are trademarks or registeredtrademarks of SAP AG in Germany and in several other countries all over the world. All other products mentionedare trademarks or registered trademarks of their respective companies.Disclaimer: SAP AG assumes no responsibility for errors or omissions in these materials. These materials areprovided “as is” without a warranty of any kind, either express or implied, including but not limited to, the impliedwarranties of merchantability, fitness for a particular purpose, or non-infringement.SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, orconsequential damages that may result from the use of these materials. SAP does not warrant the accuracy orcompleteness of the information, text, graphics, links or other items contained within these materials. SAP has nocontrol over the information that you may access through the use of hot links contained in these materials anddoes not endorse your use of third party Web pages nor provide any warranty whatsoever relating to third partyWeb pages.