017 CRM Initial Download Configuration

5/20/2018 017 CRM Initial Download Configuration

1/32

Best Practice: mySAP CRM Initial download configuration

2004 SAP AG

1

mySAP CRM Initial Download Configuration

Best Practice for Solution Management

Version Date: November 2004

The newest version of this Best Practice can always be obtained through the SAP SolutionManager or the SAP Service Marketplace.

Contents

Applicability, Goals, and Requirements........................................................................................................2

Best Practice Procedure and Verification.....................................................................................................2

Preliminary Tasks......................................................................................................................................2

Procedure..................................................................................................................................................3

Optimizing the Initial load from an R/3 based OLTP system to CRM online ........................................4

Initial load from a non-R/3 based OLTP............................................................................................. 22

Optimizing the Initial load from CRM online to CDB .......................................................................... 26

Initial download of objects after the primary initial download ............................................................. 30

Further Information .................................................................................................................................... 31


2/32


2004 SAP AG

2

Applicability, Goals, and RequirementsTo ensure that this Best Practice is the one you need, consider the following goals and requirements.

Goal of Using this Best Practice

The goal of this Best Practice is to document and recommend procedures:

For the configuration of the initial download of business data in a way that the planned volumecan be downloaded in the available time frame. The procedures focus especially on how tohandle the download of huge data volumes.

To ensure the optimum performance for the initial download of business data

To avoid unnecessary business data being downloaded

For monitoring the initial download and taking corrective actions in case of errors andperformance loss

How to handle errors occurring during the initial download of business data Minimize performance bottlenecks

Staff and Skills Requirements

System administratorswith training for the mySAP CRM middleware

Application team members to define which business data has to be exchanged

System Requirements

MySAP CRM solution landscape including CRM Release 4.0and either an R/3 based OLTPor a non-SAP based OLTPsystem.

How to Use this Best Practice

To apply this best practice, follow the steps in the document, and implement and test the applicablerecommendations.

Best Practice Procedure and Verification

Preliminary Tasks

Before performing this Best Practice, ensure that you perform the following preliminary tasks or checks inthe system:

Complete all installation and post-installation actions and procedures including customizing.

Apply all SAP recommendations from SAP Service Sessions and any SAP recommendationsresulting from customer problem messages.

Implement all current SAP Support Packages upon availability.

If an R/3 based OLTP system is connected to the CRM, implement the highest available Plug-In andpatch on the R/3 server.


3/32


2004 SAP AG

3

Procedure

Communication OLTP system CRM Middleware

The communication between the OLTP system and the CRM middleware is necessary to provide bothsystems with the necessary data to run the different business scenarios. For the CRM Solution this maybe mainly selling goods, for the OLTP system this may be mainly producing and distributing goods.

With CRM 4.0 it is possible to perform this data exchange between the CRM system and an R/3basedOLTP system or between the CRM system and a non-R/3 based OLTP system using the ExternalInterface Adapter (XIF).

Two different methods of exchange exist to handle the data transfer between CRM and OLTP systems:the one-time initial load before GoLive and the permanent exchange of data during the operation of theCRM solution.

The XIF can be used both for the initial load as well as for the permanent exchange of data. In fact, theexchange of Business Partners, Business Partner Hierarchies, Business Partner Relationships,Products, Price Conditions, and One-Order objects such as orders and contracts is supported (see CRM

table CRMXIF_BDOCIF and SAP note 561321 for details).

Where there is a mobile scenario or groupware connection via sBDocs, a data transfer between CRMonline and the consolidated database (CDB) of the CRM system is needed. For mobile sales the data isthen distributed to the mobile clients.

If a BW connection exists, data exchange between CRM online and the BW is also carried out.

The communication between CRM, OLTP system and the CDB supports three different types of dataexchanges: The Initial load to be performed once before GoLive, the Delta down and upload to beperformed after GoLive and the Synchronization download:

Initial load:The initial load from an R/3 based OLTP systemis used to download all business,customizing and condition data and insert it into CRM online with a deactivated mobile bridge. Theinitial load froma non-R/3 based OLTP system is used to download a part of the business andcondition data and insert it into both CRM online with a deactivated mobile bridge. With a mobilesales scenario the mobile bridge has to be activated after this first step and the data have than to bedownloaded from the CRM onlineto the CDB. The Initial load from a non R/3 OLTP can be runeither instead of the initial load from an R/3 based OLTP system or to download additional data, e.g.business partner data only. The next step is then the initial distribution to the mobile clients. If a BWconnection is used the initial distribution to BW should be carried out after the initial download toCRM online and CDB.

Delta down and upload: The delta down and upload is used to keep the business and master datain CRM, the OLTP system, and CDB in sync during normal operation. Therefore all new andchanged data relevant for the connected systems have to be exchanged. Download means that datais transferred from the OLTP to the CRM system and then via the mobile bridge to CDB for MobileSales; upload means transferring data from the CRM to the OLTP system. The Delta down- andupload procedure can be either setup between a CRM and an R/3 based OLTP systemor betweena CRM and a non-R/3 based OLTP system using the XIF. When using the Adapter technology forthe data exchange, only a limited range of business objects is supported.

Synchronization download: This type of download is used for customizing data and is only possiblebetween CRM and an R/3 based OLTP sys tem. In contrast to the delta download for businessobjects, customizing data is not downloaded automatically. Instead, the synchronization downloadhas to be started manually or should be scheduled for fixed time intervals (e.g. weekly or monthly) ina productive system. The queue name for these queues starts with R3AI* and can be monitored inthe monitor for initial download. With Requestselected data can be downloaded from the R/3backend to the CRM database (or vice versa) or from the CRM database to the CDB or externalsystems

.


4/32


2004 SAP AG

4

Main Focus of this Best Practice

The initial load is the most critical and challenging data transfer procedure between the OLTP systemand the CRM Middleware, especially if a large amount of data has to be transferred to the CRM

Middleware in a restricted time frame. For mobile sales the initial load from the CRM online to the CDB isalso needed. Therefore, we will focus on the initial load in this best practice document. The performanceoptimization of the delta down and upload and the Synchronization download are not currently covered inthis document.

The initial load itself contains 3 different download procedures:

Initial load of customizing data (only from an R/3 based OLTP system)

Initial download of condition data such as pricing conditions

Initial download of business data such as material, customers, sales orders

The Initial load of business data is usually the most critical part due to the amount of data that has to betransferred.

For mobile sales the following steps should be followed for the first initial download:

Download from OLTP system to CRM online Download from CRM online to CDB

Distribution of data to the mobile clients.

As the procedures for the initial load from an R/3 based OLTP systemand from a non-R/3 based OLTPsystemto CRM online are different, this best practice document is split in two major sections for thedownload to CRM online. For mobile sales we have a third section for initial download from CRM onlineto CDB.The distribution to the mobile clients and the initial load to BW is not part of this document.

In the first section of this best practice we discuss procedures for optimizing the initial load from an R/3based OLTP system to CRM online, e.g. by reducing the data volume to be transferred to the CRMsystem and by setting up parallel processing.

In the second section we give a brief overview of the procedures for optimizing the initial load from a non-R/3 based OLTP system using the External Interface Adapter (XIF)

The third section discusses a procedure for optimizing the initial load from CRM online to CDB.

Optimizing the Initial load from an R/3 basedOLTP system to CRM onlineTo optimize the performance of the initial load from an R/3 based OLTP system to CRM online severalactions have to be considered. Reducing the amount of data to be transferred can shorten the runtime.This is described in the next paragraph. Additional technical tuning is described in the followingparagraph. The major performance gain in the initial load can be achieved by using parallel processing.The procedure how to setup and test parallel processing is described in detail at the end of this chapter

Reducing Data Volume during the Init ial DownloadOne major factor impacting on the runtime of the initial download is the data volume that has to bedownloaded to CRM online. Filter settings can be used to reduce the downloaded data volume. Filtercriteria are easy to implement, as no programming is required. The procedure for applying filters andsome recommendations regarding filter settings are given in this chapter.


5/32


2004 SAP AG

5

Using filter Settings for the Initial Download

The transaction you use to specify the filter criteria for the various business objects can be found under:

CRM 4.0: Middleware >>Data Exchange >> Object Management >>

Business Objects (Transaction R3AC1 )

Customizing Objects (Transaction R3AC3 )

Conditi on Objects (Transaction R3AC5 )

Filter criteria can be defined for business objects, customizing objects and condition objects. Thebusiness objects usually have the highest data volume to be downloaded.

Some general remarks regarding filter settings:

Filter options allow the filtering of business objects at the source, at the target or at both thesource and the target for business objects. However, business data are usually filtered at thesource. Customizing or condition objects can be filtered at the source only.

Filter settings are stored in the table SMOFFILTAB on CRM site, CRMFILTAB on OLTP site and

refer to table fields.

Filters for the initial download are also used for the delta download

If you use more than one filter entry per object, filters to the same table field are linked with anOR, e.g. the filter condition VKORG = 0001, VKORG = 0002 results in a set that contains eitherthe first or the second sales organization.

Filters to different table fields are linked by AND, e.g. the filter condition VKORG = 0001, VTWEG= 01 results in objects that fulfill both conditions at the same time.

Filter conditions are only stored locally and are not contained in the transport of adapter objects.In this way filters are not transported from the development system to the production system.New filter settings have to be defined in each system.

Here are some typical examples illustrating how filter settings are used to reduce the data volume:

Activation

flag

object specific

filters


6/32


2004 SAP AG

6

Filters for Material Master

Filter out material that is not sold directly, using your CRM solution. For example, if you areproducing pumps, you normally only sell the finished products and some spare parts but not the

raw material and all components. In this case, use a filter on the field MARA-MTART (materialtype) for the download object MATERIAL in order to only download this material type from theOLTP.

Filter out configurable materials if you do not need them in your CRM solution. To do this, set afilter on field MARA-SATNR.

See also SAP note 526980 for details about the functionality of filtering Material master.

To accelerate the initial download of objects, such as the material master, see SAP Note 350176(CRM/EBP: Performance improvement during exchange of data).

The SAP Note 418886 (Download: Products not found or not changeable) explains some errormessages which occur during a further download of materials after the materials or services havebeen replicated to the CRM/EBP System successfully.

Filters for Customer Master

Filter out customers that are not required for your CRM application. For example, if you are only selling tocustomers assigned to the distribution channel "direct selling", create a filter to download only thesecustomers. To do this, create a filter on the field MVKE-VTWEG (distribution channel) for the downloadobject CUSTOMER_MAIN.

Filter conditions defined for CUSTOMER_MAIN are automatically taken into account during thedownload of object CUSTOMER_REL.

Filters for Sales Documents

Filter out old sales documents that are not relevant for your CRM solution. For example, you decidethat for running your CRM solution it is only necessary to have the sales documents of the currentbusiness year which started for your company on 01.01.2003. In this case create a filter on the field

VBAK-ERDAT (creation date) for the download object SALESDOCUMENT with operator GT. Enterthe value 20030101 (date format YYYYMMDD has to be used) in the field Low. In this case onlysales documents created from the 1

stof January 2003 onwards will be downloaded to the CRM

server. For more details regarding filter setting for sales documents see SAP Note 213733

Filter out sales documents for customers to whom you are not selling in your CRM solution. Carrythis out by using a general filter.

Note: Be aware that changing filter settings on an already productive system is critical. E.g. if you arefurther restricting filter criteria after the initial load this may lead to data inconsistencies between the CRMand the OLTP system.

Filters for keys

Please keep in mind that you have to use the keys in the filters as they are defined in the data dictionarytable in R/3. For example for customer, which has 10 digits, see OLTP table KNA1 field KUNNR. This

means you have to fill the filter entries with leading zeros. For material, which has 18 digits, see OLTPtable MARA field MATNR.

Performance of the initial downloadIn this Section you will find the most important recommendations for achieving good performance duringthe initial download. Read all sub-sections carefully and verify that the performance recommendationscan be applied to your CRM solution. In the Appendix you find a list of further SAPNet notes regardingperformance optimization. The main Note for performance issues regarding the initial download is SAPNote 350176


7/32


2004 SAP AG

7

Initial download after a test of the initial download

After a test of the initial download the application tables are already filled with business data. Should youtest the initial download for a second time within this system you will, for example, have updates on the

tables instead of inserts as during the first download. Please delete the data before you make a secondtest of the initial download or use a backup which was made before the first initial download.

Setting the middleware trace level parameters

In order to log CRM application errors, the middleware trace must be switched on during the initial load.

As only error information is required, the middleware trace level can be reduced to level "1" for all traceenvironments of the Middleware trace except for 'G' (generation). For this purpose, choose the followingpath in the SAP Menu 'Architecture and Technology -> Middleware -> Monitoring -> Message Flow ->Set up Middleware Trace' and choose 'Set default settings'.

Recommendation:Delete outdated trace and log information, schedule report SMO6_REORG on aregular basis as described in SAP Note 206439. Please take into account that the standard settingsretain the data from the last 7 days. Therefore, for the initial download it makes sense to reduce thedays to hold to one. To avoid problems with error analysis, regularly monitor the BDoc errors.

Delete trace data and BDOC message administration tables before and after theinitial download has successfully completed

During an initial download, large data sets may be written in the middleware trace and the BDocmessage store.

Recommendation:Delete the data before or after the initial download has successfully completed,schedule report SMO6_REORG for the reorganization of the log tables as described in SAP Note206439. The reorganization job deletes data in the tables. Please take into account that the standardvariant holds the data from the last 7 days. Therefore, in the case of the initial download it makes senseto reduce the value days to hold to one after you have checked that the initial download has workedcorrectly. To avoid problems with error analysis regularly monitor the BDoc errors.

For Oracle the indexes should be rebuilt as described in SAP note 435125.

Reorganize BDOC message related object links before and after the initialdownload completed successfully

During the initial download many entries may be written to the tables SMW0REL and SRRELROLES dueto BDOC message related object links. If you have already carried out a test download on the system,these tables may already be full.

Recommendation: Reorganize the data before or after the initial download has completed successfully,by scheduling the report RSRLDREL as described in SAP note 493156. In this special case of the initialdownload you should set the end date in such a way that as many links are reorganized as possible.

CRM Server Performance: RFC Queue processing

The qRFC is part of mySAP Basis Technology. qRFC releases are independent of the CRM releasecycle and ongoing performance improvements are realized in the different versions. Prior to BasisRelease 6.20 the qRFC version is an extra transport. For Basis Release 6.20 or higher, you can only

import the current qRFC version via a Support Package.Recommendation:Always use the latest qRFC version. Always update qRFC on the CRM server ANDon the connected OLTP R/3 system.

Implementation: To obtain the newest qRFC release, see SAP Note 438015.

Note:After the implementation of a new basis support package, verify if the qRFC version is still thenewest. Some basis support packages may contain older qRFC versions.

See the Appendix for a list of the actual SAP Notes containing program enhancements related to qRFC.

Recommendation: For a general overview on queuing in CRM and R/3 and information on queuenames used and their respective functions in the CRM server and R3 backend, see SAP Note 765236


8/32


2004 SAP AG

8

Data transfer using XML

The XML interface consists of an XML generating program on the OLTP Plug-In and an XML parser onthe CRM. The generating program generates an XML document from the data stream to be sent to the

CRM system, where the XML parser converts the data back to the original data stream.

In a heterogeneous system landscape (for example, if servers with different CPU architecture areinvolved), data inconsistencies during data transfer can occur caused by the Endian problem. The datatransfer in XML format solves both the Endian problem and the code page problem which may occur in adata transport.

With regard to the performance: the XML format has the performance disadvantage that the XML parserneeds CPU time. In some special cases it may be possible to deactivate the XML Conversion during theinitial download to speed up the process. With CRM 4.0 and the applicable PI on the R/3 side the optionSEND_XML = M is possible. The option SEND_XML = M (=mixed) in the table CRMRFCPAR specifiesthat only parts of the BAPI container used for the data exchange between the SAP R/3 System and theCRM Server are sent using XML. Character-type fields are transferred to subsequent systems with highperformance. For non Character-type fields, an XML conversion occurs in spite of the 'M' entry which hasa negative impact on the performance.

Recommendation: To avoid performance problems the option SEND_XML = M should be used.Information on the availability of this option can be found in SAP Note 442277. If the performance is notsufficient with this option you can check if the XML Converter can be deactivated. For details see SAPnote 333008. As rule of thumb the SEND_XML = ` is 6% faster than SEND_XML = M.

Note:Please be aware that deactivating the XML Converter may lead to data inconsistencies if thepreconditions are not met. If you change your hardware please again check if the conditions are met inorder to avoid problems after the hardware change.

Avoid ini tial download to CDB and BW

To take advantage of the parallel initial download and the filtering, the data should not be directly loadedto the CDB in the initial load from R/3 to CRM online.

Recommendation: The mobile bridge needs to be switched off toavoid the load to the CDB. With CRM

4.0 it is possible to switch it off for all objects in transaction CMWC_SMW. If the flag Data Sync Activeis not set the mobile bridge is deactivated. In addition the creation of the CSA* queues for these objectsshould be deactivated. With CRM 4.0 this is possible in transaction SMW3_00. The deactivation needs tobe carried out per BDoc type. For details see also SAP note 635697. This deactivation of the CSA*queues also switches off the delta load to BW for objects which are transferred to the BW via m flow. Ingeneral you should first make the initial load to the CRM server and then the initial load to the BW. If youneed the delta load to BW or other systems you have to reactivate the CSA* queues again after the initialdownload.

Download of currencies

The initial download of currency data may terminate with a time-out. Currency data is contained in thedownload objects DNL_CUST_CURREN and DNL_CUST_BASIS3. The reason for the timeout is thatthe table TCURR containing the exchange rates has too many records.

Recommendation: Set the block size (that is, the number of data records per BAPI call) to 1. Filter outexchange rates: Apply the filter to the field GDATU of the table TCURR.

Test runs and t ime frame for the initial downloadTo obtain an idea of how long the initial download will run (this can last several days, or even up to eventwo weeks) testing is mandatory as runtime can be critical for the project time plan. For a high volume ofdata, test phases of one or several weeks should be considered.

Procedure: Set up separate test runs for each download object for the initial download. For downloadobjects with a high volume (at least several 100 000), a test period of at least one week may benecessary.

Be aware that the initial download cannot be run in " test mode." This means that during each testrun o f the in itial download, the data really are inserted into the CRM database. There is also notool available for "deleting" the downloaded data from the CRM system after a test run. This


9/32


2004 SAP AG

9

means that you either have to perform test runs with d ifferent filter settings for the samebusiness object or to delete the data manually before performing another test run. An easiersolution is to make a backup before the initial download test and restore this backup before the

next test.After the initial download is started, entries in table TBE31 on the OLTP system are created in order totrigger the download of delta information to synchronize the information on the OLTP and the CRMsystem.

If you are performing test runs using an already productive OLTP system, please ensure that the evententries are deleted after the test initial download is finished. Therefore use transaction R3AC4 and setthe flag "inactive" for the corresponding download object. Do not forget to remove the inactive flag for thecorresponding objects immediately before you start the productive initial download. Otherwise the deltainformation will not be downloaded from the OLTP system to CRM which leads to inconsistency betweenthe two systems,

During the test runs, check for errors which occurred during the download. Errors may occur if the datadownloaded to the CRM system is not formatted in the way expected by the CRM system. For example,when downloading business partners, the phone number must be in a specific format. If the phone

number has a different format in the OLTP system, an error will occur during the initial download of thisbusiness partner, as the data cannot be entered into the CRM system.

Correct the errors which occurred during the initial download and repeat the download to check whetherthe error correction worked. Afterwards correct the errors in the productive environment also. Try torepeat the test runs until errors no longer occur.

Measure the runtime for the initial download for all download objects with a high volume. This is essentialfor estimating the time necessary to download all business objects and to optimize the setup for parallelprocessing. Please keep in mind that the extrapolation is only a rule of thumb because the downloadslows down with increasing tables. You should also refresh the table statistics.

Parallel Processing of the Init ial DownloadParallel Processing of the Initial Download is very effective for objects that have very extensive data sets,

for example, for the download of business partners, parallel processing of the initial download may benecessary to process the data in the given time frame. Parallel processing is described in detail in SAPNote 350176.

To configure parallel processing of the initial download you need to decide between two cases:

1. Data selection on the OLTP is faster than the processing on the CRM server (e.g. materials,business partners, conditions). If the bottleneck is caused by processing on the CRM server, abetter performance can be achieved by increasing the number of qRFC inbound queues on theCRM server as each queue is processed by one work process. Where there is a large datavolume it is possible that the time frame is not sufficient for finishing the initial download with oneselection process on the OLTP side. In this special case you could use requests in parallel. Perrequest you can define the number of inbound queues on CRM used per request. For details seeSAP Note 426159.

2. Data selection on the OLTP is slower than the processing on the CRM server. (E.g. for salesdocuments). In this case starting multiple downloads can increase the performance, since morework processes are now used to select the data on the OLTP R/3 system. To do this you have todefine and start multiple requests with non overlapping parameters for the same downloadobject.

Procedure:

To decide if the bottleneck is created more on the CRM server or on the OLTP server, performtest runs of the initial download for selected data and measure the run time on both sides. Theruntime information can be retrieved by evaluating the statistical records using transaction STADor by directly monitoring the process using transaction SM50. Here you should check theactivities for the user defined in the rfc destination from R/3 to CRM. If you use the Destinationwith LOGON Data in SMQR, the user is the user defined there.


10/32


2004 SAP AG

10

To decide if parallel processing is necessary on the CRM server based on these measurements,and to calculate how many parallel queues should be setup per download object, follow theprocedure described in the section Parallel Processing on the CRM Server

To setup a parallel processing on the OLTP system, follow the procedure described in thesection Parallel Processing on the OLTP Server

Parallel Processing on the CRM Server

Time Frame for Initial Download - Parallel Processing

Procedure: Check whether the time frame planned for the productive initial download in the productiveenvironment is sufficient. Therefore use the data from the check table in the section "Test runs and timeframe for the initial download" and evaluate the data as follows:

1. Define the available time frame for the initial download of all required objects. This timeframedoes not include the time required for data verification and correction.

2. Calculate the time frame needed to download the given data volume to the CRM server.

Therefore divide the given data volume for the selected download object by the download rateper hour and per job as measured in test runs and add up time per object to get the total timerequired.

As the download rate depends on customizing settings and on the data structure itself, nobenchmark is available as yet. However, as a rule of thumb we expect that one data object (e.g.onebusiness partner or one line item of an order) can be downloaded in 1 second assuming thatthe support package level is up to date, performance-relevant notes have been implemented andall parameter recommendations given in the GoingLive Analysis session have beenimplemented. This results in a download rate of around 3.600 data objects per hour.

3. Check whether the calculated time frame needed to download the given data volume to the CRMserver is shorter than the available time frame. If the calculated time frame is significantly longer,i.e. more than 25%, we can assume that the required volume cannot be downloaded in theavailable time frame. In these cases, parallel processing for this download object must be used.

To calculate the number of queues to be configured on the CRM server, see the section "Queueset up for Parallel Processing."

Test measurements for the initial download

The average download rate per hour per job for each download object is dependent on the hardware,parameter configuration and on the download data itself. To obtain exact figures for your own CRMsolution, you need to perform test downloads with a limited data volume. Please keep in mind that theextrapolation is only a rule of thumb because the download slows down with increasing table size.

Procedure: To set up and perform test downloads for the initial download proceed as follows:

Check if you can use the productive environment for the test measurement or set up a testenvironment with similar hardware. (As we do not use parallel processing for the test run, thenumber of CPUs in the test environment is not an issue and may differ significantly from the

number of CPUs in the productive environment. However, to use the measured runtime as abasis for our estimation of the average download rate in the productive environment, the testenvironment needs to have a similar CPU type (frequency and type)).

Setup test data for each download object. To get a representative value for the averagedownload rate per hour per job, there should be at least 1000 data sets available for download(for example, 1000 customers).

Specify the block size in the same way that will later be used for the productive download. E.g. ifyou will later download 100 customers per block set the block size to 100.

Set up the filter criteria in a way that ensures that no more data than necessary will bedownloaded. Example: If you have 1320 customers in the OLTP system of your test environmentfor the sales organization "0008" and you decide to download these customers for the test run,


11/32


2004 SAP AG

11

you must specify the filter condition for the download object CUSTOMER_MAIN as follows:VKORG =0008.

Be sure that the parameter settings are the same as will later be used for the initial download in

the productive environment, e.g. the setting for the flow trace level, log level

Start the test using transaction R3AS with only one queue in the CRM system (that is, withoutparallel processing).

Measure the time the job runs on the OLTP server. Use the system monitor SM50 for directmeasurement by monitoring the relevant process or evaluate the statistical records on the OLTPserver using transaction STAD

Get the total time the download queue ran on the CRM server. For this purpose either usetransaction SM50 or evaluate the statistical records on the OLTP server using transaction STAD.

For general recommendations regarding performance optimization on a test environment see forexample your GoingLive session. See also SAP note 429423 for General analysis in the initialLoad.


12/32


2004 SAP AG

12

Queue Setup for Parallel Process ing

To optimize the runtime of the initial download on the CRM middleware server, the number of downloadqueues to be run should be high. On the other hand, the available hardware resources limit the number

of download queues you can run in parallel. Please keep also in mind that the disk I/O could limit thenumber of queues which can be processed in parallel. In this section we will determine the optimumqueue set up for parallel processing.

Procedure: To setup and verify the scheduling for parallel processing of the initial download proceed asfollows:

1. Enter the information you have so far into the table below: Specify the object type and thevolume of the downloaded data (from the OLTP system to the CRM system).

2. Enter the value for the measured download rate per hour and per job. If you did not measure thedownload rate, enter the default values from the table in the section above.

3. Calculate the needed download time for serial processing. Therefore, for each download object,divide the value in the column Volume by the value in the column Download rate per hour andper job. Enter the calculated value in the column Download time in h needed (seq).

4. Calculate the maximum number of queues to be run in parallel that can be configured on theCRM server. Therefore consider two cases:

The CRM server is a Central Instance: Multiply the number of CPUs on the server by 2/3.Round to the next integer value and enter the value in the column CRM: Max. no. of Queuespossible. (For Example: Your CRM server is equipped with 8 CPUs. 2/3 of 8 CPUs would be5,33. This value would be rounded to 5)

The Database server is located on a separate machine than the application server(s): Add upall CPUs on all application servers and enter the figure in the column CRM: Max. no. ofQueues possible. (For Example: One database server with 2 CPUs, two application serverswith 3 CPUs each. Together this would result in 6 CPUs available and therefore 6 queuescan be run in parallel.)

5. Calculate the estimated download time if using parallel processing. To do this, for each download

object, divide the value in the column "Download time in h needed (seq)" by the value in thecolumn "CRM: Max. no. of Queues possible." Enter the calculated value in the column"Download time in h needed (parallel)." Please take into account that only the processing time onthe CRM side can be reduced by this parallel processing. With high parallelization you shouldtake this time into account.

6. Finally, enter a value for the available time frame for the initial download to the CRM system onthe last line. Calculate the sum of all values in the column "Download time needed (parallel)" andenter the sum on the last line.


13/32


2004 SAP AG

13

SETUP OF PARALLEL DOWNLOAD ON CRM MIDDLEWARE

Object Name Volume DownloadRate Per Hour

and Per Job

DownloadTime in h

Needed (Seq)

CRM: max. Noof Queues

Possible

DownloadTime in h

Needed(Parallel)

Material Master 40 000 5 000 8 4 2

Customer Master 120 000 2 400 50 4 12,5

Contacts 240 000 3 600 67 4 16,7

Classification: Attributes

Classification: Classes

Bill of Material

Customer Master

Customer Hierarchy

Contacts

Sales Documents

Total time frame available

(Hours)

72 Total time(Parallel)

31,2

To check whether the system is able to perform the initial download in the available time frame, comparethe two values "Total time frame available" and "Total time (parallel)" and set a rating for your evaluation:

RED:Set a red rating if the value for Total time frame available is lower than 50% of the value forTotal time (parallel).

YELLOW:Set a yellow rating if the value for Total time frame available is between 50% and 100%of the value for Total time (parallel).

GREEN:Set a green rating if the value for Total time frame available is higher than 100% of thevalue for Total time (parallel)

In addition to the above determined time you should consider time for unforeseen problemswhich may occur in the product ive initial download.

Procedure Final Conclusion:

With a green ratingwe can assume that the system will be capable of downloading the availabledocuments in the given time frame with the calculated number of queues in parallel. Thereforecontinue with the procedures for the configuration for the parallel initial download on the CRM server.

With a yellow ratingwe assume that the system will be capable of downloading the availabledocuments in the given time frame with the calculated number of queues in parallel. If the real

productive initial download performs as calculated is dependant on some issues:o Did you make your calculation with assumptions for the download rate? If yes run

performance test measurements for at least the top 3 download objects with the highestvolume. Update your calculation with the download rates measured during the test runs.

o For some download objects such as customers, the degree of parallel processing is limitedby the complexness of the relationships between customers. E.g. If customer A is related tocustomer B, customer B has to be download prior to customer A. To ensure this, customers

A and B have to be downloaded in one block and cannot be processed in parallel. Therefore,if you know that you are using complex customer relations, the degree of parallel processingfor the download object CUSTOMER_MAIN may be lower than the amount of CPUs on thesystem would allow.

Continue with the procedures for the configuration for the parallel initial download on the CRM server.


14/32


2004 SAP AG

14

With a red ratingwe assume that the system will not be capable of downloading the availabledocuments in the given time frame with the maximum number of queues to be run in parallel ascalculated above. In this case you should proceed as follows:

1. If you not already performed test measurements for at least the 3 download objects with the topvolume you should perform them as described above. With the measured average downloadtime recalculate the estimated download time and recalculate your rating.

2. Check if you can further extend the available time frame for the initial download.

3. Check if you can further decrease the volume of data to be downloaded. See the procedure andrecommendation given in the section Reducing data volume during the initial download

4. If your available time frame is still smaller them the estimated download time, try to run thedownload with a higher number of queues in parallel than calculated above. Running a highernumber of queues in parallel may lead to a CPU bottleneck situation on your CRM server. To findthe maximum number of download processes which can run in parallel, dedicated test runs arenecessary:

Double the number of queues to be run in parallel above.

Configure the system for parallel processing of the initial download as described in thesection Configuration of the parallel download processing on the CRM server.

Carry out a test run with the doubled number of queues. During this test run verify that allqueues are running. Check the CPU and memory consumption during the test run usingtransaction ST06 and ST02 on the CRM server. If the CPU & memory resources run at100% during the whole test run you may have to repeat the test run with a lower number ofdownload queues. By doing this you find the maximum number of queues that can be run inparallel on your CRM system.

With the new number for the parallel download queues, recalculate the estimated downloadtime and recalculate your rating.

If your available time frame is still smaller than the estimated download time you can be surethat you will not be able to download the planned volume in the given time frame. In this case

you should contact your hardware partner to discuss a resizing of the CRM server Because this initial download is only a temporary process, a possible solution would be to

use the test system temporarily as an application server. You would then have additionalhardware for the initial download.

Configuration of the parallel download processing on the CRM server

If your CRM system consists of several servers (E.g. one database server and two application servers)and you want to use several servers for parallel processing of the initial download on the CRM site, youhave to define an RFC server group on the CRM system including these servers and change the ASgroup accordingly in SMQR.

Implementation:To define an RFC server group, use transaction RZ12. In the QIN scheduler(transaction SMQR), you must assign this RFC server group by choosing "Edit" >> "Change AS groups."

In addition you should check that the Max. Runtimeis set to 60s which is he default value. For largequeues it makes sense to set the value to 300s.

Configuration of parallel processing for all download objects

Implementation:Make a new entry in table CRMPAROLTP on the OLTP including the parameter name"CRM_MAX_QUEUE_NUMBER_INITIAL" and consumer = "CRM." For the parameter value, enter thenumber of queues you decided to run in parallel. For example, if you want to run 5 queues in parallel,enter parameter value = 5.

Note: Parallel processing cannot be applied to all objects, e.g. for the initial download of CUSTOMIZINGobjects, which R/3 directly writes to the CRM online database (for example DNL_CUST_BASIS) theparallel processing can not be used. Therefore be sure that all necessary customizing objects have


15/32


2004 SAP AG

15

already been downloaded before configuring the parameters in table CRMPAROLTP for parallel initialdownload. In order to avoid resulting problems, remove parameter

CRM_MAX_QUEUE_NUMBER_INITIALfrom table CRMPAROLTP after the initial download has been

completed successfully. To avoid these problems define the configuration of parallel processing forspecific download objects as described below

Configuration of parallel processing for specific download objects

Parallel processing can be set up per download object.

Implementation: To process individual objects in parallel, make an entry in table CRMPAROLTP foreach download object (for example MATERIAL). For example, if you want to run 5 queues in parallel forthe download object MATERIAL, the entry in table CRMPAROLTP (Transaction SM30 in the OLTP)should look like this:

Parameter name: CRM_MAX_QUEUE_NUMBER_INITIAL

Parameter name 2: MATERIALParameter name 3:

Consumer: CRM

Parval1: 5

Note: For more detail, see SAP Notes 350176 and 510192.

Parameter settings fo r parallel processing of the ini tial download

In additional to the setup of the number of queues to be run in parallel, you have to check certainparameters on the CRM server to be sure that the configured number of queues can really be processedon the CRM server:

Be sure that the number of dialog work processes configured on the CRM server is higher thanthe number of queues you want to run in parallel. To have additional free work processesavailable for administrative purposes such as monitoring the initial download, there should be atleast 3 more dialog work processes available than the number of queues to be run in parallel.For example, if you want to run 10 queues in parallel you should have at least 13 dialog workprocesses configured on the CRM server.

Be sure that not all available dialog work processes are occupied by queue processing.Therefore, check especially the setup of the RFC resource parameter"rdisp/rfc_min_wait_dia_wp". This parameter describes the number of dialog work processes tobe kept free, so they cannot be used by incoming RFC calls. E.g. if you have 13 dialog workprocesses configured and you want a maximum of 10 to be used by RFC calls, the parameter"rdisp/rfc_min_wait_dia_wp" must be set to 3. As this parameter is set to 1 by default, you shouldincrease this parameter to at least 3 before starting the initial download. On the other hand the

parameter must not be set too high. If you have 13 dialog work processes configured and youwant 10 queues be processed in parallel, you must not set this parameter higher than 3.

Please take into account that the maximum number of parallel initial loads is restricted by theparameter MAX_PARALLEL_PROCESSES in table SMOFPARSFA. The delivered standardvalue is 5. The number of processes contains initial loads and requests. See also SAP note429423.

Parallel Processing on the OLTP Server

In some cases it may be necessary to have parallel processing on the OLTP. This can be carried out bymultiple request downloads with non-overlapping parameters in the OLTP system to shorten the runtimeof the initial download. One reason for parallel processing could be that for some download objects thedata selection on the OLTP is slower than the processing on the CRM server (e.g. for sales documents).


16/32


2004 SAP AG

16

Another reason can be that the daily backup, which has to be started at a fixed time, limits the availabletime frame for the initial download of one download object.

To run the selection and the filling of the download queues in parallel on the OLTP system there is no

special configuration necessary.To start the parallel processing on the OLTP server, start several request downloads with no overlappingparameters in parallel using transaction R3AR4. The request function is described in detail in the R/3

Adapter documentation. You can, for example, request document numbers 1 to 1000, 1001 to 2000, andso on from the OLTP ("Request"). Here, the general filters must fit the request filters. That means therequest filters must be a subset of the general filters.

Note: The maximum number of requests to be processed in parallel is limited by the parameterMAX_PARALLEL_PROCESSES in table SMOFPARSFA. The default value for this parameter is 5. Thisis described in SAP Note 429423.

Parallel Processing on OLTP and CRM Servers

In some cases, it may be necessary to use parallel processing of the initial download both on the OLTP

server and on the CRM server. One reason could be that the document volume is too high to use only asingle process for section on the OLTP site. To realize this you should use parallel requests. See detailsand restrictions in SAP note 426159.

How to Start the Init ial DownloadIn this chapter the procedure for starting the initial download is described. The transaction for starting theInitial Download can be found under

CRM 4.0: Middleware >> Data Exchange >> Initi al Load >> Start (Transaction R3AS).

Procedure: In order to ensure a correct initial download, perform the steps described in the followingsections.


17/32


2004 SAP AG

17

Deregistering the delta download queues

If you want to start the parallel initial download for a download object, you have to first deregister thedelta download queue for this download object. For a non-parallel initial download, this is not necessary

as the delta download is automatically stopped when the initial download is started. With an initialdownload using several download queues in parallel, the delta download for the selected downloadobject is also stopped. But in this case the download display in transaction R3AM1 can change to greentoo early and the block for the delta download is released, because the system may process the lastblock faster than earlier blocks. If the released delta download queue contains changes for the samedata object which is actually being processed in the running queue of the initial download, this situationcan lead to data inconsistencies. After parallel initial download you must again restart the delta queues.

Starting the initial download

Before starting the initial download, verify if the parent objects were successfully loaded. For example, ifyou want to download MATERIAL parallel within an object, you must first start parent objects"DNL_CUST_PROD0" and "CUSTOMER" separately, then wait for completion and then start the

dependent object "Material." For dependencies, refer to the entries in table SMOFOBJPAR.After starting the download for a selected download object in parallel, the system processes in parallel byblock number: For example, the first material block runs into the server inbound queue SMQ2 having thename R3AI_MATERIAL01, the second block into R3AI_MATERIAL02, and so on. The Queue Scheduleron the CRM server processes these blocks at the same time.

Recommendation: Monitor the initial download using the procedure described in the section Monitoringthe initial download

Note: With parallel initial downloads, the download display in transaction R3AM1 can change to greentoo early because the system may process the last block faster than the block prior to the last one. Forthis reason, you have to check not only the download display but also the inbound queue and the numberof data in the database (Transaction SE16) before you start additional objects. In order to avoid resultingproblems, remove parameter CRM_MAX_QUEUE_NUMBER_INITIAL from table CRMPAROLTP afterthe initial download has been completed successfully.

Initial load of Customers

To load the customer data into the CRM system, there are two different types of customer master dataaccording to the OLTP systems. One is Business partner master data for an Industry solution (e.g.:

Automotive, Pharmaceutical, Media, High Tech, Aerospace) R/3 system, and the other is Customermaster data for a standard R/3 OLTP system.

The download object used for Industry solution R/3 and non R/3 OLTP systems is BUPA_MAIN for thebusiness partners and BUPA_REL for contact persons.

The download object used for standard R/3 systems is CUSTOMER_MAIN for the business partnersand CUSTOMER_REL for contact persons.

To clarify if you have to use download objects BUPA_MAIN or CUSTOMER_MAIN for the initial load

from the OLTP, proceed as follows: If your customer data is stored in tables KNA1, KNVV, KNBK andKNB1, use CUSTOMER_MAIN to download customer master data. If your customer data is stored intable BUT000 and subsequent BUT* tables use BUPA_MAIN to download customer master data.

For example in an IS-Retail solution, customer master data is stored in the KN*-tables, which indicatesthat CUSTOMER_MAIN has to be used.

Note: If you are using an IS-Media solution as a backend system, please see also SAP Note 0436843 fordetailed requirements for the Initial download.


18/32


2004 SAP AG

18

Monitor ing the initial downloadIn this section the procedure for monitoring the initial download is described. To understand what has to

be monitored first, a description is given of how the initial download works and in which way the dataflows during the initial downloads.

A short overview over the most important monitors used during the initial download is also given.

Finally a description of what to do should errors occur during the initial download is provided.

How the initial download works

In this section we describe how the initial download works, based on the example of the download objectCUSTOMER:

Before the initial download of customers (and also contact persons), the system checks various importantdefault settings. In particular, these are the consistency of organizational management and the settingsfor the download of customers as they are described in the setup and download guide. If these defaultsettings are not correct, the system immediately issues a corresponding error message and the initialdownload is not started. In this case, the errors must first be corrected before the initial download can beperformed.

If no errors occur during this preliminary check, the initial download is started. All customers whichcorrespond to the filter criteria are selected from the R/3 system and sent in blocks of the defined blocksize (default setting 100) to the CRM system. There, the system fills the inbound queue with the namesR3AI_CUSTOMERXX. The "XX" in the queue name refers to the queue number, for example, if you setup 5 queues to run in parallel, the queues will be named "R3AI_CUSTOMER01","R3AI_CUSTOMER02", ,"R3AI_CUSTOMER05." If not previously carried out manually, all delta downloads from the R/3system are stopped. Therefore, during the initial download, delta downloads will not work. However, afterthe initial download is started, entries in table TBE31 on the OLTP system are created to trigger thedownload of delta information to synchronize the information on the OLTP and the CRM system. Theentries in the inbound queue in the CRM system are processed sequentially.

The following diagram describes the dataflow during the initial download from OLTP to CRM online:

Inbound

processing

Inbound

qRFC

mBDoc

R/3 AdapterBAPI

Outbound

qRFC

Outbound

processingValidation

CRMAdapter

3

Plug-In

OLTP Trigger

DownloadProxy

ExtractR/3

Extractor

BAPI Send

CRM

Qutbound qRFC

qRFC

1

5

6

7

Inbound qRFC

R3AS

qRFC

24


19/32


2004 SAP AG

19

The data flow consists of the following steps:

1. With the CRM transaction R3AS an initial download trigger is raised.

2. The RFCs are queued and processed in a sequence (dependent or independent objects).

3. In the backend system a function module call (extractor module call) is used to extract therequested data.

4. A data container is created with the selected data to be sent to the CRM Middleware server.

5. The data is transferred using qRFC in the form of tables to the CRM.

6. In the R/3 Adapter the BAPI is mapped to an mBDoc and then passed to the CRM Adapter.

7. After successful validation the BDoc is stored in the CRM database.

The initial load from OLTP to CRM online is finished with step 7 in case the mobile bridge and thecreation of CSA* queues are switched off.

Tools for monitoring the initial download

In general, there are 4 kinds of tools that can be used to monitor the initial download:Status monitoring:

The download monitor (R3AM1) described in chapter The Download Monitor (R3AM1) gives ageneral overview of the actual status of the initial download per download object.

Error monitoring and analysis:

The monitoring of the outbound queue in the OLTP backend described in the section "Monitoringthe outbound queues in the OLTP allows you to detect and analyze errors which occurred in theOLTP server.

The monitoring of the inbound queue in the CRM middleware described in the section"Monitoring the inbound queues in the CRM middleware" allows you to detect and analyze errorswhich occurred in the CRM server

Additional monitoring tools used to detect and analyze errors which occurred during the initial

download are described in SAP Note 429423. See also SAP Note 768503 Documentation forCRM Administration which refers to the Best Practice for BDoc Message Analysis. You shouldmonitor status E* and I* in SMW01.

Performance monitoring and analysis:

The execution of SQL statements can be monitored using the database monitor ST04 asdescribed in the section "Monitor the database cursor cache."

To find expensive SQL statements executed on the CRM server during the initial download theST05 trace tool can be used as described in the section "Performing an SQL Trace."

CRM Middleware Alert Monitoring

The CRM Middleware Alert Monitoring is based on the CCMS Alert Monitoring infrastructure. It is

accessed via transaction RZ20, where it is available under the "SAP CRM Monitor Templates" monitorsets with the name "CRM."

The CRM Middleware Alert Monitor generates alerts if critical situations occur. The processes aredisplayed as a tree structure in the monitor and the alerts are assigned to the nodes of the correspondingprocesses. The responsible person can then be informed automatically.

For more details see SAP Note 437187.

The Download Monitor (R3AM1)

The main tool to be used for monitoring the initial download is the Download Monitor on the CRMMiddleware:

CRM 4.0: Middleware >> Data Exchange >> Initial Load >> Monitor objects


20/32


2004 SAP AG

20

Time: The displayed time stamp corresponds to the start time of the processing of the recent block.

Block Nbr.: The number represents a counter for the number of successfully processed blocks.

P: a flag is set in case a parent object exist for this object.

If all the traffic lights are green, the download has completed successfully.

If a traffic light is yellow, choose Refresh and observe whether the block number increases. If so, the

download is still in progress. If not, an error has occurred during the initial download. To find and analyzethe error, check the monitors described in the following sections or follow the procedure given in thesection Correcting errors in the processing of the inbound queue.

Monitoring the outbound queues in the OLTP

To check the outbound queues on the OLTP server, execute transaction SMQ1 from the OLTP. To checkall outbound queues, enter "R3AI*" as queue name and press execute (F8). This gives a list of all activeoutbound queues with their actual number of entries in the OLTP R/3. For each download requeststarted, there will be an active download queue. If the number of entries for a selected download queueis increasing, the selection of the data in the OLTP is still active and the writing of the data in the CRMserver is slower than the selecting and preparing the data in the OLTP.

When double-clicking on a single Queue name field, additional details are displayed. In this view you

can also reactivate a queue. If capacity overload situations occur, a second attempt to execute the queueprocessing may be successful. If a second start of the queue is not successful and leads to a shortdump, you can find additional information in the short dump information displayed using transaction ST22in OLTP.

By double-clicking on the Status field additional status information is displayed

Monitoring the inbound queues in the CRM Middleware

To check the status of the inbound queues in the CRM server, use transaction SMQ2. To check alloutbound queues, enter "R3AI*" as queue name and press execute (F8). This gives you a list of all activeoutbound queues with their actual number of entries in the CRM server.


21/32


2004 SAP AG

21

When double-clicking on a single "Queue name" field, additional details are displayed. In this view youcan also reactivate a queue. By double-clicking on the "Status" field additional detailed information isgenerated. By double-clicking on the "Queue name" field, an additional detail display is generated where

you can execute a transaction by selecting function module "BAPI_CRM_SAVE" and choosing"EXECUTE LUW F6."

If the queue has the status "STOP" this may indicate a Customizing problem in the CRM application. Forfurther analysis, use transaction SMW01 as described in SAP Note 429423. See also SAP Note 768503Documentation for CRM Administration which refers to the Best Practice for BDoc Message Analysis.

Another example for a queue with the status "STOP" is the queue "R3AI_DNL_CUST_PROD2", whichstops with the status detail "the numbering scheme specified in table COMS_HIER_R3IMP does notexist."

This is a Customizing problem in the CRM/EBP application. The solution is to correct and finalize theCustomizing before continuing the initial download by restarting the queue. Please keep in mind that theInbound queue on CRM site is used only if the flag USE IN Q is set in OLTP table CRMRFCPAR. Thisflag is default delivered by SAP AG and recommended.

Monitoring the database cursor cache

To analyze the database cursor cache, use transaction ST04 and choose "Detailed Analysis" >> "SQLrequest." This shows a list of all accesses to the database since database startup. To view the mostexpensive SQL statements issued on the database, sort this list either by "buffer gets" or "disk reads."

If you mark one of these SQL statements and choose "Goto" >> "Explain SQL" from the menu, you canview the Execution Plan for the statement. If you cannot find a suitable index, you can view the statisticsby means of the "Analyze" button. For tables with statistics which are empty or nearly empty before theinitial download, you should refresh the statistics from time to time during the download in order to seethe correct statistics.

Performing an SQL Trace

Perform a detailed analysis using an SQL trace (transaction ST05). These can be carried out by stoppingthe queue and executing one LUW with an SQL trace in parallel. After this trace the queue should berestarted. In transaction ST05, the SQL trace highlights in red those tables for which performance-criticalstatements were run. You should then run a new SQL trace using Transaction ST05 to check that yourresolution of performance critical issues was successful and identify other performance critical tables.

Correcting errors in processing of the inbound queue

To correct the errors occurring during the processing of the inbound queue, read the error messages inSMW01. The queue name gives information on the object for which the error has occurred. (Forexample, queue name 'R3AI_CUSTOMER' indicates that the error has occurred in the BDOC with thename CAPGEN_OBJECT_WRITE.) The error messages can be checked in the BDOC viewer(Transaction SMW01) in the error segments.

Now edit these error messages. If you can correct all errors by correcting the Customizing in the CRMsystem, you can simply restart the stopped entry from the inbound queue in the CRM system and thesystem continues the initial download with the block that contains the incorrect customers.

If, however, you must correct one of the errors by correcting customer master data in the R/3 system,you must delete all entries from the inbound queue in the CRM system and then restart the initialdownload.

Note: For more details on correcting errors in the processing of the inbound queue, see SAP Note306567. For a general overview of how to perform an error analysis of an initial load, see SAP Note429423 and 768503.


22/32


2004 SAP AG

22

Initial load from a non-R/3 based OLTPThe communication between the non-R/3 OLTP system and the CRM middleware is necessary to

provide both systems with the necessary data to run the different business scenarios. The initial loadfrom a non-R/3 based OLTP system to a CRM system is possible as of Release CRM 3.0 using theExternal Interface Adapter (XIF).

The CRM Middleware provides the following interfaces for initial or delta loads from/towards non-SAPsystems:

XML messages / Intermediate Documents (IDocs) via ALE;

Business partner, hierarchy, relation import / export

Business Transaction (sales / service order, activity) import / export

Product import

Condition import

Invoice export

File based import;

SAP Data Transfer (DX) Workbench

ASCII Adapter

These interfaces allow data to be exchanged with legacy backend systems initially and on a regularbasis. Delta loads take place when changes to existing objects are made or when new objects arecreated. The exchange of customizing data is not relevant to the data exchange with non-SAP backendsystems. Product configuration and marketing data is also not exchanged. The mapping between forexample legacy and SAP CRM has to be defined by the customer.

In inbound transactions, incoming messages in XML or IDoc format are received by the XIF adaptervia the basic services (SOAP, ALE) and converted into the structures in the mBDoc. The CRMMiddleware then starts calling an appropriate application validation service that updates the applicationafter successful checks on the message.

In outbound transactions, creating or changing a data object within an application transaction, anmBDoc is created and transferred to the CRM Middleware. Possible external receivers for the mBDocare determined within the CRM Middleware and are transferred to the XIF adapter together with themBDoc. The data in the mBDoc is converted in the XIF adapter into an XML-like complex Data Structureand an appropriate basic service (SOAP, ALE) is started that sends the Data Object to the externalreceiver; for example, via a third party Middleware Tool.


23/32


2004 SAP AG

23

mBDoc flow

When you create or change data objects, the system generates a messaging BDoc (mBDoc) in the

outbound and passes it to the CRM Middleware. For the mBDoc, possible external recipients aredetermined within the CRM Middleware and passed to the XIF adapter. In accordance with theCustomizing settings, the messages are distributed by means of IDocs or XML/SOAP outboundprocessing. The function module CRMXIF_*_SAVE defines the CRM Online interface for the businesstransaction for external systems. The interface is implemented as an adapter for CRM Middleware (XIF)and provides both IDOC processing and processing for XML/SOAP calls. The corresponding IDOCmessage type is: CRMXIF_*_SAVE_M. The business transaction interface functions as an inbound- aswell as an outbound transaction interface and is capable of handling mass data. There you are informedabout the required Customizing. If you do not want to be informed about each change, you have to definecorresponding filter criteria in the middleware.

ASCII fi le

The ASCII file is written from the legacy system. This file has the structure of the legacy system. The

Data Transfer Workbench (DX-WB) calls the Legacy System Migration Workbench (LSMW) for mappingin the form of a job. The LSMW reads the ASCII file and converts it into the IDoc or XIF format. Then theIDoc has the structure of the complex XIF structure. Thus, the only condition for the ASCII file is that itcan be represented on the IDoc by means of LSMW.

XML/SOAP

XML/SOAP is more general. Therefore, potentially more EAI tools can offer an XML/SOAP connection. Inthis case it is important that the 3rd-party EAI (enterprise application integration) tool can offerreasonable error handling because the XML/SOAP call may also return application errors. In this case,the processing is synchronous. In general, the IDoc has a better performance. The error handling occursin the ALE inbound processing layer. That is, the 3rd party EAI does not have to have a detailed errorhandling.

Performance of the initial download

In this section are the most important recommendations for achieving good performance during the initialload to CRM using the External Interface Adapter (XIF). Go through all subsections and verify if theperformance recommendations can be applied to your CRM solution.

Deactivation of status check and generation of change documents

When loading a large number of business partners into the CRM server, the following recommendationshave to be applied:

Recommendation:Deactivate the status check and deactivate the generation of change documents forthe business partner as described in SAP notes: 493026, 456929 and 493343.

During the data transfer via the XIF adapter, the system cannot decide automatically whether this is aninitial data transfer or an exchange of changes. Thus the status check and the writing of changedocuments are always activated here. The deactivation of these two actions could improve the

performance. After the initial data transfer, it is necessary at least to deactivate the call indicator and toactivate the writing of change documents and the status check again.

See also SAP note 561321.

Recommendations when using the DX Workbench

If you use the DX Workbench apply the following recommendations:

Do not set the block size in DX workbench to a value higher than 100. This is, however, only arule of thumb and depends on the object.

Set the logging in the DX workbench to 'only errors'.

The split of the data into several files can further improve the throughput. However, this involvesan increased administrative effort for the provision of the files, scheduling of the jobs and so on.


24/32


2004 SAP AG

24

Set up a server group that is provided with as many dialog processes as possible. Use thisserver group in the DX workbench. See also the DX workbench documentation. During parallelprocessing, you must make sure that the limit for MAX ENQUEUE on the system side is set to a

high value.For additional details regarding performance optimization when loading business partner data viaDXWB/XIF see SAP note 561321

Avoid ini tial download to CDB or BW

To make full use of the advantages of parallel initial download the data should not be directly loaded tothe CDB or BW during the initial load to CRM online.

Recommendation: The mobile bridge needs to be switched off toavoid the load to CDB. With CRM 4.0it is possible to switch it off for all objects in transaction CMWC_SMW. If the flag Data Sync Active isnot set, the mobile bridge is deactivated. In addition the creation of the CSA* queues for these objectsshould be deactivated. With CRM 4.0 this is possible in transaction SMW3_00. The deactivation needs tobe carried out per BDOC type. For details see also SAP note 635697. This deactivation of the CSA*queues also switches off the delta load to BW for objects which are transferred to the BW via m flow. Ingeneral you should make the initial load to the CRM server first and then the initial load to the BW.

For some cases you may decide to have the CSA* queue active. In these cases avoid creating too manyCSA*-queue entries. Therefore change the data type related entry in Table SMOFQFIND.

As an example: one CSA-queue will be generated containing one entry for each activity document whenloading activities occur in the CRM system and the creation of CSA-queues for the activities is notswitched off as described in SAP note 635697.

Recommendation: From a performance point of view it is more efficient to have a low number of queuescontaining several entries. To reduce the number of queues for activities adjust the entry for TR_GNAME= BUS_TRANS_MSG in table SMOQFIND. To have a maximum of 99 entries per queue by using thelast 2 characters of the object instance, set FLDOFFSET to 8 and LENGTH to 2. This procedure can beused also for other object types.

Notes have to be implemented in addition to the customizing settings which have to be performed in

transaction SMW3_00 for some application Objects. The Notes are described in the table below:

SAP Note Content BDOC Included in

650624 Suppressing BDOC generation in CRM Server BUPA_MAIN,BUPA_REL

4.0 SP03

738562 Create no BDOCs for TGs if MSA is not installed TG_MSG 4.0 SP07

705953 CRM Standalone: Deactivate messaging flow forBUS_TRANS_MSG

BUS_TRANS_MSG 4.0 SP06

711267 MSA is not active -> suppression of BDOC creation CHAR_MSG,PFTPL_MSG

4.0 SP06

710606 No MSA in use, no creation of Bdocs desired CHARVAL_MSG 4.0 SP06

Test runs and t ime frame for the initial downloadTesting is mandatory in order to get an idea of how long the initial download will run (this can takeseveral days, or even up to two weeks) as runtime can be critical for the project time plan. Test phases ofone or several weeks should be considered because different errors (especially because of missingcustomizing) can occur.

For details on the recommended procedure please refer to the section Test runs and time frame for theinitial download

Parallel Processing of the Init ial DownloadParallel processing has to be used to increase the throughput on a multiprocessor server. When loadingdata from an external system to CRM, at least two different methods of parallel processing have to betaken into account.


25/32


2004 SAP AG

25

Parallel processing when using the Data transfer workbench

When using the Data transfer workbench (SXDA) parallel processing has to be implemented by creatingseveral run definitions with different file names in one sub-project and then starting runs in parallel.

In the SXDA the actual data transfer from a file is carried out in a run. When the run is started, all thetasks of the run definition are carried out in sequence when the associated programs are called.

Parallel processing when using IDOCs

Another way of loading data is using IDOCs. If the needed message type is not available you have todeveloped it yourself. The IDOCs can be created in several ways, e.g. using the SAP BusinessConnector. To load the data to the CRM database the IDOCs have to be processed using reportRBDAPP01. The make use of parallel processing using report RBDAPP01 proceed as follows:

1. Start report RBDAPP01 using transaction SE38

2. Switch to the tab Parallel Proc.

3. Mark the Checkbox Parallel Proc. Active

4. Use the F4-help to select a server group.

5. Switch to the tab IDOC selection

6. Enter the IDOC numbers to be processed or any other selection criteria you need in order to specifywhich IDOCs are to be processed

7. Define a useful package size and enter it in the field Pack size

8. Start the run

Defini tion of the Package size

The Package size defines how many IDOCs are processed in one package. To optimize the runtime youshould bundle IDOCs in packages of 10 IDOCs or more. The optimum package size can only bedetermined by a test run. For sales orders you should normally use package size 1. For another object itmay be appropriate to use a package size of 1000.

When using the parallel processing option the packages will be distributed to all available work

processes for RFC processing when report RBDAPP01 is started. When a work process has finished theprocessing of a package, the report will dispatch the next package to the free work process. Thiscontinues until all packages have been processed.

The only way to define the degree of parallel processing is by limiting the available work processes forRFC processing and to define RFC server groups

RFC Parameter settings for parallel processing

To limit the available work processes for RFC processing check the setup for the RFC resourceparameter "rdisp/rfc_min_wait_dia_wp". This parameter describes the number of dialog work processesto be kept free, so they cannot be used by incoming RFC calls. E.g. if you have 13 dialog work processesconfigured and you want a maximum of 10 to be used by RFC calls, the parameter"rdisp/rfc_min_wait_dia_wp" must be set to 3. As this parameter is set to 1 by default, you shouldincrease this parameter to at least 3 before starting the initial download. On the other hand the parametermust not be set too high. If you have 13 dialog work processes configured and you want 10 workprocesses for RFC processing in parallel, you must not set this parameter higher than 3.

To avoid overloading of your CPU resources you have to adjust this parameter according to the availableCPU resources on the server. In general the number of work processes for RFC processing should notexceed 1.5 times the number of available CPUs.

Configuration of the parallel processing on the CRM server

If your CRM system consists of several servers (E.g. one database server and two application servers)and you want to use several servers for parallel processing of the initial load, you have to define an RFCserver group on the CRM system which includes these servers.

Implementation:To define an RFC server group, use transaction RZ12. In the QIN scheduler(transaction SMQR), you must assign this RFC server group by choosing "Edit" >> "Change AS groups."


26/32


2004 SAP AG

26

Note: If you have several Application servers and one Database server we recommend excluding thedatabase server from the RFC server group to avoid overloading the database server.

Optimizing the Init ial load from CRM online toCDBIf you want to setup a mobile sales scenario, the data has to be not only loaded to the CRM server butalso to the mobile client. This makes the initial load process more complex and may also increase theneed for performance optimization for the initial load process. In the case of a mobile sales scenario, thebusiness data has to be transferred to the CRM Application tables, to the consolidated database (CDB)and to the mobile clients. To avoid performance bottlenecks on the CRM server, these three steps haveto be separated and performed in a 3-step procedure:

1. Initial load to CRM Application tables (CRM Online)

2. Initial load to consolidated database (CDB)

3. Initial load to mobile clients

The second step is discussed in this section. For the first step see sections described earlier in thisdocument dependent upon the data source. The first paragraph has some general performancerecommendations.

Before you can start the initial download from CRM online to CDB you have to activate the mobile bridgefor the corresponding object and deactivate the general switch for the mobile bridge. The CSA* queuesalso have to be reactivated. See also the section Avoid initial download to CDB.

Performance of the initial downloadIn this section you will find the most important recommendations for achieving a good performanceduring the initial download. Go through all subsections and verify if the performance recommendationscan be applied to your CRM solution. The main Note for performance issues regarding the initial

download is SAP Note 350176

Setting the middleware trace level parameters

In order to log CRM application errors, the middleware trace must be switched on during the initial load.

As only error information is required, the middleware trace level can be reduced to level "1" for all traceenvironments of the Middleware trace except for 'G' (generation). For this purpose, choose the followingpath in the SAP Menu 'Architecture and Technology -> Middleware -> Monitoring -> Message Flow ->Set up Middleware Trace' and choose 'Set default settings'.

To regularly delete outdated trace and log information, schedule report SM06_REORG on a regular basisas described in SAP Note 206439

Delete trace data and BDOC message administration tables before and after theinitial download has successfully completed

During an initial download, large data sets may be written in the middleware trace and the BDocmessage store. During the first part of the initial download the load from R/3 to CRM online, theSMWT_TRC and the BDoc store tables SMW3_BDOC*, may already be filled.

Recommendation: To delete the data before or after the initial download was completed successfully,schedule report SM06_REORG for the reorganization of the log tables as described in SAP Note206439. The reorganization job deletes data in the tables. Please take into account that the standardvariant holds the data from the last 7 days. Therefore with the initial download it makes sense to reducethe value days to hold to zero after you have checked that the initial download has functioned correctly.

For Oracle the indexes should be rebuilt as described in SAP note 435125.


27/32


2004 SAP AG

27

Reorganize BDOC message related object links before and after the initialdownload completed successfully

During the initial download many entries may be written to the tables SMW0REL and SRRELROLES dueto BDOC message related object links. These tables may already be filled during the first part of theinitial download the load from R/3 to CRM online.

Recommendation: To reorganize the data before or after the initial download has completedsuccessfully schedule the report RSRLDREL as described in SAP note 493156. In this special case (theinitial download) you should set the end date in such a way that as many links are reorganized aspossible.

Please see also SAP note 691628. With this note you can deactivate the writing of links betweenoutbound mBdoc messages and sBDoc messages.

CRM Server Performance: RFC Queue processing

The qRFC is part of mySAP Basis Technology. qRFC releases are decoupled from the CRM release

cycle and ongoing performance improvements are realized in the different versions. For releases lowerthan Basis Release 6.20 the qRFC version is an extra transport. For Basis Release 6.20 or higher, youcan only import the current qRFC version via a Support Package.

Recommendation:Always use the latest qRFC version. You should update qRFC on both the CRMserver AND the connected OLTP R/3 system.

Implementation: To obtain the newest qRFC release, see SAP Note 438015.

Note:After the implementation of a new basis support package, verify if the qRFC version is still thelatest. Some basis support packages may contain older qRFC versions. For Oracle DB with Supplement11please implement SAP note 742950to avoid performance problems.

In this section you find a list of the actual SAP Notes containing program enhancements related to qRFC.

SAP Note Content Release

539917 SQL error 601 in accessing "ARFCRSTATE" table 6.20

683459 qRFC performance improvements in the inbound scheduler 6.20, 6.30

697884 Queues in SMQ2 are not processed 6.20

742950 Performance affected on Oracle DB with Supplement 11 6.20

744869 Optimizing the performance of inbound scheduler 6.20, 6.40

Recommendation: For a general overview on queuing in CRM and R/3 and information on queuenames used and their respective functions in the CRM server and R3 backend, see SAP Note 765236

Reorganize Key-Gen Table

Table SMO9_KYTBL contains GUID keys and is used temporarily during the initial download. Aperformance loss during table lookups can occur when table SMO9_KYTBL contains too many entries

(for example, more than 50 000).Recommendation: Start report SMO9_CLEAR_KYTBL manually (via transaction SE38).

SMO9_CLEAR_KYTBL is part of the report SMO6_REORG, which should be scheduled every night.

Optimizing the performance for Oracle Database

To prepare an Oracle database for the initial download proceed as follows:

To optimize "FOR ALL ENTRIES" statements set the parameters as described in SAP note 634263.

Schedule Report SMO6_MAINT_STATISTIC in such a way that it is exited before the regular"update statistics run" begins. It could also make sense to run this report and "update statistics run"between two initial downloads.


28/32


2004 SAP AG

28

Inactive s ites

If you already have sites with assigned subscriptions you should deactivate these sites in SMOEAC toavoid filling the outbound queues. If you have not set up the assignment of subscripti

017 CRM Initial Download Configuration

Documents

Transcript of 017 CRM Initial Download Configuration