8/2/2019 86606 Bi 1003 - Data Extraction - V2
1/153
India SAP CoE, Slide 1
BI1003 Data Extraction BI1003 Data Extraction v2.0v2.0
8/2/2019 86606 Bi 1003 - Data Extraction - V2
2/153
India SAP CoE, Slide 2
Data ExtractionData Extraction
1 PrepareMe
2 TellMe
3 ShowMe
4 LetMe
5 HelpMe
8/2/2019 86606 Bi 1003 - Data Extraction - V2
3/153
India SAP CoE, Slide 3
Data ExtractionData Extraction
1 PrepareMe
2 TellMe
3 ShowMe
4 LetMe
5 HelpMe
8/2/2019 86606 Bi 1003 - Data Extraction - V2
4/153
India SAP CoE, Slide 4
PrepareMePrepareMe
In this course, we will cover 3 main sub-topics :In this course, we will cover 3 main sub-topics :
- Purpose- Purpose
- Use- Use
- Challenges- Challenges
8/2/2019 86606 Bi 1003 - Data Extraction - V2
5/153
India SAP CoE, Slide 5
PurposePurpose
This course demonstrates the various data extractionThis course demonstrates the various data extractionmethods.methods.
Extraction Methods coveredExtraction Methods covered Business Content Extraction from SAP R/3Business Content Extraction from SAP R/3 LIS & LO Extraction from SAP R/3LIS & LO Extraction from SAP R/3 CO-PA and FI-SL ExtractionCO-PA and FI-SL Extraction Generic ExtractorsGeneric Extractors Flat File ExtractionFlat File Extraction UD Connect, DB Connect, SOAP Based and Open HubUD Connect, DB Connect, SOAP Based and Open Hub
ExtractionExtraction
DataSource EnhancementDataSource Enhancement
8/2/2019 86606 Bi 1003 - Data Extraction - V2
6/153
India SAP CoE, Slide 6
UseUse
In many organizations, data is fragmented and spread acrossIn many organizations, data is fragmented and spread acrossmany databases and applications.many databases and applications.
To be useful, data must be integrated, standardized,To be useful, data must be integrated, standardized,synchronized, and enriched typically through ETLsynchronized, and enriched typically through ETL
(extraction, transformation, and loading) processes.(extraction, transformation, and loading) processes. SAP BI provides a broad set of ETL capabilities that supportSAP BI provides a broad set of ETL capabilities that support
data extraction.data extraction. With the open interfaces of SAP BI, data can be loaded fromWith the open interfaces of SAP BI, data can be loaded from
virtually any source and can handle the huge amounts ofvirtually any source and can handle the huge amounts of
transactional data typical of the enterprise landscape.transactional data typical of the enterprise landscape. Thus extraction deals with extracting the data into the BWThus extraction deals with extracting the data into the BWsystem for analysing and reporting.system for analysing and reporting.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
7/153
India SAP CoE, Slide 7
ChallengesChallenges
To extract large volumes of data from SAP R/3 by definingTo extract large volumes of data from SAP R/3 by defining
delta managementdelta management
Modifying the already available Business content andModifying the already available Business content and
satisfying user needssatisfying user needs Integrating data from non SAP systems into BWIntegrating data from non SAP systems into BW
8/2/2019 86606 Bi 1003 - Data Extraction - V2
8/153India SAP CoE, Slide 8
Data ExtractionData Extraction
1 PrepareMe
2 TellMe
3 ShowMe
4 LetMe
5 HelpMe
ShowMeShowMe
8/2/2019 86606 Bi 1003 - Data Extraction - V2
9/153India SAP CoE, Slide 9
Data Extraction : OverviewData Extraction : Overview
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAP
based
Datasource Enhancement
8/2/2019 86606 Bi 1003 - Data Extraction - V2
10/153India SAP CoE, Slide 10
Update Rules
Transfer Rules
Update Rules
Transfer Rules
ODSODS
Process FlowProcess Flow
PSAPSA
DataSourceDataSourceDataSourceDataSource
Update RulesUpdate Rules
InfoCube
DeliveryOrder
Order Delivery
OLTP System Data extractData extract Data extractData extract
Order-Deliv.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
11/153India SAP CoE, Slide 11
Process FlowProcess Flow
The data flows from the OLTP system to the PSA tablesThe data flows from the OLTP system to the PSA tableswithout any transformation.without any transformation.
If any transformation or filtration is required in the data it canIf any transformation or filtration is required in the data it canbe done in the update rules and transfer rules.be done in the update rules and transfer rules.
Then the data is loaded in the data targets like ODS. An ODSThen the data is loaded in the data targets like ODS. An ODS
object acts as a storage location for consolidated andobject acts as a storage location for consolidated andcleaned-up transaction data.cleaned-up transaction data.
The data from multiple ODS is then loaded into InfoCube andThe data from multiple ODS is then loaded into InfoCube andthus available to generate reports.thus available to generate reports.
In this example, Order and Delivery Data are extractedIn this example, Order and Delivery Data are extracted
separately, and populate separate PSA and ODS tables.separately, and populate separate PSA and ODS tables.The data is then merged within the ODS to form a new ODSThe data is then merged within the ODS to form a new ODSobject storing combined Order Delivery data. The data thenobject storing combined Order Delivery data. The data thenupdates the InfoCube with Order Delivery data.updates the InfoCube with Order Delivery data.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
12/153India SAP CoE, Slide 12
Multiple Source SystemsMultiple Source Systems
8/2/2019 86606 Bi 1003 - Data Extraction - V2
13/153India SAP CoE, Slide 13
Types of ExtractorsTypes of Extractors
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
14/153India SAP CoE, Slide 14
Types of ExtractorsTypes of Extractors
Extractors mainly are of two types:Extractors mainly are of two types: Application SpecificApplication Specific Cross ApplicationCross Application
Application Specific Extractors consist ofApplication Specific Extractors consist of BI Content Extractors (LO Cockpit)BI Content Extractors (LO Cockpit) Customer Generated Extractors (CO-PA, FI-SL and LIS)Customer Generated Extractors (CO-PA, FI-SL and LIS)
Cross Application Extractors consist of generic extractorsCross Application Extractors consist of generic extractors(based on Database table, Infoset and Function module)(based on Database table, Infoset and Function module)
8/2/2019 86606 Bi 1003 - Data Extraction - V2
15/153India SAP CoE, Slide 15
Business Content ExtractionBusiness Content Extraction
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
Business Content Extractors:
Application Specific Extractors
that are activated when
business content datasources
are transferred
8/2/2019 86606 Bi 1003 - Data Extraction - V2
16/153India SAP CoE, Slide 16
Business Content ExtractorsBusiness Content Extractors
SAP provides extractors for almost all applications. They areSAP provides extractors for almost all applications. They areprovided as an add on with the Business Content.provided as an add on with the Business Content. For each application like FI,CO and HR there are specificFor each application like FI,CO and HR there are specific
tables in the OLTP system.tables in the OLTP system. BI Content extractors consist of extract structures which areBI Content extractors consist of extract structures which are
based upon these tables and thus can be used to fetch thebased upon these tables and thus can be used to fetch thedata in BW.data in BW.
Hence when any company implements application like FI,Hence when any company implements application like FI,CO it can directly use the already available BI ContentCO it can directly use the already available BI ContentExtractors without the need to create their own.Extractors without the need to create their own.
SAP delivers these objects in Delivery version (D) and weSAP delivers these objects in Delivery version (D) and we
need to convert them in the active version (A).need to convert them in the active version (A). Business content DataSources from a Source System areBusiness content DataSources from a Source System are
available in BW for transferring data only after you haveavailable in BW for transferring data only after you haveconverted them into the active versions in the source systemconverted them into the active versions in the source systemand replicated them.and replicated them.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
17/153India SAP CoE, Slide 17
Uses of BI Content ExtractorsUses of BI Content Extractors
Built in ExtractorsBuilt in Extractors
High coverage (Applications and Industries)High coverage (Applications and Industries)
Available for both Transaction and Master dataAvailable for both Transaction and Master data
Ready to useReady to use
Reduce implementation Costs and effortsReduce implementation Costs and efforts
Delta capableDelta capable
8/2/2019 86606 Bi 1003 - Data Extraction - V2
18/153India SAP CoE, Slide 18
LO Data Extraction : OverviewLO Data Extraction : Overview
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
19/153India SAP CoE, Slide 19
LO CockpitLO Cockpit
Logistics Customizing Cockpit provides a simplified way toLogistics Customizing Cockpit provides a simplified way toextract logistics data and transfer it to SAP Businessextract logistics data and transfer it to SAP Business
Information Warehouse.Information Warehouse.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
20/153India SAP CoE, Slide 20
LO Data Extraction : Data FlowLO Data Extraction : Data Flow
As shown in the figure the document data for variousAs shown in the figure the document data for variousapplications like Customer order, delivery and billing is firstapplications like Customer order, delivery and billing is firsttransferred to various Communication Structures.transferred to various Communication Structures.
These communication structures are used to form variousThese communication structures are used to form variousextract structures in R/3 which consist of various DataSourcesextract structures in R/3 which consist of various DataSourceslike 2LIS_11_VAHDR, 2LIS_11_VAITM etc.like 2LIS_11_VAHDR, 2LIS_11_VAITM etc.
These DataSources then can be replicated to BW and assignedThese DataSources then can be replicated to BW and assignedto Infosources.to Infosources.
Then by creating transfer rules and update rules and definingThen by creating transfer rules and update rules and definingthe transformation the data is loaded into various data targetsthe transformation the data is loaded into various data targetslike InfoCube and ODS, thus available for reporting.like InfoCube and ODS, thus available for reporting.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
21/153India SAP CoE, Slide 21
V1 and V2 UpdatesV1 and V2 Updates V1 Update Synchronous UpdateV1 Update Synchronous Update
If you create/change a purchase order (me21n/me22n),If you create/change a purchase order (me21n/me22n),when you press 'SAVE' and see a success messagewhen you press 'SAVE' and see a success message(PO.... changed..), the update to underlying tables(PO.... changed..), the update to underlying tablesEKKO/EKPO has happened (before you saw theEKKO/EKPO has happened (before you saw themessage). This update was executed in the V1 workmessage). This update was executed in the V1 work
process.process.
V2 Update Asynchronous UpdateV2 Update Asynchronous Update
If you create/change a purchase order (me21n/me22n),If you create/change a purchase order (me21n/me22n),when you press 'SAVE' it takes few seconds to update towhen you press 'SAVE' it takes few seconds to update tounderlying tables EKKO/EKPO depending on systemunderlying tables EKKO/EKPO depending on systemload. This update was executed in the V2 work process.load. This update was executed in the V2 work process.
V3 Update has now replaced V1 and V2 updates.V3 Update has now replaced V1 and V2 updates.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
22/153India SAP CoE, Slide 22
V3 Update Modes in LO CockpitV3 Update Modes in LO Cockpit
There are four Update Modes in LO ExtractionThere are four Update Modes in LO Extraction
Serialized V3 UpdateSerialized V3 Update
Direct DeltaDirect Delta
Queued DeltaQueued Delta
Un-serialized V3 UpdateUn-serialized V3 Update
8/2/2019 86606 Bi 1003 - Data Extraction - V2
23/153India SAP CoE, Slide 23
Serialized V3 UpdateSerialized V3 Update
Transaction
tables
Applicationsp
ecificcustomer
exits
Update
Tables
BW Delta
queue
Staging
Engine
Periodic
Job
Delta Load
SM 13
RSA7
R3 system BW System
8/2/2019 86606 Bi 1003 - Data Extraction - V2
24/153India SAP CoE, Slide 24
Serialized V3 UpdateSerialized V3 Update
Transaction data is collected in the R/3 update tableTransaction data is collected in the R/3 update table
Data in the update tables is transferred through a periodicData in the update tables is transferred through a periodicupdate process to BW Delta queueupdate process to BW Delta queue
Delta loads from BW retrieve the data from this BW DeltaDelta loads from BW retrieve the data from this BW Deltaqueuequeue
Transaction postings lead to:Transaction postings lead to: Records in transaction tables and in update tablesRecords in transaction tables and in update tables
A periodically scheduled job transfers these postings into theA periodically scheduled job transfers these postings into the
BW delta queueBW delta queue This BW Delta queue is read when a delta load is executed.This BW Delta queue is read when a delta load is executed.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
25/153India SAP CoE, Slide 25
Direct DeltaDirect Delta
Transaction
tables
Applicationsp
ecificcustomer
exits
BW Delta
queue
Staging
Engine
Delta Load
R3 system BW System
8/2/2019 86606 Bi 1003 - Data Extraction - V2
26/153India SAP CoE, Slide 26
Direct DeltaDirect Delta
Each document posting is directly transferred into the BWEach document posting is directly transferred into the BWdelta queuedelta queue
Each document posting with delta extraction leads to exactlyEach document posting with delta extraction leads to exactlyone LUW in the respective BW delta queuesone LUW in the respective BW delta queues
Transaction postings lead to:Transaction postings lead to: Records in transaction tables and in update tablesRecords in transaction tables and in update tables
A periodically scheduled job transfers these postings into theA periodically scheduled job transfers these postings into theBW delta queueBW delta queue
This BW Delta queue is read when a delta load is executed.This BW Delta queue is read when a delta load is executed.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
27/153
India SAP CoE, Slide 27
Queued DeltaQueued Delta
Transaction
tables
Applicationsp
ecificcustomer
exits
Extraction
queue
BW Delta
queue
Staging
Engine
Periodic
Job
Delta Load
LBWQ
RSA7
R3 system BW System
8/2/2019 86606 Bi 1003 - Data Extraction - V2
28/153
India SAP CoE, Slide 28
Queued DeltaQueued Delta
Extraction data is collected for the affected application in anExtraction data is collected for the affected application in anextraction queueextraction queue
Collective run as usual for transferring data into the BW deltaCollective run as usual for transferring data into the BW deltaqueuequeue
Transaction postings lead to:Transaction postings lead to: Records in transaction tables and in extraction queueRecords in transaction tables and in extraction queue A periodically scheduled job transfers these postings into theA periodically scheduled job transfers these postings into the
BW delta queueBW delta queue This BW Delta queue is read when a delta load is executed.This BW Delta queue is read when a delta load is executed.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
29/153
India SAP CoE, Slide 29
Un-Serialized V3 UpdateUn-Serialized V3 Update
Transaction
tables
Applicationsp
ecificcustomer
exits
Update
Tables
BW Delta
queue
Staging
Engine
Periodic
Job
Delta Load
SM 13
RSA7
R3 system BW System
8/2/2019 86606 Bi 1003 - Data Extraction - V2
30/153
India SAP CoE, Slide 30
Un-Serialized V3 UpdateUn-Serialized V3 Update
Extraction data for written as before into the update tablesExtraction data for written as before into the update tableswith a V3 update modulewith a V3 update module
V3 collective run transfers the data to BW Delta queueV3 collective run transfers the data to BW Delta queue
In contrast to serialized V3, the data in the updating collectiveIn contrast to serialized V3, the data in the updating collectiverun is without regard to sequence from the update tablesrun is without regard to sequence from the update tables
Transaction postings lead to:Transaction postings lead to: Records in transaction tables and in update tableRecords in transaction tables and in update table
A periodically scheduled job transfers these postings into theA periodically scheduled job transfers these postings into the
BW delta queueBW delta queue This BW Delta queue is read when a delta load is executed.This BW Delta queue is read when a delta load is executed.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
31/153
India SAP CoE, Slide 31
New LO Data Extraction : Why ?New LO Data Extraction : Why ?
8/2/2019 86606 Bi 1003 - Data Extraction - V2
32/153
India SAP CoE, Slide 32
New LO Data Extraction : Why ?New LO Data Extraction : Why ?
Performance and Data VolumePerformance and Data Volume
Detailed extraction: You can deactivate the extraction of,
for example, scheduling data ('thin extractors' lead to
small upload volumes)
Document changes: only BW-relevant data changes areupdated (smaller upload volumes)
LIS tables are not updated: Reduced data volumes due to
removed redundant data storage
Update with batch processes: No overload of everydaywork'
8/2/2019 86606 Bi 1003 - Data Extraction - V2
33/153
India SAP CoE, Slide 33
New LO Data Extraction : Why ?New LO Data Extraction : Why ?
Simple Handling BW Customizing Cockpit: Central, single, maintenance
tool for Logistics applications No LIS functionality: No knowledge of LIS Customizing
update settings, etc, required
Function enhancement: Creating simply and withoutmodifications
Others Delta tables (SnnnBIW1/-2): duplicated update, duplicated
data storage
8/2/2019 86606 Bi 1003 - Data Extraction - V2
34/153
India SAP CoE, Slide 34
Customer Generated ExtractorsCustomer Generated Extractors
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
35/153
India SAP CoE, Slide 35
Customer Generated ExtractorsCustomer Generated Extractors
In response to demands from the customers, SAP hasIn response to demands from the customers, SAP hasdesigned the reporting facility in certain parts of the R/3designed the reporting facility in certain parts of the R/3system in such a way that it can be easily customized tosystem in such a way that it can be easily customized tomeet customers requirements.meet customers requirements.
The Business Information Warehouse must be able toThe Business Information Warehouse must be able toprocess this data, with the result that it must be possible toprocess this data, with the result that it must be possible todefine generic extractors in certain applications.define generic extractors in certain applications.
These applications are theThese applications are the Logistics Information SystemLogistics Information System
Profitability analysisProfitability analysis
Special LedgerSpecial Ledger
8/2/2019 86606 Bi 1003 - Data Extraction - V2
36/153
India SAP CoE, Slide 36
LIS ExtractionLIS Extraction
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
37/153
India SAP CoE, Slide 37
LIS Data ExtractionLIS Data Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
38/153
India SAP CoE, Slide 38
LIS Data ExtractionLIS Data Extraction
The LIS data flow can be described as follows:
Operative transaction data (sales orders, billing) is stored in
application data tables (for example VBAK/VBAP for sales
orders)
The information is transferred from an operative document to theCommunication Structures (for example MCVBAK,MCVBAP) to
update the data in the LIS).
These R/3 Communication perform the same function as the BW
Communication Structures, but they fill LIS InfoStructures in R/3
with data and not cubes in BW.
The update rules write the information from the Communication
Structure to the corresponding Information Structures.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
39/153
India SAP CoE, Slide 39
LIS Data ExtractionLIS Data Extraction
Although the term InfoStructure is used in R/3, theAlthough the term InfoStructure is used in R/3, theInfoStructure is a real transparent table and not the structureInfoStructure is a real transparent table and not the structurewhich contains data only at run time.which contains data only at run time.
LIS has many reporting tools like ABC analysis, correlation,LIS has many reporting tools like ABC analysis, correlation,
graphical displays but the limitations of the schema with onlygraphical displays but the limitations of the schema with onlyone table makes reporting slower than it would be in BW.one table makes reporting slower than it would be in BW. Thus in the LIS Extractor the data is transferred from the LISThus in the LIS Extractor the data is transferred from the LIS
InfoStructure in BW.InfoStructure in BW. LIS extractors are now obsolete and replaced by LO Cockpit.LIS extractors are now obsolete and replaced by LO Cockpit.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
40/153
India SAP CoE, Slide 40
FI-SL ExtractorsFI-SL Extractors
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
41/153
India SAP CoE, Slide 41
FI-SL ExtractorsFI-SL Extractors
FI-SL is an application in which data (planned and actual)FI-SL is an application in which data (planned and actual)from different levels of OLTP applications is combined tofrom different levels of OLTP applications is combined tomeasure business performance.measure business performance.
FI-SL includes planning functions and reporting tools. FI-SLFI-SL includes planning functions and reporting tools. FI-SLreporting in SAP R/3 is however restricted by the following :reporting in SAP R/3 is however restricted by the following :
Cross application reporting is not diverseCross application reporting is not diverse The OLTP system is optimized for transaction processing andThe OLTP system is optimized for transaction processing and
a high reporting workload would have a negative impact ona high reporting workload would have a negative impact onthe overall performance of the systemthe overall performance of the system
The solution to the above limitation is FI-SL Reporting in BW.The solution to the above limitation is FI-SL Reporting in BW.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
42/153
India SAP CoE, Slide 42
Uses of FI-SL ExtractorsUses of FI-SL Extractors
In FI-SL one is able to use alternate chart of accountsIn FI-SL one is able to use alternate chart of accounts(operative, group specific, country specific charts of(operative, group specific, country specific charts ofaccounts)accounts)
Adjustment postings (direct data entry) can be made in theAdjustment postings (direct data entry) can be made in theFI-SL systemFI-SL system
Various fiscal year variants enables one to create a weekly orVarious fiscal year variants enables one to create a weekly ormonthly reportmonthly report Validations or substitutions allow one to check or modify theValidations or substitutions allow one to check or modify the
data whenever it enters the FI-SL system.data whenever it enters the FI-SL system. Up to three currencies and two quantities can be run inUp to three currencies and two quantities can be run in
parallel in FI-SLparallel in FI-SL
Selective data retrieval - the assignment of transactions toSelective data retrieval - the assignment of transactions toparticular company code or ledgers determines whichparticular company code or ledgers determines whichledgers are to be updated.ledgers are to be updated.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
43/153
India SAP CoE, Slide 43
FI-SL Data FlowFI-SL Data Flow
8/2/2019 86606 Bi 1003 - Data Extraction - V2
44/153
India SAP CoE, Slide 44
CO-PA ExtractorsCO-PA Extractors
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
45/153
India SAP CoE, Slide 45
CO-PA ExtractorsCO-PA Extractors
Profitability Analysis (PA) is an integrated component in theProfitability Analysis (PA) is an integrated component in theSAP R/3 system.SAP R/3 system.
All of the data related to profitability from the other SAP R/3All of the data related to profitability from the other SAP R/3applications is mapped in CO-PA in accordance with theapplications is mapped in CO-PA in accordance with thecorresponding business transactions.corresponding business transactions.
This allows you to transfer into CO-PA billing document dataThis allows you to transfer into CO-PA billing document datafrom SD, cost object costs from CO-PC, and overall costsfrom SD, cost object costs from CO-PC, and overall costsfrom overall cost controlling.from overall cost controlling.
CO-PA collects all of the OLTP data for calculatingCO-PA collects all of the OLTP data for calculatingcontribution margins (sales, cost of sales, overhead costs)contribution margins (sales, cost of sales, overhead costs)
CO-PA also has powerful reporting tools and planningCO-PA also has powerful reporting tools and planningfunctionsfunctions
8/2/2019 86606 Bi 1003 - Data Extraction - V2
46/153
India SAP CoE, Slide 46
CO-PA ExtractorsCO-PA Extractors
8/2/2019 86606 Bi 1003 - Data Extraction - V2
47/153
India SAP CoE, Slide 47
CO-PA ExtractorsCO-PA Extractors During billing SD, revenues and payments are transferred toDuring billing SD, revenues and payments are transferred to
profitability segments in Profitability Analysis. At the same time,profitability segments in Profitability Analysis. At the same time,sales quantities are valuated using the standard cost of goodssales quantities are valuated using the standard cost of goodsmanufactured, as specified in the cost component split from CO-manufactured, as specified in the cost component split from CO-PC.PC.
One of the typical questions that can be answered with CO-PAOne of the typical questions that can be answered with CO-PAmodule is what are the top products and customers in ourmodule is what are the top products and customers in ourdifferent divisions.different divisions.
The wide variety of analysis and planning functions in CO-PAThe wide variety of analysis and planning functions in CO-PAallow you to plan, monitor and control the success of yourallow you to plan, monitor and control the success of yourcompany in terms of product-oriented, customer-oriented andcompany in terms of product-oriented, customer-oriented and
organizational dimensions of multidimensional profitabilityorganizational dimensions of multidimensional profitabilitysegments.segments.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
48/153
India SAP CoE, Slide 48
Generic ExtractorsGeneric Extractors
FI
SAP CRM
Types of Extractors
Application Specific Cross Application
BW Content
Extractors
Customer Generated
ExtractorsGeneric Extractors
LO COCKPIT
HR COLIS
FISL
COPA
DB View
Infoset
Function Module
Application
Specific DB Tables
Various SAP DB
tablesApplication
Specific DB Tables
FI
SAP CRM
8/2/2019 86606 Bi 1003 - Data Extraction - V2
49/153
India SAP CoE, Slide 49
Generic ExtractorsGeneric Extractors
Generic Extractors are Cross Application Extractors used inGeneric Extractors are Cross Application Extractors used in
scenarios where the other type of extractors are unavailable.scenarios where the other type of extractors are unavailable.
Generic extractors are of three types:Generic extractors are of three types:
Based on Database view/TableBased on Database view/Table Based on InfoSetBased on InfoSet
Based on Function ModuleBased on Function Module
8/2/2019 86606 Bi 1003 - Data Extraction - V2
50/153
India SAP CoE, Slide 50
Generic ExtractorsGeneric Extractors
When should you use generic extractors?When should you use generic extractors? Business Content does not contain a DataSource for yourBusiness Content does not contain a DataSource for your
application.application. The application does not feature its own generic deltaThe application does not feature its own generic delta
extraction methodextraction method
You are using your own programs in SAP R/3 to populateYou are using your own programs in SAP R/3 to populatethe tablesthe tables
The tools for generic data extraction can be used to generateThe tools for generic data extraction can be used to generatean extractor for all applications.an extractor for all applications.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
51/153
India SAP CoE, Slide 51
Flat file ExtractionFlat file Extraction
BW provides facility to load data from flat files (CSV or ASCIIBW provides facility to load data from flat files (CSV or ASCIIfiles).files).
It supports the following types of data :It supports the following types of data : Transaction dataTransaction data
Master data, either directly or flexiblyMaster data, either directly or flexibly AttributesAttributes TextText HierarchiesHierarchies
Flat file can be stored either on a local system or ApplicationFlat file can be stored either on a local system or Application
Server.Server.
But from performance point of view we store the file onBut from performance point of view we store the file onthe applications server and then load it into BW.the applications server and then load it into BW.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
52/153
India SAP CoE, Slide 52
UD Connect OverviewUD Connect Overview
With the help of the SAP Web AS J2EE connectivity, UDWith the help of the SAP Web AS J2EE connectivity, UD
Connect (Universal Data Connect) enables reporting andConnect (Universal Data Connect) enables reporting and
analysis of both SAP and non-SAP data.analysis of both SAP and non-SAP data.
Using UD Connect, you can access just about all relationalUsing UD Connect, you can access just about all relational
and multi-dimensional data sources.and multi-dimensional data sources.
UD Connect transfers the data as flat data. Multi-UD Connect transfers the data as flat data. Multi-
dimensional data is converted to a flat format when UDdimensional data is converted to a flat format when UDConnect is used.Connect is used.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
53/153
India SAP CoE, Slide 53
UD Connect OverviewUD Connect Overview
For the connection to DataSources (in BW), UD ConnectFor the connection to DataSources (in BW), UD Connectuses the J2EE Connector Architecture.uses the J2EE Connector Architecture.
BI Java Connectors that are available for various drivers,BI Java Connectors that are available for various drivers,protocols and providers as resource adapters.protocols and providers as resource adapters.
BI JDBC ConnectorBI JDBC Connector BI ODBO ConnectorBI ODBO Connector BI SAP Query ConnectorBI SAP Query Connector BI XMLA ConnectorBI XMLA Connector
8/2/2019 86606 Bi 1003 - Data Extraction - V2
54/153
India SAP CoE, Slide 54
DB Connect OverviewDB Connect Overview
Unix
Foreign
Application
ORACLE
DataBase
Purchasing
SAP BW
MS SQL
DataBaseNT
DB-Connect
8/2/2019 86606 Bi 1003 - Data Extraction - V2
55/153
India SAP CoE, Slide 55
DB Connect OverviewDB Connect Overview
A purchasing application runs on a legacy system based onan ORACLE database.
To analyze the data from the purchasing application, the data
needs to be loaded into the BW System (possibly installed on
a different database, e.g. MS SQL-Server). DB-Connect can be used to connect the DBMS of of the
purchasing application and extract data from tables or views.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
56/153
India SAP CoE, Slide 56
Data Transfer with DB Connect
8/2/2019 86606 Bi 1003 - Data Extraction - V2
57/153
India SAP CoE, Slide 57
Data Transfer with DB Connect
By default, when a BW application server is started by theSAP kernel, the system opens up a connection to thedatabase on which the SAP system runs.
All SQL commands (irrespective of whether they are Open orNative SQL) that are submitted by the SAP kernel or byABAP programs, relate automatically to this defaultconnection, meaning that they run in the context of thedatabase transaction that is active in this connection.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
58/153
India SAP CoE, Slide 58
Data Transfer with DB Connect
Connection data, such as database user name, userpassword, or database name are taken either from the profileparameters or from the corresponding environment variables(this is database-specific).
Thus DB Connect can be used to open up other database
connections in addition to the default connection and usethese connections to transfer data from tables or views into aBW system.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
59/153
India SAP CoE, Slide 59
SOAP-Based Transfer of Data
8/2/2019 86606 Bi 1003 - Data Extraction - V2
60/153
India SAP CoE, Slide 60
SOAP-Based Transfer of Data
As a rule, data transfer in BW takes place using a datarequest that is sent from BW to the Source System (pull fromthe scheduler).
You can also send the data to SAP BW using external
control. This is a data push in the SAP BW.
Data push is possible for multiple scenarios: Transferring Data Using the SOAP Service SAP Web AS Transferring Data Using Web Service
Transferring Data Using SAP XI
In all three scenarios, data transfer takes place via transfermechanisms that are sufficient for Simple Object AccessProtocol (SOAP) and are XML based.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
61/153
India SAP CoE, Slide 61
Open Hub Service
8/2/2019 86606 Bi 1003 - Data Extraction - V2
62/153
India SAP CoE, Slide 62
Open Hub Service
The open hub service enables you to distribute data from anSAP BW system into external data marts, analyticalapplications, and other applications.
With this, you can ensure controlled distribution using several
systems. The central object for the export of data is the InfoSpoke.Using this, you can define the object from which the datacomes and into which target it is transferred.
Through the open hub service, SAP BW becomes a hub ofan enterprise data warehouse. The distribution of data
becomes clear through central monitoring from thedistribution status in the BW system.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
63/153
India SAP CoE, Slide 63
Datasource EnhancementDatasource Enhancement
Need for DataSource enhancementNeed for DataSource enhancement When we require an additional field from a database table and
it is not directly available from the datasource, then we can
append the field in the extract structure of the datasource.
Then the logic to populate that field can be written in the
customer exit.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
64/153
India SAP CoE, Slide 64
1 PrepareMe
2 TellMe
3 ShowMe
4 LetMe
5 HelpMe
Data ExtractionData Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
65/153
India SAP CoE, Slide 65
BI Content ExtractorsBI Content Extractors
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
BI C t t E t t StBI C t t E t t St
8/2/2019 86606 Bi 1003 - Data Extraction - V2
66/153
India SAP CoE, Slide 66
BI Content Extractors StepsBI Content Extractors Steps
1. Activate Datasources
2. Replicate the Datasource in BW System
3. Assign InfoSource and maintain communication
structure and transfer rules
4. Maintain InfoCube and Update Rules
5. Create InfoPackage
6. Initialize Delta Process
7. Carry Out Delta Update
8/2/2019 86606 Bi 1003 - Data Extraction - V2
67/153
India SAP CoE, Slide 67
Step 1 - Activate DataSourcesStep 1 - Activate DataSources
In SAP R/3, go to
transaction Code SBIWDisplay IMG
Business Content
DataSourcesExecute
Transfer Business
Content DataSources
8/2/2019 86606 Bi 1003 - Data Extraction - V2
68/153
India SAP CoE, Slide 68
Step 1 - Activate DataSourcesStep 1 - Activate DataSources
Select the DataSourcefrom Application
Component Hierarchy.
Click on the button
Transfer DataSources/Activate the DataSource.
Assign to a transport
request.
Activate
datasources
8/2/2019 86606 Bi 1003 - Data Extraction - V2
69/153
India SAP CoE, Slide 69
Step 2 - Replicate datasourcesStep 2 - Replicate datasources
Tcode RSA1Tcode RSA1
ModellingModelling
Source SystemsSource Systems
Context MenuContext Menu
ReplicateReplicate
DataSources.DataSources.
8/2/2019 86606 Bi 1003 - Data Extraction - V2
70/153
India SAP CoE, Slide 70
Step 3- Assign the InfoSourceStep 3- Assign the InfoSource
Maintain theMaintain the
communicationcommunication
structure and Activatestructure and Activate
the transfer rules.the transfer rules.
S f C
8/2/2019 86606 Bi 1003 - Data Extraction - V2
71/153
India SAP CoE, Slide 71
Step 4 - Maintain InfoCube andStep 4 - Maintain InfoCube and
Update rulesUpdate rules
Infocube
Update rules
Step 5 Create InfoPackageStep 5 Create InfoPackage
8/2/2019 86606 Bi 1003 - Data Extraction - V2
72/153
India SAP CoE, Slide 72
Step 5 - Create InfoPackageStep 5 - Create InfoPackage
Go to InfoSources,
Select the InfoSource
Right click on the
Source System and say
create InfoPackage
Scheduler
screen
Monitor
Step 6 Initialize the deltaStep 6 Initialize the delta
8/2/2019 86606 Bi 1003 - Data Extraction - V2
73/153
India SAP CoE, Slide 73
Step 6 - Initialize the deltaStep 6 - Initialize the delta
processprocessGo to InfoPackage
and select the update
type as initialize delta
process.
And click on start toschedule the process
Monitor
Scheduling
Initialize the
delta process
I i i li i h D l PI iti li i th D lt P
8/2/2019 86606 Bi 1003 - Data Extraction - V2
74/153
India SAP CoE, Slide 74
Initializing the Delta ProcessInitializing the Delta Process
(Scheduler)(Scheduler) If you want to transfer data in the delta process, the process
must be initialized in the first data request. In this process, the selection conditions for connecting delta
uploads are set and its complete dataset is loaded into BWfrom the R/3 System.
To do this, you must select the checkbox Initialize delta
process for the Info Package under the update parameters
of the Scheduler.
Once the data is requested message appears afterscheduling, Go to Monitor to view the status and details of
the data load.
M it i th U l d P dM it i th U l d P d
8/2/2019 86606 Bi 1003 - Data Extraction - V2
75/153
India SAP CoE, Slide 75
Monitoring the Upload ProcedureMonitoring the Upload Procedure
D lt U d t (S h d l )D lt U d t (S h d l )
8/2/2019 86606 Bi 1003 - Data Extraction - V2
76/153
India SAP CoE, Slide 76
Delta Update (Scheduler)Delta Update (Scheduler)
Only the data since last
upload is requested
D lt U d t (S h d l )D lt U d t (S h d l )
8/2/2019 86606 Bi 1003 - Data Extraction - V2
77/153
India SAP CoE, Slide 77
Delta Update (Scheduler)Delta Update (Scheduler)
For high volume of transaction data, a full update here is mostlyonly justified for the first time data is transferred or if the statisticsdata has to be rebuilt following a system failure.
Delta update mechanisms that restrict the volume of data towithin realistic limits, therefore, are required to implement aperformance-oriented, periodical data transfer strategy. For e.g. when sales figures are updated every week in the
Business Information Warehouse, only the sales documentinformation that has been added or changed in the last weekshould be extracted.
LO C k itLO C k it
8/2/2019 86606 Bi 1003 - Data Extraction - V2
78/153
India SAP CoE, Slide 78
LO CockpitLO Cockpit
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
L i ti C t i i C k itL i ti C t i i C k it
8/2/2019 86606 Bi 1003 - Data Extraction - V2
79/153
India SAP CoE, Slide 79
Logistics Customizing CockpitLogistics Customizing Cockpit
The LO Cockpit contains the following functionality, as
specified in the procedure sequenced below:
Maintaining extract structures
Maintaining DataSources
Activating the update
Controlling V3 update
LO D t E t ti (SBIW)LO D t E t ti (SBIW)
8/2/2019 86606 Bi 1003 - Data Extraction - V2
80/153
India SAP CoE, Slide 80
LO Data Extraction (SBIW)LO Data Extraction (SBIW)
LO
customizing
cockpit lbwe
BW Logs
(LBWF)
Perform set up
Application
wiseDelete set
up tables(LBWG)
L i ti E t t St tL i ti E t t St t
8/2/2019 86606 Bi 1003 - Data Extraction - V2
81/153
India SAP CoE, Slide 81
Logistics Extract Structure Logistics Extract Structure
Cockpit (LBWE)Cockpit (LBWE)
Maintain
extract
structure
Set up V3
update
Activate
datasources
Individual Steps in LO ExtractionIndividual Steps in LO Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
82/153
India SAP CoE, Slide 82
Individual Steps in LO ExtractionIndividual Steps in LO Extraction
1. Maintain Extract Structure
2. Replicate the Datasource in BW System
3. Assign InfoSource and maintain
communication structure and transfer rules
4. Maintain InfoCube and Update Rules
8. Initialize Delta Process
9. Carry Out Delta Update
5. Activate Extract Structure
6. Delete setup tables/Perform Extraction
7. Set Up periodic V3 update
St 1 M i t i i th E t tSt 1 M i t i i th E t t
8/2/2019 86606 Bi 1003 - Data Extraction - V2
83/153
India SAP CoE, Slide 83
Step 1 - Maintaining the ExtractStep 1 - Maintaining the Extract
StructureStructure
Maintainextract
structure
M i t i i th E t tMaintaining the E tract
8/2/2019 86606 Bi 1003 - Data Extraction - V2
84/153
India SAP CoE, Slide 84
Maintaining the ExtractMaintaining the Extract
StructureStructure The extract structure is filled with the assigned communicationstructures. You can only use selected fields from thecommunication structures (SAP internal control fields, forexample, are not offered).
SAP already delivers extract structures, which you can enhance(by connecting to the communication structure). Every extractstructure can be maintained by you and by SAP.
After you set up the extract structure, the system automaticallygenerates it. This completes missing fields (their units andcharacteristics). The extract structure is created hierarchically
according to the communication structures. Everycommunication structure leads to the generation of asubstructure that belongs to the actual extract structure.
M i t i i D t SMaintaining DataSources
8/2/2019 86606 Bi 1003 - Data Extraction - V2
85/153
India SAP CoE, Slide 85
Maintaining DataSourcesMaintaining DataSources
Maintain
datasources
M i t i i D t SMaintaining DataSources :
8/2/2019 86606 Bi 1003 - Data Extraction - V2
86/153
India SAP CoE, Slide 86
Maintaining DataSources :Maintaining DataSources :
ProcedureProcedure There exists a DataSource(for e.g. 2LIS_11_VAITM) foreach extract structure that ismade available (for example,MC11VA0ITM) in the OLTP
System. A maintenance screen is
displayed in which you canto assign other properties tothe fields of the extractstructure Selection fields Hide fields Cancellation fields
(field is inverted whencanceled (*-1))
Step 2 ReplicatingStep 2 Replicating
8/2/2019 86606 Bi 1003 - Data Extraction - V2
87/153
India SAP CoE, Slide 87
Step 2 ReplicatingStep 2 Replicating
DataSourcesDataSources
Step 3 - MaintainingStep 3 - Maintaining
8/2/2019 86606 Bi 1003 - Data Extraction - V2
88/153
India SAP CoE, Slide 88
Step 3 - MaintainingStep 3 - Maintaining
Communication StructureCommunication Structure
All the fields in the extractstructure are shown with
the assigned info objects
in the template
Step 4 Maintaining TransferStep 4 Maintaining Transfer
8/2/2019 86606 Bi 1003 - Data Extraction - V2
89/153
India SAP CoE, Slide 89
Step 4 - Maintaining TransferStep 4 - Maintaining Transfer
RulesRules
Step 5 Maintaining InfoCubesStep 5 Maintaining InfoCubes
8/2/2019 86606 Bi 1003 - Data Extraction - V2
90/153
India SAP CoE, Slide 90
Step 5 - Maintaining InfoCubesStep 5 - Maintaining InfoCubes
and Update Rulesand Update Rules
Step 6 Activating ExtractStep 6 Activating Extract
8/2/2019 86606 Bi 1003 - Data Extraction - V2
91/153
India SAP CoE, Slide 91
Step 6 - Activating ExtractStep 6 - Activating Extract
StructuresStructures
Activate extract
structure
Step 7 Delete set up tablesStep 7 Delete set up tables
8/2/2019 86606 Bi 1003 - Data Extraction - V2
92/153
India SAP CoE, Slide 92
Step 7 - Delete set up tablesStep 7 - Delete set up tables
In transaction LBWG enter the application number
for which you want to delete the set up tables.
For e.g. For 2LIS_11_VAITM enter the number 11
and execute.
Step 8 - Initialization/SimulationStep 8 - Initialization/Simulation
8/2/2019 86606 Bi 1003 - Data Extraction - V2
93/153
India SAP CoE, Slide 93
Step 8 Initialization/Simulationp
(OLI*BW)(OLI*BW)
Give a name to
the job
Initializing ('build') the dataset for therespective extract structures is to be
done before data can be extracted
from the R/3 System.
The initialization is set up in the
OLTP. The restructuring process fillsrestructuring tables that are read
during the initialization.
It is recommended to carry this out
before, in the simulation mode. The
option of checking the extract
structure update, without actually
updating the data is available. The
results are recorded in a detailed
simulation log.
Step 8 Creating InfopackagesStep 8 Creating Infopackages
8/2/2019 86606 Bi 1003 - Data Extraction - V2
94/153
India SAP CoE, Slide 94
Step 8 - Creating InfopackagesStep 8 - Creating Infopackages
Step 9 Initializing the DeltaStep 9 Initializing the Delta
8/2/2019 86606 Bi 1003 - Data Extraction - V2
95/153
India SAP CoE, Slide 95
Step 9 - Initializing the DeltaStep 9 - Initializing the Delta
ProcessProcess
V3 Update- SettingsV3 Update- Settings
8/2/2019 86606 Bi 1003 - Data Extraction - V2
96/153
India SAP CoE, Slide 96
V3 Update- SettingsV3 Update- Settings
Set up V3 update
Set the start time
and print
parameters.
Schedule batch
jobs
V3 Update SettingsV3 Update Settings
8/2/2019 86606 Bi 1003 - Data Extraction - V2
97/153
India SAP CoE, Slide 97
V3 Update SettingsV3 Update SettingsIn Start Date
Select
immediatescheduling
Give the print
parameters
Schedule the job and
check the joboverview in SM37
Delta Queue MaintenanceDelta Queue Maintenance
8/2/2019 86606 Bi 1003 - Data Extraction - V2
98/153
India SAP CoE, Slide 98
Delta Queue MaintenanceDelta Queue Maintenance
Go to transaction
RSA7.
You can see that the
delta queue has been
successfully filled.
Step 10 Delta UpdateStep 10 -Delta Update
8/2/2019 86606 Bi 1003 - Data Extraction - V2
99/153
India SAP CoE, Slide 99
Step 10 -Delta UpdateStep 10 -Delta Update
(Scheduler)(Scheduler)
Only the data since
last upload is
requested
FI-SL ExtractorsFI-SL Extractors
8/2/2019 86606 Bi 1003 - Data Extraction - V2
100/153
India SAP CoE, Slide 100
FI-SL ExtractorsFI-SL Extractors
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
FI-SL extraction Step-by-StepFI-SL extraction Step-by-Step
8/2/2019 86606 Bi 1003 - Data Extraction - V2
101/153
India SAP CoE, Slide 101
FI SL extraction Step by StepFI SL extraction Step by Step
2. Define or Assign a DataSource for Ledger
3. Replicate the Datasource in BW System
4. Assign InfoSource and maintain
communication structure and transfer rules
5. Maintain InfoCube and Update Rules
6. Create InfoPackage
7. Initialize Delta Process
8. Carry Out Delta Update
1. Generate an extract structure for summary
table
Step 1 - Setting up an FI-SLStep 1 - Setting up an FI-SL
8/2/2019 86606 Bi 1003 - Data Extraction - V2
102/153
India SAP CoE, Slide 102
DataSourceDataSource
Step 1 - Setting up an FI-SLStep 1 - Setting up an FI-SL
8/2/2019 86606 Bi 1003 - Data Extraction - V2
103/153
India SAP CoE, Slide 103
Go to SBIWGo to SBIW
SettingsSettings
forfor
ApplicationApplicationspecific dataspecific data
sourcessources
GenerateGenerate
transfertransfer
structure forstructure fortotals tabletotals table
p g pp g p
DataSourceDataSource
Step 2 - Setting up an FI-SLStep 2 - Setting up an FI-SL
8/2/2019 86606 Bi 1003 - Data Extraction - V2
104/153
India SAP CoE, Slide 104
p g pp g p
DataSourceDataSource
Step 2 - Setting up an FI-SLStep 2 - Setting up an FI-SL
8/2/2019 86606 Bi 1003 - Data Extraction - V2
105/153
India SAP CoE, Slide 105
Step 2 Setting up an FI SLp g p
datasource :datasource :
Go to SBIWGo to SBIWSettings forSettings forapplicationapplicationspecificspecific
datasourcesdatasources FinancialFinancialaccountingaccounting createcreateinfosource forinfosource forLedgerLedger
Step 2 - Setting up an FI-SLStep 2 - Setting up an FI-SL
8/2/2019 86606 Bi 1003 - Data Extraction - V2
106/153
India SAP CoE, Slide 106
Step 2 - Setting up an FI-SLStep 2 - Setting up an FI-SL
DataSourceDataSourceThe statusThe statuscolumn showscolumn showswhether awhether aDataSource isDataSource is
assigned to theassigned to theledger already.ledger already.(Traffic light(Traffic lightgreen)green)
You select theYou select theledger to whichledger to whichyou want toyou want toassign theassign theDataSourceDataSource
Step 2 - Setting up an FI-SLStep 2 - Setting up an FI-SL
8/2/2019 86606 Bi 1003 - Data Extraction - V2
107/153
India SAP CoE, Slide 107
Step 2 - Setting up an FI-SLStep 2 Setting up an FI SL
DataSourceDataSource
FI-SL ExtractorsFI-SL Extractors
8/2/2019 86606 Bi 1003 - Data Extraction - V2
108/153
India SAP CoE, Slide 108
FI-SL ExtractorsFI SL Extractors
After generating the FI-SL DataSource rest of the steps areAfter generating the FI-SL DataSource rest of the steps are
same as explained earlier.same as explained earlier.
Replicate the DataSource and then assign it to theReplicate the DataSource and then assign it to the
infosource.infosource.
Create transfer rules and update rulesCreate transfer rules and update rules
Create InfoPackage and initialize the delta processCreate InfoPackage and initialize the delta process
CO-PA ExtractorsCO-PA Extractors
8/2/2019 86606 Bi 1003 - Data Extraction - V2
109/153
India SAP CoE, Slide 109
CO PA ExtractorsCO PA Extractors
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
CO-PA Extraction (Steps)CO-PA Extraction (Steps)
8/2/2019 86606 Bi 1003 - Data Extraction - V2
110/153
India SAP CoE, Slide 110
( p )( p )
2. Replicate the Datasource in BW System
3. Assign InfoSource and maintain
communication structure and transfer rules
4. Maintain InfoCube and Update Rules
5. Create InfoPackage
6. Initialize Delta Process
7. Carry Out Delta Update
1. Generate DataSource in OLTP
Step 1 - Generate CO-PAStep 1 - Generate CO-PA
8/2/2019 86606 Bi 1003 - Data Extraction - V2
111/153
India SAP CoE, Slide 111
Step 1 Generate CO PAStep 1 Generate CO PA
DataSourceDataSource
Go to Transaction KEB0.
Define the DataSource for the
current client of your SAP R/3
System on the basis of one ofthe operating concerns.
A standard name is always
generated. For e.g. 1_CO_PA
%CL%ERK with prefix
1_CO_PA is mandatory.
Select the
operating concern
Step 1 - Generate DataSourceStep 1 - Generate DataSource
8/2/2019 86606 Bi 1003 - Data Extraction - V2
112/153
India SAP CoE, Slide 112
Step 1 Generate DataSourceStep 1 Generate DataSource
Maintain CO-PA datasourceMaintain CO-PA datasource
8/2/2019 86606 Bi 1003 - Data Extraction - V2
113/153
India SAP CoE, Slide 113
Maintain CO PA datasource
You can then specify
which fields are to
be applied as the
selection for the CO-PA extraction.
Providing required
fields for selection
CO-PA ExtractorsCO-PA Extractors
8/2/2019 86606 Bi 1003 - Data Extraction - V2
114/153
India SAP CoE, Slide 114
CO PA ExtractorsCO PA Extractors
After generating the CO-PA DataSource rest of the steps areAfter generating the CO-PA DataSource rest of the steps are
same as explained earlier.same as explained earlier.
Replicate the DataSource and then assign it to theReplicate the DataSource and then assign it to the
infosource.infosource.
Create transfer rules and update rulesCreate transfer rules and update rules
Create InfoPackage and initialize the delta processCreate InfoPackage and initialize the delta process
Generic ExtractorsGeneric Extractors
8/2/2019 86606 Bi 1003 - Data Extraction - V2
115/153
India SAP CoE, Slide 115
Generic ExtractorsGeneric Extractors
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
Generic Data ExtractionGeneric Data Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
116/153
India SAP CoE, Slide 116
Steps for transaction DataSteps for transaction Data
1. Create DataSources for transaction Data
3. Replicate the Datasource in BW System
4. Assign InfoSource and maintain
communication structure and transfer rules
5. Maintain InfoCube and Update Rules
6. Create InfoPackage
7. Initialize Delta Process
8. Carry Out Delta Update
2. Create InfoSet/enhance table
Generic Data ExtractionGeneric Data Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
117/153
India SAP CoE, Slide 117
Steps for Master DataSteps for Master Data
1. Create DataSources for Master Data (attr)
3. Replicate the Datasource in BW System
4. Assign InfoSource and maintain
communication structure and transfer rules
5. Create InfoPackage for Master Data
6. Create InfoPackage for texts
7. Schedule InfoPackages
2. Create DataSources for texts
Step1 - Create genericStep1 - Create generic
8/2/2019 86606 Bi 1003 - Data Extraction - V2
118/153
India SAP CoE, Slide 118
DataSourceDataSourceGo to TransactionGo to TransactionRSO2RSO2 HereHere
you can selectyou can select
whether you wantwhether you want
to create ato create a
datasource fordatasource fortransaction data,transaction data,
Attributes orAttributes or
texts.texts.
Give a technicalGive a technicalname to thename to the
datasource anddatasource and
choose createchoose create
Create Generic DataSourceCreate Generic DataSource
8/2/2019 86606 Bi 1003 - Data Extraction - V2
119/153
India SAP CoE, Slide 119
Create a Generic
DataSource.Choose an application
component to which the
DataSource is to be
assigned.
Enter the description texts.
You can choose whether you
want to create the
DataSource based on
View, Function Module orInfoSet query.
Choose Generic delta
Generic Data ExtractionGeneric Data ExtractionOnly one
attribute can be
defined as delta
8/2/2019 86606 Bi 1003 - Data Extraction - V2
120/153
India SAP CoE, Slide 120
Specify the delta specific field
and its type. Maintain the
settings for generic delta.Specify the safety interval.
NOTE: safety interval should
be set so that no document is
missed even if it was not
stored in the DB table whenextraction took place.
Select delta type : New status
for changed records (i.e. after
image) This can be used with
data target ODS (AIE).Additive delta i.e. aggregated
data records (ADD) and then
choose save.
defined as delta
attribute
Generic Data ExtractionGeneric Data Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
121/153
India SAP CoE, Slide 121
Ge e c ata t act o
.
Now choose save again.
This will generate theDataSource. After
generating the
DataSource you will see
the delta update flag
selected.
Select the fields in the
selection which you want
to appear in the
Infopackage selection.
Generic Data ExtractionGeneric Data Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
122/153
India SAP CoE, Slide 122
Transport the DataSource to BW in Tcode RSA6(PostTransport the DataSource to BW in Tcode RSA6(PostProcessing of DataSources).Processing of DataSources).
The steps in BW :The steps in BW : Replicate the DataSources.Replicate the DataSources. Create necessary Info objectsCreate necessary Info objects
Generate Info source in BW, Maintain CommunicationGenerate Info source in BW, Maintain CommunicationStructure and Transfer RulesStructure and Transfer Rules
Create Info package and schedule it.Create Info package and schedule it.
Flat File ExtractionFlat File Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
123/153
India SAP CoE, Slide 123
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
Flat File ExtractionFlat File Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
124/153
India SAP CoE, Slide 124
1. Identify the data
3. Define fields of InfoSource as InfoObjects
4. Assign InfoSource and maintain
communication structure and transfer rules
5. Maintain InfoCube and Update Rules
6. Create InfoPackage
7. Schedule Flat file using extractor
2. Define Source System in AWB
Step 1 Identify the DataStep 1 Identify the Data
8/2/2019 86606 Bi 1003 - Data Extraction - V2
125/153
India SAP CoE, Slide 125
p yp yGo to MS Excel.
Create a flat file and
enter the records to
be updated in the file.
Save the file as a
.CSV file.
NOTE: The order of
the fields in the flat file
should be same as
that in thecommunication
structure of the
InfoSource.
Step 2 - Create a SourceStep 2 - Create a Source
8/2/2019 86606 Bi 1003 - Data Extraction - V2
126/153
India SAP CoE, Slide 126
pp
SystemSystemRSA1RSA1
ModellingModelling
Source SystemsSource Systems
Context MenuContext Menu
CreateCreate
Select the SourceSelect the Source
System as FileSystem as File
System .System .
Flat File ExtractionFlat File Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
127/153
India SAP CoE, Slide 127
After creating the Source System, create the Infoobjects inAfter creating the Source System, create the Infoobjects in
the BW system.the BW system.
Create an InfoSource and assign the flat file Source SystemCreate an InfoSource and assign the flat file Source System
to it and maintain the transfer rules.to it and maintain the transfer rules.
Create and ODS and maintain update rules.Create and ODS and maintain update rules.
Now create an infopackage and load the data.Now create an infopackage and load the data.
Hints while loading flat fileHints while loading flat file
8/2/2019 86606 Bi 1003 - Data Extraction - V2
128/153
India SAP CoE, Slide 128
gg
Data separator
File in CSV
format
Give the name of
the CSV file
which you have
created in theinfopackage.
Use the data
separator as ;
Hints while loading flat fileHints while loading flat file
8/2/2019 86606 Bi 1003 - Data Extraction - V2
129/153
India SAP CoE, Slide 129
gg
Be sure the delimiter in the InfoPackage is given as ; Be sure the delimiter in the InfoPackage is given as ;
You might get an error on alpha conversion. Take in mind theYou might get an error on alpha conversion. Take in mind theoption in transfer rule to convert the data from external formatoption in transfer rule to convert the data from external format
into internal.into internal.
Hints while loading flat fileHints while loading flat file
8/2/2019 86606 Bi 1003 - Data Extraction - V2
130/153
India SAP CoE, Slide 130
gg Calendar day should be ofCalendar day should be of
the format yyyymmddthe format yyyymmdd(SAP requires this format).(SAP requires this format).
Sometimes you might getSometimes you might getan error like the systeman error like the systemcannot open the file. Trycannot open the file. Try
keeping the file in Mykeeping the file in MyDocuments or in C drive.Documents or in C drive.
View the preview option inView the preview option inthe InfoPackage beforethe InfoPackage beforeloading the fileloading the file
Set the number of headerSet the number of headerrows to be ignored to 1.rows to be ignored to 1.This ignores the headingThis ignores the headingrow in the flat filerow in the flat file
DB Connect, UD Connect andDB Connect, UD Connect and
SOAP B dSOAP B d
8/2/2019 86606 Bi 1003 - Data Extraction - V2
131/153
India SAP CoE, Slide 131
SOAP BasedSOAP BasedBI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
Steps involved for DB ConnectSteps involved for DB Connect
8/2/2019 86606 Bi 1003 - Data Extraction - V2
132/153
India SAP CoE, Slide 132
1. Create a DB Connect Source System
3. Create InfoSource and assign Source System
4. Maintain communication structure and transfer
rules
5. Maintain InfoCube and Update Rules
6. Create InfoPackage
7. Initialize Delta Update
2. Create a DataSource
Steps involved for DB ConnectSteps involved for DB Connect
8/2/2019 86606 Bi 1003 - Data Extraction - V2
133/153
India SAP CoE, Slide 133
p
Create a DB Connect Source SystemCreate a DB Connect Source System
Create a DataSourceCreate a DataSource
Create InfoSource and assign the Source System created toCreate InfoSource and assign the Source System created to
it.it.
Create InfoPackage and load the data.Create InfoPackage and load the data.
Step 1 - Create a DB
C t t
8/2/2019 86606 Bi 1003 - Data Extraction - V2
134/153
India SAP CoE, Slide 134
Connect source systemDB Connection : Give the name of the
connection.
DBMS : Enter the database type to
which you want to connect in this
field. (i.e. Oracle, DB2 etc)
Enter the username and password.
Conn. Info : This field contains the
technical information that is analyzed
in native SQL in the CONNECT
statement in order to open the
database connection. e.g. In Oracle
enter the name of the database you
want to connect
You use the context menu for a
Step 2 - Generate
DataSource
8/2/2019 86606 Bi 1003 - Data Extraction - V2
135/153
India SAP CoE, Slide 135
Taking over Metadata
via DB Connect
You use the context menu for a
database Source System
Select Database Tables togenerate a DataSource for
database source systems.
First, you choose a selection of
tables for a database source
system and create a connection
to the database source system.
Next, you select the table fields
for a specific table of the
database Source System, andspecify whether you want these
table fields to be available for
selection in the BW system.
DataSource
Step 3 - Assign the
D t S t th I f S
8/2/2019 86606 Bi 1003 - Data Extraction - V2
136/153
India SAP CoE, Slide 136
Based on the
generated
DataSources for
external DB
views/tables
InfoSources in BW
can be established
as usual.
Dataflow is nowpossible!
DataSource to the InfoSource
DataSource EnhancementDataSource Enhancement
8/2/2019 86606 Bi 1003 - Data Extraction - V2
137/153
India SAP CoE, Slide 137
BI Content Extractors
LO Cockpit
FI-SL Extractors
CO-PA ExtractorsGeneric Extractors
Flat File Extraction
DB Connect, UD
Connect and SOAPbased
Datasource Enhancement
Datasource Enhancement viaDatasource Enhancement via
8/2/2019 86606 Bi 1003 - Data Extraction - V2
138/153
India SAP CoE, Slide 138
Customer ExitCustomer Exit
The different customers exits for enhancements ofThe different customers exits for enhancements oftransaction data, master data attributes, texts and hierarchytransaction data, master data attributes, texts and hierarchyDataSources areDataSources are EXIT_SAPLRSAP_001 : Transaction data DataSourceEXIT_SAPLRSAP_001 : Transaction data DataSource
EXIT_SAPLRSAP_002 : Master data DataSourceEXIT_SAPLRSAP_002 : Master data DataSource EXIT_SAPLRSAP_003 : Text DataSourceEXIT_SAPLRSAP_003 : Text DataSource EXIT_SAPLRSAP_004 : Hierarchy DataSourceEXIT_SAPLRSAP_004 : Hierarchy DataSource
Datasource enhancement StepsDatasource enhancement Steps
i l di l d
8/2/2019 86606 Bi 1003 - Data Extraction - V2
139/153
India SAP CoE, Slide 139
involvedinvolved
1. Check whether required field is available in
extract structure
3. If not available proceed with enhancement
4. Enhance extract structure with append structure
5.Unhide the custom field
6. Write appropriate customer exit for enhancement
2. If available include the same in extract structure
Step 1 - Check for availability inStep 1 - Check for availability in
8/2/2019 86606 Bi 1003 - Data Extraction - V2
140/153
India SAP CoE, Slide 140
Extract StructureExtract Structure
Applicable for DataSources where required field is available butApplicable for DataSources where required field is available but
not included by defaultnot included by default
Pre-requisite is empty delta queue and set-up tablesPre-requisite is empty delta queue and set-up tables
Step 2 - Enhance ExtractStep 2 - Enhance Extract
StructureStructure
8/2/2019 86606 Bi 1003 - Data Extraction - V2
141/153
India SAP CoE, Slide 141
StructureStructure Find the name of the extract structure for DataSourceFind the name of the extract structure for DataSource Namespace for custom field is YY or ZZNamespace for custom field is YY or ZZ
Activate append structureActivate append structure
For currency/quantity fields, the reference field should alsoFor currency/quantity fields, the reference field should also
be present in the extract structurebe present in the extract structure
Step 3 - Unhide the added fieldStep 3 - Unhide the added field
8/2/2019 86606 Bi 1003 - Data Extraction - V2
142/153
India SAP CoE, Slide 142
Note that Field Only is checkedNote that Field Only is checked
Incase field is required for selection, check SelectionIncase field is required for selection, check Selection
Step 4 - Write the logicStep 4 - Write the logic
8/2/2019 86606 Bi 1003 - Data Extraction - V2
143/153
India SAP CoE, Slide 143
Go to CMODGo to CMOD
Enter Project name and select componentEnter Project name and select component
Select appropriate Customer ExitSelect appropriate Customer Exit
Write the LogicWrite the Logic
8/2/2019 86606 Bi 1003 - Data Extraction - V2
144/153
India SAP CoE, Slide 144
Write the ABAP code for populating the custom fieldWrite the ABAP code for populating the custom field
Test Enhancement in RSA3Test Enhancement in RSA3
Replicate DataSource in BWReplicate DataSource in BW
Data ExtractionData Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
145/153
India SAP CoE, Slide 145
1 PrepareMe
2 TellMe
3 ShowMe
4 LetMe
5 HelpMe
Flat File ExtractionFlat File Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
146/153
India SAP CoE, Slide 146
Create the InfoObjects document number, order no, name ofCreate the InfoObjects document number, order no, name ofthe material, cost, currency, order creation date in the BWthe material, cost, currency, order creation date in the BWsystem.system.
Create a flat file containing data for above InfoObjectsCreate a flat file containing data for above InfoObjects Save the file in the CSV format.Save the file in the CSV format. Now create a flat file source system and InfoSource andNow create a flat file source system and InfoSource and
assign the Source System to the InfoSource.assign the Source System to the InfoSource. Create an ODS with document number as key field andCreate an ODS with document number as key field and
include the above InfoObjects.include the above InfoObjects. Create the update rules from the InfoSource you haveCreate the update rules from the InfoSource you have
created.created. Create an InfoPackage and load the file.Create an InfoPackage and load the file.
BI Content ExtractionBI Content Extraction
8/2/2019 86606 Bi 1003 - Data Extraction - V2
147/153
India SAP CoE, Slide 147
In the Source system go to transaction RSA5, activate theIn the Source system go to transaction RSA5, activate the
DataSource 2LIS_11_VAITM.DataSource 2LIS_11_VAITM.
Go to BI Content select the grouping as data flow before andGo to BI Content select the grouping as data flow before and
install the InfoSource 2LIS_11_VAITM.install the InfoSource 2LIS_11_VAITM.
Now replicate the DataSource in BW and assign it to theNow replicate the DataSource in BW and assign it to the
InfoSource and maintain transfer rulesInfoSource and maintain transfer rules
Create an ODS and update rulesCreate an ODS and update rules
Create the InfoPackage and load the ODSCreate the InfoPackage and load the ODS
LO CockpitLO Cockpit
8/2/2019 86606 Bi 1003 - Data Extraction - V2
148/153
India SAP CoE, Slide 148
Go to transaction SBIW. Check out the various options toGo to transaction SBIW. Check out the various options to
delete and fill the set up tables and simulate the batch jobsdelete and fill the set up tables and simulate the batch jobs
Go to transaction LBWE, check the option to maintain theGo to transaction LBWE, check the option to maintain the
datasources and to activate them as well as to simulate thedatasources and to activate them as well as to simulate the
V3 updateV3 update
TransactionsTransactions
8/2/2019 86606 Bi 1003 - Data Extraction - V2
149/153
India SAP CoE, Slide 149
RSA1 Administrator Workbench
RRMX Business Explorer ( BEX )
RSO2 Maintain Generic Datasource
RSA3 Extractor Checker
RSA7 Delta Queue Status
RSA6 Maintain Datasources
RSQ02 Maintain InfoSets
RSBOH1 Open Hub Destination
TransactionsTransactions
8/2/2019 86606 Bi 1003 - Data Extraction - V2
150/153
India SAP CoE, Slide 150
RSMO Monitoring
SBIW BW Config.
OLI*BW Statistical Setup
SM50 Process Overview
RSA5 Install Business Content
RSA6 Post Processing of DataSourcesLBWE Logistics Cockpit
LBWG Deleting Setup tables
LBWF BW Log
SM37 Job Overview
KEB0 Create the COPA Data Source
Course NameCourse Name
8/2/2019 86606 Bi 1003 - Data Extraction - V2
151/153
India SAP CoE, Slide 151
1 PrepareMe
2 TellMe
3 ShowMe
4 LetMe
5 HelpMe
Tips & TricksTips & Tricks
8/2/2019 86606 Bi 1003 - Data Extraction - V2
152/153
India SAP CoE, Slide 152
Before starting an SAP BW project, analyze the reportingrequirements against the standard DataSources available.
Zero-in on the standard DataSources satisfying therequirement.
If data that cannot be supplied by the standard sources, we
can:- Create a generic DataSources.
- Enhance the standard DataSource.
Some Useful sitesSome Useful sites
8/2/2019 86606 Bi 1003 - Data Extraction - V2
153/153
www.sdn.sap.com
www.help.sap.com
www.service.sap.com
Top Related