SAP BW Archiving with Nearline Storage at Esprit TT 2013 Esprit Hahne Consulting V11… · SAP BW...

SAP BW Archiving with Nearline Storage at Esprit

Claudia Ottilige, Esprit Europe GmbH Dr. Michael Hahne, Hahne Consulting GmbH 27. Februar 2013

Agenda

•  Company Esprit •  Initial situation •  NLS Best Practices •  Solution options •  Conclusions •  Project results so far

SAP BW Archiving with Nearline Storage at Esprit 2

Agenda



Esprit Company Overview

•  One of the most well-known apparel brands in the world. •  Founded in San Francisco in 1968 by Susie and Doug Tompkins •  Offering style and quality made to last. Effortless style, authentic and

easygoing living are at the heart of the brand philosophy. •  Company is represented in more than 40 countries •  More than 1,000 directly managed retail stores •  Distributes via more than 10,800 controlled space wholesale point-of-sale •  Total selling space over 1 million square meters. •  Since 1993 listed on the Hongkong stock exchange. •  Group turnover: 30,165 m HK$ (FY 2011/2012) •  Product lines: Casual, Collection, edc, others: accessoires, bodywear, shoes, kids, sports, de.corp •  Distribution channels: Wholesale, Retail, Licences

Company overview


Agenda



6

Background / initial situation (I)

•  Esprit‘s SAP BW has currently 16 TB of data, data volume still growing –  nightly data loading times too long à reports are often available too

late for the users –  potential recovery of the system would take several days –  backup time very long à one night can become too short for a backup –  in case of further growth of data volume à new backup infrastructure

needed –  makes copy of whole production system to test-system inacceptable –  database reorganisation takes very long –  data storage cost could be reduced –  assumption: SAP BW is not meant to archive data relevant for audits,

audit relevant data are stored in source systems


Background / initial situation (II)

During the last 12 months the used dabase size increased from 14,5 to 16 TB. Average growth per month: 125 GB Several housekeeping activities had positive impact on the data volume.


8

Data volume reduction by DVM

Data volume management (DVM) is a concept which tries to keep the data volume of the SAP BW system as small as possible. This includes:

–  deletion of unused data –  deletion of temporary data –  data archiving (ADK) –  near-line storage –  database re-organisation –  aggregation of data –  data compression (e.g. Infocubes) –  processes and guidelines to keep the amount of data as small as

possible –  ...


Archive Development Kit (ADK) versus Nearline Storage (NLS)

ADK (SAP BW 3.X) NLS (SAP NW 7.0 BI)

•  Classical Archiving solution for Infocubes and ODS objects based on Archive development kit (ADK)

•  Archived data must be reloaded into the SAP BW system when needed

•  Cost reduction by using cheaper storage media and through data compression

•  read-only

•  Archiving solution for Infocubes and DS objects

•  Direct access to NLS data for reporting and progagation

•  Archived data does not need to be reloaded into the SAP BW system when needed

•  Cost reduction possible through compression of data

•  NLS data can be stored on file system, DBMS, also in ADK files

•  read-only


Agenda



Data access vs. Data growth

•  Typical Data Growth from year to year

•  Typical Data Access vs. Data Growth

•  With increasing Data Growth access likelihood changes significantly SAP BW Archiving with Nearline Storage at Esprit 11

0.5 TB 1.5 TB 2.5 TB

4 TB

12 TB

100% 90%

75%

30%

10% Frequently read Infrequently read Rarely read

Information Lifecycle Management

Online Database Storage Near Line Storage

Data Archiving

Frequently read/updated data

Infrequently read data

Very rarely read data

•  Online: –  Persistent in the Data Warehouse –  Data Modelling and Layered Architectures are relevant –  Housekeeping

•  Nearline: –  Transparent Access –  „BW on a diet“

•  Offline: –  Classical

Archiving


Roll-Up & Transformation

Process

Roll-Up & Transformation

Process

Data Load Process

Data Marts

Data Acquisition Layer

Data Integration Layer “Ready to use”Data è Feeding the “Data Marts“

Granular, untransformed Data è 1:1 mapping to ERP System

Analytical layer è Mainly aggregated data

50-70% of overall

data volume

Data Management Best Practice: Enterprise Data Warehousing

Archiving mostly daily

Archiving weekly or monthly

Archiving monthly to yearly


Look-Ups in data flows

Data Warehouse Layer

Acquisition Layer

Reporting Layer

History Objects Staging ODS

Look up of historical data in

Update Rules

Nearline - Object

Adhoc reporting, Analysis Process

Designer

… Look-

Up


Look-Up API

NLS - APIDB Interface

Data AccessAPI

Analysis Process

Single Point of access to all data

– archived and non-archived

NLS - APIDB Interface

Data AccessAPI

Analysis Process

Single Point of access to all data

– archived and non-archived


DSO Look-up in Transformations with BW 7.30

•  With BW 7.30 there‘s a DSO look-up as Transformation Rule available

•  Reads also NLS partitions automatically


7.30

NLS Access in Analysis Processes with BW 7.30

•  With BW 7.30 the Source-Typ DSO in Analysis Processes reads the NLS Partitions automatically if required


Data from NLS can be included as option

7.30

Data Management Best Practice: Integration of NLS in Reporting Layer


Data Management Best Practice: Transparent Access in Queries with NLS


NLS Support für MultiProvider with 7.30

NearLine Readmode •  disabled at all •  enabled at all •  InfoProvider settings

7.30


NLS Properties in a Query with 7.30

Fixed NLS Settings •  read NLS •  do not read NLS •  see InfoProvider settings

7.30


NLS Query Variables with BW 7.30

Variable NLS Settings •  read NLS •  do not read NLS •  see InfoProvider settings


Advantages of a Data Volume Management Strategy

•  Performance –  Faster Data Loads –  Faster Queries

•  Costs –  Less storage costs –  Less administrative effort –  controlled system growth

•  Availability –  of data:

faster rollups and change runs etc. –  of system:

less downtime for backups, upgrades etc. •  Enterprise Data Warehousing

–  „Data Marts on Demand“: more flexibility, faster rollout


Agenda



SAP Defined Integration Scenario NW-BI-NLS 04S - Nearline Storage 2004s for SAP NetWeaver BW

Certified Partners (June 2012)

(see also http://www.sap.com/ecosystem/customers/directories/SearchSolution.epx )

Informatica ILM Nearline for SAP BW (Former SAND Nearline)

•  Reduce production storage costs by moving infrequently accessed data to a nearline data store.

•  Does not require any database administration.

•  Data is stored in encrypted files on standard file system, simplifying backup and restore processes as well as replication to other environments.

•  Reduce non-compliance risk with continued access to data in immutable, compressed files - compression rates as high as 98%, with average of 90%+ compression rate.

•  Contact: [email protected]

SAP NetWeaver BW Nearline Storage on DataVard OutBoard™

•  DataVard‘s OutBoard™ is the all-in-SAP nearline storage solution •  Built in ABAP, runs in your existing SAP landscape, no additional database or

hardware required •  OutBoard™ is priced per productive system landscape. No matter how much data

you load onto it. •  Compresses all InfoProviders, PSA and

Changelogs up to 95% •  Contact: [email protected]

Dat

a Fl

ow C

ontr

ol /

Proc

ess

Cha

ins

Enterprise Query, Reporting & Analysis

Enterprise Data Warehouse

BW

Acc

eler

ator

Met

a D

ata

Rep

osito

ry /

Doc

umen

ts

Info

Obj

ects

/ M

aste

r Dat

a

Out

Boa

rd™

N

ear-

Line

Sto

rage

DataSource / PSA

Data Store Object

(volatile) Data Warehouse Layer (historical)

SAP NLS Interface

(Architected) Data Marts

Analytical Engine

Change Logs

…

Database

SAP NetWeaver BW Nearline Storage on PBS CBW •  With CBW PBS provides an add-on solution for the data retrieval of archived/

nearlined BW data. •  On the basis of the SAP ADK files, the CBW solution creates aggregates on the

archive data and (optionally, if needed) index structures to enable a fast retrieval option for the archived data directly from the end-user's original query.

•  PBS CBW NLS IQ for Sybase Analytics Server IQ utilizes the Sybase Analytics Server IQ as a nearline database and offers extremely fast query response times.

•  Contact: [email protected]

Agenda



Conclusions (I)


During the next 5 years, approx. 4 – 7 TB of data can be archived only for the integration layer. (data vary because of co-existance phase of two ERP systems)

In addition to NLS data, ~ 3 GB could be stored via ADK files, as data are older than requested retention times.

vs. PY: ∑Storage:

Conclusions (II)

§ 

Integration layer is the main focus for archiving. Reporting layer is already supported by BWA to increase performance, archiving of data is not a must (alternative: keep only aggregated data for historical data). PSA and extraction layer are deleted very promptly.


Esprit‘s Evaluation (criteria)

SAP BW Archiving with Nearline Storage at Esprit

32

To compare different providers the following evaluation criteria have been applied:

•  Is separate hardware necessary? •  How is the Query performance in comparison to BWA? •  How large is the compression rate? •  How well does the software assists with the archiving process? •  Reference customers •  Support via OSS •  Compatibility with HANA •  Architectural fit with other archiving tools already available at Esprit •  Is it possible to access productive data from the test system? •  Company background

1.  Data archiving / storing DSO & Cubes in NLS

2.  Loading of data via DTP 3.  Reporting on NLS data 4.  Reloading of requests 5.  Usage in process chains

1.  OutBoard™ Cockpit §  OutBoard™ Browser §  OutBoard™ Analyzer §  Statistics

2.  Performance-optimization for fast access to NLS Data

3.  Mass archiving / Automation 4.  Storage Management (via

separate Tablespaces a.o.) 5.  Archiving Request deletion 6.  Archiving PSA & Change log

Datavard OutBoard

SAP NLS Interface OutBoard™ Features

ADK vs. NLS To have 2 archiving systems available increases complexity and cost therefore we decided to only implemented NLS solution. Motivation:

§  NLS is necessary anyway: •  fast response to new reporting requirements •  quick reload of cubes •  quick reload of data for realignments •  look-ups on archived data during data load possible

§  In case amount of data needs to be further reduced also in Reporting layer (e.g. because BWA is at it‘s limits) there are 2 possible strategies how to deal with it using the Datavard NLS solution:

à either data model can be adapted (aggregation of data) OR à data are saved in NLS, but are not made available for reporting (can be quickly retrieved if absolutely necessary à but probably performance issue)


Agenda



36

Project results up to now (I)

•  Several housekeeping activities implemented in 2011 •  Several workshops regarding archiving concept in 2011 •  Data volume scoping by SAP in October 2011 •  Analysis of SAP BW system by SAP in February 2012 •  Data volume management workshop with SAP (1 day) in Feb. 2012 •  Defined & prioritized workpackages to reduce SAP BW data

volume •  Implemented some workpackages (e.g. deleted some large unused

DSO‘s)


37

Project results up to now (II)

•  defined all objects to be archived, distinguishing between: –  Archiving development kit (ADK) or –  Nearline storage

•  defined business requirements for data retention times •  Near-line Storage

–  presentations by different nearline storage solution providers –  received offers from solution providers –  workshop with nearline storage expert –  telephone call with reference customer –  checked already available SAP archiving solutions at Esprit Europe (PBS

solution for current FI/CO system) •  Calculated business case for NLS solution •  Installed Outboard Software •  Started with archiving of first DSO‘s in October 2012


38

Summary

•  Near-line storage is state-of-the-art alternative to classical archiving

•  NLS can solve various problems: –  Load run time –  Query performance –  Data volume and storage costs –  Backup and recovery

•  SAP data volume management workshop identifies the biggest potential issues and provides recommendations

•  Quick return on investment (often less than a year)


26.02.13 Copyright Hahne Consulting GmbH 2013 39

How to get started…

•  Analysis Workshop / Fitnesstest –  Objective: identify „Quick Wins“ / low hanging fruits –  Benchmarking and Recommendations in areas of

•  System Performance and Robustness •  Data Management •  Housekeeping

–  ROI in a couple of months •  Proof of Concept

–  Test installation –  Proof of Concept

•  Project and usage –  Analysis and design –  Implementation –  Operation –  Check-up with Fitnesstest

40

Questions?


SAP BW Archiving with Nearline Storage at Esprit TT 2013 Esprit Hahne Consulting V11… · SAP BW...

Documents

Transcript of SAP BW Archiving with Nearline Storage at Esprit TT 2013 Esprit Hahne Consulting V11… · SAP BW...