SAP NetWeaver BW Near-Line Storage (NLS) …...BW powered by HANA and Sybase IQ Near-Line Storage...

Public

Rainer Uhle

Product Management SAP EDW (BW / HANA), SAP SE

SAP BW 7.40 Near-Line Storage for SAP IQ –

What's New?

© 2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Public

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a

purchase decision. This presentation is not subject to your license agreement or any other agreement

with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and SAP's

strategy and possible future developments are subject to change and may be changed by SAP at any

time for any reason without notice. This document is provided without a warranty of any kind, either

express or implied, including but not limited to, the implied warranties of merchantability, fitness for a

particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this

document, except if such damages were caused by SAP intentionally or grossly negligent.


SAP BW on HANA – Smarter, simpler, more efficient How Does BW running on RDBMS differ from BW running on HANA ?

Customer value of SAP BW powered by

SAP HANA

Excellent query performance

for improved decision making

Performance boost for Data Load processes

for decreased data latency

Accelerated In-Memory planning capabilities

for faster planning scenarios

Flexible combine EDW with HANA-native

data for real-time insights and decision making

Data persistency layers are cut off and

reduced administration efforts

Simplified data modeling and remodeling

Data intensive functions are pushed down

from BW to HANA

HANA Stack

RDBMS

Traditional Stack

SAP NetWeaver BW

Data Modeling

Planning

Data Management

OLAP Pro

ce

ss

Orc

hes

trati

on

Data Schema

&

Data

SAP BW on HANA

Data Modeling

Planning

Data

Management

OLAP Pro

ce

ss

Orc

hes

trati

on

Push Down

HANA as the Primary Database for BW and

Foundation for new Applications

any NLS

SAP

IQ-NLS


• Data is read and/or written frequently

• In HANA DB, Full In-memory handling

• Accessibility: No restrictions, all features available

• Infrequent access

• In HANA DB, on disk, Main memory for caching and processing • Accessibility: No restrictions, all features available

• Sporadic access

• Not stored in HANA DB; stored in SAP IQ Near-line DB

• Accessibility: Restricted to BW-NLS capabilities, read-only

Differentiations: Multi Temperature Data Approach

HANA Dynamic Tiering (fka Extended Storage)

Providing lower TCO by optimized data volume management

hot

warm

cold

BW-NLS Management for read-only data


BW powered by HANA and Sybase IQ Near-Line Storage (NLS) Architecture - Overview

BI Clients

SAP

HANA

MultiProvider Transient

Provider InfoCube/DSO

Near-Line SDK

SAP Netweaver BW 7.3x

NLS-

Partner

OEM

BW NLS4IQ

SAP Native

An SAP - owned BW NLS implementation for

Sybase IQ offers a fully integrated solution for

combined primary & secondary database

management:

• Primary Database (preferable SAP HANA)

• active data, maximum performance,

transactional changes, high frequent

access, very short MTTR*

• Secondary Database (always SAP IQ)

• Slave (detouchable without breaking

the whole system), non-active data,

lower performance accepted, only bulk

loads at scheduled PITs*, entirely read-

only, low frequent access, longer MTTR

acceptable

*) MTTR – Mean Time To Recover

PIT - Point in Time

SAP owned alternative to

existing NLS-Partner Solutions

Architecture


SAP HANA Database

Hot data

Introducing SAP HANA dynamic tiering Map data priorities to data management

Warm data

Primary Image

in memory

Durability

Cache /

Processing

Primary

Image on

disk

Dynamic Tiering

All in one

database

Hot Store

Classical HANA tables

Primary data image in memory

DB algorithms optimized for in-memory data

Persistence on disk to guarantee durability

Warm Store

Extended Tables

Primary data image on disk

Data processing using algorithms optimized for disk-

based data

Main memory used for caching and processing.

Hot Store Warm Store

RAM


Decision Matrix Extended Table vs. NLS on InfoProvider Types

Supported

InfoProvider

Types per

‚temperature‘

InfoCube Standard

DSO w/o DSO PSA

HOT not

recommended

not

recommended

WARM

currently not

supported

currently not

supported

COLD not supported

by SAP (PBS)

HANA

Archive

Advanced DSO

will replace

InfoCubes and

Standard DSOs

supporting

Dynamic Tiering/

Extended Tables


NLS - Interface

DAP – Design Time

• Define partitioning criteria

• Generate DAP NLS

Implementation

NLS Scenario – Design and Runtime Aspects

DBMS

InfoProvider

BW

DB

Q3/2010 Q4/2010

Q1/2010 Q2/2010

Q1/2011 Younger …

Data Archiving

Process (DAP)

• Query access to NLS

• DTP access to NLS

• Look up during Transforms

Add. Features

DAP – Scheduling /

Monitoring / Run-Time: • Describe partition

to be extracted

• Generate request

• InfoProvider

Manage Menu

• Process Chain

• Archive / Extract (DAP menu,

Process Chain

• Restore (DAP menu)

Mass

Maintenance Tool

for

DAPs, Variants,

Process Chains

available

(SP08)


Data Relocation to NLS takes place in three phases

Copy Phase:

All data of a request is copied in individually committed packages to SAP IQ

• Data is pulled from BW database

• Data is transferred across the network and loaded into NLS using IQ client side

load

• After the last data package is loaded the final transaction commit id is retrieved

• IQ Backup Event is Triggered to capture all the data that was archived

Verification Phase:

Technical key of archived records is retrieved and verified for completeness

Deletion Phase:

Before data is deleted from primary database

• Deletion proceeds only if the latest backup includes all the data from the copy

phase

• Data is deleted from the BW database and Archived Data in NLS becomes active

• IQ Backup Event is Triggered to capture the end state of the NLS database

Example for 3(4)-fold Archiving Process on single Object

Throughput

Up to

250 Mio.

rows/h

(100 GByte/h)

Load Performance

Improvements under

continous

investigation …

(still potential for

improvements)


NLS Implementation

Main Principle: Relocation - Cold NLS Data is write-protected …

RDBMS

InfoProvider

BW

Source

Q3/2010 Q4/2010

Q1/2010 Q2/2010

Q1/2011 Newer…

for Locked

update

Q3/2010

NLS - Interface

Updates for archived

records in the period

Q1 to Q3 2010 can be

managed using DTP

Error Handling after

NLS reload

Update

capabilities

coming soon!!

(‚Straggler

Management‘)


BW NLS in action - Runtime Experience

… read NLS as well

Acceptable

NLS ‚Latency

Factor‘

Further NLS Read

Performance

Improvements

based on

HANA SDA

available

(SP08)


Planned Innovations Future Direction Today

BW NLS Enhancements

NLS support for BW Inventory Cubes

Generated HANA Models of an InfoProvider

can access data in NLS

BW on HANA Query Optimizations

Further improvement of NLS Query performance

due to SDA optimizations (SDA Virtual Tables in

HANA CalcViews)

Extended Pruning for time characteristics

BW NLS Implementation Support

Mass maintenance Support (BW 7.40 SP 8) for

generating DAPs, DAP Variants, Process Chains

Further improvements on NLS operations

DBA Cockpit for integrated Sybase IQ Monitoring

(BW 7.40 SP 8 + downports up to NW 7.02)

New nearline connection mode

– Optimized handling of attached NLS in BW

system copy or migration process to HANA

– Improved operations when nearline data is

unavailable

SAP NetWeaver BW Product Roadmap Focusing – BW NLS Implementation for Sybase IQ


Optimized performance for NLS read purposes

(HANA SDA with BW 7.4 SP05)

Supported Platforms Server/Client

Sybase IQ 16.0 SP3

Sybase IQ ODBC NLS client for BW AppServer

Windows on x64 64bit, Linux (x86_64 64bit),

AIX 64bit, HP-UX, Solaris, OS/400

Error DTP Handling in case of NLS caused lock

violations

Support for Standard DSO with

BW 7.3 SP11 (resp. Note 1931784)

IQ Backup Automation

Event trigger for incremental backup in IQ after

data relocation to NLS (BW 7.0 SP32)

BW NLS Downport to BW 7.0

BW 7.0 SP32 (CW2/2014)

BW 7.0 SP28 with note 1858550

Lookup Performance (SP10)

Optimizations for lookup to archived data

using the Nearline Interface API

Throughput optimizations in archiving (SP11)

New parallelization option copy, verification

and deletion phase of a Data Archiving

Process (DAP)

High Performance Data Relocation through

multi-threaded execution of the LOAD

statement within the SAP IQ server

NLS support for additional BW InfoProviders

(planned for next BW Feature Pack)

Supporting the Advanced DataStore Object

BW NLS Enhancements (planned for next BW

Feature Pack)

Integrated handling of exceptional updates to

archived nearline time slices within BW

Processes (e.g. Advanced DataStore Object

Activation)


SAP HANA Smart Data Access and SAP BW NLS

Enhanced Business Flexibility by providing “the

logical EDW”

Data Federation in diverse EDW landscapes

Smart data access – read access to relational and non-relational

sources via ODBC

Supports data location agnostic development

No special syntax to access heterogeneous data sources

BW based Analytic Services on external data

Implemented in BW on HANA for NLS in IQ

Scenario

Make other DWHs transparent to HANA

Non-disruptive evolution from virtual table to persistent structure by

establishing ETL without major effort

Consolidating / rationalizing the DWH landscape

Consumption of HANA datamart scenarios from second HANA

database

Optimization of NLS Queries with filters on Navigational

Attributes

HANA Smart Data Access Layer

Query

BW Virtualization Layer

Composite Provider, Open ODS View

SAP HANA

Hadoop

Teradata ASE

Virtual Tables HANA Tables

IQ MSSQL

Oracle


HANA Smart Data Access for BW NLS with SAP IQ BW7.40 SP08 (RTC Sept 2014)

VirtualTable can be used in

HANA CalcView

• Initial version w/ HANA SP07

• Full plan optimizations w/ HANA SP08

BW Query execution can push OLAP

calculations to HANA (Restricted

KeyFigures, Hierarchies, …)

Further improvement of NLS Query

performance due to optimized access

SAP IQ

Facts (nearline)

HANA

BW

VIRTUAL_TABLE Facts (online) Master

data

Analytic Query

NLS-Split

CalcScenario

API API


Runtime Example from E-Procurement with 40% NLS plus SDA

Less than 500

ms runtime

with 40% NLS 669 ms

drill down into

NavAttr Region

Reduced

network traffic

for

NavAttr Region


Runtime Example from E-Procurement with 40% NLS w/o SDA

Much more

traffic and

runtime

w/o HANA SDA



BW NLS Enhancements







HANA CalcViews)












unavailable






Sybase IQ 16.0 SP3





violations


BW 7.3 SP11 (resp. Note 1931784)





BW 7.0 SP32 (CW2/2014)








Process (DAP)








Feature Pack)




Activation)








Sybase IQ 16.0 SP3





violations


BW 7.3 SP11 (resp. Note 1931784)





BW 7.0 SP32 (CW2/2014)


BW NLS Enhancements







HANA CalcViews)





based on query selection statistics



(BW 7.40 SP8 + downports up to NW 7.02)





unavailable







Process (DAP)


optional multi-threaded execution of the

LOAD statement within the SAP IQ server





Feature Pack)




Activation)


Key Points to take home …

1. Near-line Storage helps to reduce TCO and gives seamless access to massive

amounts of aged resp. inactive data to reveal trends, patterns, correlations

2. With its columnar store and compression mechanisms, SAP IQ ideally complements

SAP HANA as NLS store

3. Finally NLS also supports any kind of inventory handling in BW and opens up a new

category of high volume InfoProviders for NLS based volume optimization

4. SAP HANAs Smart Data Access capabilities guarantee optimized performance for NLS

query access even in case of complex filter criteria.

5. The read-only restriction for NLS data can soon be temporarily suspended for

exceptional update operations.

6. With the latest enhancements to their NLS solution SAP competes against all existing

NLS solutions of our certified NLS Development Partners.

SAP NLS is optimized for BW on HANA.


SAP BW Near-Line Storage RDS Solution scope covered

DAP = Data archiving process

DAP 1

DAP … N

Process Repeatable result

Install/configure

SAP Sybase IQ

Configure SAP NW BW

to use SAP Sybase IQ

as near-line storage

Implementation of data

aging strategy in SAP NW

BW for historical data

Archive historical

data to near-line

storage

Delete already archived

data from online SAP

NetWeaver BW system

Update support

package to SAP BW

7.3x or higher

The following diagram shows the scope and process flow of the solution:

Install and configure SAP Sybase IQ

Configure Near-Line storage

Implement data aging strategy in SAP NetWeaver BW for historical data.


Services Scope Overview Go live in as little as 12 weeks based on implementation options

A special step-by-step guide describes each activity

during the deployment

Service Name: Rapid deployment of SAP NetWeaver BW Near-Line Storage

Implementation Duration: 12 Weeks*

* Based on scope

** Provided as option and impacts overall efforts of the service

Core Offerings

Implementation of SAP NetWeaver BW Near-Line Storage

Identification of objects to be archived

Installation and Configuration of SAP Sybase IQ database

Installation of SAP Sybase IQ Client and configuration of

SAP NetWeaver BW to Near-Line Storage

Create up to 10 DAPs (Data Archiving Processes)

Create up to 2 Process Chains

Adjust settings for up to 10 Queries and 2 multiproviders

Optional Task**

Support Package Update


SAP BW NLS RDS Implementation Partners

• Former SAP BW NLS Development Partners also

offer implementation services for SAP BW Near-line

rapid-deployment solution

• Partner specific add-ons available on top

(e.g. NLS storage management, migration options)

• … have a look into SAP Store for more details under

store.sap.com

store.sap.com


Nearline Storage (NLS) with SAP IQ Specific features of SAP’s Nearline Solution for BW based on SAP IQ

© 2014 SAP SE or an SAP affiliate company. All rights reserved. 25

Available

• Optimized query performance when accessing NLS IQ data through HANA

Smart Data Access (SDA)

• NLS support for BW Inventory Cubes

• Automated mass generation of Data Archiving Processes based on query

selection statistics

• Generated HANA Models of an InfoProvider can access data in NLS

• Optimized lookup to archived data in NLS IQ using the Nearline Interface

API

• DBA Cockpit for integrated Sybase IQ Monitoring

• Backup Automation: Event trigger for incremental backup in IQ after data

relocation to NLS

Planned

• High Performance Data Relocation through optional multi-threaded

execution of the LOAD statement within the SAP IQ server (SP11)

• Integrated handling of exceptional updates to archived nearline time slices

within BW Processes (e.g. Advanced DataStore Object Activation)

SAP HANA

SAP BW

Interface (Advanced)

DataStore

Objects

InfoCubes

SAP

IQ

NL

S In

terf

ace


© 2014 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an

SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE

(or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark

information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its

affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or

SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing

herein should be construed as constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or

release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future

developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any

reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking

statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue

reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

SAP NetWeaver BW Near-Line Storage (NLS) …...BW powered by HANA and Sybase IQ Near-Line Storage...

Documents

Transcript of SAP NetWeaver BW Near-Line Storage (NLS) …...BW powered by HANA and Sybase IQ Near-Line Storage...