SAP HANA SPS09 - Dynamic Tiering

1 © 2014 SAP SE or an SAP affiliate company. All rights reserved.

SAP HANA SPS 09 - What’s New? HANA Dynamic Tiering

SAP HANA Product Management November 2014

(Delta from SPS 08 to SPS 09)

© 2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Public

Disclaimer

This presentation outlines our general product direction and should not be relied on in making

a purchase decision. This presentation is not subject to your license agreement or any other

agreement with SAP.

SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and

SAP’s strategy and possible future developments are subject to change and may be changed

by SAP at any time for any reason without notice.

This document is provided without a warranty of any kind, either express or implied, including

but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or

non-infringement. SAP assumes no responsibility for errors or omissions in this document,

except if such damages were caused by SAP intentionally or grossly negligent.


Agenda

Positioning

What is “SAP HANA Dynamic Tiering”, and what is its value to the customer?

Technical Details

Implementation choices

Use Cases

SAP BW and native HANA applications

Future Direction

Where are we headed?

Positioning What is “SAP HANA Dynamic Tiering”, and what is its value to the customer?


IDC predictions for 2014

Data explosion Data volumes will continue to explode to 6 billion petabytes

Social networking Social networking will become embedded

in cloud platforms and most enterprise

apps and processes

Cloud Cloud spending will surge by 25%, reaching

over $100 billion. There will be a doubling of

cloud data centers.

Internet of Things 30 billion devices, sensors in 2020 – driving

$8.9 Trillion in revenue

Mobile

CRM Data

Planning

Opportunities

Transactions

Customer Sales Order

Things

Instant Messages

Demand

Inventory

Big Data

Sales

Order

Things

Mobile Demand

Big Data

CRM Data

Customer Planning Transactions


SAP End to End Data Management for Real Time Business

Business & Consumer Applications

Big Data

SAP DATA MANAGEMENT

STORE TRANSACT PREDICT ANALYZE

Custom Development

ISVs & OEMs ERP

Internet of

Things

Workforce of

the Future

Cloud

Industries


e

SAP HANA platform

Processing Engine

Application Function Lib. & Data Models

Integration Services

SAP HANA PLATFORM Real-time transactions + end-to-end analytics

Operational

Analytics

Big Data

Warehousing

Predictive, Spatial &

Text Analytics

REAL-TIME ANALYTICS

Sense &

Respond Planning &

Optimization

Consumer

Engagement

REAL-TIME APPLICATIONS

SAP ESP

SAP ASE

Replication

Server

SAP SQL

Anywhere

SAP IQ

SAP Data

Services

Extended Application Services

SAP Data Management Portfolio End-to End Data Management & App Platform for Real-Time Business

Database

Services SAP HANA

dynamic tiering


Time Value of Data

Time

Value

Last time

accessed

Value of

immediate

data access

declines

When you

need it again

Archive Access Event

• Regulatory audit

• Business critical reference data

• Source data


Multi-Temperature Storage Options with SAP HANA

Data Temperature Storage Option SAP BW

on HANA

SAP Business Suite on

HANA

SAP HANA

Native

hot SAP HANA

In-Memory

cold

SAP HANA

dynamic tiering (1) (2) Data Aging

(Next Gen ILM) 3 Near-line Storage

(NLS)

frozen Data Archiving

(ADK)

Generally available

Combination not available

1 Early shipment available for SAP BW 7.4; General availability planned Q4/2014

2 General availability with limited scope planned Q4/2014

3 For selected business objects


SAP HANA Dynamic Tiering Key aspects at a glance

Add-on Product to SAP HANA

Manage data of different temperatures

Hot data (always in memory) – classical HANA

Cold data (disk based data store)

Introducing a new type of table:

Extended table – disk-based columnar table

SPS 9 release focus

Operational integration

Common installer

Unified monitoring and administration

Integrated backup/recovery

Initial functional scope

Transparent query processing

Cross-store optimizer

Use extended table in calculation views

Applications manage data temperatures

(no active support for aging)

SAP HANA Database

Data for daily reporting,

other high-priority data

Other data required to

operate the application

Hot

Warm


Introducing SAP HANA Dynamic Tiering Requirements from our customers

Manage data cost effectively, yet with desired performance based on SLAs

Handle very large data sets – terabytes to petabytes

Update and query all data seamlessly via HANA tables

Application defines which data is “hot”, and which data is “warm”

Native Big Data solution to handle a large percentage of enterprise data needs without Hadoop

SAP HANA

hot store

(in-memory)

SAP HANA warm store

(dynamic tiering)

Extended table

(definition)

Extended table

(data)

Fast data movement and optimized push

down query processing

All data of extended table resides in warm store

SAP HANA Database System

Hot table

(definition/data)


Hot/Warm Data Management Questions about SAP HANA Dynamic Tiering

Size and cost constraints may prohibit all in-memory solution

Not all data has the same value

Warm data has lower latency requirements than hot data

Why is warm data management important for SAP HANA?

SAP HANA dynamic tiering utilizes disk backed, smart column store technology based on Intellectual Property from SAP Sybase

SAP HANA dynamic tiering excels at ad hoc queries on structured data from terabyte to petabyte scale

SAP HANA dynamic tiering is a deeply integrated, high performance solution in a single system

Why is SAP HANA dynamic tiering the best solution for warm data

management?

Hadoop has unlimited capacity for raw data processing

Hadoop is best suited for batch processing of raw, unstructured data

Hadoop is an external data store with technical integration into HANA – with higher TCO in order to manage the additional system

What about Hadoop for warm data storage and processing?


SAP HANA Dynamic Tiering Key aspects at a glance

Data in the database

Different data temperatures

Maximum access performance

Hot data - always in memory

Reduced access performance:

Warm data - not (always) in memory

All part of the database’s data image

Data moved out of the database

Different data qualities

Available for read access

BW Near-line storage

Not accessible without IT process

Traditional archive

Data is stored and managed outside of the

application database SAP HANA Database

Data for daily reporting,

other high-priority data

Other data required to

operate the application

Hot

Warm

NLS Data that is (normally) not updated, infrequently accessed

Traditional Archive Data that‘s kept for legal reasons or similar

Externalize


Problems with temperatures There are too many options – across system boundaries

In DB

In memory

No restrictions, all features available

External to DB

Near-line Storage

Read access, no updates

In DB

On disk


hot

warm

cold

???

External to DB

Archive storage

No read access or updates

Performance

and Price Priority and

Data Volume

HANA

Archive

HANA column and

row store

Warm store of dynamic tiering /

Non-Active Data Concept

BW Near-line Storage

Traditional Archive


Problems with temperatures There are too many options – across system boundaries

In DB

In memory


External to DB

Near-line Storage

Read access, no updates

In DB

On disk


hot

warm

cold

???

External to DB

Archive storage

No read access or updates

Performance

and Price Priority and

Data Volume HANA column and

row store

Warm store of dynamic tiering /

Non-Active Data Concept

BW Near-line Storage

Traditional Archive

hot

warm

BW NLS

Archive


SAP HANA Dynamic Tiering Map data priorities to data management

Hot Store

Classic HANA tables

Primary data image in memory

DB algorithms optimized for in-memory data

Persistence on disk to guarantee durability

Warm Store

Extended Tables

Primary data image on disk

Data processing using algorithms optimized for

disk-based data

Main memory used for caching and processing.

SAP HANA Database

Primary image in memory

Durability

Cache / Processing

Primary Image

on disk

Dynamic Tiering

Hot data

Warm data

All in one

database

Hot Store Warm Store

RAM

Technical Details Implementation choices


SAP HANA Dynamic Tiering – one database / one experience for HANA application developers and admins

SAP HANA Dynamic Tiering

Reduced TCO

Optimized for performance

Single database experience

Centralized operational control

Centralized

monitoring /

admin

High speed

data ingest

Common

installer and

licensing

model

Unified

backup and

restore

Integrated

security

Optimized

query

processing

SAP

HANA

Dynamic

Tiering


SAP HANA Dynamic Tiering The overall system layout

SAP HANA with Dynamic Tiering consists of two types of

hosts:

Regular worker hosts (running the classical HANA processes:

indexserver, nameserver, daemon, xsserver,…)

– HANA hosts can be single-node or scale-out; appliance or TDI

“ES host” (running nameserver, daemon, and esserver)

– esserver is the database process of the warm store

One single SAP HANA database: one SID, one instance number

All client communication happens through index server / XS server

Hot Store

Fast data movement and optimized push down query processing

SAP HANA System with dynamic tiering service

Worker

host(*)

Worker

host

Worker

host

Client

Application

Connect

ES host

Column

Table

Row

Table

Extended

Table

Warm Store

Common Storage System

(*) Standby hosts not shown


Database Catalog

HANA Extended Tables

HANA Database

Warm

Store Data

HANA extended table

schema is part of HANA

database catalog

HANA extended table

data resides in warm

store

HANA extended table is a first

class database object with full

ACID compliance

Hot

Store

Table Definition

Data

Table Definition

Classical HANA

column/row table

Extended table

(warm table)


High Speed Data Ingest

Import from CSV files:

IMPORT FROM CSV FILE ‘bigfile.csv’ INTO t1

Bulk array insert:

INSERT INTO t1 (col1, col2, col3...) VALUES (val1, val2, val3...)

High-speed data movement between HANA tables and HANA extended tables:

INSERT INTO t_extended select c1 FROM t_hana

Concurrent inserts from multiple connections:

A HANA extended table may be a DELTA enabled table, which allows multiple concurrent writes

Warm

Extended

Table

IMPORT FROM CSV

FILE ‘data.csv’

INTO t_extended

CSV

DATA

Hot HANA

column Table INSERT...SELECT

Materialization

Data movement between hot and warm store

HANA Database


Optimized Query Processing

Parallel query processing

• Data is pulled from HANA hot store into HANA warm

store query processing engine using multiple streams,

and processed in parallel

Push/Pull query optimization and transformation

• Query operations ship to hot or warm store as

appropriate for native performance

Extended tables may be used in HANA CALC

views

• HANA Calc engine and HANA SQL engine share

extended table query performance optimizations

Joining

Grouping

Ordering

T3 T4 T1 T2


Example Query Plan

select

"account_num",

count(*) as account_count

from

VXM_FOODMART.CUSTOMER C

where

"lname" >= 'Ga' and "lname" < 'Gb'

and exists

(

select *

from

VXM_IQSTORE.PRODUCT P

where

"product_id" = "customer_id"

)

group by

"account_num"

order by

"account_num";

Customer is a native

HANA table in HANA

memory

Product is a HANA

extended table in the

warm store


HANA Monitoring and Administration

HANA Cockpit:

New, web based monitoring and administration

console for HANA Extended Storage

HANA Studio will be used for design and

modeling of HANA extended tables

HANA Cockpit displays status,

CPU/memory/storage resource utilization,

table usage statistics

Provides access to and search of server logs

and custom traces

Shows alerts triggered by extended storage

Enables administration of extended storage:

add and drop storage, or increase size of file


Unified Backup and Restore

HANA backup manages backup of both hot and warm store

Point in Time Recovery (PITR) is supported

Extended

Storage

HANA

Data backups

(manual or

scheduled) Log backups

(automatic, or

none)

Data backup

Log backup System crash

Restore

Time

t1 t2 t3

Data backups with log

backups allow restore

to Point in Time or

most recent state: t1-

> t3

Data backups alone

allow restore to specific

backup only: t1 or t2

Log area

Backup History


High Availability and Disaster Recovery

High availability

Compute node failure will result in failover to standby node (manual for warm store

nodes)

Storage failure will depend on inherent storage vendor disk mirroring and fault

tolerance capabilities

Hot and warm store should use the same storage to facilitate auto-failover in the

future

Disaster recovery

HANA without Dynamic Tiering supports continuous replication to maintain a disaster

recovery site

HANA with Dynamic Tiering will maintain a disaster recovery site through backup and

restore capabilities only

– Disaster recovery through system replication is planned for a future release

– Disaster recovery through storage replication may be added independently from

software releases Classical HANA services

Compute

node

Hot Store

Warm Store Service

Compute

node

Standby

node

Manual

Failover

Standby

node

Warm Store

Auto-

Failover

mirror

mirror


Support in SAP HANA multiple database containers (MDC)

MDC: One SAP HANA system can have multiple tenant databases

Each tenant database can be associated with zero or one extended stores

Each extended store is dedicated to exactly one tenant database

SAP HANA system with MDC and dynamic tiering

Compute node

System Database

Compute node Compute node

Tenant Database <B>

Extended Store Tenant Database <A>

Tenant Database <C>

Extended Store

Classical HANA (single-node or scale-out)

ES Host <B> ES Host <C>

Use Cases SAP BW and native HANA applications


SAP NetWeaver BW powered by SAP HANA Data Classification by Object Type

Frequent reporting and/or HANA-native operations

BW – Operational Data

Data Categories in a BW System

Staging Layer

Analytic Mart

Business Transformation

EDW Propagation

EDW Transformation

Co

rpo

rate

Me

mo

ry

Arc

hiv

e/N

LS

“Old”, “out-of-use” data – Archive, read-only, different SLAs

Limited reporting, limited HANA-native operations

Archived


SAP HANA database

Database Catalog

Extended Tables in HANA BW Use Case: Staging and Corporate Memory

Object Classification in BW

Data Sources and write-optimized DSOs

can have the property “Extended Table”

Generated Tables are of type “Extended”

All BW standard operations supported –

no changes

Only minor temporary RAM required in HANA

InfoCubes and Regular or Advanced DSOs

Generate standard column table

Hot Store Warm store

BW System

Corporate Memory

Write-optimized DSO

Staging Area

Data Source

Table

Schema

Data

PSA Table Table

Schema

Data

Active Table

Data Mart

InfoCube

Table

Schema

Data

Fact Table


SAP HANA Dynamic Tiering for Big Data

Cutting edge, in-memory platform

Transact/analyze in real-time

Native predictive, text, and spatial algorithms

Petascale extension to HANA with disk backed,

columnar database technology

Expand HANA capacity with warm/cool structured

data in HANA warm store

Tight integration between HANA hot store and HANA

warm store for optimal performance

SAP HANA with Dynamic Tiering provides native Big Data solution

Hot data

SAP HANA

Petascale, warm

structured data

HANA extended tables


SAP HANA with Dynamic Tiering Native Big Data solution for a multitude of use cases

SAP HANA Dynamic Tiering for Big Data Use Cases across Industries

Airline route profitability analysis: SAP HANA analyzes revenue, variable operating costs (fuel,

landing fees...), and fixed operating costs in real time to make decisions on network, pricing, and

marketing to determine where to fly, when, and how often. All data must be analyzed in real time.

Financial services: Stock tick data streamed into SAP HANA for immediate price fluctuation

analysis and trading actions, with historical stock price data stored in HANA extended tables for

trend analysis and portfolio management.

Telecommunications: Network service data in HANA extended tables analyzed and correlated

with customer loyalty data in SAP HANA, to anticipate customer churn and initiate customer

retention response activities.

Public utilities: enterprise data stored in SAP HANA and large amounts of smart meter data

stored in HANA extended tables, to identify operational problems, and establish incentive pricing

for more efficient energy use.

Future Direction Where are we headed?


SAP HANA Dynamic Tiering roadmap

SAP HANA dynamic tiering available to be used by any

HANA application (if the application supports the

feature)

Common installer

Unified administration and monitoring using HANA

Cockpit

Extended Storage (ES) engine is part of HANA topology

Single authentication model

Single licensing model

Combined error log / trace handling

Integrated File-based backup/recovery, including point-in

time recovery

HANA ES host scale-out and auto-failover (HA)

Disaster Recovery (SAP HANA system replication)

Further integration with respect to backup/recovery

Hybrid extended tables with rule based automatic data

movement / aging

Optimization of communication between hot and warm

store

Further unification of DDL and DML for HANA

extended tables

Further optimizer enhancements

Further extension of unique HANA capabilities to warm

store

FUTURE PLANNED


Hybrid extended tables

Automatic, rules-based, asynchronous data movement between hot and warm stores

Hot partitions in HANA memory; remaining partitions in warm store

Single HANA table that spans hot and warm stores

Hot data in

HANA tier

Warm data In

warm tier

2012 2012 Hybrid

Extended

Table aging

regulatory

audit


How to find SAP HANA documentation on this topic?

• In addition to this learning material, you can find SAP HANA

platform documentation on SAP Help Portal knowledge center at

http://help.sap.com/hana_platform.

• The knowledge centers are structured according to the product

lifecycle: installation, security, administration, development:

SAP HANA Options

SAP HANA Advanced Data Processing

SAP HANA Dynamic Tiering

SAP HANA Enterprise Information Management

SAP HANA Predictive

SAP HANA Real-Time Replication

SAP HANA Smart Data Streaming

SAP HANA Spatial

• Documentation sets for SAP HANA options can be found at

http://help.sap.com/hana_options:

SAP HANA Platform SPS

What’s New – Release Notes

Installation

Administration

Development

References

•

http://help.sap.com/hana_platform

http://help.sap.com/hana_options

© 2014 SAP SE or an SAP affiliate company. All rights reserved.

Thank you

Contact information

Richard Bremer, Courtney Claussen, Balaji Krishna, and Robert Waywell

SAP HANA Product Management

[email protected]

mailto:[email protected]


© 2014 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate

company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its

affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services

are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an

additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or

release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future

developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for

any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-

looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place

undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

http://global12.sap.com/corporate-en/legal/copyright/index.epx



SAP HANA SPS09 - Dynamic Tiering

Technology

Transcript of SAP HANA SPS09 - Dynamic Tiering