SAP HANA SPS10- SAP HANA Dynamic Tiering

23
1 © 2014 SAP AG or an SAP affiliate company. All rights reserved. SAP HANA SPS 10 - What’s New? SAP HANA dynamic tiering SAP HANA Product Management May, 2015 (Delta from SPS 09 to SPS 10) Public

Transcript of SAP HANA SPS10- SAP HANA Dynamic Tiering

Page 1: SAP HANA SPS10- SAP HANA Dynamic Tiering

1© 2014 SAP AG or an SAP affiliate company. All rights reserved.

SAP HANA SPS 10 - What’s New? SAP HANA dynamic tiering

SAP HANA Product Management May, 2015

(Delta from SPS 09 to SPS 10) Public

Page 2: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2014 SAP AG or an SAP affiliate company. All rights reserved. 2

Disclaimer

This presentation outlines our general product direction and should not be relied on in making

a purchase decision. This presentation is not subject to your license agreement or any other

agreement with SAP.

SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and

SAP’s strategy and possible future developments are subject to change and may be changed

by SAP at any time for any reason without notice.

This document is provided without a warranty of any kind, either express or implied, including

but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or

non-infringement. SAP assumes no responsibility for errors or omissions in this document,

except if such damages were caused by SAP intentionally or grossly negligent.

Page 3: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 3Public

Agenda

High availability

Automatic host-failover for dynamic tiering node

Backup & Recovery

Implementation of Backint for SAP HANA; Further enhancements in backup system

Multitenant database containers

Support for dynamic tiering in tenant databases

Administration and monitoring

New monitoring views; new cockpit tiles; new roles for DT admin/monitoring users

Further features

Support for extended tables in EIM flow graphs; upcoming Data Lifecycle Management Tool

Page 4: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 4Public

Host Auto Failover for Dynamic Tiering HostsHigh availability within database cluster

HANA plus DT is a clustered database

So you may want to set up standby hosts

Standby for either Indexserver (standard HANA) or esserver (DT)

One host cannot be standby for both at the same time

Automatic failover now also available for dynamic tiering hosts

With DT standby host, no manual intervention necessary in case

the DT host fails

System automatically activates DT standby

Database remains available during failover procedure, only DB

requests that access DT will failClassical HANA services

Compute

node

Hot Store

Warm Store Service

Compute

node

Standby

node

Auto-

Failover

Standby

node

Warm Store

Auto-

Failover

mirror

mirror

Page 5: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 5Public

Host Auto Failover for Dynamic Tiering HostsHigh availability within database cluster

How to set up DT system for host auto failover

System needs two DT hosts Add one with role “EXTENDED_STORAGE_WORKER”

Add second one with role “EXTENDED_STORAGE_STANDBY”

Both hosts need to have the same hardware layout

Add hosts during installation or later (in hdblcm(gui))

Take care of assigning correct failover group: DT worker and standby must be in the same, dedicated failover group

A host can only be standby for HANA workers or dynamic tiering workers, not for both at the same time.

If you want standby for HANA and dynamic tiering workers, you needtwo different standby hosts

Classical HANA services

Compute

node

Hot Store

Warm Store Service

Compute

node

Standby

node

Auto-

Failover

Standby

node

Warm Store

Auto-

Failover

mirror

mirror

Failover Group

„extended_storage“

Failover Group

„default“

HANA

DT

Page 6: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 6Public

Backup & Recovery in SAP HANA Dynamic Tiering (DT)Implementation of the BACKINT interface

Backup & recovery affects full DB

We always back up memory + disk store!

Works the same way as in regular scale-out:

– All hosts are included in backup

– All worker processes must be running

New HANA Delta Backup not yet for DT

Planned for future release

With SPS 10: implementation for Backint

API for 3rd party backup solutions

See next slide for info on Backint for SAP HANA

Backint is a two-party implementation

HANA-side implementation now also includes DT

– Backup, recovery, query backup catalog, delete

Tool vendors implement backup agent

Add-on certification process planned

Backup tools will need to be certified for the new

HANA+DT system

Not included in regular Backint for HANA certification

Certification will be offered after first pilot (Q3/Q4)

Dynamic

tiering

HANAIndex

server

One Data backup

Name

server

XS

engine

DT

table

spaces

Page 7: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 7Public

SAP HANA Backup and Recovery3rd party backup tools: “Backint for SAP HANA” API

SAP HANA provides a backup/recovery API for connecting 3rd party backup tools

For both data and log backups

“Backint for SAP HANA” API can be implemented by a 3rd party backup agent (certification required)

Provides functions for backup, recovery, query, delete

3rd party backup agent runs on the SAP HANA server, communicates with 3rd party backup server

Backups are transferred via pipe

Direct integration with SAP HANA:

– Data backups to Backint can be triggered/scheduled using SAP HANA studio, SQL commands, or DBA Cockpit

– Log backups are automatically written to Backint (if configured)

SAP HANA

Database

3rd Party

Backup

Server

3rd Party

Backup Agent

SAP HANA

studio

Backup/

recovery

Page 8: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 8Public

Single backup location for HANA and DT processes

Single backup location for HANA and DT

Previously (SPS 09):

DT had dedicated location for data and log backups

Parameters: global.ini | [Persistence] |

basepath_databackup_es / basepath_logbackup_es

New situation (starting with SPS 10):

DT and HANA backups always written to same

location

Parameters: global.ini | [Persistence] |

basepath_databackup / basepath_logbackup

These locations must be accessible from all hosts of

the system, including the DT host

Consequence of new backup location

Upgrade to SPS 10 breaks backup history

Upgrade preparation:

Create final data backup before upgrade

Stop database:

sapcontrol -nr <instance> -function stopsystem HDB

On file system, move old data and log backups of

dynamic tiering from previous location to global HANA

backup locations

[This ensures that system finds log backups after update]

Then upgrade HANA + DT to SPS 10-level

Post-upgrade step:

After upgrade, point-in-time recovery is unavailable

Create full database backup to enable PITR again

Page 9: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 9Public

Co-Deployment of HANA and DT on single host

Non-Productive Scenario: Co-Deployment of HANA and DT on singlehost

In SPS 10 supported, including backup/recovery

Only supported for non-productive installations

Installation

Install HANA and DT server software as usual

Do not add a second host (DT host) during installation

For data and log of DT, choose either dedicated location (hana/data_es/<SID>)or same as HANA location (e.g. /hana/data/<SID>)

Add „extended_storage_worker“ role to the HANA host

Must configure using hdbmodify – not configurable through hdblcm/hdblcmgui

/hana/shared/<sid>/global/hdb/install/bin/hdbmodify \--add_local_roles=extended_storage_worker --listen_interface=global

This starts the esserver process on the HANA host

Then create extended storage as always

Using SQL or SAP HANA Cockpit

SAP HANA host

indexserver esserver

daemon nameserver xserver

Roles „worker“ and „extended_storage_worker“

Page 10: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 10Public

Multitenant Database Containers (MDC)

MDC setups with SAP HANA dynamic tiering

SAP HANA dynamic tiering is now also available for tenant databases

Each tenant database can be associated with zero or one extended stores

Each extended store is dedicated to exactly one tenant database

Each extended store requires a dedicated dynamic tiering host

SAP HANA system with MDC and dynamic tiering

Compute node

System Database

Compute nodeCompute node

Tenant Database <B>

Extended StoreTenant Database <A>

Tenant Database <C>

Extended Store

Classical HANA (single-node or scale-out)

DT Host <B> DT Host <C>

Page 11: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 11Public

Multitenant Database Containers (MDC)

Details of using dynamic tiering in MDC setups

Installing DT in an MDC setup

Install DT software as usual During HANA installation or add at later point in time

Add DT hosts Add one „Dynamic Tiering Worker“ for each tenant DB

that needs dynamic tiering

If required, also add „Dynamic Tiering Standby“ hosts

Make sure each host has unique location for DT dataand log volumes

Create Extended Storage for tenant DBs In each tenant DB:

– Add esserver: ALTER DATABASE <name> ADD 'esserver' AT <host:port>

– Import delivery units for dynamic tiering into tenant DB

– Create extended storage in tenant DB

For full details: see administration guide for dynamictiering at http://help.sap.com/hana_options_dt (SPS 10)

Backup & Recovery with DT and MDC

File-based Backups Backup/Recovery of tenant database includes its extended

storage

No functional differences to MDC without DT

Backint DT implementation of Backint does not cover MDC

Planned for future release

Other properties and limitations in SPS 10

Cross-tenant queries Not yet supporting cross-tenant access to extended tables

Tenant isolation Strong tenant isolation is not yet supported

Licence management Single DT license for entire HANA system

DT license sets cumulative size limit for all tenant DBs

Page 12: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 12Public

Five New Monitoring Views for Dynamic Tiering with SPS 10

M_ES_OVERVIEW

Tabular information on important system properties

Version, block and buffer usage, backup status, and more

M_ES_DELTA_MEMORY

Lists all extended tables that are delta-enabled

Plus delta memory usage broken down by components

M_ES_CONNECTIONS

Lists information about processes running in dynamic tiering

– test

Connection and user ID, type of command being executed, number of threads used bythe connection, time since last request was entered through the connection, and more

M_ES_TRANSACTIONS

Lists active transactions running in dynamic tiering

Connection and user ID, state of transaction (ACTIVE, COMMITTING, etc.), start time, and more

M_ES_LOCKS

Lists information about all locks currently held in extended store

Connection and user id that has the lock, the table and row(s) that are locked, the type oflock (SHARED, READ, WRITE…), and more

DT has its own connection management

DT connections are linked to HANA connections

View M_REMOTE_STATEMENTS

– CONNECTION_ID = HANA Connection (e.g. fromM_CONNECTIONS)

– REMOTE_CONNECTION_ID = DT Connection (fromM_ES_* views)

Find all DT-connections of HANA user RICHARD:

select * from

M_REMOTE_STATEMENTS as A inner join

M_ES_CONNECTIONS as B

on A.REMOTE_CONNECTION_ID =

B.CONNECTION_ID

where A.USER_NAME = 'RICHARD'

Page 13: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 13Public

New SAP HANA Cockpit Tiles Related to Dynamic Tiering

Two new Tiles in SAP HANA Cockpit

User Tables Determine tables that are candidates for

conversion to extended tables

Dynamic Tiering Server Configuration View and Change parameterization of SAP HANA

dynamic tiering database process (esserver)

Two new roles for DT Tiles

Monitoring user (view DT tiles) sap.hana.tiering.roles::Monitoring

Administer DT (change configuration etc.) sap.hana.tiering.roles::Administration

Includes Monitoring role and system privilegeEXTENDED STORAGE ADMIN

Required only for the DT tiles You also need the regular admin/monitoring roles

to access SAP HANA Cockpit itself

Page 14: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 14Public

SAP HANA CockpitNew tile: Dynamic Tiering Server Configuration

Tile to edit esserver.ini entries:

• ‘esserver.ini’ is the configuration file for the DT server

• Click on “Edit” button at bottom right of screen to delete

or modify existing entries in the esserver.ini file.

Examples:

• delta_memory_mb: amount of memory available to

store delta enabled extended tables

• num_threads: maximum number of threads to be

used by DT server

• There is also a ‘+’ sign at the top right of the screen to

set additional DT server configuration parameters

(parameters that should be set only with the help of a

support engineer)

• These new entries become part of the esserver.ini

file

Page 15: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 15Public

SAP HANA CockpitNew Tile „Table Usage"

Find candidates for extended tables

Combine size and usage information

Only displaying HANA column tables

Table size in MB/GB

Number of read operations on table

Number of write operations on table

Convert table to extended table

Select one or multiple tables

Convert to extended tables

Control delta creation in conversion

Page 16: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 16Public

Security

Audit logging for management of extended storage

Changes to the configuration of a dynamic tiering system

New group of audit actions „SAP HANA Dynamic Tiering“

Audit creation, change and dropping of extended storage

Page 17: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 17Public

SAP HANA Enterprise Information Management

Can now be used with extended tables

Include extended tables in flow graphs

As data source or data sink

Based on tasks or stored procedures

Value Proposition

Load into extended tables from within HANA system or external data sources

Model and schedule data movement between in-memory and extended tables

Data staging in native HANA data warehouses

EIM Flow Graphs with Extended Tables

Page 18: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 18Public

SAP Data Warehousing Foundation Data Lifecycle Manager (DLM)

Optimize the memory footprint of SAP HANA

Leverage SAP HANA Dynamic Tiering, Hadoop or

SAP Sybase IQ with a tool-based approach to

setup an aging strategy

Optimize memory footprint in SAP HANA

native use cases

Define data slices on SAP HANA tables to be

displaced from SAP HANA memory

Optimize union access between the tables

Automated Data Movement between stores

Generated Stored Procedures to do data mass

movement – in and out

Schedule data movement using HANA tasks

Planned for Q3/2015

Page 19: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 19Public

Future DirectionSPS 11 and beyond (2016/2017)

TodaySAP HANA SPS 10

This is the current state of planning and may be changed by SAP at any time.

Outlook SAP HANA dynamic tieringSummary

Technical integration

– Common installer

– Integrated administration & Monitoring

– Support for DT in multitenant database containers (MDC)

Enterprise Readiness

– Host auto-failover, support for storage connector API

– Convergence of file-based backup; Implementation of Backint API

Functional integration

– Global database catalog

– Cross-store optimizer

– Support in Calculation Models

– Performance optimization for BW data loads

– Enable use of DLM tool to manage extended tables

– Enable HANA EIM to use extended tables in data flows

Extend enterprise readiness

– Delta backup mechanism

– Full backup/restore integration

– Storage Snapshots

– Integration with system replication

Focus: Data Volume Management

– Native data lifecycle management

– Scale-out for extended store

Functional integration

– Support for special functionalities

(series data, geospatial, …)

– Unstructured data, search, …

Page 20: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 20Public

Outlook: extended usage of dynamic tiering in BWUse extended tables in transformation and propagation layer

BW – Operational Data

Data Categories in a BW System

Staging Layer

Analytic Mart

Business Transformation

Co

rpo

rate

Me

mo

ry

EDW Propagation

EDW Transformation

Arc

hiv

e/N

LS

Archived

Planned with BW 7.4 EhP 1 (Q4 2015):

Use extended tables for „warm“ part of transformation and propagation layer (advanced DSOs only)

EDW Propagation

EDW Transformation

Page 21: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 21Public

PLANNED INNOVATIONS Future DirectionToday

This is the current state of planning and may be changed by SAP at any time.

(BW. 7.4 SP08 + HANA SPS 09) (planned for BW 7.4 EhP 1+ HANA SPS 10) (2016 / 2017)

Outlook: SAP BW & dynamic tieringSummary

PSA

Write-optimized DSO

“advanced“ DSO (inbound table only)

Target:

Staging area and corporate memory

~20% of BW data footprint

All major “advanced” DSOs

– Full tables / providers only

– Data separation via simple

“data flow split”

Target:

As before + „warm“ parts of

propagation & transformation layer

~40% of BW data footprint

Optimizations of reporting/read

access for data in DT

Optional integration of NLS storage

into dynamic tiering

Target:

As before + „warm“ parts of

reporting layer

Max. 70% of data footprint

Remark: SAP NLS with SAP IQ is not replaced by the BW integration of HANA dynamic tiering. NLS is a powerful and sophisticated archiving

solution. It is complementary to dynamic tiering and part of the multi-temperature data lifecycle management of BW. Depending on the scenarios,

data growth and architecture our customers have the option to use both “tiers” or just one.

Page 22: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 22Public

Where do I find more information on SAP HANA dynamic tiering?

Public Documentation:

http://help.sap.com/hana_platform

Navigate to SAP HANA Options • SAP HANA Dynamic Tiering

Covers only DT-specific information

i.e. add-on to SAP HANA platform documentation

Videos in the SAP HANA academy

http://scn.sap.com/docs/DOC-59988

Page 23: SAP HANA SPS10- SAP HANA Dynamic Tiering

© 2015 SAP SE or an SAP affiliate company. All rights reserved.

Thank you

Contact information

Dr. Richard Bremer

SAP HANA Platform Product Management

[email protected]