SAP HANA SPS10- SAP HANA Dynamic Tiering
-
Upload
sap-technology -
Category
Technology
-
view
1.370 -
download
8
Transcript of SAP HANA SPS10- SAP HANA Dynamic Tiering
1© 2014 SAP AG or an SAP affiliate company. All rights reserved.
SAP HANA SPS 10 - What’s New? SAP HANA dynamic tiering
SAP HANA Product Management May, 2015
(Delta from SPS 09 to SPS 10) Public
© 2014 SAP AG or an SAP affiliate company. All rights reserved. 2
Disclaimer
This presentation outlines our general product direction and should not be relied on in making
a purchase decision. This presentation is not subject to your license agreement or any other
agreement with SAP.
SAP has no obligation to pursue any course of business outlined in this presentation or to
develop or release any functionality mentioned in this presentation. This presentation and
SAP’s strategy and possible future developments are subject to change and may be changed
by SAP at any time for any reason without notice.
This document is provided without a warranty of any kind, either express or implied, including
but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or
non-infringement. SAP assumes no responsibility for errors or omissions in this document,
except if such damages were caused by SAP intentionally or grossly negligent.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 3Public
Agenda
High availability
Automatic host-failover for dynamic tiering node
Backup & Recovery
Implementation of Backint for SAP HANA; Further enhancements in backup system
Multitenant database containers
Support for dynamic tiering in tenant databases
Administration and monitoring
New monitoring views; new cockpit tiles; new roles for DT admin/monitoring users
Further features
Support for extended tables in EIM flow graphs; upcoming Data Lifecycle Management Tool
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 4Public
Host Auto Failover for Dynamic Tiering HostsHigh availability within database cluster
HANA plus DT is a clustered database
So you may want to set up standby hosts
Standby for either Indexserver (standard HANA) or esserver (DT)
One host cannot be standby for both at the same time
Automatic failover now also available for dynamic tiering hosts
With DT standby host, no manual intervention necessary in case
the DT host fails
System automatically activates DT standby
Database remains available during failover procedure, only DB
requests that access DT will failClassical HANA services
Compute
node
Hot Store
Warm Store Service
Compute
node
Standby
node
Auto-
Failover
Standby
node
Warm Store
Auto-
Failover
mirror
mirror
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 5Public
Host Auto Failover for Dynamic Tiering HostsHigh availability within database cluster
How to set up DT system for host auto failover
System needs two DT hosts Add one with role “EXTENDED_STORAGE_WORKER”
Add second one with role “EXTENDED_STORAGE_STANDBY”
Both hosts need to have the same hardware layout
Add hosts during installation or later (in hdblcm(gui))
Take care of assigning correct failover group: DT worker and standby must be in the same, dedicated failover group
A host can only be standby for HANA workers or dynamic tiering workers, not for both at the same time.
If you want standby for HANA and dynamic tiering workers, you needtwo different standby hosts
Classical HANA services
Compute
node
Hot Store
Warm Store Service
Compute
node
Standby
node
Auto-
Failover
Standby
node
Warm Store
Auto-
Failover
mirror
mirror
Failover Group
„extended_storage“
Failover Group
„default“
HANA
DT
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 6Public
Backup & Recovery in SAP HANA Dynamic Tiering (DT)Implementation of the BACKINT interface
Backup & recovery affects full DB
We always back up memory + disk store!
Works the same way as in regular scale-out:
– All hosts are included in backup
– All worker processes must be running
New HANA Delta Backup not yet for DT
Planned for future release
With SPS 10: implementation for Backint
API for 3rd party backup solutions
See next slide for info on Backint for SAP HANA
Backint is a two-party implementation
HANA-side implementation now also includes DT
– Backup, recovery, query backup catalog, delete
Tool vendors implement backup agent
Add-on certification process planned
Backup tools will need to be certified for the new
HANA+DT system
Not included in regular Backint for HANA certification
Certification will be offered after first pilot (Q3/Q4)
Dynamic
tiering
HANAIndex
server
One Data backup
Name
server
XS
engine
DT
table
spaces
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 7Public
SAP HANA Backup and Recovery3rd party backup tools: “Backint for SAP HANA” API
SAP HANA provides a backup/recovery API for connecting 3rd party backup tools
For both data and log backups
“Backint for SAP HANA” API can be implemented by a 3rd party backup agent (certification required)
Provides functions for backup, recovery, query, delete
3rd party backup agent runs on the SAP HANA server, communicates with 3rd party backup server
Backups are transferred via pipe
Direct integration with SAP HANA:
– Data backups to Backint can be triggered/scheduled using SAP HANA studio, SQL commands, or DBA Cockpit
– Log backups are automatically written to Backint (if configured)
SAP HANA
Database
3rd Party
Backup
Server
3rd Party
Backup Agent
SAP HANA
studio
Backup/
recovery
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 8Public
Single backup location for HANA and DT processes
Single backup location for HANA and DT
Previously (SPS 09):
DT had dedicated location for data and log backups
Parameters: global.ini | [Persistence] |
basepath_databackup_es / basepath_logbackup_es
New situation (starting with SPS 10):
DT and HANA backups always written to same
location
Parameters: global.ini | [Persistence] |
basepath_databackup / basepath_logbackup
These locations must be accessible from all hosts of
the system, including the DT host
Consequence of new backup location
Upgrade to SPS 10 breaks backup history
Upgrade preparation:
Create final data backup before upgrade
Stop database:
sapcontrol -nr <instance> -function stopsystem HDB
On file system, move old data and log backups of
dynamic tiering from previous location to global HANA
backup locations
[This ensures that system finds log backups after update]
Then upgrade HANA + DT to SPS 10-level
Post-upgrade step:
After upgrade, point-in-time recovery is unavailable
Create full database backup to enable PITR again
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 9Public
Co-Deployment of HANA and DT on single host
Non-Productive Scenario: Co-Deployment of HANA and DT on singlehost
In SPS 10 supported, including backup/recovery
Only supported for non-productive installations
Installation
Install HANA and DT server software as usual
Do not add a second host (DT host) during installation
For data and log of DT, choose either dedicated location (hana/data_es/<SID>)or same as HANA location (e.g. /hana/data/<SID>)
Add „extended_storage_worker“ role to the HANA host
Must configure using hdbmodify – not configurable through hdblcm/hdblcmgui
/hana/shared/<sid>/global/hdb/install/bin/hdbmodify \--add_local_roles=extended_storage_worker --listen_interface=global
This starts the esserver process on the HANA host
Then create extended storage as always
Using SQL or SAP HANA Cockpit
SAP HANA host
indexserver esserver
daemon nameserver xserver
Roles „worker“ and „extended_storage_worker“
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 10Public
Multitenant Database Containers (MDC)
MDC setups with SAP HANA dynamic tiering
SAP HANA dynamic tiering is now also available for tenant databases
Each tenant database can be associated with zero or one extended stores
Each extended store is dedicated to exactly one tenant database
Each extended store requires a dedicated dynamic tiering host
SAP HANA system with MDC and dynamic tiering
Compute node
System Database
Compute nodeCompute node
Tenant Database <B>
Extended StoreTenant Database <A>
Tenant Database <C>
Extended Store
Classical HANA (single-node or scale-out)
DT Host <B> DT Host <C>
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 11Public
Multitenant Database Containers (MDC)
Details of using dynamic tiering in MDC setups
Installing DT in an MDC setup
Install DT software as usual During HANA installation or add at later point in time
Add DT hosts Add one „Dynamic Tiering Worker“ for each tenant DB
that needs dynamic tiering
If required, also add „Dynamic Tiering Standby“ hosts
Make sure each host has unique location for DT dataand log volumes
Create Extended Storage for tenant DBs In each tenant DB:
– Add esserver: ALTER DATABASE <name> ADD 'esserver' AT <host:port>
– Import delivery units for dynamic tiering into tenant DB
– Create extended storage in tenant DB
For full details: see administration guide for dynamictiering at http://help.sap.com/hana_options_dt (SPS 10)
Backup & Recovery with DT and MDC
File-based Backups Backup/Recovery of tenant database includes its extended
storage
No functional differences to MDC without DT
Backint DT implementation of Backint does not cover MDC
Planned for future release
Other properties and limitations in SPS 10
Cross-tenant queries Not yet supporting cross-tenant access to extended tables
Tenant isolation Strong tenant isolation is not yet supported
Licence management Single DT license for entire HANA system
DT license sets cumulative size limit for all tenant DBs
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 12Public
Five New Monitoring Views for Dynamic Tiering with SPS 10
M_ES_OVERVIEW
Tabular information on important system properties
Version, block and buffer usage, backup status, and more
M_ES_DELTA_MEMORY
Lists all extended tables that are delta-enabled
Plus delta memory usage broken down by components
M_ES_CONNECTIONS
Lists information about processes running in dynamic tiering
– test
Connection and user ID, type of command being executed, number of threads used bythe connection, time since last request was entered through the connection, and more
M_ES_TRANSACTIONS
Lists active transactions running in dynamic tiering
Connection and user ID, state of transaction (ACTIVE, COMMITTING, etc.), start time, and more
M_ES_LOCKS
Lists information about all locks currently held in extended store
Connection and user id that has the lock, the table and row(s) that are locked, the type oflock (SHARED, READ, WRITE…), and more
DT has its own connection management
DT connections are linked to HANA connections
View M_REMOTE_STATEMENTS
– CONNECTION_ID = HANA Connection (e.g. fromM_CONNECTIONS)
– REMOTE_CONNECTION_ID = DT Connection (fromM_ES_* views)
Find all DT-connections of HANA user RICHARD:
select * from
M_REMOTE_STATEMENTS as A inner join
M_ES_CONNECTIONS as B
on A.REMOTE_CONNECTION_ID =
B.CONNECTION_ID
where A.USER_NAME = 'RICHARD'
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 13Public
New SAP HANA Cockpit Tiles Related to Dynamic Tiering
Two new Tiles in SAP HANA Cockpit
User Tables Determine tables that are candidates for
conversion to extended tables
Dynamic Tiering Server Configuration View and Change parameterization of SAP HANA
dynamic tiering database process (esserver)
Two new roles for DT Tiles
Monitoring user (view DT tiles) sap.hana.tiering.roles::Monitoring
Administer DT (change configuration etc.) sap.hana.tiering.roles::Administration
Includes Monitoring role and system privilegeEXTENDED STORAGE ADMIN
Required only for the DT tiles You also need the regular admin/monitoring roles
to access SAP HANA Cockpit itself
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 14Public
SAP HANA CockpitNew tile: Dynamic Tiering Server Configuration
Tile to edit esserver.ini entries:
• ‘esserver.ini’ is the configuration file for the DT server
• Click on “Edit” button at bottom right of screen to delete
or modify existing entries in the esserver.ini file.
Examples:
• delta_memory_mb: amount of memory available to
store delta enabled extended tables
• num_threads: maximum number of threads to be
used by DT server
• There is also a ‘+’ sign at the top right of the screen to
set additional DT server configuration parameters
(parameters that should be set only with the help of a
support engineer)
• These new entries become part of the esserver.ini
file
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 15Public
SAP HANA CockpitNew Tile „Table Usage"
Find candidates for extended tables
Combine size and usage information
Only displaying HANA column tables
Table size in MB/GB
Number of read operations on table
Number of write operations on table
Convert table to extended table
Select one or multiple tables
Convert to extended tables
Control delta creation in conversion
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 16Public
Security
Audit logging for management of extended storage
Changes to the configuration of a dynamic tiering system
New group of audit actions „SAP HANA Dynamic Tiering“
Audit creation, change and dropping of extended storage
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 17Public
SAP HANA Enterprise Information Management
Can now be used with extended tables
Include extended tables in flow graphs
As data source or data sink
Based on tasks or stored procedures
Value Proposition
Load into extended tables from within HANA system or external data sources
Model and schedule data movement between in-memory and extended tables
Data staging in native HANA data warehouses
EIM Flow Graphs with Extended Tables
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 18Public
SAP Data Warehousing Foundation Data Lifecycle Manager (DLM)
Optimize the memory footprint of SAP HANA
Leverage SAP HANA Dynamic Tiering, Hadoop or
SAP Sybase IQ with a tool-based approach to
setup an aging strategy
Optimize memory footprint in SAP HANA
native use cases
Define data slices on SAP HANA tables to be
displaced from SAP HANA memory
Optimize union access between the tables
Automated Data Movement between stores
Generated Stored Procedures to do data mass
movement – in and out
Schedule data movement using HANA tasks
Planned for Q3/2015
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 19Public
Future DirectionSPS 11 and beyond (2016/2017)
TodaySAP HANA SPS 10
This is the current state of planning and may be changed by SAP at any time.
Outlook SAP HANA dynamic tieringSummary
Technical integration
– Common installer
– Integrated administration & Monitoring
– Support for DT in multitenant database containers (MDC)
Enterprise Readiness
– Host auto-failover, support for storage connector API
– Convergence of file-based backup; Implementation of Backint API
Functional integration
– Global database catalog
– Cross-store optimizer
– Support in Calculation Models
– Performance optimization for BW data loads
– Enable use of DLM tool to manage extended tables
– Enable HANA EIM to use extended tables in data flows
Extend enterprise readiness
– Delta backup mechanism
– Full backup/restore integration
– Storage Snapshots
– Integration with system replication
Focus: Data Volume Management
– Native data lifecycle management
– Scale-out for extended store
Functional integration
– Support for special functionalities
(series data, geospatial, …)
– Unstructured data, search, …
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 20Public
Outlook: extended usage of dynamic tiering in BWUse extended tables in transformation and propagation layer
BW – Operational Data
Data Categories in a BW System
Staging Layer
Analytic Mart
Business Transformation
Co
rpo
rate
Me
mo
ry
EDW Propagation
EDW Transformation
Arc
hiv
e/N
LS
Archived
Planned with BW 7.4 EhP 1 (Q4 2015):
Use extended tables for „warm“ part of transformation and propagation layer (advanced DSOs only)
EDW Propagation
EDW Transformation
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 21Public
PLANNED INNOVATIONS Future DirectionToday
This is the current state of planning and may be changed by SAP at any time.
(BW. 7.4 SP08 + HANA SPS 09) (planned for BW 7.4 EhP 1+ HANA SPS 10) (2016 / 2017)
Outlook: SAP BW & dynamic tieringSummary
PSA
Write-optimized DSO
“advanced“ DSO (inbound table only)
Target:
Staging area and corporate memory
~20% of BW data footprint
All major “advanced” DSOs
– Full tables / providers only
– Data separation via simple
“data flow split”
Target:
As before + „warm“ parts of
propagation & transformation layer
~40% of BW data footprint
Optimizations of reporting/read
access for data in DT
Optional integration of NLS storage
into dynamic tiering
Target:
As before + „warm“ parts of
reporting layer
Max. 70% of data footprint
Remark: SAP NLS with SAP IQ is not replaced by the BW integration of HANA dynamic tiering. NLS is a powerful and sophisticated archiving
solution. It is complementary to dynamic tiering and part of the multi-temperature data lifecycle management of BW. Depending on the scenarios,
data growth and architecture our customers have the option to use both “tiers” or just one.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 22Public
Where do I find more information on SAP HANA dynamic tiering?
Public Documentation:
http://help.sap.com/hana_platform
Navigate to SAP HANA Options • SAP HANA Dynamic Tiering
Covers only DT-specific information
i.e. add-on to SAP HANA platform documentation
Videos in the SAP HANA academy
http://scn.sap.com/docs/DOC-59988
© 2015 SAP SE or an SAP affiliate company. All rights reserved.
Thank you
Contact information
Dr. Richard Bremer
SAP HANA Platform Product Management