DMM103_15573_MorgenC_1

Public

Christoph Morgen / SAP HANA Product Management, SAP SE

DMM103 New and Best Practices for Data Modeling with SAP HANA

2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Public

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a

purchase decision. This presentation is not subject to your license agreement or any other agreement

with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and SAP's

strategy and possible future developments are subject to change and may be changed by SAP at any

time for any reason without notice. This document is provided without a warranty of any kind, either

express or implied, including but not limited to, the implied warranties of merchantability, fitness for a

particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this

document, except if such damages were caused by SAP intentionally or grossly negligent.


Agenda

Data Modeling in SAP HANA

New Approaches

Best Practices

Future Look

Data Modeling with SAP HANA

Overview SAP HANA Information Views


SAP HANA Information Views Virtual Data Modeling approach as a key SAP HANA concept

SAP HANA PLATFORM

Database Layer Physical Tables

HANA Views

Virtual Data Flow Models

No Aggregations | single atomic copy of detail ata

In-Memory Engines | Performance

Multidimensional Reporting Models

Enterprise Application Virtualize Data Models

Operational Reporting | Applications | Analytics


SAP HANA Information Views Virtual Data Models for Multidimensional Scenarios

Analytical View Attribute View Column Table

Calculation View

Reporting Tools can usually directly consume HANA Calculation Views or Analytic Views.

Multidimensional Tools support Hierarchies for Navigation, Filtering and Aggregation and HANA Prompts (Variables & Input Parameters) for efficient Pre-Filtering of Data.

Calculation Views are usually build upon Analytic-, Attribute- Views, and Column Tables


SAP HANA Information Views Virtual Data Models for Normalized Data Model Scenarios

SAP HANA Calculation Views provide the means to model sophisticate views based on normalized data structures.

See SAP Note 1857202

Complex Calculation Views demand a more explicit intent and control of the modeled set-based data flow, i.e. slicing, aggregation and filtering of sets as input to joins, unions etc.

SAP HANA Calculation Views typically feed data to Business Applications, like SAP HANA XS build Applications


SAP HANA Information Views Flavors of View Modeling Approaches

Attribute Views

Compose a dimensional view with a series of attributes derived

from a collection of tables

e.g. Master Data Views

Highly re-used and shared in Analytic- and Calculation Views

Used to build Hierarchies

Hierarchies are key elements in use with Analytic View for multi-

dimensional reporting

Analytic Views

Combines Fact-Tables with Attribute-Views to Star-Schema-

or OLAP Cube-like objects for

multidimensional reporting.

Stores no aggregates and mass-aggregates on the fly

Hierarchies are key for multi-dimensional access (navigation,

filtering, slicing and aggregation)

Calculation View

Great flexibility for advanced use

Approach to model custom scenarios like

Combined use of Multiple-Fact

Tables/Analytics Views

Build Models on Normalized Data

Re-Use and stack views

Make use of custom scripted views


SAP HANA Attribute Views Modeling Attribute Information in Data Models

SAP AG 2011

What is an Attribute View?

... are the reusable dimensions objects adding context to data in the analysis or data flow.

Can be regarded as Master Data-Views

Build a semantic attribute information collection

from various source tables (e.g. join Plant to Material)

Measures cannot be modeled

Re-used as dimensions in multidimensional scenarios (Analytic Views) or

re-used to model complex master in normalized data model

approach and master data reporting scenarios

Cannot be directly consumed by multidimensional reporting clients

Semantic Attribute Information

Attribute Data Foundation


SAP HANA Attribute Views Modeling Attribute Information in Data Models

SAP AG 2011

What are the capabilities of Attribute Views?

Attribute Views support

Data Foundation join model (various joins types)

Calculated Attributes (static or dynamic calculations)

Description Mapping & Text-Join Lookup master data tables

e.g. used for handling of multi-language master data

Time Dimension Attribute Views

Sematic-Type specification

Hierarchies (Level, Parent-Child), hierarchy-use behavior

Define filter values on attributes & columns

incl. use of Input Parameters

Embedded search properties


SAP HANA Analytic Views Modeling Facts and Dimensions as a Multidimensional Data Model

SAP AG 2011

What is an Analytic View?

Can be regarded as Cube-/OLAP Star Schema-like data model

Without storing aggregated data

consumed using multi-dimensional clients or

re-used in complex data flows

Fact data from the Data Foundation is joined against modeled Dimensions (Attribute Views)

fact table contain the key figures

Measures

Dimensions describe the

key figures and enrich the data

Cardinality in star schemas is generally N:1 fact to dimension, Left Outer Joins

OLAP models are not designed to handle complex join operations

HANA Analytical Views are highly optimized for aggregating mass data

Analytical View

Left Outer Fact Table

N N

1

1

1

N

1

1

N

N


SAP HANA Analytic Views Modeling Facts and Dimensions as a Multidimensional Data Model

SAP AG 2011

What are the capabilities of Analytic Views?

Data Foundation Model for Star Schema Fact data

Measure facts derive from only one data foundation table

Measure definition from Data Foundation Facts

Distinct Count Measures, Calculated & Restricted Measures,

Currency- and Unit Conversion-Measures

Default Aggregation (sum, max, count, ...)

Re-use of slowly changing dimension data

Temporal join btw Facts and dimension data

Variables and Input Parameters

Dynamic filtering and parameter-driven calculations

UI-prompts in use with multidimensional reporting clients


SAP HANA Calculation Views Modeling custom Data Flow Scenarios

SAP AG 2011

What is a Calculation View?

Calculation Views are composite views and can be used to combine other views

Can consume other Analytical-, Attribute-, other Calculation Views & tables

Approach to model custom scenarios like

Combined use of Multiple-Fact Tables/Analytics Views

Build Models on Normalized Data

Re-Use and stack views

Make use of custom scripted views

Great flexibility for advanced use

Modeling Approaches: Graphical Modeler or SQLScript-based Editor

Calculation View

Application UI


SAP HANA Calculation Views Modeling custom Data Flow Scenarios

SAP AG 2011

What are the capabilities of graphical modeled Calculation Views?

Graphical Calculation Views

Views as client-consumable multidimensional query objects or

internal re-use objects or views without measures

Builds a composite view with facts from multiple sources

Embeds Analytical-, Attribute-, Calculation Views & tables

Data operation nodes for Union, Join, Projection & Aggregation

(No SQL or SQL Script knowledge required)

Great flexibility to build complex Virtual Data Models*

and unique capabilities like dynamic joins, .

Calculation View-data flows

Will get optimized and exploit underlying engines

(i.e. OLAP Engine) where possible, prune data flow,

push-down filters

Analytical View

Calculation View

Attribute View

Column Table

Calculation View

*SAP HANA Live


SAP HANA Calculation Views Scripting custom Data Flow Scenarios

SAP AG 2011

SQL Scripts-based Calculation Views

Use of custom SQLScript coding to build Calculation Views

Utilize SQLScript CE-functions or standard SQL-statements (do not mix)

Side-effect free structures / READ-ONLY functions

Consume data from raw tables, modeled views, stored procedures, decision tables,

Semantic output structure is graphically

modeled

Graphical modeled View versus CE Functions

Result in equal performance

(e.g. field pruning, parallelization,

join omission, .)

Scripted

New Approaches

SAP HANA Information Models


New approaches for SAP HANA Data Modeling

Value Help Views Variable and Input Parameter

Referencing data structures for value help definitions

Variable and Input Parameter support external views or tables to generate value help lists-of-values

Supported with Analytic- and Calculation Views

View Variables and Input Parameter can be mapped to variables and input parameters from external views or tables

Allows filtering and customizing value help LOVs from external views e.g. by passing to values of initially selected variables, a LOV for a

dependent variable is generated as a dynamically filtered LOV

Out-source Value Help Information to dedicated views

Benefit from faster value help dialogs

Provide consistent LOVs across consuming views

*was introduced with a revision previous to SPS07



Modeling Dimension-Calculation Views

Dimension-Type Calculation View

Similar capabilities to Attribute Views

cannot be directly consumed by multidimensional reporting clients

Composing Dimension-Attributes and -Hierarchies based on a custom CalcView-data flow

Hierarchies supporting dynamic input structures

Attribute columns used within hierarchies can ne input parameter-driven

Hierarchy properties like Root Node (e.g. Parent-Child Hierarchies) can be input parameter-driven

Usage Scenario

Dynamic hierarchy structures and properties are required

Star-Join Calculation Views (details see following slides)

Dynamically mapped attributes


New Practices for SAP HANA Data Modeling

Modeling Star Join-Calculation Views

Calculation Views with Star Join capabilities

Joining fact input data flows with multiple dimension views in a single node

Fact Input flow can be any custom calculation flow of nodes

Other, only Calculation Views of Data Category-Dimension are allowed as input

Dimension Properties

All DIM-View Attributes are automatically part of the StarJoin (incl. Hierarchies)

DIM-Views are added as shared (referenced) dimensions, changes to dimension views are immediately available

Local hierarchies can also be defined

Measures, like Counters use the dimension reference, hence can reference to attributes hidden from the output

UseCase

Make use of special Calculation View Dimension capabilities (e.g. parameterized Hierarchies) or multiple fact-table input

Important Note: Star Join-Views currently cannot regarded a replacement for Analytic View

capability, especially as they have not been yet fully optimized for aggregation performance. Further,

Star Join-Views cannot be consumed by other Calculation Views.



Modeling with Virtual Tables

Consuming remote data in SAP HANA Views

Smart Data Access allows

To access remote data like local data

Specific HANA query optimization and execution handles functional SQL compensation, automatic data type translations, filter push-down, in order to push query processing to the underlying data source system

To enable consumption

Remote data structures are registered and referenced as virtual tables

Virtual tables can also be consumed as data sources within HANA Calculation Views.

Supported external systems and restrictions are documented in SAP note 1868209



Introducing performance analysis capabilities

Performance Analysis Mode in Modeling Environment

Introduction of performance analysis hints and indicators inside the HANA Model Editor

Manually switched on or defaulted switched on

Hints and indicators about table partitioning and number of rows (threshold as preference)

Scenario indicators for

partitioned tables (icon)

and

exceeded row thresholds

Switching on

performance analysis

mode

View details pane: indication about partitioning

type by icon (hash, range, )

Performance analysis: more partitioning and row

count information.



Enhanced Plan Visualization for detailed analysis

SQL Editor > Context Menu

Visualize Plan

Explain Plan

Time Line View root cause analysis

New Operator List View

Summarized & Filter



Modeling Productivity Error Handling/HANA Answers Integration

Extended Error Handling with SAP HANA Answers (http://answers.saphana.com/)

In extension to documentation and help, the SAP HANA Answers-plugin to SAP HANA Studio enables crawling external sources of information

e.g. adds information from SCN and others

Displays crowed-sourced information

embedded in HANA Studio or outside

Integrated with HANA Studio views (job log, ), editors, wizards. Called via key from selected text

or feature.

Independent feature-plugin to install

F10 as your new friend

Best Practices



Column

Store Tables

Attribute View Analytical View

Calculation View Procedures

Avoid transfer ring large result sets between the HANA Database and client application

Reduce data transfer between views follow the volcano approach

Do calculation after aggregation & avoid calculations before aggregation on line item level

Aggregate data records (e.g using GROUP BY, Keep flags, reducing columns, .)

Filter data amount as early as possible in the lower layers .. use

Constraint filters

WHERE clause / Parameters

Analytical Privileges

SAP HANA Modeling Best Practices

General Modeling Performance Guidelines

Analytical Privileges



Modeling Scenario Multidimensional Model or not?!

Analytical View (including Attribute views) Calculation View

Multidimensional, Star Schema Combine multiple Analytical Models using Unions

(Similar Dimensions)

STAR SCHEMA MODEL

AGGREGATION

WITH OR WITHOUT

WHERE CLAUSES

MULTIPLE QUERY OJECTS

NORMALZED MODEL

COMPLEX JOINS WITH

WHERE CLAUSES

Complex Joins

Attribute View (independent)

Complex Joins including facts

Calculation View



Things to watch our for Data Movement

Filters pushed

down SELECT SPART, MATNR, WERKS, SUM(KWMENG) FROM AV WHERE WERKS = ? AND MATNR = ? GROUP BY

SelectWhere Matnr = DPC1017 Group By Spart

Refrain from moving

large datasets

between views & to

the front-end

Aggregation

While re-using Analytic Views

in Calculation Views,

carefully select, prune, filter

and slice data from a wide

and multidimensional detail-

level data structure



Things to watch our for Implicit Filter Push Down

Select Werks, Matnr, Sum(KwMeng) From Sales_Av

SELECT WERKS, SUM(KWMENGA), SUM(KWMENGB), SUM(TOTAL) FROM CV WHERE MATNR = DPC1017 GROUP BY WERKS

Select From Sales_Av Where Werks = 100 & Matnr = DPC1017

Select From Sales_Av Where Werks = 1000 & Matnr = DPC1017

Review the Visualize Plan

to make sure filters are

pushed down

Filter applied late Filter applied early



Things to watch our for Column & Row Store operators

Review the Explain Plan!

Limit the number of records used by

ROW store operators (through

Aggregation & Filters)



Things to watch our for Scripted Calculation Views

SELECT FIELDA, SUM(COUNT) FROM MODEL GROUP BY FIELDA WHERE FIELDA IN Apple, Orange, Banana

CE Functions will try to exploit underlying database engines and will push

filters down, prune columns and omit joins where possible

FIELDA COUNT

Apple 3000000

Banana 4000000

Orange 9000000

FIELDA COUNT

Apple 3000000

Banana 4000000

Orange 9000000

Filter applied & columns pruned late

Filters applied & columns pruned early.

Aggregation performed by OLAP engine

SQL is static and is always executed as defined and

intermediate result sets will be materialized



Tips and Tricks Input Parameters (Explicit filter push down)

SELECT CATEGORY, SUM(YEAR1), SUM(YEAR2) FROM MODEL GROUP BY CATEGORY

WITH PARAMETERS (

'PLACEHOLDER' = ('$$YEAR_1$$', '2011'),

'PLACEHOLDER' = ('$$YEAR_2$$', '2012'))

Constraint Filter

$$YEAR_1$$ Constraint Filter

$$YEAR_2$$

Define Input Parameters in the

Data Foundation to explicit and

compulsory Filter the data



Tips and Tricks Union with Constants for Input Source Pruning

SELECT

ORDER

SALESORG

DIVISION

AMOUNT

QUANTITY

GROUP_CONSTANT

FROM VIEW

WHERE

(GROUP_CONSTANT = A AND ORDER IN 1, 2,3)

OR

(GROUP_CONSTANT = B AND ORDER IN 6,7,8)

Constant Column

Filtering on Constant Column will prune input sources. (i.e. Model C, D, E, F are NOT executed)



Tips and Tricks Union with Constant Values (Pivot Data)

Standard Union

Pivot table using Union with Constant values



Tips and Tricks if joining Analytics Models is required

Variable (Delivery Date)

pushed down using

WHERE clauses

Variable (Created On)

pushed down using

WHERE clause

Variables or Input Parameters

for explicit filtering

exposed as UI prompts

Join Analytic Models

with caution.

Use WHERE clauses/filters to minimize the amount of

records used in the join.

Slicing / pruning of columns

Optimized join

Where clause(s) filters data-set

before Join occurs



Modeling Scenario Normalized Data Models / Virtual Data Models

Building complex relational Model Scenarios

Demands a more explicit intent and control of the modeled set-based data flow, i.e. slicing, aggregation and filtering of sets as

input to joins, unions etc.

Identify reproducible

pattern

Split big models into

smaller parts

Do not build monolithic

models



SAP HANA Live! Virtual Data Models for SAP Business Suite

Open Interfaces https | oData | SQL | MDX

Personalized

Views

Personalized

Views

Personalized

Views

Customer Extensions

SAP HANA Live (VDM)

Re-use of

Real-Time

Views

Reuse Views

Private Views

Physical Tables

SAP-

delivered

Query

Views

Query Views Top of the SAP HANA Live view

hierarchy (VDM) and provide

consumable output fields

Query Views are consumed by reports or analytical applications

Reuse Views are for use in Query Views, not for

direct consumption by reports or

analytical applications

Reuse Views represent the actual data model by exposing or translating original SAP Business Suite source

tables into Views

Private Views encapsulate SQL transformations on

single or multiple data base tables or

Reuse views

Private Views are comparable to subroutines

Completely build on SAP HANA Calculation Views



Modeling Scenario Normalized Data Models / Virtual Data Models

Optimized execution for Queries in complex relational Model Scenarios

See SAP Note 1857202 for considerations and constraints i.e. Only Tables and Calculation views are supported etc. Improved Join ordering

For stacked Calculation View data

models, the Calculation Engine can

generate an optimized SQL

statement at runtime if SQL Engine execution-flag is used

Note, the optimized execution of SQL Engine- flagged models does allow for

implicit OLAP Engine push-down in

certain scenarios.

Outlook



Future Directions for SAP HANA Data Modeling

Planned Innovations and Future Direction

Future Direction

Further unification of Attribute, Analytics, Calculation View into a

unified model

Consumption framework for Application Function Library

Procedures

Continuous Usability enhancements

Drive easiness, enhance modeler productivity, reduce complexity,

separate modeling intent from

model optimization

Planned Innovations

Editor usability

Value help entity reference

Calculation Views

Rank node

Table function as data sources

Script-based CV enhancements

Table function support

Output column structure import

History Views support (time travel support)

Spatial support: spatial joins, spatial expressions in filters

Harmonize Development- and Modeler- Studio perspectives

Harmonize object naming

Supportability

Debugging Views with drill-down analysis, join cardinality

Logging and tracing for modeler plugins (preferences)

Productivity / object re-usability

Replace node / replace node with a data source / ..

Propagate semantics from data sources / extract semantics

WebIDE

Calculation View editor

Analytic Privilege editor


Further Information

SAP Education and Certification Opportunities

www.sap.com/education

Watch SAP TechEd Online

www.sapteched.com/online

SAP Public Web

scn.sap.com http://scn.sap.com/community/developer-center/hana

www.sap.com www.saphana.com

Related Workshops/Lectures at SAP TechEd 2014

DMM161 - Introduction to Data Modeling in SAP HANA, Hands-On Workshop

DMM270 Advanced Data Modeling in SAP HANA, Hands-On Workshop


SAP d-code Virtual Hands-on Workshops and SAP d-code Online Continue your SAP d-code education after the event!

SAP d-code Online

Access replays of keynotes, Demo Jam, SAP d-code live interviews, select lecture sessions, and more!

Hands-on replays

http://sapdcode.com/online

SAP d-code Virtual Hands-on Workshops

Access hands-on workshops post-event

Starting January 2015

Complementary with your SAP d-code registration

http://sapdcodehandson.sap.com

2014 SAP SE or an SAP affiliate company. All rights reserved.

Thank you

Contact information:

Christoph Morgen

SAP HANA Product Management

SAP SE | Dietmar-Hopp-Allee 16 | 69190 Walldorf | Germany

[email protected] | www.sap.com


2014 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an

SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE

(or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark

information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its

affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or

SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing

herein should be construed as constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or

release any functionality mentioned therein. This document, or any related presentation, and SAP SEs or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for

any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-

looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place

undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

DMM103_15573_MorgenC_1

Documents

Transcript of DMM103_15573_MorgenC_1