Create hybrid olap-relational obiee models

T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com

Creating Hybrid OLAP/Relational OBIEE ModelsMark Rittman, Director, Rittman MeadVenkatakrishnan Janakiraman, BI Architect, Oracle

Friday, 12 June 2009


Who Are We?

• Mark Rittman is Technical Director at Rittman Mead‣ Oracle ACE Director, 12 Years+ of Oracle BI experience‣ Author of forthcoming Oracle Press OBIEE+ book‣ Consultant and Trainer, based in the UK‣ Blog at http://www.rittmanmead.com/blog

• Venkatakrishnan Janakiraman is BI Architect at Oracle Corporation‣ Over 6+ Years of Oracle BI Implementation/Training experience‣ BI Architect, Based out of Oracle India‣ Blog at http://oraclebizint.wordpress.com



Rittman Mead Consulting

•Oracle BI&DW Project Specialists providing consulting, training and support•Clients in the UK, USA, Europe, Middle-East•Voted UKOUG BI Partner of the Year 2008•Consultants in Europe and North America•Regular speakers at user group and Oracle events



Oracle Business Intelligence Enterprise Edition Plus


http://www.rittmanmead.com/



Oracle BI Server

•Calculation and Data Integration Engine‣ “Virtual Data Warehouse”‣Allows reporting across multiple sources‣Data accessed though metadata layer

•Connects to RDBMS and OLAP data‣Oracle Database (including Oracle OLAP)‣ IBM DB2‣Microsoft SQL Server‣ Teradata‣Microsoft Analysis Services (OLAP)‣SAP BW 'info cubes'

•Security, Summary Management•Translates incoming “logical SQL”

in to optimized, physical SQL or MDX



Data Access and the Semantic Object Layer

•Reporting data may be sourced frommultiple databases, applications‣Oracle Database (including Oracle OLAP)‣ IBM DB2‣Microsoft SQL Server‣ Teradata‣Microsoft Analysis Services (OLAP)‣SAP BW 'info cubes'‣SAP, Peoplesoft, Siebel, E-Business Suite

•Oracle BI Server joins this sourcedata together and presents alogical business model‣ Facts, Dimensions, Calculations‣ Joins, Presentation folders



Essbase Overview

•Standalone OLAP server now owned by Oracle‣Acquired as part of the Hyperion Acquisition

•Adds a fully-featured OLAP server, separate to the database, as part of theFusion Middleware family of products

•End-User focused, very popularwith business users

•Used to power many of the Hyperion performancemanagement applications

•Similar capabilities to Oracle OLAP

•Supports MDX and XML/Arather than SQL access



Essbase and Oracle Business Intelligence Enterprise Edition

•Essbase is a key part of Oracle’s Business Intelligence Foundation Layer•Essbase and Hyperion EPM can play several roles in this architecture‣As a data source‣As a data consumer‣As a portal for all Oracle BI‣As a MS Office integration point

•Essbase can benefit from Oracle BIEnterprise Semantic Model

•OBIEE can benefit from Essbasecalculations and analytics



Essbase as Data Provider and Data Consumer





OBIEE to Essbase Connectivity

• The Oracle BI Server connects to Essbase via the Essbase Client API‣ Low-level access, potential support for writeback ‣ Precludes the use of Provider Services (implications for clustering)

• Essbase Client needs to be installed on the BI Server server‣ And potentially on the workstation running Oracle BI Administrator

• BI Server generates MDX which is sent to Essbase• Able to leverage Essbase MDX functions• Time Series calculations are “function shipped” to MDX• Some (limited) support for Essbase filters,

ragged hierarchies, substitution variables etc• OBIEE 10g+Essbase integration is a

“work in progress”, some issues to be aware of



Importing Essbase Data into Oracle BI Enterprise Edition: Step 1

• Essbase cube metadata (“Outline”) is imported into BI Server Semantic Model using Oracle BI Administrator tool

• Connection is via the Essbase Client API• Enter connection details (typically admin account for import, switch later on)• Select application and database to import



Importing Essbase Data into Oracle BI Enterprise Edition: Step 2

• Import process creates the Physical Model within the Semantic Layer• This is then dragged and dropped into the Business Model and Mapping Layer, and then the

Presentation Model layer‣ Logical tables and

dimensions are automatically created‣ Measure aggregations

set to AGGR_EXTERNAL• Essbase generations

converted to dim columns• Essbase measures

converted to fact measures



DemonstrationImporting an Essbase Outline into Oracle BI Administrator



Creating Federated (Hybrid) Models

• One of the most powerful features in the BI Server it is ability to combine data sources• Multiple data sources presented as a single business model• Works across both relational and non-relational sources‣ Oracle BI Server converts non-relational sources into relational views

• In terms of Essbase integration there are two main use-cases:1.Providing drill-down from an Essbase-based model to detail-level relational data2.Combining MOLAP and relational measures into a single business model



Drill-Through From Essbase to Relational: Step 1

• Start by initially mapping in the Essbase data source into the business model• Each logical table will then have a single Logical Table Source mapping to Essbase• The screenshot below is of the logical fact table• It is currently mapped to the lowest dimension levels provided by the Essbase source• Check that any measure aggregations are supported by both relational and Essbase sources




• Bring in the new relational source into the physical layer• Drag and drop the new relational physical tables across to the business model

to create new Logical Table Sources‣ Drop each relational column onto the

corresponding (originally Essbase-derived) logical columns‣ Tells the BI Server that either source can

provide this data item (i.e. creates joins)




• Ensure that the relational LTS provides all the same dimension columns as the Essbase LTS• Gen1 columns will need to be mapped to a literal based on the dimension name




• Time dimensions will need special handling‣ Relational sources do not need to ensure unique time period names across the dimension‣ Essbase (normally) requires that all member names are unique across the dimension‣ You will therefore usually need to transform the relational names to match the Essbase ones




• Map in the relational dimension source that contains more detail• Note: you can also use your relational source to add additional attributes to the dimension• Useful for enhancing Essbase data that usually consists of just IDs and names




• Ensure logical dimension table logical key contains the new detail level• Then add new leaf level to the associated dimension




• Finally, ensure that the new relational (more detailed) LTS has the dimension leaflevel as the logical level

• This ensures that the BI Server will use this source when drilling down to transactional level




• Reports that then run at the aggregated level will automatically use Essbase• This is because it is mapped in to the Customer dimension at a higher (non-leaf) level

With set [Channel3] as '[Channel].Generations(3).members' set [Customers5] as '[Customers].Generations(5).members'select { [Measures].[Amount] } on columns, NON EMPTY {crossjoin ({[Channel3]},{[Customers5]})} properties ANCESTOR_NAMES, GEN_NUMBER on rowsfrom [SH.SH]




• Reports where the user drills to transaction level then switch the relational source• This happens because the relational source is the only one that can provide

customer data at the required level

select D1.c1 as c1, D1.c2 as c2, D1.c3 as c3, D1.c4 as c4, D1.c5 as c5, D1.c6 as c6, D1.c7 as c7from (select T1915.COUNTRY_NAME as c1, T1927.CUST_CITY as c2, T1927.CUST_EMAIL as c3, T1927.CUST_CREDIT_LIMIT as c4, concat(concat(T1927.CUST_FIRST_NAME, ' - '), T1927.CUST_LAST_NAME) as c5, T203.CHANNEL_CLASS as c6, sum(T1960.AMOUNT_SOLD) as c7, T1927.CUST_ID as c8 from COUNTRIES T1915,CUSTOMERS T1927, CHANNELS T203,SALES T1960 where ( T203.CHANNEL_ID = T1960.CHANNEL_ID and T1915.COUNTRY_ID = T1927.COUNTRY_ID and T1915.COUNTRY_NAME = 'Australia' and T1927.CUST_CITY = 'Darwin' and T1927.CUST_ID = T1960.CUST_ID ) group by T203.CHANNEL_CLASS, T1915.COUNTRY_NAME, T1927.CUST_CITY, T1927.CUST_CREDIT_LIMIT, T1927.CUST_EMAIL, T1927.CUST_ID, concat(concat(T1927.CUST_FIRST_NAME, ' - '), T1927.CUST_LAST_NAME) ) D1

order by c1, c2, c3, c4, c5, c6



DemonstrationDrill to Detail using Essbase and Relational Sources



Data Model Federation Considerations & Limitations

• Only Essbase outlines that are balanced and level-orientated can be federated• Unbalanced/ragged hierarchies do not consistently map to BMM columns

• Therefore Essbase cubes that “look like” relational cubes are the best to federate• Typically sales analysis, and cubes built to support OBIEE

• There is no “refresh” facility to bring in changed outlines into OBIEE• Therefore only “stable” outlines are suitable for OBIEE

• No issues over ASO, BSO etc• Versions of Essbase supported back to 7.x



Use Case #2 : Combining Essbase and Relational Measures

• You may have actuals data in a relational database, and forecast/budget data in Essbase• It would be useful to report on these in a combined business model• Measures of the same dimensionality can go

in the same fact table• Measures of differing dimensionality go in

separate fact tables• Users see a single combined data source• Oracle BI Server combines the data at runtime



Leveraging Essbase Calculations

• When Essbase is used as a data source, it is possible to leverage its Essbase MDX functions‣ Make use of cross-dimensional selections, forecasts, allocations, statistical functions

• Access is provided through the EVALUATE and EVALUATE_AGGR functions‣ EVALUATE used to return dimension attributes (rank, value etc)‣ EVALUATE_AGGR used to return aggregated values (sum etc)

EVALUATE_AGGR('(%1,Customers.dimension.currentmember).value/ Count(Customers.dimension.currentmember.children)', "vejanaki-lap.in.oracle.com"."SH".""."SH"."Amount")



Adding an Essbase MDX Calculation To the Business Model

• Firstly, add a new logical column to the business model• Then edit the LTS to add the EVALUATE(_AGGR) expression, using Essbase MDX function



DemonstrationMDX Calculations and EVALUATE



Essbase and Time-Series Calculations

• Oracle BI Administrator enables you to mark a dimension as a “Time Dimension”• Makes it then possible to use Time-Series calculations‣ AGO provides the value of a measure by a time offset‣ TODATE gives you the value of a measure from a time offset to now

• The Oracle BI Server “function-ships” these to the appropriate Essbase MDX function

• Generally more efficient than the equivalent SQL query

With set [Times5] as '{[Times].[01-Apr-1998], [Times].[02-Apr-1998] member [Measures].[MS1] as 'SUM({ParallelPeriod([Times].[Gen5,Times],1,[Times].currentmember)},Measures.[Quantity])'select { [Measures].[Quantity], [Measures].[MS1], [Measures].[MS2] } on columns, NON EMPTY {{[Times5]}} properties ANCESTOR_NAMES, GEN_NUMBER on rows from [SH.SH]



Essbase Function Shipping Considerations

• Only a few Oracle BI Server functions are automatically function-shipped to MDX‣ Time Series functions: AGO and TODATE

• All others are performed by the BI Server‣ MOLAP data is retrieved from Essbase via MDX‣ BI Server performs calculations (simple mathematics through to RANK etc)

• In this initial release, you may wish to perform complex analytical functions usingMDX and EVALUATE / EVALUATE_AGGR

• Future releases will function-ship more to MDX?



Using Essbase as An Aggregation Layer

• Essbase can also be used as an alternative to the Aggregate Persistence Wizardor summary tables to speed up relational queries requiring aggregation

• Business model will have originally been sourced from a relational database• The relational data is then loaded into an Essbase

database and aggregated• Essbase database is then “plugged in” to

the business model at the correct aggregation level• In OBIEE 10g / EPM 11.1, this can be performed

using Essbase Studio• In OBIEE 11g, this is planned to be an

automated process



OBIEE as a Data Source Considerations

•Data is unlikely to transform naturally into an Essbase cube‣Non-unique keys‣Presentation layer may not even feature keys‣ Illegal characters

•Will require key values to be exposed, then transformed and aliases added

•Currently several manual steps•Process likely to be automated in future release

of OBIEE



Leveraging HSS and Essbase Security

•The initial import into BI Administrator was through the “admin” Essbase account•Going forward, you may wish to leverage Essbase and HSS security in two ways‣Use HSS login to connect to OBIEE (optionally through EPM Workspace)‣Use Essbase login for Essbase connection pool in BI Administrator

•Allows single sign-on between EPM and OBIEE

•Allows you to use Essbase filters, cube security



Connection Pools and Essbase Logins : Step 1

•Normal way to leverage DB security is to use :USER and :PASSWORD in connection pool settings‣ but OBIEE 10.1.3.4+ adds @Directory Name to :USER when HSS security is used

•Login to Essbase will then fail




•To address this, firstly create a custom Init Block to capture :USER into INITUSER session variable before HSS authentication can add directory name

•Use VALUE_OF(NQ_SESSION.INITUSER) to pass across user credentials

•Note : EPM Workspace login does not pass :PASSWORD value through, cannot SSO to Essbase server, need to use BI Server filters instead




•Use VALUE_OF(NQ_SESSION.INITUSER) to pass across user credentials•Note : EPM Workspace login does not pass :PASSWORD value through, cannot SSO to

Essbase server, need to use BI Server filters instead•OBIEE 10.1.3.4.1 required for use of

Essbase filters (handles #NoAccess properly)‣Warning - reports of performance issues & problems

when using this 10.1.3.4.1 or 10.1.3.4 with 7349048patchset- Performance drop compared to 10.1.3.4- Grand Totals stop working- Reports and graphs come up empty- Need to include all filter columns in criteria etc

‣Recommendation, if possible, is to use10.1.3.4 and BI Server filters instead



DemonstrationConfiguring Essbase Security



Conclusions

• Essbase can significantly add to the capabilities of Oracle BI Enterprise Edition• The Oracle BI Server can report against Essbase in isolation, or integrated with relational data• Essbase can provide the aggregated data, the relational database can provide detail• MOLAP and ROLAP measures can be combined into a single federated model• The Oracle BI Server provides access to native Essbase MDX functions• However, there are many issues and bugs to be aware of, this is an “unfinished work”• Works best when your Essbase cube looks like a ROLAP cube, has a stable outline and

security is handled by OBIEE• Watch out for two otherwise working features not working in combination• For more information, check out the accompanying white paper, and our blogs:‣ http://www.rittmanmead.com/blog‣ http://oraclebizint.wordpress.com

• Thank you for attending this session




http://www.rittmanmead.com/blog

http://www.rittmanmead.com/blog


Creating Hybrid OLAP/Relational OBIEE ModelsMark Rittman, Director, Rittman MeadVenkatakrishnan Janakiraman, BI Architect, Oracle


Create hybrid olap-relational obiee models

Technology

Transcript of Create hybrid olap-relational obiee models