Post on 02-Jun-2018
8/11/2019 Using Oracle Data Integrator
1/54
Using Oracle Data Integrator with
Essbase, Planning and the Rest of
the Oracle EPM Products
Edward Roske
eroske@interrel.com
BLOG: LookSmarter.blogspot.com
WEBSITE: www.interrel.com
TWITTER: ERoske
mailto:eroske@interrel.comhttp://www.interrel.com/http://www.interrel.com/mailto:eroske@interrel.com8/11/2019 Using Oracle Data Integrator
2/54
2
8/11/2019 Using Oracle Data Integrator
3/54
8/11/2019 Using Oracle Data Integrator
4/54
About interRel
2008 & 2009 Oracle Titan Award winner - EPM Solution of the
year
2008 Oracle EPM Excellence Award
2009 Oracle EPM/BI Innovation Award
One of the fastest growing companies in the world
(Inc. Magazine, 08 & 09)
Two of the three Hyperion Oracle ACE Directors in the world
Founding Hyperion Platinum Partner; now Oracle Certified Partner
Focused exclusively on Oracle Hyperion EPM software
Consulting
Training
Infrastructure and Installation
Support
Software sales
4
8/11/2019 Using Oracle Data Integrator
5/54
5 Hyperion Books Available:
Essbase (7): Complete Guide
Essbase System 9: Complete Guide
Essbase System 9: End User Guide
Smart View 11: End User Guide
Essbase 11: Adm in Guide
eBooks avai lab le on Amazon Kind le
Just out!
Hyperion Planning for End Users
Coming Soon Hyperion Planning for Admins (days now)
Hyperion Financial Management (Q1 2010)
To order, check out www.lulu.com
Copyright 2007, Hyperion. All rights reserved.5
http://www.lulu.com/http://www.lulu.com/8/11/2019 Using Oracle Data Integrator
6/54
Disclaimer
These slides represent the work and opinions of the
presenter and do not constitute official positions of Oracle
or any other organization.
This material has not been peer reviewed and is presented
here with the permission of the presenter. This material should not be reproduced without the written
permission of interRel Consulting.
8/11/2019 Using Oracle Data Integrator
7/54
Agenda
What is ODI?
A Real World Example ODI and Planning
ODI Demo
A glimpse into the concepts
Tips
Getting Started
Questions and Answers
8/11/2019 Using Oracle Data Integrator
8/54
What is ODI?
Oracle Data Integrator
8/11/2019 Using Oracle Data Integrator
9/54
Oracle Data Integrator (ODI) Overview
Integration tool that replaces HAL and DIM
Used to move and transform information across systems
Processes are based on metadata of existing systems
Only metadata is stored in ODI, not data
Sometimes referred to as Sunopsis
8/11/2019 Using Oracle Data Integrator
10/54
How is ODI used with EPM applications?
Interface metadata and data between source data systems
Build dimensions
Load data
Transfer data across files and relational databases and the
EPM applications All supported with Hyperion Planning, Financial
Management and Essbase
Not supported with EPMA
8/11/2019 Using Oracle Data Integrator
11/54
Oracle ODI to EPM Functionality
Planning Essbase HFM
Metadatadiscovery &
model creation
P P P
Load dataP P P
Load MetadataP P P
Extract dataO P P
Extractmetadata O P POther Refresh to
Essbase
Calc, post and
pre MaxL
scripts
Consolidate
8/11/2019 Using Oracle Data Integrator
12/54
Marketing Message
Three main points:
Extract-Load & Transform (E-LT) architecture
Removes ETL server from infrastructure
USP = Native target database performs
transformations Declarative Design
Developer define the rules, ODI:
Writes the data flows
Manages the data flow complexity
Writes the behind-the-scenes code
Knowledge Modules
Similar to HAL adapters
Talks (and almost always writes) to almost everything
8/11/2019 Using Oracle Data Integrator
13/54
MaxL Run Alloc at ions,
Agg regate Data, etc
A Real World Integration with ODI
DW
Oracle
GL
SAP
Planning
HFM
Essbase
Fcst Data
Consol idated
Results
Sales by
Customer
DB
Accoun ts /
Enti t ies
1
Accoun ts /
Enti t ies
2
Accoun ts /
Enti t ies
3
Custom er Dimension
Sales by Cus tomer Data
4
5
6
7
8
8/11/2019 Using Oracle Data Integrator
14/54
ODI Demo
8/11/2019 Using Oracle Data Integrator
15/54
A Glimpse
ODI Components and Concepts
8/11/2019 Using Oracle Data Integrator
16/54
ODI Components
Master
Repository
(Infrastructure)
Work
Repository
(Interfaces)
Topology Manager
Configure Architecture /
Infrastructure of source
& target technologies
Designer
Design & execute metadata
models and interfaces to move
data across systems
Operator
Monitor Sessions
Review errors
Agent
Coordinates
communication &
scheduling
Security Manager
Maintain ODI Security
Metadata Navigator
Web access to
mettadata repository
Source Systems Target Systems
ODI(ELT) ExtractLoad-Transform
ODI Components
8/11/2019 Using Oracle Data Integrator
17/54
Components
GUI Purpose Users
Repository Configuration Info Metadata
OLTP RDBMS
Sunopsis Infrastructure
AdministratorsDevelopers
Operators
Designer Reverse Engineer
Develop Projects Release Scenarios
Developers
Metadata Administrators
Operator Operate Production
Monitor Sessions
Production Operators
Topology
Manager
Physical and Logical
Architecture of theinfrastructure (servers,
schemas and agents)
Infrastructure Administrators
Security
Manager
User privileges Security Administrators
8/11/2019 Using Oracle Data Integrator
18/54
\
\
Deployment
Work
Repository
(s)(Interfaces)
TopologySecurity
VersioningModels
Projects
Execution
Typically have 1 Master Repository that talks to multipleWork Repository (Dev, Test & Prod).
Can set the production work repository to execution only
Master
Repository
(Infrastructure)
8/11/2019 Using Oracle Data Integrator
19/54
Topology Manager
Representation of the Information System
Connect to resources for integration purposes
Contains:
Physical Architecture
Data Server Physical Schemas and Physical Agents
Logical Architecture (logical schemas & logical agents)
Contexts Links logical and physical architectures
Repositories Master and attached work repositories Languages - different types of languages available
8/11/2019 Using Oracle Data Integrator
20/54
A Sample Topology
DW Essbase Source
FilesPlanning
Windows Machine X
(Development Environment)
PlanSamp.
Consol
FinOp.Plan
1
Windows Machine Y
(Production Environment)
Windows Machine Z
(Production Environment)
DW PlanningEssbase Source
FilesRep.App1
Rep.App2
Rep.App1
Essbase
Rep.App2
Essbase
Rep.App1
Rep.App2
PlanSamp.
Consol
FinOp.Plan1
C:\ODI\FileC:\ODI\File
dbo.databa
sedbo.databa
se
PlanSamp.
Consol
Planning
FinOp.Plan
1
Planning
C:\ODI\File
Filedbo.database
MSSQL
Physical Agents
Logical Agents
Logical Architecture
Physical Architecture
Logical Schema
Physical SchemaData Server
8/11/2019 Using Oracle Data Integrator
21/54
Languages
Languages used to generate code for the specific
technologies
8/11/2019 Using Oracle Data Integrator
22/54
Designer
Designer includes:
Models Metadata for your source and target systems
Projects - Integrations that move data from your source
and target systems
Interfaces
Packages
Procedures
Variables
Knowledge Modules
The Designer module stores this information in a work
repository, references the topology and the security
information defined in the master repository.
8/11/2019 Using Oracle Data Integrator
23/54
Designing an Interface
Before you start building an interface you:
Import Knowledge Modules into the project
Create your Metadata Models
Then in the interface you:
Define your source and target
Define the mapping and translations (business rules)
Define the flow requirements
Execute
To automate execution, move on to:
Packages
Procedures
Variables
Sequencing
8/11/2019 Using Oracle Data Integrator
24/54
Knowledge Modules
Knowledge Modules (KMs) are components with built in
knowledge (functions and logic) that connect technologies
to ODI.
Similar to HAL Adapters, but many more connection
options.
KMs connect to the technology, extract data from it,
transform the data, check it, integrate it, etc.
You can find a complete list of KMs at
http://www.oracle.com/technology/products/oracle-data-
integrator/10.1.3/htdocs/documentation/oracledi_km_reference.pdf
8/11/2019 Using Oracle Data Integrator
25/54
Knowledge Modules
Interfaces
LKM (Loading Knowledge Modules) are used to extractdata from the source database tables and other systems
(files, middleware, mainframe, etc.).
IKM (Integration Knowledge Modules) are used to
integrate (load) data to the target system.
CKM (Check Knowledge Modules) are used to check thatconstraints on the sources and targets are not violated.
.
8/11/2019 Using Oracle Data Integrator
26/54
Knowledge Modules
Models
RKM (Reverse Knowledge Modules) are used to perform acustomized reverse-engineering of data models for a
specific technology.
JKM (Journalizing Knowledge Modules) are used to create
a journal of data modifications (insert, update and delete)
of the source databases to keep track of the changes.
8/11/2019 Using Oracle Data Integrator
27/54
Knowledge Modules
Others
SKM (Service Knowledge Modules) are used to generatethe code required for creating dataservices.
8/11/2019 Using Oracle Data Integrator
28/54
An Example: Hyperion Planning Model
Dimensions
Planning
Metadata
8/11/2019 Using Oracle Data Integrator
29/54
An Example: Hyperion Essbase Model
Dimensions
Essbase
Metadata
8/11/2019 Using Oracle Data Integrator
30/54
Interfaces
An ODI object which loads one target data store with data
from one or more sources, based on business rules
implemented as mappings
8/11/2019 Using Oracle Data Integrator
31/54
Inside the Interface: Mappings
8/11/2019 Using Oracle Data Integrator
32/54
More on Mappings
Expression
Editor
ExecuteOn
UpdateTarget
Column
8/11/2019 Using Oracle Data Integrator
33/54
Multiple Sources, Joins and Filters
Joins (similar to Joins in RDMS)
Links multiple data sources together
Uses SQL Clauses
Multiple sources in a single interface must be connected
with a join. Create joins by dragging and dropping columns together
Alternatively, you can use the expression editor to add
them manually
Advanced joins
Connect models from different technologies or model /
schemas. (Heterogeneous Join)
Connect 2 or more datastores
8/11/2019 Using Oracle Data Integrator
34/54
Join Definition
Specify
Join type
Use ISO-92 Syntax,Join Order, Auto-
calculate Order
SQL
Syntax
8/11/2019 Using Oracle Data Integrator
35/54
Filter
Reduce amount of data
processed by the interface
by applying a filter.
Funnel icon represents
filter applied on the source
Use the Expression Editor
or manually type in the
SQL.
Test filter, right-click and
select Data.
8/11/2019 Using Oracle Data Integrator
36/54
Expression Editor
8/11/2019 Using Oracle Data Integrator
37/54
Flow Tab
8/11/2019 Using Oracle Data Integrator
38/54
SUNOPSIS_MEMORY_ENGINE
SUNOPSIS_MEMORY_ENGINE sits between source and
target
Temporary staging area that is created during interfaces to
facilitate mapping and translation
Not recommended for large data sets for performancereasons
Can be difficult to debug
We recommend configuring a separate staging area
8/11/2019 Using Oracle Data Integrator
39/54
Operator: Checking Status
8/11/2019 Using Oracle Data Integrator
40/54
Automation
Packages - Group of steps to complete a workflow.
Can include:
Interfaces
OS Commands
Procedures
Workflow can include error trapping and email status
notifications
Scenarios can schedule as a scenario or call a scenario
from a batch file
Variables - can have variables defined to pass parameters
(i.e. file name, current month, etc.)
8/11/2019 Using Oracle Data Integrator
41/54
Create Error Log Files in the Interface
Enable logging inthe interface in the
files to obtain more
details or send
information to a
text file for use in
debugging.
8/11/2019 Using Oracle Data Integrator
42/54
Errors
Errors are not always ODI errors, they are typically being
sent from other technologies (Oracle EPM in our case)
You will begin to see commonality of errors but they are
not always descriptive or useful.
Plan extra time for debugging interfaces
8/11/2019 Using Oracle Data Integrator
43/54
Real Life Tips
8/11/2019 Using Oracle Data Integrator
44/54
TipsGeneral Information
Good sources of information:
interRel 3 day ODI and EPM training class (available
this fall)
OTN Forum:
http://forums.oracle.com/forums/forum.jspa?forumID=3
74
http://john-goodwin.blogspot.com/
Get the 10.1.3.5 release
No patching required for Essbase, Planning, and HFM
Available at:
http://www.oracle.com/technology/software/products/odi/index.html
http://forums.oracle.com/forums/forum.jspa?forumID=374http://forums.oracle.com/forums/forum.jspa?forumID=374http://john-goodwin.blogspot.com/http://john-goodwin.blogspot.com/http://john-goodwin.blogspot.com/http://john-goodwin.blogspot.com/http://forums.oracle.com/forums/forum.jspa?forumID=374http://forums.oracle.com/forums/forum.jspa?forumID=3748/11/2019 Using Oracle Data Integrator
45/54
Design and Development Tips
Whiteboard the Topology before configuring in ODI
Standardize naming conventions for topology components
Incorporate as many error logs and error trapping
processes as possible.
Consider using a staging area versus Sunopsis Memory
Engine. This will allow performance optimization and error
resolution. If the source is a relational database, perform as many
translations in a view as possible.
8/11/2019 Using Oracle Data Integrator
46/54
Design and Development Tips
Recommend against using comma as a file delimiter for
source files. Use | or !. Versus loading Essbase Data with ODI, write a MaxL to
load data to Essbase and add it as a step in the ODI
package.
Plan extra development time, learning curve is steep and it
is more complex than HAL.
8/11/2019 Using Oracle Data Integrator
47/54
Project Tasks: Getting Started
Plan the scope, identify source and target
Establish success criteria
Identify the resources
Define the Topology
Define your Models Create Interfaces
Unit Testing
Automate with Packages
Schedule with Scenarios
Integration Testing
Deploy
8/11/2019 Using Oracle Data Integrator
48/54
How long will take?
No short answer, depends on the complexity of the
interface. If this is the first time, allow ample time to debug
issues.
Minimize risk by:
Working with infrastructure team for connectivity issues
Keeping data transformations simple or nonexistent in the
initial development (prototype)
Then consider the opportunities to automate translations or
business rules Plan out your topology on a whiteboard first! It is critical to
understand what links to what to avoid silly mistakes.
Use Relational Sources where possible.
8/11/2019 Using Oracle Data Integrator
49/54
OTHER
8/11/2019 Using Oracle Data Integrator
50/54
Security Manager
Separate from Shared Services
Manage security for all ODI modules
Create Users & Profiles
Assign access ODI Objects (data servers, projects, etc)
Methods (Edit, Create, Delete)
Servers
8/11/2019 Using Oracle Data Integrator
51/54
Metadata Navigator
J2EE application that provides web access to the
repositories
View Flow maps, trace the source of all data and drill down
to field level
Possible to trigger and monitor processing jobs from a web
browser using this tool
By default, installed on Jakarta Tomcat Application Server
8/11/2019 Using Oracle Data Integrator
52/54
Oracle Data Profiling and Quality
Integrator extends data quality features of Oracle Data
Integrator to provide more advanced data governance
capabilities.
Data Cleansing
Duplicates
Parsing
Data Enrichment
User driven rules
Integrated with ODI
8/11/2019 Using Oracle Data Integrator
53/54
ODI and EPM Training
Coming Soon from interRel Training!
Introduct ion to ODI for EPM Adm inistrators
This class will be useful for administrators or developers thatwill design, develop and / or deploy data integrations that
update Essbase, Planning or HFM with ODI.
Includes several hours of hands on workshops !!!
8/11/2019 Using Oracle Data Integrator
54/54
Thank You.
Edward Roskeeroske@interrel.com
BLOG: LookSmarter.blogspot.com
WEBSITE: www.interrel.com
mailto:eroske@interrel.comhttp://www.interrel.com/http://www.interrel.com/mailto:eroske@interrel.com