Fusion Middleware Oracle Data Integrator

Oracle Data Integration: Improving Information Agility

Mark Rabne

Senior Regional Sales Consultant

[email protected]

Mark Rabne

Senior Regional Sales Consultant

[email protected]

Oracle Confidential

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Oracle Confidential

Agenda

• Why Unified Information Management is key to Information Agility• Oracle’s Data Integration Offering & Strategy• How Unified Information Management impacts IT

and the Business• Customer Case Studies• Summary

© 2009 Oracle Corporation 5

Oracle Confidential

Q: Why can’t I see the reporting results that I need for all my

inventory?

Accessible Information

Ensure information is connected, available, reliable, and secure. Connect information across mixed sources and targets.

Q: Why do I have so many duplicate copies of my customer data?

Why isn’t it accurate?

Trusted Information

Ensure information consistency, quality, and authoritative data for improved, actionable business insight.

Q: Why are my applications referring to last weeks retail sales numbers?

Up-to-date Information

Integrate bulk data in real-time across data warehouses, data marts, business processes, MDM applications, BI applications.

Business Challenges Managing Information Requires Accessible, Up-to-date, and Trusted Information


Oracle Confidential

OLTP & ODS

SystemsData

Warehouse, Data Mart

Oracle

PeopleSoft, Siebel, SAP

Custom Apps

Files

Excel

XML

IT Challenges to Unifying InformationWhat is it costing you to unify your data?

Enterprise Performance

Custom Reporting

PackagedApplications

BusinessIntelligence

Custom Analytics

Data Federation

Data Warehousing

Custom

Data MartsData Access

Data Silos

SQL

Java

Batch Scripts

Data Hubs

Data Migration

Data Replication

Fragmented Data Silos

Slow Performance

Poor Data Quality

OLAP

Out of sync Missed SLAs


Oracle Confidential

Agenda




Oracle Confidential

Where Does Data Integration Fit?Essential Ingredient for Information Agility

SOA Business Intelligence

Enterprise DataWarehousing

Master Data Management

Oracle Confidential

Oracle Data Integrator Enterprise Edition (ODI-EE)Optimized E-LT for improved Performance, Productivity and Lower TCO

Declarative Set-based design

Real Time Change Data Capture

E-LT Transformation vs. E-T-L

High Speed Batch Data Movement

Data Warehouse

Planning System

OLTP DB Sources

Application Sources

Legacy Sources

Pluggable Knowledge Modules


Application

ODI-EE is part of ODI Suite, which is part of Fusion Middleware

Oracle Confidential

Oracle Data QualityEnsure Data Quality as Part of the Integration Process

ODI EEODI EE

Oracle Data ProfilingOracle Data Profiling Oracle Data QualityOracle Data Quality

Better Data Visibility

Tighter Data Control

Greater Data Accuracy

Metadata Profiling of Source & Target

Duplicate Detection, Matching & Merging

Visual Data Quality Tools


Oracle Confidential

• Comprehensive

• Optimized Performance with E-LT

• Improved productivity

• Lower TCO

• Open, Java-based, SOA native

Oracle Data Integration SuiteComprehensive solution for Data Integration


Oracle Data Integration Suite

Heterogeneous Data Sources

MDM, Business Intelligence, BAM, SOA Platforms, Applications

Master Data

Data Integrator

Data Services

DataGrid

Process Manager

ServiceBus

Data Quality

Data Profiling

HumanWorkflow

DataELT/ETL

Data Migration

Data Federation

Data Replication

Data Governance

Oracle Confidential

Content Management

Services

Business Intelligence

Services

CollaborationServices

TransactionProcessing

Services

Application and Data Services

Business Process Management

Information Management

File System Real-Time Data FeedOLTP Content MgmtData Warehouse OLAP Cube

Oracle Data Integration Suite

Oracle Data Warehousing

Oracle BI Foundation OLAP OLTP

Appliance Mining Partitioning Modeling

ETL/ELT CDC Data Federation Data Services Data Quality Master Data

SQL Tooling RAC

Unified Information Strategy from OracleComplete, Integrated Platform from Storage to Scorecard


Oracle Confidential

Agenda




Oracle Confidential

Improving Information AgilityOracle Data Integration as part of the Fusion Middleware family


Oracle Confidential

Reduced IT Costs Lower TCO for integration and migration projects


Oracle Confidential

Reduced IT Costs Lower TCO for integration and migration projects


• Reduced IT costs through consolidation and data migration

Oracle Confidential

• Heterogeneous sources • Heterogeneous targets• Any Data Warehouse• Flexible data loading

– Incremental load– Changed data capture

• Slowly changing dimensions• Data integrity and consistency• Data lineage

Data Warehouse

Cube

Cube

Cube

----------------

Op

era

tion

al

An

aly

tics

Metadata

LoadTransform

Capture Changes

Incremental UpdateData Integrity

AggregateExport

Data Warehousing Using ODIReduces costs by consolidating Data silos


Oracle Confidential

• Bulk-load historical data to new application

• Transform source format to target

• Synchronize new and old applications during overlap time

• Capture changes in a bi-directional way (CDC)

Old

Ap

plic

atio

ns

Ne

w A

pp

lica

tion

Metadata

Initial bulk loadCDC for

synchronization

Transformation to new application

format

CDC for loop-back synchronization

CDC

CDC----------------

Data Migration Using ODIBulk Load Data To Ease Application Migration


Oracle Confidential

• Projected cost savings of $400M in 3 Years

• Significantly reduced the hardware footprint through combined SOA+ ODI

• ODI + SOA helped to build out a shared services layer which helped provide greater flexibility

• Consolidate database (DB2, Sybase, Informix to Oracle)

• Consolidate ERP (PeopleSoft HR/Payroll, Decommissioning SAP),

• In addition the program helped to decommission Cystal and Cognos and adopt Oracle BIEE

Results Using Oracle Data Integration

VerizonReduces costs through consolidation

• Streamline and automate business processes around new hire on boarding

• Provide better business insight and manage FiOS supply chain

• Lower the cost of rewriting SAP, PeopleSoft & Legacy interfaces (about 209) through custom code

• Lower the cost of IT infrastructure from multiple systems

Challenges


Oracle Confidential

Eliminate Custom CodeImproved flexibility and declarative design lowers implementation costs


• Reduced development costs through declarative design

• Pre-integrations (Knowledge Modules) for faster time to value

Oracle Confidential

Declarative Set-based Design – “The WHAT” Provides significant reduction in custom coding

Conventional ETL DesignConventional ETL Design

ODI Declarative Design

Define How: Built-in Templates

Define WhatYou Want

Automatically GenerateDataflow

11 22

ODI Declarative Design


Define WhatYou Want


11 22


Define WhatYou Want


11 22

Define WhatYou Want


11 2211 22

• ODI provides Declarative Set-based Design, simplifies number of steps

• Benefits:

• Significantly reduce the learning curve

• Streamline access to non-IT pros

• Shorten implementation times

• High performance every time


Oracle Confidential

Journalize

Read from CDC Source

Load

From Sources to Staging

Check

Constraints before Load

Integrate

Transform and Move to Targets

Service

Expose Data and Transformation

Services

Reverse

Engineer Metadata

• Leverage Database Optimizations: Native SQL; Native Functions; Native Loads; Native Journaling / CDC

• Tailor to an organization’s existing best practices• Ease administration work• Reduce cost of ownership

Reverse

Journalize

Load

Check

IntegrateServices

Knowledge Modules Architecture

CDC

Sources

Staging Tables

Error Tables

Target Tables

WS

WS W

S

SAP/R3

Siebel

Log Miner

DB2 Journals

SQL Server Triggers

Oracle DBLink

DB2 Exp/Imp

JMS Queues Check MS Excel

Check Sybase

Oracle SQL*Loader

TPump/ Multiload

Type II SCD

Oracle Merge

Siebel EIM Schema

Oracle Web Services

DB2 Web Services

Sample out-of-the-box Knowledge Modules

Benefits

100+ Pluggable Knowledge Modules – “The HOW” For Faster Time to Value, Key to Flexibility, Modularity, and Extensibility


Oracle Confidential

• Increases development productivity by at least 50%

• Increases market share by 50% by being able to correctly profile each client based on the risk

• Produces data interfaces that are better documented and easier to understand

• Executes loading process that used to take eight hours in less than two hours despite dramatic increases in data complexity of the transformations


Financial Case Study: InterbankReduced Development Costs for ETL

• Improve loading of data warehouse

• Being able to correctly profile each client based on the risk

• Improve the integration of banking systems, collection systems, and front office systems

• Modernize batch exchanges and EDI interfaces toward real time

Challenges


Oracle Confidential

Improve PerformanceReal-time data warehousing for real results


• High volume performance for bulk data movement through ELT

• Real-time feeds, improved overall responsiveness

Oracle Confidential

Conventional ETL ArchitectureConventional ETL Architecture

Extract LoadTransform

Next Generation ArchitectureNext Generation Architecture

“E-LT”“E-LT”LoadExtract

Transform Transform

Optimized Data Loading through E-LTThe key to improved performance and reduced costs

• E-LT provides flexible architecture for optimized performance

• Benefits:

• Leverage Set-based transformations

• Improved performance for loading, no network hop

• Takes advantage of existing hardware

26© 2009 Oracle Corporation 26

Lookup

Oracle Confidential

Changed Data CaptureEnables Real-time Data Warehousing

• Change Data Capture provides real-time updates of mixed data sources without impacting performance

• Benefits:

• Improved performance for extraction – delta based

• Operates in near-real time

• Timeliness of data

27© 2009 Oracle Corporation 27

Oracle Confidential

• “Having access to key business metrics in real-time is no longer a fantasy. In short, Oracle Data Integrator gave us the ability to make better decisions and better manage our bottom line.”

• Found a way to ensure that data warehouse was constantly updated.

• Even highly complex transformations are automated

• Supporting several terabytes of data stored in the enterprise warehouse, and millions of daily transactions


Overstock.comReal time Enterprise Data Warehouse

• Wanted to enable sales, finance, marketing and merchandising teams to have access to near real-time data so that they could make timely, more intelligent business decisions.

• Wanted to know at any point in time if company performance is meeting the target metrics.

• Needed a data integration product that could handle our high-volume loading and transformation requirements in near real time.

Challenges


Oracle Confidential

Business Benefits of Unified Information Management Agile and Intelligent for Business


Oracle Confidential

Improve Agility for your DataRe-use information for improved business performance


• Connect SOA + Data Integration for improved flexibility and agility of business processes

Oracle Confidential

Bulk Data ServicesProvides business processes integration and improved re-use

Oracle SOA SuiteOracle SOA Suite

Business Activity Monitoring

Web Services Manager

Business Rules Engine

Enterprise Service Bus

Oracle BPEL Process Manager

Bulk Data Processing

Oracle Data IntegratorOracle Data Integrator

ODI Knowledge Modules

Transform.Services

Changed Data

Capture

Data Services

ODI Connectivity Framework

• Decouple business apps from underlying data-centric applications

• Recombine existing services and app components instead of new development

• Benefit

• Enables efficiencies in transformation, data loading, data synchronization, all at improved performance


Oracle Confidential

• 75% improvement of Business Process Execution

• Together with Oracle SOA Suite (Oracle BPEL PM) provided business optimization, process visibility, exception handling

• Seamless data integration from heterogeneous sources leveraging SOA helped to create a more flexible, standards based integration approach

• Closed loop processing using BAM and Data Integration combined eliminated ordering/replenishing inventory errors by reducing inaccurate data


Ross StoresData Warehouse Agility ‘for Less’

• Hard to consolidate consistent, accurate inventory information across over 800 stores

• Lack of visibility to errors to the business

• Higher cost and time to delivery of new value

• Current state was complex, didn’t scale, and difficult to manage

• Non-standard approach required coding paradigm, inconsistent error handling

Challenges


Oracle Confidential

Impact of Clean DataReduced risk and ensure compliance


• Improve data quality, invoke data cleansing rules, data profiling

Oracle Confidential

Data Quality LifecycleEnsure Data Quality as Part of the Integration Process

Integration Integration ProcessProcess

Target

BPEL Process ManagerBPEL Process Manager

HumanHumanWorkflowWorkflow

Sources

Erro

rs

Co

rrection

s

Error Recycling

ErrorsFlow Check Data Quality Process

StaticCheck

• Data Quality and Profiling provide cleansing of data as part of integration process

• Benefits:

• Risk reduction from bad data

• Compliance – where information flows

• Improves visibility, business insight

• Trusted data


Oracle Confidential

• Increased overall revenue by $2M in one month by improving data quality

• Saved $1M in acquisition: company had bad data

• Increased direct/inside sales productivity

• Increased number of sales (more calls per day), avg order (better targeting, messaging)

• Increased cross-selling: Cross-divisional lead sharing

• 280% revenue growth in 3 years


Leading Online Job Placement Firm in North AmericaWhat’s the Return on investment for Clean Data?

• Integrate data across multiple systems, including legacy acquisition infrastructure.

• Better profile and detect issues with data among multiple systems

• Remove duplicates and eliminate errors associated with better identifying customer data

Challenges


Oracle Confidential

Improving Business Insight with Data IntegrationImproved Data Lineage – better trusted data


• End-to-end data lineage - ensure pervasive reach of BI/EPM applications to multiple source systems

Oracle Confidential

Data Warehouse/Data MartPlanning & Budgeting

Metadata-driven Data LineageMetadata-driven Data Lineage

Oracle BI EE, Essbase & ODI EE

Planning &

Budgeting

Analytic Dashboards

Improve Visibility of Unified DataOracle Data Integrator EE & Oracle BI


Oracle Confidential

Oracle OnDemand, Oracle Hyperion Planning, Oracle Data Integrator

“By using Oracle Data Integrator, we were able to extract, transform, and aggregate financial information from two distinct systems and swiftly load data into our Hyperion Planning application.

By automating the data integration process and eliminating manual steps, we have saved time and resources, greatly reduced the risk of error associated with manually entry, and have more confidence about the data critical to our key planning and decision making processes.”

.


Lower TCO for Performance ManagementUsing Oracle Data Integration

Oracle Confidential

Unique Design

Advantage

Efficiency and

Innovation for IT

Agile and Intelligent

for Business

Summary: Improving Information AgilityUsing Oracle Data Integration


Oracle Confidential

Next Steps

1. Unify enterprise information for improving information agility

2. Leverage E-LT and Change Data Capture for real-time BI/DW

3. Align business demands and IT requirements for data initiatives – Ask “What If” questions

4. Assess the existing and future needs of data across your enterprise

5. Leverage Oracle Partners and Oracle Professional Services

Additional Customer Examples

Global Case Studies

Order Replenishment, Online CatalogJ. Crew

• Simplified the end-to-end data integration with all their core IT systems and database platforms to reduce the total cost of ownership of their order replenishment and cataloging

• Benefited from the Oracle advantage in batch processing, bulk data transformation for increased performance

• Successfully leveraged SAP and Teradata integration to phase out legacy systems and ultimately reduce costs

• Order replenishment for online catalog system • Define new corporate standard for data integration• Better integrate data across 175 retail stores and 52

outlet chains

Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges

• Replace antiquated home-grown solution of scripts• Teradata bulk data support, and help integrate data

across SAP, Red Prairie Inventory systems

Sales and Marketing Data Warehouse Nestle

• Provided 100GB of actionable data to the business users• Replaced weekly insertion with daily recording of change in status with time imprint for all clients—allowing for

more effective client follow-up, for example• Improved response times by transferring reporting processes to the data warehouse• Improved reactivity for restocking requirements thanks to daily updates on stock quantity and value for each store• Provided overall view of the status of business processes, resulting in improved user communication

• Hard to consolidate consistent, accurate inventory information across over 800 multiple store chains

• Lack of visibility to errors to the business• Higher cost and time to delivery of new value

Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges

• Manage a data warehouse for operational transactions and marketing information

• Lack of high-value-added actionable data on clients, orders, stock, etc.

• Enable the marketing and CRM team to better target its campaigns and activities across 50 countries

• Anticipate future reporting challenges, in light of the strong increase in the volume of production data

Quote AttributionTitle, Company

For More Information

• Visit the Oracle Fusion Middleware 11g web site at http://www.oracle.com/fusionmiddleware11g

• Oracle Data Integration on oracle.com www.oracle.com/goto/odi

• Oracle Fusion Middleware on OTN http://otn.oracle.com/middleware

Get Started

• Datasheet: http://www.oracle.com/products/middleware/odi/docs/odiee-datasheet.pdf

• Blog: http://blogs.oracle.com/dataintegration

• Technical information available at: http://www.oracle.com/technology/products/oracle-data-integrator/index.html

• Data Integration Events http://www.oracle.com/events

Resources


Hands-On ODI WorkshopColumbus, OH – October 6th

Contact: [email protected]

Oracle Confidential

Information Management

Please rate this session now.Please rate this session now.

Thank You!!Thank You!!

Session Evaluation

Oracle Confidential

Questions


Fusion Middleware Oracle Data Integrator

Documents

Transcript of Fusion Middleware Oracle Data Integrator