2009.10.22 S308460 Cloud Data Services

Cloud Data Services - S308460

Jeff PollockSenior Director, Fusion Middleware Product Management

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.


relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

“I believe, over time, more and more

software will be delivered as a

service. I totally believe that... We

have to be good at this, or we have a


have to be good at this, or we have a

problem”

- Larry Ellison

<Insert Picture Here>

Agenda

• Data Services

• Cloud Computing

• Data Services in the Private Cloud

• Pragmatic First Steps

– Pre-Built On-Demand Data Integration


– Pre-Built On-Demand Data Integration

– Service-based ETL Architectures

– Canonical Data Access Services

• Lessons Learned– Unify and Rationalize Tools

– Build Dedicated Competency Centers


What are Data Services?

A Bit of Data Services History

• Reality Check About “Data Services”

– This new hype is about an old idea – reusable data is a good thing

– 1968 – formation of EDI document translation standards for banks

– 1986 – Design by Contract: Precondition and Postcondition driven services

– Early 1990’s – tech roots of Informatica, IBM DataStage and Ab Initio

– 1994 – Design Patterns: Factory, Adapter, Façade, Proxy, Mediator…

– Late 1990’s – the application server data services (Kiva, Net Dynamics…)

• Long Before Java (and Before EAI or SOA or Cloud Computing)


• Long Before Java (and Before EAI or SOA or Cloud Computing)

– Data interchange among: banks, railways, airlines, militaries, etc.

– Metered software access and computing services from Mainframes

– Enterprise data warehouses for analytic purposes

– Object-oriented design patterns for decoupled data access

• So What’s Different About Data Services Today?

– Modern (new and different) standards & protocols

– Business scale, complexity, and expectations

– Near-infinite elasticity

Java Programmer’s View of Data ServicesObjects & Components in my JEE and JSE infrastructure

JEE BUSINESS SERVICES

• SCA– Service Component Architecture – Key framework spec for Java Business Services

JEE DATA SERVICES

• DAS, DTO & JDO– Data Access Service – Java service for accessing RDB’s and other Source Types

– Data Transfer Object – Java object for moving data (w/in container)

– Java Data Object – Java object for data containment (POJO-oriented)


• JPA & SDO– Java Persistence API – EJB 3.0 spec for accessing external data (typically relational)

– Service Data Object – Entity objects targeted for SOA data use

• ORM– Object Relational Mapping – A generic term typically applied to some solution approach to handling

“impedance mismatch”

• Oracle Toplink – A proven industrial strength data services layer for managing object persistence

• JAXB / OXM– Java Architecture for XML Binding – Standard set of APIs for XML

– Object to XML – Toplink extended implementation of JAXB

A Classification Taxonomy for Data Services

JAVA DATA SERVICE IMPLEMENTATION PATTERNS

• DAS & DTO – Access and in-container data transfer

• JPA, JDO & SDO – Enterprise intra-container data entity transfer

• JAXB & OXM (Toplink) – Access and binding to XML data

INFRASTRUCTURE DATA SERVICE FUNCTIONAL PATTERNS

• Master Data Services – Golden Records, Hierarchies, & Policies

• Bulk Data Services – Bulk Refresh, CDC, Trickle Feeds, SCD…

• Data Access Services – Virtualization, Federated Query, ORM…

– Really Urgent Data Access – Data Grid Cache / Virtualization


– Really Urgent Data Access – Data Grid Cache / Virtualization

– Real Time Data Access – Query Federation & Layered Data Services

– Right Time Data Access – Data Hubs & Warehouses

• Data Event Services – Low Latency CDC, Trigger-Based CDC…

• Data Quality Services – Cleansing, Matching, De-Duplication…

PHYSICAL DATA SERVICE DEPLOYMENT PATTERNS

• Data Hub – Centralized cache (variation of Data Virtualization) or repository

• Data Grid – Distributed RAM-based data partitions with decentralized access

• Data Federation – Late-binding of query-time access to source data

Enterprise Data Service Functional Patterns


Example: Service-based Data IntegrationBest of Breed Data Integration as a SOA Service

Oracle Data Integration• Best Performing E-LT Architecture

• Works with any Data Warehouse

• 100% Java & SOA components

Oracle Data Replication• Changed Data Capture (CDC)

• Query Offloading / DB Tiering

Example Use Cases


High PerformanceETL & Replication

Any Data SourceData Warehouse

& OLAP

Example Use Cases• Bulk Data Transformation (any2any)

• XML/EDI Large File Handling

• SOA-driven Business Intelligence

• Load DW from SOA

• Unified Data Steward Workflow(ETL Error Hospital w/BPEL PM)

• ERP Migration, Replication / Loading

• Query Offloading & Zero Downtime

Oracle Data Integrator EE is the standard for:

• Oracle Business Intelligence• Oracle Performance Management• Oracle Database, Essbase & Tools

• Oracle Fusion Applications• Oracle Fusion AIA• Oracle Fusion Middleware

So What Are Data Services?Casting a Wide Technical Net…

• A General Software Pattern?

– Using contract-based development patterns for data distribution…

• A Java Software Pattern?

– Pick your favorite TLA: DTO, DAS, SDO, JPA, ORM, JAXB, OXM…

• A Web Services Stereotype?

– Dedicated SOAP services for reusable data manipulation and access…


– Dedicated SOAP services for reusable data manipulation and access…

• Shared Services for Enterprise Class Data Processing– Shared – common for a group of consumers/clients

– Services – contract-driven interfaces to functional behaviors

– Enterprise Class – exceptionally demanding SLA’s and KPI’s

– Data Processing – the heart-beat for all big businesses


What is Cloud Computing?

Everyone Is Talking About Cloud


What Is Cloud Computing?

*


Cloud Computing is often characterized by:

• Virtualized computing resources

• Seemingly limitless capacity/scalability

• Dynamic provisioning

* Source: Wikipedia.org

• Multi-tenancy

• Self-service

• Pay-for-use pricing

Haven’t we Heard About this Before?

*

CloudGrid


Cloud Grid Computing is often characterized by:

• Virtualized computing resources

• Seemingly limitless capacity/scalability

• Dynamic/elastic provisioning

* Source: Wikipedia.org

• Multi-tenancy

• Self-service

• Pay-for-use pricing

Cloud

Cloud Computing Benefits are Speed & Cost-Driven

Speed

Cost


Cloud Layers: SaaS, PaaS, IaaS

Applications delivered as a service to end-users over the Internet

Software as a Service

App development & deployment


Infrastructure as a Service

Platform as a ServiceApp development & deployment platform delivered as a service

Server, storage and network hardware and associated software delivered as a service

Public Clouds and Private Clouds

IaaS

PaaS

SaaS INTERNE

Public Clouds

IaaS

PaaS

SaaSINTRANET

Private Cloud


IaaS ET

Users

Public Clouds:

• Lower upfront costs

• Economies of scale

• Simpler to manage

• OpEx

Private Cloud:

• Lower total costs

• Greater control over security, compliance & quality of service

• Easier integration

• CapEx & OpEx

Both offer:

• High efficiency

• High availability

• Elastic capacity

Private Clouds are Majority of IT Spend on Cloud

Public Cloud


Public Cloudvs.

On-Premise


Data Services for the Private Cloud

Cloud Layers: Where’s the Data?




Platform as a Service

Cloud Layers: Data at Every Strata!


Data as a Service

Public Clouds Private Cloud

Corporate Cloud

Services



Platform as a Service

Storage Clusters

Hubs, ETL and Access Services

Important Challenges For Data Owners Today

Access and Distribute Data

with agility


Do it all efficiently

Scale data access while preserving

quality of service

Technology that Starts to Address the Challenges

Grid/Dynamic Resourcing

Management/ Automation

Shared Data Services


Virtualization

Data Integration: Reusable Data Components

create processes

create data services

IT

Dept App

ProcSvc

define dataintegration scenarios

ProcSvc

find data services

Department App Owner

build app

include data

Dept App

ProcSvc

ProcSvc

Dept App

ProcSvc

ProcSvc


DataQuality

DataService

Integrator

DataIntegrator

Oracle WebLogic Suite-based Application Grid

Oracle SOA SuiteOracle Data Integration Suite

Svc SvcSvc

DB

Svc

Integration

Svc

Legacy

Proc Proc Proc

Enterprise Manager

Self Svc

Data Movement & Delivery Data Integration Designer

FU

NC

TIO

NA

L

SE

RV

ICE

S

ETLData

Service

DataAccessService

DataEvent

Service

DataQualityService

DataGrid

Service

SOARegistryService

SOAMetadataService

MasterData

Service

CL

IEN

TS

MDMApplications

SOAPlatforms

OracleApplications

BusinessIntelligence

ActivityMonitoring

Fusion Applications

Kinds of Enterprise Data Integration Services


Data Movement & Delivery

Data Integration Foundation

Data Integration Designer

Data ConnectivityTE

CH

NIC

AL

PL

AT

FO

RM

DA

TA

SO

UR

CE

S

Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid | Content Management | Search

MDMApplications

SOAPlatforms

OracleApplications

BusinessIntelligence

ActivityMonitoring

Fusion Applications

Data Distribution & Delivery Services

Data Integration Foundation

Data Quality Core

Bulk Data API Data Quality API

Java SDO

Data Replication

Event-Driven (CDC)

Data Relationship Core

Data Virtualization

Matching/Dedupe’s

Cleanse/Parse

Lifecycle Mgmt

Data Classification

Oracle Data Integration Suite

Data Federation Core

Data Caching

Realtime Query

Process-Oriented

Message-Oriented

Bulk Data Core

Bulk Acquisition

Trickle Feeds

Change Data Capture

Event-Based


Data Connectivity

Metadata Services Models Relationships Lineage Object Cache

Relational Databases Non-Relational DBs Various File Formats XML Data

Packaged Applications Standard Msg Formats Message Queues Semi-Structured Data

Exchange APIs

Cluster AnalysisData Reconciliation

Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid | Content Management | Search

Query FederationComplex Transforms Integrity Constraints

Data Integration, SOA, BPM & IdM: Putting it Together

create reusable

DATAcomponents

ITDept App

UI

ProcSvc

Svc Proc UI UIProcSvc

Svc Proc UI UIProcSvc

register components

set policies

Department App Owner

authenticate

build app

discover authorized

components

Include components

UI


Svc Proc UI Self-Service Interface


Oracle SOA Suite

Oracle BPM Suite

Oracle Identity Mgt Oracle

Enterprise Manager

Oracle Data Integration

Suite

UIProcSvcSvc Proc UI UIProcSvc

policies

Cloud and Fusion Middleware Data Services

Self-Service InterfaceShared Components


ODI SuiteOracle

SOA SuiteOracle

BPM SuiteOracle

Identity Mgt Oracle Enterprise Manager

Cloud Enabling CapabilitiesKey Benefits


Cloud Enabling Capabilities

•Basis for reusable data service components

•Simple and consistent data integration framework

•Modular and heterogeneous

•Unified Management

•Development efficiency

•Deployment agility

•Elastic Scale-out

Why Oracle

•Most integrated

•Easiest to use and manage

Take-Aways

Cloud Data Services is an architecture for centralized, shared, elastic data resources internal to an enterprise that allows departments to rapidly deploy and easily manage application and analytic data.

• Fundamental Enablers:• Application server

• Data Integration

• Key Benefits:• Efficiency


• Data Integration

• SOA/BPM

• Identity Management

• Enterprise Management

• Oracle:• Most complete, comprehensive offering

• Easiest to set up, integrate, and operate

• Best performance, automation, and self-service

• Agility

• Quality of Service


Pragmatic First Steps

Leverage Pre-Packaged Integration Solutions

Siebel CRM Integration Pack for Oracle EBS Order Management

Cross Industry Process Integration Packs- Design, Sell, Plan and Execute

►Order to Cash

Agile Product Lifecycle Management integration Pack for Oracle E-Business Suite►Design to Release

Oracle CRM On Demand Integration Pack for Oracle E-Business Suite►Opportunity to Quote

�Oracle Transportation Management (Glog) Integration to E-Business Suite

�Oracle CRM On Demand Integration to Siebel CRM

�Demantra Integration Pack for Siebel CRM Consumer Goods

�Siebel Life Sciences Integration for Oracle Adverse Event Reporting System

Direct Pre-Built Integrations”- Data Integration made simple


Siebel CRM Integration Pack for Trade Promotion Management►Trade Promotion Mgmt

Oracle Comms Billing and Revenue Management Integration Pack for Oracle E-Business Suite: Revenue Accounting

Siebel CRM Integration Pack for Oracle Comms Billing and Revenue Management: Order to Bill

Siebel CRM Integration Pack for Oracle Comms Billing & Revenue Management: Agent Assisted Billing Care

►Comms Order to Bill

►Comms Customer care

Industry Process Integration Packs- Transform key business processes

►Comms Revenue Accntg

Siebel CRM Integration Pack for i-flex FLEXCUBE Account Originations►Banking Acct Originations

Siebel CRM Integration Pack for Banking Account Originations►Banking Acct Originations

Dept App 1

Dept App 2

DataService

Shared Service Enterprise

Manager

DataService

Dept App 1

Elastic ETL

Sense demand spike

Commit to a Service-based ETL Architecture

Running Oracle Data Integrator ETL as a managed Java process enables CONSISTENT management and monitoring – only Oracle can do this!


ManagerElastic ETL Services

WebLogicServer Cluser

of NodesAdjust

capacity

WebLogic Suite-based Application Grid

Consider a Canonical Data Services Architecture

Service Clients Canonical Data Services Physical Sources

AddressesCustomer

Profile

Pending Payments

Credited Payments

Orders

History

JDBC

Data Service Internal

Customer

Order

Data Service APIs

Web Srvcs

Adapters

Ja

va

, W

eb

Se

rvic

e, X

Qu

ery

, S

QL

, J

MS

Mgmt Dashboard

ESB


• Data services layer provides data access capabilities• Data services modeling organizes services for use and reuse

Repository

Enterprise Data

Payment

Adapters

Custom

Ja

va

, W

eb

Se

rvic

e, X

Qu

ery

, S

QL

, J

MS

Customer Portal

BPM


Lessons Learned

Create Data Integration Competency Centers

PeoplePeople

Data StewardsData Stewards

Data ArchitectsData Architects

ETL DevelopersETL Developers

Data Service ModelersData Service Modelers

Shared Services / SOAShared Services / SOA

ETL ToolsETL Tools

Data Quality ToolsData Quality Tools

Data ProfilingData Profiling

Data FederationData Federation

TechnologyTechnologyProcessProcess

DataGovernance

Data Profilingand Quality

Data Integration


End-to-End Lifecycle Governance

Database AdministratorsDatabase Administrators

Compliance OfficersCompliance Officers

Data FederationData Federation

Data ReplicationData Replication

Data ArchivingData Archiving

Data MaskingData Masking

Unify and Rationalize Different Vendor Tools

Solution Areas Key Buying Attributes

Data Modeling Relational, object and service models.

ETL / Bulk Data Integration Apps, DB and Java integration. Flexible ETL and ELT architectures.

Data Quality, Matching and Cleansing

Rule-based and statistical learning. Fast bulk and realtime match.

Data Profiling In place and out-of-place profiling.


Data Profiling In place and out-of-place profiling.

Data Federation Canonical modeling and data redaction.

Data Archiving Data masking, lifecycle management

Data Replication / High Availability

Log-based capture with sub-second,transaction-safe data apply

Data Grid Multi-platform limitless object scale-out.

Management and Monitoring Common metrics, logging, and charts.

Quote AttributionTitle, Company

For More Information

Get Started Resources


• Visit the Oracle Fusion Middleware 11g web site at http://www.oracle.com/goto/fmw11g/index.html

• Oracle Data Integration on oracle.com www.oracle.com/goto/odi

• Oracle Fusion Middleware on OTN http://otn.oracle.com/middleware

• Information on GoldenGate: http://www.oracle.com/goldengate

Get Started

• Datasheet: http://www.oracle.com/products/middleware/odi/docs/odiee-datasheet.pdf

• Blog: http://blogs.oracle.com/dataintegration

• Technical information available at: http://www.oracle.com/technology/products/oracle-data-integrator/index.html

• Data Integration Events http://www.oracle.com/events

Resources

Developing in the Cloud @ Middleware LoungeAt the Marriott Hotel

• Free Amazon Web Services access• $35 credit for development time

• Daily workshops: “Developing in the Cloud”

• Q&A with Amazon representatives and other experts

• Internet access• Wireless access

• SunRay machines also provided

Workshops DailyWorkshops DailyWorkshops DailyWorkshops DailyIn the Middleware Lounge

•Daily, 11:30 AM Introduction to Amazon Web Services

•Monday 2:00 PM Developing with WebCenter and Oracle Content Management in Amazon Web Services

•Tuesday 2:00 PM Developing with


• SunRay machines also provided

• Recharging stations for your phone or laptop

• Free food and beverages

• Live Twitter feeds and blog updates

• Watch keynotes and featured sessions while relaxing, recharging, and checking email

•Tuesday 2:00 PM Developing with WebCenter and Oracle Content Management in Amazon Web Services

•Wednesday 2:15 PM Developing with Oracle SOA Suite in Amazon Web Services

•Thursday 2:30 PM Developing with Oracle SOA Suite in Amazon Web Services

OpenWorld Activities for Oracle Data Integration

Oracle GoldenGate & Oracle Streams: The Future of Oracle Replication and Data Integration

Panel With Hasan Rizvi and Juan Loaiza, SachinChawla, and Ali Kutay

Tuesday, 2:30 - 3:30 PM

Moscone South Room 270

Monday 10/1211:30am Introducing Oracle GoldenGate Products Marriott Golden Gate B1

1:00 pm Event-Driven BI with Oracle Data Integrator Marriott Salon 1

2:30 pm Real-time BI Using Oracle GoldenGate + ODI Marriott Golden Gate B1

4:00 pm RIM Case Study: Using Oracle GoldenGate Marriott Golden Gate B1

5:30 pm Integrate Oracle BI Solutions with ODI Moscone Room 307

Tuesday 10/1311:30am Oracle GoldenGate for Zero-Downtime Migrations, Marriot Golden Gate B1

11:30am Enterprise Data Services in the Cloud Marriott Hotel Salon 7

1:00pm Oracle GoldenGate Deep Dive: Architecture Marriott Hotel Golden Gate B1

1:00pm Deep Dive on Real-World Use Cases for ODI Hilton Hotel Golden Gate 3

4:00 pm Bringing BPM and Data Integration Together Marriott Hotel Nob Hill CD


Hands-on Labs Marriott Hotel Golden Gate A2

Monday 11:30am, Wed 1:15pm: Introduction to ODI-EE Tuesday 4:00pm, Thurs 10:30am: BI + ODI-EE

Monday 1:00pm, Wed 5:00pm: Data Services using ODI-EE Tuesday 2:30pm, Thurs 9:00am: Advanced ODI-EE

Wednesday 10/1411:45am Ensure a Successful Siebel CRM Upgrade Moscone West L2 Room 2001

1:00pm ELT, Federation, and Replication: Overstock.com Moscone West L3 Room 3014

Thursday 10/15

9:00amOracle Information Framework: The Power of the Combined ODI and MDM Moscone South Room 304

DemoGrounds

Moscone West

ODI, OWB, W-093, W-094

Oracle GoldenGate, 3709

The preceding is intended to outline our general product direction. It is intended for information

purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any

material, code, or functionality, and should not be relied upon in making purchasing decisions.


relied upon in making purchasing decisions.The development, release, and timing of any

features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

For More Information

search.oracle.com


or

oracle.com

2009.10.22 S308460 Cloud Data Services

Documents

Transcript of 2009.10.22 S308460 Cloud Data Services