53642 Oracle Goldenga

7/24/2019 53642 Oracle Goldenga

1/68

http://www.oracle-developer-days.com

Copyright 2012, Oracle and/or its affiliates. All rights reserved

Introduction to Oracle Data Integrator

&

Combining Oracle Data Integrator and

Oracle GoldenGate

Lab Guides


2/68

Introduction to ODI & Combining ODI with OGG


Copyright 2012, Oracle and/or its affiliates. All rights reserved 2

TABLE OF CONTENTS

Hands on Lab - Introduction to Oracle Data Integrator EE................................................3Overview ..................................................................................................................................................... 4Time to Complete ...................................................................................................................................... 4Prerequisites .............................................................................................................................................. 4Architecture Overview .............................................................................................................................. 5Sources: ..................................................................................................................................................... 5Target: ......................................................................................................................................................... 5Part 0: Preparation Steps .......................................................................................................................... 6Part 1: Review Topology Setup ................................................................................................................ 7Part 2: Extracting, Loading and Transforming Product Data to Oracle ............................................. 11

Dimension Tables .................................................................................................................................. 11

High Level Architecture .......................................................................................................................... 11Part 3: Extracting, Loading and Transforming Sales Data to Oracle Dimension Tables ................. 29Optional Part 4: Extracting, Loading and Transforming Promotion Data to Oracle Dimension Tables................................................................................................................................................................... 38

High Level Architecture .......................................................................................................................... 38Optional Part 5: Creating a New ODI Package to Execute the Full Dimension and Sales Table Build.................................................................................................................................................................... 46Summary .................................................................................................................................................. 50

Hands on Lab: Combining Oracle Data Integrator and Oracle GoldenGate ......... 51Overview ................................................................................................................................................... 52Part 0: Preparation Steps ........................................................................................................................ 53Part 1: Enable Staging Model for CDC .................................................................................................. 54Part 2: Configure Oracle GoldenGate Using Generated Files ............................................................ 60

Part 3: Initialize Load to Staging and Target Tables ............................................................................ 61Part 4: Initialize CDC Process and Perform Change Operations ........................................................ 65Summary .................................................................................................................................................. 68


3/68




HANDS ON LAB -INTRODUCTION TO ORACLE

DATA INTEGRATOR EE

Abstract:

This

lab

will

highlight

to

Developers,

DBAs

and

Architects

some

of

the

best

practices

for

implementing

E-

LT

techniques

to

improve

performance

and

reduce

data

integration

costs

using

Oracle

DataIntegratorEnterprise Edition(ODI-EE).In this lab,participantswillwalk through thestepsthatareneeded

to

load

and

transform various

sources

into

a

relational

table

target.

This

will

also

include

how

to

deploy

and

manage

interfaces

using ODI-EE.

Scenario:OurcompanyisintheprocessofbuildingaBIsystemforreportingonourcustomerandsalesdata.

The

physical

schema

for

our

target

star

schema

has

been

finalized

and

we

have

been

assigned

to

buildtheinterfacesandpackagesinODItoloadthisstarschema.

Oursupportingdataisinvariousheterogeneoussystems,locatedacrosstheenterprise. Sales,customerand

country

data

will

be

imported

from

our

Sales

system

located

on

MySQL.

Our

product

management

systemcontainsourproductandcostingdataand this informationwillbeprovidedascommadelimitedextracts. Ourmarketingsystemisan internetbasedsystem;wewillbeprovidedanXMLextractofourpromotional

data

from

this

system.

The following lessons will walk us through various steps that are needed to create the interfaces andpackagesneedtoloadthereportingstarschema.


4/68




OVERVIEW

Time to Complete

Create interfaces and packages 50 Minutes

Prerequisites

Before you begin this tutorial, you should

Have a general understanding of RDBMS concepts. Have a general understanding of ETL concepts.


5/68




ARCHITECTURE OVERVIEW

Sources:

Source Type

Sales MySQL

Customers

MySQL

Countries

MySQL

Promotions

XML

Products

CSV

Costs

CSV

Target:

Source Type

Sales

Oracle 11g

Customers Oracle 11g


6/68




PART 0:PREPARATION STEPS

In these steps you will clean and setup the environment for this exercise

1. If you have used ODI on this environment previously, you need to reset the repositories to theoriginal state. If this is a new environment such as a fresh installed VM, you can skip this step.Warning: If you have custom ODI projects in the repository on this machine, they will bedeleted.

Open a terminal and enter reset_odi. PressEnter.


7/68




PART 1:REVIEW TOPOLOGY SETUP

1. The connectivity information has already been setup for this hand on lab. This information issetup within the Topology Manager of ODI. The next steps will walk you through how to reviewthis information.

2. Open the Oracle Data Integrator Studio:

Open a terminal and execute odistudio. Press Enter.

3. Go

to

the

Topology

Navigator

and

press

ConnecttoRepository

4. Select HOL Basicfrom the drop-down list of logins and click OK.

5. Within the Physical Architectureaccordion, expand the Technologies folder


8/68




6. For this HOL the connectivity information has been setup. Connectivity information is setup forFile, Oracle, MySQLand XMLsources and targets.


9/68




7. Double-click on the Oracle_DWdata server to review settings

8. Expand the data server and double-click on ORACLE_DW.OOW_HOL_DWphysical schema toreview the settings.


10/68




9. Review

any

additional

items

as

needed.


11/68




PART 2:EXTRACTING,LOADING AND

TRANSFORMING PRODUCT DATA TO ORACLE

Dimension Tables

1. The

first

interface

to

be

created

will

load

the

products

data

(PRODUCTS.CSV)

into

the

Oracle

star

schema.

The

products

data

will

be

joined

with

the

current

cost

data

(CURRENT_COSTS.csv)

file

so the product cost (UNIT_COST) can be stored in the PRODUCTS dimension table(PROD_COST)

High Level Architecture


12/68




2. Open the ODI Studio, choose the Designer tab; If not already logged in, click on Connect ToRepository

3. SelectHOLBasicfromthedrop-downlistofloginsandclickOK.

4. OpentheDesignernavigatormenuandmakesurethattheDefaultExecutionContext issettotheDevelopmentcontext(notProduction).

5. OpentheModelsaccordionintheDesignernavigatorbyclickingonthelabel.


13/68




6. Expand

the

ORACLE

DW

TRG

folder

7. The target Oracle star schema has been physically created within the RDBMS. But the logicaldata stores to represent the physical model have not been created within ODI. The logical datastores can be created by reverse engineering the Oracle ORACLE_DW_SALES.

Info: A m odel is a set of metadata def in i t ions regarding a source su ch as a databaseschema or a set of f i les. A m odel can contain m ult ip le data stores, which fo l low the

relat ional concept o f colum ns and rows and can b e database tables, structured f i les, or

XML elements w ith in an XML docum ent.

8. To

reverse

double-click

the

existing

model

ORACLE_DW_SALES.The

model

editor

is

opened

on

therightside.


14/68




9. Click

the

Selective

Reverse-Engineering

tab

10. Check the SelectiveReverse-Engineering checkbox and the Objects to Reverse Engineercheckbox.

The

list

of

tables

to

reverse

engineer

is

shown.

11. ClicktheReverseEngineer( )buttoninthetoolbar.


15/68




The reversed tables will show up in the model tree and are now logical data stores within ODI that canbe used as a source or target within an ODI Interface.

Note:The other data stores that are needed for the hands o n lab have already been created.

12. The

first

interface

to

be

created

will

load

the

products

data

(PRODUCTS.CSV)

into

the

Oracle

star

schema. Theproductsdatawillbejoinedwiththecurrentcostdata(CURRENT_COSTS.csv)fileso

the

product

cost

(UNIT_COST) can

be

stored

in

the

PRODUCTS

dimension

table

(PROD_COST)

To

create

a

new

interface,

open

the

Projectaccordion

within

the

Designernavigator:


16/68




13. ExpandtheHOLBASIC>FirstFolderfolder

14. RightclickonInterfacesandclickNewInterface

Info:An interface isadata flow tomoveand transformdata frommultiplesources intoonetarget.Itcontainsdeclarativeandgraphicalrulesaboutdatajoiningandtransformation.

15. IntheInterfaceeditornametheinterfaceINT_LD_PRODUCTS

Note:

The

staging

area

does

not

need

to

be

changed

since

we

will

use

the

target

as

the

staging

area.


17/68




16. ClicktheMappingtabbelowtheeditor

17. For

this

interface

we

will

load

the

PRODUCTS.csvfile

and

CURRENT_COSTS.csvfrom

model

FlatFileSrctoPRODUCTStablewithinthemodelORACLE_DW_SALES.

To

view

the

models

open

the

Modelsaccordion

18. Open

the

FLATFILESRCfolder

and

FlatFileSRCmodel

and

drag

the

CURRENT_COSTS and

PRODUCTSsourcecsvfilesintothesourcepanelofthedesigner


18/68




19. InserttargetdatastorebydraggingthePRODUCTStablefromORACLE_DW_SALESmodeltotheTargetDatastorepanel(calledbeforethedrop).

20. ClickYesontheconfirmationforautomapping.ODIwillperformsomemappingforyou.

21. Join

the

csv

files

by

click

the

appropriate

column

in

each

table

and

dragging

to

one

another.

The

tablesarejoinedbyPROD_IDandPROD_ID


19/68




Keep the Join Symbol in the Mapping selected and scroll down in the Property InspectortotheStagingAreaandJoinproperties (seeimagebelow).

Info:The Property Inspector show s addi t ional proper t ies of the i tem current ly selected in themapping or the target . This could be a source o r target data store, row, jo in, f i lter , or look up.

Scrol l the proper ty inspector dow n to f ind addi t ional proper t ies.

Set

the

Join

to

a

LeftOuterJoinby

clicking

the

checkbox

connected

to PRO (PRODUCTS)so

no

rowsarelostwhenloadingtothePRODUCTStable.Also changewherethejoinwillbeexecutedbyselectingStagingArea. Sinceweareusingadatatypethatdoesnotacceptjoinswemustdothis

join

in

the

staging

area.

Both

tables

will

be

loaded

to

the

target

and

then

joined.

22. TheIntegrationKnowledgeModule(IKM)forIncrementalUpdateneedsaprimarykeysettobe

able

to

differentiate

between

updated

and

new

records.

To

set

the

primary

key,

click

on

the

target

mapping

forPROD_IDandclicktheKeycheckboxinthepropertyinspectorbelow.


20/68




23. WewillnowmaptheremainingtargetcolumnsPROD_INSERT_DATE,PROD_UPDATE_DATEandPROD_COSTthatwerenotmappedbyauto-mappingearlier.

24. ODIallowsforaspecialfeaturethatwillincrementallyupdatethetargetbasedonthesourceandtargetdata. Weonlywant to insert/updatethePROD_INSERT_DATEcolumnwhere there isanew

record.

And

we

want

to

update

the

PROD_UPDATE_DATEon

every

insert/update.

Click

on

the

PROD_INSERT_DATE column

within

the

target

to

bring

up

the

transformation

mapping.

25. Within the Implementation area enter SYSDATE, you can do this directly in the text box or open

the expression editor. Also make sure to choose Target for the mapping, and uncheck theUpdate checkbox. Un-checking the update turn off updates for this column when this row isupdated based on the primary key.

26. To

set

the

transformation

for

the

PROD_UPDATE_DATEclick

the

PROD_UPDATE_DATE

mapping


21/68




27. WithinthemappingareaenterSYSDATE,youcandothisdirectlyinthetextboxoropentheexpression

editor.

Also

make

sure

to

choose

Targetfor

the

mapping,

Insertand

Updateare

checkedbydefault. Makesurebotharecheckedtoensurethecolumnisupdatedforbothinserts

and

updates.

28. The

PROD_COSTmust

also

be

mapped

to

the

UNIT_COSTof

the

CURRENT_COSTtable. To

mapthis,dragtheUNIT_COSTfieldovertothePROD_COSTtargetcolumn.


22/68




29. Thetransformationshavenowbeensetup. TosettheIKMclickontheFlowtaboftheeditor.

Note: The KMs that w i l l be used have already been imported into the project.

Info:A know ledge module (KM) is a template that represents best practices to perform an action in aninterface, such as loading from/to a certain technology (Load knowledge module or LKM), integratingdata into the target (Integration Knowledge Module or IKM), checking data constraints (CheckKnowledge Module or CKM), and others. Knowledge modules can be customized by the user.

ODI

will

select

default

KMs.

For

the

small

PRODUCT.CSVfile

the

LKMFiletoSQLshould

be

used.

ForthelargerCURRENT_COSTS.CSVfiletheLKMFiletoOracle(SQLLDR)shouldbeused.

TosettheLKMforeachsource,clickthetitleofthesource;thiswillopentheKM properties.

30. Click

the

title

of

the

PRODUCTSsource

and

select

the

LKMFiletoSQL


23/68




31. ClickthetitleoftheCURRENT_COSTSsourceandselecttheLKMFileto Oracle(SQLLDR)

Nocodingisneededbytheuser. ODIwillcreateallneededscripts.

32. IntheOptionsforLKMFiletoOracle(SQLLDR),setDELETE_TEMPORARY_OBJECTSto

false.


24/68




33. ClickthetitleStagingAreatobringuptheIKMproperties

Check

that

the

IKM

Oracle

Incremental

Update

is

being

used.

NOTE:WewillnotbeusingFLOW_CONTROLfor this lab,turn thisoption to false.Flow

controlisusedtoenforcetargettableconstraintsbeforeinsertingrows. SetDELETE_ALL

totruetooverwriteanypriorresults.


25/68




34. The

interface

has

now

been

setup

to

join

the

Products

and

Costs

file

and

load

into

the Products

table. ToexecutetheinterfaceclicktheExecute( )buttoninthetoolbar.

35. WhenpromptedtosaveclickYes

36. The

execution

context

is

Development,

and

the

agent

is

Local.ClickOK

37. ClickOKontheconfirmationwindow

38. Toreviewexecution,gototheOperatornavigator.


26/68




39. ExpandTodayandtheappropriateexecution.Theexecutionmightnothavefinished, thenitwill

showtheicon foranongoingtask.Youcanrefreshtheviewbyclicking torefreshonceor

torefreshautomaticallyevery5seconds.

Doubleclickanytasktoreviewtheexecution.

40. Click

details

or

execution

tab

for

more

information

such

as

inserts

or

updates.


27/68




41. 72 new rows should have been inserted into the Products Table.

42. GobacktotheINT_LD_PRODUCTSinterfaceeditorandselecttheMappingtab.You canviewthe

new

data

in

the

target

PRODUCTS table

from

the

designer

by

right

clicking

the

target

and

clicking

Data...


28/68




43. Review

All

Rows

44. The data has now been loaded successfully for the PRODUCTS table with new costinginformation

from

the

CURRENT_COSTS.CSVtable.


29/68




PART 3:EXTRACTING,LOADING AND

TRANSFORMING SALES DATA TO ORACLEDIMENSION TABLES

1. For

this

interface

we

will

be

de-normalizing

the

SALES

table

by

including

not

only

the

dimension

IDsbutalsothedimensionbasemembersforfasterreporting. ThiswillallowourBIapplicationtoqueryonetablefordetailedreports.

OursourcetableSALESisonMySQL,itwillbejoinedtotheCUSTOMERtablewhichisalsoon

MySQLtolookuptheCustomerName. WewillalsojoinSALEStothe PRODUCTSandtothe

PROMOTION

table

which

have

already

been

loaded

to

our

Oracle target.



30/68




2. Open the ODI Studio. To create a new interface, open the Designer navigator and ProjectaccordionontheHOL_BASIC/FirstFolder:

3. Right

click

and

select

NewInterface

4. Name

the

interface

INT_LD_SALES


31/68




5. ClicktheMappingtabanddraganddropthesourcesSALESandCUSTOMERS fromMYSQLSRC/MYSQL_SALESmodelontothesourcespanel:

6. A

join

will

automatically

be

created

for

the

SALES

and

CUSTOMER

Tables.

Click

the

join

to

view

the

join

properties.

Make

sure

the

join

is

set

to

Source.

This

will

allow

the

integration

to

take

advantage

of

the

exiting

indexesonthesource. DoingthistypeofjoininconventionalETLtoolswouldrequirecustomSQL

expressions.


32/68




7. Drag the PRODUCTS and PROMOTIONS tables from ORACLE DW TRG/ORACLE_DW_SALES intothesourcepanel.

8. Drag

the

table

SALESfrom

the

ORACLE_DW_SALESmodel

to

the

TargetDatastore area.

9. ClickYestoallowforautomapping


33/68




10. JoinPRODUCTbydraggingfromPROD_IDtotheSALEStablePROD_IDcolumn,makesuretheExecuteOnlocationinthepropertyinspectorischangedtoStagingArea.

1. JointhePROMOTIONSdatabyjoiningPROMO_IDtothePROMO_IDoftheSALEStable;makesure

the

ExecuteOnlocation

in

the

property

inspector

is

changed

to

StagingArea.


34/68




2. MapPRODUCTtoPROD_NAMEfromthejoinedPRODUCTtable.

3. MapthetargetfieldCUSTOMERbyconcatenatingthefirstandlastnameofthecustomerusingamapping

on

the

Staging Area of

CUSTOMERS.CUST_FIRST_NAME || ' ' ||CUSTOMERS.CUST_LAST_NAME


35/68




4. MapthetargetfieldPROMOTIONtoPROMO_NAMEfromthejoinedPROMOTIONS table.

5. Thedesignandmappingsarenowcomplete. ClickontheFlowtab.

The

power

of

ELT

and

ODI

is

apparent

by

how

easy

it

is

to

set

certain

transformation

on

the

source,

target

or

staging

area.

ODI

will

generate

the

flow

automatically,

we

have

only

defined

whatwewanttodo,nothowitwillbedone.

6. The

KMs

will

be

selected

by

default

for

the

source

and

target.

Source

LKM:

LKM

SQL

to

Oracle


36/68




Target

IKM:

IKM

Oracle

Incremental

Update

7. Select

the

title

of

the

Staging

Area.

Set

the

FLOW_CONTROLOption

to

false.Set DELETE_ALLto

truetooverwriteanypriorresults.

8. The

interface

has

now

been

setup

to

extract,

de-normalize

and

load

the

SALES

table

from

MySQL

to Oracle, using lookup tables on MySQL and on Oracle. To execute the interface click the

Execute( )button

in

the

toolbar.


10. TheexecutioncontextisDevelopment,andtheagentisLocal.

ClickOK


37/68





12. To

review

execution,

go

to

the

Operatornavigator.

13. Double click any step for details; click the details, or execution tab for more information. The

execution

of

the

interface

might

take

up

to

a

few

minutes

to

complete,

during

which

time

thecompletedstepswillbeshown.TheRecordStatisticsshoulddisplay183,941inserts.


38/68




OPTIONALPART4:EXTRACTING,LOADING

ANDTRANSFORMINGPROMOTION DATATOORACLEDIMENSIONTABLES

1. This

interface

will

load

the

Promotion

data

from

an

XML

file

to

the

Oracle

star

schemaPROMOTIONStable.


2. Start

the

ODIStudioif

not

already

started

and

open

the

HOL_BASIC

login

3. For

this

interface

the

source

PROMOTIONS.xmlfile

will

be

used.

ODI

has

a

built

in JDBC

driver

for

XML,

that

will

be

used

to

extract

the

data.


39/68




Info:ODI

works

withXML databyusingaspecializedJDBCdriver thatwillmapan XML schema into

multiple tables. Eachnon-leafelement is representedby a table; hierarchy is reflected throughPK-FKrelationships.

Using

XML

sources

in

interfaces

is

the

same

as

using

a

relational

DB

source.

4. To

create

a

new

interface,

open

the Designer navigator and Project accordion on the

HOL_BASIC/FirstFolder:


40/68




5. RightclickandselectNewInterface

6. NametheinterfaceINT_LD_PROMOTIONS


41/68




7. Click

on the

Mappingtab

OpentotheModelsaccordionontheleftnavigatoranddragPROMODATAintothesourcepanelof

diagram

tab.

8. Drag the PROMOTIONS data store from the ORACLE_DW_SALES model to the Target

Datastorepanel

9. Click

YESto

auto

mapping

confirmation


42/68




10. Again

for

the

logic

of

the

incremental

update

a

key

must

be

set

on

the

target

table.

To

set

the

PROMO_IDasthekeyselectthemappingandchecktheKeycheckboxbelow.

11. ThesourcedatamightcontainrecordswithaNULLbegindata.Weneedtosetafilter to ignore

these

records.

From

the

PROMODATAsource

field

drag

the

PROMO_BEGIN_DATEfield

onto

an

empty

area

of

the

diagram

background.

12. Select

the

filter

and

edit

the

Implementation

property

of

the

filter.

Complete

the

expression

to

read:

PROMODATA.PROMO_BEGIN_DATEisnotnull

13. The

date

columns

in

the

XML

file

are

setup

as

varchar

fields.

We

can

implicitly

convert

these

within

the staging area by select the Staging Area for the mapping of the two date fieldsPROMO_BEGIN_DATEand

PROMO_END_DATE.


43/68




14. ClicktheFlowtabtoreviewtheFlowandKMsettings

Note:BydefaulttheLKMSQLtoOracleKMshouldhavebeenselectedfortheXMLSource.Ifnot,selectthisLKM.EventhoughthisisanXMLfile,theJDBCdriverallowsODItointeractwiththefileas

though

it

is

a

relational

table.


44/68




15. Select

the

Staging

Area

to

review

the

IKM

setting.

By

default

the

IKMOracle IncrementalUpdate

hasbeenset.

16. Note:MakesuretosettheIKMoptionforFLOW_CONTROLtofalse.SetDELETE_ALLto true

tooverwrite

any

prior

results.

17. TheinterfacehasnowbeensetuptoloadthePromotionsXMLfiletothePROMOTIONS table. To

executetheinterfaceclicktheExecute( )buttoninthetoolbar.


19. TheexecutioncontextisDevelopment,andtheagentisLocal.

Click

OK


45/68





21. To

review

execution

go

to

theOperatornavigator.

22. Reviewtheexecutionofthestepsformore informationdoubleclickastep,andview thedetailsorexecution

tab.

The

total

number

of

inserts

for

this

session

is

503

inserts

to

the

target.

23. You have now successfully loaded the Promotions XML file to the PROMOTIONS Table on the

Oracle

DW

Sale

star

schema.


46/68




OPTIONALPART5:CREATINGANEWODIPACKAGETOEXECUTETHEFULL DIMENSIONANDSALESTABLEBUILD.

1. NowthattheinterfaceshavebeencreatedwecancreateapackagewithinODIthatwillexecuteallofthe

interfaces

in

order.

2. To

create

a

new

package,

open

the

Designernavigator

and

Projectaccordion

on

the HOL_BASIC/FirstFolder:

3. Right

click

Packages

and

select

NewPackage.

Info:Apackageisataskflowtoorchestrateexecutionofmultipleinterfacesanddefineadditionallogic,suchasconditionalexecutionandactionssuchassendingemails,callingwebservices,uploads/downloads,filemanipulation,

event

handling,

and

others.


47/68




4. NamethepackagePKG_LD_ALL

5. ClicktheDiagramtab

Drag

and

Drop

the

interfaces

from

the

left

onto

the

diagram

panel,

starting

with

the

interfaceINT_LD_PRODUCTS

Notice

the

green

arrow

on

this

interface

which

means

it

is

the

first

step.

6. DragtheinterfacesINT_LD_PROMOTIONSandINT_LD_SALESontothepanel


48/68




7. ClicktheOKarrowtoolbarbuttontoselecttheorderofprecedence.

8. Drag

and

drop

from

the INT_LD_PRODUCTSto the INT_LD_PROMOTIONStosetthe link.Then

draganddropfromINT_LD_PROMOTIONStoINT_LD_SALES.

9. Thepackageisnowsetupandcanbeexecuted.Toexecutethe interfaceclickthe Execute( )button

in

the

toolbar.

10. When prompted to save click Yes

11. The

execution

context

is

Development,

and

the

agent

is

Local.

Click

OK


49/68





13. Toreviewexecution,gototheOperatornavigatorandopenthelatestsessionexecution.


50/68




SUMMARY

You havenow successfullycompleted the Handson Lab, and have successfully loaded the Product andPromotions

dimension

table

as

well

as

de-normalizing

and

loading

the

sales

fact

table.

Once

the

interfaces

were

created

to

load

and

transform

the

data,

a

package

was

created

to

execute

all

the

interfaces

in

the

correctorder.


51/68




HANDS ON LAB:COMBINING ORACLE DATAINTEGRATOR AND ORACLE GOLDENGATE

Supported Version: Oracle Data Integrator 11.1.1.6, Oracle GoldenGate 11gR2

Objectives:

Data integration is more than simply about ETL (extract, transform, load); its also about unifyinginformation for improved business agility. This hands-on labfor developers, DBAs, and architectshighlights best practices using Oracle Data Integrator (ODI) and Oracle GoldenGate (OGG) together toperform real-time ETL. In the lab, participants will use ODI s Journalizing Knowledge Module for OGG to

generate OGG scripts based on ODI models and interfaces.

Prerequisite:

Basic knowledge of Oracle Data Integrator and GoldenGate, Installed Oracle Data Integrator, OracleGoldenGate and Oracle Database 11g instance.


52/68




OVERVIEW

In this lab we are going to add OGG CDC to an existing ODI project. The project is mostly identical to theODI Extended Hands-on Lab that covers ODI CDC mechanisms.

For simplicity the entire setup is on one machine using a single database. Real-live use cases would mostlikely use source and target on different servers, but the fundamental setup of OGG and ODI are verysimilar.

The demo machine has two separate installations of OGG, one in C:\gg_src and one in C:\gg_stg. Thesetwo locations simulate OGG installations on two separate machines.


53/68




PART 0:PREPARATION STEPS

In these steps you will clean and setup the environment for this exercise

1. Reset the environment if it has been previously used.

Warning: I f you have custom ODI projects in the reposi tory on this machine, they wi l l be deleted

Open a terminal and execute reset_odi. Press Enter.


54/68




PART 1:ENABLE STAGING MODEL FOR CDC

In this exercise you will review the existing models and projects in ODI

1. Open the Oracle Data Integrator Studio:

Open a terminal and execute odistudio. Press Enter

2. Choose the Designer tab and click on Connect To Repository

3. In the Login dialog, choose HOL GoldenGate from the Login Name list.

4. Go to the Designer navigator and click on the Models title to expand the models pane.


55/68




5. Review the models present in the resposi tory: Expand each one of the 4 modelsOracle_Source, Oracle_Staging, Oracle_Target, and Flat File Sources. The Oracle_Sourcemodel contains the tables in the original source location. Oracle_Staging contains copies oftables from the source that need to be replicated by Oracle GoldenGate in real-time. TheOracle_Taget model contains transformed tables in the final target location. The Flat FileSources model contains flat file datastores that will participate in the transformation to the target

table.


56/68




6. Enable the staging m odel for ODI CDC: The JKM for Oracle GoldenGate performs a 2-stageCDC; OGG detects changes in the source and replicates them to the staging tables; ODI isinformed of changes arriving in the staging table through its standard CDC framework; therefore

the staging tables are enabled for CDC and generate OGG scripts to perform replication fromsource to staging.

Right-click on the model Oracle_Staging, go into the submenu Changed Data Capture, andselect Add to CDC. Click Yes when asked Are you sure you want to add all the modelstables to CDC?

7. Double-click on the model Oracle_Staging to open the model editor


57/68




8. Go to the Journalizing tab.

9. In Journalizing Mode pick Consistent Set, click Ok in the Alert dialog, and in Journalizing KMpick JKM Oracle to Oracle Consistent (OGG).

10. Enter the following options (highlighted in red) to the JKM. Leave options starting with unchanged.


58/68




Values:

LOCAL_TEMP_DIR /tmp

SRC_LSCHEMA ORCL_SOURCE

SRC_DB_USER system

SRC_DB_PASSWORD oracle

SRC_OGG_PATH /app/goldengate/source

STG_MANAGER_PORT 7810

STG_OGG_PATH /app/goldengate/target

COMPATIBLE 10

11. Click Save ( ) to save the model; close the model editor.

12. Right-click on model Oracle_Staging, go into the submenu Changed Data Capture Subscriber, and select Subscribe.

13. Enter ODI as the name of a new subscriber and click the blue Arrow Down button to add tothe list.


59/68




14. Click OK to close the dialog. Click Ok on the Execution and then Information dialogs thatappear.

15. Right-click on Oracle_Staging, go into the submenu Changed Data Capture, and select Start

Journal. Click Ok on the Execution and then Information dialogs that appear.


60/68




PART 2:CONFIGURE ORACLE GOLDENGATE

USING GENERATED FILES

In this exercise you will review the existing models and projects in ODI

1. Start a Terminal window by opening the menu ApplicationsAccessoriesTerminal.

2. Enter cd /tmp/ODIS_to_ODIT1

PressEnterand then enter ls

Press Enterinto the terminal prompt. You can see two directories containing the configurationfiles for the source and staging OGG installations, as well as a Readme.txt file explaining theconfiguration of OGG using these files.

3. Enter vi Readme.txt

Press Enterto review the 6 configuration steps.

4. Enter copy_gg_files

Press Enterto perform the 6 steps described in Readme.txt


61/68




PART 3:INITIALIZE LOAD TO STAGING ANDTARGET TABLES

In this exercise you will initalize both staging and target areas with the pre-existing source data.

1. Review the data in sourc e, staging , and target table:

2. Open a Terminal and enter startGGDemo

Press Enter.

This client is a custom Java program to display changes in the 3 tables as they occur. At this point only

the source table has data. Keep this client running for later.

3. Open the ODI Studio and go to Designer > Projects.


62/68




4. Review folders: The relevant folders for this GG_HOL project are:

1. Initial Load Staging: Performs a 1-to-1 copy from the source tables to the staging tables.

2. Initial Load Target: Does an bulk load & transformation from staging to target tables3. Synchronize Target: Performs a CDC-triggered load & transformation from staging to target.

5. Expand the folder 1. Initial Load Staging and its Interfaces subfolder:

6. Right click on interface Initial Load Staging and select Execute. Click Ok on the Execution andthen Information dialogs that appear.


63/68




7. Open the ETL Demo Client to review the loading of the staging data:

8. Expand the folder 2. Initial Load Target and its Interfaces subfolder:

9. Right click on interface Full Load Customer to Target and select Execute. Click Ok on theExecutionand then Information dialogs that appear.


64/68




10. Open the ETL Demo Client to review the loading of the target data:


65/68




PART 4:INITIALIZE CDCPROCESS AND

PERFORM CHANGE OPERATIONS

1. Open the ODI Studioand go to DesignerProjects.

2. Expand the folder 3. Synchronize Target and its Packages folder.

3. Double-Click on the Sync Target package to open it. Click on the Diagram tab.

The interface consists of a perpetual loop that is triggered by a new journal log entry andprocesses new changes with the CDC Load Customer interface.

4. Click the Execute button ( ) in the toolbar to start the interface.


66/68




5. Open the Operator from the toolbar to review the process of the Sync Target interface.

6. Open the Date

Today

Sync Target tree branch in the operator. The interface is currentlyrunning and waiting for a new journal entry at OdiWaitForLogData 1.

7. Open the ETL Demo Client and double-click on a FIRST_NAME of an entry in the Source table.The entry is editable.

8. Edit the name to something else and press Enter.Watch the Staging and Target tables. After afew seconds the FIRST_NAME in the Staging table flashes yellow and after that the

concatenated CUST_NAME in the Target table.


67/68





68/68


SUMMARY

At the end of these exercises the participants understand how to:

Configure the ODI JKM for Oracle-to-Oracle with OGG

Use the generated OGG configuration to set up a CDC process

Operate and diagnose the CDC process with OGG and ODI

Questions & Clarifications:

If you have any comments or need additional information, please communicate through the Oracle Data

Integrator forum at:http://forums.oracle.com/forums/forum.jspa?forumID=374
http://forums.oracle.com/forums/forum.jspahttp://forums.oracle.com/forums/forum.jspahttp://forums.oracle.com/forums/forum.jspa

53642 Oracle Goldenga

Documents

Transcript of 53642 Oracle Goldenga