Configuring Oracle GoldenGate OGG 11gR2 local integrated capture

and using OGG for mapping and transformations

In the article you will have a look at an OGG configuration example for local

integrated capture and discover how to deploy some of the powerful OGG

features related to data mapping and transformation. Starting from OGG 11gR2

Oracle implemented an integrated capture, in addition to the already existing

transaction log capture now a day called classic capture, only from Oracle

RDBMS data sources by integrating the OGG extract with the database logmining

server. OGG mapping and transformation is a feature that works with both

classic and integrated capture enabling complex transactions to be replicated

across heterogeneous environments. I will cover each topic separately in the

following sub-sections.

Example of local integrated capture using OGG 11gR2

Example of mapping and transformation using OGG 11gR2

The article is based on OGG 11gR2 environment configuration previously

described here.

Example of local integrated capture using OGG

The integrated capture mode introduced in OGG 11gR2 benefits from the

integration with the existing Oracle Streams API. In integrated capture mode,

the OGG extracts interact with the Oracle logmining server to capture data

changes as a logical change records (LCR). The database where the Oracle

logmining server is running must be 11.2.0.3 with patch as per MOS Doc ID

1411356.1 for integrated capture mode to handle almost all objects and data

types otherwise there are restrictions. My source test database RACD and my

target database RACDB are both 11.2.0.3 and patched as per the requirements.

The integrated capture can be as follows depending on where the logminig

server is running.

Local integrated capture – logminig server is running on the database

where the transactions are executed and captured

Downstream integrated capture - logminig server is running on a

different database from the database where the transactions are

executed. The database running the transactions is configured to ship

the redo and archivelogs to the mining database.

Useful Oracle sources are Oracle Installation and Setup Guide from the

official Oracle OGG documentation accessible here and MOS notes (Best

Practice - Oracle GoldenGate 11gr2 integrated extract and Oracle Data Guard -

Switchover/Fail-over Operations Version 1.1 Document ID: 1436913.1 and

Oracle GoldenGate Best Practices: Configuring Downstream Integrated Extract

Document ID: 1485620.1).

Integrated capture mode in OGG 11gR2 enables you to process Oracle database

objects and data types that cannot be processed in the classing capture mode.

For a detailed list of Oracle database objects and data types that can be

processed in integrated/classic capture mode look at Oracle Installation and

Setup Guide.

In this section I will show an example of configuring a local integrated

capture. In a separate blog post I will show an example of a downstream

integrated capture.

The following tables will be created on both source and target databases for

illustration of OGG mapping and transformation while replicating data from

the source database to the target database.

The source objects are in test3 schema in RACD database. The target objects

are in the test3_t schema in the RACDB database. I will configure extract

extint on the source and replicat repint on the target. The tables listed

below will be created on both source and target.

create table customer ( NID varchar2(10) primary key ,

Name varchar2(400),

Address_street varchar2(400),

Adress_city varchar2(50),

Tel varchar2(4),

state_code char(2),

zip char(9)

);

create table customer_detail (

customer_id number(10) primary key,

First_name varchar2(40),

Last_name varchar2(40),

NID varchar2(10),

Tel_country varchar2(3),

Tel_area varchar2(3),

Tel_number varchar2(8),

Country varchar2(30),

Address_street varchar2(400),

address_city varchar2(50),

Tel_c_code varchar2(4),

State_name varchar2(50),

zip1 varchar2(5),

zip2 varchar2(5) );

create table customer_detail1 (

customer_id number(10) primary key,

First_name varchar2(40),

Last_name varchar2(40),

NID varchar2(10),

Tel_country varchar2(3),

Tel_area varchar2(3),

Tel_number varchar2(8),

Country varchar2(30),

Address_street varchar2(400),

address_city varchar2(50),

Tel_c_code varchar2(4),

State_name varchar2(50),

zip1 varchar2(5),

zip2 varchar2(5) );

This section will provide an overview of the steps to configure a local

integrated extract. The steps are executed in order as follows:

1. Grant permissions to the user (for local integrated capture one user

will suffice although you can use two dedicated users).

2. While logged in with dblogin add trandata for the source tables.

3. Register extract for integrated capture while logged in with dblogin

4. Create an extract for integrated capture

5. Create a source and target definition file using DEFGEN.

6. Create a parameter file for the extract using local integrated capture

7. Create a trail for the extract

8. Create a replicat

9. Start OGG extract and replicat

10. Troubleshooting tips/tricks for extract

1. Grant permissions to the extract user

I am using the same OGG_EXTRACT user for extraction as defined here for OGG

to interact to both source Oracle database and logmining server running on

the source database. Grant privileges that required for the user to interact

with the Oracle logmining server by executing the following SQL statement

while logged in to the source database using as sysdba.

SQL> exec DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE (grantee=>'ogg_extract',

privilege_type=>'capture',grant_select_privileges=>true, do_grants=>TRUE);

SQL>

PL/SQL procedure successfully completed.

SQL>

2. Add trandata on the source database tables that will be

replicated.

GGSCI (raclinux1.gj.com) 6> dblogin userid ogg_extract, password ogg_extract

Successfully logged into database.

GGSCI (raclinux1.gj.com) 8> add trandata test3.state

Logging of supplemental redo data enabled for table TEST3.STATE.

GGSCI (raclinux1.gj.com) 9> add trandata test3.customer

Logging of supplemental redo data enabled for table TEST3.CUSTOMER.

GGSCI (raclinux1.gj.com) 10> add trandata test3.customer_detail

Logging of supplemental redo data enabled for table TEST3.CUSTOMER_DETAIL.

GGSCI (raclinux1.gj.com) 11>

GGSCI (raclinux1.gj.com) 28> add schematrandata test3

2012-10-25 13:29:20 INFO OGG-01788 SCHEMATRANDATA has been added on schema

test3.

GGSCI (raclinux1.gj.com) 29>

3. Register the extract for integrated capture

GGSCI (raclinux1.gj.com) 29> dblogin userid ogg_extract, password ogg_extract

Successfully logged into database.

GGSCI (raclinux1.gj.com) 30> register extract extint database

2012-10-25 13:31:09 WARNING OGG-02064 Oracle compatibility version 11.2.0.0.0 has

limited datatype support for integrated capture. Version 11.2.0.3 required for full

support.

2012-10-25 13:32:26 INFO OGG-02003 Extract EXTINT successfully registered with

database at SCN 4629827.

GGSCI (raclinux1.gj.com) 31>

4. Create a source and a target definition files using DEFGEN

utility

Create a parameter file for generating the source definition file on

the source system.

GGSCI (raclinux1.gj.com) 4> view params defgensource

defsfile ./dirdef/source_int.def, purge

userid ddl_ogg, password ddl_ogg

table test3.state;

table test3.customer;

table test3.customer_detail;

GGSCI (raclinux1.gj.com) 5>

Run defgen utility from OGG installation directory on the source system

./defgen paramfile ./dirprm/defgensource.prm

Create parameter file for generating the target definition file on the

target system.

GGSCI (raclinux1.gj.com) 5> view params defgentarget

defsfile ./dirdef/target_int.def, purge

userid ddl_ogg, password ddl_ogg

table test3_t.state;

table test3_t.customer;

table test3_t.customer_detail;

table test3_t.customer_detail1;

GGSCI (raclinux1.gj.com) 6>

Run defgen utility from the OGG installation directory on the target

system.

./defgen paramfile ./dirprm/defgentarget.prm

Copy the source definition file to the target. Copy the target

definition file to the source.

5. Create the extract

From the Reference Guide you will see that in case or RAC database

with local integrated capture you should not specify threads. In

practice you will not be allowed to do so.

GGSCI (raclinux1.gj.com) 33> add extract extint, integrated tranlog, begin now,

threads 2

ERROR: THREADS is not allowed for INTEGRATED extract.

GGSCI (raclinux1.gj.com) 34> add extract extint, integrated tranlog, begin now,

threads 1

ERROR: THREADS is not allowed for INTEGRATED extract.

GGSCI (raclinux1.gj.com) 35> add extract extint, integrated tranlog, begin now

EXTRACT added.

GGSCI (raclinux1.gj.com) 36>

6. Create the parameter file for extract and replicat

The extract and replicat parameter files are in the appendix. Here I

will list the parameter required to set local integrated capture

extract.

TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3)

userid ogg_extract, password ogg_extrac

7. Create a trail

Create a remote trail from GGSCI.

GGSCI (raclinux1.gj.com) 36> add rmttrail ./dirdat/2x , extract extint, megabytes

20

RMTTRAIL added.

GGSCI (raclinux1.gj.com) 37>

8. Create a replicat

Look at the appendix for the replicat parameter file. Create a

replicat from GGSCI.

GGSCI (raclinux1.gj.com) 8> add replicat repint, exttrail ./dirdat/2x

REPLICAT added.

GGSCI (raclinux1.gj.com) 9>

9. Start OGG extract and replicat

Start extract and replicat as usual using the OGG start extract and

start replicat commands.

GGSCI (raclinux1.gj.com) 6> start extract extint

Sending START request to MANAGER ...

EXTRACT EXTINT starting

GGSCI (raclinux1.gj.com) 7>

GGSCI (raclinux1.gj.com) 7> start replicat repint

Sending START request to MANAGER ...

REPLICAT REPINT starting

GGSCI (raclinux1.gj.com) 8>

Note the difference between integrated extract extint compared to a

classic extract extt1.

GGSCI (raclinux1.gj.com) 11> info extract extint

EXTRACT EXTINT Last Started 2012-10-25 15:37 Status RUNNING

Checkpoint Lag 00:00:00 (updated 01:45:42 ago)

Log Read Checkpoint Oracle Integrated Redo Logs

2012-10-25 13:55:43

SCN 0.0 (0)

GGSCI (raclinux1.gj.com) 12>

GGSCI (raclinux1.gj.com) 12> info extract extt1

EXTRACT EXTT1 Last Started 2012-10-25 15:37 Status RUNNING

Checkpoint Lag 00:00:00 (updated 110:37:09 ago)

Log Read Checkpoint Oracle Redo Logs

2012-10-21 01:05:14 Thread 1, Seqno 276, RBA 9411088

SCN 0.4572497 (4572497)

Log Read Checkpoint Oracle Redo Logs

2012-10-13 15:48:49 Thread 2, Seqno 0, RBA 0

SCN 0.0 (0)

GGSCI (raclinux1.gj.com) 13>

10. Troubleshooting tips and tricks

Initially, without proper configuration I faced the errors specified in

the appendix. The following helped to work around the problems

Commented --tranlogoptions asmuser sys@ASM, asmpassword sys1 in order

to resolve

OGG-02000 Ignoring option ASMUSER because it is incompatible with an

integrated capture configuration and

OGG-02000 Ignoring option ASMPASSWORD because it is incompatible with

an integrated capture configuration.

Configured the database with the following parameters and added OGG

parameter TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700,

parallelism 3). Note that the source database is 11.2.0.3.

alter system set memory_max_target=3g scope=spfile sid='*';

alter system set memory_target=3g scope=spfile sid='*';

alter system set streams_pool_size=1g scope=spfile sid='*';

alter system set compatible='11.2.0.3.0' scope=spfile sid='*';

This helped avoiding the following errors

2012-10-25 16:29:57 WARNING OGG-00869 ORA-00600: internal error code,

arguments: [KGHFRE3], [0x08AB26040],

[], [], [], [], [], [], [], [], [], []

ORA-26914: Unable to communicate with GoldenGate capture process

"OGG$CAP_EXTINT" from outbound server "OGG$EXTINT".

2012-10-25 16:29:04 WARNING OGG-02045 Database does not have

streams_pool_size initialization parameter configured.

2012-10-25 16:29:02 WARNING OGG-02064 Oracle compatibility version

11.2.0.0.0 has limited datatype support for integrated capture.

Version 11.2.0.3 required for full support.

2012-10-25 17:30:27 WARNING OGG-02051 Not enough database memory to

service Extract in integrated capture

mode.

2012-10-25 17:32:52 WARNING OGG-01223 TCP/IP error 111 (Connection

refused).

Example of mapping and transformation using OGG 11gR2

OGG mapping and transformation features enable replication across dissimilar

heterogeneous table structures. OGG SQLEXEC allows execution of SQL on both

extract and replicat sites. In this case SQLEXEC will be used for table

lookups. OGG allows mapping to be done on the source system using the extract

TABLE parameter or on the target system using the replicat MAP parameter. For

all mappings on the target using COLMAP and transformations use a source

definition file specified with the SOURCEDEFS parameter. For all mappings on

the source using COLMAP and transformations use a target definition file

specified with the TARGETDEFS parameter. Source and target tables are

identical if:

Tables have same column names

Columns of the names are of the same type

Column order is the same in source and target columns

If source and target tables are not identical use the SOURCEDEFS/TARGETDEFS

parameters in extract/replicat parameter file and specify data definition

files generated with defgen see the 1.4.

I will illustrate it using the following tables.

SQL> desc customer

Name Null? Type

----------------------------------------- -------- ----------------------------

NID NOT NULL VARCHAR2(10)

NAME VARCHAR2(400)

ADDRESS_STREET VARCHAR2(400)

ADRESS_CITY VARCHAR2(50)

TEL VARCHAR2(40)

STATE_CODE VARCHAR2(3)

ZIP VARCHAR2(12)

SQL>

SQL> desc customer_detail

Name Null? Type

----------------------------------------- -------- ----------------------------

CUSTOMER_ID NOT NULL NUMBER(10)

FIRST_NAME VARCHAR2(40)

LAST_NAME VARCHAR2(40)

NID VARCHAR2(10)

TEL_COUNTRY VARCHAR2(3)

TEL_AREA VARCHAR2(3)

TEL_NUMBER VARCHAR2(8)

COUNTRY VARCHAR2(30)

ADDRESS_STREET VARCHAR2(400)

ADDRESS_CITY VARCHAR2(50)

TEL_C_CODE VARCHAR2(4)

STATE_NAME VARCHAR2(50)

ZIP1 VARCHAR2(5)

ZIP2 VARCHAR2(5)

SQL>

SQL> desc state

Name Null? Type

----------------------------------------- -------- ----------------------------

STATE_ID NOT NULL CHAR(2)

STATE_DESC VARCHAR2(50)

STATE_CAPITAL VARCHAR2(50)

SQL>

In the example state table is replicated as it is. Customer table from the

source is mapped to the customer_detail and customer_detail1 on the target.

The mappings and the transformations from customer to customer_detail[1] are

as follows:

Source table column Target Table column Transformation

CUSTOMER_ID NID Make it a number

FIRST_NAME NAME Extracts the first name

LAST_NAME NAME Extracts the surname

NID NID None. NID=NID

TEL_COUNTRY TEL +1 if state_code is in

US

TEL_AREA TEL Extracts first 3

characters from TEL

TEL_NUMBER TEL Extracts from 4th

position to the 10th

position from TEL

COUNTRY STATE_CODE USA if state_code is in

USA

ADDRESS_STREET ADDRESS_STREET NONE

ADDRESS_CITY ADRESS_CITY NONE

TEL_C_CODE STATE_CODE +1 if state_code is in

US

STATE_NAME STATE_CODE Gets STATE_DESK using

SQLEXEC lookup using

state table

ZIP1 ZIP Extracts first 5

characters from ZIP

ZIP2 ZIP Extracts from 6th

position to the 10th

position from ZIP

The logical mapping above is implemented at OGG level using the underlined

parameters from the extract extint and replicat repint parameter files shown

bellow with abbreviations for the sake of clarity. Note that there are two

mappings:

customer -> customer_detail : mapping is done in the replicat

customer -> customer_detail1 : mapping is done in the extract and

replicat. This is an illustration how to do mapping and transformation

on the extract in case the target is Tandem/NSK /HP SQL/MX based

system.

The complete extract and replicat parameter files are in the appendix.

GGSCI (raclinux1.gj.com) 1> view params extint

extract extint

SETENV (ORACLE_SID = "RACD1")

--tranlogoptions asmuser sys@ASM, asmpassword sys1

TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3)

targetdefs ./dirdef/target_int.def

userid ogg_extract, password ogg_extract

rmthost raclinux1, mgrport 7809

rmttrail ./dirdat/2x

table test3.state;

table test3.customer;

table test3.customer_detail;

table test3.customer target test3_t.customer_detail1,

sqlexec( id statelookup, query " select state_desc from test3.state where state_id = :state_code"

, PARAMS(state_code=state_code)),

colmap(

usedefaults,

customer_id = @NUMSTR(NID),

First_name = @STREXT(Name,1,@STRFIND(Name," ")),

Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)),

NID = NID,

Tel_country = @IF (@VALONEOF(state_code,"AL","AK", ... ), "+1", "+44"),

Tel_area = @STREXT (Tel, 1, 3),

Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK", ... ), "USA", "UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", " ...), "+1", "+44"),

State_name = @GETVAL(statelookup.state_desc),

zip1 = @strext(zip,1,5),

zip2 = @strext(zip,6,10));;

GGSCI (raclinux1.gj.com) 2> view params repint

replicat repint

--reperror(default,ignore)

SETENV (ORACLE_SID = "RACDB1")

userid ogg_replicat, password ogg_replicat

sourcedefs ./dirdef/source_int.def

discardfile ./dirrpt/repint.dsc, purge

map test3.state, target test3_3.state;

map test3.customer, target test3.customer;

map test3.customer, target test3_t.customer_detail1,

sqlexec( id statelookup1, query " select state_desc from test3_t.state where state_id =

:state_code" , PARAMS(state_code=state_code)),

colmap(

usedefaults,

ADDRESS_CITY = ADRESS_CITY,

customer_id = @NUMSTR(NID),

First_name = @STREXT(Name,1,@STRFIND(Name," ")),

Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)),

NID = NID,

Tel_country = @IF (@VALONEOF(state_code,"AL","AK", ... ), "+1", "+44"),

Tel_area = @STREXT (Tel, 1, 3),

Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK", ...), "USA", "UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", ..."), "+1", "+44"),

State_name = @GETVAL(statelookup1.state_desc),

zip1 = @strext(zip,1,5),

zip2 = @strext(zip,6,10));

map test3.customer, target test3_t.customer_detail,

sqlexec( id statelookup, query " select state_desc from test3_t.state where state_id =

:state_code" , PARAMS(state_code=state_code)),

colmap(

usedefaults,

ADDRESS_CITY = ADRESS_CITY,

customer_id = @NUMSTR(NID),

First_name = @STREXT(Name,1,@STRFIND(Name," ")),

Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)),

NID = NID,

Tel_country = @IF (@VALONEOF(state_code,"AL","AK", ...), "+1", "+44"),

Tel_area = @STREXT (Tel, 1, 3),

Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK", ...), "USA", "UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", ...), "+1", "+44"),

State_name = @GETVAL(statelookup.state_desc),

zip1 = @strext(zip,1,5),

zip2 = @strext(zip,6,10));

Let’s test it.

Insert into test3.customer and observe the transformations on the target.

insert into test3.customer values('222222222','John Smith','93 Some

street','Dallas','5555555555','TX','75209-5555');

insert into test3.customer values('333333333','Alejandro García', '46 Some

street','Wayne','6666666666','NJ','07470-5555');

insert into test3.customer values('444444444','Frank Meyer','56 Some street','Los

Angeles','7777777777','CA','90018-5555');

commit;

on the target we have the following result:

SQL> select * from customer_detail;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL

TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1

ZIP2

----------- -------------------- -------------------- ---------- --- --

- -------- ------ ----------------- ---------- ---- ---------- ----- --

---

222222222 John Smith 222222222 +1

555 5555555 USA 93 Some street Dallas +1 Texas 75209

-5555

333333333 Alejandro Garc??a 333333333 +1

666 6666666 USA 46 Some street Wayne +1 New Jersey 07470

-5555

444444444 Frank Meyer 444444444 +1

777 7777777 USA 56 Some street Los Angele +1 California 90018

-5555

s

SQL> select * from customer_detail1;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL

TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1

ZIP2

----------- -------------------- -------------------- ---------- --- --

- -------- ------ ----------------- ---------- ---- ---------- ----- --

---

222222222 John Smith 222222222 +1

555 5555555 USA 93 Some street Dallas +1 Texas 75209

-5555

333333333 Alejandro Garc??a 333333333 +1

666 6666666 USA 46 Some street Wayne +1 New Jersey 07470

-5555

444444444 Frank Meyer 444444444 +1

777 7777777 USA 56 Some street Los Angele +1 California 90018

-5555

s

SQL>

Perform an update on the source

update test3.customer set TEL='8888888888' where NID='222222222';

Verify the result on the target

SQL> select * from customer_detail;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR

ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2

----------- -------------------- -------------------- ---------- --- --- -------- ------

----------------- ---------- ---- ---------- ----- -----

222222222 John Smith 222222222 +1 888 8888888 USA

93 Some street Dallas +1 Texas 75209 -5555

333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA

46 Some street Wayne +1 New Jersey 07470 -5555

444444444 Frank Meyer 444444444 +1 777 7777777 USA

56 Some street Los Angele +1 California 90018 -5555

s

SQL> select * from customer_detail1;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR

ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2

----------- -------------------- -------------------- ---------- --- --- -------- ------

----------------- ---------- ---- ---------- ----- -----

222222222 John Smith 222222222 +1 888 8888888 USA

93 Some street Dallas +1 Texas 75209 -5555

333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA

46 Some street Wayne +1 New Jersey 07470 -5555

444444444 Frank Meyer 444444444 +1 777 7777777 USA

56 Some street Los Angele +1 California 90018 -5555

s

SQL>

Delete a record from the source

delete from test3.customer where NID='222222222';

Observe the result on the target

SQL> select * from customer_detail;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR

ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2

----------- -------------------- -------------------- ---------- --- --- -------- ------

----------------- ---------- ---- ---------- ----- -----

333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA

46 Some street Wayne +1 New Jersey 07470 -5555

444444444 Frank Meyer 444444444 +1 777 7777777 USA

56 Some street Los Angele +1 California 90018 -5555

s

SQL> select * from customer_detail;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR

ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2

----------- -------------------- -------------------- ---------- --- --- -------- ------

----------------- ---------- ---- ---------- ----- -----

333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA

46 Some street Wayne +1 New Jersey 07470 -5555

444444444 Frank Meyer 444444444 +1 777 7777777 USA

56 Some street Los Angele +1 California 90018 -5555

s

SQL>

The test concluded that the replication is successful from the source to the

target and that OGG performs the mapping and transformation as expected. Note

that on the target you have ‘Garc??a’ instead of 'Alejandro García' due to

character conversion.

Summary

In the article you had a look at the steps to configure and troubleshoot OGG

using local integrated capture extract. You had a glimpse at an example

utilizing OGG SQLEXEC and OGG functions for mapping and data transformation

for replication across dissimilar table structures.

Appendix

GGSCI (raclinux1.gj.com) 1> view params extint

extract extint

SETENV (ORACLE_SID = "RACD1")

--tranlogoptions asmuser sys@ASM, asmpassword sys1

TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3)

targetdefs ./dirdef/target_int.def

userid ogg_extract, password ogg_extract

rmthost raclinux1, mgrport 7809

rmttrail ./dirdat/2x

table test3.state;

table test3.customer;

table test3.customer_detail;

table test3.customer target test3_t.customer_detail1,

sqlexec( id statelookup, query " select state_desc from test3.state where state_id =

:state_code" , PARAMS(state_code=state_code)),

colmap(

usedefaults,

customer_id = @NUMSTR(NID),

First_name = @STREXT(Name,1,@STRFIND(Name," ")),

Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)),

NID = NID,

Tel_country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE",

"FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS",

"KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ",

"NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA

", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1",

"+44"),

Tel_area = @STREXT (Tel, 1, 3),

Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL",

"GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY"

, "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ",

"NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA",

"RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "USA",

"UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE",

"FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "K

Y", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ",

"NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA",

"RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1",

"+44"),

State_name = @GETVAL(statelookup.state_desc),

zip1 = @strext(zip,1,5),

zip2 = @strext(zip,6,10));;

GGSCI (raclinux1.gj.com) 2> view params repint

replicat repint

--reperror(default,ignore)

SETENV (ORACLE_SID = "RACDB1")

userid ogg_replicat, password ogg_replicat

sourcedefs ./dirdef/source_int.def

discardfile ./dirrpt/repint.dsc, purge

map test3.state, target test3_3.state;

map test3.customer, target test3.customer;

map test3.customer, target test3_t.customer_detail1,

sqlexec( id statelookup1, query " select state_desc from test3_t.state where state_id =

:state_code" , PARAMS(state_code=state_code)),

colmap(

usedefaults,

ADDRESS_CITY = ADRESS_CITY,

customer_id = @NUMSTR(NID),

First_name = @STREXT(Name,1,@STRFIND(Name," ")),

Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)),

NID = NID,

Tel_country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE",

"FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY",

"LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM",

"NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI",

"SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"),

Tel_area = @STREXT (Tel, 1, 3),

Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL",

"GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "L

A", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM",

"NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC"

, "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "USA", "UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE",

"FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY",

"LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM",

"NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "S

C", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"),

State_name = @GETVAL(statelookup1.state_desc),

zip1 = @strext(zip,1,5),

zip2 = @strext(zip,6,10));

map test3.customer, target test3_t.customer_detail,

sqlexec( id statelookup, query " select state_desc from test3_t.state where state_id =

:state_code" , PARAMS(state_code=state_code)),

colmap(

usedefaults,

ADDRESS_CITY = ADRESS_CITY,

customer_id = @NUMSTR(NID),

First_name = @STREXT(Name,1,@STRFIND(Name," ")),

Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)),

NID = NID,

Tel_country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE",

"FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY",

"LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM",

"NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI",

"SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"),

Tel_area = @STREXT (Tel, 1, 3),

Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL",

"GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "L

A", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM",

"NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC"

, "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "USA", "UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE",

"FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY",

"LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM",

"NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "S

C", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"),

State_name = @GETVAL(statelookup.state_desc),

zip1 = @strext(zip,1,5),

zip2 = @strext(zip,6,10));

GGSCI (raclinux1.gj.com) 3>

Errors resolved during the configuration

2012-10-25 16:29:02 INFO OGG-02000 Ignoring option ASMUSER because it is

incompatible with an integrate

d capture configuration.

2012-10-25 16:29:02 INFO OGG-02000 Ignoring option ASMPASSWORD because it is

incompatible with an integ

rated capture configuration.

2012-10-25 16:29:02 WARNING OGG-02064 Oracle compatibility version 11.2.0.0.0 has

limited datatype support

for integrated capture. Version 11.2.0.3 required for full support.

2012-10-25 16:29:04 WARNING OGG-02045 Database does not have streams_pool_size

initialization parameter co

nfigured.

2012-10-25 16:29:18 INFO OGG-02036 Integrated capture successfully attached to

logmining server OGG$EXT

INT.

2012-10-25 16:29:30 INFO OGG-01226 Socket buffer size set to 27985 (flush size

27985).

2012-10-25 16:29:30 INFO OGG-01055 Recovery initialization completed for target file

./dirdat/2x000004,

at RBA 1097.

2012-10-25 16:29:30 INFO OGG-01478 Output file ./dirdat/2x is using format RELEASE

11.2.

2012-10-25 16:29:30 INFO OGG-01026 Rolling over remote file ./dirdat/2x000005.

2012-10-25 16:29:30 INFO OGG-01053 Recovery completed for target file

./dirdat/2x000005, at RBA 1036.

2012-10-25 16:29:30 INFO OGG-01057 Recovery completed for all targets.

***********************************************************************

** Run Time Messages **

***********************************************************************

2012-10-25 16:29:57 WARNING OGG-00869 ORA-00600: internal error code, arguments:

[KGHFRE3], [0x08AB26040],

[], [], [], [], [], [], [], [], [], []

ORA-26914: Unable to communicate with GoldenGate capture process "OGG$CAP_EXTINT" from

outbound server "OGG$

EXTINT".

References

1. Best Practice - Oracle GoldenGate 11gr2 integrated extract and Oracle

Data Guard - Switchover/Fail-over Operations Version 1.1 Document ID:

1436913.1

2. Oracle GoldenGate Best Practices: Configuring Downstream Integrated

Extract Document ID: 1485620.1

3. Oracle® GoldenGate Oracle Installation and Setup Guide Release 11.2.1

E35957-01 Oracle Installation and Setup Guide 4. OGG Administrator's Guide

5. OGG Reference Guide