Oracle GoldenGate 11 · PDF fileOracle GoldenGate Best Practices: Configuring Downstream...

Oracle GoldenGate Best Practices: Configuring Downstream Integrated Extract

An Oracle White Paper

July 2012

Oracle GoldenGate Best Practices: Configuring

Downstream Integrated Extract

Document ID: 1485620.1

Last Update: 10 November 2014


2

DOCUMENT ID: 1485620.1 .............................................................................. 1

EXECUTIVE OVERVIEW ................................................................................. 5

INTRODUCTION ............................................................................................... 6

DOWNSTREAM INTEGRATED EXTRACT SETUP ......................................... 7

Evaluating capture options for a downstream deployment .................................................................. 7

Configuring redo transport from a source to the downstream mining database ............................... 8

Creating a mining user account and source database user account .................................................. 10

Set Streams pool at downstream mining database .............................................................................. 10

Preparing a downstream mining database for real-time capture ...................................................... 11 Configure the downstream database to archive local redo data ........................................................... 11 Configure the downstream database to receive and write shipped redo data locally ........................... 11 Create the standby redo log files for real-time downstream mining .................................................... 12

Create the Downstream Integrated Extract ........................................................................................ 14

OPTIONAL PARAMETERS FOR INTEGRATED EXTRACT ......................... 14

DATABASE PATCHING OR UPGRADE CONSIDERATIONS ...................... 15

EXAMPLE 1: CONFIGURATION OF REAL-TIME DOWNSTREAM MINING

DATABASE FOR INTEGRATED EXTRACT.................................................. 16

Configure the downstream database to archive the local redo data .................................................. 17

Prepare the mining database to accept redo from the source database ............................................ 17

Prepare the source database to send redo to the mining database .................................................... 18

Configure the downstream database to receive and write shipped foreign redo data ..................... 19

Set up Integrated Extract on DBMSCAP ............................................................................................ 19


3

EXAMPLE 2: CONFIGURATION OF REAL-TIME DOWNSTREAM MINING

DATABASE FOR INTEGRATED EXTRACT FOR A RAC SOURCE ............ 21

EXAMPLE 3: CONFIGURATION OF ARCHIVE LOG DOWNSTREAM

MINING DATABASE FOR INTEGRATED EXTRACT .................................... 22


Prepare the source database to send archive to the mining database ............................................... 23

Set up Extract to capture changes from archived logs sent by DBMS1 ............................................ 24


MINING DATABASE FOR INTEGRATED EXTRACT FOR A RAC SOURCE

........................................................................................................................ 25


MINING DATABASE FOR INTEGRATED EXTRACT FOR MULTIPLE

SOURCE DATABASES ................................................................................. 27

Configure the downstream mining database to archive the local redo data ..................................... 28

Prepare the first source database to send archive to the mining database ....................................... 29

Prepare the second source database to send archive to the mining database ................................... 30

Set up Extract1 to capture changes from archived logs sent by DBMS1 .......................................... 30



MINING FOR 1ST SOURCE DATABASES AND REAL-TIME DOWNSTREAM

MINING FOR 2ND SOURCE DATABASE ....................................................... 33


Prepare the first source database to send archive logs to the mining database ................................ 35

Prepare the mining database to accept redo from the 2nd

source database ...................................... 36

Prepare the 2nd

source database to send redo to the mining database .............................................. 36


4

Configure the downstream database to receive and write shipped foreign redo data ..................... 37


Set up Extract2 to capture changes from redo logs sent by DBMS2 ................................................. 38

CONSIDERATIONS FOR REAL-TIME MINING OF MULTIPLE SOURCE

DATABASES .................................................................................................. 40

CONCLUSION ................................................................................................ 40

Caution

This document touches briefly on many important and complex concepts and does not provide

a detailed explanation of any one topic since the intent is to present the material in the most

expedient manner. The goal is simply to help the reader become familiar enough with the

product to successfully design and implement an Oracle GoldenGate environment. To that end,

it is important to note that the activities of design, unit testing and integration testing which

are crucial to a successful implementation have been intentionally left out of the guide. All the

examples are provided as is. Oracle consulting service is highly recommended for any

customized implementation.


5

Executive Overview

This document is an introduction to Oracle GoldenGate’s best practices and guidelines for

configuring a downstream mining database for Integrated Extract. This document is intended

for Oracle Database Administrators (DBAs), and Oracle Developers with some basic

knowledge of Oracle GoldenGate software product. The document is intended to be a

supplement to the existing series of documentation available from Oracle.

The following assumptions have been made during the writing of this document:

The reader has basic knowledge of Oracle GoldenGate products and concepts

Referencing Oracle GoldenGate Version 11.2.1 and above

Referencing Oracle® Data Guard Concepts and Administration 11g Release 2 (11.2)

Referencing Oracle® Database Net Services Administrator's Guide 11g Release 2 (11.2)

Referencing Oracle Version 10.2 and above

Referencing OS: All Oracle GoldenGate supported platforms for Oracle


6

Introduction

Oracle GoldenGate provides data capture from transaction logs and delivery for

heterogeneous databases and messaging systems. Oracle GoldenGate (OGG) provides a

flexible, de-coupled architecture that can be used to implement virtually all replication

scenarios.

In release 11.2.1, OGG released a feature which enables downstream capture of data from a

single source or multiple sources. This feature is specific to Oracle databases only. This

feature helps customers meet their IT requirement of limiting new processes being installed

on their production source system. This feature requires some configuration of log transport

to the downstream database on the source system. It also requires an open downstream

database which is where the Integrated Extract will be installed.

Downstream Integrated Extract can be configured to process either real-time or archive logs.

The real-time mining can be configured for only one source database at a given downstream

database server at one time. If the implementation requires more than a single source

database be replicated, the archive log mode of downstream Integrated Extract must be

configured.

“Downstream”“Source”

Capture Trail

Files

Pump

“Target”

DeliveryTrail

Files

This document is a supplement to the OGG 11.2.1 Oracle Installation Guide. It has more

details and examples on how to configure downstream Integrated Extract feature which was

introduced starting with Oracle GoldenGate version 11.2.1.

Prerequisites for Implementing Downstream Capture:

Oracle Source Database Version 10.2.0.4 and above

o Grant “select on v_$database” to source OGG user if the Oracle Source

Database Version is 10.2.

Oracle Downstream Mining Database Version 11.2.0.3


7

o 11.2.0.3 Database specific bundle patch for Integrated Extract 11.2.x (Doc ID

1411356.1 ).

Downstream Integrated Extract setup

Evaluating capture options for a downstream deployment

When deploying Integrated Extract in a downstream mining database you can choose

between two options:

1. Option 1 is real-time mining. You can use the downstream mining database to

capture changes only from a single source database. You need to create standby

redo logs so that the source database can use Oracle Data Guard redo transport

to send redo to the downstream mining database as it is written to the online

redo log at the source.

Note: Only one source database can be configured for real time mine in the downstream

mining database.

2. Option 2 is archive log mining. You can use the downstream mining database to

capture changes for single or multiple source databases. You don’t need standby

redo logs in the downstream database for this configuration. Each extract

session will capture changes from the archived logs sent from a specific source

database to the downstream mining database. Using this option will incur

additional latency. The latency of Downstream Extract can be controlled by

setting the source database init.ora parameter ARCHIVE_LAG_TARGET. This

parameter is set to a number of seconds. Since this parameter forces a log switch

on all nodes, it is not recommended to set this very low. The recommended

setting should be around 15 minutes or 900 seconds.

For more information about the parameters used in this procedure, see the Oracle® Database

Reference 11g Release 2 (11.2) and Oracle Data Guard Concepts and Administration 11g

Release 2 (11.2).


8

Configuring redo transport from a source to the downstream

mining database

Complete the following steps to set up the transfer of redo log files from a source database to

the downstream mining database, and to prepare the downstream mining database to accept

these redo log files. These instructions take into account the requirements to ship redo from

multiple sources, if required. You configure an Extract process for each of those sources.

1. Configure Oracle Net so that each source database can communicate with the mining

database. The example below assumes the following and should be modified for your

environment:

DBMSCAP.EXAMPLE.COM is the downstream mining database

coe-02 is the hostname of the downstream mining server

1521 is the DBA assigned port to this downstream mining database

dmscap is the DBA assigned service name

For instructions, see Oracle® Database Net Services Administrator's Guide 11g Release 2

(11.2).

Example: (TNSNAMES.ORA)

dbmscap =

(DESCRIPTION =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = coe-02)(PORT = 1521))

)

(CONNECT_DATA =

(SERVICE_NAME = dbmscap.example.com)

)

)

2. Configure authentication at each source database and the downstream mining database

to support the transfer of redo data. Redo transport sessions are authenticated using


9

either the Secure Sockets Layer (SSL) protocol or a remote login password file. If a source

database has a remote login password file, copy it to the appropriate directory of the

mining database system. The password file must be the same at all source databases as

well as the mining database. For more information about authentication requirements for

redo transport, see Preparing the Primary Database for Standby Database Creation in

Oracle® Data Guard Concepts and Administration 11g Release 2 (11.2).

3. At each source database, configure at least one LOG_ARCHIVE_DEST_n initialization

parameter to transmit redo/archive data to the downstream mining database. Set the

attributes of this parameter as shown in one of the following examples, depending on

whether real-time or archive-log capture mode is to be used.

Example for a real-time downstream logmining server:

SQL >ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=DBMSCAP.EXAMPLE.COM

ASYNC NOREGISTER VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) REOPEN=10

DB_UNIQUE_NAME=dbmscap'

Example for an archived-log-only downstream logmining server.

SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=DMBSCAP.EXAMPLE.COM


TEMPLATE=/usr/oracle/log_for_dbms1/dbms1_arch_%t_%s_%r.log


NOTE When using an archived-log-only downstream mining database, specify a value for

TEMPLATE attribute that keeps log files from all remote source databases separated from the

local database log files, and from each other.

4. At the source database, set a value of ENABLE for the LOG_ARCHIVE_DEST_STATE_n

initialization parameter that corresponds with the LOG_ARCHIVE_DEST_n parameter on

the downstream mining database, as shown in the following example.

SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_2=ENABLE

5. At both the source database and the downstream mining database for real-time mine, set

the DG_CONFIG attribute of the LOG_ARCHIVE_CONFIG initialization parameter to

include the DB_UNIQUE_NAME of the source database and the downstream database.

For archive log mine, DG_CONFIG attribute only needs to be set at the source. The

syntax is shown in the following example.

SQL> ALTER SYSTEM SET LOG_ARCHIVE_CONFIG='DG_CONFIG=(dbms1,dbmscap)'


10

Creating a mining user account and source database user

account

To grant the downstream mining user the required privileges on the mining database and

source database, see “Assigning a database user for Oracle GoldenGate” in the installation

guide.

Set Streams pool at downstream mining database

The shared memory that is used by the logmining server comes from the Streams pool

portion of the System Global Area (SGA) in the database. Therefore, you must set

streams_pool_size high enough to keep enough memory available for the number of Extract

processes that you expect to run in integrated capture mode against a given database

instance. Note that Streams pool is also used by other components of the database (like AQ

and Streams), so make certain to take them into account while sizing the Streams pool for

Oracle GoldenGate. This database parameter setting must be saved to disk in the init.ora file

or an spfile.

By default, one integrated extract requests the logmining server to run with max_sga_size of

1GB. Thus, if you are running three Extracts in integrated capture mode in the same database

instance, you need at least 3 GB of memory allocated to the Streams pool. As an Oracle

GoldenGate best practice, keep 25 percent of the Streams pool available for other

components.

For example, if there are three Extracts in integrated capture mode, set stream_pool_size to

the following:

(# of Extracts)*(1 GB) = Minimum OGG Required Streams Pool Memory

3*(1 GB) = 3 GB

(OGG Required Streams Pool Memory) * .25 = 25% additional Memory

3 GB * .25 = .75 GB

Oracle GoldenGate Best Practice Streams Pool Setting

= OGG Required Streams Pool Memory + 25% Additional Memory

= 3 GB + .75 GB = 3.75 GB

SQL> alter system set streams_pool_size=3750M scope=both;


11

Preparing a downstream mining database for real-time capture

This procedure is required only if you want to use real-time capture at a downstream mining

database. It is not required to use archived-log-only mode.

NOTE Real-time capture can only be used for one source database at a given downstream

database server. This procedure has the following parts:

Configure the database to archive the standby redo data

Configure the database to receive and write shipped redo data locally

Create the standby redo log files

Configure the downstream database to archive local redo data

This procedure sets the archive location for the redo data that is generated locally by the

downstream mining database. This step is not a requirement, but it is a recommended best

practice.

1. At the downstream mining database, set at least one local archive log destination in the

LOG_ARCHIVE_DEST_n initialization parameter as shown in the following example:

SQL> ALTER SYSTEM SET

LOG_ARCHIVE_DEST_1='LOCATION=/home/arc_dest/local_rl_dbmscap

VALID_FOR=(ONLINE_LOGFILE,PRIMARY_ROLE)'

NOTE Log files from all remote source databases must be kept separate from local mining

database log files. For information about configuring a fast recovery area, see Oracle®

Database Backup and Recovery User's Guide 11g Release 2 (11.2).

2. Set the LOG_ARCHIVE_DEST_STATE_n initialization parameter that corresponds with the

LOG_ARCHIVE_DEST_n parameter to ENABLE, as shown in the following example.


Configure the downstream database to receive and write shipped

redo data locally

This procedure configures the downstream mining database to receive redo data from the

source database and to write the redo to the local standby redo log.


12

1. At the downstream mining database, set at least one archive log destination for the

standby log groups in the LOG_ARCHIVE_DEST_n initialization parameter as shown in the

following example.

SQL>ALTER SYSTEM SET

LOG_ARCHIVE_DEST_2='LOCATION=/home/arc_dest/srl_dbms1

VALID_FOR=(STANDBY_LOGFILE,ALL_ROLES)'

NOTE Log files from all remote source databases must be kept separate from local mining

database log files, and from each other. For information about configuring a fast recovery

area, see Oracle® Database Backup and Recovery User's Guide 11g Release 2 (11.2).

2. Set the LOG_ARCHIVE_DEST_STATE_n initialization parameter that corresponds with the

LOG_ARCHIVE_DEST_n parameter to ENABLE as shown in the following example.

SQL>ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_2=ENABLE

For more information about these initialization parameters, see Oracle Data Guard Concepts

and Administration 11g Release 2 (11.2).

Create the standby redo log files for real-time downstream mining

The following steps outline the procedure for adding standby redo log files to the

downstream mining database. The specific steps and SQL statements that are required to add

standby redo log files depend on your environment. See Oracle Data Guard Concepts and

Administration 11g Release 2 (11.2) for detailed instructions about adding standby redo log

files to a database.

1. In SQL*Plus, connect to the source database as an administrative user.

2. Determine the size of the source log file. Make note of the results.

SQL> SELECT BYTES FROM GV$LOG;

3. Determine the number of online log file groups that are configured on the source

database. Make note of the results.

SQL> SELECT COUNT(GROUP#) FROM GV$LOG;

4. Connect to the downstream mining database as an administrative user.


13

5. Add the standby log file groups to the mining database. The standby log file size must be

at least the size of the source log file size. To determine the recommended number of

standby redo logs, use the following formula:

(maximum # of logfile groups +1) * maximum # of threads

For example, if a primary database has two instances (threads) and each thread has two

online log groups, then there should be six standby redo logs ((2 + 1) * 2 = 6). Having one

more standby log group for each thread (that is, one greater than the number of the online

redo log groups for the primary database) reduces the likelihood that the logwriter process

for the primary instance is blocked because a standby redo log cannot be allocated on the

standby database. The following example shows three standby log groups.

SQL> ALTER DATABASE ADD STANDBY LOGFILE GROUP 3

('/oracle/dbs/slog3a.rdo', '/oracle/dbs/slog3b.rdo') SIZE 500M;


('/oracle/dbs/slog4.rdo', '/oracle/dbs/slog4b.rdo') SIZE 500M;



6. Confirm that the standby log file groups were added successfully.

SQL> SELECT GROUP#, THREAD#, SEQUENCE#, ARCHIVED, STATUS FROM

V$STANDBY_LOG;

The output should be similar to the following:

GROUP# THREAD# SEQUENCE# ARC STATUS

---------- ---------- ---------- --- ----------

3 0 0 YES UNASSIGNED



7. Ensure that log files from the source database are appearing in the location that is

specified in the LOCATION attribute of the local LOG_ARCHIVE_DEST_n that you set. You

might need to switch the log file at the source database to see files in the directory.


14

Create the Downstream Integrated Extract

1. Enable Supplemental logging on source database tables

GGSCI> DBLOGIN USERID ggadm1@dbms1 PASSWORD ggs123

GGSCI> ADD TRANDATA <schema>.*

2. Register Extract with the downstream mining database on the downstream server. Make

sure the ORACLE_SID is set to the name of the downstream database.


GGSCI> MININGDBLOGIN USERID ggadmcap PASSWORD ggs123

GGSCI> REGISTER EXTRACT ext1 DATABASE

3. Create Extract at the downstream mining database.

GGSCI> ADD EXTRACT ext1, INTEGRATED TRANLOG, BEGIN NOW

4. Edit Extract parameter file. The following lines must be present to take advantage

downstream Integrated Extract.

USER ggadm1@dbms1 PASSWORD ggs123

TRANLOGOPTIONS MININGUSER ggadmcap MININGPASSWORD ggs123

--For Real-Time Integrated Extract (DEFAULT VALUE)

TRANLOGOPTIONS INTEGRATEDPARAMS (downstream_real_time_mine Y)

Or

-- For Archive Log Integrated Extract

TRANLOGOPTIONS INTEGRATEDPARAMS (downstream_real_time_mine N)

5. Start Extract.

GGSCI> START EXTRACT ext1

Optional Parameters for Integrated Extract

Integrated Extract uses a database logmining server in the mining database to mine the redo

stream of the source database. The logmining server consumes two types of database


15

resources: 1) a given number of processes and 2) a given amount of shared memory. These

resource values are dependent on your available system memory and cpu. In most cases, the

default value is sufficient. However, if your system is lower in resources, you may need to

decrease these values. You can control these resources by specifying the following

parameters in the INTEGRATEDPARAMS list of the Extract TRANLOGOPTIONS parameter:

To control the amount of shared memory used by the logmining server, specify the

max_sga_size parameter with a value in megabytes. The default value is 1 GB if

streams_pool_size is greater than 1 GB; otherwise, it is 75% of streams_pool_size.

To control the number of processes used by the logmining server, specify the

parallelism parameter. The default value is 2

For example, you can specify that the logmining server uses 200 MB of memory with a

parallelism of 3 by specifying the following in the Extract parameter file:

TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 200, parallelism 3)

Database patching or upgrade considerations

Downstream capture can mine logs from lower release databases, but not from higher

release databases. So if you plan to upgrade your source database to a higher release or

patchset than the downstream mining database release, you will need to upgrade the

associated mining database first. Then you can upgrade the source database. With this in

mind, if you have multiple mining instances configured on the same host server, you should

use separate ORACLE_HOME directories for each instance. Otherwise, an upgrade of one

source database will impact all your downstream mining databases.


16

Example 1: Configuration of real-time downstream

mining database for Integrated Extract

ARC0

LNSn

Online

Redo

Log

Online

Redo

Log

LGWR

dbms1

Source

Database

LOG_ARCHIVE_DEST_1Archived

Redo

Logs

RFS

Redo

Log

Standby

Redo

Log

ARCn

Redo

Logs

Foreign

ArchivedRedoLogs

dbmscap

Mining

Database

ARC0

Online

Redo

Log

Online

Redo

Log

LGWR

Archived

RedoLogs

Local

ArchivedRedoLogs

LOG_ARCHIVE_DEST_1

LOG_ARCHIVE_DEST_2

Source Database Mining Database

Real-Time Downstream Logmining

Integrated

Capture

'SERVICE=DBMS2.EXAMPLE.COM ASYNC NOREGISTER

VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE)

DB_UNIQUE_NAME=dbms2'

dbmscap

ext1

/home/arc_dest/srl_dbms1

/home/arc_dest/local_rl_dbmscap

LOG_ARCHIVE_DEST_2Local

TrailTrail

This example captures changes from source database DBMS1 by deploying an Integrated

Extract session at a downstream mining database DBMSCAP. This example assumes that 1)

communication exists between the source and target, 2) password files have been copied, 3)

STREAMS_POOL_SIZE has been set on the mining server and 4) that the following users exist:

●User GGADM1 in DBMS1 whose credentials Extract will use to fetch data and

metadata from DBMS1. This user needs the

DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE() procedure executed to

grant the appropriate permissions. For Oracle 10.2, the user must be granted the

additional permission of “select on v_$database”

SQL> exec DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE (

grantee=>'GGADM1', privilege_type=>'capture',

grant_select_privileges=>true, do_grants=>TRUE);

● User GGADMCAP in DBMSCAP whose credentials Extract will use to retrieve logical

change records from the logmining server at the downstream mining database

DBMSCAP. It is assumed that the


17

DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE() procedure was called to

grant appropriate privileges to this user.


grantee=>'GGADMCAP', privilege_type=>'capture',


Configure the downstream database to archive the local redo

data


downstream mining database. This is not a requirement, but it is a recommended best

practice.

1. At the downstream mining database, set one local archive log destination in the

LOG_ARCHIVE_DEST_1 initialization parameter as shown in the following example:




2. Set the LOG_ARCHIVE_DEST_STATE_1 initialization parameter that corresponds with the

LOG_ARCHIVE_DEST_1 parameter to ENABLE, as shown in the following example.


Prepare the mining database to accept redo from the source

database

1. Add the standby log file groups to the downstream mining database. The following

example shows three standby log groups.

SQL> ALTER DATABASE ADD STANDBY LOGFILE GROUP 3 ('/oracle/dbs/slog3a.rdo',

'/oracle/dbs/slog3b.rdo') SIZE 500M;

SQL> ALTER DATABASE ADD STANDBY LOGFILE GROUP 4 ('/oracle/dbs/slog4.rdo',


SQL> ALTER DATABASE ADD STANDBY LOGFILE GROUP 5 ('/oracle/dbs/slog5.rdo',



18



V$STANDBY_LOG;



---------- ---------- ---------- --- ----------




3. Set DG_CONFIG at the downstream mining database.


Prepare the source database to send redo to the mining

database

1. Make sure that the source database is running with the required compatibility.

SQL> select name, value from v$parameter where name = 'compatible';

NAME VALUE

--------- ---------------------

compatible 11.1.0.7.0

The minimum compatibility setting required from Integrated Extract is 10.2.0.4.0

2. Set DG_CONFIG at the source database.

SQL> ALTER SYSTEM SET LOG_ARCHIVE_CONFIG='DG_CONFIG=(dbms1,dbmscap)';

3. Set up redo transport at the source database.


ASYNC NOREGISTER VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE)REOPEN=10


4. Enable the downstream destination.

SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_2=ENABLE;


19

Configure the downstream database to receive and write

shipped foreign redo data

1. At the downstream mining database, set one archive log destination in the

LOG_ARCHIVE_DEST_2 initialization parameter as shown in the following example.


LOG_ARCHIVE_DEST_2='LOCATION=/home/arc_dest/srl_dbms1



LOG_ARCHIVE_DEST_2 parameter to ENABLE as shown in the following example.


Set up Integrated Extract on DBMSCAP

1. Enable supplemental logging on source database DBMS1 tables.



2. Register Extract with the downstream mining database. Make sure the ORACLE_SID for

the mining database is set in the environment.


GGSCI> MININGDBLOGIN USERID ggadmcap PASSWORD ggs123


3. Create Extract at the downstream mining database.

GGSCI> ADD EXTRACT ext1 INTEGRATED TRANLOG BEGIN NOW

GGSCI> ADD EXTTRAIL ./dirdat/lt, EXTRACT ext1, MEGABYTES 200

4. Edit Extract parameter file ext1.prm. The following lines must be present to take

advantage of real-time capture.

USERID ggadm1@dbms1 PASSWORD ggs123

TRANLOGOPTIONS MININGUSER ggadmcap MININGPASSWORD ggs123



20

5. Start Extract.


NOTE Oracle does not allow real time mining of the redo logs for more than one source database. When

you need to configure multiple source databases, only one source database can be configured in real

time mode. The others have to be in archive mode.


21

Example 2: Configuration of real-time downstream

mining database for Integrated Extract for a RAC

source

ARCO

LNSn

LGWR

RAC Source

Database

Node 1

RAC Source

Database

Node 1

LOG_ARCHIVE_DEST_1

LOG_ARCHIVE_DEST_2

Archived

Redo Logs

RFS

MiningDatabase

ARC0

Online

Redo

Log

Online

Redo

Log

LGWR

Archived

RedoLogs

Local

ArchivedRedoLogs

LOG_ARCHIVE_DEST_1


Real-Time Downstream Logmining For a RAC Source

Integrated

Capture

Local

TrailTrail

'SERVICE=DBMSCAP.EXAMPLE.COM ASYNC

OPTIONAL NOREGISTER



ext1

dbmscap


Online Redo

Logs

Archived

Redo Logs

Online Redo

Logs LNSn

ARCO

LGWR

RAC Source

Database

Node 2


OPTIONAL NOREGISTER



LOG_ARCHIVE_DEST_2

LOG_ARCHIVE_DEST_1

dbms1

Standby

Redo Log

ARCn

Foreign

Archive Redo

Logs

/home/arc_dest/srl_dbms1

LOG_ARCHIVE_DEST_2

dbms1

The steps for configuring a real-time downstream mining database for a RAC source is exactly

the same as the previous example. The diagram shows the differences between a single

instance and RAC instance from an architecture perspective. But the actual steps to setup the

downstream capture are exactly the same as example 1. The LOG_ARCHIVE_DEST_2 is a

global parameter. Once it is set, it is valid for both threads.


22

Example 3: Configuration of archive log downstream

mining database for Integrated Extract

ARC0

LNSnOnline

Redo

Log

Online

Redo

Log

LGWR

Primary

Database

Source

Database

LOG_ARCHIVE_DEST_1

LOG_ARCHIVE_DEST_2

Redo

Logs

Archived

Redo

Logs

RFS

Redo

Foreign

ArchivedRedo

Log

/usr/orcl/arc_dest/dbms1/

dbms1_arch_%t_%s_%r.log

MiningDatabase

ARC0

Online

Redo

Log

Online

Redo

Log

LGWR

Archived

RedoLogs

Local

ArchivedRedoLogs

LOG_ARCHIVE_DEST_1


Archive Log Downstream Logmining

Integrated

Capture

Local

TrailTrail


OPTIONAL NOREGISTER TEMPLATE='/usr/orcl/

arc_dest/dbms1/dbms1_arch_%t_%s_%r.log



ext1

dbms1

dbmscap


The following example captures changes from database DBMS1 by deploying an archive log

Integrated Extract session at a downstream mining database DBMSCAP. This example

assumes that 1) communication exists between the source and target, 2) password files have

been copied, 3) STREAMS_POOL_SIZE has been set on the mining server and 4) that the

following users exist:










23


change records from the log mining server at the downstream mining database. It is

assumed that the DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE() procedure

was called to grant appropriate privileges to this user.

SQL> exec DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE

(grantee=>'GGADMCAP', privilege_type=>'capture',



data



practice.









.

Prepare the source database to send archive to the mining

database

1. Make certain that DBMS1 source database is running with the required compatibility.


NAME VALUE

--------- ---------------------


The minimum compatibility setting required from Integrated Extract is 10.2.0.4.0.


24

2. Set DG_CONFIG at DBMS1 source database.


3. Set Archive Log Latency (ARCHIVE_LAG_TARGET) in seconds at DBMS1 Source database.

This should be set in the init.ora file, but also can be set dynamically. This example

dynamically sets latency to the recommended 15 minutes.

SQL> ALTER SYSTEM SET ARCHIVE_LAG_TARGET=900 ;

4. Set up archive transport at DBMS1 source database.

SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=DBMSCAP.EXAMPLE.COM

ASYNC OPTIONAL NOREGISTER

TEMPLATE='/usr/orcl/arc_dest/dbms1/dbms1_arch_%t_%s_%r.log

VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE)REOPEN=10 DB_UNIQUE_NAME=dbmscap';



Set up Extract to capture changes from archived logs sent by

DBMS1

The following steps are all done on the DBMSCAP downstream mining database. Make sure

that the ORACLE_SID is set in the environment.




2. Register Extract with DBMSCAP for the DBMS1 source database.

GGSCI> DBLOGIN USERID ggadm1@dbms1 PASSWORD ggadm1pw

GGSCI> MININGDBLOGIN USERID ggadmcap PASSWORD ggadmcappw


3. Add Extract at the mining database DBMSCAP.

GGSCI> ADD EXTRACT ext1, INTEGRATED TRANLOG, BEGIN NOW

GGSCI> ADD EXTTRAIL ./dirdat/lt, EXTRACT ext1, MEGABYTES 200


25

4. Edit the Extract parameter file ext1.prm. The following lines must be present to take

advantage of archive log capture.

USERID ggadm1@dbms1 PASSWORD ggadm1pw

TRANLOGOPTIONS MININGUSER ggadmcap, MININGPASSWORD ggadmcappw


5. Start Extract.



mining database for Integrated Extract for a RAC

source

ARCO

LNSn

LGWR

RAC Source

Database

Node 1

LOG_ARCHIVE_DEST_1

LOG_ARCHIVE_DEST_2

Archived

Redo Logs

RFS



MiningDatabase

ARC0

Online

Redo

Log

Online

Redo

Log

LGWR

Archived

RedoLogs

Local

ArchivedRedoLogs

LOG_ARCHIVE_DEST_1


Archive Log Downstream Logmining RAC Source

Integrated

Capture

Local

TrailTrail






ext1

dbms1

dbmscap


Online Redo

Logs

Archived

Redo Logs

Online Redo

LogsLNSn

ARCO

LGWR

RAC Source

Database

Node 2






LOG_ARCHIVE_DEST_2

LOG_ARCHIVE_DEST_1

dbms1

Foreign

Archived

Redo Log


26

The steps for configuring an archive log downstream mining database for a RAC source is

exactly the same as the previous example. The diagram shows the differences between a

single instance and RAC instance from an architecture perspective. But the actual steps to

setup the downstream capture are exactly the same as example 3. The

LOG_ARCHIVE_DEST_2 is a global parameter. Once it is set it is valid for both threads.


27


mining database for Integrated Extract for multiple

source databases

ARCO

LNSn

LGWR

Source

Database 1

LOG_ARCHIVE_DEST_1

LOG_ARCHIVE_DEST_2

Archived

Redo Logs

RFS



Mining

Database

ARC0

Online

Redo

Log

Online

Redo

Log

LGWR

Archived

RedoLogs

Local

ArchivedRedoLogs

LOG_ARCHIVE_DEST_1

Source Databases Mining Database

Archive Log Downstream Logmining 2 Source Databases

Integrated

Capture

Local

TrailTrail

ext1

dbms1

dbmscap


Online Redo

Logs

Archived

Redo Logs

Online Redo

LogsLNSn

ARCO

LGWR

Source

Database 2






LOG_ARCHIVE_DEST_2

LOG_ARCHIVE_DEST_1

dbms2Foreign

Archived

Redo Log








Integrated

Captureext2

RFS

Local

TrailTrail

The following example captures changes from database DBMS1 and DBMS2 by deploying an

Integrated Extract session at a downstream mining database DBMSCAP. This example

assumes that 1) communication exists between the sources and target, 2) password files have

been copied, 3) STREAMS_POOL_SIZE has been set on the mining server and 4) that the

following users exist:







28



















NOTE Oracle does not allow real time mining of the redo logs for more than one source database. When

you need to configure multiple source databases, only one source database can be configured in real

time mode. The others have to be in Archive mode.

Configure the downstream mining database to archive the

local redo data



practice.




29







Prepare the first source database to send archive to the

mining database



NAME VALUE

--------- ---------------------




SQL> ALTER SYSTEM SET LOG_ARCHIVE_CONFIG='DG_CONFIG=(dbms1, dbmscap)';









VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) REOPEN=10

DB_UNIQUE_NAME=dbmscap';



30


Prepare the second source database to send archive to the

mining database

1. Make sure that DBMS2 source database is running with the required compatibility.


NAME VALUE

--------- ---------------------

















Set up Extract1 to capture changes from archived logs sent by

DBMS1


the ORACLE_SID for the downstream database is set in the downstream server environment.


31










GGSCI> ADD EXTTRAIL ./dirdat/l1, EXTRACT ext1, MEGABYTES 200






5. Start Extract.



DBMS2

The following steps are all done on the downstream DBMSCAP mining database. Make sure






32

2. Register Extract with the mining database for source database DBMS2.

GGSCI> DBLOGIN USERID ggadm2@dbms2, PASSWORD ggadm2pw

GGSCI> MININGDBLOGIN USERID ggadmcap, PASSWORD ggadmcappw


3. Create Extract at the mining database.

GGSCI> ADD EXTRACT ext2 INTEGRATED TRANLOG, BEGIN NOW




USERID ggadm2@dbms2, PASSWORD ggadm2pwd



5. Start Extract.



33


mining for 1st

source databases and real-time

downstream mining for 2nd

source database

ARCO

LNSn

LGWR

Source

Database 1

LOG_ARCHIVE_DEST_1

LOG_ARCHIVE_DEST_2

Archived

Redo Logs

RFS



Mining

Database

ARC0

Online

Redo

Log

Online

Redo

Log

LGWR

Archived

RedoLogs

Local

ArchivedRedoLogs

LOG_ARCHIVE_DEST_1

Source Databases Mining Database

Archive Log Downstream Logmining for 1st Source Database & Realtime Downstream Logmining for 2nd Source Database

Integrated

Capture

Local

TrailTrail

ext1

dbms1

dbmscap


Online Redo

Logs

Archived

Redo Logs

Online Redo

LogsLNSn

ARCO

LGWR

Source

Database 2

LOG_ARCHIVE_DEST_2

LOG_ARCHIVE_DEST_1

dbms2Foreign

Archived

Redo Log








Integrated

Captureext2

RFS

Local

TrailTrail

ARCn

Standby

Redo Logs

'SERVICE=DBMS2.EXAMPLE.COM ASYNC NOREGISTER


DB_UNIQUE_NAME=dbms2'

LOG_ARCHIVE_DEST_2

The following example captures changes from database DBMS1 in archive mode and DBMS2

in real-time mode by deploying an Integrated Extract session at a downstream mining

database DBMSCAP for each source database. This example assumes that 1) communication

exists between the sources and target, 2) password files have been copied, 3)

STREAMS_POOL_SIZE has been set on the mining server and 4) that the following users exist:







34




















data



practice.









35


Prepare the first source database to send archive logs to the

mining database



NAME VALUE

--------- ---------------------


















36

Prepare the mining database to accept redo from the 2nd

source database

1. Add the standby log file groups to the downstream mining database. The following

example shows three standby log groups.


('/oracle/dbs/slog3a.rdo', '/oracle/dbs/slog3b.rdo') SIZE 500M;







V$STANDBY_LOG;



---------- ---------- ---------- --- ----------




3. Set DG_CONFIG at the downstream mining database.


Prepare the 2nd source database to send redo to the mining

database

1. Make sure that the source database is running with the required compatibility.


NAME VALUE

--------- ---------------------



37

The minimum compatibility setting required from Integrated Extract is 10.2.0.4.0

2. Set DG_CONFIG at the source database.


3. Set up redo transport at the source database.






Configure the downstream database to receive and write

shipped foreign redo data

1. At the downstream mining database, set one archive log destination in the

LOG_ARCHIVE_DEST_2 initialization parameter as shown in the following example.


LOG_ARCHIVE_DEST_2='LOCATION=/usr/orcl/arc_dest/dbms2



LOG_ARCHIVE_DEST_2 parameter to ENABLE as shown in the following example.



DBMS1







38













5. Start Extract.


Set up Extract2 to capture changes from redo logs sent by

DBMS2

The following steps are all done on the downstream DBMSCAP mining database. . Make sure





2. Register Extract with the mining database for source database DBMS2.


GGSCI> MININGDBLOGIN USERID ggadmcap, PASSWORD ggadmcappw


39


3. Create Extract at the mining database.

GGSCI> ADD EXTRACT ext2 INTEGRATED TRANLOG, BEGIN NOW




USERID ggadm2@dbms2, PASSWORD ggadm2pwd



5. Start Extract.



40

Considerations for real-time mining of multiple source

databases

If you want real-time mining of multiple source databases, you will have to use multiple

downstream mining databases. As discussed earlier in this paper, a mining database can only

have one source database configured for real-time mine. These mining databases could be

on the same host system assuming that both databases are of the same platform type. If you

decide to go with multiple downstream mining databases on the same host, you should plan

to use different ORACLE_HOME directories for each mining database, so that you can patch

and upgrade these databases independently. This is important because the downstream

mining database should be upgraded to a new patchset or release before the associated

source database. Downstream capture can mine logs from lower release databases, but not

from higher release databases.

Conclusion

This paper has presented many different approaches to implementing downstream

Integrated Extract. With this information you can now determine which approach is best for

your environment.


41

\ Oracle GoldenGate Best Practices: Configuring

Downstream Integrated Extract

March 2012

Author:s Sourav Bhattacharya, Tracy West

Contributing Authors: Patricia McElroy

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

oracle.com

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the

contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other

warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or

fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are

formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any

means, electronic or mechanical, for any purpose, without our prior written permission.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective

owners.

AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel

and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are

trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open

Company, Ltd. 0410

Oracle GoldenGate 11 · PDF fileOracle GoldenGate Best Practices: Configuring Downstream...

Documents

Transcript of Oracle GoldenGate 11 · PDF fileOracle GoldenGate Best Practices: Configuring Downstream...