GoldenGate for DB2 to MS SQL Server · GoldenGate for DB2 to MS SQL Server Page 2 Overview of Tasks...
Transcript of GoldenGate for DB2 to MS SQL Server · GoldenGate for DB2 to MS SQL Server Page 2 Overview of Tasks...
GoldenGate for DB2 to MS SQL Server
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 1
GoldenGate for DB2 to MS SQL Server
Objective
Upon completion of this lesson, you will be able to configure GoldenGate to provide DB2 to
MS SQL Server replication.
During this lesson, you will learn how to:
���� Prepare your user environment.
���� Configure and execute the initial data load process.
���� Configure and start the change capture process of database operations.
���� Configure and start the change delivery process of database operations.
Unix DB2 to SQL Server configuration
The following diagram illustrates a GoldenGate configuration for log-based replication from
a DB2 source database on UNIX to a SQL Server target database on Windows.
UNIX/Linux server <source>
DB2 <database>
Windows <target>
SQL Server
Target <owner/schema>
Source<owner/schema>
Network
Extract
Replicat
Collector
Trails
Manager Manager
Log
GoldenGate for DB2 to MS SQL Server
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 2
Overview of Tasks
Prepare the Environment
In order to execute this lesson, the GoldenGate application must be installed on both the
source and target systems. The installation includes a sample database and scripts to generate
initial data as well as subsequent update operations. The source and target tables are created
and loaded with initial data. The GoldenGate Manager processes are also started so that other
processes may be configured and started. And finally, source definitions are generated and
transfer to the target system.
Configure Initial Data Load
Initial database load is almost always required. To initially load data across heterogeneous
databases, you have limited choices. GoldenGate provides the ability to perform initial data
load while your application remains active.
This lesson demonstrates using Extract to pull data from the source files and send it directly
to the Replicat component on the target system.
Configure Change Capture
For log-based DB2, Extract captures database changes directly from the DB2 logs and
distributes them to files known as GoldenGate trails or directly to the Delivery process.
Configure Change Delivery
Once the tables have been initially loaded with data, the Delivery process is configured to
deliver the captured change data into the target database.
UNIX/Linux <source> server
DB2 <database>
Windows <target> server
SQL Server
Target <owner/schema>
Source<owner/schema>
Network
Initial
ExtractInitial
Replicat
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 3
Exercise 1.
Prepare the Environment
� � � � � � � � � � � � � � �
Objective
The goals of this exercise are to:
���� Configure and start the Manager processes
���� Prepare source and target sample databases
���� Configure database connections
Prepare the source UNIX DB2 environment
1. Configure the Manager process on the source
Execute the following commands on the <source> system.
� Start the command interface
Shell> cd <install location> Shell> ggsci
� Specify the port that the Manager should use.
GGSCI> EDIT PARAMS MGR
-- GoldenGate Manager Parameter file PORT <port>
� Start Manager
GGSCI> START MANAGER
� Verify the results:
GGSCI> INFO MANAGER
2. Create source tables and load with initial data
Execute the scripts demo_db2_create.sql and demo_db2_insert.sql on the <source> system
to create the source tables and load with data.
Note: It is advisable to update the demo_db2_create.sql and demo_db2_insert.sql scripts to
ensure that the database connection information matches your system.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 4
Shell> cd <install location> Shell> db2 –tf demo_db2_create.sql Shell> db2 –tf demo_db2_insert.sql
Verify the results:
Shell> db2 db2 => connect to <database> db2 => describe table tcustmer db2 => describe table tcustord db2 => select * from tcustmer db2 => select * from tcustord db2 => quit
3. Add supplemental logging
Using GGSCI, log in to the database on the <source>. Use the following command to turn on
change capture for the TCUSTMER and TCUSTORD tables.
Note: This will turn on the change capture for table columns except large objects. For large
objects, the data capture changes must be turned on by column and data type.
Shell> ggsci GGSCI> DBLOGIN SOURCEDB <database>, USERID <login>, PASSWORD <password> GGSCI> ADD TRANDATA <owner/schema>.TCUSTMER GGSCI> ADD TRANDATA <owner/schema>.TCUSTORD
Verify that supplemental logging has been turned on for these tables.
GGSCI> INFO TRANDATA <owner/schema>.TCUST*
4. Configure source definition generator
Execute the following commands on the <source> system to create the DEFGEN parameter
file and add the listed parameters.
Shell> cd <install location> Shell> ggsci GGSCI> edit param defgen
DEFSFILE dirdef/source.def, PURGE SOURCEDB <database>, USERID <login>, PASSWORD <password> TABLE <owner/schema>.TCUSTMER; TABLE <owner/schema>.TCUSTORD;
GGSCI> exit
5. Execute the source definition generator
Execute the following commands on the <source> system.
Shell> defgen paramfile dirprm/defgen.prm
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 5
6. Transfer the source definition file to the target system
Shell> ftp <target> Name (<source>:): <login> Password: password ftp> ascii ftp> cd <install location>/dirdef ftp> lcd <install location>/dirdef ftp> put source.def ftp> bye
Note: To avoid overlaying existing definition files, always put the file as <db type>.def
instead of source.def (e.g. DB2.def).
Prepare the SQL Server 2005 target system
Install Manager
1. Configure Manager process on the target
Execute the following commands on the <target> system.
Note! The port numbers must be unique if you are installing GoldenGate more than once on
the same server.
Shell> cd <install location> Shell> ggsci GGSCI> EDIT PARAMS MGR
� In the parameter file, enter the following parameter, then save and close the file.
-- GoldenGate Manager Parameter file PORT <port>
� Start Manager.
GGSCI> START MANAGER
� Verify the results:
GGSCI> INFO MANAGER
Steps to create the database
2. Create the database
From Start>Programs, run SQL Server Management Studio.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 6
� Enter the name of the local SQL Server instance, then click connect.
Note: These instructions assume SQL Server Authentication which requires the
more complex, double layer, of authentication. For only Windows Authentication,
you can leave out most of the login and password entries.
� This displays the main SQL Server Management Studio dialog box. Right, right-click Database and select the New Database option.
� The New Database dialog box appears. In the Name box, type the <database> from the Lab Preparation worksheet.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 7
� Click OK to add the database.
� The main SQL Server Management Studio dialog box reappears.
3. Create SQL Server logins and users
� Right-click Security and select the New > Login.
� The New Login dialog box appears.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 8
� For Name, type the <login>.
� Select Authentication,
� If you selected SQL Server Authentication, type the <password> in the two Password fields.
� Under Defaults, select your <database> for Database, and leave Language set to the default.
� Do not click OK. Instead, from Select a page in the upper left-hand box, double click User Mapping.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 9
� In the upper Users mapped to this login area, activate the checkmark next to your database.
� In the lower Database role membership for <database> area,, check the box for the db_owner role.
� Click OK to exit.
The next step creates a schema that you will associate with the new user.
� While you have the database selected, click on the New Query button in the main
menu.
� The right side of the dialog displays the SQL Query interface. Type in:
create schema <owner/schema>;
And press the Execute button from the toolbar.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 10
Now you need to associate the new user and the schema that you have created with the new
database.
� Right click on the Security option that is under the database and select New > User.
� The New Database User dialog box appears. Enter the name of your user and press the browse button labeled with three dots (…).
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 11
� The Select Login dialog displays. Press the Browse button.
� Select your new user <login> from the list. Then press OK.
� Your selected user now displays in the object names box. Press OK to exit from Select Login.
� Back in the New Database User dialog, enter the <schema> you added as the default schema and press OK.
4. Create the ODBC system data source name
The GoldenGate Extract process connects to a SQL Server database through an ODBC (Open
Database Connectivity) connection. Both the Extract and Replicat components of GoldenGate
require that a system data source name (DSN) be established, which stores the information
about how to connect to the SQL Server.
� Click Start > Settings > Control Panel.
� Double-click Administrative Tools.
� Double-click Data Sources (ODBC) to open the ODBC Data Source Administrator dialog box.
� Click the System DSN tab, and then click the Add button. The Create New Data Source dialog box appears.
Note: For SQL Server 2000, you may use any ODBC driver supported by the
database. For SQL Server 2005, however, you must use the ODBC driver that is
included with Microsoft Data Access Components (MDAC). Both Extract and
Replicat will abend if the SQL Native Client ODBC driver that is provided with SQL
Server 2005 is used.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 12
� Select the SQL Server driver and then click Finish. The Create a New Data Source to SQL Server dialog box appears.
� For Name, type <dsn>.
� For a description, type "GG Training Data source" when creating this source DSN. This field is optional.
� For Which SQL Server do you want to connect to, select the server name.
� Click Next.
� Answer the question about how SQL Server should verify the login by selecting the option that you entered when creating the <login>. Then type <login> in Login ID
and <password> in Password.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 13
� Click Next.
� Make certain the default database is set to the database that you created; otherwise select Change the default database to: and select it. Leave the other settings to their
defaults to use ANSI.
� Click Next.
� Leave the next dialog box set to the defaults, and click Finish.
� In the confirmation, click Test Data Source to test the connection.
� Close the confirmation dialog box and the Create a New Data Source box.
Prepare the Environment
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 14
5. Create the practice tables
Execute the following commands on the <target> system.
� Go to SQL Server Management Studio, select your database and select New Query from the toolbar.
� If the correct database is not displayed in the drop down box (showing ggsdata
above), click on the arrow and select it from the list.
� Click the File > Open and navigate to the demo_mss_create.sql script.
� Click Open to open the script in the New Query window.
� Execute the script by clicking the Execute Query button on the toolbar.
� Verify the results using the following commands in the input window:
sp_help tcustmer go sp_help tcustord go
Initial Data Load
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 15
Exercise 2.
Initial Data Load using Direct Load Method
� � � � � � � � � � � � � � �
Objective
The goals of this exercise are to:
���� Configure the initial load capture task.
���� Configure the delivery of the data to the target system.
� Execute and verify the initial load of data.
Configure initial load capture
1. Add the initial load Extract batch task group
Execute the following commands on the <source> system to create an Extract process called
EINI<unique id>1.
GGSCI> ADD EXTRACT EINI<unique id>, SOURCEISTABLE
Verify the results:
GGSCI> INFO EXTRACT *, TASKS
2. Configure the initial load Extract parameter file
Execute the following commands on the <source> system to create an initial load Extract
parameter file with the listed parameters.
GGSCI> EDIT PARAMS EINI<unique id>
-- -- GoldenGate Initial Data Capture -- for TCUSTMER and TCUSTORD -- EXTRACT EINI<unique id> SOURCEDB <database>, USERID <login>, PASSWORD <password> RMTHOST <target>, MGRPORT <port> RMTTASK REPLICAT, GROUP RINI<unique id> TABLE <owner/schema>.TCUSTMER; TABLE <owner/schema>.TCUSTORD;
11 The process names used in lab exercises, for example EINIBD, are made up of 1) one character for the
GoldenGate process (E for Extract, R for Replicat); 2) three or four to describe the process type (INI for initial data load, ORA for capture from or delivery to an Oracle database, etc.) and 3) two characters to
create a unique identifier (usually your initials).
Initial Data Load
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 16
GGSCI> EXIT
Configure initial load delivery
3. Configure the initial load Replicat parameter file
Execute the following commands on the <target> system to create the initial load Replicat
parameter file with the listed parameters.
GGSCI> EDIT PARAMS RINI<unique id>
-- -- GoldenGate Initial Load Delivery for TCUSTMER -- REPLICAT RINI<unique id> USERID <login>, PASSWORD <password> SOURCEDEFS <install location>/dirdef/source.def MAP <owner/schema>.TCUSTMER, TARGET <owner/schema>.TCUSTMER; MAP <owner/schema>.TCUSTORD, TARGET <owner/schema>.TCUSTORD;
Note: Remember to use <db type>.def if you renamed the source.def when you transferred it
to the <target>.
4. Add the initial load Replicat batch task group
Execute the following commands on the <target> system.
GGSCI> ADD REPLICAT RINI<unique id>, SPECIALRUN
Verify the results:
GGSCI> INFO REPLICAT *, TASKS
5. Execute the initial load process
Execute the following commands on the <source> system.
GGSCI> START EXTRACT EINI<unique id>
Verify the results:
GGSCI> VIEW REPORT EINI<unique id>
On the <target> system:
GGSCI> VIEW REPORT RINI<unique id>
Configure Change Capture
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 17
Exercise 3.
Configure Change Capture
� � � � � � � � � � � � � � �
Objective
The goals of this exercise are to:
���� Add the Extract process that will capture changes.
���� Add the trail that will store the changes.
� Start the Extract process.
Configure change capture
1. Add the Extract group
Execute the following commands on the <source> DB2 system to create the Extract group.
GGSCI> ADD EXTRACT EDB2<unique id>, TRANLOG, BEGIN NOW
Verify the results:
GGSCI> INFO EXTRACT EDB2<unique id>
2. Create the Extract parameter file
Execute the following commands on the <source> system.
GGSCI> EDIT PARAM EDB2<unique id>
-- -- Change capture parameter file to capture -- TCUSTMER and TCUSTORD changes -- EXTRACT EDB2<unique id> SOURCEDB <database>, USERID <login>, PASSWORD <password> TRANLOGOPTIONS NOUSEREXIT RMTHOST <target>, MGRPORT <port> RMTTRAIL ./dirdat/<trail id> TABLE <owner/schema>.TCUSTMER; TABLE <owner/schema>.TCUSTORD;
Note: Record the two characters selected for your <trail id>: ______. You will need this in
the next step and when you set up the Replicat.
Close the parameter file and verify the results.
Configure Change Capture
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 18
GGSCI> VIEW PARAMS EDB2<unique id>
3. Define the GoldenGate trails
Execute the following on the <source> system.
GGSCI> ADD RMTTRAIL ./dirdat/<trail id>, EXTRACT EDB2<unique id>, MEGABYTES 5
Verify the results:
GGSCI> INFO RMTTRAIL *
4. Start the capture process
Execute the following on the <source> system.
GGSCI> START EXTRACT EDB2<unique id>
Verify the results:
GGSCI> INFO EXTRACT EDB2<unique id>, DETAIL GGSCI> VIEW REPORT EDB2<unique id>
Discussion Points
1. Identifying a remote system
What parameter is used to identify the remote target system?
_________________________________________________________________________
_________________________________________________________________________
2. Reading the transaction log
What parameter is used to set the options for reading the transaction log?
_________________________________________________________________________
What options is this lab using? What do they control?
_________________________________________________________________________
_________________________________________________________________________
Configure Change Delivery
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 19
Exercise 4.
Configure Change Delivery
Objective
The goals of this exercise are to:
���� Set up the checkpoint table on the target system.
���� Create a named group that includes the Replicat process and the checkpoint tables.
���� Configure the Replicat group by adding parameters.
���� Start the Replicat group.
Set up checkpoints
1. Edit a GLOBALS file on the target system
Execute the following commands on the <target> system.
� Edit the GLOBALS parameter file to add the checkpoint table.
Shell> cd <install location> Shell> ggsci GGSCI> EDIT PARAMS ./GLOBALS
In the text editor, type:
CHECKPOINTTABLE <owner/schema>.ggschkpt
� Record the checkpoint table owner and name, then save and close the file.
Table owner ____________________ name ___________________
Note: You could name the table anything you want, but for training purposes we are using ggschkpt.
2. Activate the GLOBALS parameters
For the GLOBALS configuration to take effect, you must exit the session in which the changes
were made. Execute the following command to exit GGSCI.
GGSCI> EXIT
Configure Change Delivery
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 20
3. Add a Replicat checkpoint table
Execute the following commands on the <target> system.
This step adds the checkpoint table that you specified when you created the GLOBALS
parameter file. It will be created in the <dsn> database.
� Run GGSCI on the target.
Shell> cd <install location> Shell> ggsci
� Execute the following commands in GGSCI.
GGSCI> DBLOGIN SOURCEDB <dsn>, USERID <login>, PASSWORD <password>
GGSCI> ADD CHECKPOINTTABLE
Configure delivery
4. Add the Replicat checkpoint group
Execute the following commands on the <target> system to create the Replicat group named
RMSS<unique id>.
GGSCI> ADD REPLICAT RMSS<unique id>, EXTTRAIL ./dirdat/<trail id>
Note: Refer to your Extract set up for the correct two-character <trail id>.
5. Create Replicat parameter file
� Execute the following command on the <target> system to edit the Replicat parameter file.
GGSCI> EDIT PARAM RMSS<unique id>
� Add the following lines to the parameter file.
REPLICAT RMSS<unique id> TARGETDB <dsn>, USERID <login>, PASSWORD <password> HANDLECOLLISIONS SOURCEDEFS ./dirdef/source.def DISCARDFILE ./dirrpt/RMSS<unique id>.DSC, PURGE MAP <owner/schema>.TCUSTMER, TARGET <owner/schema>.TCUSTMER; MAP <owner/schema>.TCUSTORD, TARGET <owner/schema>.TCUSTORD;
Note: Remember to use <db type>.def if you renamed the source.def when you transferred it
to the the <target>.
Configure Change Delivery
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 21
4. Start the Replicat process
Execute the following commands on the <target> system.
� Execute the following command in GGSCI to start the Replicat process.
GGSCI> START REPLICAT RMSS<unique id>
� Verify the results:
GGSCI> INFO REPLICAT RMSS<unique id>
Discussion points
Search in the Windows//UNIX Reference Guide for the information on the following
questions.
1. When to use HANDLECOLLISIONS
When would you use HANDLECOLLISIONS? What does it do?
_________________________________________________________________________
_________________________________________________________________________
2. What information is supplied by SOURCEDEFS?
_________________________________________________________________________
_________________________________________________________________________
3. What is the purpose of the DISCARDFILE?
_________________________________________________________________________
_________________________________________________________________________
Generate Activity and Verify Results
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 22
Exercise 5.
Generate Activity and Verify Results
� � � � � � � � � � � � � � �
Objective
The goals of this exercise are to:
���� Execute miscellaneous update, insert, and delete operations on the source system.
� Verify the delivery of the changes to the target.
� Turn off the error handling used for the initial load.
Generate database operations
1. Execute miscellaneous update, insert, and delete operations
Execute the following commands on the <source> system.
Note: It is advisable to update the demo_db2_misc.sql script to ensure that the database
connection information matches your system.
Shell> cd <install location> Shell> db2 +c –tf demo_db2_misc.sql
Verify change capture and delivery
2. Verify your results on the source system
Execute the following commands on the <source> system.
Shell> db2 db2> CONNECT TO <database> db2> select * from TCUSTMER db2> select * from TCUSTORD db2> quit
Shell> ggsci GGSCI> SEND EXTRACT EDB2<unique id>, REPORT GGSCI> VIEW REPORT EDB2<unique id>
Generate Activity and Verify Results
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 23
Verify your target SQL Server 2005 results
3. Verify your results on the SQL Server 2005 target
� Verify and record processing statistics on the <target> with the following command in GGSCI.
GGSCI> SEND REPLICAT RMSS<unique id>, REPORT GGSCI> VIEW REPORT RMSS<unique id>
� Verify the data by executing the following commands in the New Query window of SQL Server Management Studio on the source:
select * from TCUSTMER; select * from TCUSTORD;
Turn off error handling
4. Turn off initial load error handling for the running delivery process
Execute the following commands on the <target> system.
GGSCI> SEND REPLICAT RMSS<unique id>, NOHANDLECOLLISIONS
5. Remove initial load error handling from the parameter file
GGSCI> EDIT PARAMS RMSS<unique id>
Remove the HANDLECOLLISIONS parameter.
Exercise Name
� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � Page 24