Post on 10-May-2015
description
How many ways to monitor Oracle GoldenGate?
Bobby CurtisSr. Technical ConsultantEnkitec
About• Douglasville, Georgia• Senior Technical Consultant• IOUG, ODTUG, GOUSER, RMOUG• SELECT Journal Article Editor
• Expert OEM12c (2013)/Practical ODA (2014)
• Twitter: @dbasolved• Blog: http://dbasolved.com• Email: bcurtis@enkitec.com curtisbl@gmail.com
Home: www.enkitec.com/e4Registration: http://www.enkitec.com/e4/registerLocation: http://www.enkitec.com/e4/locationTraining Days Following E4: http://www.enkitec.com/e4/training-days
Agenda
• What is Oracle GoldenGate?• Use Cases• Monitoring• GGSCI• Command Line• SQL• GUI Driven
• Comprehensive software for real-time data integration in heterogeneous environments
• Benefits• High-Availabity Solutions• Real-Time Data Integration• Transactional Change Data Capture• Data Replication• Transformation of Data• Verification of Data
• Primary Replication Tool in Oracle Database 12c (12.1.0.1)• Oracle Upgrade Guide, Sections
8.1.6 & 8.1.7
What is Oracle GoldenGate?
Use Cases
Monitoring
• Five Different Approaches• Manual• Scripts Based• SQL Based• Oracle GoldenGate Director• Oracle Enteprise Manager 12c
Monitoring (Manual)
• Manual Monitoring• Simplest Approach• Monitoring parameters are used inside
of parameter files (Manager/Extract/Replicat)
• Uses GoldenGate Service Command Interface (GGSCI)
• GoldenGate Error Log (ggserr.log)
Monitoring (Manual)
Parameter Process Description
REPORT EXTRACT/REPLICAT Specify the interval at which EXTRACT or REPLICAT generates interim runtime statistics in a process report
REPORTCOUNT EXTRACT/REPLICAT Report a count of transaction records that EXTRACT or REPLICAT processed since startup
REPORTROLLOVER EXTRACT/REPLICAT Force report files to age on a regular schedule, instead of when a process starts
UPREPORTMINUTESUPREPORTHOURS
Manager Specify the frequency with which Manager reports, EXTRACT, and REPLICAT processes run
LAGREPORTMINUTESLAGREPORTHOURS
Manager Specify the interval at which Manager checks for EXTRACT and REPLICAT lag
LAGCRITICALSECONDSLAGCRITICALMINUTESLAGCRITICALHOURS
Manager Specify a lag threshold that is considered critical, and to force a warning message to the error log when the threshold is reached
LAGINFOSECONDSLAGINFOMINUTESLAGINFOHOURS
Manager Specify a basic lag threshold; if lag exceeds the specified value, Oracle GoldenGate reports lag information to the error log. If the lag exceeds the value specified with the LAGCRITICAL parameter, Manager reports the lag as critical; otherwise, it reports the lag as an informational message
Monitoring (Manual)
Command Process DescriptionINFO MANAGER STATUS MANAGER
Manager Determine whether or not the Manager process is running. If Manager is running, the port number is displayed. This command is an alias for STATUS MANAGER
SEND MANAGER Manager Retrieve the status of the active Manager process or to retrieve dynamic port information as configured in the Manager parameter file
INFO EXTRACT EXTRACT The status of EXTRACT (STARTING, RUNNING, STOPPED or ABENDED). LAG EXTRACT EXTRACT Determine a true lag time between EXTRACT and the data source. LAG EXTRACT calculates
the lag time more precisely than INFO EXTRACTbecause it communicates with EXTRACT directly, rather than reading a checkpoint position in the trail
SEND EXTRACT EXTRACT Communicate with a running EXTRACT process. The request is processed as soon as EXTRACT is ready to accept commands from users.
STATUS EXTRACT EXTRACT Determine whether or not EXTRACT is runningINFO REPLICAT REPLICAT Retrieve the processing history of a REPLICAT group. The output of this command includes:
The status of REPLICAT (STARTING, RUNNING, STOPPED or ABENDED). STARTING means that the process has started but has not yet locked the checkpoint file for processing.
(Oracle database) The REPLICAT mode: non-integrated or integrated.
Whether or not REPLICAT is in coordinated mode and, if so, how many threads it currently uses.
Approximate REPLICAT lag.
The trail from which REPLICAT is reading.
Etc …
The basic command displays information only for online (continuous) REPLICAT groups. Tasks are excluded.
LAG REPLICAT REPLICAT Determine a true lag time between REPLICAT and the trail. LAG REPLICAT estimates the lag time more precisely than INFO REPLICAT because it communicates with REPLICAT directly rather than reading a checkpoint position
SEND REPLICAT REPLICAT Communicate with a starting or running REPLICAT process. The request is processed as soon as REPLICAT is ready to accept commands from users
STATUS REPLICAT REPLICAT Determine whether or not REPLICAT is running
Monitoring (Scripts)
• Script Based• Using manual commands in scripts• Any script type• Shell• Perl• Etc…
• Write once, Run multiple times• Repeatable process with no
memorization• Portable between environments
Monitoring (Scripts)
• Info_All.sh
#!/bin/sh#info_all.shexport GGS_HOME=/opt/oracle/product/ggate $GGS_HOME/ggsci <<EOFinfo allexitEOF
Monitoring (Scripts)• gg_mem_mon.sh
PROCESSES="mgr ggcmd extract replicat"for GGPROCESS in $PROCESSESdo FLAG=`ps -ef | grep $GGPROCESS` if [ $_ ] then "No GoldenGate Process Found" else ps -C $GGPROCESS -O rss | awk '{print $2/1024, "MB", $12}' | sort -k 2 fidoneexit 0
Monitoring (SQL)
• Ability to monitor from SQL*Plus or SQL Developer
• Hybrid in nature (Scripts/SQL)• Can only see what is going on currently• Moving parts• Script to write text file• Need to schedule run
• Use of External Tables
Monitoring (SQL)• gg_monitor_sqldev.pl
.[Code Removed]open (GGPROC, ">$outfile") or die "Unable to open file";foreach (@buf){ if(/MANAGER/||/JAGENT/||/EXTRACT/||/REPLICAT/)
{no warnings 'uninitialized'; chomp;my ($program, $status, $group, $lagatchkpt, $timesincechkpt) = split(" ");
if ($group eq "") {$group = $program;
} if ($lagatchkpt eq "" || $timesincechkpt eq "") {$lagatchkpt = "00:00:00";
$timesincechkpt = "00:00:00";}
print GGPROC "$program|$status|$group|$lagatchkpt|$timesincechkpt\n"; }}close (GGPROC);
Monitoring (SQL)
• Golden Gate specific tables• 11.2.0.3 (few)• 11.2.0.4 (little more)• 12.1.0.1 (moving
forward)• 50 Objects (to
date)
Monitoring (Director)
• Multi-tier, client-server application• Can manage GoldenGate from a remote
client• Many moving parts• See Architecture (Next Slide)• Client – GUI interface for users• Mid-Tier Server – Weblogic,
Application & Monitor agent• Central Repository – i.e. Database
Monitoring (Director)
Monitoring (OEM12c)
• Quickly becoming monitoring tool of choice
• GoldenGate monitoring available since 12.1.0.1
• Support via a plug-in• JAgent is used to interact with OEM
Agent/Plug-In • Enabled via GLOBALS file• ENABLEMONITORING
parameter
Monitoring (OEM12c)
• GoldenGate Plug-in Installed• Oracle Management Server• Oracle Agent• Current Release (12.1.0.1)
• Ensure update of plug-in• Setup -> Extensibility -> Self Update
Monitoring (OEM12c)
• GoldenGate Access• Deployed on
OMS/Agent• JAgent started• Targets ->
GoldenGate
Monitoring (OEM12c)
Image provided by Maaz Anjum (here)
Monitoring (OEM12c)
• Hybrid Approach• Metric Extensions
• Metric Extensions• Uses Scripts/More Texted based• Can be deployed to any agent• Does not have up/down issues with
JAgent/Agent interaction
Monitoring (OEM12c)• gg_metric_ext_am.pl
#!/usr/bin/perl -w##use strict;use warnings; #Static Variables my $gghome = "/oracle/app/product/12.1.2/ggate"; #Program my @buf = `$gghome/ggsci << EOFinfo allEOF`; foreach (@buf){ if(/EXTRACT/||/REPLICAT/) { s/\s+/\|/g; print $_."\n"; }}
Monitoring (OEM12c)
• Metric Extension• Enterprise -> Monitoring -> Metric
Extensions• Metric seen under All Metrics for
Hosts (yellow star)• Packaged and deployed to multiple
agents
Monitoring (OEM12c)
• Metric Extension -> All Metrics
Monitoring (Bugs)
Topic Bug Resolution
Manual Bugs are associated with application software MOS Note ID: 1298817.1
Scripts Often occur due to user errors within the scripts Normal code reviews would minimize error in scripts
Oracle GoldenGate Director A few bugs! Bugs have been documented in MOS MOS Note ID: 1298817.1
Oracle Enterprise Manager 12c Few bugs have been identified. Oracle recommends using the plug-in to monitoring Oracle GoldenGate 11.2.0.1.10 or later
MOS Note ID: 1495998.1 (Berkley Database Data Store Bug)
No Monitoring Solution is 100%!Choose wisely for your data!
Monitoring (SQL Developer)
• Community Project (CGGM)• Extension Based• Hybrid approach• Perl/Shell• SQL• XML/Java
• Open Source• GitHub (underway)
Monitoring (SQL Developer)
• Navigation• Minor SQL
• Editor • SQL Driven
Script Repo Location: http://tinyurl.com/kscsdnw
Script Repo
About• Douglasville, Georgia• Senior Technical Consultant• IOUG, ODTUG, GOUSER, RMOUG• Expert OEM12c (2013)/Practical
ODA (2014)• Twitter: @dbasolved• Blog: http://dbasolved.com• Email: bcurtis@enkitec.com curtisbl@gmail.com