Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases...

50
Smart Plug-In for MySQL Databases for HP Operations Manager for Windows (OMW) Installation and Configuration Guide Revision: September 2016

Transcript of Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases...

Page 1: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

for HP Operations Manager for Windows (OMW)

Installation and Configuration Guide

Revision: September 2016

Page 2: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

2

Legal Notices

The information contained in this document is subject to change without notice.

Warranty

Bull GmbH makes no warranty of any kind with regard to this document, includ-

ing, but not limited to, the implied warranties of merchantability and fitness for a

particular purpose. Bull GmbH shall not be held liable for errors contained herein

or direct, indirect, special, incidental or consequential damages in connection

with the furnishing, performance, or use of this material whether based on war-

ranty, contract, or other legal theory.

Copyright Notices

© Copyright 2016 Bull GmbH. This document contains proprietary information

and is protected by copyright. All rights are reserved. No part of this document

may be copied, reproduced, translated to another language, transferred to elec-

tronic media, or transferred into machine readable format without the prior writ-

ten consent of Bull GmbH.

Trademark Notices

HP® and OpenView® are registered trademarks of the Hewlett-Packard Enterprise

in the United States and other jurisdictions.

Java™ is a trademark of Oracle Co.

Linux is a U.S. registered trademark of Linus Torvalds.

Microsoft®, Windows®, Windows® 2000, Windows Server™ 2003, Windows Server

2008, Windows Server 2012, Windows Vista and Windows NT® are either regis-

tered trademarks or trademarks of Microsoft Corporation in the United States

and/or other countries.

MySQL® is a registered trademark of MySQL AB, Sun Microsystems and Oracle in

the United States, the European Union and other countries. MySQL® Network™ is

a trademark of MySQL AB and Oracle.

UNIX is a registered trademark of the Open Group.

Sun™ and Solaris™ are trademarks of Sun Microsystems, Inc and Oracle.

All other product names are the property of their respective trademark or service

mark holders and are hereby acknowledged.

Page 3: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

3

Installation and Configuration

Written and developed by Bull GmbH, Germany.

Contact Information

Bull GmbH

Von-der-Wettern-Str. 27

D-51149 Köln

Germany

Phone: +49 (0)-211-399-415-30

E-Mail: [email protected]

Websites:

Atos: http://www.atos.net

Bull DataCenter Products: http://www.bes-bull.com

Bull: http://www.bull.com

Page 4: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

4

Table of Contents

1 Introduction... ............................................................................5

Product Documentation............................................................................. 5

About the Document ................................................................................. 6Conventions ...........................................................................................................6

HP Software Product Naming ................................................................................6

Document Overview ..............................................................................................7

2 Installation Requirements ..........................................................8

Before Installing the Smart Plug-In for MySQL Databases.........................8

Management Server Requirements........................................................... 8

Prerequisites on the MySQL Database Servers........................................10

3 Installation ...............................................................................12

Directory Layout.......................................................................................12

Installation on OVO/W 7.5 or OMW using Windows Clusters .................14

Reinstallation of the SPI for MySQL Databases........................................15

Upgrade from Version 1.x, 2.x .................................................................15

De-Installation..........................................................................................17

4 Configuration ...........................................................................18

Overview ..................................................................................................18

Managing MySQL Database Servers ........................................................18Step 1: Check the prerequisites ...................................................................18

Step 2: Deploy the instrumentation.............................................................19

Step 3: Generate the License Request and obtain the Licenses ..................20

Step 4: Configure the SPI..............................................................................23

Step 5: Deploy the Policies...........................................................................25

Customize the SPI for MySQL Databases .................................................27Tools.....................................................................................................................27

Tools in MySQL SPI Admin ................................................................................................ 28

Tools in MySQL SPI Licensing ............................................................................................ 30

Tools in MySQL SPI Tools .................................................................................................. 32

Monitors ..............................................................................................................33

Policies .................................................................................................................34

The Policy Group Mandatory ............................................................................................ 35

The Policy Group Event ..................................................................................................... 36

The Policy Group Performance ......................................................................................... 38

The Policy Group Custom.................................................................................................. 39

Count Monitors............................................................................................................... 39

Business Process and Custom Query Monitors .............................................................. 40

Replication Monitoring ................................................................................................... 41

5 Appendix..................................................................................44

Capacity Planning related Metrics ...........................................................44

Policies .....................................................................................................44

Metric Classes ..........................................................................................49

Page 5: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

5

Installation and Configuration

Introduction...

... to the Bull Smart Plug-In for MySQL Databases for HP Operations Manager.

The Bull Smart Plug-in (SPI) for MySQL Databases enables professional monitoring

and problem detection for the MySQL database instances. The SPI adds MySQL

database monitoring and management capabilities to the HP OpenView Opera-

tions/HP Operations Manager products.

With MySQL databases used more and more in productive environments, it

becomes a key requirement to effectively manage and monitor these databases.

The SPI for MySQL Databases integrates its performance and event detection fea-

tures with HP Operations Manager for Unix and the HP Operations Manager for

Windows. It can help to optimize business processes in the IT department, to pro-

actively prevent problem situations, to pin-point problems and even to identify

future performance behaviors based on the historical performance data.

The Smart Plug-In for MySQL Databases is designed to monitor the state and

throughput of the database instances and physical resources related to the data-

base such as buffers, disks and memory. Additional metrics support the capacity

planning process. The log file analyzer tracks any messages reported by the data-

base system and generates events for the OVO/OM operators. Performance mon-

itors are used for state monitoring (thresholds) and data collection that can be

afterwards used by reporting services for historical and trend analysis. Configura-

tion changes are tracked and Security relevant occurances are reported.

The SPI is easy to configure and maintain. To automate the necessary configura-

tion steps as far as possible, the SPI will discover most of the configuration set-

tings automatically just by having the access parameters to the database instance

specified.

Product Documentation

In this document you will find introductory information for the integration of

MySQL database environments into HP Operations Manager/Windows using the

SPI for MySQL Databases. The installation steps, configuration tasks, and concepts

that are necessary to monitor such environments with OMW are covered.

The SPI for MySQL Databases product documentation consist of this PDF file. To

get in depth information about the concepts and functionality of MySQL database

and HP Operations Manager the according product documentation is recom-

mended as further reading

• MySQL Reference Documentation (http://dev.mysql.com/doc)

• HP Operations Manager for Windows Online Help

Page 6: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

6

About the Document

Conventions

The following typographical conventions are used in this manual:

• Emphasis are printed in italic font.

• Commands, selectable menu options, and Variables are printed in bold font.

• Text and other items on the computer, listings or path specifications are

printed in Computer font.

• Text that you enter is printed in Computer bold font.

HP Software Product Naming

With the integration of the former Mercury product line (BAC, SiteScope, Diagno-

sis, etc) and Peregrine ServiceCenter the OpenView product names were

changed. The documentation uses the new product names, bur for the sake of

clarity find the according old names listed below.

Table 1: Product Name Mapping

Old Product Name New Product Name

HP OpenView Operations for Unix

(OVO/U)

HP Operations Manager for Unix

(OMU)

HP OpenView Operations for Win-

dows (OVO/W)

HP Operations Manager for Windows

(OMW)

HP OpenView Performance Mana-

ger

HP Performance Manager

HP OpenView Smart Plug-Ins HP Operations Smart Plug-Ins

Page 7: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

7

Installation and Configuration

Document Overview

Chapter 1 „Installation Requirements“ on page 8 explains the prerequisites and

configuration steps that should be performed before the installation of the SPI for

MySQL Databases.

Chapter 2 „Installation“ on page 12 explains the installation of the SPI for MySQL

Databases.

Chapter 3 „Configuration“ on page 18 guides through the configuration steps

after the initial installation and explains the areas that can be customized with

the appropriate HP Operations Manager methods.

Page 8: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

8

Installation Requirements

Before Installing the Smart Plug-In for MySQL

DatabasesBefore starting the installation be sure to verify that:

• the OM/Windows Server is up and running

• the OM/Windows Agent is installed on all MySQL Database Servers that

should be monitored

• the OM/Windows Agent can communicate without problems with the Man-

agement Server

Management Server Requirements

The Smart Plug-In for MySQL Databases requires the software versions and disk

space listed in the tables below. The specified Patches are superseded in the

meantime by new cumulative patches and therefore should be used as bare mini-

mum patch level information. The list of actual available patches needs to be ver-

ified at installation time with the HP OpenView support site at https://

softwaresupport.hpe.com.

Table 2: Software Prerequisites - Server

Product Patch Level

Windows XP, 2003, 2008,

2012

Service Packs as specified for OMW

Make sure that the MySQL SPI Installation pak-

kage matches the 32 or 64bit version of the

Operating System and OMW Installation, other-

wise the installation will fail.

Operations Manager

Windows 8.1/8.5 Server

and Agents

The Agents must have at least patch level

A.8.52.6

Operations Manager

Windows 9.0 Server and

Agents

The server must have at least patch level

OMW_00124

Page 9: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

9

Installation and Configuration

The OVO/Windows utility vpstat.exe can be used to verify the installed patches

Java runtime 1.5, 1.6,

1.7, 1.8

The JRE 1.5.0_14, 1.6.0_4, 1.7.0_5 or later can

be used. JAVA_HOME should be set.

With OMW 9 the java runtime coming with the

OMW server installation can be used, but the

path needs to be added to the System Environ-

ments PATH variable (e.g. C:\Program

Files\HP\HP BTO Software\nonOV\jre\b\bin)

Table 3: Disk Space

Directory Required Space

<OVInstall Dir>\bin 1 MB

<OVInstall Dir>\install 8MB

<OVShare Dir>a\mysqlspi

a. This location is for OVO/W 7.21 the same as <OVInstall

Dir>/shared

1 MB

<OVShare Dir>b\instrumentation

b. This location is for OVO/W 7.21 the same as <OVInstall

Dir>/shared

8 MB

Table 2: Software Prerequisites - Server

Product Patch Level

Page 10: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

10

Prerequisites on the MySQL Database Servers

In addition to the disk space requirement of the OMW Agent, the instrumenta-

Table 4: Software Prerequisites- Agent

Product Patch Level

HP-UX 11i

Novell OES / Linux

RedHat Enterprise Linux 4,

5, 6, 7

Solaris 9, 10, 11

Suse Linux 8, 9, 10, 11, 12

Suse Linux Enterprise Ser-

ver 8, 9, 10, 11, 12

Windows XP SP2, Vista

Windows 2003 Server

Windows 2008 Server

Windows 2012 Server

The Service Packs and Updates as specified for

OMW must be installed.

The supported platforms and versions are

restricted according to the OMW Agent sup-

port matrix.

(other OMW supported Linux distributions will

work, but might require to manually copy the

instrumentation files into the according plat-

form directory on the server)

Starting with OMW 8.5 the SPI uses category

based deployment packages.

Java runtime 1.5, 1.6, 1.7,

1.8

The JRE 1.5.0_14 or 1.6.0_4 or later can be

used. JAVA_HOME should be set.

Perl 5.6 Perl 5.6 or later can be used and must be confi-

gured in the search PATH. This Perl version

usually comes with the OVO agent.

OVO/Windows Agents

OMW/Agents

The Agents must have at least version A.7.33,

A.8.52.6, A.11.x and the default instrumenta-

tion must be deployed (i.e. SPI DDF Tools from

the SPI Data Collector Instrumentation)

MySQL Database It is strongly recommended to use a supported

MySQL Network or Enterprise version in pro-

ductive environments. Version 4.1 and 5.x with

the latest Updates are supported with the SPI.

Binary compatible versions of MariaDB are also

working.

Page 11: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

11

Installation and Configuration

tion for the MySQL Databases requires space for the executables and

processing.

The OMW Agent version on each MySQL Database server must match the patch

level described in „Software Prerequisites - Server“ on page 8.

Before the configuration of the SPI is started, the MySQL instance that should be

monitored has to be started. A user account with access rights for localhost con-

nections and at least Select privileges on the internal mysql database must exist.

If a Firewall or Worm Protection Tool is used, it must be configured to allow

access to the MySQL Ports and the SPI’s data collector port (4611) for the local

host.

Table 5: Disk Space on the MySQL Database Server

Directory Required Space

%OvAgent-

Dir%a\bin\instrumentation

a. OvAgentDir corresponds by default to

%ProgramFiles%\HP OpenView\Installed Packages\

{790c06b4-844e-11d2-972b-080009ef8c2a}

2 MB

%OvAgentDir%\mysqlspi 5 MB for Windows Systems only

(dependent on the number of data-

base instances to be monitored)

/opt/OV/mysqlspi 5MB for Unix Systems only (depen-

dent on the number of database

instances to be monitored)

%OvAgentDir%b\databases

b. Using the build-in OV Performance component. If the

separate Performance Agent (MeasureWare) is used,

the data resides in the default location for the log data.

20 MB for Coda Performance data

files

Page 12: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

12

Installation

The installation of the SPI for MySQL Databases 4 will add new items to the con-

figuration of the OMW Management Server. It is strongly recommended to

backup the OMW configuration and database before the installation is started.

Refer to the Online Help of OMW (keyword Backup) for instructions to perform

the backup.

The SPI for MySQL Databases 4 comes as MSI package. The installation executes

VBScripts to add configuration information to the OMW Management Server. If

an Antivirus Tool is installed on the Server, disable any Script blocking functional-

ity before the installation is started.

It is recommended that there is no OM user session running during the instal-

lation. If an OM user is logged on, he needs to reload the configuration after

the installation in order to see the newly added items.

The package installation must be performed on the Management Server by a user

which has administrative rights on the local system and is a member of the OVO/

Windows administrative group HP-OVE-ADMINS.

Note: The Administrator account on the Management Server has administra-

tive rights for OMW and can perform the installation.

The installation is started by a simple double click on the MSI file. After placing

the files in the below described locations, the installation will automatically

upload the configuration into the Management Server.

Directory Layout

The SPI for MySQL Databases 4 files are placed in the following directories:

Table 6: Directory Layout

Directory Contents

<OVInstall Dir>\install\

SPI_for_MySQLDB

MySQL DB SPI Policies

Page 13: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

13

Installation and Configuration

<OVShare Dir>a\mysqlspi MySQL DB SPI files used by ser-

ver based licensing Tools

<OVShare Dir>\Instrumenta-

tion\HPUX\B11.<*>\SPI for

MySQLDB

MySQLDB SPI Instrumentation

for Unix Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\Debian 4.X\SPI

for MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\Miracle v4.0\SPI

for MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\Novell OES\SPI

for MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\Red Hat<8,9>\SPI

for MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\Red Hat

EL<3.0,4.0,5.0>\SPI for

MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\SuSE

[Server]<8,9,10>\SPI for

MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\LINUX\Turbo 8.X\SPI for

MySQLDB

MySQLDB SPI Instrumentation

for Linux Agents

<OVShare Dir>\Instrumenta-

tion\Solaris\<9,10>\SPI for

MySQLDB

MySQLDB SPI Instrumentation

for Unix Agents

Table 6: Directory Layout

Directory Contents

Page 14: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

14

Installation on OVO/W 7.5 or OMW using Windows

Clusters

The SPI for MySQL Databases 4 supports environments in which the OVO/Win-

dows server is running on a Windows Cluster. To install the SPI on a clustered

server make sure that the OVO/W server package is running on the first cluster

node. Install the package on this node. The package performs all necessary con-

figurations to work on all cluster nodes. Therefore the package must not be

installed on any further cluster node, otherwise unexpected problems might

occur when the package gets de-installed or upgraded.

<OVShare Dir>\Instrumenta-

tion\Windows <Server

2003\5.2, Server 2008\6.0,

Vista\6.0, XP\5.1>\SPI for

MySQLDB

MySQL DB SPI Instrumentation

for Windows Agents

For OMW 8 and 9:

<OVShare Dir>\Instrumenta-

tion\Categories\SPI for MySQLDB\Win-

dows

MySQLDB SPI Instrumentation

for Windows Agents

For OMW 8 and 9:

<OVShare Dir>\Instrumenta-

tion\Categories\SPI for MySQLDB\UNIX

MySQLDB SPI Instrumentation

for Unix Agents except Linux

For OMW 8 and 9:

<OVShare Dir>\Instrumenta-

tion\Categories\SPI for MyS-

QLDB\UNIX\LINUX

MySQLDB SPI Instrumentation

for Linux Agents

a. The OVShare Dir is the shared installation directory spe-

cified for OVOW 7.5 Server installations in Cluster Envi-

ronments. For OVOW 7.21 this is the same as the

<OVInstall> directory.

Table 6: Directory Layout

Directory Contents

Page 15: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

15

Installation and Configuration

Reinstallation of the SPI for MySQL Databases

In order to cleanly re-install the SPI for MySQLDB it is necessary to cleanup some

of the OVO/W data manually. OVO/W does not support to remove of the data by

the installation itself. Therefore following steps must be performed manually

before the installation of the new package is started:

1. Undeploy all MySQLDB Policies from any Node

2. Remove all versions of the MySQLDB Policies from the Server

3. Remove the MySQLDB Tools from the Console.

After these steps are completed, re-install the MSI package.

Upgrade from Version 2.x, 3.x

It is strongly recommended to backup the OVO data before the configuration

is changed or the installation is started.

Page 16: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

16

Before installing the SPI for MySQL Databases 4 review the existing policies if

there are any changes that need to be kept. In case there are any custom changes

that should be kept, note down the policy name or change.

Make sure that you have received and installed the Version 4 License Keys.

The License Keys of previous versions will be no longer accepted.

Perform these steps if you upgraded from OVO/Windows 7.x to 8.x with the

SPI 1.x installed

In this special case the upgrade from 7.x to 8.x might have migrated the

instrumentation to the new folder structure of Operations Manager 8.x. The

new version of the SPI uses the simplified folder structure of OMW 8, which

does not match with the migrated structures. To avoid conflicts please first

deinstall the old version of the SPI. Then locate the migrated instrumentation

folders on the OMW server. These are located in

%OvShareDir%\Instrumentation\Categories\SPI for

MySQLDB

Move the folder ’SPI for MySQLDB’ to some temporary location in case you

have custom query files in there, that you want to copy later back to the new

instrumentation packages. Otherwise simply delete the folder and all subfold-

ers.

Close the OVO Consoles and install the package.

The installation will upload it’s policies as version 3.0 to avoid conflicts with any

custom changes that should have a version of 2.x. The recommended upgrade

procedure is to redo the changes in the new policy version.

Since most of the old MySQL SPI policies will work without change with the new

instrumentation, another (but not recommended way) will be to delete the new

3.0 version of the according policy.

Attention when upgrading from version 2.x: Do not delete the version 3.0 of the

following policies: MySQLSPI_Messages, MySQLSPI_InnoDBRead,

MySQLSPI_InnoDBSize, MySQLSPI_InnoDBWrite, MySQLSPI_MyISAMDBSize!

Redistribute the instrumentation to all nodes running the SPI for MySQL as

described in „Step 2: Deploy the instrumentation“ on page 19 and then redistrib-

ute the latest policies to these nodes.

Page 17: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

17

Installation and Configuration

De-Installation

Before de-installing the SPI for MySQL Databases 4 from the OMW Management

Server system, the configuration must be removed from OM.

It is strongly recommended to backup the OM data before the configuration

is changed.

Since OM does not know which configuration belongs to a specific SPI, some

steps need to be performed manually.

1. Undeploy all MySQLDB Policies from any Node

2. Remove all versions of the MySQLDB Policies from the Server

3. Remove the MySQLDB Tools from the Console.

The package de-installation must be performed on the Management Server by a

user which has administrative rights on the local system and is a member of the

OVO/Windows administrative group HP-OVE-ADMINS.

In an clustered OMW installation, make sure that the OMW server package is run-

ning on the cluster node on which the installation was done and perform the de-

installation on this node. This is normally the first cluster node, on which the

OMW server was installed.

Note: The Administrator account on the Management Server has administra-

tive rights for OMW and can perform the de-installation.

On the Management Server select Add/Remove Software... from the Windows

Control Panel. Select the entry MySQLDB-SPI and press the Remove button. The

uninstaller will remove the integration files.

Page 18: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

18

Configuration

Overview

The installation adds specific entries for the SPI for MySQL Databases to the HP

Operations Manager configuration areas listed below.

• Instrumentation

The programs, scripts, and monitors used to instrument the MySQL Database

for the OVO Agent

• Policies

The configuration information for the monitors

• Tools

Tools to perform the configuration tasks for the SPI for MySQL Databases

Tools for the operational monitoring of the MySQL Databases

These entries will be described in detail in the following chapters.

Managing MySQL Database Servers

To monitor MySQL Database servers with the SPI for MySQL Databases some

preparation on the server systems is required. Please refer also to „Prerequisites

on the MySQL Database Servers“ on page 10.

Start the OM Console as user with administrative rights to perform the configura-

tion steps outlined in the next paragraphs.

For easier management and configuration put the MySQL Database servers into a

separate Node Group ’MySQL DB Servers’.

Step 1: Check the prerequisites

First check that the OM Agent and MySQL Database versions match the prerequi-

sites.

Ensure that JAVA_HOME is set. If for some reason JAVA_HOME can not be

pointed towards the correct Java version, the Environment variable

MYSQLSPI_JAVA_HOME can be used to specify the location of the Java version

that the SPI should use.

Page 19: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

19

Installation and Configuration

The perl executable must be in the Agents search path. The perl version installed

by the OM Agent can be used.

For Agents on the Windows platform be aware that changes to the environ-

ment variables might require a reboot of the system. The Windows Service

Control Manager that spawns the OM Agent process does not recognize such

environment changes.

Make sure that the MySQL Database server instances are started and their ports

are locally accessible.

The SPI uses a background process (Data Engine) that handles all data collection

and calculation tasks. This Data Engine requires a locally accessible TCP port for

internal communication. By default it will use port 4611. If this port is in use, the

Engine will search the next higher ports, until a free port is detected. Any used

firewall or worm protection tool must be configured to allow local access to these

ports.

An user account with at least Select access to the mysql database and all data-

base/table combinations that will be used by the monitors must exist in each

MySQL Server instance. If custom monitors or the Synchronization Monitor will

be used, the user account must have full access permissions to the according

tables.

Step 2: Deploy the instrumentation

The installation of the SPI for MySQL Databases added the SPI’s instrumentation

as separate package to the instrumentation folders of OMW. The instrumentation

must be deployed to each MySQL Database Server that should be monitored.

Page 20: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

20

If the MySQL Database Servers are contained in a separate Node Group right click

on this Group and select ’All Tasks->Deploy instrumentation’ from the popup

menu. If there is no separate Node Group, this step needs to be performed for

each system.

In the Dialog select the entry ’SPI for MySQLDB’ and the ’SPI DataCollector’, then

press OK. The progress of the deployment can be monitored by selecting Deploy-

ment Jobs in the Policy Management folder in the Navigation tree. In the

Deployment Jobs Window on the right side any pending or failed deployments

will be listed.

Step 3: Generate the License Request and obtain the

Licenses

The SPI requires to have a valid license installed before the monitoring can be

activated. The licenses have to be generated specifically for each MySQL Data-

base server system. The following steps show how to install a permanent or test

license on each system:

1. Generate the license request file for a server

Page 21: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

21

Installation and Configuration

In the Navigation pane select Tools->SPI for MySQL DB->MySQL SPI

Licensing->Generate License Request.

In the Dialog select all systems for which new licenses must be created and

Launch... the tool.

2. The tool will create on each selected system the license request file

<OVAgentDir>/mysqlspi/tmp/<hostname>.req.

This file contains some information that is necessary to create a valid license

(i.e. hostname as it is discovered by the SPI, SPI version, platform).

3. Evaluation licenses can be requested per email from

[email protected].

Send the generated files along with the required number of instances.

Instances refer to the number of MySQL Servers on the system, each instance

can serve multiple databases and multiple instances can run concurrently on a

single system. The category of instances is no longer required for the license

generation. It describes the use and size of a database instance. The size is

determined by the number of databases running in this instance, the number

of concurrent users and the number of queries that are performed.

4. After the license request was approved, the license file will be send to the

registered E-mail address. This license file must be placed on the OVO server.

The tools will by default use the following location and filename:

%PROGRAM FILES%/HP OpenView/data/shared/mysqlspi/

Page 22: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

22

new.lic.

A different filename can be used, but requires to change the filename in the

next step.

5. To update the license list with the latest retrieved license file, the new or

changed license needs to be merged with the existing global license file. Run

the tool Tools->SPI for MySQL DB->MySQL SPI Licensing->Update

Licenses.

If the default location and name of the new license file was used press

Launch.... Otherwise update the path and filename in the parameters field

before launching the tool.

6. After all new license files are added, the updated license file needs to be acti-

vated. Run the tool Tools->SPI for MySQL DB->MySQL SPI Licensing-

>Activate Update Licenses. This step will copy the updated global license to

the SPI’s instrumentation folders for distribution.

7. Redistribute the SPI’s instrumentation to the managed servers to update the

license file on these systems. The detailed steps are described in „Step 2:

Deploy the instrumentation“ on page 19.

Page 23: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

23

Installation and Configuration

Step 4: Configure the SPI

The SPI’s Data Engine is performing most of the configuration steps itself, but it

must be informed about the database instances that should be monitored.

Before deploying the SPI’s policies the MySQL database instances and connection

information should be configured on each server. If the SPI does not already col-

lect data from a MySQL database instance on the system, the SPI’s Data Engine

must be started to be able to process the configuration information. This step is

not necessary if instances should be added to a server that is already monitoring

any other instance as the Data Engine will be already running. To startup the

Engine for the initial configuration run the tool

Tools->SPI for MySQL DB->MySQL SPI Admin->Start Data Engine.

Page 24: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

24

The Data Engine will generate a warning message that it’s configuration does not

contain any instances to monitor. After the Engine is started, use the tools

Tools->SPI for MySQL DB->MySQL SPI Admin->Add/Delete/List MySQL

DB Instance until all database instances are configured correctly.

The Add MySQL DB Instance Tool requires the following parameters:

• <name>: Specify a descriptive name that is unique on that server and allows

to easily identify the database instance. This name will be used in the moni-

tors and messages as object name. The name can be up to 254 characters in

length and should not contain characters with special meaning like

whitespaces, quotes or non printable characters

• <port>: Place the port number that the MySQL database instance is using

• <user>: Specify the MySQL user account that the SPI will use to monitor this

database instance. The user account must have with access rights for local-

host connections and at least Select privileges on the internal mysql database.

If custom monitors are used, the account must have full access rights to the

according tables.

• <password>:Specify the password for the user account. The password will be

stored in encrypted format in the SPI configuration files.

Page 25: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

25

Installation and Configuration

The Engine will check the entered values and discover all relevant other informa-

tion that is necessary to monitor the database instance. The tool will display a

success message if the database instance was added to the list of monitored

instances. If there is an error in the specified connection data or the instance is

not running, the Tool will display an error notification and reject to add the speci-

fied instance. To verify if all instances are configured successfully run the tool List

MySQL DB Instance. The tool displays the list of all instances that the SPI will

monitor.

The Engine will automatically generate and activate the necessary configuration

for the integration with the installed OpenView Performance Component or

Agent. The data collection of the performance data will be started for each con-

figured instance independently from the policy based threshold monitoring.

Step 5: Deploy the Policies

In the navigation tree open the Policy Groups->SPI for MySQLDB. The poli-

cies are separated into four groups.

The group Mandatory must be always deployed to a monitored MySQL database

server. It contains special policies that are responsible for startup and monitoring

of the SPI’s Data Engine and processing of the SPI’s messages.

Page 26: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

26

The Performance Policy group contains all threshold monitoring policies for per-

formance values. The metric used in these policies are also forwarded to the Per-

formance agent for online analysis and history reporting.

Policies that monitor the MySQL database for suspicious states and unusual

behaviour are contained in the Event policy group.

The policies in the group CustomMonitors must be customized specifically to the

environment or require specific preparations in the MySQL databases. Policies

from this group should be never assigned and distributed before these customiza-

tions are done.

Details about the customizations are covered in chapter „Customize the SPI for

MySQL Databases“ on page 27.

To deploy the policies select the policy group or single policies and press the right

mouse button. In the popup menu select All Tasks->Deploy on...

In the Dialog select the servers that should be managed or an according Node

Group and press OK.

Page 27: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

27

Installation and Configuration

The progress of the deployment can be monitored by selecting Deployment Jobs

in the Policy management folder in the Navigation tree. In the Deployment Jobs

Window on the right side any pending or failed deployments will be listed.

After the policies are deployed, the SPI for MySQL will monitor all configured

MySQL database instances.

Customize the SPI for MySQL Databases

The SPI for MySQL Databases extends the available Instrumentation, Tools, and

Policies of HP Operations Manager by according items to monitor and manage

MySQL database instances. These items can be customized according to the spe-

cific needs with the standard OM mechanisms. The details are listed in the follow-

ing chapters.

Tools

The tools of the SPI for MySQL Databases are organized into three Tool Groups

contained in the Group ’SPI for MySQLDB’:

Page 28: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

28

Tools in MySQL SPI Admin

The Tool Group ’MySQL SPI Admin’ contains tools to configure and manage the

MySQL Database integration as well as some support tools.

• MySQL SPI Support Group

This group contains tools that will be used in trouble shooting situations. The

tools allow to change the log level of the SPI and to show the actual environ-

ment setting of the SPI.

• Engine Access Add/Delete/List

The SPI uses a background process (Data Engine) that handles all data collec-

tion and calculation tasks. This Engine requires a locally accessible TCP port for

internal communication. By default it will use port 4611. In some environ-

ments it might make sense to allow remote connections to the Engine. The

Engine uses an internal ’hosts allowed’ list. Only hosts that are configured in

this list are allowed to communicate with the Engine. The ’Engine Access’

tools are used to maintain the contents of this access control list. The local

host is always allowed to access the Engine and can not be removed.

• Config Add/Delete/List

These tools are used to configure the monitored MySQL instances on each

system. The use of the tools is described in the chapter „Step 4: Configure the

SPI“ on page 23.

• Create Replication Master Table

This tool is used to create a special table which will be used if the replication

Page 29: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

29

Installation and Configuration

of MySQL databases has to be monitored. Refer to the paragraph about the

Custom Replication Monitor for more details.

• MySQL SPI client

This generic tool gives access to the Engine process of the MySQL SPI. It can be

used to query a variety of different informations from MySQL or to execute

SPI specific commands. The available functionality and parameters are dis-

played, if the tool is called with the ’help’ parameter.

• Pause/Resume Threshold Monitoring

This tool will stop or restart the threshold monitoring of the MySQL SPI. If the

threshold monitoring is paused, no performance values will be submitted to

the OVO monitoring agent.

Nevertheless the performance data will be still collected and submitted to the

performance component or agent. These tools should be used together with

the OVO functionality to disable the according policies to implement sched-

uled outages.

• Recompile CODA/MWA metric class

During normal operation the necessary configuration information for the HP

Performance component or agent is compiled at the configuration time of the

MySQL SPI instances. In case this configuration was accidentally deleted or the

type of performance agent was switched, this tool allows to create the neces-

sary configuration information for the integration of the performance data

again.

• Start/Stop Data Engine

The Start Data Engine tool is only used during the initial configuration steps to

add the first MySQL database instance. It will startup the SPI’s Engine process.

This happens in normal operation automatically triggered by the ’MySQL Sta-

tus’ policy.

The Stop tool is used to stop the Engine process, which is in normal operation

always running. The tool will be used e.g. if a new version of the MySQL SPI

program files are distributed to the agent, to trigger the start of the new

Engine version. The Engine will be automatically restarted by the ’MySQL Sta-

tus’ policy. Disable this policy if the Engine should remain stopped e.g. for sys-

tem maintenance.

• Start/Stop Perf. Metric collection

These tools allow to stop and re-start the submittal of the performance data

to the HP Performance component or agent. In normal operation the data is

submitted in 5 minute intervals. If this should be paused e.g. for some system

maintenance, the submittal can be suppressed by using this tool. If the collec-

tion is started again, the data will be submitted for the next interval.

The threshold monitoring will continue to work even if the submittal of the

performance data to the HP Performance component or agent is stopped.

Page 30: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

30

• Show MySQLSPI status

The status tool shows the actual runtime values of the Engine like the used

port number, the state of the Performance collection and threshold monitor-

ing.

Tools in MySQL SPI Licensing

The Application Group ’MySQL SPI Licensing’ contains tools to manage the

installed SPI Licenses. The detailed explanation of the license tools and the

license process can be found in chapter „Step 3: Generate the License Request

and obtain the Licenses“ on page 20.

• Generate License Request

The tool will create on each selected system the license request file

<OVAgentDir>/mysqlspi/tmp/<hostname>.req.

This file contains some information that is necessary to create a valid license

(i.e. hostname as it is discovered by the SPI, SPI version, platform). This license

request file must be submitted to the license website to retrieve an updated

license file.

• Update Licenses

The MySQL SPI works with a global license list. The license files that are

retrieved for each node must be merged into the global license list. This tool

takes as parameter the path and file name of a license file and updates the

global license list with the latest retrieved license file.

Page 31: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

31

Installation and Configuration

• Activate Updated Licenses

After all single license files are merged into the global license list, this tool pre-

pares the global list for the distribution to each managed MySQL server. After

running this tool the instrumentation for the MySQL SPI must be redistributed

to all managed MySQL servers that might be impacted by the license change.

• Show All Licenses

This tool lists all license entries that are available in the global license list.

• Show Node License

This tool shows the license that is available on the selected MySQL database

server. In addition the tool displays the number and type of licenses that it

assumes to be necessary based on the MySQL Instance usage patterns.

Page 32: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

32

Tools in MySQL SPI Tools

The Group ’MySQL SPI Tools’ contains tools to query runtime and configuration

information about the monitored MySQL instances.

• Config - List MySQL DB Instance

This tool lists all configured MySQL database instances and instance names on

the selected node.

• Show ...

All tools require to specify as first parameter the configured instance name of

the MySQL instance that should be used or the parameter ’-all’. The tools will

show the result for the specified instance or all configured instances. The

instance names can be listed by running the above described tool List

MySQL DB Instance.

• Show MySQL config changes and Show MySQL config changes(24h)

These tools list the MySQL configuration changes. The output lists all parame-

ters that changed together with their change history. The change history

shows the actual value and all (or for the last 24 hours) old values with the

time when the change was detected.The output will look like the following:

----Configuration changes for ’mydbsv1’ since: 2016-11-

09 10:28:30

connect_timeout='6'

>>> 2016-11-09 10:28:30='5'

>>> 2016-11-20 01:27:16='4'

Page 33: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

33

Installation and Configuration

The first line shows the time when the first configuration change of any parameter was

detected for this instance. The second line lists the parameter and it’s current value.

The next lines are prefixed with ’>>>’ and show the history values and the times at

which the change to a new value was detected. If more than one parameter was

changed this section will be repeated for every parameter.

The printing of the history can be restricted by a time parameter as shown in the Show

MySQL config changes(24h) sample tool. Only changes that occured during the speci-

fied time interval will be printed. If only the line with the actual value is displayed, the

last change to this parameter happend before the specified time interval.

• Show Table Sizes displays the size in MB for each table including their index

files.

Monitors

The SPI for MySQL Databases is designed to control the MySQL Database Server

instances as closely as possible while keeping the additional resource consump-

tion as low as possible. All data collections and calculations are centralized in a

single background process, the SPI’s Data Engine. This Engine samples the data

every 5 minutes, does all necessary calculations like counter delta and average

aggregations and passes the results to the performance agent. The policies do not

need to trigger resource consuming database log-on procedures but just trigger

the connector to submit values and events to the HP OM Monitor Agent or Mes-

sage Interceptor Interface.

The Data Engine and the utility scripts to access the Engine are placed by OM into

the directory

%OvAgentDir%\bin\instrumentation on the managed MySQL data-

base server.

The MySQL SPI uses a central directory to store configuration information, runt-

ime information and logfiles. The files in this directory must not be changed or

deleted manually. The directory resides in

Table 7: MySQL SPI Working Directory

Platform Directory

Windows Systems %OvAgentDir%\mysqlspi

Unix Systems /opt/OV/mysqlspi

Page 34: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

34

Policies

The Policies of the SPI for MySQL Databases are used to configure the monitors

from the HP Operations Manager perspective. The Policies define among other

parameters which values are monitored, which thresholds should be used, in

which intervals the monitors are started, and attributes that are assigned to a

message. This section will only briefly explain the structure of the MySQL SPI Pol-

icies. For the details about the use and configuration of Policies refer to the OMW

Online Help.

The Policies are contained in four subgroups of the Policy Group ’SPI for

MySQLDB’. The policies are also accessible according to the type of Policy in the

Navigation tree under the different types listed in Policy management->Policy

grouped by type.

All monitors are using the Object field to report back the according instance that

the returned values belongs to. For the majority of the monitors this represents

the database instance. For other monitors it contains the full qualified table name

or file system that belongs to the submitted value. This information allows to eas-

ily identify any impacted services as well as it enables the definition of object spe-

cific threshold values. For detailed information refer to the topic ’instance filter’

in the OMW online help.

Page 35: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

35

Installation and Configuration

The policies can be renamed. The threshold policies expect in the first two param-

eters the internal metric name and the OMW policy name. The internal metric

name uniquely identifies to the SPI the requested monitor. The OMW policy vari-

able $NAME is used to pass the monitored values back to the OM monitor agent.

When changing the policy name or creating a new policy, the first parameter

must remain unchanged, the second parameter will automatically set new policy

name and does not need to be changed.

The Policy Group Mandatory

The two policies in this group must be assigned to each MySQL database server

that should be monitored.

The policy MYSQL_STATUS is responsible for checking in regular intervals the

status of the SPI’s Data Engine and the database instances. Beside the collection

and monitoring of the availability data for the MySQL Database instances, this

monitor handles the situation that the Data Engine is stopped. The monitor will

automatically start the Data Engine if it is not running or responding. In normal

operation the Engine is always running in the background collecting and calculat-

ing the statistics. It will be only stopped on manual request e.g. to allow the

upgrade of the SPI or after a system reboot. If the Data Engine should remain

stopped e.g. for a scheduled outage, this policy needs to be disabled during this

time. For a detailed description about Disabling Policies refer to the OWM docu-

mentation.

Page 36: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

36

The template MessagesMySQLSPI captures all administrative messages that

the SPI might send, log file messages that the MySQL database engine generates

as well as messages that the SPI generates for not threshold based monitors.

Administrative messages will be send in case of SPI or MySQL configuration or

licensing errors.

The monitoring functionalities for configuration problems, long running transac-

tions, configuration changes and similar are submitting their information with

opcmsg to this template for further processing.

The MySQL database engine generates in it’s log files multi-line entries, which can

not be handled without preprocessing by OM’s log file monitoring functionality.

The Data Engine has the build-in capability to process such log files, to analyze

which lines belong together and enrich the log message with database instance

information. This enriched log messages are submitted via opcmsg to this tem-

plate.

The Policy Group Event

This policy group contains monitors that are important to detect unusual behav-

iour or activities for MySQL Databases in production environments.

The monitors will trigger a message (event) if the configured thresholds are

exceeded, but due to the exceptional nature of the monitors, their values are not

stored in the Performance Agent for History Reporting.

Page 37: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

37

Installation and Configuration

The monitored areas cover the usage of data structure and security related com-

mands. The selection of monitors that should be activated depends heavily on

the usage profile of the MySQL database.

For an ISP environment the altering of table structures and indexes by customers

will be a common behaviour, but for a large transactional database of an online

shop the altering of a single table or index might cause severe performance

impacts for hours. The alteration of user rights in environments with customer

data or other sensitive information should be also closely monitored. In such

environments the user roles and access rights are carefully designed. A combina-

tion of grant/revoke statements might indicate that someone got access to a user

account with the grant privileges and adds temporarily access privileges for confi-

dential data to his account and afterwards revokes these privileges to avoid

detection of the security violation.

Notified by the generated message the MySQL Administrator can react in time

and cancel e.g. running alter table commands or take further steps to pinpoint

security issues. The tracking of these events also enables to document reasons for

service level violations.

Page 38: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

38

The Policy Group Performance

This policy group contains the threshold monitors that measure the Key Perfor-

mance Indicators and several supporting metrics that provide insights on quality

of configuration settings, instance load and throughput of the MySQL Database

instances.

The metric values or the base metrics that are used to calculate aggregate metrics

are stored for further analysis and history reporting in the OV performance agent.

The collection of the metrics to the performance agent does not require any sep-

arate configuration and is independent from the selection and configuration of

the threshold policies.

The policies can be customized with the standard OM mechanisms. The polling

interval should be a multiple of 5 minutes to achieve optimal calculation of met-

rics based on rates. Polling intervals less than 5 minutes will not result in more

details, but will be cached by the Data Engine to avoid load impacts on the data-

base. The thresholds can be adjusted for each MySQL Database Instance by using

Instance Filters. The configured Instance name of the MySQL Database or the

monitored object name is used as threshold Object.

The details regarding the available policies, the meaning of the monitored metric

and possible corrective actions can be found in the appendix „Capacity Planning

related Metrics“ on page 44 and in the policies themselves.

Page 39: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

39

Installation and Configuration

The Policy Group Custom

This policy group contains policies that must not be assigned to a system before

they are customized specifically to the environment and any necessary specific

configuration steps have been performed.

Count Monitors

The policy Count Monitor is used to monitor the number of rows in a specific

table. This feature is often used to detect problems in the application throughput

or responsiveness by monitoring tables that contain e.g. outstanding job

requests.

To configure a Count Monitor policy copy the existing Sample policy or create a

new policy and set the policy program call as follows:

mysqlspi ROWSCOUNT <$NAME> <instancename> <dbname>

<tablename>

• mysqlspi ROWSCOUNT <$NAME>: required, do not change these

• <instancename>: replace this with the name that is configured for this

instance of the MySQL Database

• <dbname>: replace with the database name in which the table resides

• <tablename>: replace with the table name for which the row count will be

collected

Page 40: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

40

Business Process and Custom Query Monitors

Business processes often include steps that are dependent on some sort of data-

base transactions. To monitor such business processes and identify problem

areas, it is required to check the behaviour of the involved database transactions.

For some processes the checking of the number of entries in certain tables gives

enough information to identify the bottleneck in the process line. This can be

achieved with the Row Count monitor described in the previous chapter.

Often a more complex query is required e.g. to calulate time differences between

an order entry and the order processing or similar. In addition the monitoring of

results and response times of a typical process transaction by using synthetic

transactions is mandatory to verify the health and performance of the overall sys-

tem.

All this functionality can be easily implemented with the SPI. The template

Sample_CustomQuery Monitor gives an ready to use example of how to config-

ure such monitors.

These monitors consist of a policy and a file containing the SQL query that needs

to be performed. The name of the SQL query file must match the name of the

policy with an “.sql” extension.

In the "Program name" field for Measurement Threshold monitors use the follow-

ing syntax:

mysqlspi MYQUERY <$NAME> <INSTANCENAME|-all> {count |

value(FieldName) | time}

or

mysqlspi MYQUERY <$NAME> <INSTANCENAME|-all> {count |

value(FieldName) | time} [param1]="value1"

[param2]="value2" ...

• MYQUERY - keyword (fix) to identify the metric as query metric

• <$NAME> - will be resolved as name of the policy

• INSTANCENAME - name of monitored instance as configured on remote sys-

tem (MySQLSPI Version 2.00 and higher can use '-all' to perform query on all

configured instances)

• {count | value(FieldName) | time} - available functions• count - just count the number of rows in the result set

• value(FieldName) - returns value of specified field (must be convertable to double)

• time - returns query processing time (milliseconds).

• [param1]="value1" [param2]="value2"... - if a query needs to be supplied with

parameters these can be specified at the end of the call.• Parameter names MUST be enclosed with brackets [...] and exact the same notation

must be used in the query template.

Page 41: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

41

Installation and Configuration

• Value MUST be enclosed with double quotes ("). These quotes will be removed dur-

ing replacing.

• The MySQLSPI will replace all entries in the supplied SQL template with the specified

values.

• Multiple query parameters should be separated be spaces.

The values and execution times of the monitors will be stored in the

MYSQL_STATS_EXT performance class of the Performance Agent or Performance

Component automatically.

Steps to create a Query Monitor:1. Prepare and test the SQL query

2. Create a new Measurement Threshold policy

3. Store SQL query in a file named <PolicyName>.sql

4. Place the <PolicyName>.sql file in the instrumentation folders on the server side

(<OVShare Dir>\Instrumentation\*) and redeploy instrumenta-

tion

or4. Transfer the <PolicyName>.sql file to the MySQLSPI data directory directly on the

managed node(s)

(Windows: %OvAgentDir%\mysqlspi; Unix: /opt/OV/mysqlspi)

5. Assign and deploy the created policy to the managed node(s).

Replication Monitoring

The Synchronization Monitor allows to monitor the synchronization delays

occurring between MySQL database instances that are configured to replicate

data. MySQL replication is often used to enable load balancing across multiple

database servers and in high availability environments. The MySQL data replica-

tion functionality allows to replicate data from a Master system to one or more

replication Slave systems. These Slave systems can be replication Masters for

other replication Slave systems, allowing a replication chain. Caution: a cycle in

the replication chain is not recommended and would also require special consid-

erations in the policy configuration.

Before configuring the necessary policies, make sure that the MySQL replication

is configured and working.

To setup the monitoring functionality special tables are required to track the rep-

lication progress and two policies must be created.

The special tables have to be created in the replication Master and Slave database

with a slightly different definition.

The configured user account for the SPI must have the according privileges to cre-

ate tables in the target database instance.

Page 42: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

42

The tool MySQL SPI Admin->Create Replication Master Table can be used to

create the table on the replication Master. It requires the following parameters to

be specified:

• <instancename>: replace this with the name that is configured for this

instance of the MySQL Database

• <dbname>: replace with the database name in which the table resides

• <tablename>: replace with a name for the new table

Next use the tool MySQL SPI Admin->Create Replication Slave Table on the

Slave systems. If the replication is configured to cover all tables, it will have cre-

ated a copy of the above mentioned Master table. However, the Slave table is

slightly different and requires a trigger. Therefore it is mandatory to re-create this

table with this tool to enable the Synchronization Monitor functionality. The tool

requires the following parameters

• <su>: replace this with the name of an MySQL account with super user privi-

leges. These special privileges are required to be able to create a trigger on

the new table

• <password>: replace this with the password of above specified user

account

• <instancename>: replace this with the name that is configured for this

instance of the MySQL Database

• <dbname>: replace with the database name in which the table resides

• <tablename>: replace with a name for the new table. This name has to be

the same name as specified on Master

After the tables are created the MySQL replication configuration must be

reviewed to make sure that the configured Master table replicates to the correct

Slave table.

After these steps have been performed, the Master and Slave policies can be con-

figured. Copy the existing Sample policies or create a new policies and set the

program calls as follows:

mysqlspi SLAVESTATUS <$NAME> <instancename> <dbname>

<tablename>

mysqlspi MASTERTRIGGER <$NAME> <instancename> <dbname>

<tablename>

• <$NAME>: required, do not change it

• <instancename>: replace this with the name that is configured for this

instance of the MySQL Database

• <dbname>: replace with the database name in which the table resides

• <tablename>: replace with the above created table name

The Master policy has to be distributed to the replication master system and the

Slave policy to all the slaves.

Page 43: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

43

Installation and Configuration

The Master policy will update for every execution a time stamped entry in the

specified master table. The MySQL replication will replicate this entry to all

Slaves. The Slave policy will check the replicated table in each polling interval for

the record that was replicated from the Master database. Based on the arrival

time the policy will calculate and return the replication delay between the Slave

and Master database.

To calculate the replication delay the monitor on the Slave systems tries to con-

nect to the SPI Data Engine on the Master system to identify differences in the

system time. If this connection is not possible e.g. due to firewall settings, the cal-

culated replication delay time might have an offset according to any time differ-

ence of the two systems. A negative replication delay could be detected if the

time on the Master is behind the time of the Slave system. Since this would mean

that the data is replicated before it was inserted on the Master, the monitor will

ignore such values. To avoid such problems it is highly recommended to use a

Time Server to ensure that the system times are synchronized or to make the SPI

Data Engine port on the Master accessible for access from the Slave system.

There is an important difference in the functionality that is available for MySQL

versions before 5.0.13 and later releases. For MySQL versions before 5.0.13 the

monitor can only calculate the delay time based on the difference of the times-

tamp in the replicated table and the actual system time at which the Slave moni-

tor detects the replicated record. Therefore the calculated delay time is only an

indicator based on the number of monitor polling intervals that have passed

before the data was replicated. The thresholds should be set as multiples of the

configured polling interval.

For MySQL database versions 5.0.13 and later the monitor can calculate the delay

time with a precision of 1 second. It uses a trigger on the Slave table which

updates a timestamp value as soon as the record is replicated from the Master

database. This functionality will not work if the default behaviour of the MySQL

sysdate() functionality is changed.

Therefore the database option --sysdate-is-now must not be set.

Page 44: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

44

Appendix

Capacity Planning related Metrics

Several of the provided metrics are not only important for the ongoing monitor-

ing, but as well for the capacity planning. Typically the metrics related to the

usage like the number of queries or commits, the concurrent queries or connec-

tions, Cache utilizations and prunes, CPU and disk usage are used to determine

future capacity requirements.

Based on the collected historical information calculations can be performed to

get an idea how the system will behave if certain parameters change or e.g. just

the amount of data continues to grow like in the past. These calculations allow to

plan in advance necessary upgrades to a system like adding some disks or mem-

ory, before the system runs into a bottleneck.

To further support this capacity planning process the SPI for MySQL Databases 4

collects some additional metrics that are important for capacity planning, but not

so interesting for the operational monitoring. These metrics are related to the

individual database sizes and storage engine types. The metric values are logged

directly to Performance Agent in the new data class MYSQL_STATS_EXT.

In case these metrics should be also included in the monitoring process, an

according Policy must be defined. Please contact our support team for the white-

paper describing this topic.

Policies

Table 8: Policies

MonitorName Description

MySQLSPI_AbortedClientsDelta Delta aborted clients on the polling inter-

val.

MySQLSPI_AbortedConnection

sRate

Delta aborted clients on the polling inter-

val.

MySQLSPI_AlterDbDelta Monitors numbers of ‘ALTER DATABASE’

queries on the polling interval.

MySQLSPI_AlterTableDelta Monitors numbers of ‘ALTER TABLE’ queries

on the polling interval.

Page 45: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

45

Installation and Configuration

MySQLSPI_AvgAbortedClients Average per minute aborted clients

MySQLSPI_AvgBytesReceived Average per minute bytes received

MySQLSPI_AvgBytesSent Average per minute bytes sent

MySQLSPI_AvgCpuUsed Average CPU utilization (%) on the polling

interval

MySQLSPI_AvgHandlerCommit Average per minute commits

MySQLSPI_AvgOpenedTables Average per minute opened tables

MySQLSPI_AvgQueries Average per minute queries

MySQLSPI_AvgSelectFullJoin Average per minute selects with full joins

MySQLSPI_AvgSelectFullRangeJ

oin

Average per minute selects with full range

joins

MySQLSPI_AvgSelectRange Average per minute selects with range con-

ditions

MySQLSPI_AvgSelectScan Average per minute selects scans

MySQLSPI_AvgSlowLaunchThre

ads

Average per minute of the number of

threads that have taken more than

slow_launch_time seconds to create.

MySQLSPI_AvgSlowQueries Average per minute of the number of que-

ries that have taken more than

long_query_time seconds

MySQLSPI_AvgSortMergePasse

s

Average per minute of the number of

merge passes that the sort algorithm has

had to do.

MySQLSPI_AvgThreadsCreated Average per minute of created threads

MySQLSPI_AvgThreadsRunning Average threads running

MySQLSPI_BinlogCacheUsage Binlog cache usage ratio

MySQLSPI_ConcurrentQueries Number of actually running queries.

Table 8: Policies

MonitorName Description

Page 46: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

46

MySQLSPI_Connections Delta of connections on the polling interval

(successful or not) to the MySQL server

MySQLSPI_CreateFunctionDelt

a

Monitors numbers of ‘CREATE FUNCTION’

queries on the polling interval.

MySQLSPI_CreateIndexDelta Monitors numbers of ‘CREATE INDEX’ que-

ries on the polling interval.

MySQLSPI_CreateTableDelta Monitors numbers of ‘CREATE TABLE’ que-

ries on the polling interval.

MySQLSPI_DeltaHandlerRollba

ck

Delta rollbacks per minute

MySQLSPI_DeltaQCacheMemP

runes

The number of queries that were deleted

from the query cache because of low

memory.

MySQLSPI_DeltaTmpDiskTables Number of temporary tables created on

disk during last polling interval

MySQLSPI_DeltaTmpTables Number of temporary tables created during

last polling interval

MySQLSPI_DropFunctionDelta Monitors numbers of ‘DROP FUNCTION’

queries on the polling interval.

MySQLSPI_DropIndexDelta Monitors numbers of ‘DROP INDEX’ queries

on the polling interval.

MySQLSPI_DropTableDelta Monitors numbers of ‘DROP TABLE’ queries

on the polling interval.

MySQLSPI_DropUserDelta Monitors numbers of ‘DROP USER’ queries

on the polling interval.

MySQLSPI_FreeDiskInnodbLogs Free disk space on the volume used by

InnoDB for log files.

Table 8: Policies

MonitorName Description

Page 47: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

47

Installation and Configuration

MySQLSPI_FreeDiskInnodb Remaining free space for the InnoDB data.

Depending on the configuration of the

InnoDB storage, the metric will return the• Free space within the allocated InnoDB File(s) if autoex-

tent is turned off

• Free space within the allocated InnoDB File(s) + the

remaining space on the file system or if set until the con-

figured MAXSIZE is reached

MySQLSPI_FreeDiskSpace Free disk space on the volume where data-

dir of the MySQL is located

MySQLSPI_GrantsDelta Monitors numbers of ‘GRANT’ queries on

the polling interval.

MySQLSPI_InnoDBBufferHitRat

e

InnoDB Buffer cache hits ratio

MySQLSPI_InnoDBBufferPoolW

aits

InnoDB Buffer pool waits ratio

MySQLSPI_InnoDBLockWaits InnoDB lock waits

MySQLSPI_InnoDBLogWaits InnoDB log waits

MySQLSPI_InnoDBRead Average per minute data read (MB/min) for

InnoDB engine

MySQLSPI_InnoDBSize Size of the Databases using InnoDB (MB)

MySQLSPI_InnoDBWrite Average per minute data written (MB/min)

for InnoDB engine

MySQLSPI_KeyReadRatio Key reads ratio

MySQLSPI_KeyWriteRatio Key writes ratio

MySQLSPI_MaxOpenedTables Peak of opened tables per minute on the

polling interval

MySQLSPI_MaxThreadsCache Highwater of threads cache utilization (%)

MySQLSPI_MemoryUsed Memory used (MB) by mysqld

MySQLSPI_MyISAMDBSize MyISAM size (MB)

Table 8: Policies

MonitorName Description

Page 48: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

48

MySQLSPI_MyISAMTblLockRati

o

MyISAM table lock ratio

MySQLSPI_OpenTables Number of currently open tables

MySQLSPI_QueryCacheFragme

nts

The number of free memory blocks/frag-

mentation in the query cache.

MySQLSPI_QueryCacheHitRate Query cache hits ratio

MySQLSPI_ResponseTime Response time in ms of ‘show status’ query

MySQLSPI_RevokeAllDelta Monitors numbers of ‘REVOKE ALL’ queries

on the polling interval.

MySQLSPI_RevokesDelta Monitors numbers of ‘REVOKE’ queries on

the polling interval.

MySQLSPI_Sample_Count_Mo

nitor

This is a sample row count monitor. Please

change:

<instancename> - name of mysql instance.

<dbname>, <tablename> - specifies target

table.

MySQLSPI_Sample_CustomQu

ery

Sample for the Business Process / Query

Monitor.

MySQLSPI_STATUS MySQL DB status monitor

MySQLSPI_Sync_Master Monitor needs to be assigned to the master

of the master-slave synchronization. Return

value 1 indicates successful update of one

row in the master table. The

MySQLSPI_Sync_Slave will monitor

updated record on the slave instance.

MySQLSPI_Sync_Slave Monitor needs to be assigned to the slaves

of the master-slave synchronization.

Returns delay of the replication in number

of seconds. This value is not precise and

have deviation value from [0...polling inter-

val]

Table 8: Policies

MonitorName Description

Page 49: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

49

Installation and Configuration

Metric Classes

The SPI for MySQL Databases 4 creates the following Metric Classes for the Per-

formance Agent or Component:

MySQLSPI_TableFreeCache Free cache available for opening tables

MySQLSPI_ThreadPoolFreeSize Thread pool free size

MySQLSPI_MaxThreadsCache Highwater of thread cache utilization (%)

MySQLSPI_ThreadsRunning Number of threads actually running

MySQLSPI_TmpTablesOnDiskRa

tio

Rate for disk based tmp tables

Table 9: Metric Classes

Class Name Description

MYSQL_STATS The SPI logs the measured performance

values as metrics of this class

MYSQL_STATS_COUNTERS The SPI logs the values of the Row Count

Monitors as metrics of this class

MYSQL_STATS_EXT The SPI logs the Database Sizes and the

response time and values of the Business

Process and Query Monitors as metrics of

this class.

Table 8: Policies

MonitorName Description

Page 50: Smart Plug-In for MySQL Databases - bes-worldline.com€¦ · The Smart Plug-In for MySQL Databases is designed to monitor the state and ... • Commands, selectable menu ... The

Smart Plug-In for MySQL Databases

50

Bull GmbH

Von-der-Wettern-Str. 27

D-51149 Köln

Germany

Phone: +49 (0) 211-399-415-30

Bull DataCenter Products: http://www.bes-bull.com

Bull: http://www.bull.com

Atos: http://www.atos.net

mail: [email protected]