Zrm Mysql Zfs

8/14/2019 Zrm Mysql Zfs

1/27

Using ZFS TM Snapshots WithZmanda Recovery Manager for MySQL TM

on OpenSolaris TM 2008.11 OSCaryl Takvorian

May 2009

Sun Microsystems, Inc.

This article was published by BigAdmin at:http://www.sun.com/bigadmin/features/articles/zrm_mysql_zfs.jsp

To keep track of the latest content published by BigAdmin, subscribeto the BigAdmin newsletter: http://www.sun.com/bigadmin/newsletter/ .
http://www.sun.com/bigadmin/features/articles/zrm_mysql_zfs.jsphttp://www.sun.com/bigadmin/newsletter/http://www.sun.com/bigadmin/newsletter/http://www.sun.com/bigadmin/features/articles/zrm_mysql_zfs.jsp


2/27

Copyright 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.

U.S. Government Rights - Commercial software. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements. Use is subject to license terms.This distribution may include materials developed by third parties.

Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is aregistered trademark in the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd. X/Open is a

registered trademark of X/Open Company, Ltd.

AMD, Opteron, the AMD logo, the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation in the United States and other countries.

Sun, Sun Microsystems, the Sun logo, MySQL, OpenSolaris, Solaris, Sun BluePrints, Sun Fire, SunSolve, and ZFS aretrademarks or registered trademarks of Sun Microsystems, Inc. or its subsidiaries in the United States and other countries.

This product is covered and controlled by U.S. Export Control laws and may be subject to the export or import laws inother countries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited. Export or reexport to countries subject to U.S. embargo or to entities identified on U.S.export exclusion lists, including, but not limited to, the denied persons and specially designated nationals lists is strictly

prohibited.

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

2 Using ZFS Snapshots With Zmanda Recovery Manager for MySQL on OpenSolaris 2008.11 OS


3/27

Table of ContentsIntroduction................................................................................................................................................5Architecture Overview...............................................................................................................................6

OS Configuration..................................................................................................................................6The ZFS File System.............................................................................................................................6

Zpool Configuration.........................................................................................................................7ZFS Configuration............................................................................................................................7

Configuration of Zmanda Recovery Manager 3.0 for MySQL.............................................................8Installing Dependencies Using pkgadd ...........................................................................................9Installing Dependencies Using IPS..................................................................................................9Installing ZRM...............................................................................................................................10

Types of Backup Available to ZRM...................................................................................................11Raw-Copy Backup..........................................................................................................................11Logical Backup...............................................................................................................................11Raw-Quick (No-Copy) Backup .....................................................................................................11

Performance Testing Setup......................................................................................................................13System Setup.......................................................................................................................................13MySQL Software Release...................................................................................................................13OpenSolaris Operating System Release..............................................................................................13Zpool Configuration............................................................................................................................13ZFS Configuration...............................................................................................................................14ZRM Software Release.......................................................................................................................14db_STRESS Load Generator...............................................................................................................14

1. Creating the test databases..........................................................................................................142. Creating test tables in pool2/mysql............................................................................................143. Generating the dataset.................................................................................................................154. Loading the dataset into the tables..............................................................................................155. Creating users.............................................................................................................................156. Creating a test scenario ..............................................................................................................15

Testing Methodology..........................................................................................................................15Performance Test Results.........................................................................................................................17

1. MyISAM Results with Idle Database..............................................................................................172. InnoDB Results with Idle Database................................................................................................173. MyISAM Results with a 15 Percent Load on the Database ...........................................................194. InnoDB Results with a 15 Percent Load on the Database ..............................................................205. MyISAM Results with a 90 Percent Load on the Database ...........................................................216. InnoDB Results with a 90 Percent Load on the Database ..............................................................22

Product Information.................................................................................................................................23OpenSolaris Operating System...........................................................................................................23

The ZFS File System...........................................................................................................................23The Sun Fire X4500 Server.................................................................................................................23Zmanda Recovery Manager 3.0 for MySQL.......................................................................................23

Summary..................................................................................................................................................24About the Author ....................................................................................................................................24For More Information..............................................................................................................................25Licensing Information..............................................................................................................................27



4/27

IntroductionThe objective of this report is to show how to deploy Zmanda Recovery Manager 3.0 for MySQL onthe OpenSolaris 2008.11 OS and how ZRM performs on a Sun Fire X4500 system using the ZFS

file system. In particular this article demonstrates how ZRM takes advantage of ZFS snapshottechnology to offer a best-in-class open source data-protection solution for MySQL.

It is intended to be read in conjunction with the article Using Sun Fire X4540 Server With Zmanda Recovery Manager 2.2 for MySQL Database by Thomas Hanvey (Sun Microsystems) and DmitriJoukovski and Ken Crandall (Zmanda), October 2008, which is available athttp://www.sun.com/bigadmin/features/articles/zrm_mysql_sfx4540.jsp .

All the steps described in this document were carried out on a single Sun Fire X4500 system whichacted as the ZRM backup server and the MySQL database server.

For more information on the OpenSolaris 2008.11 OS and Zmanda technology, see the ProductInformation section at the end of this article. For further details on ZRM Enterprise, see:http://zmanda.com/zrm-mysql-enterprise.html .

Zmanda Recovery Manager for MySQL is available for download from the Zmanda network:http://network.zmanda.com/ .

http://www.sun.com/bigadmin/features/articles/zrm_mysql_sfx4540.jsphttp://zmanda.com/zrm-mysql-enterprise.htmlhttp://network.zmanda.com/http://www.sun.com/bigadmin/features/articles/zrm_mysql_sfx4540.jsphttp://zmanda.com/zrm-mysql-enterprise.htmlhttp://network.zmanda.com/


5/27

Architecture Overview

OS ConfigurationFor this guide, ZRM was deployed on a Sun Fire X4500 server running the OpenSolaris 2008.11release.

For more information on the Sun Fire X4500 server, see the Product Information section at the end of this article.

The system used during this test is a server with two dual-core AMD TM Opteron 285 processors runningat 2593 MHz with 16 Gbyte of RAM and 48 SATA drives of 512 Gbyte each, for a total of 24 Tbyte of storage capacity.

The two boot drives are mirrored using ZFS volume management capabilities and the remaining 46drives are available for backup data.

The ZFS File SystemFor more information on ZFS, see the Product Information section at the end of this article.


Figure 1: Architecture Overview


6/27

One of the most interesting features of ZFS when combined with backup technologies is its snapshotcapabilities. A snapshot is a read-only copy of a file system or volume which can be created veryquickly. Thanks to ZFS copy-on-write characteristics, snapshots are essentially a free operation, whichinitially consume no additional space within the pool. However, as data within the active datasetchanges, the snapshot consumes space by continuing to reference the old data and so prevents the datafrom being freed back to the pool.

ZFS snapshots include the following features: Persist across system reboots. Use no separate backing store. Snapshots consume disk space directly from the same storage

pool as the file system from which they were created. Recursive snapshots are created quickly as one atomic operation. The snapshots are created

together (all at once) or not created at all. The benefit of atomic snapshot operations is that thesnapshot data is taken at one consistent time, even across descendent file systems.

Zpool Configuration

For the purposes of this guide, the storage available on the Sun Fire X4500 server was split into twopools: A backup pool made of 24 drives and two spares in a RAIDZ2 configuration. This provided 11

Tbyte of capacity for backups with a very high level of protection against drive failure andperformance sufficient to handle 300 to 350 Mbyte/sec of backup traffic over the network.

A mysql pool made of 12 drives and two spares in a RAIDZ2 configuration was used to storethe MySQL databases.

First, we created the RAIDZ2 pool zmanda with six drives. Then, we added the other drives by groupsof six and verified at the end that we ended up with a pool with 10.9 Tbyte of physical space.

Here are zpool creation commands for pool zmanda :# zpool create -f zmanda raidz2 c3t1d0 c4t1d0 c6t1d0 c7t1d0 c8t1d0 c9t1d0# zpool add zmanda raidz2 c3t2d0 c4t2d0 c6t2d0 c7t2d0 c8t2d0 c9t2d0# zpool add zmanda raidz2 c3t5d0 c4t5d0 c6t5d0 c7t5d0 c8t5d0 c9t5d0# zpool add zmanda raidz2 c3t6d0 c4t6d0 c6t6d0 c7t6d0 c8t6d0 c9t6d0# zpool add zmanda spare c8t0d0 c9t0d0

We used a similar command to create the pool pool2 .

Here are the zpool creation commands for pool pool2 :# zpool create -f pool2 raidz2 c3t3d0 c4t3d0 c6t3d0 c7t3d0 c8t3d0 c9t3d0# zpool add -f pool2 raidz2 c3t7d0 c4t7d0 c6t7d0 c7t7d0 c8t7d0 c9t7d0# zpool add -f pool2 spare c8t4d0 c9t4d0

Finally we can check the ZFS pools just created:# zpool listNAME SIZE USED AVAIL CAP HEALTH ALTROOTpool2 5.44T 225K 5.44T 0% ONLINE -rpool 464G 13.3G 451G 2% ONLINE -zmanda 10.9T 252K 10.9T 0% ONLINE -



7/27

ZFS ConfigurationNext, we need to create the ZFS file system which will be used by ZRM and MySQL. We will needthree different ZFS file systems:

zmanda/zrm to hold the ZRM backups zmanda/logs to hold the MySQL binlogs pool2/mysql to hold the MySQL data files

So, first let's create the ZFS file system where ZRM will store the backup images, and mount it in thedefault location expected by ZRM:

# zfs create -o mountpoint=/var/lib/mysql-zrm zmanda/zrm

Then, create the ZFS file system where the MySQL data files will be stored, and the third file system tostore the MySQL log files.# zfs create zmanda/logs# zfs create pool2/mysql

Finally we can list the file systems available to us:# zfs listNAME USED AVAIL REFER MOUNTPOINTpool2 1.35T 2.22T 1023G /pool2pool2/mysql 356G 2.22T 356G /pool2/mysqlrpool 185G 272G 60K /rpoolrpool/ROOT 150G 272G 18K legacyrpool/ROOT/opensolaris-1 150G 272G 121G legacyrpool/dump 7.95G 272G 7.95G -rpool/export 19.1G 272G 19K /exportrpool/export/home 19.1G 272G 18.9G /export/homerpool/swap 8.16G 280G 216M -zmanda 50.0G 7.08T 38.0K /zmandazmanda/logs 815K 7.08T 815K /zmanda/logszmanda/zrm 87.9K 7.08T 87.9K /var/lib/mysql-zrm

Configuration of Zmanda Recovery Manager 3.0 for MySQLFor more information on ZRM, see the Product Information section at the end of this article.

The Sun Fire X4500 server is configured as a ZRM server with Zmanda Management Console (ZMC).To install ZRM and ZMC, follow these overall steps, which are explained in more detail in subsequentsections.

1. From the Zmanda Network web site (which requires registration), download the following: ZRM installer binary for Solaris 10 systems with Intel or AMD Opteron processors:

ZRM-enterprise-3.0-installer-intel.bin Prerequisite packages: solaris-10-zrm-dependency-intel.tar.gz

1. Extract the prerequisite packages, solaris-10-zrm-dependency-intel.tar.gz , andinstall all the included packages.

2. Extract the binary package and run the Zmanda Rapid Installer to install ZRM.

3. After the installation is complete, the ZMC Welcome Page is displayed, and you can beginconfiguring backup sets.



8/27

There are two ways of installing the required dependencies, using pkgadd or using Image PackagingSystem (IPS).

Installing Dependencies Using pkgadd

The first option is to download the dependency package provided by Zmanda from the ZmandaNetwork at http://network.zmanda.com . The required dependency packages are provided in acompressed tar archive file, solaris-10-zrm-dependency-intel.tar.gz .

Extract all the packages:# gtar zxvf solaris-10-zrm-dependency-intel.tar.gz

The dependency packages provided are as follows: SMCcoreu CSWgcc3corert CSWgcc3g++rt SMCgpgme SMCgrep SMClgcc346 SMClgpger SMCliconv SMClintl SMCpcre SMCsudo SMCtar

These packages are also available from Sunfreeware at http://sunfreeware.org/ and fromBlastwave at http://blastwave.org .

Install each package using pkgadd :

# pkgadd -d

Installing Dependencies Using IPSThe second option is to use the new OpenSolaris Image Packaging System. The most popular Solarispackaging repositories such as http://blastwave.org and http://sunfreeware.org have alreadyconverted the software they have on offer to the new IPS package format, and setting OpenSolaris touse their repository is one single step.

Add the repository to the list of authorized package repositories:# pkg set-authority -O http://blastwave.network.com:10000 blastwave

Let's make sure that everything is OK:# pkg authority -HAUTHORITY URLblastwave http://blastwave.network.com:10000/opensolaris.org (preferred) http://pkg.opensolaris.org:80/

Now we can start installing the CSW packages required by ZRM.

http://network.zmanda.com/http://sunfreeware.org/http://blastwave.org/http://blastwave.org/http://sunfreeware.org/http://network.zmanda.com/http://sunfreeware.org/http://blastwave.org/http://blastwave.org/http://sunfreeware.org/


9/27

Note that the names of the Blastwave packages available for OpenSolaris IPS start with IPS instead of the traditional CSW; for example, IPScommon instead of CSWcommon.

# pkg install IPScommonDOWNLOAD PKGS FILES XFER (MB)Completed 1/1 1/1 0.00/0.00PHASE ACTIONSInstall Phase 146/146PHASE ITEMSReading Existing Index 9/9Indexing Packages 1/1

To show the ease of use of the OpenSolaris IPS packaging system, we'll install the complete list of required packages in one single command:

# pkg install IPScurl IPSbzip2 IPSbdb4 IPScurlrt IPSexpat IPSfindutils IPSgfile IPSggettextIPSgtar IPSiconv IPSlibidn IPSlibnet IPSoldaprt IPSossl IPSossldevel IPSosslrt IPSosslutilsIPSreadline IPSsasl IPSsunmath IPSzlib IPSglib2 IPSgcc3g++rt IPSmtx

Once finished, let's check that we have everything. (Once installed, the Blastwave packages use theirold CSW names.)

# pkginfo | grep CSWapplication CSWbdb4 berkeleydb4 - Embedded database libraries andutilities application CSWbzip2 bzip2 - a high-quality block-sorting filecompressorsystem CSWcommon common - common files and dirs for CSW packagesapplication CSWcurl curl - Library and tool for common Internetprotocolsapplication CSWcurlrt curlrt - Library for common Internet protocolsRuntimeapplication CSWexpat expat - XML Parser Toolkitsystem CSWfindutils findutils - A set of utilities for searching a filesystem

application CSWgcc3corert gcc3corert - GNU C Compiler Run Timeapplication CSWgcc3g++rt gcc3g++rt - GNU C++ Compiler Run Timesystem CSWgfile fileutils - GNU file utilitiessystem CSWggettext ggettext - GNU gettextapplication CSWglib2 glib2 - the low-level core library for GTK+ andGNOMEapplication CSWgtar gtar - GNU tape archiversystem CSWiconv libiconv - GNU iconv libraryapplication CSWlibidn libidn - Implementation of the Stringprep, Punycodeand IDNAsystem CSWlibnet libnet - the libnet packet construction libraryapplication CSWmtx mtx - SCSI Media Changer and Backup Device Controlapplication CSWoldaprt openldap_rt - OpenLDAP runtime libraries (oldaprt)application CSWossl openssl - Openssl meta packageapplication CSWossldevel openssl_devel - Openssl development supportapplication CSWosslrt openssl_rt - Openssl runtime libraries

application CSWosslutils openssl_utils - Openssl binaries and related toolssystem CSWreadline readline - library to enable interactive lineeditingapplication CSWsasl sasl - Simple Authentication and Security Layersystem CSWsunmath libsunmath - Sun maths libraryapplication CSWzlib zlib - Zlib Data Compression Library



10/27

Installing ZRMInstalling Zmanda Recovery Manager 3.0 for MySQL is a simple task of running the Zmanda RapidInstaller which contains the ZRM Server, the ZRM Client, as well as the Zmanda Management Console(ZMC) web user interface.

As we are connected to the Sun Fire X4500 through an SSH connection, we'll start the Rapid Installerin text mode, as such:# ./ZRM-enterprise-3.0-installer-intel.bin -mode text

Once completed, the following software components are installed: The ZRM 3.0 for MySQL software Zmanda Management Console with MySQL, PHP, Apache, PHPmyadmin, Perl and various

Perl modules and their dependencies. These are installed under the /opt/zmanda/zrm and/opt/zmanda/ common directories.

To start or stop ZRM run:# /etc/init.d/zmc_zrm_init start | stop | status

Types of Backup Available to ZRM Depending on the datastore selected, ZRM offers a number of different types of backup.

Raw-Copy BackupA raw-copy backup makes a copy of the binary disk image of databases stored on non-transactionalstorage engines by using mysqlhotcopy . It works on live databases, but the whole database (or thetables being backed up if only a few tables are selected) is locked for the duration of the backup.Although raw backups can be restored more quickly than logical backups, they can only be restored tothe same version of MySQL server on the same platform architecture.

If any of the databases or tables are stored on a transactional storage engine (such as InnoDB), a logicalmysqldump backup is taken instead.

Logical BackupChoosing this option forces a mysqldump backup that copies MySQL binary logs regardless of thestorage engine. MySQL binary logs track and save all database server transactions as a list of SQLstatements. To implement a logical backup strategy, binary logging must be enabled on the MySQLserver, and a path to the log files must be supplied (the default is /var/lib/mysql ) in the BinaryLog Path field.

Logical backup method works with all MySQL storage engines except the MySQL cluster NDBstorage engine. Logical backups can also be restored to any platform architecture or database thatsupports SQL.

Because logical backups require a read lock on the database(s) or tables being backed up, they can havea greater impact on the applications using the MySQL database. Logical backups also result inincreased restore times, as restoring the data is accomplished by re-playing the transactions against thetarget database instead of just copying files.



11/27

Raw-Quick (No-Copy) BackupAs its name suggests, this type of backup is the quickest on offer, but it is only available with rawbackups using snapshots (offered by ZFS and other filesystems). When the Quick (No-copy) snapshotoption is enabled, ZRM for MySQL uses the snapshot itself as the backup rather than transferring thedata into a standard backup archive on the ZRM server.

Quick snapshot backups are appropriate for large databases and for databases that have high transactionrates. In addition to eliminating data transfer bottlenecks during backup, quick snapshot backups alsoprovide much faster database restoration than other backup methods.

However, because quick snapshot backups do not copy the data off of the MySQL server, they do notprotect data against server media failure. For this reason, quick snapshot backups can be converted atany time into standard backups stored on the ZRM server by using the Convert Backup option on theReports menu tab.

Sun Microsystems' ZFS file system includes snapshot capability, which facilitates near-instantaneoushot backups and rapid restores. ZRM makes use of this functionality by using an optional snapshotplug-in that integrates ZRM for MySQL with ZFS. It creates temporary snapshots of the ZFS volumeson which to perform a full backup. When snapshots are enabled, ZRM for MySQL can perform

backups with minimal impact on MySQL applications. Database writes will be blocked only duringsnapshot creation, which typically takes less than a second regardless of database size.



12/27

Performance Testing Setup

System SetupThis section describes the system architecture and configuration implemented for the performance test.

The hardware used during this test was a Sun Fire X4500 server with two dual-core AMD Opteron 285CPUs running at 2593 MHz with 16 Gbyte of RAM and 48 SATA drives of 512 Gbyte each.

The two boot drives are mirrored using ZFS volume management capabilities and the remaining 46drives are available for backup data.

As a general rule, no specific OS tuning was done on the system, nor were the binaries of both MySQLand ZRM optimized in any way. The objective of the tests was to deploy the solution out-of-the-box.

The only tuning performed was to limit the amount of memory available to the ZFS ARC cache to4 Gbyte by adding the following line to /etc/system :

set zfs:zfs_arc_max = 4294967296

MySQL Software ReleaseThe performance tests were carried out using MySQL 5.1 for Solaris 10 x64 (or x86_x64)./opt/mysql/mysql/bin/mysql Ver 14.14 Distrib 5.1.30, for pc-solaris2.10 (x86_64) usingreadline 5.1

OpenSolaris Operating System ReleaseOpenSolaris 2008.11 snv_101b_rc2 X86

Copyright 2008 Sun Microsystems, Inc. All Rights Reserved.Use is subject to license terms.

Assembled 19 November 2008

Zpool ConfigurationAs seen previously, the storage available on the Sun Fire X4500 server was split into two pools:

A backup pool made of 24 drives and two spares in a RAIDZ2 configuration. This provided11 Tbyte of capacity for backups with a very high level of protection against drive failure andperformance sufficient to handle 300 Mbyte/sec to 350 Mbyte/sec of backup traffic over thenetwork.

A mysql pool made of 12 drives and two spares in a RAIDZ2 configuration was used to storethe MySQL databases.



13/27

ZFS ConfigurationThree ZFS file systems were created:

zmanda/zrm to hold the ZRM backups zmanda/logs to hold the MySQL binlogs pool2/mysql to hold the MySQL data files

ZRM Software ReleaseThe following software components were installed:

The ZRM 3.0 for MySQL software Zmanda Management Console with MySQL, PHP, Apache, PHPmyadmin, Perl and various

Perl modules and their dependencies. These are installed under the /opt/zmanda/zrm and/opt/zmanda/ common directories.

ZRM was configured with: Level 0 backups No compression

db_STRESS Load Generator db_STRESS is an open source tool ( http://dimitrik.free.fr/db_STRESS.html ) that can be usedto quickly generate large MySQL databases. Installing and configuring db_STRESS for our particularneeds involved the following steps.

1. Creating the test databases- small_isam- med_isam

- large_isam

- small_innodb

- med_innodb

2. Creating test tables in pool2/mysql# /opt/mysql/mysql/bin/mysql -u root small_isam < BenchBASE_tab.sql# /opt/mysql/mysql/bin/mysql -u root small_innodb < BenchBASE_tab.sql# /opt/mysql/mysql/bin/mysql -u root large_isam < BenchBASE_tab.sql

# /opt/mysql/mysql/bin/mysql -u root med_innodb < BenchBASE_tab.sql# /opt/mysql/mysql/bin/mysql -u root med_isam < BenchBASE_tab.sql

http://dimitrik.free.fr/db_STRESS.htmlhttp://dimitrik.free.fr/db_STRESS.html


14/27

3. Generating the dataset# /opt/db_STRESS-kit/bin/bench64 3821145 to create a dataset of roughly 10GB# /opt/db_STRESS-kit/bin/bench64 38211450 to create a dataset of roughly 100GB# /opt/db_STRESS-kit/bin/bench64 382114500 to create a dataset of roughly 1000GB

4. Loading the dataset into the tables# /opt/db_STRESS-kit/mysql/load_mysql.sh small_isam# /opt/db_STRESS-kit/mysql/load_mysql.sh small_innodb# /opt/db_STRESS-kit/mysql/load_mysql.sh med_isam# /opt/db_STRESS-kit/mysql/load_mysql.sh med_innodb# /opt/db_STRESS-kit/mysql/load_mysql.sh large_isam

5. Creating usersCreating a user called dim with password dimitri and giving this user access to all the databases:mysql> GRANT ALL PRIVILEGES ON *.* TO 'dim'@'localhost' IDENTIFIED BY 'dimitri' with GRANTOPTION;

6. Creating a test scenarioWe created two different scenarios corresponding roughly to 15 percent CPU utilization of the systemby starting one WebX client and 90 percent CPU utilization by starting 30 WebX clients. (Note: WebXis a software package bundled with the open source db_STRESS package.)

Scenario 1:Starting one WebX client with the following command generated about 15 percent CPU utilization:

WebX x.db_STRESS -AntiDEAD 2000 -MAX 100000 -Base $dbname@localhost -User "dim" -Passwd"dimitri" -ID 0 -Group Level20 -Timeout 1 -TotalTIME $duration -RW 0 -SHMKEY 555 -MaxID2000

Scenario 2:Starting 30 WebX clients, all with the following command generated about 90 percent CPU utilization:

WebX x.db_STRESS -AntiDEAD 2000 -MAX 100000 -Base $dbname@localhost -User "dim" -Passwd"dimitri" -ID 0 -Group Level20 -Timeout 1 -TotalTIME $duration -RW 0 -SHMKEY 555 -MaxID2000

Testing MethodologyTesting the performance of ZRM 3.0 for MySQL was done by conducting backups of several localMySQL databases with tables of various sizes and storage engines, with the database either idle, 15percent or 90 percent busy.

Each backup was performed as a Level 0 backup with no compression and three types of backups wereperformed:

Raw-quick backups using ZFS snapshots Raw-copy backups Logical backups



15/27

Each type of backup was conducted on the main two storage engines available to MySQL tables: MyISAM InnoDB

What was measured was the overall time taken by ZRM to complete the backup, as reported by ZRMusing the mysql-zrm-reporter command:

mysql-zrm-reporter --where backup-set=$dbname --output report.$dbname.$i --fieldsbackup-date,backup-set,backup-size,backup-time,read-locks-time,flush-logs-time,backup-type,raw-databases,raw-databases-snapshot,logical-databases,backup-status

Each test was carried out three times in a row (except for the tests with the large database) and both theslowest and fastest results were discarded.

The following actions were taken before each measured run: Delete all previous backups from /var/lib/mysql-zrm . Delete all previous ZFS snapshots created by ZRM. Stop all active MySQL clients (the db_STRESS clients). Stop the MySQL server. Unmount / mount the ZFS file systems, in order to remove any caching effect from ZFS from

consecutive runs. Start the MySQL server.

Note: Time did not allow us to conduct recovery tests.



16/27

Performance Test ResultsA test result marked as" n/a "means that this particular test took too long to complete in a reasonabletime and was therefore not measured.

1. MyISAM Results with Idle DatabaseThe first set of results were obtained by measuring the throughput while backing up idle databasesmade of various MyISAM tables sizes.

Table 1: MyISAM Results with Idle Tables

Backup Type Table Engine Size (GB) Time (s) Throughput (MB/s)

raw-quick MyISAM 10.66 1 10,915.84

raw-copy MyISAM 10.66 68 160.53

logical MyISAM 11.44 575 20.37

raw-quick MyISAM 117.41 1 120,227.84

raw-copy MyISAM 117.41 759 158.40


raw-quick MyISAM 1065.84 1 1,091,420.16

raw-copy MyISAM 1065.84 6467 168.77

logical MyISAM 1065.84 n/a 0.00

The first observation that can be made here is that the raw-quick type of backup, which uses the ZFSsnapshot capabilities, can be not only very fast but independent of the size of the dataset.

The second observation is that a logical backup, also known as hot-backup, takes a lot longer tocomplete than a raw-copy backup, but of course enables the backup to take place while the databaseis still available to other users.

2. InnoDB Results with Idle DatabaseThe second set of results were obtained by measuring the throughput while backing up idle databasesmade of various InnoDB tables sizes.



17/27

Table 2: InnoDB Results with Idle Tables


raw-quick InnoDB 10.82 1 11,079.68

raw-copy InnoDB 10.82 91 121.75

logical InnoDB 8.08 411 20.13

raw-quick InnoDB 300.41 4 76,904.96

raw-copy InnoDB 300.41 2314 132.94

logical InnoDB 300.41 n/a n/a

Here as well, we can see that the raw-quick type of backup makes the most of the ZFS snapshotcapabilities.

We can also see that a raw-copy backup of an InnoDB table has a lower throughput compared to aMyISAM table. This is because behind the scenes, ZRM uses mysqlhotcopy when dealing withMyISAM tables and mysqldump for InnoDB tables. The logical backups however are consistent onboth types of storage engines.


Figure 2: Throughput Represented on a Logarithmic Scale: With Idle DatabaseMyISAM quick MyISAM copy MyISAM logical InnoDB quick InnoDB copy InnoDB logical

10

100

1,000

10,000

100,000

1,000,000

10,000,000

10,920

161

20

11,080

122

20

120,230

158

20

76,905

133

1,091,420

169

Throughput with Idle Database

smallmediumlarge

T h r o u g

h p u

t ( M B / s )


18/27

3. MyISAM Results with a 15 Percent Load on the Database The next set of results involved repeating the same measurement as previously, but this time whiledb_STRESS was running MySQL queries on the databases generating a 15 percent load.

First we measured MyISAM results.

Table 3: MyISAM Results with a 15 Percent Load


raw-quick MyISAM 10.66 11 992.35






raw-quick MyISAM 1065.84 102 10,700.20

raw-copy MyISAM 1065.84 9622 113.43

logical MyISAM 1065.84 n/a n/a

Raw-quick backups are again much quicker than all other types of backup in this scenario, but

because we're backing up a live database under load, we can see the effect of the delay while we'rewaiting to grab the lock on the tables before performing the snapshots.

The impact of the 15 percent load is quite noticeable on the backup throughput compared to the figuresfor an idle database but it is still quite reasonable.



19/27

4. InnoDB Results with a 15 Percent Load on the Database Then we move on to InnoDB results.

Table 4: InnoDB Results with a 15 Percent Load

Backup Type Table Engine Size (GB) Time (s) Throughput (MB/s)raw-quick InnoDB 10.66 67 162.92



raw-quick InnoDB 300.41 34 9047.64

raw-copy InnoDB 300.41 2661 115.60


As mentioned above, raw-quick backups are again much quicker than all other types of backup inthis scenario, but because we're backing up a live database under load, we can see the effect of thedelay while we're waiting to grab the lock on the tables before performing the snapshots.

The impact of the 15 percent load is quite noticeable on the backup throughput compared to the figuresfor an idle database but it is still quite reasonable.



20/27

5. MyISAM Results with a 90 Percent Load on the Database Finally, for the last set of results we used db_STRESS to heavily load the databases at 90 percent loadand repeated all the measurements.

First are the MyISAM results.

Table 5: MyISAM Results with a 90 Percent Load





raw-quick MyISAM 106.6 40 2,728.96




Figure 3: Throughput Represented on a Logarithmic Scale: With Database at 15% Load

MyISAM quick MyISAM copy MyISAM logical InnoDB quick InnoDB copy InnoDB logical10

100

1000

10000

100000

992

115

20

465

72

18

360

85

18

9048

116

18

10700

113

Throughput with Database at 15% Load

smallmediumlarge

T h r o u g

h p u

t ( M B / s )


21/27


raw-quick MyISAM 1065.84 876 1,245.91

raw-copy MyISAM 1065.84 72202 15.12

logical MyISAM 1065.84 n/a n/a

Apart from the raw-quick backups, we can see here that the overall throughput is well below theprevious results, since the backup process is competing with the mysqld process for CPU time.

In almost all use-cases, it would be a much better strategy to off-line such a heavily loaded databasebefore taking a backup.

6. InnoDB Results with a 90 Percent Load on the DatabaseThen come the InnoDB results.

Table 6: InnoDB Results with a 90 Percent Load






raw-copy InnoDB 300.41 7722 39.84


The first raw-quick backup result on the 30.43 Gbyte dataset seemed oddly out of line compared toother results collected.



22/27


Figure 4: Throughput Represented on a Logarithmic Scale: With Database at 90% Load

MyISAM quick MyISAM copy MyISAM logical InnoDB quick InnoDB copy1

10

100

1,000

10,000

79

15

3

3526

2,729

23

40

531

53

1,246

15

Throughput with Database at 90% Load

small

mediumlarge

T h r o u g

h p u

t ( M B / s )


23/27

SummaryThis guide is a quick introduction to configuring the Sun Fire X4500 server as a backup server forZmanda Recovery Manager 3.0. We have seen in this document how to deploy the Zmanda RecoveryManager for MySQL on a system running OpenSolaris. We have successively learned how to:

Create a ZFS storage pool using RAIDZ2 Create a ZFS file system Install packages using the new OpenSolaris Image Packaging System Deploy ZRM in this environment

We have demonstrated the use of ZRM's ZFS snapshot integration in protecting two of the majorMySQL storage engines under varying workloads to help better understand the value and positioning of this solution for Sun customers.

ZRM's usage of ZFS snapshot technology has shown its benefits to system and database administratorsas it can reduce the time needed to back up a database, independently of its size. Table 1 demonstratedthat the time needed to make a backup of a 1 TB MyISAM table could be reduced from 1h47mn to 1s.In this test, we could take a backup of a 1 Tbyte database close to 100 percent utilization in just a fewminutes, or only a few seconds if the database were not heavily loaded. With ZRM's capability toconvert backups, nearline snapshots can be copied to standard backup stored on the ZRM backupserver without interfering with database utilization. (Please note: Results may vary.)

Product Information

OpenSolaris Operating SystemOpenSolaris 2008.11 OS supports Dynamic Tracing (DTrace), ZFS, cryptographic infrastructures, IP

filter and User and Process Rights Management, and Predictive Self-Healing, which can furtherenhance system reliability.

The ZFS File SystemZFS is a 128-bit general-purpose file system that spans the desktop to the data center, where space isshared dynamically between multiple file systems from a single storage pool. Physical storage can beadded to or removed from storage pools dynamically, without interrupting services. This capability canprovide new levels of flexibility, availability, and performance.

The Sun Fire X4500 Server The Sun Fire X4500 server has reached end-of-life and is no longer orderable. It is superceded by thenext-generation Sun Fire X4540 Server . The Sun Fire X4500 server can deliver the performance of afour-way x64 server and provides 48 SATA drives with a total capacity of 48 Tbyte in 4U of rack space. Its bigger brother, the Sun Fire X4540 server, is powered by two Quad-Core AMD OpteronCPUs operating at 2300 MHz and provides the same amount of storage capacity.

http://www.sun.com/servers/x64/x4540/http://www.sun.com/servers/x64/x4540/


24/27

Zmanda Recovery Manager 3.0 for MySQLZmanda developed ZRM for MySQL with the explicit goal of providing simple, secure, and consistentbackups with minimal impact on applications and users. With ZRM for MySQL, DBAs can deployMySQL databases in mission-critical, high-transaction environments with confidence because theyknow their data is protected by Zmanda. True to its open source mission, ZRM doesn't use anyproprietary data formats. Users can recover their MySQL data even without ZRM being installed.

ZRM eases day-to-day backup and recovery tasks with its web-based Zmanda Management Console(ZMC). The management console integrates with the Zmanda Network to provide on-linedocumentation and other support services for ZRM. Because it is open source and supports commonstandards, as well as open data formats for backups, ZRM can free you from vendor lock-in.

Enabling advanced backup and recovery practices, such as raw, logical or hot-backups (Back up LiveMySQL databases without impacting applications and users), ZRM provides Continuous DataProtection (CDP) for MySQL by combining file system-level snapshots along with data from MySQLbinary logs. File system snapshots help reduce database server downtime, especially when workingwith large databases. ZRM uses a flexible framework of plug-ins for taking snapshots on a range of operating platforms.

Zmanda offers three versions of ZRM for MySQL: the ZRM Enterprise edition, ZRM Cluster editionand ZRM Community edition. The Enterprise edition includes a graphical installer, ZmandaManagement Console, command-line interface and plug-in framework.

About the Author Caryl Takvorian is currently a Staff Engineer at Sun Microsystems. He joined Sun in 1998 in what isnow known as ISV Engineering where he assists ISVs to make the best use of Sun's technologies. Histechnical skills are mainly in the Solaris OS and multithreaded applications and development tools.Caryl also leads the Telecommunications segment for ISV Engineering.

For More InformationZmanda Documentation

Zmanda Recovery Manager for MySQL web sitehttp://www.zmanda.com/zrm-mysql-enterprise.html , including ZRM datasheet (PDF): http://www.zmanda.com/zrm-datasheet.pdf

Zmanda Recovery Manager for MySQL is available for download from the Zmanda network:http://network.zmanda.com/

Zmanda Network documentation for Zmanda Recovery Manager (requires registration):http://network.zmanda.com/documentationEnterprise.php , includes:

ZRM: Guide to MySQL Backup & Recovery and Backup and Recovery Benchmark for MySQLwith ZRM :http://network.zmanda.com/zrm-whitepapers.php

BigAdmin article: Sun Fire X4540 Server as Backup Server for Zmanda's Amanda Enterprise2.6 Software : http://www.sun.com/bigadmin/features/articles/zmanda_sfx4540.jsp

http://mysqlbackup.zmanda.com/http://mysqlbackup.zmanda.com/http://mysqlbackup.zmanda.com/http://mysqlbackup.zmanda.com/http://www.zmanda.com/zrm-datasheet.pdfhttp://network.zmanda.com/http://network.zmanda.com/documentationEnterprise.phphttp://network.zmanda.com/documentationEnterprise.phphttp://network.zmanda.com/zrm-whitepapers.phphttp://www.sun.com/bigadmin/features/articles/zmanda_sfx4540.jsphttp://mysqlbackup.zmanda.com/http://mysqlbackup.zmanda.com/http://www.zmanda.com/zrm-datasheet.pdfhttp://network.zmanda.com/http://network.zmanda.com/documentationEnterprise.phphttp://network.zmanda.com/zrm-whitepapers.phphttp://www.sun.com/bigadmin/features/articles/zmanda_sfx4540.jsp


25/27

Backup and recovery tests for MyISAM and InnoDB engines with Zmanda Recovery

Manager for MySQL: http://solutions.mysql.com/solutions/white-papers/partner_wp_zmanda_backup.php

Sun Storage J4400 Array as Disk Storage for Zmanda Recovery Manager for MySQLDatabase: http://www.sun.com/bigadmin/features/articles/zrm_mysql_j4400.jsp

Sun Fire Server Information Sun Fire servers: http://www.sun.com/servers/x64/x4540/

Sun Fire X4500 server document collection:http://docs.sun.com/app/docs/prod/sf.x4500#hic

BigAdmin article: Using Sun Fire X4540 Server With Zmanda Recovery Manager 2.2 for MySQL Database :http://www.sun.com/bigadmin/features/articles/zrm_mysql_sfx4540.jsp

OpenSolaris Information OpenSolaris 2009.06 Image Packing System Guide (PDF):

http://opensolaris.org/os/project/pkg/files/ips.pdf

OpenSolaris web site: http://opensolaris.org

ZFS Documentation Solaris ZFS Administration Guide (part of the Solaris 10 System Administration Collection):

http://docs.sun.com/app/docs/doc/819-5461

Sun documents at http://docs.sun.com

ZFS Best Practices Guide on the Solaris Internals web site:http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

MySQL Documentation MySQL site on sun.com: http://www.sun.com/software/products/mysql/index.jsp mysql.com web site: http://www.mysql.com/ MySQL documentation site: http://dev.mysql.com/doc/ MySQL training courses: http://www.sun.com/software/products/mysql/training.jsp

MySQL download sites:

Get MySQL: http://www.sun.com/software/products/mysql/getit.jsp MySQL Downloads: http://dev.mysql.com/downloads/

http://solutions.mysql.com/solutions/white-papers/partner_wp_zmanda_backup.phphttp://solutions.mysql.com/solutions/white-papers/partner_wp_zmanda_backup.phphttp://www.sun.com/bigadmin/features/articles/zrm_mysql_j4400.jsphttp://www.sun.com/servers/x64/x4540/http://docs.sun.com/app/docs/prod/sf.x4500#hichttp://www.sun.com/bigadmin/features/articles/zrm_mysql_sfx4540.jsphttp://opensolaris.org/os/project/pkg/files/ips.pdfhttp://opensolaris.org/http://docs.sun.com/app/docs/doc/819-5461http://docs.sun.com/http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guidehttp://www.sun.com/software/products/mysql/index.jsphttp://www.mysql.com/http://dev.mysql.com/doc/http://www.sun.com/software/products/mysql/training.jsphttp://www.sun.com/software/products/mysql/getit.jsphttp://www.sun.com/software/products/mysql/getit.jsphttp://dev.mysql.com/downloads/http://dev.mysql.com/downloads/http://solutions.mysql.com/solutions/white-papers/partner_wp_zmanda_backup.phphttp://solutions.mysql.com/solutions/white-papers/partner_wp_zmanda_backup.phphttp://www.sun.com/bigadmin/features/articles/zrm_mysql_j4400.jsphttp://www.sun.com/servers/x64/x4540/http://docs.sun.com/app/docs/prod/sf.x4500#hichttp://www.sun.com/bigadmin/features/articles/zrm_mysql_sfx4540.jsphttp://opensolaris.org/os/project/pkg/files/ips.pdfhttp://opensolaris.org/http://docs.sun.com/app/docs/doc/819-5461http://docs.sun.com/http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guidehttp://www.sun.com/software/products/mysql/index.jsphttp://www.mysql.com/http://dev.mysql.com/doc/http://www.sun.com/software/products/mysql/training.jsphttp://www.sun.com/software/products/mysql/getit.jsphttp://dev.mysql.com/downloads/


26/27

Discussions Zmanda forums: http://forums.zmanda.com/ MySQL forums: http://forums.mysql.com/ Sun Storage forums: http://forum.java.sun.com/category.jspa?categoryID=66 Sun Hardware - Servers forums: http://forums.sun.com/forum.jspa?forumID=830

Other Sun Resources Sun training courses at http://www.sun.com/training/ Sun BluePrints wiki ( http://wikis.sun.com/display/BluePrints/Main , especially the

following:

Storage BluePrints page:http://wikis.sun.com/display/BluePrints/Storage+BluePrints

Sun BluePrints article: Optimize MySQL Server on Sun x64 Servers and Storage :http://wikis.sun.com/display/BluePrints/Optimize+MySQL+Server+on+Sun+x64+Servers+and+Storage

Sun Storage Administration wiki:http://wikis.sun.com/display/StorageAdmin/Home

BigAdmin Storage Tech Tips wiki:http://wikis.sun.com/display/BigAdmin/Storage+Tech+Tips

BigAdmin Storage Resource Collection (includes community content):http://www.sun.com/bigadmin/collections/storage.html

Sun Storage web site: http://www.sun.com/storagetek/index.jsp Storage Stop Blog: http://blogs.sun.com/storage

Support Sun resources:

Register your Sun gear: https://inventory.sun.com/inventory/ Services: http://www.sun.com/service/index.jsp SunSolve SM Online: http://sunsolve.sun.com

Community system administration experts: http://www.sun.com/bigadmin/content/communityexperts/

Events of interest to users of Sun products Sun and MySQL Around the WorldMeetup Mashup:

http://www.sun.com/events/mysqltour/index.jsp

Find a MySQL Meetup Group near you: http://mysql.meetup.com/

http://forums.zmanda.com/http://forums.mysql.com/http://forum.java.sun.com/category.jspa?categoryID=66http://forums.sun.com/forum.jspa?forumID=830http://www.sun.com/training/http://wikis.sun.com/display/BluePrints/Mainhttp://wikis.sun.com/display/BluePrints/Storage+BluePrintshttp://wikis.sun.com/display/BluePrints/Optimize+MySQL+Server+on+Sun+x64+Servers+and+Storagehttp://wikis.sun.com/display/BluePrints/Optimize+MySQL+Server+on+Sun+x64+Servers+and+Storagehttp://wikis.sun.com/display/StorageAdmin/Homehttp://wikis.sun.com/display/BigAdmin/Storage+Tech+Tipshttp://www.sun.com/bigadmin/collections/storage.htmlhttp://www.sun.com/storagetek/index.jsphttp://blogs.sun.com/storagehttps://inventory.sun.com/inventory/http://www.sun.com/service/index.jsphttp://sunsolve.sun.com/http://www.sun.com/bigadmin/content/communityexperts/http://www.sun.com/events/mysqltour/index.jsphttp://mysql.meetup.com/http://forums.zmanda.com/http://forums.mysql.com/http://forum.java.sun.com/category.jspa?categoryID=66http://forums.sun.com/forum.jspa?forumID=830http://www.sun.com/training/http://wikis.sun.com/display/BluePrints/Mainhttp://wikis.sun.com/display/BluePrints/Storage+BluePrintshttp://wikis.sun.com/display/BluePrints/Optimize+MySQL+Server+on+Sun+x64+Servers+and+Storagehttp://wikis.sun.com/display/BluePrints/Optimize+MySQL+Server+on+Sun+x64+Servers+and+Storagehttp://wikis.sun.com/display/StorageAdmin/Homehttp://wikis.sun.com/display/BigAdmin/Storage+Tech+Tipshttp://www.sun.com/bigadmin/collections/storage.htmlhttp://www.sun.com/storagetek/index.jsphttp://blogs.sun.com/storagehttps://inventory.sun.com/inventory/http://www.sun.com/service/index.jsphttp://sunsolve.sun.com/http://www.sun.com/bigadmin/content/communityexperts/http://www.sun.com/events/mysqltour/index.jsphttp://mysql.meetup.com/


27/27

Worldwide developer events and Sun Tech Days: http://developers.sun.com/events/ Current events: http://www.sun.com/events/index.jsp

Licensing InformationUnless otherwise specified, the use of this software is authorized pursuant to the terms of the licensefound at http://www.sun.com/bigadmin/common/berkeley_license.html .

f
http://developers.sun.com/events/http://www.sun.com/events/index.jsphttp://www.sun.com/bigadmin/common/berkeley_license.htmlhttp://developers.sun.com/events/http://www.sun.com/events/index.jsphttp://www.sun.com/bigadmin/common/berkeley_license.html

Zrm Mysql Zfs

Documents

Transcript of Zrm Mysql Zfs