Learner Guide - Kingscliff & Murwillumbah IT · It supports high-availability features like...

ICTPRG412

Configure and maintain databases

Learner Guide

© Copyright, 2016 by North Coast TAFEnow

Date last saved: 7 March 2016 by Amanda Walker Version: 1.0 # of Pages = 45

John Chapman – Content writer and course adviser

TAFEnow Resource Development Team – Instructional and

graphic design

Copyright of this material is reserved to the Crown in the right of the State of New South Wales.

Reproduction or transmittal in whole, or in part, other than in accordance with the provisions of the Copyright Act, is

prohibited without written authority of North Coast TAFEnow.

Disclaimer: In compiling the information contained within, and accessed through, this document ("Information")

DET has used its best endeavours to ensure that the Information is correct and current at the time of publication but

takes no responsibility for any error, omission or defect therein. To the extent permitted by law, DET and its

employees, agents and consultants exclude all liability for any loss or damage (including indirect, special or

consequential loss or damage) arising from the use of, or reliance on, the Information whether or not caused by any

negligent act or omission. If any law prohibits the exclusion of such liability, DET limits its liability to the extent

permitted by law, to the re-supply of the Information.

Third party sites/links disclaimer: This document may contain website contains links to third party sites. DET is not

responsible for the condition or the content of those sites as they are not under DET's control. The link(s) are

provided solely for your convenience and do not indicate, expressly or impliedly, any endorsement of the site(s) or

the products or services provided there. You access those sites and use their products and services solely at your

own risk.

Contents Getting Started .................................................................................................................................. i

About this unit .................................................................................................................................................................... i

Elements and performance criteria ............................................................................................................................. i

Icon Legends....................................................................................................................................................................... ii

Topic 1 ............................................................................................................................................... 1

Getting started ................................................................................................................................................................... 1

Database management .................................................................................................................................................. 3

Selecting management software ................................................................................................................................ 4

Installing MySQL ................................................................................................................................................................ 6

Topic 2 ............................................................................................................................................... 7

Introduction ........................................................................................................................................................................ 7

Configure database instance, components and services .................................................................................. 8

Monitor the database ...................................................................................................................................................... 9

Manage database security ...........................................................................................................................................10

Create and administer users .......................................................................................................................................11

Create and manage permissions to perform database operations .............................................................12

Create and manage roles .............................................................................................................................................13

Database integrity and optimisation .......................................................................................................................14

Perform data-management tasks .............................................................................................................................17

Summary ............................................................................................................................................................................19

Topic 3 – Backup and restore techniques ..................................................................................... 21

Introduction ......................................................................................................................................................................21

Backup/restore using phymyAdmin ........................................................................................................................22

Backup/restore using mysqldump.exe and mysql.exe .....................................................................................28

Automating backups .....................................................................................................................................................30

Summary ............................................................................................................................................................................30

Topic 4 ............................................................................................................................................. 31

Introduction ...................................................................................................................................................................... 31

Profiling .............................................................................................................................................................................. 32

Simple optimisations .................................................................................................................................................... 34

Indexes ............................................................................................................................................................................... 35

Partitioning ....................................................................................................................................................................... 36

Documentation ............................................................................................................................................................... 38

Summary ............................................................................................................................................................................ 39

Getting Started

About this unit

This unit describes the skills and knowledge required to install a database, manage data, data

access and data security, and improve database performance.

It applies to individuals responsible for the maintenance and coordination of database

operations. They usually work in an organisation, providing daily services as database

administrators, database developers, database coordinators, or application developers.

Elements and performance criteria

Elements define the essential outcomes of a unit of competency. The Performance Criteria

specify the level of performance required to demonstrate achievement of the Element. They

are also called Essential Outcomes.

Follow this link to find the essential outcomes needed to demonstrate competency in this

Unit: http://training.gov.au/Training/Details/ICTPRG412

i | P a g e I C T P R G 4 1 2 _ L G _ V 1 . 0 . D O C X

T A F E n o w

http://training.gov.au/Training/Details/ICTPRG412

Icon Legends

Learning Activities

Learning activities are the tasks and exercises that assist you in gaining a

clear understanding of the content in this workbook. It is important for you

to undertake these activities, as they will enhance your learning.

Activities can be used to prepare you for assessments. Refer to the

assessments before you commence so that you are aware which activities

will assist you in completing your assessments.

Readings (Required and suggested)

The required reading is referred to throughout this Learner Guide. You will

need the required text for readings and activities.

The suggested reading is quoted in the Learner Guide, however you do not

need a copy of this text to complete the learning. The suggested reading

provides supplementary information that may assist you in completing the

unit.

Reference

A reference will refer you to a piece of information that will assist you with

understanding the information in the Learner Guide or required text.

References may be in the required text, another textbook on the internet.

ii | P a g e I C T P R G 4 1 2 _ L G _ V 1 . 0 . d o c x T A F E n o w

Topic 1

Getting started

Welcome to ICTPRG412 - Configure and maintain databases. This unit will give you the skills

and knowledge required to install a database, manage the data, control access, and improve

database performance. This unit is applicable to people who work as database administrators,

database developers, database coordinators or application developers or anybody whose

responsibilities include the maintenance and coordination of database operations.

The unit is very action oriented - the unit guides are almost all one large activity. So you will

not see the activity icon used, but you should be doing all of the activities described in the

unit guides to get the most out of this unit.

To do these tasks you’ll need to be able to:

> Evaluate interdependencies between data fields

> Examine and define system requirements for data

1 | P a g e I C T P R G 4 1 2 _ L G _ V 1 . 0 . D O C X

T A F E n o w

> Review requirements and determine the availability of information

> Interact with developer to determine data requirements

> Liaise with management on report and result matters

> Read and interpret specifications developed by business and technical experts

> Write a report

> Address common issues relating to the operation of a computer system

> Perform basic troubleshooting to diagnose system problems

> Perform basic data management such as insert, update and delete operations.

Before you start this unit, you’ll need access to a PC running Windows, OSX or Linux, and basic

knowledge of:

> database design

> desktop operating system

> relational databases

> computer hardware and components

> computer networking and components

> Structured Query Language (SQL)

After completing this first topic, understanding and installing database administration tools,

you will be able to:

> install and configure database software

> manage database files, integrity, users and security

> create backup and restore databases

> monitor database performance

Introduction

To configure and maintain databases, we will need a database management program. This

can have a command-line or graphical interface.

But first, let’s review the basics of relational databases.

A database is a container that holds structured data tables. These tables can be thought of

like spreadsheets. Each table consists of records (analogous to rows in a spreadsheet).

Records consist of fields (like columns in a spreadsheet) which contain the data.

2 | P a g e I C T P R G 4 1 2 _ L G _ V 1 . 0 . d o c x T A F E n o w

Figure 1 – Logical relationships in database architecture

A database program usually runs as a service (i.e. in the background) and can serve many

different databases. This running program is called a database instance.

The actual data for the database resides on some sort of storage - usually a disk. Users (and

applications) communicate with it via the database program via network connections.

While we’re focussing on relational databases due to their popularity, it should be noted that

there are also specialised databases like document databases, key-value stores, and self-

contained disk-only databases like SQLite.

Common database operations which operate on the data are Create, Read, Update and Delete

(acronym CRUD). You may have covered those concepts in other units. This unit considers

configuration and management of the database.

Database management

Database management is distinct from database design and usage and entails:

> Installing and configuring database software

> Managing databases, database integrity, users and security

> Creating backups of and restoring databases

> Monitoring database performance


T A F E n o w

> Optimising the database

With that in mind, we will need software to help us perform these tasks by communicating

with the database service instance on our behalf.

Figure 2 - Connectivity relationships between database applications, server and physical storage

Selecting management software

Even though SQL is a standard language, all database vendors adhere to the standard to

varying degrees and also extend it. This means that management software works with only

the database it was designed to work with. Thus, all popular databases come with their own

management software. This software typically provides a graphical user interface to many of

the underlying management tools.

You typically select database software for a project based on project requirements

(functionality, performance, scalability, cost, etc.) and you’re stuck with the management

software that comes with that database.

Popular database software includes:

> Microsoft’s SQL Server

> Oracle database

> MySQL


For this unit, we will be using the popular and powerful MySQL and its companion

management program called MySQL Workbench. MySQL is a powerful feature-rich database

and suite of tools. It supports high-availability features like replication, caching, partitioning,

and migration. Incredibly, MySQL is open source so it is free to use, free to distribute, and

there is no sign up, subscriptions, or advertisements. It's also cross-platform.

MySQL suite

MySQL is available for Windows, Linux, and OSX. See the document Installing XAMPPV1.0 to

install:

> Apache server

> MySQL

There are also packages for various Linux distributions at that URL if your distribution's

package manager doesn't have version 5.6 or higher.

There is an entire suite of software available from the MySQL project. What you need for this

unit is: XAMPP. At the time of this writing, these were available for Windows in a single

installation file.

Technically, you don’t actually need to be running a MySQL server if you have access to an

existing database server somewhere on your network or on the internet. But it will be a good

idea since we can populate a local server with our own test data, without fear of breaking

something important in an existing system.

XAMPP/LAMP/WAMP/ and phpMyAdmin

Another popular MySQL database management program is called phpMyAdmin. Being PHP

based, this program is run by a web server and is interfaced to via a browser. If you end up

managing the database for a website, chances are good that you will use phpMyAdmin for

management.

If you would like to use phpMyAdmin on your local machine to learn about database

configuration and management, look into a suite of open source programs called WAMP. The

“AMP” stands for Apache (the web server), MySQL (the database), and PHP (the language) and

means that all three have been packaged together. The “W” stands for Windows. “LAMP” is

available for Linux.


T A F E n o w

https://share.tafensw.edu.au/share/file/e482fb2a-0ef1-4f14-a697-2aa1a5dbe4fc/1/Installing%20XAMPP_V1.0.pdf

The “AMP” software stack is an entire web server, all free and open source. Installing this will

allow you to install and run MySQL and phpMyAdmin.

Generality

You will find the concepts similar between all of these management programs.

For simplicity, it is recommended you use MySQL and its associated management program

XAMPP for this unit.

Installing MySQL

Use the document about Installing XAMPP from the Database Bookshelf on the Learning and

activity tab for this unit in TAFEnow.


Topic 2

Introduction

There are hundreds of settings for a MySQL server. We will look at a few important ones. We

will also set up a new user, exploring the difference between privileges and roles as we go.

We will look at easy database maintenance tasks, like checking the database integrity and

optimising tables. Finally, we will look at ways to easily change data in the database using

management tools.

Also, the term “server” sometimes refers to the software providing a service to the user such as

a database, and sometimes refers to the hardware that software is running on. Currently

running server software is sometimes also called an instance (for example by MySQL), or a

service (for example by Windows). Throughout this unit we will simply use the term “server”

for the MySQL database software which is running in the background and connecting us to

the underlying database files.


T A F E n o w

Configure database instance, components and services

The database instance is the actual database program running in the background and waiting

for connections from client software. It may also be called a server or service. Configuring the

database instance is done through the MySQL Workbench.

Use the document XAMPP Features in the Database Bookshelf on the Learning and activity

section for this unit in TAFEnow to get an introduction to XAMPP.

Start the server using the XAMPP control panel and start both:

> Apache

> MYSQL

Figure 3 - The XAMPP control panel, showing both Apache and MySQL are running.

Open XAMPP typing http://localhost/xampp in a browser address bar.


http://localhost/xampp

Figure 4 – The XAMPP homepage

Monitor the database

A major part of maintaining databases is monitoring their health and performance. XAMPP

offers many ways to see your database server's health.

Many of the sections to monitor are included in the document XAMPP FeaturesV1.0.

These include:

> Status - click status in the navigation at left

Figure 5 – The status of a XAMPP server

This shows that

> MySQL is activated

> PHP – the scripting language often used with MySQL, allows webpages to display MySQL

data


T A F E n o w

> SSL secure sockets layer is configured

> CGI is not configured as it is not required on this server

> SSI is activated and is used with PHP

Documentation for the server is provided via the documentation link in the navigation.

Manage database security

Access to the databases is by user, and each user can perform particular actions (also known

as rights or privileges) on particular tables. The root user, set up during installation, has all

rights on all tables. We will be covering database security configuration further as we progress

through this topic.

It is really important that you consider what security requirements are when creating and

maintaining a database. Never assume these requirements, seek advice and input from the

owner of the database so that the data is correctly secured. Seemingly insignificant databases

may have security requirements that are not apparent, e.g. commercially sensitive

information, compliance with legislation (e.g. Privacy Act) or internal business policies.

Click security in the navigation bar at left


XAMPP rates each feature as

> Unsecure – the first one being that on this home network, pages are viewable by anyone.

> Secure - root has a password, phpMyAdmin has a password.

> Unknown - FTP is not installed, so XAMPP does not know its status.

Click http://localhost/security/xamppsecurity.php to add passwords for:

> root

> phpMyAdmin

Create and administer users

Login to phpMyAdmin.

Click the users tab at the top of the screen

Figure 6 – The XAMPP user in a system

To create a new user:

Click add User


T A F E n o w

http://localhost/security/xamppsecurity.php

In the first section, the login information is requested.

Type the name of the user e.g. newUser

Type the hostname e.g. localhost

Type a password and re-type it in the next field or generate one.

Create and manage permissions to perform database operations

The next section offers that chance to create a database especially for the user.

This is usually ignored – it is a security risk for general users!


The most significant part of the page is further below.

In this section, the access for the newUser is created.

In most business cases, a new user will only have access in the Data section.

Some will even have SELECT only – so they can’t change any data.

Typically, a user will have INSERT and UPDATE as well.

Check SELECT in the Data section and click Go

We have now created a user with the absolute minimal set of privileges required to do their

job. This is a good philosophy when it comes to security – minimal privileges reduce the

chance that a user will be able to cause damage or access data they are not supposed to.

Create and manage roles

By careful selection of privileges, users can be given access to a database so that they do not

have any more rights than is appropriate.

In most cases, users will have access such as:

1 SELECT only. This is the most restrictive access and allows users the right to view data.

2 SELECT, INSERT, UPDATE. This allows users to add and modify existing data.


T A F E n o w

Note: both of these allow access to data only. It is unlikely that any further access is required,

for these will change the structure of the database, which may have disastrous results!

Database integrity and optimisation

All industrial strength databases include tools to verify the integrity of the database and/or

tables within a database. In the case of MySQL, there is both a MySQL Workbench way of

checking table integrity, and a command line way of checking. We will look at both.

By “integrity”, we are referring to the consistency and correctness of the data and structure

within the database. A bug in the storage engine behind the database (InnoDB or MyISAM in

the case of MySQL) would be one cause of errors within the database. Data errors from

problems with storage might be another.

Note that the database ‘world’ is created using the instructions in the document XAMPP

Features in the Database Bookshelf on the Learning and activity section for this unit in

TAFEnow; you should have created this earlier as part of the introduction to XAMPP.

To check the integrity of the table

Select the table, world.

On the structure tab, a list of tables within the database “world” is presented. At the bottom

we see a Check All box.

Check this.

In the ‘with selected’ dropdown choose:

Check table

Now we will select all three tables within the database and

Click on the Analyze Table button.

If all is well with the tables, a screen like this appears:


Click OK. Let’s also run “Check Table”:

In a similar fashion, choose Analyse table.

A similar output result of the word “check” instead of “analyze”:


T A F E n o w

Another option is to “Optimize Table” which reorganises the physical storage of table data

and associated index data to reduce storage space and improve I/O efficiency when accessing

the table.


There are no features of these tables to support optimisation. This depends on other

properties, not included.

Perform data-management tasks

Data management can mean many things. This concerns getting data into and out of the

MySQL database when we look at how to perform backups later in the Unit Note for Topic 3.

For now, let’s explore how to use Excel and phpMyAdmin to interact with data in databases,

and we’ll see how we can even edit data from within Excel.

Editing data with Excel

Use the import data function in phpMyAdmin. This is demonstrated in the XAMPP Features

document in the Database Bookshelf on the Learning and activity section for this unit in

TAFEnow.

Editing data with XAMPP

This is not the intended method of modifying data! The more usual method is to run a query,

but those with administrative rights run the risk of compromising data this way.

Select the country table in the database browser.

Change the population of Australia to 24 000 000. To do this:

Double click the population. The field becomes editable.

Enter 24000000

Note the feedback - one row affected.

Export query results


T A F E n o w

We can export this query result as a Comma Separated Values (.csv) format file. This is a very

generic plain text file format, readable by many different tools, including Excel, MS Access,

and many programming languages.

To export, the contents of a table,

Select the table in the database browser

Click the Export tab at the top

In the Format dropdown:

Choose CSV

Click Go.

Locate the file you just exported, and open it with a text editor to see the format and the

contents. The file will be in the default downloads folder for your browser.


Editing data with SQL

The alternative here is to run an SQL query. Change the population of Australia to 23 000 000

as follows:

Go to the SQL tab.

Type UPDATE country SET couPopulation=23000000 WHERE countryID=12

Click Go

Click Browse to verify the change

This is a more controlled way of making changes to data.

Summary

We’ve seen how to use XAMPP to perform several database configuration and management

tasks. We’ve seen how to monitor the server, change server settings, and manage the data

within the database. In the next Topic, we’ll be looking at backup and restore techniques.


T A F E n o w

Topic 3 – Backup and restore techniques

Introduction

In this topic we’ll explore several ways to backup and restore MySQL databases, with an

emphasis on reliability and uptime. Backing up MySQL could refer to backing up the data, the

database structure, or even the Workbench settings. For our purposes, we’ll assume

something of a real-life application where the real value is in the entire database (structure

and data) and we’re trying to ensure our organisation can quickly recover from a disaster

(hardware failure, theft, data loss, etc.) with minimal loss of work, and minimal server

downtime.


T A F E n o w

Backup/restore using phymyAdmin

Concepts

At their most basic, backups of MySQL databases are simply SQL files filled with commands to

recreate the database structure and insert data into that structure. In the event we need to

restore, we simply direct MySQL to execute the backup file, like a script.

If MySQL’s binlog feature is turned on, all transactions to the database are being recorded.

This can be useful after restoring the database from the SQL backup files to apply the changes

made to the database since the last backup was made, thus bringing the database up to a

point just before the disaster. You could think of the SQL backups as full backups, and the bin

logs as incremental backups. See the mysqlbinlog command.

Backups with phpMyAdmin

Backing up using phpMyAdmin is done via the Export tab on the left side of the main window.

Select the database in the database list.

Click the Export tab

The dialog that follows confirms the export.


This is the easiest option:

> Leave the quick option selected

> Leave SQL as the format

Click Go

Use your browsers save dialog to complete the task.

By default, your browser use the downloads folder.

Open this file in a text editor to examine it.

This file is over 600 lines a sample of this appears below:


T A F E n o w

Here, the SQL commands to create the database, as it was when the export was done, are

visible.

There are:

> CREATE TABLE commands

> INSERT commands

> ALTER commands

As you can see, you get one SQL file per table. The prefix for all these files is the database

name, with the once prefixed “mysql” being internal MySQL tables containing things like

settings and users.


LEARNING ACTIVITIES ACTIVITY 1

Pick one of the SQL files and open it with a text editor to examine the contents.

If you are familiar with SQL, you will immediately recognise the contents.

Notice that the backup file contains both the database structure (the CREATE statements), and

the data (the INSERT statements). With any SQL-generating backup method, always make

sure you are getting both structure and data in the output.

The second backup method, “Export to Self-Contained File”, does the same thing, but just

outputs the entire contents (i.e. all databases) served by the instance to a single file.

There are advantages and disadvantages to each method. A single file might be easier to

compress, email, or otherwise transfer, but doesn’t give you the same control when restoring

as separate files will.

Restoring with phpMyAdmin

To restore the database, it will be necessary to import this file.

Of course, the database will need to have all its tables dropped to do this.

This confirms that the database is empty.


T A F E n o w

Go to the Import tab.

Use the file dialog, to browse for the backup file, by default, world.sql

Leave all options as the default

Click Go

If the import was successful, the SQL commands in the file were executed to give this result:

Click the Structure tab


All the tables will be returned as before.

The options on this screen are pretty self-explanatory, so click “Start Import” and observe the

progress. Pick a backup file corresponding to one of the example databases.

If you use MySQL’s binary logs feature, which save the transactions done to the database, you

can use the logs to bring a table up to the current time after a restore process.


T A F E n o w

REFERENCE REFERENCE 1

There’s a lot more to backups. You should do more research about some of the issues and

capabilities of MySQL.

> The Binary Log (MySQL, n.d.)

> Point-in-Time (Incremental) Recovery (MySQL, n.d.)

Backup/restore using mysqldump.exe and mysql.exe

Concepts

As we noticed, MySQL Workbench was using separate executables to perform the backup and

restore operations. Let’s see how to do that ourselves, which will be useful if we need more

control, or if we want to automate the process.

Will need the command line, so will run “cmd.exe” using Windows Key-R. Navigate to the

server binary files, which will vary depending on the installation. Or, put the binary folder in

your system’s path.

Now run mysqldump as follows.

Figure 7 – Running mysqldump from the command line

We’re dumping the “world” database, as the root user. The -p option says we want to be

asked for a password. When you enter your password and hit “enter”, your screen will fill with

SQL commands. That is not terribly useful. If you know the command line, you’ll know that

you can redirect this output to a file using the “>” character. Let’s do that:


http://dev.mysql.com/doc/refman/5.7/en/binary-log.html

http://dev.mysql.com/doc/refman/5.7/en/point-in-time-recovery.html

C:\> mysqldump –u root –p world > C:\Users\Greg\Desktop\world.sql

We’ve just made a backup. Notice the entire database has gone into a single file, not one

table per file as with MySQL Workbench.

This required manual entering of a password. That’s not acceptable for an automated

solution (for example in a script). This is where the backup user that we created in the first

Unit Guide comes in. Because that user only has read access to the database, hard coding the

password in a script is less risky than putting the root user’s password in a script. So the

backup command would look like this:

C:\> mysqldump –u backup –-password=Password world >

C:\Users\Greg\Desktop\world.sql

The mysqldump command is very flexible, allowing you to back up just one table within that

database, or all databases on the server (with –-all-databases). Use —-help to see the

documentation. Don’t forget about piping to command’s output to the more command with

|more to paginate.

In a real-world situation, the desktop is a bad place for backups. A major part of ensuring the

continuity of the business you’re responsible for is making sure something smart gets done

with those backup files. Saving them on different media (e.g. DVDs, SSDs), and at different

physical locations is essential.

Also, recovering to a time in the past is often important – what if it took a week for someone

to notice that 90% of a particular table had been deleted? Would the backup that contained

that data have been overwritten by subsequent backups?

REFERENCE REFERENCE 2

MySQL has a commercial version, and an enterprise backup product is available for that. (MySQL,

n.d.)

Restoring with MySQL

To restore the table from the file we created above we simply run:

C:\> mysql –u root –p world < C:\Users\Greg\Desktop\world.sql


T A F E n o w

http://www.mysql.com/products/enterprise/backup.html

We have to tell MySQL from the command line to use the database “world” because there is

no USE statement within that SQL file.

That’s all there is to it. The benefit of the command line is that once you have the commands

worked out, you can put them in scripts (or batch files) so they can be run automatically.

Unlike the GUI, a tested command line is more reliable since there’s no chance of ticking the

wrong box accidentally and creating useless backups.

Automating backups

The next step would be to automate backups so that they always run without any human

needing to remember them. This is best done using the command line utility mysqldump. In

windows, you would use power shell to create backups named after the date and time, and

possibly rotating the backups so that they didn’t fill up the disk. Then, using the task

scheduler, you would have this shell script run periodically.

Under Linux, you would create a shell script to do the same as above, and use a cron job to

run it periodically.

LEARNING ACTIVITIES ACTIVITY 2

If you are not familiar with scheduling tasks using the tools detailed here, use the internet or other

available resources to research use of scheduling tools on your preferred operating system and

record the basic steps for future reference.

Summary

We’ve seen simple ways to make backups and restore from them (Workbench, and

mysqldump/mysql), and we’ve seen a more sophisticated method (using MySQL’s replicate

feature). The MySQL documentation has a section on backup and recovery strategies.

(MySQL, n.d.).

In the next Topic, we’ll be looking at backup and restore techniques.


http://dev.mysql.com/doc/refman/5.7/en/backup-and-recovery.html

Topic 4

Introduction

In this topic we cover a few topics that don’t fit elsewhere – tuning and optimising the

database for performance, and documentation.

There are a few operations you can perform to optimise the database, and a few features

which you can set up to increase performance.

We will first look at profiling, which will allow us to measure database performance and see

what improvements our optimisations make. Then we’ll look at simple commands and

settings which might improve your databases performance and are part of database

maintenance. Then we’ll look at slightly more advanced topics of indexing, and partitioning.


T A F E n o w

You can think of performance tuning as falling into three categories:

> Server settings and features

> Database / table structure and organisation

> Database organisation on disk (physical)

We also cover documentation, which is a necessary evil in order to keep track of all these

changes you make to the database for yourself and for anybody who does these somewhat

tedious parts of your job for you in the future. It is also a requirement of any ICT professional

working in an enterprise environment.

Profiling

If we want to improve performance, we need to be able to measure it, so that we can see what

effect our optimisations and tuning have.

Profiling allows us to measure how long various components of a query take.

Go to the phpMyAdmin home page,

Click Variables

Scroll to profiling

Click Edit set to value to ON click save

Select the world database

Go to the SQL tab

Type SELECT * FROM country;

At the right Check profiling


If we re-run the same SELECT query, and examine its profile, we’ll see “Opening tables” takes

considerably less time with the second SELECT:


T A F E n o w

This could be because the file is now open on disk and can more quickly accept a second

SELECT statement. You should run several queries that are the same, and examine the

statistics for each. You’ll note that some of the parameters vary quite considerably between

queries, even though you haven’t changed any settings. Thus, running enough queries to get

a real baseline is essential for determining if your changes have a positive effect on database

performance.

In particular, “Sending data” and “statistics” seem to vary quite a bit.

Simple optimisations

Optimize table

This command reorganises the physical structure of table data and indexes. This is similar to

defragmentation of a physical storage device. Running it is as easy as this:

FROM the database browser select the database,

Choose a table. At the dropdown box at the bottom, choose optimise

Tuning key_buffer_size and table_open_cache

From the phpMyAdmin homepage go to the Variables tab.

Scroll to the values:

> keybuffe rsize

set this to 64MiB if it is set to 16MiB

> table_open_cache

this is set to 2000 by default. Leave this setting.

See this Tuning Server Parameters (MySQL, n.d.) for further information.


http://dev.mysql.com/doc/refman/5.7/en/server-parameters.html

Indexes

Concepts

In MySQL, indexes are sets of pointers to rows in your table which improve the speed of

SELECT statements. When we create an index we tell MySQL which columns of a table are

likely to be part of a WHERE clause.

Don’t create indexes for every column of the table since that would negate the performance

increase.

Creating indexes

Go to the structure tab of the country table.

Against the couName field

Click Index

Click ok to accept the ALTER TABLE

To create a new index, simply click in the space below PRIMARY in the Index Name column.

Name your index, and then choose a type. INDEX is the most common index type. You can

read about the others here (MySQL, n.d.).

MySQL has a powerful feature where it will analyse your SELECT statement and tell you if it is

using indexes

All we need to do is prepend the word EXPLAIN to the query as follows:

EXPLAIN SELECT * FROM `country` WHERE couName = 'Australia'


T A F E n o w

http://dev.mysql.com/doc/refman/5.7/en/optimization-indexes.html

We set the type to INDEX as that’s the most common and straightforward. Consult the

documentation to learn about the other types of indexes. Now, if we re-run our EXPLAIN

query:

In the “Extra” column we can see the index is now being used and fewer rows were consulted

(15 vs. 4080) in order to match the query.

Partitioning

Concepts

MySQL has the ability to physically split a table into two or more parts, storing them on

different physical locations. If these locations are different disks, significant throughput gains

can be realised - effectively parallelising your database operations.

Partitioning your database tables may yield another significant improvement - in backups. If

you use incremental backups and split your table so that new records go into one partition,

one of your partitions wouldn't be changing, which would reduce the size of your incremental

backups.

Tables are split horizontally, by rows. Vertical splits are possible, but not with MySQL. Vertical

splitting can manually be achieved by dividing a table into multiple tables (some columns in

table 1, the rest in table 2).

A table is partitioned by one of:

> HASH - MySQL uses a function you provide to determine which partition to pull rows

from.

> KEY - Like HASH, but MySQL uses the MD5 algorithm to calculate a hash.

> RANGE - You specify a range of column values which MySQL uses to determine which

partition the data is in.

> LIST - You specify values of a column which MySQL users to determine which partition

the data is in.

KEY is probably the simplest – we only need to supply it a field that is unique to each record

(your PRIMARY KEY usually).


Setting up the partitions

USE the SQL page to type:

ALTER TABLE country PARTITION BY HASH(countryID) PARTITIONS 2;

Checking the partitions

The partitioned table appears in:

C:\xampp\mysql\data\world

Here we can see the country table has been split up into two partitions because there are "p0"

MYD (data) and MYI (index) files, as well as "p1" files of those two types.

Partitioning issues

So how do we have those partitions served by another disk? In Unix, we can simply move the

file to another location, and put a symbolic link to that location. NTFS also supports symbolic

links with the "mklink" command (from the command line).

But as of v5.6, MySQL with symbolic links under Windows/NTFS is problematic. With the setup

in place, database engine errors were encountered whenever the table was accessed. Online

discussions indicated that hard links worked better, but hard links cannot cross device

boundaries, eliminating the main benefit of relocating the partitions.

There's a difference between supporting the data and index files being located on another

drive, and supporting a partition being located on another drive. Even locating un-

partitioned data files on another drive might not be supported, as can been seen by this help

text in Workbench:

That reads: “This works only for MyISAM tables only and not on some operating systems

(Windows)”. Where performance is concerned, there's limited value in having partitions of a

table sitting on the same physical disk. So it can be concluded that partitioning is of limited

value under Windows. This feature works well on Linux.


T A F E n o w

Removing partitions

The correct way to remove partitioning is by:

ALTER TABLE country REMOVE PARTITIONING;

If you check the data folder, you will see that your data and index files have been merged back

into one file each.

Documentation

Documentation isn’t anybody’s favourite task, but it’s a requirement in modern workplaces

and can make your job easier and make your performance better. Professionals document

well.

What might you document as a database administrator or developer?

> Database design and structure – diagrams and reasons.

> Backup strategy and implementation - how it works, where are they kept?

> Restore process – In the event of different types of failures, how do you get the business

back on its feet?

> Other processes - Document processes for common operations (for example creating

new users). There is no need to recreate the MySQL documentation, but things specific

to your organisation, your implementation, or perhaps just pointers to the

documentation will go a long way towards helping someone take on your job in the

event you move on.

> Bugs and issues - There are specialised applications and databases (e.g. MantisBT) for

keeping track of issues, who is assigned to them, and what the resolution was. This ends

up being documentation of what changes were made to the database, which may be

useful in the future if the same or similar problem is encountered. You should document

reasons for changes, which are often the most difficult things to remember.

> Changes - If an issue database isn’t used, a spreadsheet documenting changes made over

time might be useful to you for remembering what happened, or for someone new to

understand the history of the project.


Summary

Optimising and tuning can keep your MySQL database working at its peak performance. The

MySQL documentation has a section on optimising (MySQL, n.d.).

Partitioning is an easy way to spread the load across different devices.

Documenting what you do and why you do it can help keep your sanity and help you move

on to other interesting and challenging tasks.


T A F E n o w

http://dev.mysql.com/doc/refman/5.7/en/optimization.html

Learner Guide - Kingscliff & Murwillumbah IT · It supports high-availability features like...

Documents

Transcript of Learner Guide - Kingscliff & Murwillumbah IT · It supports high-availability features like...