Reliable MySQL Using Replication
description
Transcript of Reliable MySQL Using Replication
What is replication?
• Allows 2 or more databases to maintain state between them
• MySQL 3.23 and up supports asynchronous replication
• One server acts as a master for one or more slaves
• Slaves pull data to be replicated from server’s binary logs and execute the relevant statements locally
Replication basics…
• One master supports multiple slaves• Each node on the network is assigned a unique
identifying number• Each slave attempts to connect to the master and
fetches data to be replicated
MasterSlave
SlaveSlaveSlaves
Replication basics…
• Clients perform data modification on master server
• INSERT, SELECT, DELETE, LOAD DATA• EXECUTE in MySQL 5.0 and above
Master SlaveClient
DATA DATAINSERT INTO …
Replication basics…
• Immediately following execution of command on master, the command is written to the local binary log
• Additionally, the master records its unique ID (to prevent endless loops in circular replication scenarios) and the timestamp for use with statements which use NOW(), etc.
Master SlaveClient
DATADATA
INSERT INTO …
Binary Log
Replication basics…
• If the slave is online, the command is transmitted to the slave in parallel (well, immediately following) to being written in the local binary log
• Otherwise, when the slave next connects it will receive a list of all pending statements from the master server’s binary log
• The slave’s replication IO thread stores the command in the local relay log
Master SlaveClient
DATADATA
INSERT INTO …
INSERT INTO …
Relay Log
Replication Thread
Replication basics…
• Once the data is received in the slave’s relay log, the slave SQL thread executes the command locally, bringing the slave up-to-date with the master
• In MySQL 3.23, the IO and SQL threads were just one thread. In later versions this was changed to boost performance
Master SlaveClient
DATADATA
INSERT INTO …
INSERT INTO …
Relay Log
Replication Thread
DATA
Replication Strategies
• Load balancing – single write, distributed read
Master
Slave Slave Slave
Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT)
Client (INSERT)
Replication Strategies
• Load balancing – single write, distributed read• Load balancing – circular read/write
MySQLServer
MySQLServer
MySQLServer
Client
INSERT INTO…
INSERT INTO …
Client
Replication Strategies
• Load balancing – single write, distributed read• Load balancing – circular read/write• High availability (hot failover)
Client Master
DATA
Slave
DATA
DATADATADATA
Replication Strategies
• Load balancing – single write, distributed read• Load balancing – circular read/write• High availability (hot failover)• Snapshot backups
Master
Slave
Backup with LOCK TABLES (or single transaction)
ClientINSERT / SELECT
ClientINSERT / SELECT
Simple Replication Setup
• Modify my.cnf to include a unique server-id for each node
• On master server, ensure that log-bin (binary logging) is enabled in my.cnf
• On slave, configure login credentials on master, either via my.cnf or CHANGE MASTER TO statement
• Copy initial data snapshot from master to slave• Configure initial binary log position on slave• Start replication with SLAVE START command
Configure master
my.cnf
-------------
[mysqld]
server-id = 1
log-bin
Configure slave
my.cnf
-------------
[mysqld]
server-id = 2
master-user = someuser
master-password = secret
master-host = ip.of.master
Initial dataset
• Binary log provides a record of all modifications to master database starting from a fixed point: when binary logging was activated
• If all binary logs exist on master from initial install of MySQL, the slave(s) can use these to bring themselves up-to-date
• Otherwise, a snapshot of the master must be taken, using mysqldump –master-data, to provide an initial dataset for the slave(s)
• If only MyISAM tables are used, the LOAD DATA FROM MASTER statement may be used on the slave(s)
Configure log position
MASTER mysql> SHOW MASTER STATUS;+---------------------------+----------+--------------+------------------+| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |+---------------------------+----------+--------------+------------------+| vmware-mirimar-bin.000002 | 79 | | |+---------------------------+----------+--------------+------------------+
SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘vmware-mirimar-bin.000002’, MASTER_LOG_POS=79;SLAVE mysql> START SLAVE;
Using CHANGE MASTER TO
• MASTER_HOST
• MASTER_USER
• MASTER_PASSWORD
• MASTER_LOG_FILE
• MASTER_LOG_POS
Advanced CHANGE MASTER TO
• MASTER_PORT
• RELAY_LOG_FILE
• RELAY_LOG_POS
• MASTER_SSL
• SSL: CA/CAPATH/CERT/KEY/CIPHER
Confirming it works
SLAVE mysql> SHOW SLAVE STATUS\G*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: vmware-mirimar Master_User: someuser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: vmware-mirimar-bin.000002 Read_Master_Log_Pos: 79 Relay_Log_File: vmware1-mirimar-relay-bin.000002 Relay_Log_Pos: 250 Relay_Master_Log_File: vmware-mirimar-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 79 Relay_Log_Space: 250
Seconds_Behind_Master: 0
Permissions
• Slaves need REPLICATION SLAVE permission on master for basic usage
• If LOAD TABLE FROM MASTER or LOAD DATA FROM MASTER statements are used, slave will also need SUPER and RELOAD privileges
Internal Threads
• Since MySQL 4.0, replication slaves run two threads
• IO thread continuously receives updates from master and writes to local relay log
• SQL thread continuously executes statements in relay log
IO thread isolation
• Isolating IO thread means that slave won’t have to wait for long-executing statements to finish executing before retrieving data from master
• Also, slave will continue reading data from master if a statement creates a data conflict
SQL thread isolation
• SQL thread isolation allows for replication in an environment without a continuous link between slave and masters
• If master fails (or slave simply has no access), the IO thread will try to reconnect endlessly (waiting 60 seconds between attempts)
• SQL thread will continue processing relay logs even while IO thread is unable to connect to master
Master Thread
• Additionally, the master server runs the Binlog Dump thread
• This thread is simply dedicated to scanning the binary logs on the master and sending updates to the connected slave
• If this thread isn’t running, it means that replication isn’t running – more accurately, that no slaves are currently connected
Status files
• 2 status files for replication’s use• Their use is to record the state of
replication between server shutdown and startup
• master.info records information about the slave’s master server
• relay-log.info records information about the local relay logs
Information in master.info
• Master log file• Read master log pos• Master Host• Master User• Password (will not be shown in SHOW SLAVE
STATUS)• Master Port• Connect Retry• In MySQL 4.1+, SSL options are stored if SSL is
used
Information in relay-log.info
• Relay log file
• Relay log pos
• Relay master-log pos
• Exec master-log pos
Backup master
• Master backups can be accomplished with mysqldump
• Care must be taken to ensure the following 2 special considerations:
1. Consistent snapshot of master date (via lock tables for MyISAM or single transaction for InnoDB)
2. Recording of binary log information, for use on slaves (master-data)
Backup master files
• If a file-system level backup is required, care should be taken to manually record binary log name and position via SHOW MASTER STATUS statement.
• To ensure consistency between backup and binary log position, the tables should be locked via FLUSH TABLES WITH READ LOCK immediately before backup (and SHOW MASTER STATUS)
• LEAVE THE CLIENT CONNECTED!!!• After backup finishes, execute UNLOCK TABLES to
release the read lock
Backup slave
• Same idea as master file system backup
• Instead of recording position, it’s enough to backup the master.info and relay-log.info files
• Instead of acquiring global read lock, it’s enough to STOP SLAVE before backup and START SLAVE once backup finishes
Live demo
• Time permitting, we’ll show a short demonstration of a simple unidirectional replication setup
For more information
• MySQL documentation
• 5.0 documentation http://mirror.mirimar.net/mysql/doc/refman/5.0/en/replication.html
• 4.1 documentation http://mirror.mirimar.net/mysql/doc/refman/4.1/en/replication.html
Thank You!
For more information:
Issac Goldstand [email protected]
http://www.beamartyr.net/
http://www.mirimar.net/