Seminar : "The Future of MySQL - Roadmap to Success" session MySQL High Availability Options
-
Upload
software-park-thailand -
Category
Technology
-
view
2.440 -
download
2
description
Transcript of Seminar : "The Future of MySQL - Roadmap to Success" session MySQL High Availability Options
<Insert Picture Here>
MySQL High Availability Options
Ryusuke KajiyamaMySQL Principal Sales Consultant, Asia Pacific and Japan
2
Designing for High Availability
Copyright Oracle Corporation 2010 2
3
Selecting the Right HA Architecture
Copyright Oracle Corporation 2010 3
4
MySQLCluster
MySQLCluster
MySQLServer
Application / Web / Web AP
• MySQL ReplicationAsynchronous replication
• MySQL+DRBD (for Linux)Shared Nothing Active/Passive
Application / Web / Web AP
Application / Web / Web AP
Shared disk
• Shared Disk BasedActive/Passive
Application / Web / Web AP
• MySQL ClusterShared Nothing Active/Active
MySQLServer
MySQLServer
MySQLServer
MySQLServer
MySQLServer
AsynchronousReplication
SynchronousReplication
Fail Over on failure
Fail Over on failure
Load Balancing
SynchronousReplication
Load Balancing
MySQL High Availability Option
5
Common MySQL HA SolutionsMySQL Replication is Common Foundation for MySQL HA
Copyright Oracle Corporation 2010 5
Requirements Replication Cluster MySQL &DRBD
AvailabilityAutomatic Fail Over No Yes Yes
Fail Over Time Varies < 1 sec < 5 min
Resynch of Data No Yes Yes
Geographic Redundancy Yes Yes No
ScalabilityLoad Balancing Scale-Out Yes No
Read Intensive Yes Yes No
Write Intensive No Yes No
# of Nodes 100’s (reads) 255 1 Active
SQL FunctionalityPrimary Key Lookups Yes Yes Yes
Simple JOINs Yes Yes Yes
Complex JOINs Yes No Yes
Transactions Yes Yes Yes
6
MySQL Replication Overview
• Native in MySQL• Used for Scalability
and HA• Asynchronous as
standard• Semi-Synchronous
support added inMySQL 5.5
• Each slave addsminimal load onmaster
Copyright Oracle Corporation 2010 6
7
MySQL ReplicationDelivering Read Scalability
• Used by leading web properties for scale-out• Reads are directed to slaves, writes to master• Delivers higher performance & scale with efficient resource utilization
Clie
nts
Slaves Master
MySQL Replication
Copyright Oracle Corporation 2010 7
8
Master > Slave
Masters > Slave (Multi-Source)
Master < > Master (Multi-Master)
Master > Slaves
Circular (Multi-Master)
Master > Slave > Slaves
MySQL Replication Topologies
9
Building on ReplicationFailure Detection & Failover
• Linux Heartbeat implements heartbeat protocol between nodes• Failover initiated by Cluster Resource Manager (Pacemaker) if heartbeat
message is not received• Virtual IP address failed over to ensure failover is transparent to apps
Copyright Oracle Corporation 2010 9
10
Application / Web / Web AP
■MySQL Replication+ IP address failover
Application / Web / Web AP
Shared disk
■RedHat Cluster + Sharedstorage for Master + Slave
MySQLServer
MySQLServer
MySQLServer
MySQLServerAsynchronous
Replication
Fail Over on failureLoad Balancing
MySQLServer
Master(Read + Write)
Slave(Read)
Master(Read + Write) Asynchronous
Replication
Slave(Read)
Standby
MySQL HA Configuraiton
11
Application / Web / Web AP
■MySQL Replication+ IP address failover
Application / Web / Web AP
Shared disk
■RedHat Cluster + Sharedstorage for Master + Slave
MySQLServer
MySQLServer
MySQLServer
MySQLServer
Fail Over on failure
MySQLServer
AsynchronousReplication
Slave(Read)
Fail Over on failure
New Master(Read + Write)
New Master(Read + Write)
In case of failure of Master Server
12
MySQL 5.5 Replication Features1. Semisynchronous replication
Improved resilience by having master wait for slave toreceive events.
2. Slave fsync tuning & Automatic relay log recoveryTune fsyncs so corruption is less likely on slave crashes.Let the slave recover from corrupted relay logs.
3. Replication HeartbeatHave a more precise failure detection mechanism.Avoid spurious relay log rotation when the master is idle.
4. Per server replication filteringInstruct slave to discard events from a master with aspecific server id.
Copyright Oracle Corporation 2010 12
13
MySQL 5.5 Replication Features5. Precise Slave Type Conversions
Use different types on master and slave and getautomatic type promotion and demotion when using RBR
6. Individual Log FlushingSelectively flush server logs when using 'FLUSH LOGS'
7. Safe logging of mixed transactionsReplicate transactions containing both InnoDB andMyISAM changes
Copyright Oracle Corporation 2010 13
14
Application
Master
Connection Thread
Data Binlog
Slave
DataRelaylog
Commit
ChangingData
ChangingBinlog
Replication
Response
ChangingData
Asynchronous Replication
15
Application
Master
Connection Thread
Data Binlog
Slave
DataRelaylog
Commit
ChangingData
ChangingBinlog
Replication
ResponseChanging
Data
Response
Semi-synchronous Replication
16
“A high-performance, distributed memory object cachingsystem, generic in nature, but intended for use in speeding updynamic web applications by alleviating database load” *
* http://www.socialtext.net/memcached/index.cgi?faq
• Created by Danga Interactive to speed up LiveJournal’s 20million+ dynamic page views per day for 1 million+ users
• Significantly dropped database load, delivering faster pageloads, better resource utilization, and faster access todatabases
• Perfect for dynamic sites that generate high database load
• Used by Facebook, YouTube, Wikipedia, others!
What is Memcached?
17
• Created to speed up blogging site LiveJournal• 20 million+ dynamic page views per day• 1 million+ users
• Results…• Faster page loads• Lowered database load• Better resource utilization• Faster access to databases
• Perfect for dynamic sites that generate high database load
Why was Memcached created?
18
• Application is modified so data isread from memcached not thedatabase
• In the event the data is stale ornon-existent…
– data is read from thedatabase
– written into memcached
• Next request for the same data isretrieved from memcached
Typical Use Case: Read/Pass-Through
19
Memcached Functions for MySQL
• Overview– Uses UDF API and libmemcached– Manage memcached Cluster via SQL– Read through Cache– Write through Cache
• Installation– CREATE FUNCTION memc_servers_add
RETURNS INT SONAME"libmemcached_functions_mysql.so";
20
memcached UDF Example
– Creating Trigger which kicks UDF is one of the best practices
21
Linux Heartbeat, Block-Replication & MySQL
• Distributed Replicated Block Device (DRBD)– Runs over standard IP networks– Distributed storage– Similar to network RAID
• Synchronous• Characteristics
– Higher complexity to install and configure– No special networking components (except Heartbeat)– Excellent performance (blocks vs. rows of data)– Manages inconsistencies of data during a failure– Hides the complexity of many recovery actions– Linux heartbeat manages fail over and virtual IPs
22
DRBD Architecture
23
MySQL w/ Shared Storage & Clustering Agents
• Active/Passive likely configuration– Multiple instances not allowed concurrent access to same
data files• Automated management
– Virtual IPs– Fail over– Data synchronization– Mounting file systems
• Characteristics– High cost (storage, hardware, software)– Idle resources– Longer fail over times– High initial complexity– Many options and proven vendors
24
MySQL Cluster
• Shared-Nothing Clustering Solution• Synchronous (2-phase commit)• Fast Automatic Fail Over• High Performance• High Transactional Throughput• No Special Component Requirements• In-Memory & Disk Data Support• Heart-beat protocol
25
MySQL Cluster Data Nodes
MySQL Cluster Application Nodes
MySQLClusterMgmt
Clients
MySQLClusterMgmt
MySQL Cluster ArchitectureParallel Database with no SPOF: High Read & Write Performance & 99.999% uptime
26
Out of the Box Scalability: Data Partitioning
• Data partitioned across Data Nodes• Rows are divided into partitions, based on a hash of all or part of the primary
key• Each Data Node holds primary fragment for 1 partition
– Also stores secondary fragment of another partition
• Records larger than 8KB stored as BLOBs
27
Geographic Replication
Cluster 1
Synchronousreplication
Cluster 2
MyISAM MyISAM InnoDB
Asynchronousreplication
• Synchronous replication withina Cluster node group for HA
• Bi-Direction asynchronousreplication to remote Cluster forgeographic redundancy
• Asynchronous replication tonon-Cluster databases forspecialised activities such asreport generation
• Mix and match replication types
28
High Throughput, Low Latency Transactional Performance
• MySQL Cluster delivered:– 250k TPM, 125k operations per second– Average 3ms response time– 4.3x higher throughput than previous MySQL Cluster 6.3 release
1 4 8 12 16 20 24 28 32 36 400
2500050000
75000
100000125000
150000
175000
200000
225000
250000
275000
DBT2 Benchmark, 4-MySQL Cluster Data Nodes
MySQL C luster 7.0MySQL C luster 6.3
Number of MySQL Server Nodes
Tran
sact
ions
Per
Min
ute
http://www.mysql.com/why-mysql/benchmarks/mysql-cluster/
29
SQL Node(MySQL)
NDB API
Data Node(NDB Storage Engine)
X
Low-Level Access via NDB API
• High performance C++ API• Implements indexes, scans, transactions & events• ACID-compliant• Object-oriented error-handling• Additional performance features not available in SQL
30
MySQL Cluster Connector for Java
• New Domain Object Model PersistenceAPI (ClusterJ) :– Java API– High performance, low latency– Feature rich
• JPA interface built upon this new Javalayer:– Java Persistence API compliant
• Implemented as an OpenJPA plugin– Uses ClusterJ where possible, reverts to
JDBC for some operations– Higher performance than JDBC– More natural for most Java designers– Easier Cluster adoption for web
applicationsData Nodes
Network
32
• Application: Service Delivery Platform– Roaming platform to support 7m roaming subscribers per day FIFA World Cup 2010– Database supports AAA, routing, billing, messaging, signalling, payment processing– MySQL Cluster 7.1 delivered 1k TPS on 1TB data with carrier-grade availability
• Key business benefits– Local carriers to monetize new subscribers– Users enjoy local pricing with full functionality of their home network– Reduced deployment time by 75%
32
”MySQL Cluster 7.1 gave us the perfect combination of extreme levels of transactionthroughput, low latency & carrier-grade availability. We also reduced TCO by being able
to scale out on commodity server blades and eliminate costly shared storage”
- Phani Naik, Head of Technology at Pyro Group
Learn More: http://www.mysql.com/why-mysql/case-studies/mysql_cs-pyro_telecoms.php
3333
“Since deploying MySQL Cluster as our eCommerce database, we have had continuous uptime with linear scalability enabling us to exceed our most stringent SLAs”
— Sean Collier, CIO & COO, Shopatron Inc
Shopatron: eCommerce Platform
• Applications– Ecommerce back-end, user authentication,
order data & fulfilment, payment data &inventory tracking. Supports severalthousand queries per second
• Key business benefits– Scale quickly and at low cost to meet
demand– Self-healing architecture, reducing TCO
• Why MySQL?– Low cost scalability– High read and write throughput– Extreme availability
Learn More: http://www.mysql.com/why-mysql/case-studies/mysql_cs_shopatron.php
34
MySQLCluster
MySQLCluster
MySQLServer
Application / Web / Web AP
• MySQL ReplicationAsynchronous replication
• MySQL+DRBD (for Linux)Shared Nothing Active/Passive
Application / Web / Web AP
Application / Web / Web AP
Shared disk
• Shared Disk BasedActive/Passive
Application / Web / Web AP
• MySQL ClusterShared Nothing Active/Active
MySQLServer
MySQLServer
MySQLServer
MySQLServer
MySQLServer
AsynchronousReplication
SynchronousReplication
Fail Over on failure
Fail Over on failure
Load Balancing
SynchronousReplication
Load Balancing
MySQL High Availability Option