Percona and Continuent present: Multi-Data Center MySQL with Continuent Tungsten
Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
Transcript of Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
1/36
Continuent4/14/08
Continuent Tungsten:
Proxies on Steroids for HAand Performance!Technology Preview
Robert Hodges, CTO of Continuent
April 15, 2008
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
2/36
2 Continuent
What We Will Cover / Defining the Scale-Out Problem
/ A Hard Look at MySQL Replication for Scale-Out
/ Can We Do Better?
/ Replication Reloaded: Tungsten Scale-Out Stack
/ How to Get Software and Get Started Today
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
3/36
3 Continuent
A Word from Our Sponsor: Continuent
/ Company Leading provider of database availability solutions for open source
databases
/ Products uni/cluster multi-master database clustering that replicates data across
multiple databases and load balances reads Works with off-the-shelf database servers Uses database virtualization to provide a seamless client interface
/ Value Low-cost open source business critical solutions Highly available data Raise performance and hardware utilization through load balancing Remove chance of data loss
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
4/36
4 Continuent ConfidentialConfidential
4 Continuent
Defining the Scale-Out Problem
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
5/36
5 Continuent
Requirement: Availability and Performance
/ Data Availability Bring up alternate database after local failure Bring up alternate database after site loss Keep both sites on-line for immediate failover Maintain/upgrade/migrate without disruption Restart/restore following data loss or outage due to human error Ensure commits to multiple databases and/or sites Always accept transactions no matter what Stay available in grid environments
/ Performance Scale reads transparently Scale writes transparently Spread transaction load across multiple active sites Rapid capacity scaling up/down in grid environments
/ Nirvana: increased data availability andperformance scalability by adding commodityhardware
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
6/36
6 Continuent
The Dream - Flexible Scale-Out
S c a l a
b i l i t y
Availability
Db
App
Db
Db
App
App
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
7/36
7 Continuent ConfidentialConfidential7 Continuent
A Hard Look at MySQL Replication
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
8/36
8 Continuent
MySQL Replication for Scale-Out
/ MySQL database replication has set the high bar for horizontal scale-out on large websites for many years
/ Replication is baked-in to MySQL Initial set-up takes only a few minutes Handy SQL commands to configure masters and slaves
/ Flexibly copies all or some databases/ Offers a choice of statement or row replication
Row replication only in MySQL 5.0 and previous releases
/ Works across WANs, hence can be used to link
multiple sites/ In short, its a key reason why MySQL is so popular
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
9/36
9 Continuent
But Is It Really That Easy?
/ Replication is hard -- no single approach solves allproblems
/ Master is the weak point in the MySQL design Replacing a crashed master when there are multiple slaves Sync between binlog and database
Loss of committed transactions/ Latency due to single threaded update on slaves
/ Administration is harder than it looks Parameter settings: innodb_flush_log_at_trx_commit=2 Provisioning, resynchronizing, monitoring
Automating failover / Read scaling requires application changes
/ Not a write scaling solution
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
10/36
10 Continuent
A Hard Problem -- Master Election after a Crash
Slave
Master
Slave
Slave
NewNewMaster Master
WhereWhere was I??was I??
WhereWhere was I??was I??
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
11/36
11 Continuent
Customer Feedback from Our Clustering Work
/ Use cases for data base clustering driven byHA/scaling/both
/ Analysis of US/Europe sales shows followingpatterns:
Almost 100% of customers are looking for HA solution Scalability was a requirement for ~30%
/ What Customers Need Transparent, easy-to-deploy HA Protection from data loss/corruption Higher database performance with fewer application
changes
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
12/36
12 Continuent
MySQL vs. Customer Requirements
/ MySQL replication is excellent for implementing readscaling on custom applications
For the do-it-yourself set, its the best thing out there
/ BUT--It does not suit everyone:
/ Users who need easy-to-deploy availability
/ Users who cannot afford to lose data
/ Users who cannot afford intrusive applicationchanges
/ Users who are not MySQL experts
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
13/36
13 Continuent ConfidentialConfidential13 Continuent
Can We Do Better?
New Technologies for Scale-Out
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
14/36
14 Continuent
Continuent Database Clustering History
/ Continuent implemented m/cluster for MySQL -- thefirst successful commercial cluster based on groupcommunications
/ Our current uni/cluster product implements clustersusing middleware and off-the-shelf databases
/ We have supported MySQL and PostgreSQL for years
/ Strong experience in commercial/open sourcesolutions
Sequoia -- Multi-master state-machine clustering Myosotis -- Fast native-to-JDBC proxies Hedera -- Generic group communications adapters Bristlecone -- Testing tools for scale-out architectures
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
15/36
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
16/36
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
17/36
17 Continuent ConfidentialConfidential17 Continuent
Replication Reloaded:
The Tungsten Scale-Out Stack
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
18/36
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
19/36
19 Continuent
Tungsten Overview
/ Data Services integrate multiple copies of data withinand across sites into a single highly available,performant database
/ Database virtualization with controlled data latencyand consistency
/ Database-neutral replication designed for highavailability
/ Cluster-aware management framework backed byexcellent graphical tools
/ Stack organization of technology to allowsubstitution and extensions
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
20/36
20 Continuent
An Open, Database-Neutral Scale-Out Stack
Tungsten Connector Tungsten Connector Native Wire ProtocolNative Wire Protocol ProxyingProxying ;; Connection semanticsConnection semantics
SequoiaSequoiaSynchrononousSynchrononous
Multi-Master ReplicationMulti-Master Replication
Tungsten Replicator Tungsten Replicator AsynchronousAsynchronous
Master/SlaveMaster/Slave ReplicationReplication
Tungsten Manager Tungsten Manager Cluster-awareCluster-aware service management for LAN/WANservice management for LAN/WAN
HederaHederaGroupGroup CommComm AdaptersAdapters
BristleconeBristleconeScale-OutScale-Out Perf Perf TestingTesting
Group CommunicationsGroup CommunicationsJGroups,JGroups, AppiaAppia , Spread, Spread
DBMS ServersDBMS ServersMySQLMySQL ,, PostreSQLPostreSQL , Oracle, Oracle
Application StacksApplication Stacks
PHP,PHP, PerlPerl , Java, Ruby, etc., Java, Ruby, etc.
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
21/36
21 Continuent
Why Are Stacks So Important?
/ No single solution can solve all data availability andscaling use cases
/ Stack approach allows you to reuse components ondifferent scale-out problems
/ The stack is an extensible framework
Tungsten Connector (Tungsten Connector ( MyosotisMyosotis ))
SequoiaSequoia TungstenTungsten Replicator Replicator
Tungsten Manager Tungsten Manager
HederaHederaBristleconeBristlecone
CachingCaching
PartioningPartioning
Certification-Certification-BasedBased
ReplicationReplication
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
22/36
22 Continuent
Virtualized SQL Connection Semantics
Application StacksApplication StacksPHP,PHP, PerlPerl , Java, Ruby, etc., Java, Ruby, etc.
NodeNode NodeNode NodeNode NodeNode
All UpdatesVisible
Own UpdatesVisible
UpdatesEventually
Visible
Tungsten Data Service
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
23/36
23 Continuent
High-Availability Replication
Master-master replication/ No master failover headaches
/ Automatic load balancing
Master-slave replication/ Wider range of supported SQL
/ Faster performance
/ Works over WAN
NodeNode
NodeNodeNodeNode
NodeNode
NodeNodeNodeNode
SequoiaSeq uoia Tungsten Replicator Tung sten Replicator
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
24/36
24 Continuent
Managing within and across Sites
HostHost
ServiceServiceManager Manager
Replicator Replicator NodeNode
HostHost
ServiceServiceManager Manager
Replicator Replicator NodeNode
HostHost
ServiceServiceManager Manager
Connector Connector
Connector Connector HostHost
ServiceServiceManager Manager
Connector Connector
Connector Connector
Site 1Site 1
AdministrativeAdministrativeClientClient
JMXJMX
Group CommunicationGroup Communication
Site 2Site 2
HostHostHostHostHostHost
Event BridgeEvent Bridge
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
25/36
25 Continuent
GUI Tools Demo
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
26/36
26 Continuent
Now -- Multi-Master Data Services
/ Ensure constant availability and avoid master failurescenarios
/ Focus: Scalable and easily manageable multi-master replication
High quality management tools
Automatic load balancing of read requests 3+ master nodes Seamless connections from native clients and Java Support for latest MySQL versions and table types Solaris and Linux platforms
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
27/36
27 Continuent
Java Clients
Now -- uni/cluster 2008
Client ApplicationsLAMP Stack Clients(PHP, Perl, C, Ruby, etc.)
uni/cluster connector uni/cluster JDBC Driver
DBServer
DBServer
DBServer
DBServer
Clientsuni/cluster Driver
Controller
JDBC Driver
Controller
JDBC Driver
Controller
JDBC Driver
Controller
JDBC Driver
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
28/36
28 Continuent
Phase 1 - Master/Slave Data Services
/ Constant availability with high transparency andupdate performance/ Focus: Seamless upgrade from existing standalone
database One-step software installation Rapid data service configuration Lightweight, multi-platform management tools No application code changes
/ High availability replication Automatic and manual promotion of slave to master Bounded latency between slave and master Transparent proxying with read distribution to idle slaves
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
29/36
29 Continuent
Phase 1 - Master/Slave Data Service
ConnectorsFailover, Connection Semantics
Client Client
T u n g s t e n
M a m a g e m e n
t T o o
l s
Service Management
Master DBServer
Slave DBServer
ReplicationNode Mgr
Slave DBServer
ReplicationNode Mgr
ReplicationNode Mgr
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
30/36
30 Continuent
Phase 2 - Cross-Site Data Services
/ Ensure availability and scaling across sites/ Focus: Cross-site clustering with efficient failover
Replication of data to secondary site databases Minimal, understandable application changes Accept transactions across active systems on multiple sites
/ WAN clustering features Tools and admin model for cross-site management Handle WAN quality of service (latency, transient failures) SQL partitioning
/ Emphasis on WAN solutions that work without corner cases or data inconsistencies
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
31/36
31 Continuent
Phase 2 - Cross-Site Data Services
Data Service Connectors
Read/WriteApplications
Slave DBServer
Master DBServer
Slave DBServer
Connectors
Site 2Site 2SiteSite 11
Read/WriteApplications
Read-OnlyApplications,
Reports
Read-OnlyApplications,
Reports
DNS Load Balancing
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
32/36
32 Continuent
ConfidentialConfidential32 Continuent
How to Get Software and Get
Started Today
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
33/36
33 Continuent
Technology Roadmap
/ Uni/cluster 2007.1 is available today!/ Uni/cluster 2008 available in late Q2
GUI administrative tools Tungsten management framework Multiple controllers
/ Tungsten scale-out stack projects will be publishedsoon
/ Tungsten master/slave services will be available earlyQ4
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
34/36
34 Continuent
Getting Software
/ For access to commercial and open source products,visit us at the following web addresses
www.continuent.com
www.continuent.org
/ Visit us at the Continuent booth in the exhibit hall
/ Watch for more information on Tungsten very soon!
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
35/36
35 Continuent
For More on Scale-Out Performance
/ Come hear about scale-out testing using Bristlecone
Portable Scale-OutBenchmarks for MySQL
Wednesday 10:50-11:50am
Ballroom D
-
8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation
36/36