Scaling MySQL using Fabric
-
Upload
karthik-pr -
Category
Technology
-
view
159 -
download
1
description
Transcript of Scaling MySQL using Fabric
![Page 1: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/1.jpg)
Scaling Persistent Store UsingMySQL FABRIC
P.R.KARTHIKMySQL DBA
![Page 2: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/2.jpg)
About US
Karthik.P.R• 4 + years of Industry experience as MySQL DBA.
• 1+ year at Yahoo! Administrating MySQL servers.
• Area of Focus MySQL HA and MySQL Sharding
• Student : M.Sc ( FOSS)
• Blogger : remotemysqldba.blogspot.in
![Page 3: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/3.jpg)
PROGRAM AGENDA
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
![Page 4: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/4.jpg)
Scaling MySQL
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
• Handling Shards
![Page 5: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/5.jpg)
Scaling MySQL
• Starts with a single node
![Page 6: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/6.jpg)
Scaling MySQL
• As reads and writes grows
Scale Vertically.
• Increase system resources• Memory• CPU • Hard Disk
![Page 7: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/7.jpg)
Scaling MySQL
Scaling reads using replication. • Replication is the best solution.• Replication is Async.• Failover has to be set.• Single writer
Read Queries Write Queries
![Page 8: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/8.jpg)
Scaling MySQL
Scaling reads using Galera • Read/ Write on any node• Synchronous Replication• A good HA solution.• Easy to add nodes.
Write / Read Queries
![Page 9: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/9.jpg)
Scaling MySQL
Scaling Writes
• Replication can’t scale writes• Partitioning is needed.• Distributes the writes• Horizontal scaling or Sharding.
ID 1 - 1000 ID 1001 - 2000
![Page 10: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/10.jpg)
Scaling MySQL
Components .
•Shard Key– Range– Hash– List
•Meta data store•Managing data set•High Availability of shards.•Database and schema changes.
![Page 11: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/11.jpg)
Scaling MySQL
Sharding Architecture
Proxy layer
State Store
Shard A
Shard B
Shard C
Application
![Page 12: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/12.jpg)
Scaling MySQL
Advantages of Sharding
•Improved Performance•Smaller Data set on local node.•Handling large data set•Scale well horizontally.•Only small data set is affected on a node failure.
Disadvantages of Sharding
•Managing the shards •Code change is need at some cases.•Splitting Shards•Maintaining the HA
![Page 13: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/13.jpg)
Available Sharding Tools
• Sharding is used widely in Large Scale MySQL Deployments.• Large organization built their own tools for sharding.
Popular MySQL Shards,
Facebook• Twitter• Tumblr• Flickr• Youtube• Dropbox
![Page 14: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/14.jpg)
Available Sharding Tools
Sharding Tools.
1) Saclebase ( Closed Source)2) Jetpants (Tumblr ) 3) Vitess ( Youtube )4) MySQL Fabric ( Oracle MySQL )
![Page 15: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/15.jpg)
MySQL FABRIC
![Page 16: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/16.jpg)
MySQL FABRIC
• MySQL Fabric is the frame work to maintain shards and High Availability.• Downloaded from MySQL Utilities.• Good CLI commands.• Works on MySQL > 5.6.10• XML RPC for Python , php and Java.• Minimized downtime of shard.
![Page 17: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/17.jpg)
MySQL FABRIC ARCHITECTURE
Application XML-RPC Connector
XML-RPC
FABRICNODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store
![Page 18: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/18.jpg)
MySQL FABRIC
FABRIC Components.
• Fabric node.• Fabric aware connectors ( XML RPC )• Server Groups.
– Global Group– Shard Group
• Back Store
![Page 19: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/19.jpg)
MySQL FABRIC
Fabric node
• Stores Fabric Config• Shard Moving• Shard splitting• High Availability monitoring
Back Store
• Global tables info• Shard key info and mapping• Shard monitoring
Fabric Node Back store
![Page 20: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/20.jpg)
MySQL FABRIC
Fabric aware connectors
• Supports Python/ php / Java Connectors• Fetch Shard info• Local cache• Avoids the proxy hop.
Application XML-RPC Connector
![Page 21: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/21.jpg)
MySQL FABRIC
Server Groups
• Stores actual data – Global Group– Shard Group
• Primary – Read / Write• Secondary – Reads ( failover)• Spare ( On request )
Primary Node Secondary Node
Spare Node
Server Group
![Page 22: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/22.jpg)
MySQL FABRIC
Server Groups ( Global Group )
• Stores Global Tables and schema• Schema changes• Replicates to Shard groups
Global Group Shard Group
Global Writes
Fabric Node
![Page 23: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/23.jpg)
MySQL FABRIC ARCHITECTURE ( Again )
Application XML-RPC Connector
XML-RPC
FABRICNODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store
![Page 24: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/24.jpg)
MySQL FABRIC
Advantages:
• Ease of Failover with GTID.• Shard Split is easy • RPC avoid network hops.
Dis Advantages
• Fabric node will be single point of failure • No Cross Shard . ( Shard query can help here)• Limited Connectors.
![Page 25: Scaling MySQL using Fabric](https://reader033.fdocuments.in/reader033/viewer/2022061207/5485bbbfb4af9f33148b4b65/html5/thumbnails/25.jpg)
THANK YOU