CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity
-
Upload
davie-stokes -
Category
Technology
-
view
253 -
download
1
description
Transcript of CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity
![Page 1: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/1.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
Insert Picture Here
MySQL Scaling§ Dave Stokes
MySQL Community Manager
@stoker
slideshare.net/davestokes
23rd August 2014
![Page 2: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/2.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.2
Safe Harbor Agreement
.The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated into
any contract. It is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making purchasing decision.
The development, release, and timing of any features or functionality
described for Oracle’s products remains at the sole discretion of Oracle.
![Page 3: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/3.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Insert Picture Here
MySQL Releases since 2010
§ Enterprise Monitor 2.2, 2.3
§ Cluster 7.1, 7.2, 7.3
§ Cluster Manager 1, 1.1, 1.3
§ Workbench 5.2, 6, 6.1
§ Database 5.5, 5.6, DMR 5.7
§ Backup 3.5, 3.7, 3.1
§ Oracle VM for MySQL
§ Oracle Product Certifications
§ Windows Installer
§ Utilities 1.0.6
§ Audit
§ Fabric
● Under promise/Over Deliver
● More Developers
● More QA
● More Support
● Making $$
● Classes popular
– Instructor lead
– Self-paced
● We're hiring!
![Page 4: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/4.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
You want to get a bigger, badder MySQL --BUT what are the costs???
Budget
Complexity
Application
Opportunity
Sanity.
![Page 5: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/5.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.5
Simple Stuff
![Page 6: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/6.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.6
Option 0
Run the Latest, Greatest MySQL
MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15%
--Make sure your config file matches your hardware
--ANALYSE TABLE
--Check for SLOW QUERIES, QUERIES w/o INDEXES
--TUNE QUERIES
This is the least complex, less upsetting, and usually lowest budget cost option with NO changes to your application!
![Page 7: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/7.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.7
Option 0.1
Add in a Memcached Layer
● Cache for quick access
● Session Data
● Sports Scores
● Opens up NoSQL access to InnoDB later
This is the second least complex, less upsetting, and usually lowest budget cost
option but does make you change your application to become 'cache aware'.
![Page 8: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/8.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.8
Up Time versus Response Time versus Access
Things to be engineered/negotiated upfront
Up Time
– How much down time can you afford?
Response Time
– How long is too long to wait?
Access
– How many do you need to serve simultaneously.
![Page 9: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/9.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Insert Picture Here
Simple Level 1 -Splitting Reads
![Page 10: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/10.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.10
Use MySQL Replication to Split Read/Writes
![Page 11: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/11.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.11
READ Splitting
Most Common Scaling Technique
● More complex environment
● Application Changes
● Easier with PHP & Java
● Low opportunity and sanity costs
● Less risk of one server stopping operations
![Page 12: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/12.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.12
Read Splitting—Easy Way
Use MySQL Utilities to copy master, set up failover
● mysqlserverclone
● mysqlfailover
– When master dies, most up to date slave (GTID) promoted automatically
● Mysqlrplcheck & mysqlrplshow
![Page 13: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/13.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Insert Picture Here
Simple Level 2Moderately Easy, BetterAvailability
![Page 14: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/14.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.14
Distributed ReplicatedBlock Device (DRBD)
● High Availability
More complex environment
● Proven, supported
● Please practice fail over
and back
● Easy to manage but
not HA panacea
![Page 15: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/15.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.15
Cloud
High Availability, low cost (sometimes)
Slightly more complex environment
● Proven, supported
● Best for ephemeral data, may not make upper management
comfortable (or you)
● Easy to manage but not a HA panacea
![Page 16: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/16.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Insert Picture Here
Complex Level N^NNever, Ever Down
![Page 17: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/17.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.17
MySQL ClusterFor 99.999% up time, upper bound on response time
Extremely Complex
● No single point of
failure = lots of $$
hardware
● Not general
purpose MySQL
Auto sharding
● Add/Subtract nodes at will
● Replicate between
geographically separated
locations
● Need a smart staff
![Page 18: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/18.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.18
MySQL Cluster Shards Data Automatically
● Single or multiple data node
failures do not bring down a
cluster
● Easy to add/remove data
nodes
![Page 19: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/19.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.19
MySQL Cluster Topology
● Nodes
– MGMT – startup/add node
– Data
– Mysqld
● Can be engineered for no
single point of failure
● Can replicate across data centers
![Page 20: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/20.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Insert Picture Here
BigData
![Page 21: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/21.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.21
Ask ten DBAs to define 'Big Data' and You WillGet At Least Eleven Answers
● BIG as massive amounts
– Define MASSIVE
● BIG as in fast
● Define fast
● BIG as lots of eyes
● BIG as all of the the above!!!
● How many simultaneous users
![Page 22: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/22.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.22
The Easiest Way to Get Speed
NoSQL/Memcached to InnoDB/NBD
● 9x faster by skipping parser, optimizer
● Key/Value pair access
and SQL access of same data
![Page 23: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/23.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.23
Easiest Way to Gain Size
Columnar Storage Engines for MySQL
● Not InnoDB
● Unique features, quirks
● Need > 26,000,000,000 rows to start seeing 'sweet spot'
● Use new server, don't scrimp
See Calpont and Inforbright
![Page 24: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/24.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.24
Hadoop-- getting data from MySQL to Hadoop
80% of Hadoop Installations feed from MySQL
● Hadoop is made up of several components
● Twisty little maze, all alike
● Use Apache Bigtop
– Tested for interoperability
● Cloudera & Horton Works have great tutorials
● Kinda 'SQL-ish'
● Squoop and MySQL Hadoop Applier to move data
![Page 25: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/25.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.25
![Page 26: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/26.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.26
Partitioning and Sharding /Dividing and Conquering
Partitions built-in to MySQL
● Nice management tool
– Rolling quarters/months
– Fast drops
● No change to application but
higher complexity, low
opportunity cost
Sharding
● Much more complex
application and operations
● Not off shelf
● Hard to re-engineer
● Works
![Page 27: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/27.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.27
MySQL Fabric
● Lets you shard and reshard
● Very new
● Looking for feedback
● Released in May
![Page 28: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/28.jpg)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Insert Picture Here
Wrap up
![Page 29: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/29.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.29
Goals – What are you trying toaccomplishDefine problem completely
● Who produces, who consumes
data
● Negotiate performance rates
● Plan for failure, catastrophe
● Design for long life
What does end solution look/feel like?
● Platforms
● Speed
● Access areas
● Security
● Serenity
![Page 30: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/30.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.30
MySQL Marinate
● Free virtual class to learn MySQL run by the Boston MySQL Users Group
– 1 chapter a week
– Homework graded helpfully
![Page 31: CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity](https://reader034.fdocuments.in/reader034/viewer/2022051323/54b6eac84a7959aa218b4669/html5/thumbnails/31.jpg)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.31
Q&A
[email protected] @stoker
slideshare.net/davestokes