Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014
-
Upload
mysql-brasil -
Category
Technology
-
view
250 -
download
2
description
Transcript of Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014
![Page 1: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/1.jpg)
Architecture and Design of MySQL Powered Applications
Peter ZaitsevMay 8, 2014
![Page 2: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/2.jpg)
About Presentation
What's in your Toolbox ?
What Questions you should ask ?
Common MySQL Architecture
Patterns
2
![Page 3: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/3.jpg)
LiveJournal Architecture 10 Years Ago3
![Page 4: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/4.jpg)
10 Years ago we see
Load Balancers
FrontEnd vs BackEnd Web Server Separation
MemcacheD (Caching)
Global Database for Shared Data
Read Write Splitting
Sharding “Database Clusters”
4
![Page 5: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/5.jpg)
History Lesson
MySQL is quite Mature
Battle tested Architecture Patterns
Most Problems have proven solutions
5
![Page 6: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/6.jpg)
YOUR TOOLBOX NOW
Well… certain things have changed
6
![Page 7: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/7.jpg)
Very Powerful Hardware
• 16GB to 512GBLots of
Memory
• 1000 IOPS to 100.000 IOPS
• 10ms latency to 0.1ms IO Latency
Very Fast Storage
• 4 Cores to 32 CoresMany CPU
Cores
7
![Page 8: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/8.jpg)
Reasonable Software
•MySQL
•Linux Kernel
•File Systems
•Etc..
Can use this
Hardware Pretty Well
8
![Page 9: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/9.jpg)
Cloud
Dynamic Scalability
Throw Away Server Mentality
Agility
Less involved Operations
9
![Page 10: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/10.jpg)
Automation
Development Testing Deployment Operations
10
![Page 11: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/11.jpg)
New Languages and Frameworks
More Agile
Less “system” code
More Client Side Development
Less Database Dependence
ORM
11
![Page 12: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/12.jpg)
Multiple “Clients”
Desktop Web Browsers
Mobile Web Browsers
“Apps”
API
12
![Page 13: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/13.jpg)
Not Only “MySQL”
• Memcache, Redis
• RabbitMQ
• Cassandra, MongoDB
• Hadoop
• Column Store Engines, Vertica
Right Tools
for Right Job!
13
![Page 14: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/14.jpg)
More HA Options
Mature MySQL Replication
Percona XtraDB Cluster, Galera
MySQL Cluster
Proprietary cloud HA
14
![Page 15: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/15.jpg)
Ready solutions for Replication Management
MHA
PRM
Continuent Tungsten
15
![Page 16: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/16.jpg)
Making Sharding Better
• Clustrix
• MemSQL
• ScaleDB
Not Quite MySQL
• ScaleArc, ScaleBase, Tesora
• MySQL Proxy, MaxScale, ProxySQL
Proxy Solutions
• Vitess
• JetPants
• MySQL Fabric
OpenSource Frameworks
16
![Page 17: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/17.jpg)
ARCHITECTURE QUESTIONS
17
![Page 18: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/18.jpg)
Basics
Make some Right decisions Early
Your first Architecture will not be perfect
Some poor choices are very expensive to fix
18
![Page 19: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/19.jpg)
Consultants are like Lawyers
You can waste a lot of money with them
Getting right advice on the right topics will save you a lot of trouble in the end
19
![Page 20: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/20.jpg)
Dimensions to consider
Application Scale
High Availability
Team Experience
20
![Page 21: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/21.jpg)
Approach to Scaling
Avoid Over-Engineering
Give yourself some
runway
Do Some Capacity Planning
21
![Page 22: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/22.jpg)
High Availability
Real High Availability is Really Expensive
Medium Availability is getting more affordable
People are often leading cause of downtime
22
![Page 23: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/23.jpg)
Team Experience
Chose Solutions which your Team can Support
Often Developers know little about Databases
Right Development, Operational Process Is a Key
Tools to Support your Process
• Percona Toolkit, Percona Cloud Tools
• Systematically find and review new queries
23
![Page 24: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/24.jpg)
Simplicity
• Less complexity lower costsUse fewer
components when possible
• Monitor
• Troubleshoot
Each components needs experience
to
• “Follow the Crowd”
• Unless you have a reason not toThere is safety in
numbers
24
![Page 25: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/25.jpg)
World of “Small” Applications
• Really
Majority of Applications are
Small
• Especially with support of caching
May well run on single database
instance
• Active intranet site for company with 200K employees
• Ecommerce site supporting $10M+ in Sales
Such “Small” Applications can
be significant
25
![Page 26: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/26.jpg)
Lets do Some Math
Modern MySQL
Instance Handle 100K
QPS
20 Queries per User
Interaction
5K User Interactions Per Second
144M User Interactions a
Day (Considering
3x peak to average)
10M Daily Active users
(with Medium
Engagement)
26
![Page 27: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/27.jpg)
PRACTICAL CHOICES
Basic Architecture Patterns
27
![Page 28: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/28.jpg)
Baseline
Single MySQL Instance
No Caching
No HA
No Supplemental Technologies
28
![Page 29: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/29.jpg)
Simple High Availability
HA with DBaaS (Amazon RDS)
HA with MySQL Replication
HA with Percona XtraDB Cluster
29
![Page 30: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/30.jpg)
What do you need to Scale ?
Reads
Writes Data Size
30
![Page 31: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/31.jpg)
Load Management Tricks
Move the load
In Time (Doing Batch Processing
at Night)
In Space (Use Slave for Reports)
31
![Page 32: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/32.jpg)
Queueing
Used by Most high volume architectures
Deal with Load Spikes
Increase Reliability
Scale parts of the system which need to
Many solutions
• RabbitMQ
• Redis
• Gearman
32
![Page 33: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/33.jpg)
Scaling Reads
Replication (and Read Write Splitting)
Caching (Memcache, Redis etc)
Percona XtraDB Cluster (Read Anywhere)
33
![Page 34: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/34.jpg)
Scaling Writes
• Using Different Servers for Different Parts of Application
Functional Partitioning
• Horizontal Partitioning over many servers Sharding
34
![Page 35: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/35.jpg)
Scaling Data Size
•Depends on the Application and Needs
•10TB good rule of thumb
How Much We Can
Practically Store in Single
MySQL Instance
these days
35
![Page 36: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/36.jpg)
How Is it Possible
“Online” Schema Change
MySQL 5.6
Pt-online-schema-change (Percona
Toolkit)
Fast Backup
Percona Xtrabackup
MySQL Enterprise Backup
Modern Filesystem/Volume
Managers
Fast Network
10Gbps affordable
High Performance
Storage
Flash/SSD
Compression
Innodb
TokuDB
36
![Page 37: Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b79f7d4a795993718b4679/html5/thumbnails/37.jpg)
Summary
MySQL is Mature. Solutions Available for Many Problems
Make few choices right in the beginning
Hardware and Software advances allow to go a long way with simple architectures
37