ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016...
Transcript of ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016...
![Page 1: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/1.jpg)
ProxySQLUseCaseScenarios
Percona Webinar Nov 23 2016
Alkin Tezuysal René Cannaò
![Page 2: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/2.jpg)
2
Who we are
• Alkin Tezuysal Sr. Technical Manager, Percona @ask_dba • René Cannaò MySQL SRE, Dropbox CEO, ProxySQL @proxysql
![Page 3: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/3.jpg)
3
Top 5 reasons to use ProxySQL
• Improvedatabaseopera:ons.
• Understandandsolveperformanceissues.
• Createaproxylayertoshieldthedatabase.
• AddHigh-Availabilitytodatabasetopology.
• EmpowertheDBAswithgreattool.
![Page 4: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/4.jpg)
4
ProxySQL Highlights
Scalability HighAvailability
AdvancedQuerySupport Manageability
![Page 5: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/5.jpg)
5
Use case overview - Scalability
Connec:onPoolingandMul:plexing
Addconnec:onpoolinglayer
Reduceconnec:onthreadoverhead
Read/WriteSplitMorescalabilityforanyMySQLtopology
Mustforanyheavyworkloads
Read/WriteSharding
Effortlessshardingimplementa:on
WithoutDev.supportandcostsavings
![Page 6: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/6.jpg)
6
Use case overview - High Availability
SeamlessFailover
Gracefulfailoversupport
Queryrerou:ng
LoadBalancingAllowseasyscaling
ForbePeru:liza:onofservers
ClusterAwareSupportsPXC/Galeraimplementa:ons
Smallfootprintforbiggain
![Page 7: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/7.jpg)
7
Use case overview - Advanced Queries
Querycaching Cachefrequentlyuseddata
Addsanothercachelayer
Queryrewrite AddSQLflexibility
Fasterproblemsolving
Queryblocking Adddatabaseawarefirewall
WithoutApp.support
![Page 8: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/8.jpg)
8
Use case overview - Advanced Queries
QuerymirroringAudit,Analyze,Review,CacheWarming
Allowbenchmarkingonlivedata
QuerythroPlingSetpriori:za:onforimportantqueries
Allowsmanualinterven:ontoproblemqueries
Query:meout Addanother:meoutlayer
Flexiblequerylevel:meout
![Page 9: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/9.jpg)
9
Use case overview - Manageability
AdminU:lity Authen:ca:onsupport
Limituseraccesstodatabasepool
Run:mereconfigura:on
Ontheflymodifica:ons
Configuredatabasewithoutrestartordown:me
Monitoring Statscollec:onandrepor:ng
Inves:gatedatabaseworkload
![Page 10: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/10.jpg)
10
Scalability with ProxySQL - Connection Pooling
• Any application without persistent connection to database
• PHP applications to be specific without built in connection pool
Application MySQL ProxySQL
• Reduces number of new connections to the database
![Page 11: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/11.jpg)
11
Scalability with ProxySQL – Connection Multiplexing
• Any application with persistent connection to database
• Java applications to be specific with built in connection pools
Application Pool
MySQL
• Reduce connections similar to Aurora
• Testing being performed for 300K database connections
Application Pool Application
Pool
ProxySQL
![Page 12: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/12.jpg)
12
Scalability with ProxySQL - Read/Write Split
• On the fly Read / Write implementation
• Use read_only flag to switch traffic
Application
MySQL Master
• Load balancing made easy
MySQL Slave
Write hostgroups
Read hostgroups MySQL
Slave
![Page 13: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/13.jpg)
13
Scalability with ProxySQL – User and schema level sharding
• Granular sharding per username and schema
• Backend pooling based on user activity to specific schema
Application User A
ProxySQL
• Use advanced sharding to parallelize queries
• Scale beyond the sharding per host limitations.
Application User B
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
![Page 14: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/14.jpg)
14
MySQL Slave MySQL Master
High Availability with ProxySQL – Seamless failover
• Neither VIP setup nor service discovery needed
• Use read_only flag to switch traffic
Application Pool MySQL Master
• Integration with other HA Managers
Application Pool Application
Pool
ProxySQL
![Page 15: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/15.jpg)
15
High Availability with ProxySQL – Improve Multi DC implementation
• No connection pools latency on SSL connections.
• Costly delays across the water.
MySQL Master ProxySQL
MySQL Master
ProxySQL
Application
DC1
DC2
![Page 16: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/16.jpg)
16
High Availability with ProxySQL – Adoption to Clustering
• PXC / Galera / Group Replication
MySQL
ProxySQL
•
ProxySQL
Application
Node 1
Application
Application
MySQL
Node 2
MySQL
Node 3
![Page 17: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/17.jpg)
17
Advanced Queries with ProxySQL – Caching
• Improve performance on read intensive workloads
• Reduce slave provisioning
Application A
ProxySQL
• Improved query cache over default implementation
• Query aware caching over general caching
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Cache
MySQL Slave
MySQL Slave
![Page 18: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/18.jpg)
18
Advanced Queries with ProxySQL – Query Timeout
• Built in query killer a.k.a query sniper
• Adjustable threshold based on query rule
Application A
ProxySQL
• No idle or long running queries
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Timeouts
MySQL Slave
MySQL Slave
![Page 19: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/19.jpg)
19
Advanced Queries with ProxySQL – Firewall
• Protect database from unwanted traffic
• Stop unwanted user, account , application (new code)
Application A
ProxySQL
• Fast modification to database behavior
• Added protection for DDOS and other attacks.
Application B
Query Blocking
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
![Page 20: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/20.jpg)
20
Advanced Queries with ProxySQL – Query rewrite engine
• Most wanted feature by DBAs
• Rewrite queries overloading the database on the fly.
Application A
ProxySQL
• Simply buy time until application can be modified
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Rewriting
MySQL Slave
MySQL Slave
![Page 21: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/21.jpg)
21
Advanced Queries with ProxySQL – Query retry
• Server failures or maintenance do not loose a query.
• Fails over and resubmits the query to another host.
Application A
ProxySQL
• Improved operation and new way of handling slaves.
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Retry
MySQL Slave
MySQL Slave
![Page 22: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/22.jpg)
22
Advanced Queries with ProxySQL – Query routing
• Selective routing based on importance.
• Use metrics based on stats_mysql_query_digest and decide
Application A
ProxySQL
• Go beyond read/write split.
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Routing
MySQL Slave
MySQL Slave
![Page 23: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/23.jpg)
23
Advanced Queries with ProxySQL – Query mirroring
• Mirror incoming queries to different back ends
• Use it for audit, analytics, cache warming, benchmarking.
Application A
ProxySQL
Application B
Query Mirroring
MySQL Master
Host A
Host B
MySQL Slave
MySQL Master
MySQL Slave
![Page 24: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/24.jpg)
24
Advanced Queries with ProxySQL
Querycaching QueryTimeout
QueryRetry QueryBlocking
QueryRou:ngandMirroring
![Page 25: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/25.jpg)
25
Clustered ProxySQL at scale
Tested with:
● 8 app servers with 3k clients’ connections each (24k total) ● 4 middle layer proxysqls processing 4k connections each from local
proxysqls (16k total) ● 256 backends/shard (meaning 256 routing rules) processing 600
connections each (150k total) • Up to 1M client connections • 1600 backend servers • 2000 query rules • 100k distinct users • up to 750k QPS
![Page 26: ProxySQL Use Case Scenarios - Percona...ProxySQL Use Case Scenarios Percona Webinar Nov 23 2016 Alkin Tezuysal René Cannaò 2 Who we are • Alkin Tezuysal Sr. Technical Manager,](https://reader036.fdocuments.in/reader036/viewer/2022062919/5f01c4107e708231d400ee8a/html5/thumbnails/26.jpg)
DATABASE PERFORMANCE MATTERS