Practical my sql performance optimization
-
Upload
marian-marinov -
Category
Engineering
-
view
64 -
download
3
Transcript of Practical my sql performance optimization
Practical MySQL Practical MySQL Performance Performance OptimizationOptimization
Marian HackMan MarinovMarian HackMan Marinov<[email protected]><[email protected]>
Chief System ArchitectChief System ArchitectSiteGroundSiteGround
Who am I?Who am I? Who am I?Who am I?
❖ Chief System Architect of Siteground.com
❖ Sysadmin since 1996
❖ Organizer of OpenFest, BG Perl Workshops,
LUG-BG and similar :)
❖ Teaching Network Security and Linux System
Administration at Sofia University
❖ MySQL 5.7
❖ Or its equivalent from MariaDB or Percona
Requirements... Requirements...
❖ Why am I making this presentation?
❖ Where is the information coming from?
MySQL Optimization Documentation
❖ I have skipped the SQL examples as they made the presentation too big ☹
DISCLAMERDISCLAMER
❖ Manual monitoring
❖ Using Monyog
MonitoringMonitoring
How MySQL uses indexesHow MySQL uses indexes
❖ What type of indexes are there?
How MySQL uses indexesHow MySQL uses indexes
❖ What type of indexes are there?
B-Tree
Hash
FULLTEXT
Spatial
❖ What type of indexes are there?
❖ Which is used for what ?
hash
used for = or <=>
can not be used in ORDER BY
only whole keys can be used for search
How MySQL uses indexesHow MySQL uses indexes
❖ What type of indexes are there?
❖ Which is used for what ?
hash
How MySQL uses indexesHow MySQL uses indexes
❖ What type of indexes are there?
❖ Which is used for what ?
b-tree
can be used for =, >, >=, <, <=, or BETWEEN comparisons
can be used with LIKE, only if it doesn't start with %
can not be used in the form
column1 LIKE column2
can limit the size of the index (prefix)
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when the index does not cover all the columns in the where clause
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when the index does not cover all the columns in the where clause
when the types don't match: integer used for string index(column)
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when the index does not cover all the columns in the where clause
when the types don't match: integer used for string index(column)
when != is used
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when the index does not cover all the columns in the where clause
when the types don't match: integer used for string index(column)
when != is used
when you have a multicolumn index but the column you are referencing is not the first one and you don't have a separate index
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when you have a Btree index but you use distinct on the column
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when you have a Btree index but you use distinct on the column
when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when you have a Btree index but you use distinct on the column
when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN
when LIKE is used with % in the begining e.g. LIKE '%name'
How MySQL uses indexesHow MySQL uses indexes
❖ When is index NOT used
when you have a Btree index but you use distinct on the column
when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN
when LIKE is used with % in the begining e.g. LIKE '%name'
order of columns in indexes MATTERS
How MySQL uses indexesHow MySQL uses indexes
1410864 rows
1 row in set (10.46 sec)
1 row in set (8.33 sec)
1 row in set (6.66 sec)
Index hintsIndex hints
❖ IGNORE INDEX
Index hintsIndex hints
❖ IGNORE INDEX USE INDEX❖
Index hintsIndex hints
❖ IGNORE INDEX
❖ USE INDEX
❖ FORCE INDEX
Index hintsIndex hints
❖ "Waiting for query cache lock"
Abusing query cacheAbusing query cache
❖ "Waiting for query cache lock"
❖ SELECT *,NOW() FROM TABLE
Abusing query cacheAbusing query cache
❖ "Waiting for query cache lock"
❖ SELECT *,NOW() FROM TABLE
❖ SQL_NO_CACHE
Abusing query cacheAbusing query cache
❖ "Waiting for query cache lock"
❖ SELECT *,NOW() FROM TABLE
❖ SQL_NO_CACHE
❖ SET SESSION query_cache_type = OFF;
Abusing query cacheAbusing query cache
❖ "Waiting for query cache lock"
❖ SELECT *,NOW() FROM TABLE
❖ SQL_NO_CACHE
❖ SET SESSION query_cache_type = OFF;
❖ Аs of MySQL 5.6.8 query cache is now disabled by default
Abusing query cacheAbusing query cache
❖ Large result sets hammer the network...
Abusing query cacheAbusing query cache
❖ Large result sets hammer the network...
❖ For caching large results move the cache, closer to the application by caching the results in ProxySQL
Abusing query cacheAbusing query cache
❖ InooDB uses only a single index per query
Optimizing InnoDB queriesOptimizing InnoDB queries
❖ InooDB uses only a single index per query
❖ If an indexed column cannot contain any NULL values, declare it as NOT NULL
Optimizing InnoDB queriesOptimizing InnoDB queries
❖ InooDB uses only a single index per query
❖ If an indexed column cannot contain any NULL values, declare it as NOT NULL
❖ START TRANSACTION READ ONLY
Optimizing InnoDB queriesOptimizing InnoDB queries
❖ use ANALYZE TABLE or myisamchk --analyze
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
❖ avoid complex SELECT queries on MyISAM tables that are updated frequently
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
❖ avoid complex SELECT queries on MyISAM tables that are updated frequently
❖ MyISAM supports concurrent inserts
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
❖ avoid complex SELECT queries on MyISAM tables that are updated frequently
❖ MyISAM supports concurrent inserts
concurrent_insert = 0 (disabled)
concurrent_insert = 1 (auto)
concurrent_insert = 1 (always, even with deleted rows)
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
❖ avoid complex SELECT queries on MyISAM tables that are updated frequently
❖ MyISAM supports concurrent inserts
❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
❖ avoid complex SELECT queries on MyISAM tables that are updated frequently
❖ MyISAM supports concurrent inserts
❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables
❖ Don't split large(columns) tables
Optimizing MyISAM queriesOptimizing MyISAM queries
❖ use ANALYZE TABLE or myisamchk --analyze
❖ myisamchk --sort-index --sort-records=1
❖ avoid complex SELECT queries on MyISAM tables that are updated frequently
❖ MyISAM supports concurrent inserts
❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables
❖ Don't split large(columns) tables
❖ use DELAY_KEY_WRITE=1
Optimizing MyISAM queriesOptimizing MyISAM queries
mysql> SELECT @@optimizer_switch\G
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,
index_merge_sort_union=on,
index_merge_intersection=on,
engine_condition_pushdown=on,
index_condition_pushdown=on,
mrr=on,mrr_cost_based=on,
block_nested_loop=on,batched_key_access=off,
materialization=on,semijoin=on,loosescan=on,
firstmatch=on,duplicateweedout=on,
subquery_materialization_cost_based=on,
use_index_extensions=on,
condition_fanout_filter=on,derived_merge=on
Controlling the OptimizerControlling the Optimizer
MySQL 5.7only
❖ Merging multiple index scans on a SINGLE table
index_merge=off (default on)
index_merge_union=on
index_merge_sort_union=on
index_merge_intersection=on
low selectivity indexes are very slow with intersect
Controlling the OptimizerControlling the Optimizer
❖ Selecting different merge algorithms per table (only for 5.7)
Batched Key Access join /*+ BKA(t1) */
Block Nested-Loop join /*+ BNL(t1, t2) */
Controlling the OptimizerControlling the Optimizer
❖ Optimizing subqueries
Controlling the OptimizerControlling the Optimizer
❖ Optimizing subqueries
Materialization strategy
Controlling the OptimizerControlling the Optimizer
❖ Optimizing subqueries
Materialization strategy
Exists strategy
Controlling the OptimizerControlling the Optimizer
❖ Optimizing subqueries
Materialization strategy
Exists strategy
(usually preferred when the sub queries can produce NULL results)
Controlling the OptimizerControlling the Optimizer
❖ Optimizing subqueries
Materialization strategy
Exists strategy
(usually preferred when the sub queries can produce NULL results)
subquery_materialization_cost_based
Controlling the OptimizerControlling the Optimizer
❖ use SQL_SMALL_RESUL
❖ If all columns in the index are numeric MySQL will read only the INDEX
❖ Avoid large amounts of values in IN statements
values in an IN() list count as a predicates combined with OR
use eq_range_index_dive_limit to control the index dives in range comparisons(IN) 5.7
range optimization can't be used with NOT IN()
/*+ NO_RANGE_OPTIMIZATION(t4 PRIMARY) */
❖ Index Condition Pushdown 5.7
General optimizationGeneral optimization
❖ Avoid full table scans
--max-seeks-for-key=1000
SSD or NVMe
General optimizationGeneral optimization
# Turn tracing on (it's off by default):
SET optimizer_trace="enabled=on";
SELECT ...; # your query here
SELECT * FROM INFORMATION_SCHEMA.OPTIMIZER_TRACE;
# possibly more queries...
# When done with tracing, disable it:
SET optimizer_trace="enabled=off";
Using the tracerUsing the tracer
"filesort_summary": {
"rows": 100,
"examined_rows": 100,
"number_of_tmp_files": 0,
"sort_buffer_size": 25192,
"sort_mode": "<sort_key, packed_additional_fields>"
}
Using the tracerUsing the tracer
❖ Consider using INSERT with multiple VALUES lists
❖ Updates on tables with many indexes may become slow
❖ Updates with selects in them may block because of query cache lock contention
❖ Deletes also suffer from the above problems
Changing dataChanging data
❖ Splitting less frequently used columns from large(column) tables using foreign keys
Optimizing architectureOptimizing architecture
ACID RainACID RainConcurrency-Related Attacks onConcurrency-Related Attacks on
Database-Backed Web ApplicationsDatabase-Backed Web Applications
ACID RainACID RainConcurrency-Related Attacks onConcurrency-Related Attacks on
Database-Backed Web ApplicationsDatabase-Backed Web Applications
Marian HackMan Marinov<[email protected]>
THANK YOU THANK YOU THANK YOU THANK YOU
Marian HackMan Marinov<[email protected]>