Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 ·...
Transcript of Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 ·...
![Page 1: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/1.jpg)
Percona Live 2017Santa Clara, California | April 24-27, 2017
MySQL INDEX CookbookHow to Build the Best INDEX for aGiven SELECT
Rick James
![Page 2: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/2.jpg)
AgendaLimitations: InnoDB indexes only, not FULLTEXT orSPATIAL
Agenda:• Definitions• Examples• Algorithm• What Works; What Doesn't• PRIMARY KEY• Other Issues• Table Patterns
![Page 3: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/3.jpg)
Definitions
Terminology -- (will be repeatedas we go)
![Page 4: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/4.jpg)
Syntax Keywords• PRIMARY KEY is a UNIQUE KEY • plus "clustered"• UNIQUE KEY is an INDEX • plus a uniqueness constraint• Synonymous: "INDEX" "KEY"
• FULLTEXT, SPATIAL, HASH not being discussed
![Page 5: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/5.jpg)
More Types of indexes• "Secondary index" • not PRIMARY, hence not "clustered"• "Clustered" • PRIMARY KEY lives with the data• "Covering" • All the columns of the SELECT are in the index • Don't have more than, say, 5 columns• "Composite" (aka "compound") • Multiple columns: INDEX(a,b)
![Page 6: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/6.jpg)
Filtering - "Equal" WHERE x = 123
WHERE str = 'foo'
![Page 7: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/7.jpg)
Filtering - "IN" WHERE b IN (1, 2) • If single item, works like = • If multiple items, maybe like =, maybe like range
WHERE x IN ( SELECT … ) • Optimizes poorly • Turn into JOIN – Example…
![Page 8: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/8.jpg)
IN to JOIN Example SELECT … FROM t WHERE some_test AND x IN ( SELECT x FROM … );⇒ SELECT … FROM t JOIN ( SELECT x FROM … ) b USING(x) WHERE some_test;
![Page 9: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/9.jpg)
Filtering - "Range"A sequence of consecutive values • x < 123 • x BETWEEN 100 and 199 • str LIKE 'foo%' • No: str LIKE '%foo'
![Page 10: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/10.jpg)
"Index Merge"• A single SELECT will use at most one INDEX. • A few (very few) exceptions. • Called "index merge"
mysql.rjweb.org/doc.php/index1
![Page 11: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/11.jpg)
Definitions - Q&A1 question (hold rest until end)
![Page 12: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/12.jpg)
Examples
Some Simple Examples --Develop an Algorithm later
![Page 13: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/13.jpg)
Single Filter SELECT … WHERE a = 11 SELECT … WHERE a >= 11
INDEX(a) -- perfect INDEX(a, b) -- good
SELECT … WHERE name = 'Rick' SELECT … WHERE name LIKE 'R%'
INDEX(name) -- perfect INDEX(name, b) -- good
![Page 14: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/14.jpg)
Multiple '=' Filters WHERE a = 12 AND bb = 345 WHERE bb = 345 AND a = 12
INDEX(a, bb) -- perfect INDEX(a) -- somewhat good INDEX(bb) -- somewhat good
• Order in WHERE does not matter • assuming AND'd
![Page 15: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/15.jpg)
Equal and Range WHERE a = 12 AND bb > 345 WHERE bb > 345 AND a = 12
• INDEX(a, bb) – perfect • INDEX(a) – somewhat good • INDEX(bb) – somewhat good • INDEX(bb, a) – no better than (bb)!
![Page 16: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/16.jpg)
Two RangesPunt!
WHERE a > 12 AND bb > 345No index with both a and bb is fully useful
Ditto for "=" plus multiple "ranges":
WHERE c = 9 AND a > 12 AND bb > 345Either might be useful: INDEX(c, a) INDEX(c, bb)
![Page 17: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/17.jpg)
CoveringExamples above have an exception…
IF all columns in the SELECT are in the index, then the indexis "covering", hence at least a little better
SELECT x FROM t WHERE y = 5; INDEX(y,x) • The algorithm says just INDEX(y)
SELECT x FROM t WHERE y > 5 AND q > 7; INDEX(y,q,x) • y or q first (that's as far as the Algorithm goes); then other two
![Page 18: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/18.jpg)
BTree - 1Technically it is a B+Tree.
This is the structure of the indexes being discussed.• Very efficient at • Locating a single row, given the key • Scanning a range of rows
en.wikipedia.org/wiki/B+_tree
![Page 19: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/19.jpg)
BTree - 2The data (clustered with PRIMARY KEY) is also aBTree.
• Leaf nodes of the Data BTree • contains entire rows• Leaf nodes of the Secondary index BTree • contains secondary key and PRIMARY KEY
Rule of Thumb: Fanout ~100x
![Page 20: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/20.jpg)
Examples - Q&A1 question (hold rest until end)
![Page 21: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/21.jpg)
Algorithm
Build the best INDEX
![Page 22: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/22.jpg)
First, some Caveats• No OR• No IN• Just a bunch of filters ANDd together in the WHEREclause
We'll fold those in later
![Page 23: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/23.jpg)
Step 1 - Equals• Find all filters of the form col = constant • Put those column names in the INDEX first • In any order • "Cardinality" does not matter
![Page 24: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/24.jpg)
Step 2• You can add one more column • Range, or • GROUP BY, or • ORDER BY
![Page 25: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/25.jpg)
Step 2a - RangeIf you have a "range" filter, add its column.
Then stop; no further columns will help.
![Page 26: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/26.jpg)
Step 2b - GROUP BY• If • No range, and • All of the WHERE is handled
• Then • Add all the columns of the GROUP BY to the index • In the same order
• And stop
![Page 27: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/27.jpg)
Step 2c - ORDER BY• If all are true: • No range, • All of the WHERE is handled, • No GROUP BY, • Have ORDER BY with all ASC or all DESC (Ver 8.0 relaxes this)
• Then • Add all the columns of the ORDER BY to the index • In the same order
![Page 28: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/28.jpg)
GROUP BY + ORDER BY + LIMITIf you consumed all • consumed all of WHERE, and • consumed all of GROUP BY, and • ORDER BY is • missing, or • identical to GROUP BY (or DESC)
Then, you can consume the LIMIT…
![Page 29: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/29.jpg)
Consume the LIMIT• Avoid "temporary" and "filesort"• Looks only at LIMIT rows, not all the rows
• It does not make much sense to have a LIMITwithout an ORDER BY.
• OFFSET rows must be stepped over
![Page 30: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/30.jpg)
ORDER BYSometimes the Optimizer decides to• Ignore WHERE• Use index suitable for ORDER BY
Sometimes good, sometimes not.
Perhaps add an INDEX aimed just at ORDER BY
![Page 31: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/31.jpg)
Algorithm - Q&A1 question (hold rest until end)
![Page 32: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/32.jpg)
What Works; What Doesn't
Issues that help/hurt indexing
![Page 33: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/33.jpg)
Index killers - functionsThese don't let you use an index:
• Implicit or explicit functions DATE(dt) = '…', LOWER(s) = '…' CAST(s …) = '…', x = '…' COLLATE…
en.wikipedia.org/wiki/Sargable
![Page 34: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/34.jpg)
Index killers - others• Leading wildcard s LIKE '%…'
• Different tables t1.x = 8 AND t2.y = 11 • INDEX(x) or INDEX(y) may be useful
• Negatives • NOT IN, NOT EXISTS, and LEFT JOIN..IS NULL • new versions of MySQL/MariaDB may work better
![Page 35: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/35.jpg)
Flags - badTRUE/FALSE or other low carndinality columns arenot worth indexing: WHERE flag = TRUE • won't use INDEX(flag)
OK in combo: WHERE flag = TRUE AND dt > '…' • will use INDEX(flag, dt)
![Page 36: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/36.jpg)
UNION for ORSometimes it is useful to turn OR into UNION.
WHERE a = 1 OR x = 4
This shows adding a LIMIT:
( SELECT … WHERE a = 1 ORDER BY … LIMIT 5 ) UNION ALL ( SELECT … WHERE x = 4 ORDER BY … LIMIT 5 ) ORDER BY … LIMIT 5;
Switch to UNION DISTINCT if you need dedup.
![Page 37: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/37.jpg)
UNION with OFFSETTo get the 10th 'page':
( SELECT … ORDER BY … LIMIT 50 ) UNION ALL ( SELECT … ORDER BY … LIMIT 50 ) ORDER BY … LIMIT 45, 5;
Pagination:mysql.rjweb.org/doc.php/pagination
![Page 38: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/38.jpg)
ASC / DESC ORDER BY a ASC, b ASC ORDER BY a DESC, b DESC • Both work with INDEX(a,b); the second is slightly
less efficient
ORDER BY a ASC, b DESC INDEX( a ASC, b DESC ) • (pre-8.0): ASC and DESC are ignored in index, so index
can't be used
![Page 39: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/39.jpg)
Prefix - INDEX(foo(5)) - poor• Use for TEXT or BLOB• Do not use otherwise• Often the Optimizer will eschew the index
• UNIQUE(foo(5)) is "wrong" • uniqueness check on only 5 chars• INDEX(last(3), first) • won't get past last
![Page 40: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/40.jpg)
Using temporary, Using filesortThis is often necessary.It is not the villain by itself.
• GROUP BY team ORDER BY score • Leads to second temp+sort
![Page 41: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/41.jpg)
DATEs - bad casesTempting, but cannot use index because the columnis hiding in an explicit or implicit function:
date LIKE '2016-12%'
LEFT(date, 7) = '2016-12'
YEAR(date) = 2016
Instead…
![Page 42: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/42.jpg)
DATEs - goodRange, so index possible: date >= '2016-12-01' AND date < '2016-12-01' + INTERVAL 3 MONTH
Avoids problems with • Month/year boundaries & Leap days • Last second (BETWEEN is "inclusive") • Works for DATE, DATETIME(6), TIMESTAMP
![Page 43: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/43.jpg)
What Works/Doesn't - Q&A1 question (hold rest until end)
![Page 44: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/44.jpg)
PRIMARY KEY
PRIMARY KEY issues
![Page 45: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/45.jpg)
What [not] to use for PKChoices for PRIMARY KEY• (usually best) "Natural" column(s)• (decent fallback) AUTO_INCREMENT • Make it UNSIGNED and NOT NULL • BIGINT (8 bytes) is usually overkill• (terrible for huge table) UUID/GUID/MD5 • Randomness ⇒ I/O ⇒ Slow• (usually bad) No PK • Some maintenance operations must have PK
![Page 46: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/46.jpg)
Natural benefits• Avoids need for AUTO_INCREMENT• Faster access by that column• Works fine in most cases• Might lead to "covering"
![Page 47: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/47.jpg)
AUTO_INCREMENT benefits• Less 'bulky' • Shrinks secondary keys • A copy of PK is in every Secondary key
![Page 48: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/48.jpg)
Burning IDs (gaps)Some operations waste AUTO_INCREMENT idsbecause they allocate the id before seeing if theyneed it
• INSERT IGNORE …• INSERT … ON DUPLICATE KEY UPDATE …• REPLACE … (mostly replaced by IODKU)
Beware of hitting the max value for the id!
![Page 49: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/49.jpg)
PRIMARY KEY - Q&A1 question (hold rest until end)
![Page 50: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/50.jpg)
Other Issues
Miscellany
![Page 51: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/51.jpg)
More than one INDEX• A SELECT will (usually) use only one INDEX. • Each subquery or UNION counts separately • So, they may use different indexes
![Page 52: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/52.jpg)
Tweaks• Avoid USE/FORCE/IGNORE INDEX,STRAIGHT_JOIN • except in desperation
• LIMIT 9999999999 • tricks Optimizer into doing an otherwise unnecessaryORDER BY
![Page 53: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/53.jpg)
767 LimitationErr: "max key length is 767" usually happens withVARCHAR(255) CHARACTER SET utf8mb4.
• Workaround: do one of • Upgrade to 5.7.7 for 3072 byte limit • Change 255 to 191 on the VARCHAR • ALTER .. CONVERT TO utf8 • but disallows Emoji and some Chinese
• Use a "prefix" index (ill-advised) • Reconfigure (for 5.6.3 - 5.7.6) • Barracuda + innodb_file_per_table + innodb_large_prefix +
dynamic/compressed
![Page 54: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/54.jpg)
Redundant indexes - waste PRIMARY KEY(id) UNIQUE (id) -- Drop
INDEX(a, b) INDEX(a) -- Drop
INDEX(a, b) INDEX(b, a) -- May be redundant
![Page 55: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/55.jpg)
Signs of a Newbie• No PRIMARY KEY• No composite indexes• "But I indexed everything"• Redundant indexes • eg, PRIMARY KEY(id), KEY(id)• "Commajoin" • FROM a , b WHERE a.x=b.x AND c=1 ⇒ • FROM a JOIN b ON a.x=b.x WHERE c=1
![Page 56: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/56.jpg)
PARTITIONPartitioning has a lot of limitations on indexes.
Try to avoid partitioning by building better indexes.
mariadb.com/kb/en/mariadb/partition-maintenance/
![Page 57: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/57.jpg)
JOINs• Designing INDEXes for a JOIN • Design index for first table • Design index for next table • Etc
• Which is "first"? • Not necessarily the order specified • LEFT JOIN may force left table before right • Optimizer prefers table with WHEREs
![Page 58: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/58.jpg)
JOIN example SELECT … FROM a JOIN b ON where a.x = b.y WHERE b.z = 123
• First b with INDEX(z)• Then a with INDEX(x)
![Page 59: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/59.jpg)
Other Issues - Q&A1 question (hold rest until end)
![Page 60: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/60.jpg)
Table Patterns
Some Patterns
![Page 61: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/61.jpg)
Many:Many Mapping CREATE TABLE student_class ( id_student … NOT NULL, id_class … NOT NULL, … optional atributes …, PRIMARY KEY(id_student, id_class), INDEX (id_class, id_student) ) ENGINE=InnoDB;
Notes ⇒
![Page 62: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/62.jpg)
Many:Many notes• No AUTO_INCREMENT id• Small ids (MEDIUMINT, etc)• UNSIGNED & NOT NULL• InnoDB – to get clustered PK• INDEX provides opposite path• Conditionally insert: • INSERT IGNORE …, or • INSERT … ON DUPLICATE KEY UPDATE …
![Page 63: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/63.jpg)
Normalization CREATE TABLE Hosts ( id MEDIUMINT UNSIGNED -- 3 byte NOT NULL AUTO_INCREMENT, name VARCHAR(…) NOT NULL, PRIMARY KEY(id), UNIQUE(name) -- uniq; lookup ) ENGINE=InnoDB; -- clustering
![Page 64: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/64.jpg)
wp_postmeta CREATE TABLE wp_postmeta ( post_id …, meta_key …, meta_value …, PRIMARY KEY(post_id, meta_key), INDEX(meta_key) ) ENGINE=InnoDB;
• AUTO_INCREMENT was a waste
• Much better 'natural' PK; InnoDB to get clustering
• Use 191 if necessary; not "prefix" index
![Page 65: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/65.jpg)
Table Patterns - Q&A1 question (hold rest until end)
![Page 66: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/66.jpg)
Closing Let the questions flow! Rate My Session
These Slides / longer versionmysql.rjweb.org/slides/cook.pdfmysql.rjweb.org/doc.php/index_cookbook_mysql
Rick: [email protected]/stackoverflow.com - tag [mysql] [indexing]
![Page 67: Rick's MySQL Documents - Percona Live 2017mysql.rjweb.org › slides › cook.pdf · 2017-04-22 · MySQL INDEX Cookbook How to Build the Best INDEX for a Given SELECT Rick James.](https://reader034.fdocuments.in/reader034/viewer/2022042407/5f21ad939fbbee72597055b6/html5/thumbnails/67.jpg)