Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf ·...

53
Dimitri Vanoverbeke Practical MySQL Performance Optimization

Transcript of Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf ·...

Page 1: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

Dimitri Vanoverbeke

Practical MySQL Performance Optimization

Page 2: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 2

Contents of the presentation

How do we approach performance optimisations

Discuss practical optimisation tips.

What tools exist?

Page 3: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 3

MySQL performance

MySQL Performance

does not Matter

Page 4: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 4

What matters?

My Website/App is

important

Page 5: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 5

Things to Note

Performance issues can be unrelated to MySQL

Maybe the solution is not MySQL

Page 6: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 6

Many other tools

Cassandra MongoDB Hadoop ElasticSearch Redis

Page 7: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 7

Putting it in Practice

Keep an unbiased view

Focus on the Stack

Page 8: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 8

Performance “sandbags”

Security

Manageability

Compatibility

Compliance

Ease of use by developers

Page 9: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 9

“Good Enough”

KNOW WHEN TO

STOP

Page 10: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 10

So what is important in MySQL?

Queries!!!!

•  Inserts •  Selects •  Updates •  Deletes

Page 11: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 11

What are we aiming for?

Quickly, Efficiently, Reliably…

Page 12: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 12

Quickly

Good Response Time

For all Users and Use Cases

Page 13: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 13

Efficiency

Resources

CPU, Memory, IO, Disk Space

Dollars

Human Effort

Page 14: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 14

Reliably

High Availability!

High Availability and Performance Come together

Page 15: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 15

Consider things for queries

•  But avoid focus on extreme outliers

Look beyond averages

•  Daily and Weekly cycle •  Growth of load and Data Size

Make sure to look at trends

over time

•  Data size changes? Think about the Future

Page 16: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 16

Schema

Look at the Schema and queries together

•  Minor Schema changes •  Data Architecture

Page 17: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 17

Schema Tips

Learn indexing •  http://bit.ly/1rAtamE

Design Schema for Data access •  Rethink your schema

Technical voodoo •  Partitioning and Sharding •  Normalization and denormalization

Page 18: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 18

Other Performance influences

Infrastructure •  Storage systems, virtualization, etc.

Operating system and configuration •  Disk scheduler, FS, etc…

MySQL Version •  Are you using Percona Server, MariaDB, MySQL?

MySQL configuration •  The default configuration of Hell…

Page 19: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 19

Optimization Process

Low Hanging Fruit

•  MySQL Configuration

•  OS settings •  Indexes •  Caching

Medium Level

•  InfraStructure •  MySQL

Version

Hard Changes

•  Major schema changes

•  Application architecture changes

Page 20: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 20

QUICK TIPS

Page 21: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 21

Hardware

CPU

Memory

Disk

Network

Page 22: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 22

OS choices are simple

Linux

Server Grade

Recent

Well supporting MySQL

Page 23: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 23

OS tuning

Defaults are good for Common workload

Filesystem: EXT4 or XFS

More details: http://bit.ly/MySQLonLinux

Page 24: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 24

MySQL Version

New versions typically improve performance

Scalability, improved optimizer

Expect some regression

Try maybe a different flavor? Percona Server 5.6

Page 25: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 25

MySQL config optimisations

Do NOT run with defaults

Do not ever obsess with tuning

More details: http://bit.ly/1vth5Cu

Page 26: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 26

MySQL tuning in 1 slide

•  max_connections •  log_bin •  innodb_buffer_pool_size •  innodb_log_file_size •  innodb_flush_log_at_trx_commit •  innodb_flush_method=O_DIRECT

Page 27: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 27

TOOLS

Page 28: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 28

Tools to Consider

•  pt-query-digest (Percona Toolkit) •  MySQL Enterprise Monitor •  Monyog •  VividCortex •  Percona Cloud Tools (Free Beta)

•  http://cloud.percona.com

Page 29: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 29

pt-query-digest

Command Line tool from Percona Toolkit

Process Logs Manualy

Powerful Filtering

Storing History

Page 30: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 30

pt-query-digest example

Page 31: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 31

Top Queries

Page 32: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 32

Query Details

Page 33: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 33

More Query Details

Page 34: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 34

Percona Cloud Tools

Page 35: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 35

Query Profile

Page 36: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 36

Query Details

Page 37: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 37

Graphs!

Page 38: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 38

Different Stats and Metrics

Page 39: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 39

Process Support

Discover New Queries in the Application

Mark them as “Needing Attention” or “Reviewed”

Tag queries with information about application or user facing function

Discuss it with the team and record changes using Comments

Page 40: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 40

Choose Queries

Page 41: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 41

Review and Comment

Page 42: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 42

Where to Analyze ?

• Development to catch bad queries early

• Production to catch good queries starting to go bad

Both !

Page 43: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 43

Query Performance can show modifying parameters

Workload Changes

Data Size

Changing Query Plans

Upgrading MySQL Version

Changing MySQL Settings

Hardware Changes

Page 44: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 44

But Why Slow ?

• MySQL Level • OS Level • Hardware/Environment Level

Need to know more than query

level information

Page 45: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 45

“Metrics”

System and MySQL Information

Hundreds of metrics captured

1 second resolution capture

Real-Time Graphs Update

Page 46: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 46

Design

Graphs != Data

Can plot metrics you have available when you need it

Graphite + Ease of Use

Page 47: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 47

MySQL Level

Page 48: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 48

Beyond Average

Page 49: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 49

System Information

Page 50: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 50

Zoom In as Needed

Page 51: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 51

More To Come

Active Development in Progress

Try it out, its free http://cloud.percona.com

Let us know what you think!

Page 52: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 52

FINAL WORDS

Page 53: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •

www.percona.com 53 www.percona.com

Dim0 [email protected]

Thank You!