MySQL Performance Schema - percona.com · Metadata Always existed SHOW commands Information Schema...

MySQL Performance Schemain 1 hour

March, 21, 2019

Sveta Smirnova

•Why Performance Schema?

•How it Works?

•How to Tune?

•How to Ease?

Table of Contents

2

Why Performance Schema?

MetadataSchemaSupported charsetsetc.

RuntimeResource usageAcquired locksRunning queries

Difference

Metadata vs Runtime

4

MetadataAlways existed

• SHOW commands• Information Schema

RuntimeWas not perfect

• Status variables• Extensions for

Information Schema

History

Metadata vs Runtime

4

MetadataInformation Schema

RuntimePerformance Schema

Now

Metadata vs Runtime

4

• No design issues like in Information Schema• No locking reads

Almost

• Optimized• It took one and a half major versions

• Consistently extending• New features in every release

Easy to add your own• Being mature

Performance Schema

5

https://bugs.mysql.com/bug.php?id=93365

https://bugs.mysql.com/bug.php?id=93365

https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_PFS.html





Performance Schema

5





Easy to add your own

• Being mature

Performance Schema

5






Performance Schema

5


Instruments

Tables

Variables

102

1,229

44

87

1,067

42

52

589

31

17

237

16Percona Server

8.05.75.65.5

Performance Schema Over Years

6

5.5: Server insights$ iostat -p nvme0n1 5

...

avg-cpu: %user %nice %system %iowait %steal %idle

73.02 0.00 12.03 6.37 0.00 8.59

Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn

nvme0n1 1235.80 5.60 16662.40 28 83312

nvme0n1p3 1158.00 5.60 16662.40 28 83312

Features

7

5.5: Server insightsmysql> select thread_id, EVENT_NAME, sum(NUMBER_OF_BYTES/1024) sum_kb from events_waits_history

-> where NUMBER_OF_BYTES > 0 group by thread_id, EVENT_NAME order by sum_kb desc;

+-----------+---------------------------------------------------+-------------+

| thread_id | EVENT_NAME | sum_kb |

+-----------+---------------------------------------------------+-------------+

| 10 | wait/io/file/innodb/innodb_parallel_dblwrite_file | 1312.0000 |

| 117 | wait/io/file/sql/binlog | 3.4023 |

| 113 | wait/io/file/sql/binlog | 1.7012 |

| 58 | wait/io/file/sql/query_log | 0.2031 |

+-----------+---------------------------------------------------+-------------+

4 rows in set (0.00 sec)

Features

7

5.5: Server insights5.6: Statements

mysql> SELECT THREAD_ID TID, SUBSTR(SQL_TEXT, 1, 50) SQL_TEXT, ROWS_SENT RS,

-> ROWS_EXAMINED RE,CREATED_TMP_TABLES,NO_INDEX_USED,NO_GOOD_INDEX_USED

-> FROM performance_schema.events_statements_history

-> WHERE NO_INDEX_USED=1 OR NO_GOOD_INDEX_USED=1\G

********************** 1. row **********************

TID: 10124

SQL_TEXT: select emp_no, first_name, last_name from employee

RS: 97750

RE: 397774

CREATED_TMP_TABLES: 0

NO_INDEX_USED: 1

NO_GOOD_INDEX_USED: 0

...

Features

7

5.5: Server insights5.6: Statements5.6: Stages

mysql> SELECT eshl.event_name, sql_text, eshl.timer_wait/1000000000000 w_s

-> FROM performance_schema.events_stages_history_long eshl

-> JOIN performance_schema.events_statements_history_long esthl

-> ON (eshl.nesting_event_id = esthl.event_id)

-> WHERE eshl.timer_wait > 1*10000000000\G

*************************** 1. row ***************************

event_name: stage/sql/Sending data

sql_text: SELECT COUNT(emp_no) FROM employees JOIN salaries USING(emp_no)

WHERE hire_date=from_date

w_s: 0.8170

1 row in set (0.00 sec)

Features

7

5.7: Metadata Locksmysql> select id, db, state, info, time from information_schema.processlist;

+----+--------------------+---------------------------------+----------------------------+------+

| id | db | state | info | time |

+----+--------------------+---------------------------------+----------------------------+------+

| 96 | sbtest | | NULL | 65 |

| 4 | NULL | Waiting on empty queue | NULL | 5451 |

| 94 | performance_schema | executing | select id, db, state, i... | 0 |

| 95 | sbtest | Waiting for table metadata lock | alter table sbtest1 add... | 53 |

+----+--------------------+---------------------------------+----------------------------+------+


Features

7

5.7: Memory usage$ pidstat -r -p 13103

Linux 4.15.0-43-generic (delly) 01/20/2019 _x86_64_ (4 CPU)

09:36:55 PM UID PID minflt/s majflt/s VSZ RSS %MEM Command

09:36:55 PM 1001 13103 3.09 0.02 12133668 8228772 50.97 mysqld

Features

7

5.7: Memory usagemysql> select event_name, count_alloc, count_free,

-> CURRENT_NUMBER_OF_BYTES_USED/1024/1024 as used_mb

-> from memory_summary_global_by_event_name where CURRENT_NUMBER_OF_BYTES_USED > 0

-> order by used_mb desc limit 8;

+-------------------------------+-------------+------------+---------------+

| event_name | count_alloc | count_free | used_mb |

+-------------------------------+-------------+------------+---------------+

| memory/sql/JOIN_CACHE | 15 | 6 | 9216.00000000 |

| memory/innodb/ut0link_buf | 2 | 0 | 24.00004578 |

| memory/innodb/ut0new | 6 | 0 | 16.07953835 |

| memory/innodb/lock0lock | 6241 | 0 | 10.92063904 |

| memory/mysys/KEY_CACHE | 3 | 0 | 8.00205994 |

| memory/innodb/buf0dblwr | 378 | 370 | 5.80024719 |

| memory/innodb/memory | 15686935 | 15682985 | 4.90249634 |

| memory/innodb/ut0pool | 1 | 0 | 4.000022

+-------------------------------+-------------+------------+-----------


Features

7

5.7: Replicationmysql> show slave status \G

*************************** 1. row ***************************

Slave_IO_State: Waiting for master to send event

Master_Host: 127.0.0.1

Master_User: root

Master_Port: 13000

Connect_Retry: 60

Master_Log_File: master-bin.000002

Read_Master_Log_Pos: 63810611

Relay_Log_File: [email protected]

Relay_Log_Pos: 1156

Relay_Master_Log_File: master-bin.000001

Slave_IO_Running: Yes

Slave_SQL_Running: No

Replicate_Do_DB:

Replicate_Ignore_DB:

...

Features

7

5.7: Replication• IO Thread

replication connection status

replication connection status

• SQL Threadreplication applier configuration

replication applier filters

replication applier global filters

replication applier status

replication applier status by coordinator

replication applier status by worker

• Group replicationreplication group member stats

replication group members

Features

7

5.7: Variablesmysql> select * from variables_by_thread where variable_name=’join_buffer_size’;

+-----------+------------------+----------------+

| THREAD_ID | VARIABLE_NAME | VARIABLE_VALUE |

+-----------+------------------+----------------+

| 129 | join_buffer_size | 262144 |





...





| 148 | join_buffer_size | 1073741824 |

+-----------+------------------+----------------+


Features

7

5.7: Variablesmysql> select * from status_by_thread

-> where variable_name=’Handler_write’;

+-----------+---------------+----------------+


+-----------+---------------+----------------+

| 71 | Handler_write | 94 |

| 83 | Handler_write | 477 | -- Most writes

| 84 | Handler_write | 101 |

+-----------+---------------+----------------+


Features

7

5.7: Variablesmysql> select * from user_variables_by_thread;

+-----------+---------------+----------------+


+-----------+---------------+----------------+

| 71 | baz | boo |

| 84 | foo | bar |

+-----------+---------------+----------------+


Features

7

8.0: Errorsmysql> select ERROR_NUMBER NUM, ERROR_NAME, SUM_ERROR_RAISED SUM, FIRST_SEEN, LAST_SEEN

-> from events_errors_summary_global_by_error where SUM_ERROR_RAISED > 0

-> order by SUM_ERROR_RAISED desc;

+------+---------------------------+-------+---------------------+---------------------+

| NUM | ERROR_NAME | SUM | FIRST_SEEN | LAST_SEEN |

+------+---------------------------+-------+---------------------+---------------------+

| 1213 | ER_LOCK_DEADLOCK | 17986 | 2019-01-20 20:51:34 | 2019-01-20 23:05:43 |

| 1287 | ER_WARN_DEPRECATED_SYNTAX | 207 | 2019-01-20 20:51:41 | 2019-01-20 23:03:29 |

| 1158 | ER_NET_READ_ERROR | 96 | 2019-01-20 20:57:36 | 2019-01-20 23:05:44 |

| 1205 | ER_LOCK_WAIT_TIMEOUT | 72 | 2019-01-20 20:52:19 | 2019-01-20 23:04:57 |

| 1295 | ER_UNSUPPORTED_PS | 64 | 2019-01-20 20:51:33 | 2019-01-20 23:03:26 |

| 3719 | ER_DEPRECATED_UTF8_ALIAS | 33 | 2019-01-20 20:50:33 | 2019-01-20 20:50:34 |

| 1064 | ER_PARSE_ERROR | 2 | 2019-01-20 21:11:33 | 2019-01-20 21:11:33 |

| 1317 | ER_QUERY_INTERRUPTED | 2 | 2019-01-20 21:24:23 | 2019-01-20 21:29:37 |

| 1054 | ER_BAD_FIELD_ERROR | 1 | 2019-01-20 22:14:57 | 2019-

| 1066 | ER_NONUNIQ_TABLE | 1 | 2019-01-20 21:23:45 | 2019

| 1160 | ER_NET_ERROR_ON_WRITE | 1 | 2019-01-20 21:51:36 | 2019-

...

Features

7

5.7: Prepared statements5.7: Stored routines5.7: Metadata Locks5.7: Memory usage5.7: Replication5.7: Variables8.0: Errors

• More

Features

7

How it Works?

• Wraps the diagnosed codelocker = PSI_RWLOCK_CALL(start_rwlock_wrwait)(

&state, lock->pfs_psi, PSI_RWLOCK_TRYEXCLUSIVELOCK,

file_name, static_cast<uint>(line));

ret = rw_lock_x_lock_func_nowait(lock, file_name, line);

if (locker != NULL) {

PSI_RWLOCK_CALL(end_rwlock_wrwait)(locker, static_cast<int>(ret));

}

Gets the Data

9

CREATE TABLE ‘events_NAME_current‘ (

‘[OWNER_]THREAD_ID‘ bigint(20) unsigned NOT NULL,

‘[OWNER_]EVENT_ID‘ bigint(20) unsigned NOT NULL,

‘END_EVENT_ID‘ bigint(20) unsigned DEFAULT NULL,

‘EVENT_NAME‘ varchar(128) NOT NULL,

‘SOURCE‘ varchar(64) DEFAULT NULL,

‘TIMER_START‘ bigint(20) unsigned DEFAULT NULL,

‘TIMER_END‘ bigint(20) unsigned DEFAULT NULL,

‘TIMER_WAIT‘ bigint(20) unsigned DEFAULT NULL,

‘OBJECT_SCHEMA‘ varchar(64) DEFAULT NULL,

‘OBJECT_NAME‘ varchar(512) DEFAULT NULL,

‘OBJECT_TYPE‘ varchar(64) DEFAULT NULL,

‘OBJECT_INSTANCE_BEGIN‘ bigint(20) unsigned NOT NULL,

‘NESTING_EVENT_ID‘ bigint(20) unsigned DEFAULT NULL,

‘NESTING_EVENT_TYPE‘ enum(’TRANSACTION’,’STATEMENT’,’STAGE’,’WAIT’) DEFAULT NULL,

EVENT-SPECIFIC-FIELDS

) ENGINE=PERFORMANCE_SCHEMA

Stores the Data

10

• SQL• Same basic structure for events * tables• Others are unified whenever possible

Returns the Data

11

How Does Performance Schema Works?

12

Instruments: Get the Data

13

• Every instrumented call is wrapped intotwo Performance Schema calls

• For rarely used calls - no performance impact• Memory allocation• Statement-related data

• Significant impact for often used

Performance Note

14

• Table of 2048 rows• select * from joinit;• Test preparation

mysql> truncate performance_schema.events_waits_history_long;

Query OK, 0 rows affected (0.02 sec)

-- OR

mysql> truncate performance_schema.events_statements_history_long;

Query OK, 0 rows affected (0.02 sec)

Performance Example

15

• Table of 2048 rows• select * from joinit;• The test

mysql> \P sha1sum

PAGER set to ’sha1sum’

mysql> select * from joinit;

7f188e9c9fd628613eab85366f0729604b8b5e3e -


Performance Example

15

• Table of 2048 rows• select * from joinit;• events waits*

mysql> select event_name, count(*) from performance_schema.events_waits_history_long

-> where thread_id=42 group by EVENT_NAME order by ‘count(*)‘ desc;

+---------------------------------------------+----------+

| event_name | count(*) |

+---------------------------------------------+----------+

| wait/synch/mutex/innodb/rw_lock_debug_mutex | 322 |

| wait/synch/sxlock/innodb/hash_table_locks | 12 |

| wait/synch/mutex/innodb/trx_mutex | 12 |

| wait/synch/mutex/innodb/fil_system_mutex | 12 |

| wait/synch/mutex/sql/THD::LOCK_query_plan | 4 |

| wait/synch/mutex/sql/THD::LOCK_thd_data | 3 |

| wait/synch/mutex/sql/LOCK_table_cache | 2 |

...

Performance Example

15

• Table of 2048 rows• select * from joinit;• events statements*

mysql> select event_name, count(*) from performance_schema.events_statements_history_long

-> where thread_id=42 group by EVENT_NAME;

+----------------------+----------+

| event_name | count(*) |

+----------------------+----------+

| statement/sql/select | 1 |

+----------------------+----------+

1 row in set (0.02 sec)

Performance Example

15

• Table of 2048 rows• select * from joinit;• Depends on the instrument!

Performance Example

15

Consumers: Store the Data

16

• Must be supported by the component• Collects data only after they are enabled• Never frees allocated memory

Performance Schema Limitations

17

How to Tune?

• Defaults• ON

5.7: Only global, thread, statements and transactionsinstrumentation

8.0: Memory and MDL

• All other instruments/consumers disabled

Performance Schema Defaults

19

• Defaults• ON

5.7: Only global, thread, statements and transactionsinstrumentation

8.0: Memory and MDL• All other instruments/consumers disabled

Performance Schema Defaults

19

• Configuration options• setup * tables

Two Places

20

• performance schema = ON

• Instruments and consumers, enabled at startup• Pattern

--performance-schema-consumer-consumer name=value

--performance-schema-instrument=’instrument name=value’

• Examples--performance-schema-consumer-events transactions history long=ON

--performance-schema-instrument=’%=OFF’

--performance-schema-instrument=’memory/innodb/%=ON’

--performance-schema-instrument=’wait/lock/metadata/sql/mdl=ON’

Startup Configuration

21

• performance schema = ON• Limits of table size and number of

instrumented instances• Not dynamic!• Examples

performance schema accounts size

performance schema max thread instances

performance schema max thread classes

• Instruments and consumers, enabled at startup• Pattern

--performance-schema-consumer-consumer name=value







21

• performance schema = ON• Instruments and consumers, enabled at startup

• Pattern--performance-schema-consumer-consumer name=value







21

• setup * tables

Runtime Configuration

22

• setup * tables• Actors: user connections

mysql> select * from setup_actors;

+------+------+------+---------+---------+

| HOST | USER | ROLE | ENABLED | HISTORY |

+------+------+------+---------+---------+

| % | pfs | % | YES | YES |

| % | root | % | NO | NO |

| % | pfs2 | % | YES | NO |

+------+------+------+---------+---------+



22

• setup * tables• Threads: thread classes

mysql> select * from setup_threads\G

...

*************************** 5. row ***************************

NAME: thread/sql/one_connection

ENABLED: YES

HISTORY: YES

PROPERTIES: user

VOLATILITY: 0

DOCUMENTATION: NULL

...


22

• setup * tables• Threads: thread classes

...

*************************** 42. row ***************************

NAME: thread/sql/slave_io

ENABLED: YES

HISTORY: YES

PROPERTIES: singleton

VOLATILITY: 0

DOCUMENTATION: NULL

...


22

• setup * tables• Objects

• Types of events, their names and schemamysql> select * from setup_objects;

+-------------+---------------+-------------+---------+-------+

| OBJECT_TYPE | OBJECT_SCHEMA | OBJECT_NAME | ENABLED | TIMED |

+-------------+---------------+-------------+---------+-------+

| EVENT | % | % | NO | NO |

| FUNCTION | % | % | NO | NO |

| PROCEDURE | % | % | NO | NO |

| TABLE | % | % | NO | NO |

| TRIGGER | % | % | NO | NO |

| EVENT | my_schema | % | YES | YES |

| TABLE | my_schema | % | YES | YES |

+-------------+---------------+-------------+---------+-------+



22

• setup * tables• Consumers: what to store

mysql> select * from setup_consumers;

+----------------------------------+---------+

| NAME | ENABLED |

+----------------------------------+---------+

| events_stages_current | YES |

| events_stages_history | YES |

| events_stages_history_long | NO |

| events_statements_current | YES |

| events_statements_history | YES |

| events_statements_history_long | NO |

| events_transactions_current | YES |

| events_transactions_history | YES |

| events_transactions_history_long | NO |

...


22

• setup * tables• Instruments: what to instrument

mysql> select NAME, ENABLED, TIMED, PROPERTIES, VOLATILITY from setup_instruments

-> order by rand() limit 3;

+-------------------------------------------+---------+-------+------------+------------+

| NAME | ENABLED | TIMED | PROPERTIES | VOLATILITY |

+-------------------------------------------+---------+-------+------------+------------+

| wait/synch/mutex/pfs/LOCK_pfs_share_list | NO | NO | singleton | 1 |

| memory/csv/Transparent_file | YES | NULL | | 0 |

| wait/synch/mutex/sql/THD::LOCK_thd_data | YES | YES | | 5 |

+-------------------------------------------+---------+-------+------------+------------+



22

How to Ease?

• Views on Performance Schema tables• Stored routines

• Easier configuration• Shortcuts to typical use cases• 5.7+: Installed by default• Before: github.com/mysql/mysql-sys

sys Schema

24

github.com/mysql/mysql-sys

• Views on Performance Schema tables• Stored routines• Easier configuration• Shortcuts to typical use cases

• 5.7+: Installed by default• Before: github.com/mysql/mysql-sys

sys Schema

24


• Views on Performance Schema tables• Stored routines• Easier configuration• Shortcuts to typical use cases• 5.7+: Installed by default• Before: github.com/mysql/mysql-sys

sys Schema

24


• statement analysis• statements with full table scans• statements with runtimes in 95th percentile• statements with sorting• statements with temp tables• statements with errors or warnings• memory by thread by current bytes• More

Nicer Views

25

• Performance Schema• SELECTTHREAD ID, DIGEST, ROWS SENT RS,ROWS EXAMINED, CREATED TMP TABLES,NO INDEX USED, NO GOOD INDEX USEDFROMperformance schema.events statements historyWHERE NO INDEX USED=1 ORNO GOOD INDEX USED=1

• sys schema• select * fromsys.statements with full table scans

Compare!

26

• sys schema• select * fromsys.statements with full table scans

Compare!

26

• Enablecall sys.ps_setup_enable_consumer(YOUR_CONSUMER);

call sys.ps_setup_enable_instrument(YOUR_INSTRUMENT);

Easier configuration

27

• Disablecall sys.ps_setup_disable_consumer(YOUR_CONSUMER);

call sys.ps_setup_disable_instrument(YOUR_INSTRUMENT);


27

• Examinecall sys.ps_setup_show_disabled_consumers();

call sys.ps_setup_show_disabled_instruments();

call sys.ps_setup_show_enabled_consumers();

call sys.ps_setup_show_enabled_instruments();


27

Performance Schema Dashboard

23:05:47

PMM

28


23:32:15

PMM

28


22:27:15

PMM

28


18:08:15

PMM

28

Blog of MySQL developers team

Mark Leith: author of sys schema

Official reference manual

Webinar ”Performance Schema for MySQLTroubleshooting”PMM Demo

More information

29

http://mysqlserverteam.com/category/mysql/performance-schema/

http://mysqlserverteam.com/category/mysql/performance-schema/

http://www.markleith.co.uk/

http://www.markleith.co.uk/

https://dev.mysql.com/doc/refman/8.0/en/performance-schema.html

https://dev.mysql.com/doc/refman/8.0/en/performance-schema.html

https://www.percona.com/resources/webinars/performance-schema-mysql-troubleshooting



https://pmmdemo.percona.com

https://pmmdemo.percona.com

www.slideshare.net/SvetaSmirnova

twitter.com/svetsmirnova

github.com/svetasmirnova

Thank you!

30

http://www.slideshare.net/SvetaSmirnova

www.slideshare.net/SvetaSmirnova

https://twitter.com/svetsmirnova

twitter.com/svetsmirnova

https://github.com/svetasmirnova

github.com/svetasmirnova

DATABASE PERFORMANCEMATTERS

MySQL Performance Schema - percona.com · Metadata Always existed SHOW commands Information Schema...

Documents

Transcript of MySQL Performance Schema - percona.com · Metadata Always existed SHOW commands Information Schema...