MySQL Buffer Management

1/31/2017 1

Mijin Ahnmeeeeejin@gmail.com

Contents

1/31/2017 2

• Overview

• Buffer Pool

• Buffer Read

• LRU Replacement

• Flusher

• Doublewrite Buffer

• Synchronization

OVERVIEW

1/31/2017 3

Overview

1/31/2017 4

• Buffer Pool

– Considering memory hierarchy

– Caching frequently accessed data into DRAM like a cache memory in CPU

– Exploit locality

Overview

• InnoDB Architecture

1/31/2017 5

Handler API

Transaction (trx)

Logging & Crash

Recovery(log & recv)

Mini Transaction

Lock(lck)

Cursor (cur)

Row (row)

B-tree (btr)

Page (page)

Buffer Manager (buf)

Free space / File Management (fsp / fil)

Overview

• Buffer Manager– Buffer Pool (buf0buf.cc) : buffer pool manager

– Buffer Read (buf0read.cc) : read buffer

– LRU (buf0lru.cc) : buffer replacement

1/31/2017 6

Overview

• Buffer Manager– Flusher (buf0flu.cc) : dirty page writer & background flusher

– Doublewrite (buf0dblwr.cc) : doublewrite buffer

1/31/2017 7

BUFFER POOL

1/31/2017 8

Lists of Buffer Blocks

• Free list

– Contains free page frames

• LRU list

– Contains all the blocks holding a file page

• Flush list

– Contains the blocks holding file pages that have been modified in the memory but not written to disk yet

1/31/2017 9

DatabaseBuffer

TailHead D D D

Main LRU list

Free list

Flush list D D

Buffer Pool Mutex

• The buffer buf_pool contains a single mutex

– buf_pool->mutex: protects all the control data structures of the buf_pool

• The buf_pool->mutex is a hot-spot in main memory

– Causing a lot of memory bus traffic on multiprocessor systems when processors alternatively access the mutex

– A solution to reduce mutex contention• To create a separate lock for the page hash table

1/31/2017 10

Buffer Pool Struct

• include/buf0buf.h: buf_pool_t

1/31/2017 11

Buffer Pool Struct

1/31/2017 12

Buffer Pool Struct

1/31/2017 13

Buffer Pool Struct

1/31/2017 14

Buffer Pool Init

• buf/buf0buf.cc: buf_pool_init()

1/31/2017 15

Buffer pool init per instance

Buffer Pool Init

• buf/buf0buf.cc: buf_pool_init_instance()

1/31/2017 16

Create mutex for buffer pool

Buffer Pool Init

1/31/2017 17

Initialize buffer chunk

Buffer Pool Init

1/31/2017 18

Create page hash table

Buffer Chunk

• Total buffer pool size

= x * innodb_buffer_pool_instances * innodb_buffer_pool_chunk_size

1/31/2017 19

Buffer Chunk Struct

• include/buf0buf.ic: buf_chunk_t

1/31/2017 20

[0] [1] [3][2]

[…] […] [N][…]blocks

frames

Buffer chunk mem

• buf/buf0buf.cc: buf_chunk_init()

Buffer Pool Init

1/31/2017 21

Allocate chunk mem(blocks + frames)

Buffer Pool Init

1/31/2017 22

Allocate control blocks

Allocate frames(Page size is aligned)

Buffer Pool Init

1/31/2017 23

Add all blocks to free list

Initialize control block

Buffer Control Block (BCB)

• The control block contains

– Read-write lock

– Buffer fix count• Which is incremented when a thread wants a file page to be fixed in a buffer

• The buffer fix operation does not lock the contents of the frame

– Page frame

– File pages• Put to a hash table according to the file address of the page

1/31/2017 24

Buffer Control Block (BCB)

1/31/2017 25

Frame ptr

RW lock

Lock hash value

buf_block_t

Table space ID

Page offset

Buffer fix count

IO fix

List (LRU, free, flush)

buf_page_t

• include/buf0buf.h: buf_block_t

BCB Struct

1/31/2017 26

• include/buf0buf.h: buf_block_t

BCB Struct

1/31/2017 27

• include/buf0buf.h: buf_page_t

BCB Struct

1/31/2017 28

Page identification

BCB Struct

1/31/2017 29

BCB Struct

1/31/2017 30

• buf/buf0buf.cc: buf_block_init()

BCB Init

1/31/2017 31

Set data frame

... Create block mutex & rw lock

BUFFER READPART 1 : READ A PAGE

1/31/2017 32

MySQL Buffer Manager: Read

Database on Flash SSD

DatabaseBuffer

3. Read a page

1. Search free list

Head D D D

Main LRU List

Free list

Dirty Page Set

Scan LRU List from tail

Double Write Buffer

2. Flush Dirty Pages

Buffer Read

• Read a page (buf0rea.cc)

– Find a certain page in the buffer pool using hash table

– If it is not in the buffer pool, then read a block from the storage• Allocate a free block for read (include buffer block)

• Two cases

– Buffer pool has free blocks

– Buffer pool doesn’t have a free block

• Read a page

1/31/2017 34

Buffer Read

1/31/2017 35

Buffer Read

• buf/buf0buf.cc: buf_page_get_gen()

1/31/2017 36

Get the buffer pool ptr using space & offset

** 2 important things **1) ID of a page is (space, offset) of the page2) Buffer pool – page mapping is mapped

Buffer Read

• include/buf0buf.ic: buf_pool_get()

1/31/2017 37

Make a fold number

Buffer Read

• Why fold?

– They want to put pages together in the same buffer pool if it is the same extents for read ahead

1/31/2017 38

Fold 0 Fold 1

Buffer Read

1/31/2017 39

Get page hash lock before searching in the hash table

Set shared lock on hash table

Buffer Read

1/31/2017 40

Find a page in the hash table

Page doesn’t exist in buffer pool

Read the page from the storage

success

Buffer Read

1/31/2017 41

Buffer Read

1/31/2017 42

If it failed to read target page, go to the first part of the loop

Buffer Read

• buf/buf0rea.cc: buf_read_page()

1/31/2017 43

Buffer Read

• buf/buf0rea.cc: buf_read_page_low()

1/31/2017 44

Buffer Read

1/31/2017 45

Allocate buffer block for read

Buffer Read

• buf/buf0buf.cc: buf_page_init_for_read()

1/31/2017 46

Buffer Read

1/31/2017 47

Get free block: see this later

Buffer Read

1/31/2017 48

Initialize buffer page for current read

Buffer Read

• buf/buf0buf.cc: buf_page_init()

1/31/2017 49

Buffer Read

1/31/2017 50

Buffer Read

1/31/2017 51

Insert a page into hash table

Buffer Read

1/31/2017 52

Add current block to LRU list: see this later

Set io fix to BUF_IO_READ

Buffer Read

1/31/2017 53

Increase pending read count;How many buffer read were requested but not finished

Buffer Read

• We allocate a free buffer and control block

• And the block was inserted into hash table and LRU list of corresponding buffer pool

• Now, we need to read the real content of the target page from the storage

1/31/2017 54

Buffer Read

1/31/2017 55

Read a page from storage: see this later

Buffer Read

1/31/2017 56

Complete the read request

Buffer Read

• buf/buf0buf.cc: buf_page_io_complete()

1/31/2017 57

Get io type (In this case, BUF_IO_READ)

Buffer Read

1/31/2017 58

Get io type (In this case, BUF_IO_READ)

Page corruption check based on checksum in the page

Buffer Read

1/31/2017 59

Set io fix to BUF_IO_NONE

Buffer Read

1/31/2017 60

Decrease pending read count

BUFFER READPART 2 : AFTER GOT BLOCK

1/31/2017 61

Buffer Read

1/31/2017 62

Set access time

Buffer Read

1/31/2017 63

Do read ahead process (default=false)

LRU REPLACEMENTPART 1 : ADD BLOCK

1/31/2017 64

LRU Add Block

1/31/2017 65

Add current block to LRU list

LRU Add Block

• buf/buf0lru.cc: buf_LRU_add_block()

1/31/2017 66

LRU Add Block

• buf/buf0lru.cc: buf_LRU_add_block_low()

1/31/2017 67

If list is too small, then put current block to first of the list

LRU Add Block

• buf/buf0lru.cc: buf_LRU_add_block_low()

1/31/2017 68

Else, insert current block to after LRU_old pointer

LRU REPLACEMENTPART 2 : GET FREE BLOCK

1/31/2017 69

LRU Get Free Block

• This function is called from a user thread when it needs a clean block to read in a page

– Note that we only ever get a block from the free list

– Even when we flush a page or find a page in LRU scan we put it to free list to be used

1/31/2017 70

LRU Get Free Block

• iteration 0:

– get a block from free list, success: done

– if there is an LRU flush batch in progress:• wait for batch to end: retry free list

– if buf_pool->try_LRU_scan is set• scan LRU up to srv_LRU_scan_depth to find a clean block

• the above will put the block on free list

• success: retry the free list

– flush one dirty page from tail of LRU to disk (= single page flush)• the above will put the block on free list

• success: retry the free list

1/31/2017 71

LRU Get Free Block

• iteration 1:

– same as iteration 0 except:• scan whole LRU list

• iteration > 1:

– same as iteration 1 but sleep 100ms

1/31/2017 72

LRU Get Free Block

• buf/buf0lru.cc: buf_LRU_get_free_block()

1/31/2017 73

Get buffer pool mutex

Get a free block from the free list

LRU Get Free Block

1/31/2017 74

Getting a free block succeeded

LRU Get Free Block

1/31/2017 75

If already background flushed started, wait for it to end

LRU Get Free Block

1/31/2017 76

Find a victim page to replace and make a free block

LRU Get Free Block

• buf/buf0lru.cc: buf_LRU_scan_and_free_block()

1/31/2017 77

LRU Get Free Block

• buf/buf0lru.cc: buf_LRU_scan_and_free_block()

1/31/2017 78

LRU Get Free Block

• buf/buf0lru.cc: buf_LRU_free_from_common_LRU_list()

1/31/2017 79

Try to free it

LRU Get Free Block

• buf/buf0lru.cc: buf_LRU_free_page()

1/31/2017 80

Get hash lock & block mutex

LRU Get Free Block

• buf/buf0lru.cc: buf_LRU_free_page()

1/31/2017 81

If current page is dirty and not flushed to disk yet, exit

• buf/buf0lru.cc: buf_LRU_free_page()– After func_exit, we are on clean case!

LRU Get Free Block

1/31/2017 82

• buf/buf0lru.cc: buf_LRU_block_remove_hashed()

LRU Get Free Block

1/31/2017 83

LRU Get Free Block

1/31/2017 84

If we have free block(s), go to loop

LRU Get Free Block

1/31/2017 85

If we failed to make a free block, do a single page flush

FLUSH A PAGEPART 1 : SINGLE PAGE FLUSH

1/31/2017 86

Flush a Page

• Flushing a page by

– A background flusher batch flush (LRU & flush list)

– A single page flush in LRU_get_free_block()

• Background flusher

– Regularly check system status (per 1000ms)

– Flush all buffer pool instances in a batch manner

1/31/2017 87

Single Page Flush

1/31/2017 88

buf_LRU_get_free_block()

buf_flush_single_page_from_LRU()

buf_flush_ready_for_flush() buf_flush

_page()buf_flush_write_block_low()

buf_dblwr_write_single_page()

fil_io()

fil_flush()

buf_page_io_complete()

• buf/buf0flu.cc: buf_flush_single_page_from_LRU()

Single Page Flush

1/31/2017 89

Full scan

• buf/buf0flu.cc: buf_flush_single_page_from_LRU()

Single Page Flush

1/31/2017 90

Check whether we can flush current block and ready for flush

Try to flush it; write to disk

• buf/buf0flu.cc: buf_flush_ready_for_flush()

Single Page Flush

1/31/2017 91

If the page is already flushed or doing IO, return false

• buf/buf0flu.cc: buf_flush_page()

– Writes a flushable page from the buffer pool to a file

Single Page Flush

1/31/2017 92

Get lock

• buf/buf0flu.cc: buf_flush_page()

Single Page Flush

1/31/2017 93

Set fix and flush type

• buf/buf0flu.cc: buf_flush_write_block_low()

Single Page Flush

1/31/2017 94

Flush log (transaction log – WAL)

Single Page Flush

1/31/2017 95

Doublewrite off case

Write the page to dwb, then write to datafile;See this in dwb part

Single Page Flush

1/31/2017 96

Sync buffered write to disk; call fsync by fil_flush()

FLUSH A PAGEPART 2 : BATCH FLUSH

1/31/2017 97

Batch Flush

• Background flusher (= page cleaner thread)

– Independent thread for flushing a dirty pages from buffer pools to storage

– Regularly (per 1000ms) do flush from LRU tail or

– Do flush by dirty page percent (configurable)

• Thread definition

– buf/buf0flu.cc: DECLARE_THREAD(buf_flush_page_cleaner_thread)

1/31/2017 98

• buf/buf0flu.cc: DECLARE_THREAD(buf_flush_page_cleaner_thread)

Background Flusher

1/31/2017 99

Run until shutdown

• buf/buf0flu.cc: DECLARE_THREAD(buf_flush_page_cleaner_thread)

Background Flusher

1/31/2017 100

Nothing hasbeen changed

Something hasbeen changed!

• buf/buf0flu.cc: buf_flush_LRU_tail()

• Clears up tail of the LRU lists:

– Put replaceable pages at the tail of LRU to the free list

– Flush dirty pages at the tail of LRU to the disk

• srv_LRU_scan_depth

– Scan each buffer pool at this amount

– Configuable: innodb_LRU_scan_depth

LRU List Batch Flush

1/31/2017 101

1/31/2017 102

buf_flush_LRU_tail()

buf_flush_LRU()

buf_flush_batch()

buf_do_LRU_batch()

buf_flush_LRU_list_batch()

buf_LRU_free_page()

fil_io()

fil_flush()

buf_page_io_complete()

buf_flush_page_and_try_neighbors()

buf_flush_page()

buf_flush_write_block_low()

buf_dblwr_add_to_batch()

buf_dblwr_flush_

buffered_writes()

1/31/2017 103

Per buffer pool instance

1/31/2017 104

Chunk size = 100

• buf/buf0flu.cc: buf_flush_LRU()

1/31/2017 105

Batch LRU flush

• buf/buf0flu.cc: buf_flush_batch()

1/31/2017 106

Do LRU batch

• buf/buf0flu.cc: buf_do_LRU_batch()

1/31/2017 107

• buf/buf0flu.cc: buf_flush_LRU_list_batch()

1/31/2017 108

Get the last page from LRU

1/31/2017 109

• buf/buf0flu.cc: buf_flush_ready_for_replace()

1/31/2017 110

Check whether current page is clean page or not

1/31/2017 111

It there is any replaceable page, free the page

1/31/2017 112

Else, try to flush neighbor pages

• buf/buf0flu.cc: buf_flush_page_and_try_neighbors()

1/31/2017 113

Flush page, but no sync

1/31/2017 114

Add the page to the dwb buffer;See this in dwb part

• Now, victim pages are gathered for replacement

• We need to flush them to disk

• We can do this by calling buf_flush_common()

1/31/2017 115

• buf/buf0flu.cc: buf_flush_LRU()

1/31/2017 116

• buf/buf0flu.cc: buf_flush_common()

1/31/2017 117

• flush all pages we gathered so far• write the pages to dwb area first• then issue it to datafile; See this later

DOUBLEWRITE BUFFERPART 1 : ARCHITECTURE

1/31/2017 118

Double Write Buffer

• To avoid torn page (partial page) written problem

• Write dirty pages to special storage area in system tablespace priori to write database file

1/31/2017 119

Database on Flash SSD

DatabaseBuffer

TailHead D D D

Main LRU List

Free list

Dirty Page Set

Scan LRU List from tail

Double Write Buffer

ibdata1-space0datafiles

DWB Architecture

1/31/2017 120

• include/buf0dblwr.cc: buf_dblwr_t

DWB Struct

1/31/2017 121

• include/buf0dblwr.cc: buf_dblwr_t

DWB Struct

1/31/2017 122

DOUBLEWRITE BUFFERPART 2 : SINGLE PAGE FLUSH

1/31/2017 123

Single Page Flush

1/31/2017 124

• buf/buf0dblwr.cc: buf_dblwr_write_single_page()

Single Page Flush

1/31/2017 125

# of slots for single page flush= 2 * DOUBLEWRITE_BLOCK_SIZE – BATCH_SIZE= 128 – 120= 8

Single Page Flush

1/31/2017 126

If all slots are reserved, wait until current

dblwr done

Find a free slot

Single Page Flush

1/31/2017 127

Write to datafile(synchronous)

Sync system tablespace for dwb area in disk

Write block to dwb area in disk(synchronous)

• buf/buf0dblwr.cc: buf_dblwr_write_block_to_datafile()

Single Page Flush

1/31/2017 128

Issue write operation to datafile

In single page flush, sync = true

Single Page Flush

1/31/2017 129

Sync buffered write to disk; call fsync by fil_flush()

Single Page Flush

1/31/2017 130

Get io type (In this case, BUF_IO_WRITE)

Single Page Flush

1/31/2017 131

Set io fix to BUF_IO_NONE

Single Page Flush

1/31/2017 132

Single Page Flush

• buf/buf0flu.cc: buf_flush_write_complete()

1/31/2017 133

Remove the block from the flush list

Single Page Flush

• buf/buf0dblwr.cc: buf_dblwr_update()

1/31/2017 134

Free the dwb slot of the target page

Single Page Flush

• Single page flush is performed in the context of the query thread itself

• Single page flush mode iterates over the LRU list of a buffer pool instance, while holding the buffer pool mutex

• It might have trouble in getting a free doublewrite buffer slot (total 8 slots)

• In result, it makes the overall performance worse

1/31/2017 135

DOUBLEWRITE BUFFERPART 3 : BATCH FLUSH

1/31/2017 136

Batch Flush

1/31/2017 137

• buf/buf0dblwr.cc: buf_dblwr_add_to_batch()

Batch Flush

1/31/2017 138

If another batch is already running, wait until done

Batch Flush

1/31/2017 139

If all slots for batch flush in dwb buffer is reserved,

flush dwb buffer

Batch Flush

1/31/2017 140

After flushing, copy current block to buf_dblwr->first_free

• buf/buf0dblwr.cc: buf_dblwr_flush_buffered_writes()

Batch Flush

1/31/2017 141

• buf/buf0dblwr.cc: buf_dblwr_sync_datafiles()

Batch Flush

1/31/2017 142

fil_flush() os_file_flush() os_file_fsync() fsync()

Batch Flush

1/31/2017 143

Change batch running status

Exit mutex

Nobody won’t be here except me!

Batch Flush

1/31/2017 144

Issue write op for block1(synchronous)

If current write uses only block1, then flush

Batch Flush

1/31/2017 145

Issue write op for block2(synchronous)

Batch Flush

1/31/2017 146

Flush (fsync) system table space

Write all blocks to datafile(asynchronous)

• After submitting aio requests,

– fil_aio_wait()

– buf_page_io_complete()

– buf_flush_write_complete()

– buf_dblwr_update()

Batch Flush

1/31/2017 147

• buf/buf0dblwr.cc: buf_dblwr_update()

Batch Flush

1/31/2017 148

Flush datafile

Reset dwb

SYNCHRONIZATION

1/31/2017 149

InnoDB Synchronization

• InnoDB implements its own mutexes & RW-locks for buffer management

• Latch in InnoDB

– A lightweight structure used by InnoDB to implement a lock

– Typically held for a brief time (milliseconds or microseconds)

– A general term that includes both mutexes (for exclusive access) and rw-locks (for shared access)

1/31/2017 150

Mutex in InnoDB

• The low-level object to represent and enforce exclusive-access locks to internal in-memory data structures

• Once the lock is acquired, any other process, thread, and so on is prevented from acquiring the same lock

1/31/2017 151

Mutex in InnoDB

1/31/2017 152

• Example code in InnoDB

– buf/buf0buf.cc: buf_wait_for_read()

Get current IO fix of the block

Mutex in InnoDB

1/31/2017 153

• Example code in InnoDB

– buf/buf0flu.cc: buf_flush_batch()

Flush the pages in the buffer pool

RW-lock in InnoDB

• The low-level object to represent and enforce shared-access locks to internal in-memory data structures

• RW-lock includes three types of locks

– S-locks (shared locks)

– X-locks (exclusive locks)

– SX-locks (shared-exclusive locks)

1/31/2017 154

S SX X

S Compatible Compatible Conflict

SX Compatible Conflict Conflict

X Conflict Conflict Conflict

RW-lock in InnoDB

• S-lock (Shared-lock) – provides read access to a common resource

• X-lock (eXclusive-lock)– provides write access to a common resource

– while not permitting inconsistent reads by other threads

• SX-lock (Shared-eXclusive lock)– provides write access to a common resource

– while permitting inconsistent reads by other threads

– introduced in MySQL 5.7 to optimize concurrency and improve scalability for read-write workloads.

1/31/2017 155

RW-lock in InnoDB

1/31/2017 156

• Example code in InnoDB (S-lock)

– buf/buf0buf.cc: buf_page_get_gen()

Search hash table

RW-lock in InnoDB

1/31/2017 157

• Example code in InnoDB (X-lock)

– buf/buf0buf.cc: buf_page_init_for_read()

Insert a page into the hash table

MySQL Buffer Management

Software

Transcript of MySQL Buffer Management

MySQL Introduction - Prof. Bhavana Khivsara · Introduction to MySQL MySQL is the most popular Open Source Relational SQL Database Management System. MySQL is one of the best RDBMS

Competitive Buffer Management with Packet Dependencies

LIQUIDITY BUFFER MANAGEMENT · Liquidity buffer management - October 2016 #2. Customized asset management to meet banks’ specific requirements When management is delegated, investment

MySQL 8.0 Performance - FOSDEM · MySQL 8.0 : Re-Designed REDO REDO Log Buffer Log Writer Log Flusher •New design tradeoffs… • 1 user / low load => event-driven is slower •

1 Introduction to MySQL & SQL. MySQL Relational Database Management System Relational Database Management System Competitors Competitors –Oracle –IBM.

Efficient Buffer Management

Playout Buffer Management

MySQL Tutorial Introduction to Database. Introduction of MySQL MySQL is an SQL (Structured Query Language) based relational database management system.

Social Media Management Tool Buffer Tutorial

Buffer Rope … · Title: Buffer Rope and Buffer Management in a Make To Stock Environment.pdf Author: manu.raj Created Date

Data Management: Databases - MySQL Introduction for ...€¦ · 1.3. MySQL MySQL is a Relational Database Management System (“RDBMS”). It is used by most modern websites and web-based

WATERWAY BUFFER MANAGEMENT PLAN

PHP & MySQL Project on Employee Management System

Mastering phpMyAdmin 3.1 for Effective MySQL Management

vRealize Operations (vROps) Management Pack for MySQL

Buffer Management for Lossless Service in Shared Buffer ...

Online Library Management System PHP MySql

modern servers PostgreSQL and Hugepages:Working with an ... · MySQL & PostgreSQL - database cache MySQL: InnoDB's Buffer Pool innodb_buffer_pool_size The buffer pool is an area in

Lecture 3 - The Stack. Buffer Management

SQL Management Studio for MySQL - User's Manualdownload2.sqlmanager.net/download/mystudio/doc/mystudio.pdf · 9 SQL Management Studio for MySQL ... sales via the SQL Management Studio