1 / 129 - MySQL · 2017-05-08 · High Availability High availability is a characteristic of a...

1 / 129

2 / 129

MySQL InnoDB Cluster

When MySQL High Availability is Made Easy

3 / 129

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended forinformation purpose only, and may not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality, and should not be relied up inmaking purchasing decisions. The development, release and timing of any features orfunctionality described for Oracle´s product remains at the sole discretion of Oracle.

Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.

4 / 129

about.me/lefred

Who am I ?


5 / 129

Frédéric Descamps@lefredMySQL EvangelistHacking MySQL since 3.23devops believerMySQL Community Manager since May2016 (🎂)living in Belgium


6 / 129

MySQL InnoDB Cluster

Easy MySQL High Availability


7 / 129

InnoDB

cluster

Ease-of-Use

Extreme Scale-Out

Out-of-Box Solution

Built-in HA

High P erformance

Everything Integr ated


8 / 129

Our vision in 4 steps

MySQL Document Store

Relational & Document Models

MySQL HA

Out-Of-Box HA

Read Scale-Out

Async Replication + A uto F ailover

Write Scale-Out

Sharding

E1

E3

E2 E4


9 / 129

Step 2´s Architecture

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

y

S

Q

L

S

h

e

l

l

I

n

n

o

D

B

c

l

u

s

t

e

r

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

p

M

M


10 / 129

Step 3´s ArchitectureA

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

y

S

Q

L

S

h

e

l

l

I

n

n

o

D

B

c

l

u

s

t

e

r

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

p

M

M

S1 S2 S3 S4 S...


11 / 129

Step 4´s ArchitectureA

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

y

S

Q

L

S

h

e

l

l

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

I

n

n

o

D

B

c

l

u

s

t

e

r

M

p

M

M

S2 S3 S4 S...S1

I

n

n

o

D

B

c

l

u

s

t

e

r

M

p

M

M

S1 S2 S3 S4 S...

I

n

n

o

D

B

c

l

u

s

t

e

r

M

p

M

M

S1 S2 S3 S4 S...

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

replicaset 1

replicaset 2

replicaset 3


12 / 129

Group Replication: heart of MySQL InnoDBCluster


13 / 129

Group Replication: heart of MySQL InnoDBCluster


14 / 129

But before going further...


15 / 129

What is

High Availability ?


16 / 129

High AvailabilityHigh availability is a characteristic of a system, which aims to ensure an agreed level ofoperational performance, usually uptime, for a higher than normal period.

There are three principles of systems design in reliability engineering which can helpachieve high availability:

Elimination of single points of failure. This means adding redundancy to the systemso that failure of a component does not mean failure of the entire system.Reliable crossover. In redundant systems, the crossover point itself tends to becomea single point of failure. Reliable systems must provide for reliable crossover.Detection of failures as they occur. If the two principles above are observed, then auser may never see a failure. But the maintenance activity must.

Source: WikipediaCopyright @ 2017 Oracle and/or its affiliates. All rights reserved.

17 / 129

Database Redundancy

How to achieve it ?


18 / 129

Database Redundancy

How to achieve it ?

Multiple solutions exist, some worse than the others ;-)


19 / 129

Database Redundancy

How to achieve it ?


use of share storage


20 / 129

Database Redundancy

How to achieve it ?


use of share storageuse of share blocs by network (drbd)


21 / 129

Database Redundancy

How to achieve it ?


use of share storageuse of share blocs by network (drbd)use of MySQL replication


22 / 129

Database Redundancy

How to achieve it ?


use of share storageuse of share blocs by network (drbd)use of MySQL replication

Of course the last one is the technique most spread, more flexible and more reliable.


23 / 129

MySQL ReplicationThere are multiple types of MySQL replication:


24 / 129


asynchronous replication (async)


25 / 129


asynchronous replication (async)semi-synchronous replication (semi-sync)


26 / 129


asynchronous replication (async)semi-synchronous replication (semi-sync)group replication - since (5.7.17) !!


27 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)


28 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)slaves can be laging (having delay), there is no delivery guarantee (unless for semi-sync)


29 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)slaves can be laging (having delay), there is no delivery guarantee (unless for semi-sync)replicas provisioning is manual


30 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)slaves can be laging (having delay), there is no delivery guarantee (unless for semi-sync)replicas provisioning is manualdata consistency on replicas is assumed and needs a manual verification


31 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)slaves can be laging (having delay), there is no delivery guarantee (unless for semi-sync)replicas provisioning is manualdata consistency on replicas is assumed and needs a manual verificationcomplex topologies can be built


32 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)slaves can be laging (having delay), there is no delivery guarantee (unless for semi-sync)replicas provisioning is manualdata consistency on replicas is assumed and needs a manual verificationcomplex topologies can be builtit´s even possible to create circular replication (not recommended)


33 / 129

Asynchronous Replication(s)replicas (also called slaves) stream the replication logs from a unique master (multi-source replication is possible since 5.7.6)slaves can be laging (having delay), there is no delivery guarantee (unless for semi-sync)replicas provisioning is manualdata consistency on replicas is assumed and needs a manual verificationcomplex topologies can be builtit´s even possible to create circular replication (not recommended)write operations are safe only from a unique global master


34 / 129

MySQL HA Architectures with asynchronousreplicationIt exists a lot of possible architectures and as much tools to enforce them.


35 / 129


They are all based on the same principle:


36 / 129



1 master


37 / 129



1 master1 or many replicas (slaves)


38 / 129



1 master1 or many replicas (slaves)

If the master as an issue, the most accurate slave needs to takeover the role and bepromoted as master for tall the remaining slaves still online.


39 / 129

Automation


40 / 129

AutomationAnd this is the most complicated part...


41 / 129


It exists a large amount of external tools that can be used to achieve this task. Most ofthem are only compatible with GNU/Linux.


42 / 129



Their problem is that they create complexity in the architecture's design. You need to beDBA & sysadmin to manage those solutions:


43 / 129




MySQL-Utilities (mysqlrplcheck, mysqlrpladmin)


44 / 129




MySQL-Utilities (mysqlrplcheck, mysqlrpladmin)Pacemaker with MySQL dedicated OCF


45 / 129




MySQL-Utilities (mysqlrplcheck, mysqlrpladmin)Pacemaker with MySQL dedicated OCFcustom solutions...


46 / 129

MySQL Group Replication

but what is it ?!?


47 / 129


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQL


48 / 129


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theory


49 / 129


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistency


50 / 129


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistencyGR implements conflict detection and resolution


51 / 129


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistencyGR implements conflict detection and resolutionGR allows automatic distributed recovery


52 / 129


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistencyGR implements conflict detection and resolutionGR allows automatic distributed recoverySupported on all MySQL platforms !!

Linux, Windows, Solaris, OSX, FreeBSD


53 / 129

MySQL Group Communication System (GCS)MySQL Xcom protocol


54 / 129

MySQL Group Communication System (GCS)MySQL Xcom protocolReplicated Database State Machine


55 / 129

MySQL Group Communication System (GCS)MySQL Xcom protocolReplicated Database State MachinePaxos based protocol


56 / 129

MySQL Group Communication System (GCS)MySQL Xcom protocolReplicated Database State MachinePaxos based protocolits task: deliver messages across the distributed system:


57 / 129


atomically


58 / 129


atomicallyin Total Order


59 / 129


atomicallyin Total Order

MySQL Group Replication receives the Ordered 'tickets' from this GCS subsystem.


60 / 129

And for users ?


61 / 129

And for users ?no longer necessary to handle server fail-over manually or with a complicated script


62 / 129

And for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault tolerance


63 / 129

And for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault toleranceGR enables update-everywhere setups


64 / 129

And for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault toleranceGR enables update-everywhere setupsGR handles crashes, failures, re-connetcs automatically


65 / 129

And for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault toleranceGR enables update-everywhere setupsGR handles crashes, failures, re-connetcs automaticallyAllows an easy setup of a MySQL service high available !


66 / 129

OK, but how does it work ?


67 / 129


it's just magic !


68 / 129


it's just magic !

... no, in fact the writesets replication is synchronous and then certification and apply ofthe changes are local to each nodes and asynchronous.


69 / 129


it's just magic !

... no, in fact the writesets replication is synchronous and then certification and apply ofthe changes are local to each nodes and asynchronous.

not that easy to understand... right ? As a picture is worth a 1000 words, let's illustratethis...


70 / 129



71 / 129



72 / 129



73 / 129



74 / 129



75 / 129



76 / 129



77 / 129



78 / 129



79 / 129



80 / 129



81 / 129



82 / 129

CertificationCertification is the process that only needs to answer the following unique question:


83 / 129


can the write (transaction) be applied ?


84 / 129


can the write (transaction) be applied ?based on unapplied earlier transactions


85 / 129


can the write (transaction) be applied ?based on unapplied earlier transactionssuch conflicts must come for other members/nodes


86 / 129



happens on every member/node


87 / 129



happens on every member/nodeshould be deterministic on every node


88 / 129



happens on every member/nodeshould be deterministic on every noderesults are not reported to the group


89 / 129




pass: enter in the apply queue


90 / 129




pass: enter in the apply queuefail: drop the transaction


91 / 129





serialized by the total order in GCS/XCOM + GTID


92 / 129





serialized by the total order in GCS/XCOM + GTIDcost is based on trx size (# rows & # keys)


93 / 129

GTIDGTIDs are the same as those used by asynchronous replication.

mysql> SELECT * FROM performance_schema.replication_connection_status\G ************************** 1. row *************************** CHANNEL_NAME: group_replication_applier GROUP_NAME: afb80f36-2bff-11e6-84e0-0800277dd3bf SOURCE_UUID: afb80f36-2bff-11e6-84e0-0800277dd3bf THREAD_ID: NULL SERVICE_STATE: ON COUNT_RECEIVED_HEARTBEATS: 0 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00 RECEIVED_TRANSACTION_SET: afb80f36-2bff-11e6-84e0-0800277dd3bf:1-57, f037578b-46b1-11e6-8005-08002774c31b:1-48937 LAST_ERROR_NUMBER: 0 LAST_ERROR_MESSAGE: LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00


94 / 129

but transactions use the Group´s GTID

mysql> show master status\G ************************** 1. row *************************** File: mysql4-bin.000001 Position: 1501 Binlog_Do_DB: Binlog_Ignore_DB: Executed_Gtid_Set: afb80f36-2bff-11e6-84e0-0800277dd3bf:1-57, f037578b-46b1-11e6-8005-08002774c31b:1-48937


95 / 129

Requirementsexclusively works with InnoDB tables only


96 / 129

Requirementsexclusively works with InnoDB tables onlyevery tables must have a PK defined


97 / 129

Requirementsexclusively works with InnoDB tables onlyevery tables must have a PK definedonly IPV4 is supported


98 / 129

Requirementsexclusively works with InnoDB tables onlyevery tables must have a PK definedonly IPV4 is supporteda good network with low latency is important


99 / 129

Requirementsexclusively works with InnoDB tables onlyevery tables must have a PK definedonly IPV4 is supporteda good network with low latency is importantmaximum of 9 members per group


100 / 129

Requirementsexclusively works with InnoDB tables onlyevery tables must have a PK definedonly IPV4 is supporteda good network with low latency is importantmaximum of 9 members per grouplog-bin must be enabled and only ROW format is supported


101 / 129

Requirements (2)enable GTIDs


102 / 129

Requirements (2)enable GTIDsreplication meta-data must be stored in system tables

--master-info-repository=TABLE --relay-log-info-repository=TABLE


103 / 129



writesets extraction must be enabled

--transaction-write-set-extraction=XXHASH64


104 / 129



writesets extraction must be enabled

--transaction-write-set-extraction=XXHASH64

log-slave-updates must also be enabled


105 / 129

Limitationsbinlog checksum is not supported (but we have checksum in the replication channel)

--binlog-checksum=NONE


106 / 129



savepoints were not supported before 5.7.19 & 8.0.1


107 / 129



savepoints were not supported before 5.7.19 & 8.0.1SERIALIZABLE is not supported as transaction isolation level


108 / 129



savepoints were not supported before 5.7.19 & 8.0.1SERIALIZABLE is not supported as transaction isolation levelhttp://lefred.be/content/mysql-group-replication-limitations-savepoints/http://lefred.be/content/mysql-group-replication-and-table-design/


109 / 129

http://lefred.be/content/mysql-group-replication-limitations-savepoints/

http://lefred.be/content/mysql-group-replication-and-table-design/


110 / 129

MySQL Router (GA!)MySQL Router is lightweight middleware that provides transparent routing between yourapplication and backend MySQL Servers. It can be used for a wide variety of use cases,such as providing high availability and scalability by effectively routing database traffic toappropriate backend MySQL Servers.


111 / 129


MySQL Router doesn´t require any specific configuration. It configures itself automatically(bootstrap) using MySQL InnoDB Cluster´s metadata.


112 / 129


MySQL Router doesn´t require any specific configuration. It configures itself automatically(bootstrap) using MySQL InnoDB Cluster´s metadata.

The MySQL Router development will be focusing on sharding.


113 / 129

ProxySQL has native support for GroupReplication which makes it a good choice foradvanced users.

ProxySQLIf you need some specific features that are not yet available in MySQL Router, liketransparent R/W splitting, then you can use your software of choice.

We are also collaborating with ProxySQL.


114 / 129


115 / 129

MySQL Shell (GA!)The MySQL Shell is an interactive Javascript, Python, or SQL interface supportingdevelopment and administration for the MySQL Server and is a component of the MySQLServer. You can use the MySQL Shell to perform data queries and updates as well asvarious administration operations.


116 / 129

MySQL Shell (2)The MySQL Shell provides:


117 / 129


Both Interactive and Batch operations


118 / 129


Both Interactive and Batch operationsDocument and Relational Models


119 / 129


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scripting


120 / 129


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scriptingTraditional Table, JSON, Tab Separated output results formats


121 / 129


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scriptingTraditional Table, JSON, Tab Separated output results formatsMySQL Standard and X Protocols


122 / 129


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scriptingTraditional Table, JSON, Tab Separated output results formatsMySQL Standard and X Protocolsand more...


123 / 129

MySQL Shell (3)PREVIEW: the new Shell´s prompt is nice and clear !


124 / 129

Deploying a MySQL using MySQL Shell´sadminAPI:mysql-js> var i1 = 'root@instance01:3306';mysql-js> var i2 = 'root@instance02:3306';mysql-js> var i3 = 'root@instance02:3306';

mysql-js> dba.checkInstanceCon�guration(i1);mysql-js> dba.checkInstanceCon�guration(i2);mysql-js> dba.checkInstanceCon�guration(i3);

mysql-js> shell.connect(i1);mysql-js> var cluster = dba.createCluster('InnovationDay');

mysql-js> cluster.checkInstanceState(i2); mysql-js> cluster.addInstance(i2);

mysql-js> cluster.checkInstanceState(i3);mysql-js> cluster.addInstance(i3);


125 / 129

instance01.yaml :

classes: - innodbcluster

innodbcluster::mysql_serverid: 1

Thanks to the MySQL Shell we can automate everything ;-)

https://github.com/lefred/puppet-lefred-innodbcluster

Super easy using hiera:

common.yaml:

innodbcluster::mysql_root_password: StRongP4ssw0rD!innodbcluster::mysql_bind_interface: eth1innodbcluster::cluster_name: InnovationDayinnodbcluster::grant::user: fredinnodbcluster::grant::password: fredinnodbcluster::seed: instance01


126 / 129

https://github.com/lefred/puppet-lefred-innodbcluster

example (2)With hiera-eyaml you can even encrypt your password:

innodbcluster::mysql_root_password: > ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEAhqKUTXZ/4L8/aL3XARMfDBEI+s5HPshPg9BI ... FLfovstrb8zmcbk5yb/KD0lDM8Elas0lrnpk8MxwNfKw+hB299JFp8ldAtUk ODIieTA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBJEeoyzHtW/WGpbiUz gcXTgBAnb1gGrBZAfAiv/ztwuZ9z]innodbcluster::mysql_bind_interface: eth1innodbcluster::cluster_name: InnovationDayinnodbcluster::grant::user: rootinnodbcluster::grant::password: > ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEAhqKUTXZ/4L8/aL3XARMfDBEI+s5HPshPg9BI ... FLfovstrb8zmcbk5yb/KD0lDM8Elas0lrnpk8MxwNfKw+hB299JFp8ldAtUk ODIieTA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBJEeoyzHtW/WGpbiUz gcXTgBAnb1gGrBZAfAiv/ztwuZ9z]


127 / 129

Resourceshttp://lefred.be/content/mysql-innodb-cluster-mysql-shell-starter-guide/

http://lefred.be/content/mysql-innodb-cluster-automated-installation-with-puppet/

http://lefred.be/content/category/mysql/group-replication/

https://www.slideshare.net/lefred.descamps/mysql-devops-webinar

http://mysqlserverteam.com/innodb-cluster-in-opc/


128 / 129

http://lefred.be/content/mysql-innodb-cluster-mysql-shell-starter-guide/

http://lefred.be/content/mysql-innodb-cluster-automated-installation-with-puppet/

http://lefred.be/content/category/mysql/group-replication/

https://www.slideshare.net/lefred.descamps/mysql-devops-webinar

http://mysqlserverteam.com/innodb-cluster-in-opc/

Thank you !

Any Questions ?


129 / 129

1 / 129 - MySQL · 2017-05-08 · High Availability High availability is a characteristic of a...

Documents

Transcript of 1 / 129 - MySQL · 2017-05-08 · High Availability High availability is a characteristic of a...