3 / 147 · MySQL 8.0 InnoDB Cluster is a very easy High Availability solution for MySQL Frédéric...

3 / 147

MySQL HA so easy ? That's insane !

MySQL 8.0 InnoDB Cluster is a very easy High Availability solution forMySQL

Frédéric Descamps - MySQL Community Manager - Oracle

4 / 147

Save the date !

Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.

5 / 147

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended forinformation purpose only, and may not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality, and should not be relied up inmaking purchasing decisions. The development, release and timing of any features orfunctionality described for Oracle´s product remains at the sole discretion of Oracle.


6 / 147

about me - http://about.me/lefred

Who am I ?


7 / 147

http://about.me/lefred

Frédéric Descamps@lefredMySQL EvangelistHacking MySQL since 3.23devops believerliving in Belgium 🇧🇪http://lefred.be


8 / 147

http://lefred.be/

MySQL InnoDB Cluster: vision "A single product - MySQL - with high availability and scaling features baked in;providing an integrated end-to-end solution that is easy to use."


9 / 147

MySQL InnoDB Cluster: vision "A single product - MySQL - with high availability and scaling features baked in;providing an integrated end-to-end solution that is easy to use."

InnoDB

cluster

Ease-of-Use

Extreme Scale-Out

Out-of-Box Solution

Built-in HA

High Performance

Everything Integrated


10 / 147

One Product: MySQL

All components created togetherTested togetherPackaged together

Easy to Use

One client: MySQL ShellEasy packagingIntegrated orchestrationHomogenous servers

Flexible and Modern

SQL and NoSQL togetherProtocol BuffersAsynchronous APIDeveloper friendly

Support Read/Write Scale Out

Shared clustersFederated system of N replica sets

supporting cross shardoperations

Each replica set manages a shard

MySQL InnoDB Cluster: goals


11 / 147

MySQL InnoDB Cluster Architecture

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

y

S

Q

L

S

h

e

l

l

I

n

n

o

D

B

c

l

u

s

t

e

r

A

p

p

l

i

c

a

t

i

o

n

M

y

S

Q

L

C

o

n

n

e

c

t

o

r

M

y

S

Q

L

R

o

u

t

e

r

M

p

M

M


12 / 147

Let's start with the demo, shall we ?


13 / 147


14 / 147


15 / 147


16 / 147

Group Replication: heart of MySQL InnoDBCluster


17 / 147

Group Replication: heart of MySQL InnoDBCluster


18 / 147

MySQL Group Replication

but what is it ?!?


19 / 147


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQL


20 / 147


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theory


21 / 147


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistency


22 / 147


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistencyGR implements conflict detection and resolution


23 / 147


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistencyGR implements conflict detection and resolutionGR allows automatic distributed recovery


24 / 147


but what is it ?!?

GR is a plugin for MySQL, made by MySQL and packaged with MySQLGR is an implementation of Replicated Database State Machine theoryGR allows to write on all Group Members (cluster nodes) simultaneously whileretaining consistencyGR implements conflict detection and resolutionGR allows automatic distributed recoverySupported on all MySQL platforms !!

Linux, Windows, Solaris, OSX, FreeBSD


25 / 147

And what does it means for users ?


26 / 147

And what does it means for users ?no longer necessary to handle server fail-over manually or with a complicated script


27 / 147

And what does it means for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault tolerance


28 / 147

And what does it means for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault toleranceGR enables update-everywhere setups


29 / 147

And what does it means for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault toleranceGR enables update-everywhere setupsGR handles crashes, failures, re-connetcs automatically


30 / 147

And what does it means for users ?no longer necessary to handle server fail-over manually or with a complicated scriptGR provides fault toleranceGR enables update-everywhere setupsGR handles crashes, failures, re-connetcs automaticallyAllows an easy setup of a MySQL service high available !


31 / 147

Default = Single Primary ModeBy default, MySQL InnoDB Cluster runs in Single Primary Mode.


32 / 147


mysql> show global variables like 'group_replication_single_primary_mode';+---------------------------------------+-------+| Variable_name | Value |+---------------------------------------+-------+| group_replication_single_primary_mode | ON |+---------------------------------------+-------+


33 / 147


mysql> show global variables like 'group_replication_single_primary_mode';+---------------------------------------+-------+| Variable_name | Value |+---------------------------------------+-------+| group_replication_single_primary_mode | ON |+---------------------------------------+-------+

In Single Primary Mode, a single member acts as the writable master (PRIMARY) andthe rest of the members act as hot-standbys (SECONDARY).

The group itself coordinates and configures itself automatically to determine whichmember will act as the PRIMARY, through a leader election mechanism.


34 / 147


35 / 147

MySQL RouterMySQL Router is lightweight middleware that provides transparent routing between yourapplication and backend MySQL Servers. It can be used for a wide variety of use cases,such as providing high availability and scalability by effectively routing database trafficto appropriate backend MySQL Servers.


36 / 147

MySQL RouterMySQL Router is lightweight middleware that provides transparent routing between yourapplication and backend MySQL Servers. It can be used for a wide variety of use cases,such as providing high availability and scalability by effectively routing database trafficto appropriate backend MySQL Servers.

MySQL Router doesn´t require any specific configuration. It configures itselfautomatically (bootstrap) using MySQL InnoDB Cluster´s metadata.


37 / 147

ProxySQL has native support for GroupReplication which makes it a good choicefor advanced users.

ProxySQLIf you need some specific features that are not yet available in MySQL Router, liketransparent R/W splitting, then you can use your software of choice.

We are also collaborating with ProxySQL.


38 / 147

MySQL ShellThe MySQL Shell is an interactive Javascript, Python, or SQL interface supportingdevelopment and administration for the MySQL Server and is a component of the MySQLServer. You can use the MySQL Shell to perform data queries and updates as well asvarious administration operations.


39 / 147

MySQL Shell (2)The MySQL Shell provides:


40 / 147


Both Interactive and Batch operations


41 / 147


Both Interactive and Batch operationsDocument and Relational Models


42 / 147


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scripting


43 / 147


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scriptingTraditional Table, JSON, Tab Separated output results formats


44 / 147


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scriptingTraditional Table, JSON, Tab Separated output results formatsMySQL Standard and X Protocols


45 / 147


Both Interactive and Batch operationsDocument and Relational ModelsCRUD Document and Relational APIs via scriptingTraditional Table, JSON, Tab Separated output results formatsMySQL Standard and X Protocolsand more...


46 / 147

New since MySQL 8.0.11The Shell improved a lot and uses all the enhancements part of MySQL 8:


47 / 147


set persists


48 / 147


set persists

restart

This makes the remote administration very easy and cloud friendly.


49 / 147

don´t forget, this is a HA solution !


50 / 147

Thank you !

Any Questions ?

share your 💕 for MySQL on social media using @MySQL #DataOpsBarcelona

more at http://lefred.be/content/category/mysql/group-replication/http://mysqlhighavailability.com/


51 / 147

http://lefred.be/content/category/mysql/group-replication/

http://mysqlhighavailability.com/

want to know more?


52 / 147

GR is nice, but how does it work ?


53 / 147


it´s just ...


54 / 147


it´s just ...


55 / 147


it´s just ...

... no, in fact the writesets replication is synchronous and then certification and apply ofthe changes are local to each nodes and asynchronous.


56 / 147


it´s just ...

... no, in fact the writesets replication is synchronous and then certification and apply ofthe changes are local to each nodes and asynchronous.

not that easy to understand... right ? As a picture is worth a 1000 words, let´s illustratethis...


57 / 147



58 / 147



59 / 147



60 / 147



61 / 147



62 / 147



63 / 147



64 / 147



65 / 147



66 / 147



67 / 147



68 / 147



69 / 147

MySQL Group Replication (full transaction)


70 / 147



71 / 147



72 / 147



73 / 147



74 / 147



75 / 147



76 / 147



77 / 147



78 / 147

MySQL Group Communication System (GCS)MySQL Xcom protocol


79 / 147

MySQL Group Communication System (GCS)MySQL Xcom protocolReplicated Database State Machine


80 / 147

MySQL Group Communication System (GCS)MySQL Xcom protocolReplicated Database State MachinePaxos based protocol


81 / 147

MySQL Group Communication System (GCS)MySQL Xcom protocolReplicated Database State MachinePaxos based protocolits task: deliver messages across the distributed system:


82 / 147


atomically


83 / 147


atomicallyin Total Order


84 / 147


atomicallyin Total Order

MySQL Group Replication receives the Ordered 'tickets' from this GCS subsystem.


85 / 147

Group Replication : Total Order Delivery - GTID


86 / 147



87 / 147



88 / 147



89 / 147



90 / 147



91 / 147



92 / 147



93 / 147



94 / 147



95 / 147



96 / 147



97 / 147



98 / 147



99 / 147



100 / 147



101 / 147



102 / 147



103 / 147



104 / 147

Group Replication: return commitAsynchronous Replication:


105 / 147

Group Replication: return from commit (2)Semi-Sync Replication:


106 / 147

Group Replication: return from commit (3)Group Replication:


107 / 147

Does this mean we can have a distant node andalways let it ack later ?


108 / 147


NO!


109 / 147


NO!

There will be always a penalty to pay. For example if you have 3 members, every 3rd writethe system will be affected by the lartency... why ?


110 / 147


NO!

There will be always a penalty to pay. For example if you have 3 members, every 3rd writethe system will be affected by the lartency... why ?

That´s because the system has to wait for the noop (single skip message) from the“distant” node.

More info: http://lefred.be/content/mysql-group-replication-about-ack-from-majority/


111 / 147

http://lefred.be/content/mysql-group-replication-about-ack-from-majority/

Group Replication : Optimistic LockingGroup Replication uses optimistic locking


112 / 147


during a transaction, local (InnoDB) locking happens


113 / 147


during a transaction, local (InnoDB) locking happensoptimistically assumes there will be no conflicts across nodes (no communication between nodes necessary)


114 / 147


during a transaction, local (InnoDB) locking happensoptimistically assumes there will be no conflicts across nodes (no communication between nodes necessary)cluster-wide conflict resolution happens only at COMMIT, during certification


115 / 147


during a transaction, local (InnoDB) locking happensoptimistically assumes there will be no conflicts across nodes (no communication between nodes necessary)cluster-wide conflict resolution happens only at COMMIT, during certification

Let´s first have a look at the traditional locking to compare.


116 / 147

Traditional locking


117 / 147

Traditional locking


118 / 147

Traditional locking


119 / 147

Traditional locking


120 / 147

Traditional locking


121 / 147

Traditional locking


122 / 147

Optimistic Locking


123 / 147

Optimistic Locking


124 / 147

Optimistic Locking


125 / 147

Optimistic Locking


126 / 147

Optimistic Locking


127 / 147

Optimistic Locking


128 / 147

Optimistic Locking

The system returns error 149 as certification failed:

ERROR 1180 (HY000): Got error 149 during COMMIT


129 / 147

Such conflicts happen only when using multi-primary group !


130 / 147

CertificationCertification is the process that only needs to answer the following unique question:


131 / 147


can the write (transaction) be applied ?


132 / 147


can the write (transaction) be applied ?based on unapplied earlier transactions


133 / 147


can the write (transaction) be applied ?based on unapplied earlier transactionssuch conflicts must come for other members/nodes


134 / 147



happens on every member/node


135 / 147



happens on every member/nodeshould be deterministic on every node


136 / 147



happens on every member/nodeshould be deterministic on every noderesults are not reported to the group


137 / 147




pass: enter in the apply queue


138 / 147




pass: enter in the apply queuefail: drop the transaction


139 / 147





serialized by the total order in GCS/XCOM + GTID


140 / 147





serialized by the total order in GCS/XCOM + GTIDcost is based on trx size (# rows & # keys)


141 / 147





serialized by the total order in GCS/XCOM + GTIDcost is based on trx size (# rows & # keys)first committer (and ack from majority) wins rule


142 / 147

Drawbacks of optimistic lockinghaving a first-committer-wins system means conflicts will more likely happen whenwriting on multiple members with:


143 / 147


large transactions


144 / 147


large transactions

long running transactions


145 / 147


large transactions

long running transactions

hotspot records


146 / 147

Thank you !

Any Questions ?

share your 💕 for MySQL on social media using @MySQL #DataOpsBarcelona

more at http://lefred.be/content/category/mysql/group-replication/http://mysqlhighavailability.com/


147 / 147

http://lefred.be/content/category/mysql/group-replication/

http://mysqlhighavailability.com/

3 / 147 · MySQL 8.0 InnoDB Cluster is a very easy High Availability solution for MySQL Frédéric...

Documents

Transcript of 3 / 147 · MySQL 8.0 InnoDB Cluster is a very easy High Availability solution for MySQL Frédéric...