MySQL Cluster - voxxed days belgrade 2015

7/23/2019 MySQL Cluster - voxxed days belgrade 2015

1/32

MySQL Cluster

tips & tricks

Bogdan KecmanMySQL Principal Technical [email protected]


2/32

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purnot be incorporated into any contract. It is not a commitment to deliver any material, code, or functnot be relied upon in making purchasing decisions. The development, release, and timing of any feafunctionality described for Oracles products remains at the sole discretion of Oracle.


3/32

Cloud

Web & Enterprise OEM & ISVs

Industry Leaders Rely on MySQL
http://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=Nhttp://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=N


4/32

MySQL Powers The Web

Over 50 million Tweets/day. 143,200 Tweets/sec in Aug 2013

Many petabytes of data. 11.2 Million Row changes & 2.5 birows read /sec handled in MySQL

6 billion hours of video watched each month

Globally-distributed database with 100 terabytes of user-reladata based on MySQL Cluster


5/32

Hosting IaaS, PaaS

SaaS

The #1 Database in the Cloud


6/32

MySQL Cluster Data Nodes InnoDB

Architectural overview

SE s1 SE s2MyISAM


7/32

Whos Using MySQL Cluster?


8/32

MySQL Cluster Architecture


9/32

MySQL Cluster NODES (1/2)

Data nodes (ndbd or ndbmtd) Stores data and indexes

In memory Non-indexed data can be stored on disk

Stores schema definition Check pointed to disk Transaction coordination Online backup All connect to each other

Management nodes (ndb_mgmd) Distributing configuration Logging Monitoring Act as Arbitrator to prevent split-brain scenarios Not crucial for Cluster operation (if not working cluster still works properly,

only limit is that no new nodes can start if no mgm nodes are around)

One is enough, two are perfect, three are too much


10/32

MySQL Cluster NODES (2/2)

SQL nodes (mysqld) The NDBCLUSTER Storage Engine is actually API node Transparent for most applications Used to create tables Used for geo-replication Can act as arbitrator Connects to all Data Nodes

API nodes (your own executable) Application written using NDB API

C C++ Java

FAST No SQL parsing No Optimizer

Examples

Ndbcluster storage engine, ndb_restore, memcached-ndbcluster plugin, l


11/32

High Availability

Fragmentation Table data is split among data nodes

Synchronous replication Each fragment is stored NoOfReplicas times

Heartbeating

Automatic failover

Online backups

Online Updates


12/32

High Performance

Performance boost In-memory Shared IO load As many SQL/API nodes as you like (up to 200) Direct access trough NDB API

Performance killers

Network (latency creates big problem) With 1gbit nic up to 10 data nodes works without problem With modern 10gbit nic up to 40 data nodes can run ok

Joins Huge improvements in 7.4 but still joins on distributed data a

going to suffer performance, especially on slower network Blobs


13/32

Scaling

Up to 254 nodes

Add data nodes on-line Note that adding data nodes does not only increase storage capa

but also increase your IO capacity too.

Geographical Replication

Multi channel replication (note it is always idempotent)


14/32

Accessing NDBCLUSTER data

SQL (via MySQL connector/php,java,ruby,python..., odbc, MySQL C-API.

MEMCACHED (add ndbcluster driver to memcached server)

ndbAPI (C/C++ API)

ClusterJ, JPA, ClusterJPA, LDAP

ClusterJMySQL

JDBC

Apps

JPA

JNIPython Ruby

ClusterJPA

Apps Apps Apps Apps Apps

Node.js

JSON

Apps

mod-ndb

Apache

Apps

ndb-eng

Memcached

Apps Apps

NDB API (C++)

MySQL Cluster Data Nodes

Apps

PHP PERL

Apps


15/32

Domain Object Model Persistence API

Java API

High performance, low latency

Feature rich

JPA interface built upon this new Java l

Java Persistence API compliant

Implemented as an OpenJPA plu

Uses ClusterJ where possible, revertoperations

Higher performance than JDBC

More natural for most Java designer

Easier Cluster adoption for web app

ClusterJ/JPA


16/32

Memcached is a distributed memory b

key/value store with no persistence to NoSQL, simple API, popular with develo

MySQL Cluster already provides scalab

performance with NoSQL (hashed) acce

persistence

Provide the Memcached API but map to N

Writes-in-place, so no need to invalidat Simplifies architecture as caching & dat

into 1 tier

Access data from existing relational tab

Memcached


17/32

Traditional Memcached Architecture

Memcache

httpd memcached

memcached

memcachedmemcache key

PHP/Perl

friends:12389


18/32

NDB & Memcache Architecture: Memcache protocol + NDB stora

Dmemcached

Memcache

Client

Application

D

NDB Engine


19/32

Memcached/MySQL Cluster latency

memcachetest -


20/32

Cluster & Memcached Configured Schema

prefix key value

key value

Prefix Table Key-col Val-col policy

town: map.zip town code cluster

Config tables

town ... code

maidenhead ... SL6

map.zip

Application view

SQL view


21/32

Node.js NoSQL API

Native JavaScript access to MySQL C

End-to-End JavaScript: browser to the a

Storing and retrieving JavaScript object

MySQL ClusterEliminate SQL transformation

Implemented as a module for node.j

Integrates full Cluster API library within

Couple high performance, distribute

performance distributed database

V8 JavaScript Engine

MySQL Cluster Node.js Module

MySQL Cluster Data Nodes

Clients


22/32

MySQL Cluster NoSQL API for Node.js

// Constructor

function Tweet(user, message) {this.id= UUID.generate();this.timestamp = Date.now();this.user = user;this.message = message;

}

// Server Startup

var nosql = require('mysql-js');

var sessionFactory =nosql.connectSync('ndb');

nosql.mapClass(Tweet, 'tweets');

Application Code

// Create a tweet

function newTweet(err, dbSession, httpvar tweet = new TweethttpReq.user,httpReq.message);

dbSession.persist(twe

}

function onNewTweetReqerr, httpReq){

essionFactory.openSenull, newTweet, ht

}


23/32

Need for Speed?

C++ programming interface

Provides direct access to data nodesNo MySQL server needed (still it is recommended to MySQL to manipulate schema)

No SQL layer (no parser, no optimizer, no )

Query batchingAsync transactions

NDB Events

NDB API


24/32

Cant all be that good?

Portability (all requests are hardcoded into your appl

Less flexibilityschema changes need to be hardcoded into your app

to change a simple query you have to change and recompile y

No privileges (everyone have access to everything)

No security (if you can telnet to a data node port youaccess to everything)

No triggers, views, stored procedures

No auditting


25/32

NDB API

1. Start transaction

2. Define operations3. Execute operations

4. Commit / Abort transaction

Life of a transaction


26/32

NDB API

A transaction is started by getting an NdbTransaction

An Ndb object can have maximum of 1024 parallel tr

NdbTransaction * t = ndb->startTransactiif (t == NULL){

printerr(could not start transaction\return (-1);}

Starting a Transaction


27/32

NDB API

An NdbOperation object is created with the getNdbO

A table name or a NdbDictionary::Table* needs to be

NdbOperation * op = t->getNdbOperation(

if (op == NULL){//handle error

}

Getting an NdbOperation Object


28/32

NDB API

insertTuple()

readTuple()writeTuple()

updateTuple()

deleteTuple()

op->readTuple();

Defining the operation type

NDB API EXAMPLE ( k )


29/32

NDB API EXAMPLE (pk access)NdbTransaction * trans = ndb->startTransaction();if (trans == NULL){ printerr(could not start transaction\n); retuNdbOperation * op = trans->getNdbOperation(City);if (op == NULL) return -1;op->readTuple();

int idvalue = 3236;op->equal(ID, idvalue);

int population = 0;op->getValue(Population, (char*)&population);

char name[35];op->getValue(Name, name);

if (trans->execute(NdbTransaction::Commit, NdbOperation::AbortOnErr== -1){

printerr(transaction was not successful\n);return (-1);

}trans->close();printf(The City %s has the population of %d\n, name, population)

NDB API


30/32

NDB API

Joining tables with NDB API is way more complex the

The basic principle is easy nested FOR loopsThe method for retrieving the rows depends on the tinvolved and possible indexes

It is very difficult to do dynamic optimization

In principle the join method has to be decided when the program (coding time)

Joining tables

SQL NDB API d i ( d / d t l ith


31/32

SQL vs NDB API speed comparison (reads/second note logarithm


32/32

Thank You!

Questions?

Bogdan KecmanMySQL Principal Technical [email protected]

MySQL Cluster - voxxed days belgrade 2015

Documents

Transcript of MySQL Cluster - voxxed days belgrade 2015