MySQL Cluster NoSQL Memcached API

MySQL Cluster: NoSQL Memcached API

•  Meet New Requirements: Add persistence, sharding, fault tolerance

•  Ease-of-Use: Standard Memcached libraries

•  Simplification: Eliminate cache-coherency & consolidate data tiers

•  Flexible Data Model: Schema / Schema-less storage

•  Flexible Access Patterns: SQL and NoSQL

•  Flexible Deployments: values stored in Cluster, Memcached or both

•  Key / Value Performance: Native access to MySQL Cluster

Copyright © 2012 Oracle Corp.

<Insert Picture Here>

Program Agenda

•  Overview, MySQL Cluster

•  Memcached API

•  Memcache Overview •  MySQL Cluster (NDB) & Memcache setup •  Sample Application

•  Resources to Get Started


MySQL Cluster

• Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics

HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations

99.999% AVAILABILITY

• Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Toos LOW TCO

• Key/Value + Complex, Relational Queries • SQL + Memcached + Java + JPA + HTTP/REST + C++ SQL + NoSQL

• High Load, Real Time Performance • Predictable Low-Latency, Bounded Access Time REAL-TIME


MySQL Cluster 7.2 70x Faster JOINs

Fastest Ramp…Ever

8x Higher Per Node Performance

writes

NoSQL Memcached API

Geo-Distributed Clusters

Memcached API



Memcached Overview

Memcache Client

Application memcached

memcached

memcached memcache key friends:12389!

hash key to find data

hash key to pick server


Memcache Protocol Elements

• Key Up to 250 characters • Expire Time A number in seconds, up to 30 days •  Flags 32-bit number for application use • CAS ID 64-bit incrementing version number on value • Value Value stored for key; traditionally up to 1 MB.


Memcache Protocol Commands (1)

•  get key Fetch value for this key •  set key value Store this data •  add key value Store this data (strict insert) •  replace key value Store this data (strict update) •  cas key value cas-id Update with version number check


Memcache Protocol Commands (2)

•  append key value Append text to current value •  prepend key value Insert text before current value •  incr key amount Add to current stored integer value •  decr key amount Subtract from current stored value •  delete key Delete a key and value •  flush_all Delete all stored values on the server •  stats stat-name Fetch server statistics


Traditional architecture: Memcached & MySQL

MySQL Slave

more storage (on disk)!more powerful protocol (SQL)!

higher throughput !

lower latency!

mysql client

memcached Memcache

Client

Application


NDB & Memcache Architecture: Memcache protocol + NDB storage

MySQL Cluster

Data Node memcached

Memcache Client

Application

MySQL Cluster

Data Node

NDB Engine


Goals • Access stored data directly from memcache client •  Memcached perspective:

•  MySQL Cluster is a write-scalable, replicated data store •  with reliable in-memory storage, •  plus on-disk storage when data is too big for memory.

•  MySQL Cluster perspective: •  memcache is a high performance API •  providing easy access to in-memory data, •  plus an extra layer of caching when data is on disk.


Measured Latency memcachetest -t 2 -M 7000 -c 25000 !


Configuration Overview NDB Memcache


Anatomy of a Memcache key

user:1248

the prefix the database key


A key-prefix mapping

Memcache key

prefix Cluster Container Cache

Policy


key prefix Cluster Con-

tainer Cache Policy

Server Role

ID

A memcache server role


tainer Cache Policy


tainer Cache Policy


tainer Cache Policy


Getting Started MySQL Cluster and Memcached


“Cluster on a laptop”

• Configuration file cluster.ini • Management server ndb_mgmd • Data node ndbd • MySQL server mysqld • Memcached server memcached

(All of these are present in MySQL Cluster 7.2 distribution)

Cluster Configuration File

A single central file for the whole cluster Loaded by the management server on --initial And then stored in cache for future restarts Sections:

[MGM] [DB DEFAULT] [NDBD] [API]


[MGM] section: Management Server

[MGM]NodeId=1datadir=/Users/jdd/sandbox/ndbhostname=localhost


[DB DEFAULT] Section: data node parameters

[DB DEFAULT]NoOfReplicas=1TimeBetweenGlobalCheckpoints=12000HeartbeatIntervalDbApi=15000DataMemory=180MIndexMemory=60M


[NDBD] Sections: individual data node

[NDBD]NodeId=2


[API] Sections for MySQL & Memcached servers

[API][API][API][API][API][API][API]


Complete cluster.ini file [MGM]NodeId=1datadir=/Users/jdd/sandbox/ndbhostname=localhost[DB DEFAULT]NoOfReplicas=1TimeBetweenGlobalCheckpoints=12000HeartbeatIntervalDbApi=15000DataMemory=180MIndexMemory=60M[NDBD]NodeId=2[API][API] ...


Start the Management Server

% ndb_mgmd -f config-file-path


Start the Data Node

% ndbd


Configure & start mysqld (my.cnf file)

[mysqld]ndbcluster


Load NDB Memcache configuration tables

% cd /usr/local/mysql/share/memcache-api% mysql -u root < ndb_memcache_metadata.sql



Sample Application


"Twitter-like" Application

•  Two Tables •  authors •  tweets

•  Three Perl Functions •  Create a new author record •  Insert a new tweet •  Increment an author's tweet count

Required Perl Modules

% cpan install Cache::Memcached% cpan install UUID::Tiny


Sample Application

% mysql -u rootmysql> CREATE DATABASE demo; use demo;


Data Definition: Authors

CREATE TABLE authors ( author varchar(15) not null, date_created timestamp, tweets bigint unsigned not null default 0, followers bigint unsigned not null default 0, following bigint unsigned not null default 0, PRIMARY KEY (author)) ENGINE=ndb;


Data Definition: Tweets

CREATE TABLE tweets ( id varchar(36) primary key, author varchar(15), time_stamp timestamp, tweet varchar(140), index(time_stamp), index(author))ENGINE=ndb;


Configuration: Use the ndbmemcache config schema

mysql> use ndbmemcache; show tables;


INSERT into containers SET name = 'tweets_table' , db_schema = 'demo' , db_table = 'tweets', key_columns = 'id' , value_columns = 'time_stamp,author,tweet';

INSERT into key_prefixesSET key_prefix = 'tweet:', policy = 'ndb-only', container = 'tweets_table';

Configuration: Create container & key_prefixes



Two key_prefixes for authors table

•  newauthor: •  ADD an author record and set date_created •  tweetcount: •  INCR tweet count for author

INSERT into containers SET name = 'author_new' , db_schema = 'demo' , db_table = 'authors',key_columns = 'author' ,

value_columns = 'date_created';

INSERT into key_prefixesSET key_prefix = 'newauthor:', policy = 'ndb-only', container = 'author_new';

Configuration: New Author


INSERT into containers SET name = 'author_count', db_schema = 'demo', db_table = 'authors', key_columns = 'author', increment_column = 'tweets';

INSERT into key_prefixesSET key_prefix = 'tweetcount:', policy = 'ndb-only', container = 'author_count';

Configuration: Tweet Count


% cd /usr/local/mysql% bin/memcached -E lib/ndb_engine.so

Memcached: Start the server

17-Aug-2012 13:53:32 PDT NDB Memcache 5.5.25-ndb-7.2.7 ...Contacting primary management server (localhost:1186) ... Connected to "localhost:1186" as node id 4.Retrieved 6 key prefixes for server role "default_role".Server started with 4 threads.Priming the pump ... done [0.661 sec].


use Cache::Memcached;use UUID::Tiny;our $mc = new Cache::Memcached( { 'servers' => [ "localhost:11211" ]});

Perl: Startup Code


Perl: Add a new author

## Takes one parameter: authorsub new_author { my $author = shift; my $time = time(); my $key = "newauthor:$author"; $mc->add($key, $time);}


## Take two parameters: author and tweetsub insert_tweet { my $author = shift; my $tweet = shift; my $id = create_UUID_as_string(UUID_V4); my $time = time(); # Insert the tweet my $key = "tweet:$id"; my $value = "$time" ."\t". $author ."\t". $tweet; $mc->add($key, $value);}

Perl: Insert a Tweet


## Increment the author's tweet count.## Take one parameter: authorsub increment_tweetcount { my $author = shift; my $key = "tweetcount:$author"; $mc->incr($key, 1);}

Perl: Increment Tweet Count


Download MySQL Cluster + Memcached API

http://www.mysql.com/downloads/cluster/#downloads

Most secure, scalable MySQL Database, Development / Management / Monitoring Tools, backed by Oracle Premier Lifetime Support

Oracle Premier Support

Oracle Product Certifications/Integrations

MySQL Enterprise Security

MySQL Enterprise Scalability

MySQL Enterprise Backup

MySQL Enterprise Monitor/Query Analyzer

MySQL Workbench

MySQL Cluster CGE

MySQL Cluster

Manager

•  Straight from the Source •  Largest Team of MySQL Experts •  Direct Access to MySQL Support Engineers •  Backed by MySQL Developers •  Global 24/7/365, 29 languages •  Forward Compatible Hot Fixes •  MySQL Maintenance Releases •  Unlimited Incidents •  Knowledge Base •  MySQL Consultative Support

Oracle Premier Support for MySQL Rely on The Experts - Get Unique Benefits

"The MySQL support service has been essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com

Only From Oracle

Getting Started

Learn More

Get MySQL Cluster 7.2 Review Documentation

Scaling Web Databases Guide www.mysql.com/cluster/

Download Today http://www.mysql.com/downloads/cluster/

Copyright 2011 Oracle Corporation 50

https://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-ndbmemcache.html

Summary

Scale Web Services with 99.999% Availability

Native Memcached NoSQL Access

Try it out Today!

Copyright 2011 Oracle Corporation 51

MySQL Cluster Architecture

Data Nodes

Node Group 1

F1

F3

F3

F1

Nod

e 1

Nod

e 2

Node Group 2

F2

F4

F4

F2

Nod

e 3

Nod

e 4

Application Nodes

Cluster Mgmt

Cluster Mgmt

REST JPA

MySQL Cluster - Extreme Resilience

Data Nodes

Node Group 1

F1

F3

F3

F1

Nod

e 1

Nod

e 2

Node Group 2

F2

F4

F4

F2

Nod

e 3

Nod

e 4

Application Nodes

Cluster Mgmt

Cluster Mgmt

REST JPA

MySQL Cluster NoSQL Memcached API

Technology

Transcript of MySQL Cluster NoSQL Memcached API