NoSQL & SQL- The best of both worlds

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

NoSQL and SQL: The Best of Both Worlds

Andrew [email protected]

2nd October 2014


Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


MySQL Customers

World’s Most Popular Open Source Database

WebSaaS, Hosting

OEM / ISV’s

Telecom

Enterprise

http://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=N



















SQL NoSQL


Session Agenda• NoSQL – What are people looking for?• RDBMS – What advantages do they still have?• How MySQL Delivers the Best of Both Worlds

– MySQL Cluster• NoSQL attributes: Scale-out, performance, ease-of-use, schema flexibility, on-line operations• NoSQL APIs

– Key-Value store access to InnoDB (Memcached)


Types of NoSQL stores

Key-Value

• Cassandra• Memcached• BigTable• Hadoop• Voldermort

Document

• MongoDB• CouchDB

Graph

• Neo4J• FlockDB


Key-Value Store


Document Store


Graph Database


• Massive scalability– No application-level sharding

• Performance• High Availability/Fault Tolerance• Ease of use

– Simple operations/administration– Simple APIs– Quickly evolve application & schema

Scalability

Performance

HA

Ease of use

What NoSQL must deliver


NoSQL

Simple access patterns

Compromise on consistency for performance

Ad-hoc data format

Simple operation

SQL

Complex queries with joins

ACID transactions

Well defined schemas

Rich set of tools

• No best single solution fits all• Mix and match

Still a role for SQL (RDBMS)?

Scalability

Performance

HA

Ease of use

SQL/Joins

ACID Transactions


The Temptation to Jettison Relational Model

• Relational Model:– Data stored in multiple tables– Many lookups to retrieve a user’s

timeline– Is it worth the effort in setting up

this complex data model?

• Simpler just to store as one document?

The allure of document stores

Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy

user

friend

post

comment

like

liker

commenter

many

many

many

one

one

many

https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy



• Document Model:– Entire timeline in a single

document (row)– Single lookup to retrieve the

user’s timeline– Brilliantly efficient model when

the document truly contains self-contained information

• Like a real-world document!

The allure of document stores{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}



• These are all people who have their own data that users will want to view or click through:– Name– url– Timeline

• Easy to represent with FKs in a relational model

But when the data isn’t self contained…

user

friend

post

comment

like

liker

commenter

many

many

many

one

one

many



• Do I store all data for all friends and likers again at every point they appear in the document?

• Massive amount of repeated data– Wasted space– Needs to be kept in sync (and you don’t

have transactions to provide consistency)

The allure of document stores{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}



• The reality is that the developer will store the user-ids instead

• Developer is responsible for implementing ‘joins’ in the application– e.g. scanning all timelines for where a

user made a comment or liked a post

• No transactions to ensure consistency between documents

The allure of document stores{name: 83746251, url: ‘…’ stream:[ {friend:{ name: 9384726153, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: 83746251}, {…} ], likes: [83746251, 750730283, 2938493820] }, {…},{…},… ] }, {…},{…},…}


MySQL Cluster Overview• Auto-Sharding, Multi-Master• ACID Compliant, OLTP + Real-Time Analytics

HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure• Self Healing + On-Line Operations99.999% AVAILABILITY

• In-Memory Optimization + Disk-Data• Predictable Low-Latency, Bounded Access TimeREAL-TIME

• Key/Value + Complex, Relational Queries• SQL + Memcached + JavaScript + Java + HTTP/REST & C++SQL + NoSQL

• Open Source + Commercial Editions• Commodity hardware + Management, Monitoring ToolsLOW TCO


Who’s Using MySQL Cluster?


MySQL Cluster Architecture

MySQL Cluster Data Nodes

Clients

Application Layer

Data Layer


MySQL Cluster Scaling


Clients

Application Layer

Data Layer


MySQL Cluster HA


Clients

Application Layer

Data Layer


On-line Operations• Scale the cluster (add & remove nodes on-line)• Repartition tables• Upgrade / patch servers & OS• Upgrade / patch MySQL Cluster• Back-Up• Evolve the schema on-line, in real-time


1.2 Billion UPDATEs per Minute

• NoSQL C++ API, flexaSynch benchmark

• 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB

• ACID Transactions, with Synchronous Replication

2 4 6 8 10 12 14 16 18 20 22 24 26 28 300

5

10

15

20

25


Mil

lio

ns

of

UP

DA

TE

s p

er

Se

co

nd


Join Performance

• Web-Based Content Management System– JOINs 11-tables, 33.5k rows– Returns 2k rows, 19 columns per row

MySQL Cluster 7.1 MySQL Cluster 7.20

10

20

30

40

50

60

70

80

90

100

Query Execution Time Seconds

87.23 seconds

1.26 seconds

70x Faster

Scalability a

Performance a

HA a

Ease of use

SQL/Joins a

ACID Transactions a


Better performance and operational simplicityMySQL Cluster 7.4.1 DMR

Performance gain over 7.347% (Read-Only)38% (Read-Write)

• Faster node restarts


Creating & running your first Cluster

Download & Extract• edelivery.oracle.com• www.mysql.com• dev.mysql.com

Configure• Cluster-wide

“config.ini”• Per-mysqld “my.cnf”

Start processes• Management Nodes• Data Nodes• MySQL Servers


Scalability a

Performance a

HA a

Ease of use a

SQL/Joins a

ACID Transactions a


MySQL Cluster Auto-Installer

• Fast configuration• Auto-discovery• Workload optimized• Repeatable best practices

Specify Workload

Auto-Discover

Define TopologyDeploy


MySQL Cluster Manager1. Download MCM/Cluster package from edelivery.oracle.com:2. Unzip3. Run agent, define, create & start Cluster!$> bin\mcmd --bootstrapMySQL Cluster Manager 1.1.2 started

Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862

Configuring default cluster 'mycluster'...

Starting default cluster 'mycluster'...

Cluster 'mycluster' started successfully

ndb_mgmd NOVA:1186

ndbd NOVA

ndbd NOVA

mysqld NOVA:3306

mysqld NOVA:3307

ndbapi *

Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306 -u root


MCM: Upgrade Cluster

mysql> upgrade cluster--package=7.3 mycluster;


Scalability a

Performance a

HA a

Ease of use a

SQL/Joins a

ACID Transactions a


NoSQL Access to MySQL Cluster data

Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps AppsJPA

Cluster JPAPHP Perl Python Ruby JDBC Cluster J JS Apache Memcached

MySQL JNI Node.JS mod_ndb ndb_engNDB API (C++)



Schema-Free apps

• Rapid application evolution– New types of data constantly

added– No time to get schema extended– Missing skills to extend schema– Initially roll out to just a few users– Constantly adding to live system


Cluster & Memcached – Schema-Free

<town:maidenhead,SL6>

key value


key value

Key Value

town:maidenhead SL6

generic table

Application view

SQL view


Cluster & Memcached - Configured Schema


prefix key value


key value

Prefix Table Key-col Val-col policy

town: map.zip town code cluster

Config tables

town ... code ...

maidenhead ... SL6 ...

map.zip

Application view

SQL view


MySQL 5.6 Memcached with InnoDB

8 32 128 5120

10000

20000

30000

40000

50000

60000

70000

80000

Memcached API

SQL

Client Connections

TP

S

Clients and Applications

MySQL ServerMemcached Plug-in

innodb_memcached

local cache(optional)

Handler API InnoDB API

InnoDB Storage Engine

mysqld process

SQL Memcached Protocol

Up to 9x Higher “SET / INSERT” Throughput


Node.js NoSQL API• Native JavaScript access to MySQL Cluster

– End-to-End JavaScript: browser to the app & DB– Storing and retrieving JavaScript objects

directly in MySQL Cluster – Eliminate SQL transformation

• Implemented as a module for node.js– Integrates Cluster API library within the web

app

• Couple high performance, distributed apps, with high performance distributed database

• Optionally routes through MySQL Server– Use with InnoDB!


NoSQL API for Node.js & FKs

FKs enforced on all APIs:{ message: 'Error', sqlstate: '23000', ndb_error: null, cause: {message: 'Foreign key constraint violated: No parent row found [255]', sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } }


Oracle University MySQL Training ServicesPrepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications

“Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013

Premier Support customers eligible to save 20% on learning credits.

Benefits Expert-led training to support your MySQL learning needs Flexibility to train in the classroom or online Hands-on experience to gain real world experience Key skills needed for database administrators and developers

• MySQL for Beginners MySQL for Database Administrators MySQL Performance Tuning MySQL Cluster – NEW - Register Your Interest! MySQL and PHP - Developing Dynamic Web Applications MySQL for Developers MySQL Developer Techniques

MySQL 5.6 Database Administrator MySQL 5.6 Developer

To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql

Top Courses for Administrators and Developers

Top Certifications

RECENTLY RELEASED!!ALL NEW!! MySQL Cluster Training

To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the

MySQL Cluster Course

http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getpage?page_id=3

http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getpage?page_id=609&get_params=dc:D62018GC10,p_preview:N&sc=WWOU14047594MPP002C005


Next StepsLearn More• www.mysql.com/cluster• Authentic MySQL Curriculum: http://oracle.com/education/mysql

Try it Out• dev.mysql.com/downloads/cluster/

Let us know what you think• clusterdb.com• @clusterdb• forums.mysql.com/list.php?25

NoSQL & SQL- The best of both worlds

Software

Transcript of NoSQL & SQL- The best of both worlds