NoSQL & SQL- The best of both worlds

NoSQL and SQL: The Best of Both Worlds

Andrew Morgan@andrewmorganclusterdb.com

2nd October 2014

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

MySQL Customers

World’s Most Popular Open Source Database

WebSaaS, Hosting

OEM / ISV’s

Telecom

Enterprise

SQL NoSQL

Session Agenda• NoSQL – What are people looking for?• RDBMS – What advantages do they still have?• How MySQL Delivers the Best of Both Worlds

– MySQL Cluster• NoSQL attributes: Scale-out, performance, ease-of-use, schema flexibility, on-line operations• NoSQL APIs

– Key-Value store access to InnoDB (Memcached)

Types of NoSQL stores

Key-Value

• Cassandra• Memcached• BigTable• Hadoop• Voldermort

Document

• MongoDB• CouchDB

• Neo4J• FlockDB

Key-Value Store

Document Store

Graph Database

• Massive scalability– No application-level sharding

• Performance• High Availability/Fault Tolerance• Ease of use

– Simple operations/administration– Simple APIs– Quickly evolve application & schema

Scalability

Performance

Ease of use

What NoSQL must deliver

Simple access patterns

Compromise on consistency for performance

Ad-hoc data format

Simple operation

Complex queries with joins

ACID transactions

Well defined schemas

Rich set of tools

• No best single solution fits all• Mix and match

Still a role for SQL (RDBMS)?

Scalability

Performance

Ease of use

SQL/Joins

ACID Transactions

The Temptation to Jettison Relational Model

• Relational Model:– Data stored in multiple tables– Many lookups to retrieve a user’s

timeline– Is it worth the effort in setting up

this complex data model?

• Simpler just to store as one document?

The allure of document stores

Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy

friend

comment

commenter

• Document Model:– Entire timeline in a single

document (row)– Single lookup to retrieve the

user’s timeline– Brilliantly efficient model when

the document truly contains self-contained information

• Like a real-world document!

The allure of document stores{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}

• These are all people who have their own data that users will want to view or click through:– Name– url– Timeline

• Easy to represent with FKs in a relational model

But when the data isn’t self contained…

friend

comment

commenter

• Do I store all data for all friends and likers again at every point they appear in the document?

• Massive amount of repeated data– Wasted space– Needs to be kept in sync (and you don’t

have transactions to provide consistency)

The allure of document stores{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}

• The reality is that the developer will store the user-ids instead

• Developer is responsible for implementing ‘joins’ in the application– e.g. scanning all timelines for where a

user made a comment or liked a post

• No transactions to ensure consistency between documents

The allure of document stores{name: 83746251, url: ‘…’ stream:[ {friend:{ name: 9384726153, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: 83746251}, {…} ], likes: [83746251, 750730283, 2938493820] }, {…},{…},… ] }, {…},{…},…}

MySQL Cluster Overview• Auto-Sharding, Multi-Master• ACID Compliant, OLTP + Real-Time Analytics

HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure• Self Healing + On-Line Operations99.999% AVAILABILITY

• In-Memory Optimization + Disk-Data• Predictable Low-Latency, Bounded Access TimeREAL-TIME

• Key/Value + Complex, Relational Queries• SQL + Memcached + JavaScript + Java + HTTP/REST & C++SQL + NoSQL

• Open Source + Commercial Editions• Commodity hardware + Management, Monitoring ToolsLOW TCO

Who’s Using MySQL Cluster?

MySQL Cluster Architecture

MySQL Cluster Data Nodes

Clients

Application Layer

Data Layer

MySQL Cluster Scaling

Clients

Application Layer

Data Layer

MySQL Cluster HA

Clients

Application Layer

Data Layer

On-line Operations• Scale the cluster (add & remove nodes on-line)• Repartition tables• Upgrade / patch servers & OS• Upgrade / patch MySQL Cluster• Back-Up• Evolve the schema on-line, in real-time

1.2 Billion UPDATEs per Minute

• NoSQL C++ API, flexaSynch benchmark

• 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB

• ACID Transactions, with Synchronous Replication

2 4 6 8 10 12 14 16 18 20 22 24 26 28 300

Join Performance

• Web-Based Content Management System– JOINs 11-tables, 33.5k rows– Returns 2k rows, 19 columns per row

MySQL Cluster 7.1 MySQL Cluster 7.20

Query Execution Time Seconds

87.23 seconds

1.26 seconds

70x Faster

Scalability a

Performance a

Ease of use

SQL/Joins a

ACID Transactions a

Better performance and operational simplicityMySQL Cluster 7.4.1 DMR

Performance gain over 7.347% (Read-Only)38% (Read-Write)

• Faster node restarts

Creating & running your first Cluster

Download & Extract• edelivery.oracle.com• www.mysql.com• dev.mysql.com

Configure• Cluster-wide

“config.ini”• Per-mysqld “my.cnf”

Start processes• Management Nodes• Data Nodes• MySQL Servers

Scalability a

Performance a

Ease of use a

SQL/Joins a

ACID Transactions a

MySQL Cluster Auto-Installer

• Fast configuration• Auto-discovery• Workload optimized• Repeatable best practices

Specify Workload

Auto-Discover

Define TopologyDeploy

MySQL Cluster Manager1. Download MCM/Cluster package from edelivery.oracle.com:2. Unzip3. Run agent, define, create & start Cluster!$> bin\mcmd --bootstrapMySQL Cluster Manager 1.1.2 started

Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862

Configuring default cluster 'mycluster'...

Starting default cluster 'mycluster'...

Cluster 'mycluster' started successfully

ndb_mgmd NOVA:1186

ndbd NOVA

mysqld NOVA:3306

mysqld NOVA:3307

ndbapi *

Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306 -u root

MCM: Upgrade Cluster

mysql> upgrade cluster--package=7.3 mycluster;

Scalability a

Performance a

Ease of use a

SQL/Joins a

ACID Transactions a

NoSQL Access to MySQL Cluster data

Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps AppsJPA

Cluster JPAPHP Perl Python Ruby JDBC Cluster J JS Apache Memcached

MySQL JNI Node.JS mod_ndb ndb_engNDB API (C++)

Schema-Free apps

• Rapid application evolution– New types of data constantly

added– No time to get schema extended– Missing skills to extend schema– Initially roll out to just a few users– Constantly adding to live system

Cluster & Memcached – Schema-Free

<town:maidenhead,SL6>

key value

Key Value

town:maidenhead SL6

generic table

Application view

SQL view

Cluster & Memcached - Configured Schema

prefix key value

key value

Prefix Table Key-col Val-col policy

town: map.zip town code cluster

Config tables

town ... code ...

maidenhead ... SL6 ...

map.zip

Application view

SQL view

MySQL 5.6 Memcached with InnoDB

8 32 128 5120

Memcached API

Client Connections

Clients and Applications

MySQL ServerMemcached Plug-in

innodb_memcached

local cache(optional)

Handler API InnoDB API

InnoDB Storage Engine

mysqld process

SQL Memcached Protocol

Up to 9x Higher “SET / INSERT” Throughput

Node.js NoSQL API• Native JavaScript access to MySQL Cluster

– End-to-End JavaScript: browser to the app & DB– Storing and retrieving JavaScript objects

directly in MySQL Cluster – Eliminate SQL transformation

• Implemented as a module for node.js– Integrates Cluster API library within the web

• Couple high performance, distributed apps, with high performance distributed database

• Optionally routes through MySQL Server– Use with InnoDB!

NoSQL API for Node.js & FKs

FKs enforced on all APIs:{ message: 'Error', sqlstate: '23000', ndb_error: null, cause: {message: 'Foreign key constraint violated: No parent row found [255]', sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } }

Oracle University MySQL Training ServicesPrepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications

“Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013

Premier Support customers eligible to save 20% on learning credits.

Benefits Expert-led training to support your MySQL learning needs Flexibility to train in the classroom or online Hands-on experience to gain real world experience Key skills needed for database administrators and developers

• MySQL for Beginners MySQL for Database Administrators MySQL Performance Tuning MySQL Cluster – NEW - Register Your Interest! MySQL and PHP - Developing Dynamic Web Applications MySQL for Developers MySQL Developer Techniques

MySQL 5.6 Database Administrator MySQL 5.6 Developer

To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql

Top Courses for Administrators and Developers

Top Certifications

RECENTLY RELEASED!!ALL NEW!! MySQL Cluster Training

To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the

MySQL Cluster Course

Next StepsLearn More• www.mysql.com/cluster• Authentic MySQL Curriculum: http://oracle.com/education/mysql

Try it Out• dev.mysql.com/downloads/cluster/

Let us know what you think• clusterdb.com• @clusterdb• forums.mysql.com/list.php?25

NoSQL & SQL- The best of both worlds

Software

Transcript of NoSQL & SQL- The best of both worlds

Bridging SQL and NoSQL

SQL gene in NoSQL

SQL vs NoSQL: The NoSQL way

Sql vs nosql joomlafestival

SQL or NoSQL - TrueNorthPHP

Migrar SQL Para Nosql

NoSQL and SQL - Open Analytics Summit

NoSQL and SQL - Why Choose? Enjoy the best of both worlds with MySQL

Generating NoSQL from SQL

NoSQL and SQL Databases

Comparing sql and nosql dbs

NoSQL & SQL - FOSDEM & SQL Blending the best of both worlds ... Oracle reserves the right to change the timing and content of any future release ... Memcached NoSQL Access with InnoDB

SQL and NoSQL in SQL Server

iForum 2015: SQL vs. NoSQL

SQL or NoSQL - how to choose

SQL vs Nosql - Cartoon strip

NoSQL - WordPress.com · จาก SQL สู่NoSQL • ต้องการประมวลผลข้อมูลจำนวนมากbig data) ( และรองรับผู้ใช้

NoSQL for the SQL Server Pro

NoSQL, no SQL injections?

NoSQL or Not Only SQL - Montana Technological University · NoSQL or Not Only SQL . Reasons to go to a NoSQL database: ... SQL NoSQL ACID: • Atomic • Consistent • Isolation