ScaleDB Transactional Shared Disk storage engine for MySQL Moshe Shadmon, Founder, CTO.
-
Upload
julian-mcmillan -
Category
Documents
-
view
230 -
download
5
Transcript of ScaleDB Transactional Shared Disk storage engine for MySQL Moshe Shadmon, Founder, CTO.
ScaleDBTransactional Shared Disk storage engine for MySQL
Moshe Shadmon, Founder, CTO
The product
Storage Layer
MySQL-ScaleDB Interface
DBMS/Apps
Storage Engine
ScaleDB Benefits
Enables open source DBMS (eg: MySQL) to operate as clustered, high end database
Scales applications in cloud environments No need to change the applications Provides high performance Provides unified data store
Shared Disk vs. Shared Nothing
Shared Nothing Shared Disk
Masters
Slaves
SQL ServerDB2 (Unix)MySQLPostgreSQL
Oracle RACIBM - DB2, IMS (MF)ScaleDB
Shared Disk vs Shared Nothing
Shared Nothing Shared Disk
Storage Per DBMS machine Shared storage (NAS/SAN)
Scalability By data partitioning By adding CPU
Scaling complexity High Low
High Availability Master-Slave Multi-Masters
HA complexity High Low
Implemented By MySQLPostgreSQLSQL ServerDB2 (Unix)
Oracle RACIBM on mainframe (DB2/IMS)ScaleDB
Shared disk provides “out of the box” scalability and availability
Currently available with the high end commercial databases only
Server 1
OSS DBMSOSS
DBMS
ScaleDBScaleDB
VM
The Virtualized Cloud Database
Local DiskLocal Disk
OSS DBMSOSS DBMS
Storage EngineStorage Engine
MySQL Server Server 2
OSS DBMSOSS
DBMS
ScaleDBScaleDB
VM
OSS DBMSOSS
DBMS
ScaleDBScaleDB
VM
OSS DBMSOSS
DBMS
ScaleDBScaleDB
VM
OSS DBMSOSS
DBMS
ScaleDBScaleDB
VM
Shared StorageShared Storage
Shared Disk
Shared Nothing
Standby Cluster ManagerStandby Cluster Manager
Deploying ScaleDB
…
ScaleDB Cluster ManagerScaleDB Cluster Manager
Node 1
DBMSDBMS
ScaleDBScaleDB
Node 2
DBMSDBMS
ScaleDBScaleDB
Node N
DBMSDBMS
ScaleDBScaleDB
ApplicationApplicationApplication Layer
Database Layer
(Physical or VM nodes)
Storage LayerShared StorageShared Storage Shared StorageShared Storage
ScaleDB
Benefits of ScaleDB Technology
High Availability – No single point of failure
If a node fails – a different node takes over If the cluster manager fails – the standby cluster manager is
initiated to manage the cluster
Scalability No limit on the number of nodes that can join the cluster Algorithms to minimize the need to communicate
Performance Innovative, high performance, general purpose indexing
ScaleDBCluster Manager
ScaleDB
Node
ScaleDB APIScaleDB API
Transaction ManagerTransaction Manager
IndexManager
IndexManager
DataManager
DataManager
Buffer ManagerBuffer Manager
Lock Manager
Local Lock Manager
Local Lock Manager
Global Lock CoordinatorGlobal Lock Coordinator
Log ManagerLog Manager
RecoveryManagerRecoveryManager
Storage ManagerStorage Manager
Global Recovery Manager
Global Recovery Manager
Global LockManager
Global LockManager
Global BufferManager
Global BufferManager
File SystemFile System
Storage DevicesStorage DevicesStorage DevicesStorage Devices
ScaleDB’s Internal Architecture
ApplicationsApplications XMLXML
ScaleDBCluster Manager
ScaleDB
Node
Read Customer Info
Read Customer Info
File SystemFile System
Storage DevicesStorage DevicesStorage DevicesStorage Devices
Distributed Lock Manager
Index translates logical id to Row ID
Index translates logical id to Row ID
Shared DBMS lock
Shared DBMS lock
Shared Table lock
Shared Table lock
Shared Row lockShared
Row lock
DBMS XDBMS X
Table YTable Y
Block ZBlock Z
Node 1 R/W
Node 1 R/W
Node 1 R
Node 2 R/W
Node 2 R/W
Node 2 R
Node 2 R/W
H List
H List
H List
W List
Shared Disk in the Cloud
Cloud provides Scalability and HA
Scalability - Enables dynamic allocation of computing and storage
Availability – Multiple servers over the same physical data High performance
Reduces Management Overhead
Reduced tuning and partitioning support No manual slave promotion or replication
Performance / Scalability
Benchmarks
Join operations using Multi-Table index
Sysbench
TPCC
Example multiple MySQL + ScaleDB nodes on Amazon EC2
Demo
Benchmark multi-table index
0
10
20
30
40
50
60
Engine X Join ScaleDB MTI ScaleDB 2 Nodes
Queries/Sec
Sysbench Performance on EC2
775 TPS
415
685
954
1224
1494
1764
2033
Number of Nodes1
2303 TPS
1 2 3 4 5 6 7 8
Transactions Per Second
ScaleDB
Other Engine
Current & Future Work
Optimizing TPCC – soon to be published
Integrating with other environments
GA – later this year
Demo