Couchbase Live Europe 2015: Introducing Multi Dimensional Scale: A New Architecture for Workload...

Multi-Dimensional Scaling

A new Architecture for Workload Optimization

in Couchbase Server 4.0

Cihan BiyikogluDir. Product Management

1

©2015 Couchbase Inc. 2

Agenda

Brief History of Scaling Scaling up and out

NoSQL Workloads and Scalability Model Core Data operations, Indexing and Query

Introducing Multi-Dimensional Scaling Services and Independent Scaling

Demo Q&A


Disclaimer

Couchbase Server 4.0 is still in development. Detail presented in this presentation may change based on customer feedback and other factors by the time the final version of the product is released.

History of Scaling


Question

Few million people are looking for a setup to efficiently live and interact. What is the most efficient way to build this infra?A) Build one giant high-rise?B) Build some mid-rises?C) Build many single-family homes


Scaling Up

Build one big high-rise Vertical Scaling

Cluster processors – hyper-threading to cores Locally partition workload among processors Communicate over memory

Great for fast processing but limited in scalability and elasticity


Scaling out

Build a large community of single-family houses Horizontal Scaling

Cluster commodity HW Partition workload among nodes Communicate over network

Great for scaling and elasticity but slower communication


So what is the right model

?

NoSQL Workloads &Scalability Model

©2014 Couchbase, Inc. ©2015 Couchbase Inc. 10

NoSQL Workloads

One Database Many Workloads Core Data Processing: GETs & SETs for given key Indexing: Index maintenance and lookups Querying: Combine index and data with complex just-in-

time data re-shaping, ordering, grouping, aggregations and more

Varying resource requirements - CPU, RAM, I/O, Network

Varying methods to optimize latency & throughput for each

10


Scalability Model TodayHomogenous Scaling

Each node get a slice of the workload Simple to do…

But... Workloads compete and interfere with each other Cant fine tune each workload

- Core Data operation are partition-able so great with wider fan-out- Indexing and Query not always partition-able so worse with wider fan-out

Index Service

Couchbase Cluster

Query Service

Data Service

node1 node8

Introducing Multi-Dimensional Scaling


Modern Architecture

What is Multi-Dimensional Scaling? MDS is the architecture that enables independent scaling of data, query and indexing workloads.

Index Service

Couchbase Cluster

Query Service

Data Service

node1 node8


Modern Architecture

Isolated Service for minimized interference Independent “zones” for Query, Index and Data Services

Minimize indexing and query overhead on core KV operations

Index ServiceGlobal

Secondary Indexes

Couchbase Cluster

Query Service

Data ServiceViews and Geo Views

node1 node8


Modern Architecture Independent Scalability for Best Computational Capacity per

Service

Heavier indexing (index more fields) : scale up index service nodesMore RAM for query processing: scale up query service nodes

Couchbase Cluster

node1 node8 node9

Data Service

Index ServiceQuery

Service

Services ArchitectureData, Index & Query

©2014 Couchbase Inc.

Full Cluster Architecture

17

STORAGE

Couchbase Server 1

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache

Cluster ManagerCluster

Manager

Managed Cache

Storage

Data Service

Index Service

Query Service

STORAGE

Couchbase Server 2

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Manager

Managed Cache

Storage

Data Service

Index Service

Query Service

STORAGE

Couchbase Server 3

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Manager

Managed Cache

Storage

Data Service

Index Service

Query Service

STORAGE

Couchbase Server 4

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Manager

Managed Cache

Storage

Data Service

Index Service

Query Service

STORAGE

Couchbase Server 5

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Manager

Managed Cache

Storage

Data Service

Index Service

Query Service

STORAGE

Couchbase Server 6

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Manager

Managed Cache

Storage

Data Service

Index Service

Query Service


Data Service

Projector & Router

New in Couchbase Server 4.0

Data Service: GET & SET operations and Map/Reduce Views Arch mostly the same as 3.0

Index Service: new Global Secondary Indexes (GSI) Query Service: new Query processor (N1QL)

Query Service

Query Processor

cbq-engine

Index Service

SupervisorIndex maintenance &

Scan coordinator

Index#2

Index#1

Query Processorcbq-engine

Bucket#1

Bucket#2

DCP StreamIndex#4Index#3

...

Bucket#2

Bucket#1


Recap

MSD enables unprecedented control on scalability with Couchbase Server Separate out competing workloads to independent services Independently scale each service “zone” within the cluster

Couchbase Server with MDS maximizes scalability and performance Improves scale and performance to degrees not possible

with other NoSQL or Big Data engines on premise or in the cloud

Improved price/performance and squeezes more performance and throughput for mission critical systems


Couchbase Server 4.0

Couchbase Server 4.0

Download the Developer Preview in a few weeks…

Couchbase.com/

Q&ACihan Biyikoglu

[email protected]@cihangirb

mailto:[email protected]

Couchbase Live Europe 2015: Introducing Multi Dimensional Scale: A New Architecture for Workload...

Software

Transcript of Couchbase Live Europe 2015: Introducing Multi Dimensional Scale: A New Architecture for Workload...