CouchConf Israel 2013_Couchbase Tour

2

Announcement: local vendor at Israel via

3

When you tweet…

#CouchConf

4

Stay for the Q&A Session!

Beer. Wine. Snacks. Don’t miss it.

5

Submit questions for the Q&A session...

Email: [email protected]: @couchbase

...or write it down and hand it to any of the Couchbase staff!

6

Tell Us What You Think!

DO THIS

GET THIS

7

WiFi is available...

Network: exodiaPassword: 1234567890

9

Couchbase NoSQL Leadership

Leading NoSQL database companyOpen Source development & business modelBehind Couchbase open source project

Most mature, reliable and widely deployed solution>5,000 paid production deployments worldwide, over 350 customers

Headquarters in Silicon Valley (Mountain View, CA)~100 employees including >60 in engineering/product>80% of commits to Couchbase, memcached, Apache CouchDB

Document-oriented NoSQL databaseFocused on interactive internet and mobile applications

Provide more flexible, higher performance, more scalable database than relational alternative

10

Market Adoption – Customers

Internet Companies Enterprises

More than 350 customers -- 5,000 production deployments worldwide

11

Introduction to Couchbase Server 2.0

Sharon BarrVP of Engineering

12

2.0

NoSQL Document Database

Couchbase Server

13

Easy Scalabili

ty

Consistent High

Performance

Always On

24x365

Grow cluster without application changes, without downtime with a single click

Consistent sub-millisecond read and write response times

with consistent high throughput

No downtime for software upgrades, hardware maintenance, etc.

Couchbase Server 2.0

JSONJSONJSON

JSONJSON

PERFORMANCE

Flexible Data Model

JSON document model with no fixed schema.

14

Flexible Data Model: Relational vs Document Data Model

Relational data model Document data modelCollection of complex documents with

arbitrary, nested data formats andvarying “record” format.

Highly-structured table organization with rigidly-defined data formats and

record structure.

JSONJSON

C1 C2 C3 C4

JSON

{

}

15

RDBMS Example: User Profile

Address Info

1 DEN 30303CO

2 MV 94040CA

3 CHI 60609IL

User Info

KEY First ZIP_idLast

4 NY 10010NY

1 Frank 2Weigel

2 Ali 2Dodson

3 Mark 2Azad

4 Steve 3Yen

ZIP_id CITY ZIPSTATE

1 Frank 2Weigel

To get info about specific user, you perform a join across two tables

16

All data in a single document

Document Example: User Profile

{ “ID”: 1, “FIRST”: “Frank”, “LAST”: “Weigel”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” }

JSON

= +

17

Flexible Data Model

• No need to worry about the database when changing your application

• Records can have different structures, there is no fixed schema

• Allows painless data model changes for rapid application development

{ “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA”}

JSONJSON

JSON JSON

18

Couchbase Server Features

Built-in clustering – All nodes equal

Data replication with auto-failover

Zero- downtime maintenance

Clone to grow and scale horizontally

Built-in managed cached

Monitoring and administration APIs and GUI

SDK for a variety of languages

19

New in 2.0

JSON support Indexing and Querying

Cross data center replicationIncremental Map Reduce

JSONJSONJSON

JSONJSON

20

Additional Couchbase Server 2.0 Features

Append-only storage layer

Online compaction

Better working set management

Reduce server warm-up time

21

storage interface

Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpRE

ST m

anag

emen

t API

/Web

UI

8092Couch View

CouchStoreAuto compaction

Memcached Interface

Couchbase Server 2.0 Architecture

Couc

h AP

I

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 - 21199

Erlang/OTP

Distributed Indexing

Hash table cache Write/replicaQueues Cluster ManagerData Manager

23

Couchbase deployment

Data Flow

Cluster Management (8091)

Web Application

CouchbaseClient Library

Data ports

24

COUCHBASE OPERATIONS

25

33 2

Single node - Couchbase Write Operation2

Managed Cache

Dis

k Q

ueue

Disk

Replication Queue

App Server

Couchbase Server Node

Doc 1Doc 1

Doc 1

To other node

26

33 2

Update Operation2

Managed Cache

Dis

k Q

ueue

Replication Queue

App Server


Doc 1’

Doc 1

Doc 1’Doc 1

Doc 1’

Disk

To other node

27

33 2

Cache Eviction2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 6Doc 5Doc 4Doc 3Doc 2

Doc 1

Doc 6 Doc 5 Doc 4 Doc 3 Doc 2

Managed Cache

Disk

To other node

28

GET

Doc

1

33 2

Read Operation2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 1Doc 1

Managed Cache

Disk

To other node

29

33 2

Cache Miss2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 3Doc 5 Doc 2Doc 4

Doc 6 Doc 5 Doc 4 Doc 3 Doc 2

Doc 4

GET

Doc

1

Doc 1

Doc 1

Managed Cache

Disk

To other node

30

Partitioning The Data – vbucket (internal partitions) map

31

COUCHBASE SERVER CLUSTER

Cluster wide - Basic Operation

• Docs distributed evenly across servers

• Each server stores both active and replica docsOnly one server active at a time

• Client library provides app with simple interface to database

• Cluster map provides map to which server doc is onApp never needs to know

• App reads, writes, updates docs

• Multiple app servers can access same document at same time

User Configured Replica Count = 1

READ/WRITE/UPDATE

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 1

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc

SERVER 2

Doc 8

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

REPLICA

Doc 6

Doc 3

Doc 2

Doc

Doc

Doc

REPLICA

Doc 7

Doc 9

Doc 5

Doc

Doc

Doc

SERVER 3

Doc 6

APP SERVER 1

COUCHBASE Client Library

CLUSTER MAP


CLUSTER MAP

APP SERVER 2

Doc 9

32

Cluster wide - Add Nodes to Cluster

• Two servers addedOne-click operation

• Docs automatically rebalanced across clusterEven distribution of docsMinimum doc movement

• Cluster map updated

• App database calls now distributed over larger number of servers

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc

Doc 8 Doc

Doc 9 Doc

Doc 2 Doc

Doc 8 Doc

Doc 5 Doc

Doc 6

READ/WRITE/UPDATE READ/WRITE/UPDATE

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2



33

Cluster wide - Fail Over Node

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc 9

Doc 8

Doc Doc 6 Doc

Doc

Doc 5 Doc

Doc 2

Doc 8 Doc

Doc

• App servers accessing docs

• Requests to Server 3 fail

• Cluster detects server failedPromotes replicas of docs to activeUpdates cluster map

• Requests for docs now go to appropriate server

• Typically rebalance would follow

Doc

Doc 1 Doc 3

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2



34

DEMO TIME

35

Indexing and Querying – The basics

• Define materialized views on JSON documents and then query across the data set • Using views you can define

• Primary indexes • Simple secondary indexes (most common use case)• Complex secondary, tertiary and composite indexes• Aggregations (reduction)

• Indexes are eventually indexed • Queries are eventually consistent with respect to

documents• Built using Map/Reduce technology

• Map and Reduce functions are written in Javascript

36

33 2

Eventually indexed Views – Data flow2

Managed Cache

Dis

k Q

ueueDisk

Replication Queue

App Server


Doc 1Doc 1

Doc 1

To other node

View engine

Doc 1

37


Cluster wide - Indexing and Querying


ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 1

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2

Doc 9

• Indexing work is distributed amongst nodes

• Large data set possible

• Parallelize the effort

• Each node has index for data stored on it

• Queries combine the results from required nodes

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 2

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

Doc 9

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 3

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

Doc 9

Query

38

Cross Data Center Replication – The basics

• Replicate your Couchbase data across clusters• Clusters may be spread across geos• Configured on a per-bucket basis• Supports unidirectional and bidirectional operation• Application can read and write from both clusters

(active – active replication)• Replication throughput scales out linearly• Different from intra-cluster replication

39

33 2

Cross data center replication – Data flow2

Managed Cache

Dis

k Q

ueue

Disk

Replication Queue

App Server


Doc 1Doc 1

Doc 1

To other node

XDCR Queue

Doc 1

To other cluster

40

Cluster wide - XDCR

COUCHBASE SERVER CLUSTERNY DATA CENTER

ACTIVE

Doc

Doc 2

SERVER 1

Doc 9

SERVER 2 SERVER 3

RAM

Doc Doc Doc

ACTIVE

Doc

Doc

Doc RAM

ACTIVE

Doc

Doc

DocRAM

DISK

Doc Doc Doc

DISK

Doc Doc Doc

DISK

COUCHBASE SERVER CLUSTERSF DATA CENTER

ACTIVE

Doc

Doc 2

SERVER 1

Doc 9

SERVER 2 SERVER 3

RAM

Doc Doc Doc

ACTIVE

Doc

Doc

Doc RAM

ACTIVE

Doc

Doc

DocRAM

DISK

Doc Doc Doc

DISK

Doc Doc Doc

DISK

41

DEMO TIME

42

Demo: The next big social game

3 Objects (documents) within game:• Players•Monsters• Items

Gameplay:• Players fight monsters•Monsters drop items• Players own items

43

Player Document

{"jsonType": "player","uuid": "35767d02-a958-4b83-8179-616816692de1","name": "Keith4540","hitpoints": 75,"experience": 663,"level": 4,"loggedIn": false

}

Player ID

44

Item Document

{"jsonType": "item","name": "Katana_e5890c94-11c6-65746ce6c560","uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560","ownerId": "Dale9887"

}

Item ID

Player ID

45

Monster Document

{"jsonType": "monster","name": "Bauchan9932","uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec","hitpoints": 370,"experienceWhenKilled": 52,"itemProbability": 0.5050581341872865

}

Monster ID

46

GAME ON!

47

Full Text Search Integration

• Elastic Search is good for ad-hoc queries and faceted browsing• Couchbase adapter uses XDCR to push mutations to ESDocs are

indexed by Elastic Search• Couchbase ES Adapter is cluster-aware

ElasticSearch

Unidirectional Cross Data Center Replication

48

Full Text Search

49

Couchbase SDKs

Java SDK

.Net SDK

PHP SDK

Ruby SDK

…and many more

Java client API

User Code

Couchbase Server

CouchbaseClient cb = new CouchbaseClient(listURIs,"aBucket", "letmein");

cb.set("hello", 0, "world");cb.get("hello");

http://www.couchbase.com/develop

Couchbase Java Library (spymemcached)

50

QUESTIONS?

50

51

THANK YOU

COUCHBASE SIMPLE, FAST, ELASTIC NOSQL

[email protected]@sharonyb

CouchConf Israel 2013_Couchbase Tour

Documents

Transcript of CouchConf Israel 2013_Couchbase Tour