Three things you need to know about document data modelling

Three things you need to know about document data modellingMatthew RevellDirector of Developer Advocacy, Couchbase

1

We are still learning

2

©2014 Couchbase Inc.

Book learnin'

3

Our access pattern influences the data model

4


Ad-hoc versus predictable

5

Application layer computation Off-load computationPredictable

queriesKey-value: pre-computed answers Views

Ad-hoc queries N1QL and key-value with manual indexes

N1QL and views

Modelling for key-value

6


Principles for key-value modelling

7

Pre-compute answers asynchronouslyStore object stateChoose when to embed data and when to refer to itDesign your keys well

Pre-computing answers

8


We're used to asking questions

9


A library of answers

10


Answer-oriented databases

11

http://martinfowler.com/bliki/AggregateOrientedDatabase.html


Save the answers for later use

12

Embed or refer?

13


An e-commerce order

14


The same order as a document

15


The same order as a document

16


When to embed data

17

You should embed data when:

Speed trumps all elseSlow moving dataNo duplicationApplication layer can keep multiple copies of

same data in sync


When to referdata

18

You should refer to data when:

Consistency is a priorityThe data has large growth potential

Key design

19


Three ways to build a key

20

Key design is as important as document design.

There are three broad types of keys:

Human readable/deterministic: e.g., an email address

Computer generated/random: e.g., a UUIDCompound: e.g., a UUID with a deterministic

portion


Human readable/deterministic key

21

public class user {

private String name;private String email;private String streetAddress;private String city;private String country;private String postCode;private String telephone;private Array orders;private Array

productsViewed;}

{ "name": "Matthew Revell", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ], “productsViewed”: [8, 33, 99, 100] }

Key: [email protected]


Counter key pattern

22

Application

user_id = incr(“counter_key")

add(user_id, data)

Creating a new user

add(email_address, user_id)

Application

key = get("[email protected]")

get(key)

Finding a user by email address

mailto:[email protected]


Multiple look-up documents

23

u::count

1001

u::1001

{ "name": “Matthew Revell", "facebook_id": 16172910,

"email": “[email protected]”,“password”: ab02d#Jf02K

"created_at": "5/1/2012 2:30am",“facebook_access_token”: xox0v2dje20,“twitter_access_token”: 20jffieieaaixixj }

fb::16172910

1001

nflx::2939202

1001

twtr::2920283830

1001

em::[email protected]

1001

em::[email protected]

1001

uname::mrevell

1001


Compound keysu::1001

{ "name": "Matthew Revell", "email": "[email protected]", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ], “productsViewed”: [8, 33, 99, 100]

}



{ "name": "Matthew Revell", "email": "[email protected]", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ]}

u::1001::productsviewed

{"productsList": [

8, 33, 99, 100]

}



{ "name": "Matthew Revell", "email": "[email protected]", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ]}

u::1001::productsviewed

{"productsList": [

8, 33, 99, 100]

}

p::8

{

id": 1,"name": "T-shirt","description": "Red Couchbase shirt","quantityInStock": 99,"image": "tshirt.jpg”

}

p::8::img

“http://someurl.com/tshirt.jpg”

Data modelling for N1QL

27


N1QL

28

• N1QL implements much of SQL++• Dive into arrays and objects• NEST data from JOINs• UNNEST data• Gracefully handles MISSING data


A N1QL Example

29

SELECT * FROM `travel-sample` r JOIN `travel-sample` a ON KEYS r.airlineid WHERE r.sourceairport="LHR" AND r.destinationairport = "SFO";


JOINs and keys

34

JOINs work on primary keys and secondary keys.

They JOIN across and within keyspaces (for now, that means buckets).

Airlines:

airline_24 ← Key (“primary key”){ "id": "24", "type": "airline", "name": "American Airlines", "iata": "AA", "icao": "AAL", "callsign": "AMERICAN", "country": "United States", "active": "Y"}

Routes:

route_5966 ← Key { "id": "5966", "type": "route", "airline": "AA", "airlineid": "airline_24", ← This is the foreign key "sourceairport": "MCO", "destinationairport": "SEA", "stops": "0", "equipment": "737", "schedule": [... ]}

Next Steps

Couchbase Developer Portal

developer.couchbase.com

40

http://developer.couchbase.com/

Forums

http://forums.couchbase.com

41

http://www.meetup.com/Couchbase-Dublin/



Three things you need to know about document data modelling

Technology

Transcript of Three things you need to know about document data modelling