KVIV / NoSQL : the new generation of database servers

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

NoSQLde nieuwe generatie van database serversKVIV IT - 3/6/2010

http://www.flickr.com/photos/wolfgangstaudt/2215246206/


Who am I

» Steven Noels - [email protected]

»Outerthought : scalable content applications

»makers of Daisy and Lily open source CMS

2


An evolution driven by pain.


History

4

hierarchical databases

IMS

OODBMS

XMLDB RDBMS

1. standardization

2. simplification


History

5

RDBMS NOSQL

cachingdenormalisationshardingreplication ...

3. pain

4. rethinkingthe problem


Scalability


Numbers of scale

7

http://qos.doubleclick.net/counters/


Types of scaling

8

» scaling for usage» volume of users

» volume of data

availabilityreplication

» scaling types of ops» concurrent read

» concurrent write

partioningconsistency

distribution


Scaling

9

database app server users


Scaling

10



Scaling: vertical partitioning

11



Scaling: horizontal partitioning

12

databases app servers users

complexity


Scaling through architecture

13

data layer app layer users


Distributed systems are hard !


8 fallacies of distributed computing» The network is reliable.

» Latency is zero.

» Bandwidth is infinite.

» The network is secure.

» Topology doesn't change.

» There is one administrator.

» Transport cost is zero.

» The network is homogeneous.

15

Pete

r D

euts

ch a

nd Ja

mes

Gos

ling

http://en.wikipedia.org/wiki/Computer_network

http://en.wikipedia.org/wiki/Computer_network

http://en.wikipedia.org/wiki/Latency_%28engineering%29

http://en.wikipedia.org/wiki/Latency_%28engineering%29

http://en.wikipedia.org/wiki/Throughput

http://en.wikipedia.org/wiki/Throughput

http://en.wikipedia.org/wiki/Computer_security

http://en.wikipedia.org/wiki/Computer_security

http://en.wikipedia.org/wiki/Network_topology

http://en.wikipedia.org/wiki/Network_topology

http://en.wikipedia.org/wiki/Network_administrator

http://en.wikipedia.org/wiki/Network_administrator


Scaling relational systems

16

»When scaling relational systems you loose their advantages but retain their overhead

»The pain is all about locking (i.e. writes)

»Caching alleviates the read pain to the cost of complexity

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 17

The Perspective of Cost


Enter NoSQL


Cambrian Explosion

19


?Buzz-oriented development


Cambrian Explosion

21

N-O-SQL


Common themes

23

» SCALE SCALE SCALE

» new datamodels

» devops

»N-O-SQL

»The Cloud :technology is of no interest anymore


New Data

» sparse structures

»weak schemas

» graphs

» semi-structured

» document-oriented

24


NoSQL

»Not a movement.

»Not ANSI NoSQL-2010.

»Not one-size-fits-all.

»Not (necessarily) anti-RDBMS.

»No silver bullet.

25


NoSQL = pro Choice

26


NoSQL = toolbox

27


The NOSQL footprint

28

AC

ID,

sim

ple

oper

atio

nal

const

rain

ts

free-structured or sparse data

SQL

NOSQL

referential integrity,typed data

high

ly scalable an

davailab

le (com

plex

ity)

HBase

Cassandra

CouchDB

MongoDB

neo4j


NOSQL, if you need ...

» horizontal scaling (out rather than up)

» unusually common data (aka free-structured)

» speed (especially for writes)

» the bleeding edge

29


SQL/RDBMS, if you need ...

» SQL

»ACID

» normalisation

» a defined liability

30


Theory


Academic background

»Amazon Dynamo

»Google BigTable

» Eric Brewer CAP theorem

32


Amazon Dynamo

33

» coined the term ‘eventual consistency’

» consistent hashing


Consistent hashing

34

- node C+ node D

http://www.lexemetech.com/2007/11/consistent-hashing.html

»multi-dimensional column-oriented database

» on top of GoogleFileSystem

» object versioning


Google BigTable

35


CAP theorem

36

strong consistency

highavailability

partition-tolerance


CAP

»Strong Consistency: all clients see the same view, even in the presence of updates

»High Availability: all clients can find some replica of the data, even in the presence of failures

»Partition-tolerance: the system properties hold even when the system is partitioned

37


Culture Clash

38

»ACID» highest priority: strong

consistency for transactions

» availability less important

» pessimistic

» rigorous analysis

» complex mechanisms

» BASE» availability and scaling

highest priorities

» weak consistency

» optimistic

» best effort

» simple and fast

spectrum


Availability ≠ total async !


The Enterprise Service Bus

40

✘bus =

congestion


Bus systems

41

» objects don’t fit in a pipe

» object ➙ message

» serialization / de-serialization cost

»message size

» queuing = cost


Use a mixture of both

»async + sync

42

stuff which matters !


Processing large datasets :

Map/Reduce


Hadoop: HDFS + MapReduce» single filesystem + single execution-space

44


MapReduce example: WordCount

45


Hadoop ecosystem» Hadoop Common

» Subprojects

» Chukwa: A data collection system for managing large distributed systems.» HBase: A scalable, distributed database that supports structured data storage for

large tables.» HDFS: A distributed file system that provides high throughput access to application

data.» Hive: A data warehouse infrastructure that provides data summarization and ad hoc

querying.

» MapReduce: A software framework for distributed processing of large data sets on compute clusters.

» Pig: A high-level data-flow language and execution framework for parallel computation.

» ZooKeeper: A high-performance coordination service for distributed applications.» Mahout: machine learning libaries

46

http://hadoop.apache.org/chukwa/

http://hadoop.apache.org/chukwa/

http://hadoop.apache.org/hbase/

http://hadoop.apache.org/hbase/

http://hadoop.apache.org/hdfs/

http://hadoop.apache.org/hdfs/

http://hadoop.apache.org/hive/

http://hadoop.apache.org/hive/

http://hadoop.apache.org/mapreduce/

http://hadoop.apache.org/mapreduce/

http://hadoop.apache.org/pig/

http://hadoop.apache.org/pig/

http://hadoop.apache.org/zookeeper/

http://hadoop.apache.org/zookeeper/


Processing large datasets with MR

47

»Benefit from parallellisation

» Less modelling upfront (ad-hoc processing)

»Compartmentalized approach reduces operational risks

»AsterData et al. have SQL/MR hybrids for huge-scale BI


Market overview


Four Trends

»Trend 1 : Data Size

»Trend 2 : Connectedness

»Trend 3 : Semi-structure

»Trend 4 : Architecture

49


2006 2007 2008 2009 2010

0

250

500

750

1000

161

253

397

623

988

ExaBytes (10!") of data stored per year

3

Trend 1: Data size

Data source: IDC 2007

Each year more and more digital data is created. Over two years we create more digital data than all the data created in history before that.


Trend 2: Connectedness

4

Text documents

1990

Info

rmat

ion

conne

ctiv

ity

Folksonomies

Tagging

User-generated

contentWikis

RSS

Blogs

Hypertext

2000 2010 2020

web 1.0 web 2.0 “web 3.0”

Ontologies

RDF

Giant

Global

Graph (GGG)

Over time data has evolved to be more and more interlinked and connected.Hypertext has links,Blogs have pingback,Tagging groups all related data


Trend 3: Semi-structure

5

! Individualization of content

• In the salary lists of the 1970s, all elements had exactly one job

• In the salary lists of the 2000s, we need 5 job columns! Or 8?

Or 15?

!All encompassing “entire world views”

• Store more data about each entity

!Trend accelerated by the decentralization of content generation

that is the hallmark of the age of participation (“web 2.0”)


Trend 4: Architecture

6

DB

Application

1980s: Mainframe applications



7

DB

Application

1990s: Database as integration hub

Application Application


DBDB DB


8

Application

2000s: (moving towards) Decoupled serviceswith their own backend

Application Application


Products overview


We welcome the Polyglot Persistence overlords.


Categories

» key-value stores

» column stores

» document stores

» graph databases

58


Key-value stores

» Focus on scaling to huge amounts of data

»Designed to handle big loads

»Often: cfr. Amazon Dynamo

» ring partitioning and replication

»Data model: key/value pairs

59


Key-value stores

»Redis

»Voldemort

»Tokyo Cabinet

60


Redis

»REmote DIctionary Server

» http://code.google.com/p/redis/

» vmware

61


Redis Features» persisted memcache, ‘awesome’

» RAM-based + persistable

» key ➙ values: string, list, set

» higher-level ops

» i.e. push/pop and sort for lists

» fast (very)

» configurable durability

» client-managed sharding

62


Voldemort

» http://project-voldemort.com/

» LinkedIn

63


Voldemort

» persistent

» distributed

» fault-tolerant

» hash table

64


Voldemort

65

API: GET, PUT,DELETE


Voldemort

66

routing logic moving up the stack,smaller latency


Column stores

»BigTable clones

» Sparseness!

»Data model: columns ➙ column families ➙ ACL

»Datums keyed by: row, column, time, index

» Row-range ➙ tablet ➙ distribution

67


Column stores

»BigTable

»HBase

»Cassandra

68


BigTable

» http://labs.google.com/papers/bigtable.html

»Google

» layered on top of GFS

69


HBase

» http://hadoop.apache.org/hbase/

» StumbleUpon / Adobe / Cloudera

70


HBase

» sorted» distributed» column-oriented»multi-dimensional» highly-available» high-performance

» persisted» storage system

» adds random access reads and writes atop HDFS

71


HBase data model

72

»Distributed multi-dimensional sparse map

»Multi-dimensional keys:(table, row, family:column, timestamp) → value

»Keys are arbitrary strings

»Access to row data is atomic


Storage architecture

73

© lars george


Cassandra

» http://cassandra.apache.org/

»Rackspace / Facebook

74


Cassandra

»Key-value store (with added structure)

»Reliability (identical nodes)

» Eventual consistent

»Distributed

»Tunable

» Partitioning

» Replication

75

CA

P


Cassandra applicability

76

FIT

» Scalable reliability (through identical nodes)» Linear scaling»Write throughput» Large Data Sets

NO FIT

» Flexible indexing»Only PK-based

querying»Big Binary Data» 1 Row must fit in

RAM entirely


Document databases

»≈ K/V stores, but DB knows what the Value is

» Lotus Notes heritage

»Data model: collections of K/V collections

»Documents often versioned

79


Document stores

»CouchDB

»MongoDB

»Riak

»MarkLogic

80


CouchDB

» http://couchdb.apache.org/

» couch.io

81


CouchDB

» fault-tolerant

» schema-free

» document-oriented

» accessible via a RESTful HTTP/JSON API

82


CouchDB documents

{ “_id”: ”BCCD12CBB”, “_rev”: ”AB764C”, “type”: ”person”, “name”: ”Darth Vader”, “age”: 63, “headware”: [“Helmet”, “Sombrero”], “dark_side”: true }

83


CouchDB REST API

»HTTP

» PUT /db/docid

»GET /db/docid

» POST /db/docid

»DELETE /db/docid

84


CouchDB Views»MapReduce-based

» Filter, Collate, Aggregate

» Javascript

85

function (Key, Values) { var sum = 0; for(var i in Values) sum += Values[i]; return sum; }

function (doc) { for(var i in doc.tags) emit(doc.tags[i], 1); }

map reduce


CouchDB

» be careful on semantics

» replication ≠ partioning/sharding !

» distributed database = distributable database

» sharded / distributed deploymentrequires proxy layer

86


MongoDB

» http://www.mongodb.org/

» 10gen

87


MongoDB

» cfr. CouchDB, really

» except for:

»C++

» performance focus

» runtime queries (mapreduce still available)

» native drivers (no REST/HTTP layering)

» no MVCC: update-in-place

» auto sharding (alpha)

88


Graph databases

» Focus on modeling structure of data - interconnectivity

» Scale, but only to the complexity of data

»Data model: property graphs

89


Graph databases

»Neo4j

»AllegroGraph (RDF)

90


Neo4j

» http://neo4j.org/

»Neo Technology

91


Neo4j» data = nodes + relationships + key/value properties

92


Neo4j

»many language bindings, little remoting

» ‘whiteboard’ friendly

» scaling to complexity (rather than volume?)

» lots of focus on domain modelling

» SPARQL/SAIL impl for triple geeks

»mostly RAM centric (with disk swapping & persistence)

93


Market maturization


Rise of integrators

»Cloudera (H-stack)

»Riptano (Cassandra)

»Cloudant (hosted CouchDB)

» (Outerthought: HBase)

95


VC capital

»Cloudera

» couch.io

»Neo

» 10gen

»many others

96


Experiences & (h)in(d)sights


the fireside conversations

http://www.flickr.com/photos/52641994@N00/516394238/


NOSQL applicability

»Horizontal scaling

»Multi-Master

»Data representation

» search of simplicity

» data that doesn’t fit the E-R model(graphs, trees, versions)

» Speed

99


Tool selection

» be careful on the marketeese:smoke and mirrors beware!

»monitor dev list, IRC, Twitter, blogs

»monitor project ‘sponsors’

»mix-and-match: polyglot persistency

»DON’T NOSQL WITHOUT INTERNAL SYS ARCHS & DEV(OP)S !

100


Our Context: Lily

» cloud-scalable content store and search repository

» successor (in many ways) of Daisy


aptness

complexity

inte

rnet

ente

rpri

seco

rpor

ate

com

mun

ity

NOSQL}S

QL}

102


Lily essentials

» (open source)

» scalable store (Apache HBase)

» and search (Apache SOLR)

» content repository

»α due mid 2010

»www.lilycms.org or @outerthought

103


Choosing a NoSQL store for Lily: step I

» automatic scaling to large data sets

» fault-tolerance

» flexible datamodel with sparse data

» commodity hardware

» efficient random access

» community-based open source

» Java if possible

104


» need for consistency

» atomic single-row updates

»M/R for index regeneration

105

Choosing a NoSQL store for Lily: step II


HBase» datamodel with column families and cell

versioning

» ordered tables with range scans

»HDFS for blob storage

»Apache

106

Choosing a NoSQL store for Lily: step III

lily simplified architectureIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 107

Lily client

client

client

Lily store node

store node

store node

distributed process coordination

and configuration (ZooKeeper)

query indexerupdate

WAL M/RMQ

documents2ary

indexes

WAL /

MQ

}

}

}

Lily Store Server

HBase Region Server

Hadoop DFS

index replica

replica replica

} SOLR

inverted index

REST


lily architecture


lily distributed architecture


Distribution,durability,and availabilityis everywhere


When combining store and search, make sure your (search) index doesn’t become the store.


Key lessons learned

112

» unlearning normalization is very difficult

» integrity checking in code = not so bad

» doing joins in code can be very liberating

» importance of keyspace design

» secondary indexing

» data de-normalization = size! (x3)

» schema vs. code flexibility?

» distribution is everywhereand you shouldn’t forget about it


Mis-use cases

» SQL (or ORM) is a prerequisite

»Deeply hierarchical datasets (unless graph)

»Data integrity is listed on DBA job description

»High-security apps (enforced in DB)

»Transactional data (banking)

»Usage is highly unpredictable, combinatorial, or likely to change suddenly

113


Reading material

»Amazon Dynamo, Google BigTable, CAP

» http://nosql.mypopescu.com/

» http://nosql-database.org/

» http://twitter.com/nosqlupdate

» http://highscalability.com/

114


Questions?

115

http://www.flickr.com/photos/leehaywood/4237636853/


» [email protected]

» @stevenn

Thanks for your attention !

KVIV / NoSQL : the new generation of database servers

Technology

Transcript of KVIV / NoSQL : the new generation of database servers