XebiCon'16 : Data Lake Done Right ! Par Matthieu Blanc, Data Architect chez Xebia
Xebia Knowledge Exchange (may 2010) - NoSQL : Using the right tool for the right job
-
Upload
michael-figuiere -
Category
Technology
-
view
1.937 -
download
1
Transcript of Xebia Knowledge Exchange (may 2010) - NoSQL : Using the right tool for the right job
NoSQLUsing the right tool for the right job
www.xebia.fr / blog.xebia.fr 2
No SQL
www.xebia.fr / blog.xebia.fr 3
No SQL
Not Only/ I /I
www.xebia.fr / blog.xebia.fr 4
No SQL
Not Only/ I /I/
/
www.xebia.fr / blog.xebia.fr 5
No SQL
Not Only/ I /I/
/Relational
www.xebia.fr / blog.xebia.fr 6
Need to store some data ?
Oracle
SQL Server
Sybase
DB2
MySQL
PostgreSQL
www.xebia.fr / blog.xebia.fr 7
RDBMS are great for relational
storage !
www.xebia.fr / blog.xebia.fr 8
RDBMS may not be fully trivial…
…what a pity if you just wanted
a put / get store
www.xebia.fr / blog.xebia.fr 9
How to achievescalability and
high availabilitywith a RDMS ?
www.xebia.fr / blog.xebia.fr 10
Memcached can help to reduce the
pressure on RDBMS
www.xebia.fr / blog.xebia.fr 11
But this wasn’t enough for some
people…
www.xebia.fr / blog.xebia.fr 12
The birth of Dynamo at
Amazon.com…
Fill cart Checkout Payment Process order Prepare Send
Write must ALWAYS be available Only key-value is required
Temporal unavailability is acceptableRich storage is preferred for reporting
www.xebia.fr / blog.xebia.fr 13
So they started to create their own
database that would match their
needs… Table de hachage
persistante
www.xebia.fr / blog.xebia.fr 14
So they started to create their own
database that would match their
needs…
Table de hachage
persistante
www.xebia.fr / blog.xebia.fr 15
Consistent hashing spreads keys
uniformally across the partitions
www.xebia.fr / blog.xebia.fr 16
Sharding brings scalability,
replication brings high availability
www.xebia.fr / blog.xebia.fr 17
But failures still have to be handled
efficiently…
www.xebia.fr / blog.xebia.fr 18
In case of failure of a node, the 2 other nodes serve its load
www.xebia.fr / blog.xebia.fr 19
In this configuration,
clients know only few nodes which
act as proxy to other nodes…
www.xebia.fr / blog.xebia.fr 20
Gossip protocol to let everybody know node B has failed…
www.xebia.fr / blog.xebia.fr 21
Eventual consistency…
N = Number of replicas
R = Number of read responses to wait for
W = Number of write aknowledges to wait for
www.xebia.fr / blog.xebia.fr 22
The ratio between N, R and W makes
everything…
www.xebia.fr / blog.xebia.fr 23
The ratio between N, R and W makes
everything…
www.xebia.fr / blog.xebia.fr 24
The ratio between N, R and W makes
everything…
www.xebia.fr / blog.xebia.fr 25
www.xebia.fr / blog.xebia.fr 26
Key-value storesare simple…
www.xebia.fr / blog.xebia.fr 27
Several competitors…
www.xebia.fr / blog.xebia.fr 28
Riak…
www.xebia.fr / blog.xebia.fr 29
Riak…
www.xebia.fr / blog.xebia.fr 30
Document oriented databases are an evolution of key-
value stores
www.xebia.fr / blog.xebia.fr 31
MongoDB is the leader so far…
… 10gen said they are the « MySQL of
NoSQL »
www.xebia.fr / blog.xebia.fr 32
MongoDB is the leader so far…
www.xebia.fr / blog.xebia.fr 33
MongoDB is the leader so far…
www.xebia.fr / blog.xebia.fr 34
Graph databases are great to store…
graphs !
www.xebia.fr / blog.xebia.fr 35
Neo Technology is the leader so far
with Neo4j…
www.xebia.fr / blog.xebia.fr 36
Neo4j…
www.xebia.fr / blog.xebia.fr 37
Column-oriented databases have a
very particular data model…
www.xebia.fr / blog.xebia.fr 38
CassandraVs
HBase !
www.xebia.fr / blog.xebia.fr 39
Cassandra adds SuperColumns to
this model…
www.xebia.fr / blog.xebia.fr 40
Cassandra was developped at
Facebook, to store the user messages
www.xebia.fr / blog.xebia.fr 41
Now also used by Digg, Twitter, Rackspace,…
www.xebia.fr / blog.xebia.fr
What a model…
www.xebia.fr / blog.xebia.fr 43
An example…
www.xebia.fr / blog.xebia.fr 44
Cassandra Demo