MongoDB @ Frankfurt NoSql User Group
-
Upload
chris-harris -
Category
Technology
-
view
809 -
download
5
description
Transcript of MongoDB @ Frankfurt NoSql User Group
Chris Harris Email : [email protected]
Twitter : cj_harris5
Frankfurt NoSQL User Group
Wednesday, 22 February 12
As simple as possible, but no simpler
Depth of functionality
Sca
labi
lity
& P
erfo
rman
ce
MemcachedKey / Value
RDBMS
Wednesday, 22 February 12
Terminology
RDBMS MongoDBTable CollectionRow(s) JSON DocumentIndex IndexJoin Embedding & LinkingPartition ShardPartition Key Shard Key
Wednesday, 22 February 12
Here is a “simple” SQL Modelmysql> select * from book;+----+----------------------------------------------------------+| id | title |+----+----------------------------------------------------------+| 1 | The Demon-Haunted World: Science as a Candle in the Dark || 2 | Cosmos || 3 | Programming in Scala |+----+----------------------------------------------------------+3 rows in set (0.00 sec)
mysql> select * from bookauthor;+---------+-----------+| book_id | author_id |+---------+-----------+| 1 | 1 || 2 | 1 || 3 | 2 || 3 | 3 || 3 | 4 |+---------+-----------+5 rows in set (0.00 sec)
mysql> select * from author;+----+-----------+------------+-------------+-------------+---------------+| id | last_name | first_name | middle_name | nationality | year_of_birth |+----+-----------+------------+-------------+-------------+---------------+| 1 | Sagan | Carl | Edward | NULL | 1934 || 2 | Odersky | Martin | NULL | DE | 1958 || 3 | Spoon | Lex | NULL | NULL | NULL || 4 | Venners | Bill | NULL | NULL | NULL |+----+-----------+------------+-------------+-------------+---------------+4 rows in set (0.00 sec)
Wednesday, 22 February 12
Flexible “Schemas”
{ “author”: “brendan”, “text”: “...” }
{ “author”: “brendan”, “text”: “...”, “tags”: [“mongodb”, “nosql”] }
Wednesday, 22 February 12
The Same Data in MongoDB
{ "_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"), "title" : "Programming in Scala", "author" : [ { "first_name" : "Martin", "last_name" : "Odersky", "nationality" : "DE", "year_of_birth" : 1958 }, { "first_name" : "Lex", "last_name" : "Spoon" }, { "first_name" : "Bill", "last_name" : "Venners" } ]}
Wednesday, 22 February 12
Cursors
> var c = db.test.find({x: 20}).skip(20).limit(10)> c.next()> c.next()...
$gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $or,$not, $mod, $size, $exists, $type, $elemMatch
query
first N results + cursor id
getMore w/ cursor idnext N results + cursor id or 0
...
Wednesday, 22 February 12
db.blogs.ensureIndex({author: 1})
1 = ascending-1 = descending
An index on _id is automatic.
For more use ensureIndex:
Creating Indexes
Wednesday, 22 February 12
Compound Indexes
db.blogs.save({ author: "James", ts: new Date() ...});
db.blogs.ensureIndex({author: 1, ts: -1})
Wednesday, 22 February 12
db.blogs.save({ author: "James", title: "My first blog" ...});
db.blogs.ensureIndex({title: 1}, {unique: true})
Unique Indexes
Wednesday, 22 February 12
db.blogs.save({ title: "My First blog", stats : { views: 0, followers: 0 }});
db.blogs.ensureIndex({"stats.followers": -1})
db.blogs.find({"stats.followers": {$gt: 500}})
Indexing Embedded Documents
Wednesday, 22 February 12
db.blogs.save({ title: "My First blog", comments: [ {author: "James", ts : new Date()} ]});
db.blogs.ensureIndex({"comments.author": 1})
db.blogs.find({"comments.author": "James"})
Indexing Embedded Arrays
Wednesday, 22 February 12
db.blogs.save({ title: "My Second blog", tags: ["mongodb", "NoSQL"]});
db.blogs.ensureIndex({tags: 1})
db.blogs.find({tags: "NoSQL"})
Multikeys
Wednesday, 22 February 12
• From 1.8.0• Query resolved in index only• Need to exclude _id from items projected
db.blogs.save({ author: "James", title: "My first blog"});
db.blogs.ensureIndex({author: 1})
db.blogs.find({author: "James"}, {author: 1, _id:0}))
Covered Indexes
Wednesday, 22 February 12
• From 1.8.0• Key value included if and only if the value is present• Reduces size of index• Limited to a single field
db.blogs.ensureIndex({loc: 1}, {sparse: true})
// loc key stored for Ben & Sarah onlydb.blogs.save({author: "Jim"})db.blogs.save({author: "Ben", loc: null})db.blogs.save({author: "Sarah", loc: "CA"})
Sparse Indexes
Wednesday, 22 February 12
db.blogs.save({ loc: { long: 40.739037, lat: 40.739037 }});
db.blogs.save({ loc: [40.739037, 40.739037]});
db.blogs.ensureIndex({"loc": "2d"})
Geospatial• Geo hash stored in B-Tree• First two values indexed
Wednesday, 22 February 12
Types of outage
• Planned• Hardware upgrade• O/S or file-system tuning• Relocation of data to new file-system / storage• Software upgrade
• Unplanned• Hardware failure• Data center failure• Region outage• Human error• Application corruption
Wednesday, 22 February 12
• A cluster of N servers• Any (one) node can be primary• Consensus election of primary• Automatic failover• Automatic recovery• All writes to primary• Reads can be to primary (default) or a secondary
Replica Set features
Wednesday, 22 February 12
How MongoDB Replication works
Member 1
Member 2
Member 3
•Set is made up of 2 or more nodes
Wednesday, 22 February 12
How MongoDB Replication works
Member 1
Member 2
PRIMARY
Member 3
•Election establishes the PRIMARY•Data replication from PRIMARY to SECONDARY
Wednesday, 22 February 12
How MongoDB Replication works
Member 1
Member 2
DOWN
Member 3
negotiate new master
•PRIMARY may fail•Automatic election of new PRIMARY if majority exists
Wednesday, 22 February 12
How MongoDB Replication works
Member 1
Member 2
DOWN
Member 3
PRIMARY
•New PRIMARY elected•Replication Set re-established
Wednesday, 22 February 12
How MongoDB Replication works
Member 1
Member 2RECOVERING
Member 3
PRIMARY
•Automatic recovery
Wednesday, 22 February 12
How MongoDB Replication works
Member 1
Member 2
Member 3
PRIMARY
•Replication Set re-established
Wednesday, 22 February 12
http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg
Wednesday, 22 February 12
http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg
Wednesday, 22 February 12
Write scaling - add Shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
Wednesday, 22 February 12
Write scaling - add Shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
shard3
node_c3
node_b3
node_a3
Wednesday, 22 February 12
MongoDB Sharding
• Automatic partitioning and management
• Range based
• Convert to sharded system with no downtime
• Fully consistent
Wednesday, 22 February 12
How MongoDB Sharding works
> db.posts.save( {age:40} )
-∞ +∞
-∞ 40 41 +∞
•Data in inserted•Ranges are split into more “chunks”
Wednesday, 22 February 12
How MongoDB Sharding works
> db.posts.save( {age:40} )> db.posts.save( {age:50} )
-∞ +∞
-∞ 40 41 +∞
41 50 51 +∞
•More Data in inserted•Ranges are split into more“chunks”
Wednesday, 22 February 12
How MongoDB Sharding works
> db.posts.save( {age:40} )> db.posts.save( {age:50} )> db.posts.save( {age:60} )
-∞ +∞
-∞ 40 41 +∞
41 50 51 +∞
61 +∞ 51 60
Wednesday, 22 February 12
Balancing
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
17
21
13
18
22
14
19
23
15
20
24
16
29
33
25
30
34
26
31
35
27
32
36
28
41
45
37
42
46
38
43
47
39
44
48
40
mongos
balancerconfig
config
config
Chunks!
Wednesday, 22 February 12
Balancingmongos
balancerconfig
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
ImbalanceImbalance
Wednesday, 22 February 12
Balancingmongos
balancer
Move chunk 1 to Shard 2
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
Wednesday, 22 February 12
Balancingmongos
balancerconfig
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
1
Wednesday, 22 February 12
Balancingmongos
balancerconfig
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
1
Wednesday, 22 February 12