Introduction to MongoDB
75
Mark Hillick Email : [email protected] Twitter : @markofu MongoDB Introduction Wednesday 31 October 12
-
Upload
mark-hillick -
Category
Technology
-
view
491 -
download
2
description
Introduction
Transcript of Introduction to MongoDB
Mark HillickEmail : [email protected]
Twitter : @markofu
MongoDB Introduction
Wednesday 31 October 12
Apologies
Source: http://blog.stackoverflow.com/wp-content/uploads/jarrod-lair-1.jpg
Source: http://abcnews.go.com/blogs/headlines/2012/10/hurricane-sandy-live-updates/
mark != dev
= no prep :(
Wednesday 31 October 12
NoSQL
Wednesday 31 October 12
What?
• A collection of very different products
• Not relational
• No joins
• Currently not using SQL
• Allows much more flexible data structures than RDBMS
Wednesday 31 October 12
10gen & MongoDB
Wednesday 31 October 12
Who? What?
• 10gen created MongoDB in 2007
• Founded by Eliot Horowitz and Dwight Merriman
• Inspired by work on DoubleClick, ShopWiki and Panther Express
• Over 170 people
• Funded by: Sequoia, Flybridge Capital, Union Square Ventures & New Enterprise Associates
• Offices in New York (HQ), Palo Alto, London, Dublin and Sydney
Wednesday 31 October 12
When?
• Coding started fall 2007
• First production site March 2008 - businessinsider.com
• Open Source - AGPL
• Version 0.8 – first official release February 2009
• Version 1.0 – August 2009
• Version 2.0 – September 2011
• Version 2.2 - September 2012
Wednesday 31 October 12
Terminology
RDBMS MongoDB
Table Collection
Row(s) JSON Document
Index Index
Join Embedding & Linking
Partition Shard
Partition Key Shard Key
Wednesday 31 October 12
Data Snippet
Wednesday 31 October 12
Here is a “simple” SQL Modelmysql> select * from book;+----+----------------------------------------------------------+| id | title |+----+----------------------------------------------------------+| 1 | The Demon-Haunted World: Science as a Candle in the Dark || 2 | Cosmos || 3 | Programming in Scala |+----+----------------------------------------------------------+3 rows in set (0.00 sec)
mysql> select * from bookauthor;+---------+-----------+| book_id | author_id |+---------+-----------+| 1 | 1 || 2 | 1 || 3 | 2 || 3 | 3 || 3 | 4 |+---------+-----------+5 rows in set (0.00 sec)
mysql> select * from author;+----+-----------+------------+-------------+-------------+---------------+| id | last_name | first_name | middle_name | nationality | year_of_birth |+----+-----------+------------+-------------+-------------+---------------+| 1 | Sagan | Carl | Edward | NULL | 1934 || 2 | Odersky | Martin | NULL | DE | 1958 || 3 | Spoon | Lex | NULL | NULL | NULL || 4 | Venners | Bill | NULL | NULL | NULL |+----+-----------+------------+-------------+-------------+---------------+4 rows in set (0.00 sec)
Wednesday 31 October 12
The Same Data in MongoDB
{ "_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"), "title" : "Programming in Scala", "author" : [ { "first_name" : "Martin", "last_name" : "Odersky", "nationality" : "DE", "year_of_birth" : 1958 }, { "first_name" : "Lex", "last_name" : "Spoon" }, { "first_name" : "Bill", "last_name" : "Venners" } ]}
Wednesday 31 October 12
CRUD Operations
Wednesday 31 October 12
Insert
• You should see something like:
“Inserted 1 record(s) in 2ms”
• db.test.insert({name: “mark”, surname: “hillick”, weight: 170})
Wednesday 31 October 12
Find
• db.test.find({name: “mark”})
• You should see something like:
“{ "_id": ObjectId("509108ff707a9dd29312054e"), "name": "mark", "surname": "mark", "weight": 170 }”
Wednesday 31 October 12
Update
• db.name.update({name: 'mark'}, {$set: {weight: 169}})
• You should see something like:
“Updated 1 existing record(s) in 0ms”
Wednesday 31 October 12
Delete
• You should see something like:
“Removed 1 record(s) in 0ms”
• db.name.remove({name: “mark”})
Wednesday 31 October 12
Installation
Wednesday 31 October 12
Where to Download
• Binaries: http://www.mongodb.org/downloads
• Subscription: http://ww.mongodb.org/mongodb-subscriber-edition-download
• SSL & SNMP
• Source: https://github.com/mongodb/mongo
• Packages: RH-type & Ubuntu/Debian
Wednesday 31 October 12
What now?
• How To Install: http://docs.mongodb.org/manual/installation/
• Release Notes: http://docs.mongodb.org/manual/release-notes/
Wednesday 31 October 12
Other Documentation
• Binaries - http://www.mongodb.org/downloads
• New: http://docs.mongodb.org/manual/
• Tutorials• Descriptions• Configurations• Security (personal favourite)
• Old: http://www.mongodb.org/display/DOCS/Home
Wednesday 31 October 12
Replication
Wednesday 31 October 12
Types of outage
• Resilience
• Reliability
• Protect against planned & unplanned outages
Wednesday 31 October 12
How MongoDB Replication works
Member 1
Member 2
Member 3
•Set is made up of 2 or more nodes
Wednesday 31 October 12
How MongoDB Replication works
Member 1
Member 2PRIMARY
Member 3
•Election establishes the PRIMARY•Data replication from PRIMARY to SECONDARY
Wednesday 31 October 12
How MongoDB Replication works
Member 1
Member 2DOWN
Member 3
negotiate new master
•PRIMARY may fail•Automatic election of new PRIMARY if majority exists
Wednesday 31 October 12
How MongoDB Replication works
Member 1
Member 2DOWN
Member 3PRIMARY
•New PRIMARY elected•Replication Set re-established
Wednesday 31 October 12
How MongoDB Replication works
Member 1
Member 2RECOVERING
Member 3PRIMARY
•Automatic recovery
Wednesday 31 October 12
How MongoDB Replication works
Member 1
Member 2
Member 3PRIMARY
•Replication Set re-established
Wednesday 31 October 12
Sharding
Wednesday 31 October 12
http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg
Wednesday 31 October 12
http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg
Wednesday 31 October 12
MongoDB Scaling - Single Node
write
read
node_a1
Wednesday 31 October 12
Write scaling - add Shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
Wednesday 31 October 12
Write scaling - add Shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
shard3
node_c3
node_b3
node_a3
Wednesday 31 October 12
MongoDB Sharding
• Automatic partitioning and management
• Range based
• Convert to sharded system with no downtime
• Fully consistent
Wednesday 31 October 12
How MongoDB Sharding works
> db.posts.save( {age:40} )
-∞ +∞
-∞ 40 41 +∞
•Data in inserted•Ranges are split into more “chunks”
Wednesday 31 October 12
How MongoDB Sharding works
> db.posts.save( {age:40} )> db.posts.save( {age:50} )> db.posts.save( {age:60} )
-∞ +∞
-∞ 40 41 +∞
41 50 51 +∞
61 +∞ 51 60
Wednesday 31 October 12
Split Example
Wednesday 31 October 12
{ name: “Jared”, email: “[email protected]”,}{ name: “Scott”, email: “[email protected]”,}{
> db.runCommand( { shardcollection: “test.users”, key: { email: 1 }} )
Wednesday 31 October 12
Chunks 1
-∞ +∞
Wednesday 31 October 12
-∞ +∞
Split!This is a chunk
This is a chunk
Chunks 4
Wednesday 31 October 12
Balancing Example
Wednesday 31 October 12
Balancing 1
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
17
21
13
18
22
14
19
23
15
20
24
16
29
33
25
30
34
26
31
35
27
32
36
28
41
45
37
42
46
38
43
47
39
44
48
40
mongos
balancerconfig
config
config
Chunks!
Wednesday 31 October 12
Balancing 2mongos
balancerconfig
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
ImbalanceImbalance
Wednesday 31 October 12
Balancing 3mongos
balancer
Move chunk 1 to Shard 2
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
Wednesday 31 October 12
Balancing 4mongos
balancerconfig
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
1
Wednesday 31 October 12
Balancing 4mongos
balancerconfig
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
6
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
1
Wednesday 31 October 12
Balancing 5mongos
balancer
Chunk 1 now lives on Shard 2
config
config
config
Shard 1 Shard 2 Shard 3 Shard 4
5
9
16
10
2
7
11
3
8
12
4
21 22 23 24 33 34 35 36 45 46 47 48
Wednesday 31 October 12
Drivers
Wednesday 31 October 12
Documentation & Features
• Doc: http://www.mongodb.org/display/DOCS/Drivers
• Official
• C, C++, Java, PHP, Python, Ruby, Scala & more
• Community
Wednesday 31 October 12
Operators & Indexes
Wednesday 31 October 12
Operators
> var c = db.test.find({x: 20}).skip(20).limit(10)> c.next()> c.next()...
$gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $or,$not, $mod, $size, $exists, $type, $elemMatch
Wednesday 31 October 12
db.blogs.ensureIndex({author: 1})
1 = ascending-‐1 = descending
An index on _id is automatic.
For more use ensureIndex:
Creating Indexes
Wednesday 31 October 12
Compound Indexes
db.blogs.save({ author: "James", ts: new Date() ...});
db.blogs.ensureIndex({author: 1, ts: -‐1})
Wednesday 31 October 12
db.blogs.save({ title: "My First blog", comments: [ {author: "James", ts : new Date()} ]});
db.blogs.ensureIndex({"comments.author": 1})
db.blogs.find({"comments.author": "James"})
Indexing Embedded Arrays
Wednesday 31 October 12
db.blogs.save({ loc: { long: 40.739037, lat: 40.739037 }});
db.blogs.save({ loc: [40.739037, 40.739037]});
db.blogs.ensureIndex({"loc": "2d"})
Geospatial• Geo hash stored in B-Tree• First two values indexed
Wednesday 31 October 12
Security
Wednesday 31 October 12
Documentation & Features
• New: http://docs.mongodb.org/manual/security/
• Linux• Windows• AMI• Authentication• SSL
• NoSQL solutions behind traditional RDBMs
Wednesday 31 October 12
Future
• Role-based Authentication
• Kerberos
• LDAP (external authorisation)
Wednesday 31 October 12
Roadmap
Wednesday 31 October 12
2.2 Features
• Aggregation Framework (no need for map-reduce)•m/r single thread @ present
• TTL Collections
• Concurrency Improvements
• Tag Aware Sharding
• Fully Supported Read Semantics
• Improved Authentication
Wednesday 31 October 12
Future
• 2.4
• Security, security, security
• Faster Aggregation
• Hash-based shard keys
• More ops
Wednesday 31 October 12
Get Involved
Wednesday 31 October 12
Where
• Open-Source Community
• Contribute - tell us your thoughts!!!
• Support provided By 10gen & Community on Google Groups [ groups.google.com/group/mongodb-user]
• jira.mongodb.org: features, commercial support, security, bugs, drivers etc
Wednesday 31 October 12
Education
Wednesday 31 October 12
https://education.10gen.com/
Learn for Free :)
Wednesday 31 October 12
Meetup Groups
Wednesday 31 October 12
MUGs
• Right Here in Dublin: http://mongodb.meetup.com/cities/ie/dublin/ :)
• Complete List: http://www.mongodb.org/display/DOCS/MongoDB+User+Groups+(MUGs)
Wednesday 31 October 12
Dublin
• Next MeetUp: Thursday, December 6th @ EngineYard
• Poll: http://www.meetup.com/DublinMUG/polls/
Source: http://www.feedhenry.com Source: http://www.iamstaggered.com/wp-content/uploads/2011/03/dublin.jpg
Wednesday 31 October 12
Finally
Wednesday 31 October 12
Thanks
Source: http://smallbiztrends.com/wp-content/uploads/2010/01/question-things.jpg
Wednesday 31 October 12
