MongoDB dessi-codemotion

Massimiliano Dessì – [email protected] – Jug Sardegna


Speaker

Software architect/developer Pronetics/Sourcesense

Founder Spring Italian User Group

Chairman JugSardegna

Committer/Contributor OpenNMS - MongoDB

Author Spring 2.5 Aspect Oriented Programming

http://www.packtpub.com/aspect-oriented-programming-with-spring-2-5/book


MongoDB Philosophy


Some MongoDB production deployments

http://www.mongodb.org/display/DOCS/Production+Deployments

http://www.mongodb.org/display/DOCS/Production+Deployments


Main Features● Document Oriented

- Documents (objects) map nicely to programming language data types

- Dynamically-typed (schemaless) for easy schema evolution

- No joins and no transactions for high performance

and easy scalability

● High Performance

- No joins and no transactions makes reads and writes fast

- Indexes with indexing into embedded documents and arrays

- Optional asynchronous writes

● Rich Query Language

● Easy scalability

- "slaveOK" reads are distributed over replicated servers

- Automatic sharding (auto-partitioning of data across servers)

- Reads and writes are distributed over shards

- No joins and no transactions make distributed queries easy and fast

● High Availability

- Replicated servers with automatic master failover

● Indexing

● Stored JavaScript

● Fixed-size collection

● File storage

● MapReduce


No Sql Injection

Mongo is invulnerable to injection attacks, no code execution


Document as Basic unit of Data in BSON (Binary JSON) format{ "name" : "MongoDB",

"info" : { "storage" : "Binary JSON (BSON)",

"full index" : "true",

"scale" : "Autosharding",

"query" : "Rich document-base queries",

"replication" : "Replica sets",

"atomic modifiers" : "Fast in place update",

"binary content" : "GridFS",

"batch operation" : "Map/Reduce”,

"js server side" : ”true”

}

"greeting": {"international" : "Hello, world!", "italy" :"Ciao Mondo !" }

}, "_id” : "024x6f279578a64bb0666945"

Document: an Ordered set of keys with associated values


Grouping

SQL Table contains Rows

MONGO Collection and subcollections contains Documents

* Document Limit: Larger than 4 Mb, the entire text of War and Peace is 3.14Mb

Collection are are created dynamically and automatically grow in size to fit additional data


A single instance of MongoDB can host

multiple independent databases, each of

which can have its own collections

and permissions.

Photo from http://www.aibento.net/


No Join's cost

● SQL SELECT * FROM posts

INNER JOIN posts_tags ON posts.id = posts_tags.post_id

INNER JOIN tags ON posts_tags.tag_id == tags.id

WHERE tags.text = 'politics' AND posts.vote_count > 10;

● MONGO db.posts.find({'tags': 'politics', 'vote_count': {'$gt': 10}});


Collections Schema free

Documents within a single collection can have any number of different "shapes”

In theory, each document in a collection

can have a completely different structure;

in practice, a collection's documents

will be relatively uniform.


Driver

C, C#, C++, Clojure, D, Delphi, Erlang,

Factor, Fantom, F#, Go, Groovy,

Haskell, Java, Javascript, Lua, Nodejs,

ObjectiveC, Perl, PHP, Python, R,

Ruby, Scala, Scheme (PLT), Smalltalk http://www.mongodb.org/display/DOCS/Drivers

http://www.mongodb.org/display/DOCS/Drivers


Common operationsFastest : Fire and Forget

Command with response : getLastError

Examples with

Server side Javascript via mongo shell

Java via MongoDB Official 10gen Driver

Scala via Casbah Official 10gen scala driver + Salat serializer


Inserting

//mongo shell

db.dcComicsCollection.insert({"name" : "bruce", "surname" : "wayne", "alias" : "batman"})

//Java

Map fields = batman.toMap()

dcComicsCollection.insert(BasicDBObjectBuilder.start(fields).get()) //java driver

//scala

dcComicsCollection.insert(grater[BlogPost].asDBObject(post))


Removing

//mongo shell

db.dcComicsCollection.delete({"alias" : "superman"})

//java

Map query = batman.getMap()

DBObject obj = BasicDBObjectBuilder.start(fields).get()

dcComicsColl.remove(obj);

//scala

dcComicsCollection.remove(grater[BlogPost].asDBObject(post)


Updating (The schema can be changed dinamically)//mongo shell

var hero = db.dcComicsColl.findOne({"alias" : "batman"});

hero.gadget = {"car" : "batmobile”};

db.dcComicsColl.update({”aias” : ”batman"}, hero, true);

//java

DbObject query = BasicDBObjectBuilder.start().add("surname",”wayne”).get();

DbObject hero = BasicDBObjectBuilder.start().add("gadget",”batmobile”).get();

dcComicsCollection.update(query, hero, false, true);

//scala

val query = MongoDBObject("name" -> "bruce")

val hero = MongoDBObject("gadget" -> "batmobile")

dcComicsCollection.update(query, hero, true, false)

* The blue value it's the upsert, update or insert if not present


Querying//mongo shell

db.dcComicsColl.find({"alias":"batman"}) // all field of the doc

db.dcComicsColl.find({"alias":"batman"},{”surname":1})//surname desc

//java

DBObject query = BasicDBObjectBuilder.start().add("alias", "batman").get();

DBObject out = BasicDBObjectBuilder.start().add("surname", "1").get()

DBCursor cursor = dcComicsColl.find(query, out);

//scala

val query = MongoDBObject("alias" -> "batman")

val obj = dcComicsColl.findOne(query)


Modifier

Partial updates$set (set a key, or add if not present)

$inc (with numbers)

$push (add to the end of an array)

$ne

$addToSet

$each

$pop (remove from the end)

$pull (remove element that match criteria)


Query Criteria

Conditionals$lt $lte $gt $gte

start = new Date("01/01/20111")

db.mycollection.find({"registered" : {"$lt" : start}})

db.dcComicsColl.find({

"surname":{"$ne":"parker"},

"name":{"$ne":"peter"}

})


Query Criteria

Conditionals

$in $or, $nin, $not <, <=, >, >=, $ne, $mod, $all, $size, $exists

db.attendees.find({”external_num" : {"$nin" : [1356, 525, 874]}})

db.attendees.find({"$or" : [{"ticket_no" : 513}, {"name" : foo}]})


Query Regex

Perl Compatible Regular Expression //mongo shell

db.mycollection.find({"name" : /foo?/i})

//java

String pattern = new StringBuilder().append(character).append("*").toString();

DbObject query =BasicDBObjectBuilder.start()

.add("surname", java.util.regex.Pattern.compile(pattern)).get()

List<DBObject> objects = coll.find(query).toArray();

//scala

coll.find(MongoDBObject("surname" -> ".*yne$".r))) {...


Query on array and grouping

Array inside a document $all $size $slice

Grouping count distinct group finalize $key


Javascript as part of a query

db.mycollection.find({

"$where" : function (){

for (var current in this) {

….

}

}

})


Indexing

db.mycollection.ensureIndex({"name" : 1, "info" : 1 }

Geospatial indexing db.mycollection.ensureIndex({"gps" : "2d"})

db.star.trek.ensureIndex( {"light-years" : "2d"}, {"min”:-10, "max”:10})

Yes it's a Foursquare feature :)


GridFS

GridFS is a specification for storing large files such video, photos, blob in MongoDB .

GridFS uses two collections to store data:● files contains the object metadata ● chunks contains the binary chunks with some additional accounting

information


Map Reducemap = function() {

for (var key in this) {

emit(key, {count : 1});

};

}

reduce = function(key, emits) {

total = 0;

for (var i in emits) {

total += emits[i].count;

}

return {"count" : total};

}


Scaling


Shard

Sharding = break up collections into smaller chunks

Splitting data and storing different portions of the data on different machines, also

know as partitioning

The chunks can be distributed across shards so that each shard is responsible for a subset of the total data set.

A shard is a container that holds a subset of a collection’s data. A shard is either a single mongod server (for development/testing) or a replica set (for production). Thus, even if there are many servers in a shard, there is only one master, and all of the servers contain the same data.


Mongos

The client don't know what shard has what data, or even that data is broken up on different shards. In front of the shard run routing process called mongos.

This router know when the data are located and the client can see a normal mongod, like a noshard environment.

This is the router process and comes with all MongoDB distributions. It basically just routes requests and aggregates responses. It doesn’t store any data or configuration information. (Although it does cache information from the config servers.)


Config server

Config servers store the configuration of the cluster: which data is on which shard. Because mongos doesn’t store anything permanently, it needs somewhere to get the shard configuration. It syncs this data from the config servers.


Replication

Replica set, clustering with automatic failover

Master is elected by the cluster and may change to another node if the current master goes down.

This election process will be initiated by any node that cannot reach the primary.

The highest-priority most-up-to-date server will become the new primary.

The replication is asynchronous


Scala Webapp Demo


Admin Interface


Mongo Shell


Mac Client MongoHub


Books

http://www.mongodb.org/display/DOCS/Books

http://www.mongodb.org/display/DOCS/Books


References

● MongoDB : http://www.mongodb.org/● Casbah: https://github.com/mongodb/casbah● Salat: https://github.com/novus/salat/● Slides scala code example: https://github.com/desmax74/codemotion-2010

http://www.mongodb.org/

https://github.com/mongodb/casbah

https://github.com/novus/salat/

https://github.com/desmax74/codemotion-2010


Thanks !

Massimiliano Dessì

http://jroller.com/desmax

http://twitter.com/desmax74

http://www.linkedin.com/in/desmax74

http://wiki.java.net/bin/view/People/MassimilianoDessi

http://www.jugsardegna.org/vqwiki/jsp/Wiki?MassimilianoDessi

http://twitter.com/desmax74

http://www.linkedin.com/in/desmax74

http://wiki.java.net/bin/view/People/MassimilianoDessi

http://www.jugsardegna.org/vqwiki/jsp/Wiki?MassimilianoDessi

MongoDB dessi-codemotion

Technology

Transcript of MongoDB dessi-codemotion