MongoDB

Telefónica I+D

Lugar y Fecha

Telefónica I+D 2

Mongodb is...

A document-oriented storage.JSON-style documents with dynamic schemas offer simplicity and power.

Querying.Rich, document-base queries.

Fast In-Place Updates.Atomic modifiers for contention-free performance.

Full index Support.Index on any attribute.

Auto-Sharding.Scale horizontally without compromising funcionality.

Replication & High Availability.Mirror across LAN for scale.

Map/ReduceFlexible (but limited) data processing.

Telefónica I+D 3

Mongodb is Not...

A ACID database.

A batch procesing platform.

Telefónica I+D 4

Mongodb Architecture

MongoDB is a document-oriented database, but this not implica that it has the same concepts of RDMS, one of this concepts is the “database”, within a MongoDB instance you can have zero or more databases, each acting as high-level containers for everything else

A database can have zero or more “collections”, A collection shares enough in common with a traditional table, a collection is the same as a table with a indeterminate number of columns.

Collections is made up of zero or more documents. Again, a document can safely be thought of as a “row”.

A document is made up one or more fields, which yo can probable guess are a lot like “columns”.

Indexes in MongoDB function much like their RDBMS counterparts.

Cursors are different than the other five concepts but they are important enough. The import thing to understand about cursors is that when you ask MongoDB for data, it returns a cursor, which we can do things to, such as counting or skipping ahead, without actually pulling down data.

Telefónica I+D 5

Mongodb Architecture

Mongo uses memory mapped files to access data, which results in large numbers being displayed in tools like top for the mongod process.

Mongo eliminates the need (in some cases) for a separate object caching layer. Queries that result in file system RAM cache hits are very fast as the object's representation in the database is very close to its representation in application memory. Also, the MongoDB can scale to any level and provides an object cache and database integrated together, which is very helpful as there is no risk of retrieving stale data from the cache. In addition, the complex queries a full DBMS provides are also possible.

MongoDB supports write-ahead journaling of operations to facilitate fast crash recovery and durability in the storage

Telefónica I+D 6

Mongodb Api

Data Type Type Number

Double 1

String 2

Object 3

Array 4

Binary data 5

Object id 7

Boolean 8

Date 9

Null 10

Regular Expresion 11

Telefónica I+D 7

Mongodb Api

The conection to the databases front the aplications are a propiatery protocol that runs over TCP/IP in binary format. The apis have a control the write behavior for with various options, as well as exception raising on error conditions. NONE: No exceptions are raised, even for network issues

NORMAL: Exceptions are raised for network issues, but not server errors

SAFE: Exceptions are raised for network issues, and server errors; waits on a server for the write operation

FSYNC_SAFE: Exceptions are raised for network issues, and server errors and the write operation waits for the server to flush the data to disk

REPLICAS_SAFE: Exceptions are raised for network issues, and server errors; waits for at least 2 servers for the write operation

Telefónica I+D 8

Mongodb Api

Insert: it is the basic method for adding data into MongoDB. When we insert a new document into a collection this will add an “_id” key to the document before save it. When insert a new object into the collection, the driver coverts the data structure into BSON, which it then sends to the database. Then it saves the document, and regenerate the _id index, and all the secondary index that the new object has.

db.collection.insert(doc);

db.collection.save(doc); // updates if exists; inserts if new (_id)

Delete: it is the basic method for deleting data. db.videos.remove( { rating : { $lt : 3.0 }, $atomic : true } )

Telefónica I+D 9

Mongodb Api

Query: MongoDB's support for dynamic (ad hoc) queries. MongoDB supports a number of query objects for fetching data.

Db.collection.find(query,fields)

Method: count()

sort()

limit()

skip()

Telefónica I+D 10

Mongodb Api

Query Modifier Operations

$gt (>) , $gte(>=), $lt (<), $gle(<=) db.collection.find({ "field" : { $gt: value } } );

$all db.things.find( { a: { $all: [ 2, 3 ] } } );

$exists db.things.find( { a : { $exists : true } } ); present

$mod db.things.find( { a : { $mod : [ 10 , 1 ] } } )

$ne db.things.find( { x : { $ne : 3 } } );

$in db.things.find({j:{$in: [2,4,6]}});

$nin db.things.find({j:{$nin: [2,4,6]}});

$or db.foo.find( { $or : [ { a : 1 } , { b : 2 } ] } )

$size db.things.find( { a : { $size: 1 } } );

$type db.things.find( { a : { $type : 2 } } );

$elemMatch t.find( { x : { $elemMatch : { a : 1}})

$regex db.customers.find( { name : { $regex : 'acme.*corp', $options: 'i' } } );

$where db.myCollection.find( { $where: "this.a > 3" } );

Telefónica I+D 11

Mongodb Api

Update: replaces the document matching criteria entirely with objNew. If you only want to modify some fields, you should use the atomic modifiers below.

db.collection.update( criteria, objNew, upsert, multi ).

Arguments criteria - query which selects the record to update;

objNew - updated object or $ operators (e.g., $inc) which manipulate the object

upsert - if this should be an "upsert"; that is, if the record does not exist, insert it

multi - if all documents matching criteria should be updated

Telefónica I+D 12

Mongodb Api

Update Modifier Operations$inc { $inc : { field : value } }

$set { $set : { field : value } }

$unset { $unset : { field : 1} }

$push { $push : { field : value } }

$pushAll { $pushAll : { field : value_array } }

$addToSet { $addToSet : { field : value } }

$pop { $pop : { field : 1 } }{ $pop : { field : -1 } }

$pull { $pull : { field : _value } }

$pullAll { $pullAll : { field : value_array } }

$rename { $rename : { old_field_name : new_field_name } }

$bit {$bit : {field : {and : 5, or : 2}}}

$ ( positional number) {$inc:{'comments.$.votes':1}}

Telefónica I+D 13

Mongodb Api

Upserts : it is a special type of update, in where if no document is found to upate, a new document will be created in a atomic operation.

db.jobs.findAndModify({query: {},update : {}, fields: {}, new: false } )

Arguments

Query {}

Sort {}

Remove N/A

Update N/A

New False

Fields All fields

upsert False

Telefónica I+D 14

Mongodb Api

Example in console.

Telefónica I+D 15

Mongodb Index

These indexes are implemented as "B-Tree" indexes.

the indexes in MongoDB work a lot like indexes in a relational database: they help improve query and sorting performance, the indexes are created manual, and could be unique, ascending or descending.

Index on _id is always created and can`t be deleted.

I can index on a key inside of an embedded document.

Telefónica I+D 16

Mongodb Index

Example in console.

Telefónica I+D 17

Mongodb Sharding

MongoDB supports auto-sharding, and auto.balancing.

Sharding is an approach to scalability which separates your data across multiple servers.

With sharding we instead scale horizontally to achieve the same computacional/storage/memory footprint from smaller servers.

We will show the vertically scale db and the horizontally scaled db for comparison.

A sharded collection has a shard key. The collection is partitioned using this key. In this example we uses ident.

{ident:…, data:…}

Telefónica I+D 18

Mongodb Sharding

There are three components in sharding.

Mongos: routing procesing.

Config servers: store the metadata.

Mongod: store the data.

Range ShardIdent in [-inf,200) 2

Ident in (200, 400] 4

…

Ident in [1000,inf) 10

Telefónica I+D 19

Mongodb Sharding

Metadata is maintained on chunks which are represented by shard key ranges.

Range ShardIdent in [-inf,200) 2

Ident in (200, 400] 4

…

Ident in [1000,inf) 10

Telefónica I+D 20

Mongodb Sharding

Querys with the shard key are sent to the node who has this shard.

Query ShardsFind({ident :{$gt 100, $lt:150}) 2

Telefónica I+D 21

Mongodb Sharding

Querys with the shard key and the data are in some shard, it as sent to these nodes.

Query ShardsFind({ident :{$gt 100, $lt:250}) 2,3

Telefónica I+D 22

Mongodb Sharding

Querys with the no shard key are sent to all nodes.

Query ShardsFind({data:{$gt 100, $lt:250}) 2,3

Telefónica I+D 23

Mongodb Sharding

Example in console.

Telefónica I+D 24

Mongodb Replication

MongoDB replication works similarly to how relational database replication works. Writes are sent to a single server, the master, which then synchronizes itself to one or more other servers, the slaves. If the master goes down, a slave can be promoted to act as the new master.

While replication can improve performance (by distributing reads), its main purpose is to increase reliability.

There are two types of replication:

Master slaver replication.

Replica sets.

Telefónica I+D 25

Mongodb Replication

Master slaver replication.

Setup master: bin/mongod –master –oplogSize 100

Setup slaver: bin/mongod --slave --source <IP>[:<port>] --slavedelay 1 --autoresync

Telefónica I+D 26

Mongodb Replication

Replica sets. mongod --replSet “setname”

Shell in one nodecfg = {

... _id : “setname",

... members : [

... { _id : 0, host : “host1" },

... { _id : 1, host : “host2" },

... { _id : 2, host : “host3" } ] }

rs.initiate(cfg)

rs.status()

Options

arbiterOnly buildIndexes

hidden priority

slaveDelay votes

tags

Telefónica I+D 27

Mongodb Map&Reduce

DB is useful for batch processing of data and aggregation operations.

Telefónica I+D 28

Mongodb Tools

Mongodump: export to binary data

Mongoexport: export to csv

Mongofiles: load data into mongodb

MongoImport: import txt, csv or json data

mongoRestore: mongorestore takes the output from mongodump: and restores it.

mongoSniff: This utility is to MongoDB what tcpdump is to TCP/IP

Mongostat: Use the mongostat utility to quickly view statistics on a running mongod instance