Hands on with Ruby & MongoDB

Hands on with MongoDBRuby &

^

Austin on Rails ★ October 27, 2009

WYNNNETHERLAND

Wednesday, October 28, 2009

No SQL?x

x


When does NOSQL make sense?

★Your data is stored and retrieved mainly by primary key, without complex joins.

★You have a non-trivial amount of data, and the thought of managing lots of RDBMS shards and replication failure scenarios gives you the fear.

http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/




Key value stores

Project VoldemortRingoScalarisKaiDynomiteMemcacheDBThruDBCouchDBCassandraHBaseHypertableRedisTokyo Cabinet/Tyrant


Some very coolprojects in this space


http://project-voldemort.com/

http://project-voldemort.com/

http://github.com/tuulos/ringo/tree/master

http://github.com/tuulos/ringo/tree/master

http://code.google.com/p/scalaris/

http://code.google.com/p/scalaris/

http://kai.wiki.sourceforge.net/

http://kai.wiki.sourceforge.net/

http://github.com/cliffmoon/dynomite/tree/master

http://github.com/cliffmoon/dynomite/tree/master

http://memcachedb.org/

http://memcachedb.org/

http://code.google.com/p/thrudb/

http://code.google.com/p/thrudb/

http://couchdb.apache.org/


http://code.google.com/p/the-cassandra-project/

http://code.google.com/p/the-cassandra-project/

http://hadoop.apache.org/hbase/

http://hadoop.apache.org/hbase/

http://hypertable.org/




http://tokyocabinet.sourceforge.net/spex-en.html

http://tokyocabinet.sourceforge.net/spex-en.html



Tokyo Cabinet: Popular with Rubyists, Big in Japan


Tokyo Cabinet: Popular with Rubyists, Big in Japan

★Lightning fast★Works best for !at objects★Tokyo Tyrant for network access

http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/




More document-oriented solutions


CouchDB

Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API.





Erlang + Javascript


Map + reduce


Very cool


Plenty o’ Ruby to go around


Ruby libraries for CouchDB

CouchRest Basic model RelaxDB CouchPotato CouchFoo ActiveCouch

http://www.slideshare.net/brianthecoder/couchdb

From theAustin on Rails talk




Document stores:Throw out everything

you learned about DB design

almost*

* more on this later


SQL CouchDB

Prede!ned, explicit schema Dynamic, implicit schema

Uniform tables of data Collection of named documents with varying structure

Normalized. Objects spread across tables. Duplication reduced.

Denormalized. Docs usually self contained. Data often duplicated.

Must know schema to read/write a complete object Must know only document name

Dynamic queries of static schemas Static queries of dynamic schemas

http://damienkatz.net/files/What is CouchDB.pdfhttp://damienkatz.net/files/What is CouchDB.pdf


http://damienkatz.net/files/What%20is%20CouchDB.pdf




SQL CouchDB

Prede!ned, explicit schema Dynamic, implicit schema

Uniform tables of data Collection of named documents with varying structure

Normalized. Objects spread across tables. Duplication reduced.

Denormalized. Docs usually self contained. Data often duplicated.

Must know schema to read/write a complete object Must know only document name

Dynamic queries of static schemas Static queries of dynamic schemas

http://damienkatz.net/files/What is CouchDB.pdfhttp://damienkatz.net/files/What is CouchDB.pdf

The devil's in the details






because this is BIG


Runs like Hayes.Hits like Mays.


MongoDB x


Let's just skip to here already!

x


Introducing

★Built For Speed★Dynamic Queries and Indexes★Replication and Failover★Sharding★Map / Reduce

can be very fast


MongoDB is great for

★Websites

★Caching

★High volume, low value

★High scalability

★Storage of program objects and JSON

stash the hash


Not as great for

★Highly transactional

★Ad-hoc business intelligence

★Problems requiring SQL


Installation

★mkdir -p /data/db

★download pre-built for OSX and unzip to /usr/local/

★cp -R /usr/local/pathtomongo/bin /usr/local/bin

★sudo gem install mongo

★sudo gem install mongo_ext

★sudo gem install mongo_mapper

Native C extensions ( go turbo! )

Ruby driver for MongoDB


Contents of mongo/bin

★mongod - The MongoDB server

★mongo - the JavaScript interactive shell

★mongoexport - export data as JSON or csv

★mongoimport - As advertised

★mongodump - Like mysqldump

★mongorestore - Restore from mongodump "les

★mongos - Auto-sharding module (getting better with every build)

what's in the box?

why? What did you think it was?


Some new terms


When I saydatabase


When I saydatabase

thinkdatabase


Well that one isn't new...

When I saydatabase

thinkdatabase


Databases in MongoDB

★Made up of multiple collections

★Are created on-the-!y when "rst referenced


When I saycollection


When I saycollection

thinktable


Collections in MongoDB

★Schema-less

★For grouping documents into smaller query sets (speed)

★ Indexable by one or more key

★Are created on-the-!y when "rst referenced

★Capped collections: Fixed size, older records dropped after limit reached

but typed!


When I saydocument


When I saydocument

thinkrecord or row


Document

★Stored in a collection

★Can have _id key that works like primary keys in MySQL

★Supports relationships: subdocument or db reference


Document Storage (BSON)

{ author: 'joe', created: Date('03-28-2009'), title: 'Yet another blog post', text: 'Here is the text...', tags: [ 'example', 'joe' ], comments: [ { author: 'jim', comment: 'I disagree' }, { author: 'nancy', comment: 'Good post' } ]}

http://www.mongodb.org/display/DOCS/BSON







Sure wish JSON did this...







Sure wish JSON did this...

B is for Binary




That looks like JSON


So does this

jason.to_json


Where's the Ruby?


Querying

db.collection.find({'first_name': 'John'}) # finds all Johns

db.collection.find({'first_name': /^wynn/i}) # regex

db.collection.find_first({'_id':1}) # finds first with _id of 1

db.collection.find({'age': {'$gte': 21}}) # finds possible drinkers

db.collection.find({'author.first_name':'John'}) # subdocument

db.collection.find({$where:'this.age >= 6 && this.age <= 18'})


Querying

$in, $nin, $all, $ne, $gt, $gte, $lt, $lte, $size, $where

:fields (like :select in active record)

:limit, :offset for pagination

:sort ascending or descending [['foo', 1], ['bar', -1]]

count and group (uses map/reduce)

db.collection.mapReduce(mapfunction,reducefunction[,options]);

More


10 Gen projects on the hub

★mongo-ruby-driver http://github.com/mongodb/mongo-ruby-driver

★mongorecord http://github.com/mongodb/mongo-activerecord-ruby


http://github.com/mongodb/mongo-ruby-driver




http://github.com/mongodb/mongo-activerecord-ruby




MongoMapper

★Mongo is not MySQL

★DSL for modeling domain should also teach you Mongo

★ It sounded fun

★Just released version 0.5.6

from @jnunemaker


MongoMapper

★Mongo is not MySQL

★DSL for modeling domain should also teach you Mongo

★ It sounded fun

★Just released version 0.5.6

from @jnunemaker

I voted for "Nunemapper"


Features

★Typecasting

★Callbacks (ActiveSupport Callbacks)

★Validations

★Connection and database can differ per document

★Create and Update with single or multiple

★Delete and Destroy and _all counterparts

★Find: id, ids, :all, :"rst, :last

★Associations

Be careful. Ordering can be tricky.


Example

class User include MongoMapper::Document key :name, String, :required => true, :length => 5..100 key :email, String, :required => true, :index => true key :age, Integer, :numeric => true key :active, Boolean, :default => true one :address many :articlesend

class Address include MongoMapper::Document key :street, String key :city, String key :state, String, :length => 2 key :zip, Integer, :numeric => true, :length => 5end

Included as module, not subclassed (this may change soon).


MongoDB fun

★Capped collections (think memcache, actually used for replication)

★Upserts db.collection.update({'_id':1}, {'$inc':{'views':1}})

★Multikeys (think tagging and full text search)

★GridFS and auto-sharding


So, is it ready for Prime Time?

★Disqus

★SourceForge

★TweetCongress, GovTwit -- Floxee.com

★TweetSaver.com

★Mozilla Ubiquity Herd

Ask these folks!


Lessons learned in production

★The laws of computing are still in effect

★ Indexes are important no matter what the salesman told ya about performance

★Data modeling. Deep or Wide?

★MongoDB and MongoMapper are in active development

the fine print

The answer is yes!

Very responsive yet very volatile changes!


How can you help?

★We need an awesome admin GUI

★Port some plugins (might get easier with ActiveModel support coming soon)

★Build something cool


Resources

http://mongodb.orghttp://www.10gen.com http://groups.google.com/group/mongomapperhttp://groups.google.com/group/mongodb-userhttp://squeejee.com

http://wynnnetherland.com

and thanks for having me!

Questions? I'm @pengwynn on Twitter

the very new blog

the very cool company behind MongoDB


http://mongodb.com

http://mongodb.com

http://www.10gen.com

http://www.10gen.com

http://groups.google.com/group/mongomapper

http://groups.google.com/group/mongomapper

http://groups.google.com/group/mongodb-user

http://groups.google.com/group/mongodb-user

http://squeejee.com

http://squeejee.com




Hands on with Ruby & MongoDB

Technology

Transcript of Hands on with Ruby & MongoDB