Eliot Horowitz@eliothorowitzMongoBerlin

October 4, 2010

Scaling with MongoDB

Scaling

•Storage needs only go up•Operations/sec only go up•Complexity only goes up

Scaling by Optimization

•Schema Design•Index Design•Hardware Configuration

Horizontal Scaling

•Vertical scaling is limited•Hard to scale vertically in the cloud•Can scale wider than higher

Schema

•Modeling the same data in different ways can change performance by orders of magnitude

•Very often performance problems can be solved by changing Schema

Embedding•Great for read performance•One seek to load entire object•One roundtrip to database•Writes can be slow if adding to

objects all the time•Should you embed comments?

Indexes

•Index common queries•Make sure there aren’t duplicates:

(A) and (A,B) aren’t needed•Right-balance indexes keep

working set small

RAM Requirements

•Understand working set•What percentage of your data has

to fit in RAM?•How do you figure this out?

Hardware

•Disk performance•How many drives•What about ec2?•Network performance

Read Scaling•One master at any time•Programmer determines if read hits

master or a slave•Pro: easy to setup, can scale reads very

well•Con: reads are inconsistent on a slave •Writes don’t scale•Good for read heavy applications

One Master, Many Slaves

•Custom Master/Slave setup•Have as many slaves as you want•Can put them local to appservers•Good for 90+% read heavy

applications (Wikipedia)

Replica Sets•High Availability Cluster•One master at any time, up to 6 slaves•A slave automatically promoted to

master if failure•Drivers [will] support auto routing of

reads to slaves if programmer allows•Good for applications that need high

write availability but mostly reads (Commenting System)

•Many masters, even more slaves•Can scale reads and writes in two

dimensions•Add slaves for inconsistent read

scaling and redundancy•Add Shards for write and data size

scaling

Sharding

Common Setup•Typical setup is 3 shards with 3

servers per shard: 3 masters, 6 slaves•One massive collection, dozen non-

sharded•Can add sharding later to an existing

replica set with no down time•Can have sharded and non-sharded

collections

Use Cases

•Millions of user profiles•User activity stream•Photos•Logging

Download MongoDBhttp://www.mongodb.org

and let us know what you think@eliothorowitz @mongodb

10gen is hiring!http://www.10gen.com/jobs