Sharding with MongoDB -- MongoNYC 2012
-
Upload
tyler-brock -
Category
Technology
-
view
1.214 -
download
2
Transcript of Sharding with MongoDB -- MongoNYC 2012
![Page 2: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/2.jpg)
Philosophy
Concepts
Architecture
Mechanics
![Page 3: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/3.jpg)
Philosophy
![Page 4: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/4.jpg)
Philosophy
MongoDB is a database for developers.
![Page 5: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/5.jpg)
Build
Philosophy
![Page 6: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/6.jpg)
BuildScale
Philosophy
![Page 7: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/7.jpg)
How to Draw an Owl
Philosophy
![Page 8: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/8.jpg)
How to Draw an Owl
Philosophy
![Page 9: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/9.jpg)
> db.runCommand({enablesharding: "<dbname>" })
> db.runCommand({ shardcollection: "<namespace>", key: <shardkeypatternobject> })
Draw Two Circles
Philosophy
![Page 10: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/10.jpg)
Concepts
![Page 11: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/11.jpg)
datastore
app
Read/Write
Simple Web Application
![Page 12: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/12.jpg)
What happens when your working set exceeds memory?
![Page 13: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/13.jpg)
What happens if your write load is enormous?
![Page 14: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/14.jpg)
datastore
app
Vertical Scaling
![Page 15: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/15.jpg)
app
Vertical Scaling
datastore
![Page 16: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/16.jpg)
app
Vertical Scaling
datastore
appapp
68 GB RamRaid10 EBS
![Page 17: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/17.jpg)
datastore
app
Vertical Scaling
appapp
128 GB RamRaid10 SSD
![Page 18: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/18.jpg)
![Page 19: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/19.jpg)
app
datastoredatastoredatastore
Horizontal Scaling
60gb
![Page 20: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/20.jpg)
app
datastoredatastore datastore
20gb 20gb 20gb
Horizontal Scaling
![Page 21: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/21.jpg)
Routing Logic
app
datastoredatastore datastore
20gb 20gb 20gb
Horizontal Scaling
metadata
![Page 22: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/22.jpg)
Routing Logic
app
datastoredatastore datastore
20gb 20gb
Horizontal Scaling
metadata
60gb
![Page 23: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/23.jpg)
app
Routing Logic
Balancer
datastoredatastore datastore
20gb 20gb
Horizontal Scaling
metadata
60gb
![Page 24: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/24.jpg)
app
Routing Logic
Balancer
datastoredatastore datastore
Horizontal Scaling
metadata
30gb 30gb 30gb
![Page 25: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/25.jpg)
Architecture
![Page 26: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/26.jpg)
Really is just a mongod (or replica set)Where your data lives
mongod
Shard
![Page 27: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/27.jpg)
Mongod started with --configsvr optionMust have 3 (or 1 in development)Data is commited using 2 phase commit
config
Config Server
![Page 28: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/28.jpg)
mongos
Acts just like shard router / proxyOne or as many as you wantLight weight -- can run on App serversCaches meta-data from config servers
mongos
![Page 29: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/29.jpg)
Routing Logic
Balancingmetadata
datastore datastoredatastore
![Page 30: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/30.jpg)
metadata
datastore
mongos
datastoredatastore
![Page 31: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/31.jpg)
metadata
datastore
mongos
datastoredatastore
app
![Page 32: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/32.jpg)
datastore
mongos
config
datastoredatastore
app
![Page 33: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/33.jpg)
datastore
mongos
config
datastoredatastore
config
config
app
![Page 34: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/34.jpg)
mongos
config
mongod mongodmongod
config
config
app
![Page 35: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/35.jpg)
mongos
config
mongod mongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
config
config
app
![Page 36: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/36.jpg)
mongos
config
mongod mongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
config
config
app
![Page 37: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/37.jpg)
Mechanics
![Page 38: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/38.jpg)
How does MongoDB balance my data?
![Page 39: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/39.jpg)
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
Keys
test.users
![Page 40: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/40.jpg)
> db.runCommand({
})
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
shardcollection: “test.users”,
Keys
key: { email: 1 }
test.users
![Page 41: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/41.jpg)
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
shardcollection: “test.users”,
Keys
key: { email: 1 }
test.users
![Page 42: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/42.jpg)
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
Keys
key: { email: 1 }
test.users
![Page 43: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/43.jpg)
Chunks
-∞ +∞
![Page 44: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/44.jpg)
Chunks
-∞ +∞
![Page 45: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/45.jpg)
Chunks
-∞ +∞
Split!
![Page 46: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/46.jpg)
Chunks
-∞ +∞
Split!This is a chunk
This is a chunk
![Page 47: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/47.jpg)
Chunks
-∞ +∞
![Page 48: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/48.jpg)
Chunks
-∞ +∞
![Page 49: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/49.jpg)
Chunks
-∞ +∞
Split!
![Page 50: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/50.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 51: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/51.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 52: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/52.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 53: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/53.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 54: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/54.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Split this big chunk into 2
chunks
![Page 55: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/55.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 56: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/56.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
These chunks have split
![Page 57: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/57.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 58: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/58.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Shard1, move a chunk to
Shard2
![Page 59: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/59.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 60: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/60.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Shard1, move another chunk
to Shard3
![Page 61: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/61.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 62: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/62.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Shard1, move another chunk
to Shard4
![Page 63: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/63.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 64: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/64.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 65: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/65.jpg)
How does MongoDB route my queries?
![Page 66: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/66.jpg)
Routed Request
mongos
shard shard shard
![Page 67: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/67.jpg)
Routed Request1
mongos
shard shard shard
1. Query arrives at Mongos
![Page 68: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/68.jpg)
Routed Request1
2
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a single shard
![Page 69: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/69.jpg)
Routed Request1
2
3
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a single shard
3. Shard returns results of query
![Page 70: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/70.jpg)
Routed Request1
2
3
4
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a single shard
3. Shard returns results of query
4. Results returned to client
![Page 71: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/71.jpg)
Scatter Gather Request
shard shard shard
mongos
![Page 72: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/72.jpg)
Scatter Gather Request1
1. Query arrives at Mongos
shard shard shard
mongos
![Page 73: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/73.jpg)
Scatter Gather Request1
1. Query arrives at Mongos
2 22
shard shard shard
mongos2. Mongos broadcasts queryto all shards
![Page 74: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/74.jpg)
Scatter Gather Request1
1. Query arrives at Mongos
2 22
3 33
shard shard shard
mongos2. Mongos broadcasts queryto all shards
3. Each shard returns resultsfor query
![Page 75: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/75.jpg)
Scatter Gather Request1
41. Query arrives at Mongos
2 22
3 33
shard shard shard
mongos2. Mongos broadcasts queryto all shards
3. Each shard returns resultsfor query
4. Results combined andreturned to client
![Page 76: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/76.jpg)
mongos
Distributed Merge Sort Req.
shard shard shard
![Page 77: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/77.jpg)
mongos
Distributed Merge Sort Req.1
shard shard shard
1. Query arrives at Mongos
![Page 78: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/78.jpg)
mongos
Distributed Merge Sort Req.1
22 2
shard shard shard
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
![Page 79: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/79.jpg)
mongos
Distributed Merge Sort Req.1
22 2
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
![Page 80: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/80.jpg)
mongos
Distributed Merge Sort Req.1
22 2
4 44
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
4. Results returned to mongos
![Page 81: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/81.jpg)
mongos
Distributed Merge Sort Req.1
5
22 2
4 44
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
4. Results returned to mongos
5. Mongos merges sorted results
![Page 82: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/82.jpg)
mongos
Distributed Merge Sort Req.1
6
5
22 2
4 44
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
4. Results returned to mongos
5. Mongos merges sorted results
6. Combined results returned to client
![Page 83: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/83.jpg)
Queries
By Shard Key Routed db.users.find({email: “[email protected]”})
Sorted by shard key
Routed in order db.users.find().sort({email:-1})
Find by non shard key
Scatter Gather db.users.find({state:”NY”})
Sorted by non shard key
Distributed merge sort
db.users.find().sort({state:1})
![Page 84: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/84.jpg)
Writes
Inserts Requires shard key db.users.insert({ name: “Bob”, email: “[email protected]”})
Removes Routed db.users.delete({ email: “[email protected]”})
Removes
Scattered db.users.delete({name: “Bob”})
Updates Routed db.users.update( {email: “[email protected]”}, {$set: { state: “NY”}})
Updates
Scattered db.users.update( {state: “CA”}, {$set:{ state: “NY”}} )
![Page 85: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/85.jpg)
How do I choose my shard key?
![Page 86: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/86.jpg)
Choose a field that is common to your queries.
Rule of Thumb
![Page 87: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/87.jpg)
Write Scaling
Writes should be distributed.
![Page 88: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/88.jpg)
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Bad { time : 1 }
Writes should be distributed
![Page 89: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/89.jpg)
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Bad { time : 1 }
Better {node:1, application:1, time:1}
Writes should be distributed
![Page 90: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/90.jpg)
Query Isolation & Data Locality
Queries should be routed to one shard.
![Page 91: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/91.jpg)
Bad {msg: 1, node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken”}
Queries should be routed to one shard
![Page 92: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/92.jpg)
Better {node: 1, time: 1}
Bad {msg: 1, node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken”}
Queries should be routed to one shard
![Page 93: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/93.jpg)
Cardinality
Chunks should be able to split.
![Page 94: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/94.jpg)
Bad {node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Chunks should be able to split
![Page 95: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/95.jpg)
Better {node:1, time:1}
Bad {node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Chunks should be able to split
![Page 96: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/96.jpg)
Configuration
![Page 97: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/97.jpg)
mongodmongodmongod
Bring up mongods or Replica Sets
mongod mongodmongod
mongod mongodmongod
RS RS RS
mongod --shardsvrmongod --replSet --shardsvr
![Page 98: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/98.jpg)
config
mongodmongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
Bring up Config Servers
config
config
mongod --configsvr
![Page 99: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/99.jpg)
config
mongodmongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
Bring up Mongos
config
config
mongos
mongos --configdb <list of configdb uris>
![Page 100: Sharding with MongoDB -- MongoNYC 2012](https://reader033.fdocuments.in/reader033/viewer/2022042700/554f726ab4c9058a148b53ed/html5/thumbnails/100.jpg)
> use admin> db.runCommand({"addShard": <shard uri>})
Connect to Mongos+ Add Shards
Enable Sharding
> db.runCommand( { enablesharding : "<dbname>" } );
> db.runCommand( { shardcollection : "<namespace>", key : <key> });
Shard a Collection