Time Series Data in MongoDB Senior Solutions Architect, MongoDB Inc. Massimo Brignoli #mongodb.
Shortcuts around the mistakes I've made scaling MongoDB
-
Upload
iconara -
Category
Technology
-
view
841 -
download
0
description
Transcript of Shortcuts around the mistakes I've made scaling MongoDB
![Page 1: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/1.jpg)
SHORTCUTSAROUND THEMISTAKES I’VEMADE SCALING
MONGODB
Theo, Chief Architect atonsdag 21 september 11
![Page 2: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/2.jpg)
What we doWe want to revolutionize the digital advertising industry by showing that there is more to ad analytics than click through rates.
onsdag 21 september 11
![Page 3: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/3.jpg)
Ads
onsdag 21 september 11
![Page 4: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/4.jpg)
Data
onsdag 21 september 11
![Page 5: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/5.jpg)
Assembling sessionsexposure
pingping
ping ping
ping
event
event
ping
session➔ ➔
onsdag 21 september 11
![Page 6: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/6.jpg)
Crunching
session
session
session
session
sessionsession
session session
session
session
session
session
session
➔ ➔ 42
onsdag 21 september 11
![Page 7: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/7.jpg)
Reports
onsdag 21 september 11
![Page 8: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/8.jpg)
What we doTrack ads, make pretty reports.
onsdag 21 september 11
![Page 9: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/9.jpg)
That doesn’t sound so hard
onsdag 21 september 11
![Page 10: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/10.jpg)
That doesn’t sound so hardWe don’t know when sessions end
onsdag 21 september 11
![Page 11: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/11.jpg)
That doesn’t sound so hardWe don’t know when sessions endThere’s a lot of data
onsdag 21 september 11
![Page 12: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/12.jpg)
That doesn’t sound so hardWe don’t know when sessions endThere’s a lot of dataIt’s all done in (close to) real time
onsdag 21 september 11
![Page 13: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/13.jpg)
Numbers
onsdag 21 september 11
![Page 14: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/14.jpg)
Numbers40 Gb data
onsdag 21 september 11
![Page 15: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/15.jpg)
Numbers40 Gb data50 million documents
onsdag 21 september 11
![Page 16: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/16.jpg)
Numbers40 Gb data50 million documentsper day
onsdag 21 september 11
![Page 17: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/17.jpg)
How we use MongoDB
onsdag 21 september 11
![Page 18: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/18.jpg)
How we use MongoDB“Virtual memory” to offload data while we wait for sessions to finish
onsdag 21 september 11
![Page 19: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/19.jpg)
How we use MongoDB“Virtual memory” to offload data while we wait for sessions to finishShort time storage (<48 hours) for batch jobs
onsdag 21 september 11
![Page 20: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/20.jpg)
How we use MongoDB“Virtual memory” to offload data while we wait for sessions to finishShort time storage (<48 hours) for batch jobsMetrics storage
onsdag 21 september 11
![Page 21: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/21.jpg)
Why we use MongoDB
onsdag 21 september 11
![Page 22: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/22.jpg)
Why we use MongoDBSchemalessness makes things so much easier, the data we collect changes as we come up with new ideas
onsdag 21 september 11
![Page 23: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/23.jpg)
Why we use MongoDBSchemalessness makes things so much easier, the data we collect changes as we come up with new ideasSharding makes it possible to scale writes
onsdag 21 september 11
![Page 24: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/24.jpg)
Why we use MongoDBSchemalessness makes things so much easier, the data we collect changes as we come up with new ideasSharding makes it possible to scale writesSecondary indexes and rich query language are great features (for the metrics store)
onsdag 21 september 11
![Page 25: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/25.jpg)
Why we use MongoDBSchemalessness makes things so much easier, the data we collect changes as we come up with new ideasSharding makes it possible to scale writesSecondary indexes and rich query language are great features (for the metrics store)It’s just… nice
onsdag 21 september 11
![Page 26: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/26.jpg)
Btw.
onsdag 21 september 11
![Page 27: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/27.jpg)
Btw.We use JRuby, it’s awesome
onsdag 21 september 11
![Page 28: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/28.jpg)
A story in 7 iterations
onsdag 21 september 11
![Page 29: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/29.jpg)
secondary indexes and updates1st iteration
onsdag 21 september 11
![Page 30: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/30.jpg)
secondary indexes and updates1st iteration
One document per session, update as new data comes alongOutcome: 1000% write lock
onsdag 21 september 11
![Page 31: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/31.jpg)
#1Everything is aboutworking around the
GLOBALWRITELOCK
onsdag 21 september 11
![Page 32: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/32.jpg)
MongoDB 2.0.0
db.coll.update({_id: "xyz"}, {$inc: {x: 1}}, true)
db.coll.update({_id: "abc"}, {$push: {x: “...”}}, true)
onsdag 21 september 11
![Page 33: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/33.jpg)
MongoDB 1.8.1
db.coll.update({_id: "xyz"}, {$inc: {x: 1}}, true)
db.coll.update({_id: "abc"}, {$push: {x: “...”}}, true)
onsdag 21 september 11
![Page 34: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/34.jpg)
using scans for two step assembling2nd iteration
Instead of updating, save each fragment, then scan over _id to assemble sessions
onsdag 21 september 11
![Page 35: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/35.jpg)
using scans for two step assembling2nd iteration
Outcome: not as much lock, but still not great performance. We also realised we couldn’t remove data fast enough
onsdag 21 september 11
![Page 36: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/36.jpg)
#2Everything is aboutworking around the
GLOBALWRITELOCK
onsdag 21 september 11
![Page 37: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/37.jpg)
#3Give a lot of
thought to your
PRIMARYKEY
onsdag 21 september 11
![Page 38: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/38.jpg)
partitioning3rd iteration
onsdag 21 september 11
![Page 39: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/39.jpg)
partitioning3rd iteration
We came up with the idea of partitioning the data by writing to a new collection every hour
onsdag 21 september 11
![Page 40: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/40.jpg)
partitioning3rd iteration
We came up with the idea of partitioning the data by writing to a new collection every hourOutcome: lots of complicated code, lots of bugs, but we didn’t have to care about removing data
onsdag 21 september 11
![Page 41: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/41.jpg)
#4Make sure you can
REMOVE OLD DATA
onsdag 21 september 11
![Page 42: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/42.jpg)
sharding4th iteration
onsdag 21 september 11
![Page 43: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/43.jpg)
sharding4th iteration
To get around the global write lock and get higher write performance we moved to a sharded cluster.
onsdag 21 september 11
![Page 44: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/44.jpg)
sharding4th iteration
To get around the global write lock and get higher write performance we moved to a sharded cluster.Outcome: higher write performance, lots of problems, lots of ops time spent debugging
onsdag 21 september 11
![Page 45: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/45.jpg)
#5Everything is aboutworking around the
GLOBALWRITELOCK
onsdag 21 september 11
![Page 46: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/46.jpg)
#6SHARDINGIS NOT A
SILVER BULLETand it’s buggy,
if you can, avoid it
onsdag 21 september 11
![Page 47: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/47.jpg)
onsdag 21 september 11
![Page 48: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/48.jpg)
#7IT WILL FAIL
design for it
onsdag 21 september 11
![Page 49: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/49.jpg)
onsdag 21 september 11
![Page 50: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/50.jpg)
onsdag 21 september 11
![Page 51: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/51.jpg)
moving things to separate clusters5th iteration
onsdag 21 september 11
![Page 52: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/52.jpg)
moving things to separate clusters5th iteration
We saw very different loads on the shards and realised we had databases with very different usage patterns, some that made autosharding not work. We moved these off the cluster.
onsdag 21 september 11
![Page 53: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/53.jpg)
moving things to separate clusters5th iteration
We saw very different loads on the shards and realised we had databases with very different usage patterns, some that made autosharding not work. We moved these off the cluster.Outcome: a more balanced and stable cluster
onsdag 21 september 11
![Page 54: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/54.jpg)
#8Everything is aboutworking around the
GLOBALWRITELOCK
onsdag 21 september 11
![Page 55: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/55.jpg)
#9ONE DATABASEwith one usage pattern
PER CLUSTER
onsdag 21 september 11
![Page 56: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/56.jpg)
#10MONITOR
EVERYTHINGlook at your health
graphs daily
onsdag 21 september 11
![Page 57: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/57.jpg)
monster machines6th iteration
onsdag 21 september 11
![Page 58: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/58.jpg)
monster machines6th iteration
We got new problems removing data and needed some room to breathe and think
onsdag 21 september 11
![Page 59: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/59.jpg)
monster machines6th iteration
We got new problems removing data and needed some room to breathe and think Solution: upgraded the servers to High-Memory Quadruple Extra Large (with cheese).
onsdag 21 september 11
![Page 60: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/60.jpg)
monster machines6th iteration
We got new problems removing data and needed some room to breathe and think Solution: upgraded the servers to High-Memory Quadruple Extra Large (with cheese).
♥Ionsdag 21 september 11
![Page 61: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/61.jpg)
#11Don’t try to scale up
SCALE OUT
onsdag 21 september 11
![Page 62: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/62.jpg)
#12When you’re out of ideas
CALL THE EXPERTS
onsdag 21 september 11
![Page 63: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/63.jpg)
partitioning (again) and pre-chunking7th iteration
onsdag 21 september 11
![Page 64: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/64.jpg)
partitioning (again) and pre-chunking7th iteration
We rewrote the database layer to write to a new database each day, and we created all chunks in advance. We also decreased the size of our documents by a lot.
onsdag 21 september 11
![Page 65: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/65.jpg)
partitioning (again) and pre-chunking7th iteration
We rewrote the database layer to write to a new database each day, and we created all chunks in advance. We also decreased the size of our documents by a lot.Outcome: no more problems removing data.
onsdag 21 september 11
![Page 66: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/66.jpg)
#13Smaller objects means a smaller database, and a smaller database means
LESS RAM NEEDED
onsdag 21 september 11
![Page 67: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/67.jpg)
#14Give a lot of
thought to your
PRIMARYKEY
onsdag 21 september 11
![Page 68: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/68.jpg)
#15Everything is aboutworking around the
GLOBALWRITELOCK
onsdag 21 september 11
![Page 69: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/69.jpg)
#16Everything is aboutworking around the
GLOBALWRITELOCK
onsdag 21 september 11
![Page 70: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/70.jpg)
KTHXBAI
@iconaraarchitecturalatrocities.com
burtcorp.com
onsdag 21 september 11
![Page 71: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/71.jpg)
Since we got time…
onsdag 21 september 11
![Page 72: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/72.jpg)
Safe modeTips
onsdag 21 september 11
![Page 73: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/73.jpg)
Safe modeTips
Run every Nth insert in safe mode
onsdag 21 september 11
![Page 74: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/74.jpg)
Safe modeTips
Run every Nth insert in safe modeThis will give you warnings when bad things happen; like failovers
onsdag 21 september 11
![Page 75: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/75.jpg)
Avoid bulk insertsTips
onsdag 21 september 11
![Page 76: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/76.jpg)
Avoid bulk insertsTips
Very dangerous if there’s a possibility of duplicate key errors
onsdag 21 september 11
![Page 77: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/77.jpg)
EC2Tips
onsdag 21 september 11
![Page 78: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/78.jpg)
EC2Tips
You have three copies of your data, do you really need EBS?
onsdag 21 september 11
![Page 79: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/79.jpg)
EC2Tips
You have three copies of your data, do you really need EBS?Instance store disks are included in the price and they have predictable performance.
onsdag 21 september 11
![Page 80: Shortcuts around the mistakes I've made scaling MongoDB](https://reader033.fdocuments.in/reader033/viewer/2022051816/54579575af795994188b516d/html5/thumbnails/80.jpg)
EC2Tips
You have three copies of your data, do you really need EBS?Instance store disks are included in the price and they have predictable performance.m1.xlarge comes with 1.7 TB of storage.
onsdag 21 september 11