Mongodb, Node.js and You: PART I
-
Upload
mitch-pirtle -
Category
Technology
-
view
915 -
download
1
description
Transcript of Mongodb, Node.js and You: PART I
![Page 1: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/1.jpg)
Node.js, MongoDB and You: Part I
Mitch Pirtle jsDay 2014, Verona Italy - @jsdayit
![Page 2: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/2.jpg)
First, tell me about yourselves.
![Page 3: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/3.jpg)
New to Node?
![Page 4: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/4.jpg)
Come on, be honest!
![Page 5: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/5.jpg)
New to Node?
![Page 6: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/6.jpg)
New to MongoDB?
![Page 7: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/7.jpg)
Does your Javascript totally suck?
![Page 8: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/8.jpg)
Ok my Javascript totally sucks.
![Page 9: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/9.jpg)
Now about me.
![Page 10: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/10.jpg)
Mitch Pirtle
• Recovering Joomla! founder
• Mongo Master
• Starting companies since 1995
• Musician, skate punk, football coach
• American idiot living in Turin
![Page 11: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/11.jpg)
Important Mitch Facts
• I am not cool. However I have been called perky.
• I am not Rich. My name is Mitch. Such is life.
• I am internet famous. Just to be clear: Internet Famous + $1.50 = $1.50
![Page 12: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/12.jpg)
About this talk.
![Page 13: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/13.jpg)
Ok, technically there are three talks today.
![Page 14: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/14.jpg)
• Session 1: All about MongoDB (this one)!
• Session 2: All about Node.js (that’s next)
• Session 3: The coolness of both together
![Page 15: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/15.jpg)
That’s a lotta lotta stuff to
cover
STAY ALERT
![Page 16: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/16.jpg)
All About MongoDB
• Brief introduction to MongoDB
• CONSOLE!
• Really cool discoveries and surprises
• Shameful admissions and painful stories
![Page 17: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/17.jpg)
In The Beginning• We had relational databases. Back then they
were called “databases” and that’s where you stored your data.
• Primary focus: atomicity, consistency, reliability.
• Was normal to spend 6 hours. ON ONE QUERY.
• I love vacuum tubes, keep you warm in winter.
• Life was good.
![Page 18: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/18.jpg)
What Happened
• Hello, Internet!
• Databases became immediate source of pain for scale, performance
• Traffic grew, along with it came bigger expectations, infinitely more complexity, a slew of new platforms, and Big Data™
![Page 19: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/19.jpg)
That sure looks like a nail to me.
![Page 20: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/20.jpg)
Troubled Relations
• Web languages gravitated toward objects, not 3NF entites/relations
• Size of data needed to live on more than one physical machine
• Performance requirements needed to be far better
![Page 21: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/21.jpg)
Along came sharding
• Can split your data across multiple machines
• Also splits your query load across multiple machines
• Like RAID for your data, right?
![Page 22: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/22.jpg)
What sharding brought along for the ride
• How do you back this stuff up?
• How do you spread a group query across N machines again?
• How do you run a join query that spans a sharded table?
![Page 23: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/23.jpg)
All those hours, spent mastering 3NF and
procedural programming
![Page 24: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/24.jpg)
IMPORTANT LESSON:
![Page 25: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/25.jpg)
It is REALLY hard to scale a relational database
engine.
![Page 26: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/26.jpg)
The common approach pushed logic out of the database back
into the application tier.
![Page 27: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/27.jpg)
Then why use a relational database in the first place?
![Page 28: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/28.jpg)
Then there was…
![Page 29: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/29.jpg)
The Promises of MongoDB• Speed - crazy whack-daddy fast
• Simplicity - JSON documents FTW
• Embedded documents
• 16MB limit
• Scale - sharding, multimaster out of the box
• Yes, I said whack-daddy.
![Page 30: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/30.jpg)
ENOUGH TALKBRING ON THE CONSOLE
![Page 31: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/31.jpg)
Wait, there’s more• Fulltext: Allows for compound indexes, supports
many languages
• Sharding: You can scale collections across N machines
• GridFS: Simple interface to store files in your database (CONSOLE!)
• Multimaster: Replica Sets make it possible for read slaves, failover, redundancy
![Page 32: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/32.jpg)
Now some cool stories
![Page 33: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/33.jpg)
Mini Case Study: Totsy• First ecommerce site to rely on MongoDB for all
data. Everything. Even product images and associated media.
• I suspected it would be fast.
• I suspected we could develop quickly. (This was important, as they only let me hire one guy.)
![Page 34: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/34.jpg)
So how fast was it?
![Page 35: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/35.jpg)
Launch story• Went live with MongoDB on a quad-core
consumer grade el-cheapo machine, only 2GB RAM.
• I was terrified.
• Over a million moms waiting for the launch.
• Upon launch, load was 0.05. Highest it ever got was around 0.5.
![Page 36: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/36.jpg)
Was development quicker?
![Page 37: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/37.jpg)
Development impact• Simple models make for less code. There were
no sixteen-table joins, no ORM, one result had all the data needed from a single query.!
• Less code makes for less bugs. No more six-hour query debugging marathons. No more learning why UNION was faster than JOIN…
• Less bugs leaves time for more code. Did I mention they only let me hire one guy?
![Page 38: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/38.jpg)
Even moar impact
• Used GridFS for all media storage.
• Allowed free MD5 checking for duplicates.
• Allowed storage of metadata per file (views, comments, rates, whatever else we wanted).
• No need for NFS, clumsy rsync cronjobs, high costs of NAS or iSCSI.
![Page 39: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/39.jpg)
Now some sad stories.
![Page 40: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/40.jpg)
The perils of schemaless
• Started prototyping quickly enough
• Made a couple changes to user model
• Made some more changes…
• WHUPS WHY FIFTEEN KINDS OF USER?!?!
![Page 41: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/41.jpg)
Remember: Always update existing data
when changing models.
![Page 42: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/42.jpg)
Everything in the database!
• Backups were brutal
• Forgot to separate GridFS data from main database
• Totally unprepared for the operational impact
![Page 43: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/43.jpg)
Remember: Operational impact BEFORE you
launch.
![Page 44: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/44.jpg)
Stump the Geek™
![Page 45: Mongodb, Node.js and You: PART I](https://reader034.fdocuments.in/reader034/viewer/2022042623/54620936af7959ba618b4aeb/html5/thumbnails/45.jpg)
Thanks!
• AboutMe
• @mitchitized - Twitter
• spacemonkey - GitHub
• LinkedIn - I’M AVAILABLE!