Stacktrace Berlin RC.2
38
Bits of Experience DB Scaling, Cloud Hosting, Code Quality
-
Upload
oliver-seemann -
Category
Technology
-
view
92 -
download
2
description
Slides of my talk at Stacktrace meetup on 2014-04-03 in Berlin at Hub:raum.
Transcript of Stacktrace Berlin RC.2
- 1. Bits of Experience DB Scaling, Cloud Hosting, Code Quality
- 2. Hi! Nice to meet you.
- 3. I want to offer some insights Everything's different, there's not one set of rules for everyone.
- 4. Small Dev team, big system 10 Applications, Up to 5 stages, 36 machines.
- 5. Our main tool is Python Also lots of SQL; More JavaScript than we'd like; Even some Perl.
- 6. A few Tera Bytes of data Most of it in PostgreSQL. We like PostgreSQL a lot.
- 7. Agenda Things I'll dive into a little.
- 8. 1. Scaling Data What we do, why it works.
- 9. 2. To cloud or not to cloud We've mostly run on bare metal, but how much longer?
- 10. 3. Complexity and Code Quality A few thoughts.
- 11. There's no universal right or wrong I just want to provide an additional perspective.
- 12. We're storing advertising data Lots of data, but not quite Big Data.
- 13. A SaaS business Isolated customers, varying amounts of data.
- 14. All relational, structured data We aggregate and filter data a lot. SQL is great for that.
- 15. Starting small One database, relational model by the book.
- 16. Trying built-in PG partitioning It's useful, but very limited.
- 17. Maybe use schemas? Then we'd lose other schema use cases.
- 18. Partition using databases Each customer his own db. Scales very well.
- 19. Very good tool support DBs can be easily created, copied, dumped, restored, dropped.
- 20. Isolation also benefits security Nothing to rely on, but helpful nonetheless.
- 21. There are downsides, of course Overhead, connection management, no quick global queries.
- 22. For us, it's great Your mileage may vary.
- 23. Dedicated Hardware Good performance for little money.
- 24. Roots precede AWS Why move a working infrastructure?
- 25. B2B: growth better predictable Elasticity requirements are limited.
- 26. We're missing: Proper Failover Automatic failover of all components is not that trivial.
- 27. For a while: mixed setup Frontend on AWS. Async Backend on our HW. IPSEC in between.
- 28. Now: Something different Software development is hard.
- 29. Start-up vs. Enterprise How different can it be?
- 30. When is a start-up a start-up? And when not?
- 31. Lots of uncertainty Building, testing, learning.
- 32. Building good systems In a startup that's not always easy.
- 33. Time is always a constraint In small companies as in big.
- 34. Clean code is not what sells Unfortunately.
- 35. Fast now, right later? Technical debt will cost you big time
- 36. Small but steady Little things add up
- 37. Delete code, drop features Code is a liability.
- 38. Thanks for listening Time for questions.