Lessons Learned at Wakoopa
-
Upload
menno-van-der-sman -
Category
Technology
-
view
2.208 -
download
4
description
Transcript of Lessons Learned at Wakoopa
![Page 1: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/1.jpg)
![Page 2: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/2.jpg)
Lessons Learnedwhile building Wakoopa
![Page 3: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/3.jpg)
Menno van der SmanDeveloper at Wakoopa
since May 2007
![Page 4: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/4.jpg)
Wakoopa is about software
![Page 5: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/5.jpg)
![Page 6: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/6.jpg)
MacPC
![Page 7: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/7.jpg)
200million
![Page 8: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/8.jpg)
Does it scale?
![Page 9: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/9.jpg)
Scaling the front is easy
![Page 10: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/10.jpg)
Scaling the back is harder
![Page 11: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/11.jpg)
Limits spawn creativity
![Page 12: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/12.jpg)
One server
at Railsmachinesliced into staging, production and db
MySQL 4.1Apache + Mongrel
![Page 13: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/13.jpg)
Background queue
![Page 14: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/14.jpg)
Lots of tools already
BackgroundRB, DelayedJob, Daemons etc
![Page 15: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/15.jpg)
Rolling our own
database basedusing script/runner
processing multiple items in one loopone cache update per loop
![Page 16: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/16.jpg)
Smart queries
INSERT INTO hourly_usage (...) VALUES (...) ON DUPLICATE KEY UPDATE active_seconds = active_seconds + ..., idle_seconds = idle_seconds + ...
UPDATE developers SET active_seconds = active_seconds + ...
Allows for concurrency
![Page 17: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/17.jpg)
Size = rowsize x rows
Make rowsize smaller
![Page 18: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/18.jpg)
sometimes `id` shouldn’t be the
Primary Key
![Page 19: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/19.jpg)
Enter composite_primary_keys
![Page 20: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/20.jpg)
Overrides default PK
Original idea by Dr NicEnhanced for Rail 2.1 by Darrin Holsthttp://compositekeys.rubyforge.org/
class HourlyUsage < ActiveRecord::Base set_primary_keys :user_id, :software_id, :used_at ...end
![Page 21: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/21.jpg)
Archive old datathat’s not frequently used
![Page 22: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/22.jpg)
HourlyUsageto
HourlyUsageStorage
SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file'
LOAD DATA INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage
avoids long locks
![Page 23: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/23.jpg)
Upcoming improvements
![Page 24: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/24.jpg)
MySQL 5.1Partitioning
CREATE TABLE hourly_usage_storage ( ... )PARTITION BY RANGE (software_id) ( PARTITION p0 VALUES LESS THAN (...), PARTITION p1 VALUES LESS THAN (...), PARTITION p2 VALUES LESS THAN (...), PARTITION p3 VALUES LESS THAN (...), .... PARTITION pn VALUES LESS THAN MAXVALUE);
Available in MySQL 5.1
Makes ‘WHERE software_id = ...’ very fast
![Page 25: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/25.jpg)
And many more plans
MySQL 5.1Memcached
SphinxRails 2.1
Amazon EC2
Designing a new infrastructure
![Page 26: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/26.jpg)
Announcing a new blog
Inside Wakoopahttp://inside.wakoopa.com
![Page 27: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/27.jpg)
Questions?
http://wakoopa.com/jobs
Looking for C++ and frontend awesomeness
or suggestions?
![Page 28: Lessons Learned at Wakoopa](https://reader034.fdocuments.in/reader034/viewer/2022051816/545b5b9ab1af9f4a1d8b4d36/html5/thumbnails/28.jpg)