PutPlace's use of Amazon Cloud Services
8
Secure ● Organise ● Share …your Digital Lifetime
-
Upload
joe-drumgoole -
Category
Technology
-
view
2.118 -
download
0
description
Joe Drumgoole CTO of PutPlace talks about how PutPlace uses Amazon's web services.
Transcript of PutPlace's use of Amazon Cloud Services
Secure ● Organise ● Share…your Digital Lifetime
Slide No. 2 Proprietary and confidential – PutPlace 2008
PutPlace
•Online Backup Service•Mac and Windows Clients•Provide Sharing integration with
Flickr/YouTube (Real Soon!)•Original BP planned to spend 1m on
data-centre in 2007
Slide No. 3 Proprietary and confidential – PutPlace 2008
PutPlace Grid
PutPlace S3 Storage Grid
PutPlace EC2 Compute Grid
Database Server
App Server
App Server
App Server
App Server
Task Server
Task Server
Task Server
App Server
...
Mac Client Windows Client
...
Production Bucket
OAT Bucket
Media Bucket
Load Balancer
Grid Monitor
Slide No. 4 Proprietary and confidential – PutPlace 2008
The PutPlace Grid
Slide No. 5 Proprietary and confidential – PutPlace 2008
Key Stats
• Been using Amazon EC2/S3 in production since Jan ‘08• Public paid service launched in Oct ’08• Backing up around 15000 user files everyday• Spend
- EC2 – USD $ 1200 a month- S3 – USD $ 470 a month (S3 costs approximately double each month)- Elastic IP – Use it but its really slow to respond- No Simple DB (we use Postgres)- No SQS (slow, non-determininistic, pricey)- No EBS ( waiting and seeing how others perform)- Keen on CDN!
Slide No. 6 Proprietary and confidential – PutPlace 2008
Things We Use
• Perlbal – Load balancing• Postgres – Database• Django – App Servers and Task Managers• Nagios – Internal Monitoring• Monitis – External Monitoring• Hamachi – Virtual Private Network (but no more free )• Hudson – Build Service• Lots of Hosted Services
- Monitis- DynDNS- GoDaddy- Google Docs- Google Sites- Google Email
Slide No. 7 Proprietary and confidential – PutPlace 2008
How we cut costs
• Big chunk of EC2 cost is DB large instance- Partition DB and run on smaller instances
• Storage is a variable cost-compression and deduplication help
• We get hammered for puts, posts, lists- Switch to MogileFS
• Dirty little secret of online backup – nobody downloads- Bandwidth is a marginal cost
• Automation- Can bring up (and down) a new grid in 10 minutes
Slide No. 8 Proprietary and confidential – PutPlace 2008
The Future
• Gearman for Task Management• CDN for static content• Choice of Storage/Grid in Europe or US• More use of regions for fault tolerance• More automation around monitoring and deployment• More stats• Archive Service?
