Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails
-
Upload
jonathan-weiss -
Category
Technology
-
view
4.954 -
download
2
description
Transcript of Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails
![Page 1: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/1.jpg)
Utilizing Amazon S3 and EC2in Rails
Jonathan Weiss, Peritor Wissensmanagement GmbHRailsConf Europe Berlin, 2007
![Page 2: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/2.jpg)
2
Starting Point
One machine:
• Apache
• Ruby / Rails
• MySQL
![Page 3: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/3.jpg)
3
Worst Case Populariy
![Page 4: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/4.jpg)
4
A Difficult Path
![Page 5: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/5.jpg)
5
Problem: Backup
• High availability
• Redundancy
• Very big data sets
![Page 6: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/6.jpg)
6
Problem: File System
• Important files have to be accessed by many servers
• NFS / Samba not practical
![Page 7: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/7.jpg)
7
Problem: Spontaneous Traffic
![Page 8: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/8.jpg)
8
Problem: Load Fluctuation
![Page 9: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/9.jpg)
9
Don‘t reinvent the wheel!
![Page 10: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/10.jpg)
10
Amazon Web Services
Simple Storage Service - S3
Redundant Data Store
$0,15 per GB data per month
$0,10 - $0,20 per GB traffic
Virtual server per hour
$0,10 per CPU hour
$0,10 - $0,20 per GB traffic
Elastic Compute Cloud - EC2
…….
E-Commerce S..
![Page 11: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/11.jpg)
11
S3 - Simple Storage Service
• Redundant storage - as much as you like
• max. 5 GB per object
• Organized in „Buckets“
• Web Service API for uploads
• Downloads via
• Web Service
• HTTP / HTTPS
• BitTorrent
![Page 12: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/12.jpg)
12
S3 - Buckets
• Unique over all S3
• Contains manykey-value-metadata tupel
• Cannot contain other buckets!
• Key can contain „/“
MyBucket_name
S3
foto_5.jpg
backups/januar.zip
fotos/2007/001.png
www.peritor.com
site/screen.css
site/script.js
FreeBSD-6.2.iso
![Page 13: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/13.jpg)
13
S3 with AWS::S3Upload
Download
![Page 14: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/14.jpg)
14
EC2 - Elastic Compute Cloud
• Based on XEN virtualization
• On demand virtual servers - controlled with Web Service API
• 1,7 GHz
• 1,75 GB RAM
• 160 GB local file system (not persistent!)
• Use your favorite Linux distro (Linux 2.6.16),Amazon Machine Images (AMI) are stored on S3
• ACLs for hosts/ports access control
![Page 15: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/15.jpg)
15
EC2 ToolsList availabe images
Start a new instance
Login with SSH
Shutdown instance
![Page 16: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/16.jpg)
16
amazon-ec2 gemSetup
Usage
![Page 17: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/17.jpg)
17
And now?
How does this solvemy problems?
![Page 18: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/18.jpg)
18
S3 - Backup
• s3sync.rb
• Brackup
• Jungle Disk
• S3 FUSE
• s3DAV
• Duplicity
• S3Browser
• Firefox S3 Organizer extension
• …
![Page 19: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/19.jpg)
19
s3sync.rb
Backup
Restore
![Page 20: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/20.jpg)
20
S3 Asset Host
![Page 21: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/21.jpg)
21
S3 Asset Host
Setup DNS
Rails configuration
![Page 22: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/22.jpg)
22
S3 Asset Host
welcome.rhtml template
Output
![Page 23: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/23.jpg)
23
S3 - Authenticated User Data
![Page 24: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/24.jpg)
24
attachment_fu Rails plugin
Setup
![Page 25: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/25.jpg)
25
attachment_fu Rails plugin
Upload
Download
![Page 26: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/26.jpg)
26
On-Demand Computing with EC2
Time based, e.g. with cron
![Page 27: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/27.jpg)
27
On-Demand Computing with EC2
Load based, e.g. with Monit
![Page 28: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/28.jpg)
28
On-Demand Computing with EC2
![Page 29: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/29.jpg)
29
EC2 for extra capacity
![Page 30: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/30.jpg)
30
Load Balancer / Proxy
Example mod_proxy_blancer:
• Talks to multiple backend servers(Mongrel)
• Central Proxy/Load-Balancerconfiguration that has knowledgeabout nodes
• Typically proxy restart on configchange
![Page 31: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/31.jpg)
31
Swiftiply
Swiftiply Proxy:
• Multiple backend clients have apersistent connection to the backendport
• Proxy listens on cluster port forrequests and forwards them
No re-configuration
![Page 32: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/32.jpg)
32
Swiftiply Proxy
Configuration
Start
Installation
![Page 33: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/33.jpg)
33
Swiftiplied Mongrel
• Gem plugin that patches Mongrel
• Transforms Mongrel in Swiftiply client
• Experimental
Start
or
![Page 34: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/34.jpg)
34
EC2 on Demand before Swiftiply
![Page 35: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/35.jpg)
35
EC2 on Demand with Swiftiply
![Page 36: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/36.jpg)
36
EC2 on Demand with Swiftiply
![Page 37: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/37.jpg)
37
Ressources
• Amazon Web Serviceshttp://aws.amazon.com
• Swiftiplyhttp://swiftiply.swiftcore.org
• Attachment_fuhttp://svn.techno-weenie.net/projects/plugins/attachment_fu/
• Elastic Railshttp://www.elasticrails.com
• Capazonhttp://capazon.rubyforge.org
• RightScalehttp://www.rightscale.com
![Page 38: Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails](https://reader034.fdocuments.in/reader034/viewer/2022052618/55493477b4c905054d8b459a/html5/thumbnails/38.jpg)
38
Peritor Wissensmanagement GmbH
Lenbachstraße 212157 Berlin
Internet: www.peritor.comEmail: [email protected]
© Peritor Wissensmanagement GmbH - All Rights Reserved
Telefon: +49 (0)30 69 40 11 94Telefax: +49 (0)30 69 40 11 95