Werner Vogels @ FOWA Feb 07
-
Upload
carsonsystems -
Category
Business
-
view
4.079 -
download
0
Transcript of Werner Vogels @ FOWA Feb 07
1
Web Scale Computing:Compete on Ideas, Not Resources
Werner Vogels
CTO – Amazon.com
2
Fast Forward to 33 minutes
3 4
5
What if…
• Launching a new business on the web was simple?
• You only had to focus on “the business”?
• You could manage growth more easily?
6
What if…
• Launching a new business on the web was simple?• You only had to focus on “the business”?• You could manage growth more easily?
What if you only had to compete on ideas?
2
7 8
PUSH VS PULL
• Demand is anticipated
• Top down design
• Centralized design
• Procedural
• Tightly coupled
• Resource centric
• Restricted participation
• Limited re-engineering
• Demand is uncertain
• Emergent design
• Decentralized
• Modular
• Loosely coupled
• People centric
• Open participation
• Rapid incremental innovation
9
Forces Driving Alternative Resource Models
• Increasing Uncertainty
• Growing Abundance
• Intensifying Competition
• Growing Power of Customers
• Greater Focus on Learning and Improvisation
10
DESIGN
DEPLOY
EXECUTE& MONITOR
REFINE
Push Model
11
FIND
CONNECT
INNOVATE
REFLECT
Pull Model
12
Resource Management in an Uncertain World
• Acquire resources on demand
• Release resources when no longer needed
• Pay for what you use
• Leverage other’s core competencies
3
13
A New Web Business using the Push Model
Your IdeaSuccessful
ProductUndifferentiated “Heavy Lifting”
Hardware CostsSoftware CostsMaintenance
Expertise
Load Balancing
Scaling and Physical Growth
Costs to run idle serversBandwidth ManagementServer Hosting
14
The 70 / 30 Switch
30 % of time, energy, and dollars on differentiated value creation
70 % of time, energy, and dollars on undifferentiated heavy lifting
15 16
How can you ever build an application that needs to
• scale to 100% - 1000% increase popularity?
• provide 4 nines uptime?
• survive a complete datacenter failure?
• survive a network partition
• While keeping cost low at the same time?
17
Chapter 4: Priorities – Scale Later – It is too hard to get right
18
The Reliability of Hard Disks
Failure Trends in a Large Disk Drive PopulationEduardo Pinheiro, Wolf-Dietrich Weber and Luiz André Barroso
4
19
Challenges to just get the infrastructure right
• Networks, hardware, operating systems, databases, middleware all fail all the time
• How to beat the CAP theorem
• Datacenters also fail (tornados, heat waves)
• To use scale as a cost-effective advantage you need really large numbers
• Operations is just plain hard, large & small
• Invest intellectual bandwidth to build the infrastructure
• Invest funds for something that may not happen (yet)
20
Resources in the Pull Model: Web-Scale Computing
Scalable Infrastructure that allow your applications to meet infinite demand, cheaply and reliably
• Turn huge fixed costs into variable
• Scale up and down as your business does
• Pay as you go
• Leverage other’s core competencies
• Focus on your Idea
21
Amazon S3, EC2 & SQS
Scalable – Increase or decrease capacity in minutes
Cost-Effective – Low rate, pay-as-you-go
Reliable – Runs on Amazon’s proven infrastructure
Simple – SOAP- and REST-based Web Service Calls
Compatible – Use Amazon EC2 and S3 together to receive free data transmission between services, decreased latency, and consistency.
22
Amazon Simple Storage Service (S3)
• Storage for the internet via web service calls
• Private and public storage options
• $.15 per GB/ per Month to store data
• $.20 per GB/ per Month to transfer data
• Use Cases: Unlimited Data Storage, Media Sharing/ Distribution, Archiving, Server Back-ups
23 24
020406080
100120140
2002 2003 2004 2005 2006
millions ofphotos
Smugmug’s growth
5
25
Smugmug & Amazon S3
• Saved $474,000 in first 7 months of operation
• Pre-S3:– Buying $40K/month of new hardware– Without S3 this would now be $80K/month
• Post-S3:– Selling excess hardware on eBay– Costs even lower than projected
• Expecting savings of $1M to $2M in 200726
27 28
29
S3 Explorer
filicio.us
Jungle Disk
S3 Firefox Organizer MyOwnDB
30
Amazon Elastic Compute Cloud (EC2)
Virtual Computing EnvironmentScalable Capacity – Pay as You Go
$.10 per Server Hour$.20 per GB of data transfer
Use cases: Load testing, Time or Traffic-based Scaling, Simulation and Analysis, Rendering, SaaSPlatform
6
31 32
33 34
Infrastructure:
Amazon Simple Queue Service
Amazon Simple Storage Service
Amazon Elastic Compute Cloud
E-Commerce:
Amazon E-Commerce Service
Amazon Historical Pricing
AWS Product Family
Web Search:
Alexa Web Search
Alexa Web Information Service
Alexa Top Sites
Alexa Site Thumbnail
Workforce / Workflow:
Amazon Mechanical Turk
35
http: / /aws.amazon.com