Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer...

36
Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb

Transcript of Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer...

Page 1: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Running Your Startup on Amazon Web Services

Alex Iskold

Founder/CEO AdaptiveBlue

Feature Writer ReadWriteWeb

Page 2: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Amazon Web Services (AWS) - robust, scalable and affordable

infrastructure for cloud computing.

This session is about:

Page 3: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

We will cover:

The Basics of Cloud Computing The Benefits of Using Amazon Web Services Amazon Web Services Stack How AdaptiveBlue is using AWS How you can get started using AWS

Page 4: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Basics of Cloud Computing

Page 5: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Basics of Cloud Computing

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

Scalable Hardware Layer

Page 6: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Basics of Cloud Computing

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

Scalable Hardware Layer

Software Infrastructure Layer

GridService

StorageService

Queue Service

Page 7: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Example: Storage Service

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

StorageService

StorageService

StorageService

StorageService

New Server

The data is automaticallyre-partitioned/re-balanced totake advantage of the new server

Page 8: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

5 Benefits of Using Amazon Web Services

Page 9: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

1. Pay-per use model

You are only charged for disk space, CPU time and bandwidth that you use.

Page 10: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

2. Instant scalability

Your Service automatically scales on AWS stack.

Page 11: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

3. Reliable/Redundant/Secure

Infrequent outages (so far).Data is redundant in the cloud.

All services have built-in security.

* Soon will add Comprehensive to the list.

Page 12: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

4. Most services accessed via simple REST/SOAP API

Libraries are available in all major languages.Minimal learning curve.

Page 13: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

5. Amazon - Experience & Commitment

Amazon has been doing this for ~15 years. Company-wide commitment to AWS.

Page 14: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Amazon Web Services Stack

Page 15: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Simple DatabaseService (SimpleDB)

Elastic Cloud Service (EC2)

Simple StorageService (S3)

Simple QueueService (SQS)

Page 16: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Elastic Compute Cloud (EC2)

On-demand grid

Page 17: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

EC2 in a Nutshell

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

MachineImage

(OS + Apps)

Usage:

• Create Machine Image• Deploy the image to S3• Start 1 or more instances• Use it as regular machine(s)

Main Options:

• Dynamic/Static IPS• Choose cores• Choose locations• Persistence via EBS

Page 18: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Sample EC2 Use CasesBatch Processing

All instances are configured with the same code. Each instance operates on a subset of data. Partitions are specified in configuration file.

Web Service

All instances are configured with the same code. One or more instances are configured as load balancers (HAProxy for

example). DNS Server distributes requests between load balancers.

Page 19: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

EC2 vs. Web Hosting Company

Good

Instantly add new instances Full-control over the machines and choice of the environment Likely cheaper (but depends on your exact situation)

Bad

Need to put the images together and manage instances No dedicated technical support (but there is premium support and RightScale solutions)

Page 20: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Simple Storage Service (S3)

Large media and everything else storage.

Page 21: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

S3 in a Nutshell

Client

Idea:

Put/Get objects into bucketsbased on unique keys.

Main Features:

• Public/Private access.• Support for large objects.

Amazon S3

Bucket 1 Bucket N…

Put object Get object

Page 22: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Sample S3 Use CasesImage/Video storage

Put your media once on S3 and then serve it up Reads are 10 times cheaper than writes!

Serialize your Java Objects

Define unique key based on the object attributes Write out binary serialized version to a stream Write bytes to S3 Read them back when needed

Page 23: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Simple Database Service (SimpleDB)

Basic database and indexing service

Page 24: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Simple DB in a Nutshell

Client

Idea:

Create flat database with auto-indexed tables.

Main Features:

• Each attribute is indexed.• Record structure is flexible.• Basic operators in queries• Supports sorting.

Simple DB Domain

Record 1

Put recordGet recordQuery records

Key1 Attributes: A1,A2…

Record N

Key2 Attributes: A1,A2…

Page 25: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Sample SimpleDB Use Cases

Index Media files stored on S3

Use the same key as on S3 Write the record with each metadata element as attribute

Store flat objects

Use SimpleDB as a storage for non-nested data

Page 26: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Simple Queue Service (SQS)

Infinite Asynchronous Queue

Page 27: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

SQS in a Nutshell

Writer

Idea:

Create an infinite asynchronous queue.

Main Features:

• Multiple queues• Upto 4K messages• Message Locking

SQS Queue

Message 1

SendMessage

ReceiveMessage

Message N

Reader

Page 28: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Sample SQS Use Cases

Twitter Friend Update

For each update generate a task to update friends Process updates in order

Publish/Subscribe

Post messages to the queue to inform multiple subscribers

Process Pipeline

Use different queues to put, for example, and order through a pipeline.

Page 29: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

How AdaptiveBlue usesAmazon Web Services

Page 30: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

AdaptiveBlue Overview

We develop browser enhancement technologies that deliver contextual relevant information into the user current page:

QuickTime™ and a decompressor

are needed to see this picture.

1. What did your friends and other people think about it?

2. How popular is this book right now?

3. What are additional relevant links for this book?

Page 31: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

AdaptiveBlue & AWS Background

One of the first companies to use AWS Started with using S3 Now using S3, SimpleDB, EC2 and

Commerce Service

Page 32: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

AdaptiveBlue AWS Architecture

Web Browser 1 Web Browser N…

SimpleDB

EC2

S3

WS1

LB

WSn WS1

LB

WSn

Page 33: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

AdaptiveBlue & AWS

On EC2: We are running web service and load balancers

On S3: We are storing XML representation of books, music, movies,

stocks, wines, etc. as well as XML of user profiles.

On SimpleDB: Records of interactions between users and things they are looking

at, liking or commenting. Records that map user ids to other services Records that map between our internal object keys and URLs for

this object around the web.

Page 34: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Noteworthy usage

We turn SimpleDB into relational database via redundancy.

We have person and object domains. Each time when the user interacts with an object we write a record with USERNAME/OBJECT_KEY pair into 2 domains - one object domain, one person domain.

This way we can instantly retrieve recent objects for a person and recent people for each object.

Page 35: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

AdaptiveBlue Recommends AWS

We love Amazon Web Services and couldn’t have built AdaptiveBlue without them.

Thanks to AWS we scale today to hundreds of thousands of users .

Our current costs are < 2K per month.

Amazon team has been nothing less than outstanding in supporting us and helping us architect correct solutions.

We recommend AWS to everyone who asks, because we believe that it offers unprecedented head start, scalability and savings.

Page 36: Running Your Startup on Amazon Web Services Alex Iskold Founder/CEO AdaptiveBlue Feature Writer ReadWriteWeb.

Amazon Web Services Resources

All you need - basic info, getting started, pricing, tutorials, code,

forums - all on AWS home page:

http://tinyurl.com/ynfsn6