AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox...

60
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Boyd McGeachie, Business Development November 29, 2016 Save Up to 90% and Run Production Workloads on Spot Featuring IFTTT and Mapbox CMP307

Transcript of AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox...

Page 1: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Boyd McGeachie, Business Development

November 29, 2016

Save Up to 90% and

Run Production Workloads on Spot Featuring IFTTT and Mapbox

CMP307

Page 2: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

On-Demand

Pay for compute

capacity by the hour

with no long-term

commitments

For spiky workloads,

or to define needs

Amazon EC2 Consumption Models

Reserved

Make a low, one-time

payment and receive

a significant discount

on the hourly charge

For committed

utilization

Spot

Bid for unused

capacity, charged at a

Spot Price which

fluctuates based on

supply and demand

For time-insensitive

or transient

workloads

Page 3: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

With Spot ,the rules are simple

Markets where the price of compute changes based on

supply and demand.

You’ll never pay more than your bid. When the market exceeds your bid, you get 2 minutes to

wrap up your work.

Page 4: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

$0.27 $0.29$0.50

1b 1c1a

8XL

$0.30 $0.16$0.214XL

$0.07 $0.08$0.082XL

$0.05 $0.04$0.04XL

$0.01 $0.04$0.01L

C3

$1.76

On

Demand

$0.88

$0.44

$.22

$0.11

Show me the markets!

Each instance family

Each instance size

Each Availability Zone

In every region

Is a separate Spot Market

Page 5: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

50% Bid

75% Bid

You pay the

market

price

Bid Price Vs Market Price

25% Bid

Page 6: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

EC2 Best practices

Fault tolerance

for Spot

Stateless Multi-AZ Loosely coupledInstance Flexibility

Page 7: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 8: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Maps

Directions Geocoding

Mobile

Developer tools

Analysis

Page 9: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Mapbox maps power over 5,000

platforms ranging from social to

mobility apps

Page 10: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 11: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 12: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 13: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 14: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 15: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 16: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 17: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

21,014,573,573 probes per week

605,940,678 miles per week

Page 18: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

S3 BucketAmazon

DynamoDBMobile Apps

Amazon

Route 53

Amazon

Kinesis

Processing

Persistence

APIAuto Scaling

groupSpot Fleet

Auto Scaling

groupSpot Fleet

Page 19: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Consumption of compute hours

has increased by 1044% since

last year

Page 20: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

We will do over 500 million

hours of compute this year

Page 21: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

But margins increased

Page 22: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 23: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Q: How much did you have to

change in order to switch to Spot?

Page 24: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

A: Following general best

practices makes it easy to use

Spot Instances.

Page 25: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Best Practices

• Diversify across Availability Zones, regions, and the types of

instances you use to ensure stability

• Reduce your cold start time

• Break up large jobs into smaller pieces

Page 26: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Diversify for stability

• 10 completely isolated regions, multiple Availability Zones, and

multiple instance types

• The Spot market is segmented by these dimensions, so each one

adds an additional layer of protection against price spikes and

interruptions

• Use Spot Fleet: bid on multiple types of instances

• ECS makes it easy to deploy applications across multiple types of

instances

Page 27: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Spotswap

• Manages Spot priceouts for a Spot Fleet by activating backup On-

Demand capacity.

• Polls the termination notification endpoint on each instance and tags

instances that are about to be terminated

• When enough Spot Instances are about to be terminated, we

automatically launch On-Demand instances to pick up the slack

• When Spot prices go back down, we shut down the more expensive

On-Demand instances

Page 28: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 29: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Reduce your cold start time

• Bake software into an AMI or Docker image for fast loading

• Minimize state on the EC2 using S3, DynamoDB, or Amazon Kinesis

• Goal should be to have your application bootstrap in less than 2

minutes

• ECS helps us quickly deploy applications and move them around to

different EC2 instances - seconds instead of minutes

Page 30: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Break up large jobs

Page 31: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Long running job?

Instance 1

15 hours

Page 32: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Break it up!

Instance 1

15 hours

Page 33: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

And finish it faster

Instance 1

Instance 2

Instance 3

5 hours

Use the same number of compute hours, but how many EC2

instances you run decides how quickly the work gets done.

Page 34: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Minimize lost work when there is a failure

Long jobs

Short jobs

Failure

Page 35: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Example: how to break up a text file

• Take a text file with 10 million lines

• Divide number of lines by 10000

• Generate a SQS message for each

chunk of 10000 lines (1000

messages)

• Have your workers start reading at

the specified point in the file

• Deliver output to S3

Page 36: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Impact

• Spot interruptions are relatively rare for the instance types we use,

so the fallback is only triggered a 1-2 times per month.

• We are running on discounted Spot Instances more than 98% of the

time.

• On our maps service alone, this has resulted in an 90% savings on

our EC2 costs each month.

Page 37: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

$$$ Gotchas

Spot enables you to run many more instances that you

normally would, so watch out for any small costs that add

up

Detailed monitoring at 1-minute granularity costs about

$0.005 per hour, but if you do 2 million hours, it adds up to

almost $10K

EBS volumes are still full price on Spot so use instance

store

Page 38: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Amazon EC2 Spot – in the wild

1) We make this easy, using

the Spot Bid Advisor.

2) With deliberate pool

selection and bidding, you

keep your Spot Instance as

long as you need to.

3) And with new features like

Spot Fleet diversified, we do

the heavy lifting for you...

Page 39: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Spot Bid Advisor – aws-spot-labs

Page 40: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Spot Fleet helps you

Launch Thousands of Spot Instanceswith one RequestSpotFleet call.

Get Best PriceFind the lowest priced horsepower that works for you.

or

Get Diversified ResourcesDiversify your fleet. Grow your availability.

And

Apply Custom WeightingCreate your own capacity unit based on your application

needs

Page 41: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

It is easy! aws ec2 request-spot-fleet --spot-fleet-request-config file://config.json {

"IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role", "TargetCapacity":

"100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil":

"2016-09-14T07:00:00Z", "TerminateInstancesWithExpiration": true,

"LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType":

"c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId":

"ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId":

"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large",

"WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId":

"subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge",

"WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId":

"subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge",

"WeightedCapacity": 16, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId":

"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge",

"WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId":

"subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge",

"WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId":

"subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge",

"WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId":

"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge",

"WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }

Page 42: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

An easy to use interface that

lets you launch spare EC2

instances in seconds

Helps you select and bid on the

EC2 instances that meet your

applications requirements

Simple to use dashboard lets

you modify and manage your

application’s compute capacity

EC2 Spot Console

Page 43: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Spot Fleet – Plays by your rules

Page 44: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Spot Fleet – Your preferences

Page 45: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Diversification with EC2 Spot Fleet

Multiple EC2 Spot instances

selected

Multiple Availability Zones

selected

Pick the instances with similar

performance characteristics, e.g.

c3.large, m3.large, m4.large,

r3.large, c4.large

Page 46: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

One connection, countless possibilities

Page 47: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 48: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

43 Million

9.5 Million

360+

1 Billion

80 Million

Applets created

Users on the platform

Services launched

Runs per month

Service activations

Page 49: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT Infrastructure 2014

• Ruby on Rails

• Dedicated web instances

• Dedicated worker instances

Page 50: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT Infrastructure 2014

• Experimenting with Spot bidding for workers

• Spot Auto Scaling groups with fixed bid prices

• Started to depend on Spot capacity

• ~50% savings over On-Demand

• Approach vulnerable to market fluctuations

Page 51: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Enter Spot Fleet

Page 52: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT on Spot Fleet

• Launch worker AMI via Spot Fleet into a mix of markets

• AMI configured to scale based on instance type

• Moved all workers over to Spot Fleet management

Page 53: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Good, but not great...

Page 54: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)
Page 55: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT Infrastructure Now

• All applications in Docker containers

• Containers scheduled by Mesos and Marathon

• Heterogenous (web containers alongside worker, etc.)

Page 56: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT Infrastructure Today

• Reserved instances for Zookeeper & leaders

• Cluster managed entirely by Spot Fleet

• Initially in 48 different Availability Zones

• Mesos instance AMI auto-boots and connects to cluster

• Mesos nodes advertise available resources

Page 57: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT Instance Weights

• Looked at memory and CPU usage across cluster

• Calculated max containers per instance type

• Set instance weights to relative calculations

• Allocation metrics reported to CloudWatch

Page 58: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

IFTTT Results

• Estimated cost of 75% less than On-Demand

• Bin-packing adds extra savings over previous system

• Spot Fleet manages capacity automatically

• Infrastructure abstraction frees up developers

Page 59: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Thank you!

Page 60: AWS re:Invent 2016: Save up to 90% and Run Production Workloads on Spot - Featuring IFTTT and Mapbox (CMP307)

Remember to complete

your evaluations!