AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web...

79
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. lIan Ward, Mapbox CTD304 How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide Ian Ward, Mapbox Engineering November 30, 2016

Transcript of AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web...

Page 1: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

lIan Ward, Mapbox

CTD304

How Mapbox Uses the AWS Edge to

Deliver Fast Maps for Mobile, Cars,

and Web Users Worldwide

Ian Ward, Mapbox Engineering

November 30, 2016

Page 2: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Who am I?

Page 3: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

13 people from Mapbox at re:Invent

Check out our booth

Page 4: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

What I'll talk about

Page 5: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

What is Mapbox?

Cost and performance

Configuration

Operations

Security

Page 6: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

What is Mapbox?

Page 7: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

A mapping platform for businesses

Page 8: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 9: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 10: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 11: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Mapbox maps power over 5,000 platforms

ranging from social to mobility apps

Page 12: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Design and data tools

Page 13: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Mapbox Studio

Page 14: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

APIs and SDKs

Page 15: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

/v4/mapbox.streets/z/x/y.png

256 px

256 px 8KB to 25KB

Page 16: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

zoom 15 is

1.1 billion tileszoom 17 is

17 billion tiles

Page 17: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

/geocoding/v5/mapbox.places/-77.03,38.91.json

~ 3 KB

Page 18: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

/directions/v5/mapbox/{profile}/{coordinates}

Page 19: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 20: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Map matching API

Uploads API

Dataset API

Distance API

and more...

Page 21: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

?access_token=$JWT

Page 22: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

API traffic is global

Page 23: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Couple days of

SDK data, 95%

NYC road

coverage

Page 24: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Where is our infrastructure?

Page 25: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

6 to 9 regions

60+ edge locations

Page 26: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Over 200 million MAU

Page 27: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Billions of requests a day

Page 28: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Cost and performance

Page 29: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Started in us-east-1,

with Amazon CloudFront

Page 30: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Client

GLOBAL 9 REGIONS 2 REGIONS

DNS

CDN

Route 53

Cloudfront

DNSRoute 53

ELB

App

servers

Cache

Object store

Database

S3

Dynamodb

Page 31: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Cache

Page 32: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Why use both max-age and s-maxage?

Page 33: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Cost: cache hit versus miss

For 1,000,000 requests

Origin Edge

EC2: $6.24 Requests: $1.00

ELB: $0.80 Transfer: $2.13

RDS: $6.96

Edge: $3.13

Total: $17 Total: $3

Page 34: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

82% cheaper

Page 35: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Performance: cache hit versus miss

Hits are 96% faster

than misses

ex. 12 ms versus 300 ms

Page 36: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Conditional Requests

Origin

ETag

LastModified

CloudFront

If-Match with ETag

If-None-Match with Etag

If-Modified-Since

Page 37: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

1,000,000,000 Conditional GETs

524 bytes at $54

25,000 bytes at $2125

Page 38: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Takeaways

CDN from the start

Know your content

Tune over time

Page 39: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Configuration

Page 40: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

60 behaviors and 13 origins

Page 41: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Order matters.

Page 42: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 43: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Options matter.

Page 44: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Test it. Automate it.

Page 45: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Takeaways

CloudFront has flexible options

Find ways to manage this complexity

Page 46: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Operations

Page 47: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

9 regions

Page 48: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Name Region

origin-maps.mapboxapi.com us-east-1

origin-maps.mapboxapi.com us-west-2

origin-maps.mapboxapi.com eu-west-1

...

Amazon Route 53 latency-based routing

Page 49: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

What could go wrong?

Bad code

Bad deploy

Regional network issues

Page 50: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Automatic failover is not always such

Page 51: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Manual failover is error prone

Page 52: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Automated manual failover

Page 53: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

https://github.com/mapbox/cfn-config

Page 54: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

$ cfn-config update geocoder-production

Page 55: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 56: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Did it work?

Page 57: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 58: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Other ways we check edge status

Page 59: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

CloudFront logs in Amazon Redshift

Page 60: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 61: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 62: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 63: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 64: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Takeaways

Tools for manual processes

Use familiar tools and workflows

Learn from your logs

Page 65: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Security

Page 66: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Scraping and DDoS

Page 67: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Fingerprint your bad traffic

Page 68: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Lots of traffic behind single IP

Unknown proxy

Single user agent

Irregular content profile

Page 69: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 70: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

AWS WAF:

a web application firewall service

Page 71: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

WAF CloudFront

Origin

Good users

Bad users

Page 72: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

How to update?

Page 73: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

$ cfn-update waf-production

Page 74: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)
Page 75: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Single computers and data centers

Page 76: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Takeaways

Ability to detect abnormal traffic

Keep log data accessible

Be familiar with your options to block

Page 77: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Conclusions

Page 78: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Thank you!

Page 79: AWS re:Invent 2016: How Mapbox Uses the AWS Edge to Deliver Fast Maps for Mobile, Cars, and Web Users Worldwide (CTD304)

Remember to complete

your evaluations!