The Matchchat AWS experienceDave Lavery, CTOJuly 2014

What Is Matchchat?

A commenting plugin for football/sports sites A real time chat/presence service A competition platform An ad server serving 1.4 million users a week A poll application Serves content above/below the fold

Starting Point

Single instance server (LAMP) - not AWS Pusher for real time messages

Growth Curve

Scaling Point 1

Improve response times Solution → move server to EC2 (Ubuntu) MySQL database on RDS (medium)

Scaling Point 2

Support more publishing partners Solution → resize the EC2 server Shutdown the server, resize server, restart There's a limit!

Scaling Point 3

Pusher cost reduction Solution → implement Faye on single Node.js

server (m1 medium)

Scaling Point 4

PHP session for every API call Running out of file handles Running Cron job every 5 minutes to

housekeep sessions Solution → move to Dynamo DB sessions

Scaling Point 5

Scale to support larger partners Solution → multiple EC2 instances with load

balancer in front Abandoned!

Scaling Point 6

Same problem exists No capacity for large scale sysadmin activity Solution → automate with Elastic Beanstalk Multiple load balanced EC2 instances (m1

medium), auto scaling on latency (???) SES (email notifications), Cloudwatch, SNS for

alarms and PHP exceptions External monitoring via Pingdom

Scaling Point 7

Real time messaging response times falling Solution → upgrade Faye server to multiple,

load balanced EC2 instances (medium), backed by Redis database for channel context

Scaling Point 8

High usage of EC2 servers serving static app content

Solution → serve app via S3/Cloudfront

Scaling Point 9

High cost of PHP sessions when only needed for registration

Solution → remove PHP sessions from API calls and reduce Dynamo DB and EC2 usage

Scaling Point 10

Wish to start tracking page visits within our app Solution → increase database size (to m3

xlarge)

Scaling point 11

Reduce MySQL load Reduce I/O blocking in API, aligning latency

with CPU usage Solution → make use of SQS and 'eventual

consistency'

Scaling Point 12

Improve PHP script performance Solution → implement APC cache in Elastic

Beanstalk

Scaling Point 13

More performance improvements to PHP Solution → implement profiling via xhprof/xhgui,

backed by Mongo DB (carefully selected sample rate)

Scaling Point 14

Wish to serve ads to our audience Solution → implement our own MVP ad server More requests per plugin load Solution → serve ad content from CloudFlare

CDN (supports CORS)

Scaling Point 15

Need to improve server creation time when auto scaling

Solution → create Matchchat AMI ready configured to support our app

Future Scaling

Audience increase to 100 million unique visitors monthly

Solution → more usage of SQS, database replication/sharding, Faye scaling, big data

Thank you for your patience!

@dlavery62

uk.linkedin.com/in/dlavery62/