HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD

Using Amazon's Elastic Computing Cloud ( and a few too many Star Wars analogies) to create a scalable server infrastructure for School of Everything.

David O'Dwyer (SupportFour) Peter Brownell (Code Positive / School of Everything)

Dr Clonelove:

WHAT'S AHEADHow we built a server cluster on Amazon EC2 for a Drupal based website.It's all about how to add and remove servers on the fly.

The set upWhy use the cloud ?An overview of our server infrastructure

How we got started with the cloudSome sysadmin religious mumbo-jumboGeneral best-practice for any cluster

TechnologyThe key tools to make it happen

Image by - http://www.flickr.com/photos/powerpig/

THE SILVER LININGWhy choose the cloud ?

Startups are poorPay for servers by the hourAdd a new server in a few minutesTurn them off when not needed

http://flickr.com/photos/balakov/2468552226

SCALE DOWNBe able to grow and shrinkThe key word really is “Elastic”Know how you will add and remove new servers

Photo credit: http://www.flickr.com/photos/34558813@N07/

BASIC STRUCTURECode built on DrupalMachine images run Debian LennyMySQL database with Replication & MySQL proxyMany custom scripts

TIME FOR AN EYE TESTTIME FOR AN EYE TEST

TIME FOR AN EYE TESTTIME FOR AN EYE TEST

THE BIG PICTURE

WHAT WE BUILTBasic configuration – grow as needed

2 web-servers (small instance) 1 primary database (small instance) 2 slave databases 2 off-cloud machines for monitoring and backups

EVERYONA MAKES MISTAKESSimple concepts to avoid ultimate disasta

THE ENDWork Backwards

Non-persistence – all changes lost on power downMust be able to rebuild everything - automaticallyishStart with your plan for complete failureMake sure you can test your backup system

ULTIMATE FAILURE

Be able to rebuild from backupsDocument and blogFire Drills – practice makes perfect

Assume that everything may just vanish

CLOUD CITYNever have faith in just one allySet up a base off the cloud Keep a backup on the groundMonitor from outsideMail smarthost to avoid auto greylisting

IT'S A TRAP!What does it look like – working or not.

Don't trust anything! Monitor as much as possible - Clues for where problems arise - Capacity planning

We use Nagios and Munin.

NOAH'S ARK PRINCIPLE:Expect the worst and have two of everythingTraditionally expensiveAdding new servers takes timeYou may just end up with a lot of hardware doing nothing

http://www.thebricktestament.com/genesis/the_flood/gn07_07_10.html

IT'S MUCH EASIER WITH CLONES

Configure one original, and make many copies, in multiple zonesOnly pay for the time you useNoah says: “It's not having two of everything running, it's being able to”

http://www.flickr.com/photos/jeremymates/2362399109/

Virtual machines are cheap – but you still need to plan

THE PIECESPutting the cluster together

STORAGE YOU CAN RELY ONHow safe is your data Local disk – EC2 Ephemeral Storage (/mnt)EBS - Elastic Block Store Point-in-time snapshots

CLONE 2 CLONE FILESYSTEM

Avoid dependencies

Local copy of uploaded files GlusterFS syncs files instantlyPrimary webserver uses persistent storage

Image by - http://www.flickr.com/photos/powerpig/

Gluster File System

SMALLER IS BETTER

Database replication – easy and reliableMultiple small images as database serversMySql proxy makes cluster transparent

MySQL(tm) is MyFriend

http://www.flickr.com/photos/powerpig/MySQL is a registered trademark of MySQL inc, no it's Sun Microsystems, no, woops looks like it's Oracle Inc. now. How weird is that ? Anyway, we like it, and are happy to endorse them – however owns them.

Best practice guide: http://www.mysql.com/why-mysql/white-papers/mysql_wp_best_practices_mysql_ec2.php

THE BALANCING ACTRound Robin DNS is the cheap solution for load balancingEach machine configures dynamic DNS on first startupInvestigating new amazon load balancing

Image by - http://www.flickr.com/photos/powerpig/

WHAT WE SAIDCloud infrastructure is not alien, it's just a little less solid – it can grow or shrink.The cloud forces best practice – no time for slacker sysadmins!

Better safe than sorryRemember Noah, avoid dependenciesWork backwards – rebuild, restore, backup, operateKeep a base in the real worldMonitor everything

Managing growthLoads of local storage, persistent elastic blocks, instant snapshots and S3Synchronise user uploads with GlusterFSScale MySQL with replication and MySQL ProxyGrowing number of load balancing options

MailMail smarthost off-cloud to avoid greylisting

Image by - http://www.flickr.com/photos/powerpig/

http://supportfour.com

http://schoolofeverything.com/

Want to learn more about Amazon Web Services ? Sign up on School of Everything and tag yourself with AWS.

http://schoolofeverything.com/subject/AWS

The Church of Jar Jar paid a visit to the Star Wars Expo in London. They sang and danced, the cast all queued up for a picture with us. Then they were thrown out.

Your humble presenters