Leveraging Elastic Web-Scale Computing with AWS

Leveraging Elas-c Web-‐Scale Compu-ng with AWS Shiva Narayanaswamy

EC2 Basics Instance Lifecycle EC2 Instance Types Using Amazon Machine Images Bootstrapping EC2 Instances Monitoring EC2 with CloudWatch Autoscaling

v

EC2 Basics

Virtual Servers in the Cloud •  One instance to thousands of instances •  In any public AWS region •  Create, start, stop, configure, monitor as desired •  Install any software: web, business, client/server, batch processing •  Pay only for capacity you use •  Variety of cost models Amazon EC2

v

EC2 Basics: cost models

On-‐Demand Reserved Spot Dedicated

Pay upfront in exchange for hourly prices that are 50-‐75% lower than

On-‐Demand

Pay for compute capacity by the hour. No long-‐term

commitments

Bid for unused Amazon EC2 capacity

Launch instances in VPC on dedicated customer hardware

Customers can combine mul1ple purchase types to op1mize pricing based on current and forecast capacity needs.

Spiky workloads CommiRed uSlizaSon Time-‐insensiSve workloads Highly sensiSve workloads

v

Provisioning and Lifecycle

• Create -> Start -> Stop -> Terminate • Manually in console • Automate via API (or other tools) • Automatically based on demand

(demand curve)

v

Instance Types GPU

Enabled General Purpose

Storage and IO OpSmized

Compute OpSmized

Memory OpSmized

M3 C3 I2

CG1 M1 C1 CR1 CC2 HI1 HS1

G2

M3 C3 I2 HS1

M2

R3 G2

Added Instance

Types

v

Amazon Machine Images

Your machine images

AMIs you have created from EC2 instances Can be kept private or shared with other

accounts

Amazon maintained

Set of Linux and Windows images Kept up to date by Amazon in each

region

Community maintained

Images published by other AWS users Managed and maintained by Marketplace

partners

v

Amazon Machine Images

v

Bootstrapping: metadata and userdata

•  Every EC2 Instance has access to local instance metadata and userdata service

Instance request

User data

Instance

Meta-‐data service

v


• Metadata: immutable informa1on about the instance • Accessible from within the instance via HTTP at hQp://169.254.169.254/latest/meta-‐data/

•  Script(s) on instance may retrieve useful informa1on about the instance, such as:

•  Host name •  AMI ID •  Instance ID •  Public/Private DNS •  Availability Zone

v


• User Data: pass up to 16KB of text to an instance on launch • Accessible from within the instance via HTTP at hQp://169.254.169.254/latest/user-‐data/

•  Text can be parsed by script on instance and used to configure the machine

v

Custom script on AMI

(script_runner.py) fetches userdata,

parses it, and configures EC2 Instance

on boot


v • CloudInit executes UserData on first boot if UserData begins with: •  #! (Linux) •  <script> (Windows; technically, EC2Config, not CloudInit, does this)

• CloudInit is installed on Amazon Linux, Ubuntu, and RHEL AMIs •  EC2Config is installed on Windows Server AMIs • Both may be installed on other distribu1ons via a package repo or source

Bootstrapping: UserData and CloudInit

v • UserData to install Apache and MySQL on boot, and aQach an EIP:

#!/bin/bash # Install Apache, PHP, and MySQL yum install –y httpd mysql-server # Attach an Elastic IP to this instance ec2-associate-address \ 23.34.45.56 \

-i $(curl http://169.254.169.254/latest/meta-data/instance-id)

Bootstrapping: UserData and CloudInit

v

Bootstrapping

Bake an AMI

Start an instance

Configure the instance

Create an AMI from your instance

Start new ones from the AMI

Configure dynamically

Launch an instance

Use metadata service and cloud-‐init to perform ac1ons on

instance when it launches

Use config management tools like Puppet/Chef/Opsworks

vs

v

Bootstrapping

Bake an AMI Configure dynamically

Build your base images and setup custom ini1alisa1on

scripts

Maintain your ‘golden’ base

Use bootstrapping to pass custom informa1on in and perform post launch tasks.

+

Sweet spot

v

Bootstrapping: AMIs

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Java App Stack

Example full stack required to run your

application.

Let’s use the 3 bootstrapping

techniques

v

Bootstrapping: AMI bake

Fully-functional AMI is pre-build and

ready to launch from the AMI inventory

Inventory of AMIs

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Amazon EC2

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Java AMI

v

Bootstrapping: Configure dynamically

Base OS AMI

An AMI with minimal components (OS,

J2EE, and Chef/Puppet) is launched.

All configuration occurs via Chef/

Puppet after instance launch

Inventory of AMIs

Amazon EC2

OS AMI

Fetch on boot

Linux

JEE

Your Code

S3

Hibernate

Tomcat

Log4J

Spring

Struts

Apache

Linux

JEE

Linux

JEE

Chef/Puppet

Chef/Puppet

scripts

v

Bootstrapping: Sweet spot

Partially-configured AMI

A “Golden Image” is launched, with

scripts fetching/installing app code

and other supporting components on

boot

Inventory of AMIs

Amazon EC2

Java AMI

Your Code

S3

Log4J

Spring

Struts

Linux

JEE

Hibernate

Tomcat

Apache

Fetch on boot

Fetch on boot

Linux

JEE

Hibernate

Tomcat

Apache

Linux

JEE

Hibernate

Tomcat

Apache

Linux

JEE

Hibernate

Tomcat

Apache

Linux

JEE

Hibernate

Tomcat

Apache

Why do this?

Automa1on Less fingers, less mistakes

Availability Drive higher

availability with self-‐healing

Security Instances locked down by default

Flexible Shell,

Powershell, CloudForma1on,Chef, Puppet, OpsWorks

Scale Manage large scale

deployments and drive autoscaling

Efficiency Audit and manage your estate with less 1me & effort

Do Don’t

Some dos and don’ts

Use IAM roles

Go keyless if you can

Strike a balance between AMI and dynamic bootstrapping

Put your API access keys into code (and then publish to GIT) or bake

into AMIs (and share)

L

v

Monitoring EC2 with CloudWatch

v

Types of Scaling

• Ver1cal Scaling •  Changing instance size •  Increasing EBS Capacity

• Horizontal Scaling •  Adding / removing instances •  ELB •  Autoscaling

r3.8xlarge

c3.2xlarge

m3.medium

m3.medium m3.medium m3.medium

m3.medium m3.medium m3.medium

v

Ver-cal Scaling

• Different EC2 instance type •  High memory instances •  High CPU instances •  High I/O instances •  High storage instances

•  Easy to change instance sizes • Will hit an endpoint eventually •  Requires instance to be stopped

r3.8xlarge

c3.2xlarge

m3.medium

v

Tradi-onal IT Usage PaJerns

On and Off Fast Growth

Variable peaks Predictable peaks

v

Tradi-onal IT Usage PaJerns



Poor Service

WASTE

v

Cloud IT Usage PaJerns (Auto Scaling)



v

Auto Scaling •  Automa1c resizing of compute clusters based on demand •  Define minimum and maximum number of instances •  Define when scaling out and in occurs •  Use metrics collected in Amazon CloudWatch to drive scaling •  Run Auto Scaling for On-‐Demand and Spot instance types •  Its Free! Amazon

CloudWatch

Usage Metrics

Scaling Instruc1ons

Auto Scaling Group

Queue Metrics

Auto Scaling

Describes what Auto Scaling will create when adding

Instances -‐ Similar to ec2-‐run-‐instances API command

AMI

Instance Type Security Group Instance Key Pair

Only one ac1ve launch configura1on at a 1me

Auto Scaling will terminate instances with old launch

configura1on first rolling update

Auto Scaling managed grouping of EC2 instances

Automa1c health check to maintain pool size

Automa1cally scale the number of instances by policy – Min, Max,

Desired

Automa1c Integra1on with ELB

Automa1c distribu1on & balancing across AZs

Parameters for performing an Auto Scaling ac1on

Scale Up/Down and by how much

ChangeInCapacity (+/-‐ #) ExactCapacity (#)

ChangeInPercent (+/-‐ %)

Cool Down (seconds)

Policy can be triggered by CloudWatch events

Launch Configuration Auto-Scaling Group Auto-Scaling Policy

v

Scaling plan

•  Scale based on demand • Manual scaling •  Scale based on schedule • Maintain current instance levels at all 1me

Auto Scaling

v

Auto Scaling Lifecycles

v

Autoscaling

v

Availability Zone A Availability Zone B

Autoscaling: Auto Scaling Group

v Latency

CloudWatch Auto Scaling

ELB

Auto scaling Group

Autoscaling: ELB + CloudWatch

v •  Tools Used:

•  CloudForma1on script – •  Create a mul1-‐AZ, load balanced and Auto Scaled sample web site running on an Apache Web Server (m1.small). The applica1on is configured to span all Availability Zones in the region and is Auto-‐Scaled based on the CPU u1liza1on of the web servers.

•  Bees with Machine Guns – Performance tes1ng tool •  A cloudforma1on script that spins up a dis1buted performance tes1ng tool based on apache eb tool. This tool will hit the ELB with 1000’s of concurrent requests for a total of 100’s of thousands of request, thus loading the web server behind the ELB.

•  Expected result •  The Apache web server will scale to serve traffic without any customer impact.

Autoscaling: DEMO

v •  CloudForma1on script (Auto scaling apache web server) •  Auto-‐scaling group configura1on:

•  Min: 1 •  Max: 3 •  Cooldown: 300

•  Scaling Policies: •  Scaling Up:

•  CPU U1liza1on > 20% for 1 consecu1ve period of 60 seconds •  Ac1on: Add 1 instance •  Then wait: 60 seconds before next opera1on

•  Scaling Down: •  CPU U1liza1on < 10% for 2 consecu1ve periods of 60 seconds •  Ac1on: Remove 1 instance •  Then wait: 60 seconds before next opera1on

•  Bees with Machine guns(NASTY)

Demo Information

v

Autoscaling isn’t one size fits all

•  Choose the right metrics •  CPU Usage •  Queue Depth •  Number of concurrent users

•  Scale too aggressively •  Overprovisioning: increases costs •  Bounciness: Add more than you need and have to par1ally scale back shortly aser scaling up, increasing costs.

•  Scale too 1midly •  Poor performance •  Outages due to lack of capacity

•  Scale out early / Scale in slowly

Stop doing these: Provisioning and fixing servers

Trea1ng compute as physical things Thinking of compute as a finite commitment

and start doing these

Security Build systems secure by

default

Elas1city Stateless autoscaling

applica1ons

Replace not fix Build from scratch, don’t

fix something Unconstrained Say goodbye to

tradi1onal capacity planning

Be cost aware Tag resources, play with

instance types

Automa1on Create instances when you need them, drop

them when not

v

What’s more?

• AQach / Detach Instances from Auto Scaling Groups • Place instances into Standby State to Troubleshoot • Hold instances in Pending state for installing sosware / retrieve logs • Create an Auto Scaling Group / Launch Configura1on based on a running instance

• Auto scaling Lifecycle hooks

v

QuesSons?

Leveraging Elastic Web-Scale Computing with AWS

Technology

Transcript of Leveraging Elastic Web-Scale Computing with AWS