Flying Through the Clouds: Piloting in the World of AWS

Presented by Eric Tendian

Eric Tendian@EricTendian/in/erictendianERIC@TENDIAN.IO

ABOUT ME

Chief Technology Officer for YSFlight Headquarters

Web Solutions Consultant for Tendian.io

Hacker, tinkerer, developer

AVIATION

YSFLIGHT HEADQUARTERS

YSFLIGHT HEADQUARTERS

Online community for flight simulator pilots

Discussion forums

Based around the game YSFlight

YSFHQ.com

OUR PHILOSOPHY

Make the users work to get what they want

Bugs are fun!

Approach the cloud with caution

Windows Server FTW!

JUST KIDDING.

Our real philosophy:

Give the user the best experience, always

Pilots want fast & free

Fix, test, deploy

Use cloud whenever possible

What it really looks like

HOW DID WE ACCOMPLISH THIS?

AMAZON WEB SERVICES

http://www.youtube.com/watch?v=jOhbTAU4OPI

OUR SETUP

One EC2 micro instance

One RDS micro instance

Multiple S3 buckets

Cloudflare DNS

WHAT WE LEARNED

Start small

Make use of all AWS resources

Cloud is $$$ when managed poorly

Developers love AWS

HOW CAN I USE AWS?

Development machine(s)

Testing on the cloud

Side projects

Startups

Static website

EXCITED? LET’S GET STARTED.

Getting Started with AWSEric TendianWeb Solutions Consultant, Tendian.io

Launching aninstance

Region

Regions

Regions Region

US-WEST (N.California) EU-WEST (Ireland)

ASIA PAC(Tokyo)

ASIA PAC(Singapore)

US-WEST (Oregon)

SOUTH AMERICA (SaoPaulo)

US-EAST (Virginia)

GOV CLOUD

ASIA PAC(Sydney)

Wizard

Launch Wizard

Choose operating system

Launch Wizard

Launch!

Launch

Launch Confirmation

InstanceDNS name

Public Address

Instance DNS name

SSH

SSH

EC2

Compute

Vertical ScalingFrom $0.02/hr

Elastic Compute Cloud (EC2)Basic unit of compute capacityRange of CPU, memory & local disk options18 Instance types available, from micro to cluster compute

Feature Details

Flexible Run Windows or Linux distributions

Scalable Wide range of instance types from micro to cluster compute

Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created

Full control Full root or administrator rights

Secure Full firewall control via Security Groups

Monitoring Publishes metrics to Cloud Watch

Inexpensive On-demand, Reserved and Spot instance types

VM Import/Export Import and export VM images to transfer configurations in and out of EC2

256

128

64

32

16

8

4

2

11 2 4 8 16 32 64

EC2 Compute Units128 256

EC2 instance types

High I/O 4XL 60.5 GB35 EC2 Compute Units16 virtual cores2*1024 GB SSD-based local instancestorage

Mem

ory

(GB)

Small 1.7 GB,1 EC2 ComputeUnit1 virtual core

Micro 613 MBUp to 2 ECUs (forshort bursts)

Large 7.5 GB4 EC2 Compute Units2 virtual cores

Hi-Mem XL 17.1 GB6.5 EC2 ComputeUnits2 virtual cores

Hi-Mem 2XL 34.2 GB13 EC2 Compute Units4 virtual cores

Hi-Mem 4XL 68.4 GB26 EC2 Compute Units8 virtual cores

High-CPU Med 1.7GB5 EC2 ComputeUnits2 virtual cores

High-CPU XL 7 GB20 EC2 ComputeUnits8 virtual cores

Medium 3.7 GB,2 EC2 ComputeUnits1 virtual core

M3 XL 15 GB13 EC2 Compute Units 4 virtualcoresEBS storage only

M3 2XL 30 GB26 EC2 Compute Units 8 virtualcoresEBS storage only

Extra Large 15 GB8 EC2 ComputeUnits4 virtual cores

Cluster GPU 4XL 22 GB33.5 EC2 Compute Units,2 x NVIDIA Tesla “Fermi”M2050 GPUs

Cluster Compute 4XL 23 GB33.5 EC2 Compute Units

Cluster Compute 8XL 60.5GB88 EC2 Compute Units

High Storage 8XL 117 GB35 EC2 Compute Units,24 * 2 TB ephemeraldrives10 GB Ethernet

Hi-Mem Cluster Compute 8XL244 GB88 EC2 Compute Units16 virtual cores240 GB SSD

EC2 instance types

Light Spiky

EC2 Compute Units

Mem

ory

(GB)

SpecialStorage

Instanc e

AMI

Amazon Machine Image

Running or Stopped machine

AZ

EBS EBS EBS

Availability Zone

EBSSnapshots

S3

EBS EBS EBS

S3 Buckets

Region

EC2 terminology

More details!

Sign up:aws.amazon.com

Sign up

1 2 3 4 5

5Sign up

1 2 3 4

5Sign up

1 2 3 4

4 5Sign up

1 2 3

You will needCredit card information – you won’t pay unless you use resources

A telephone – on which to receive an automated security call

3 4 5Sign up

1 2

You will needCredit card information – you won’t pay unless you use resources

A telephone – on which to receive an automated security call

Best practiceSetup billing alerts so you can be notified when levels of spend arereached

If you have existing accounts, consider using consolidated billing to bringthem together under one payment

Sign up

1 2 3 4 5

750 hours of Amazon EC2 Linux/RedHat/Suse Micro Instance usage750 hours of Amazon EC2 Microsoft Windows Server Micro Instance usage 750 hours of an Elastic Load Balancer30 GB of Amazon Elastic Block Storage 5 GB of Amazon S3 standard storage100 MB of storage, 5 units of write capacity, and 10 units of read capacity for Amazon DynamoDB* 25 Amazon SimpleDB Machine Hours and 1 GB of Storage1,000 Amazon SWF workflow executions*1,000,000 Requests of Amazon Simple Queue Service*1,000,000 Requests, 100,000 HTTP and 1,000 email notifications for Amazon Simple Notification Service* 10 Amazon CloudWatch metrics, 10 alarms, and 1,000,000 API requests*15 GB of bandwidth out aggregated across all AWS services750 hours of Amazon RDS for SQL Server Micro DB Instance usage20 GB of RDS database storage10 million RDS I/Os20 GB of backup storage for your automated RDS database backups and any user-initiated DB Snapshots 20 minutes of SD transcoding or 10 minutes of HD transcoding in Amazon Elastic Transcoder*

Free tier http://aws.amazon.com/free/

2 3 4 5Sign up

1

Sign up

1Key pairs

2 3 4 5

Logging in to an instance

Public Key

Inserted by Amazon into each EC2 instance that

you launch

Private Key

Downloaded and stored by you

Standard SSH RSA Key pair

Public/Private Keys

Public key provided by AWS to EC2 instance for secure, personalized, initial, non-generic access

Supports NIST and other security standards for providing non-default user access

Instance key pairs

EC2Instance

Comms securedwith private key

4 5Sign up

1Key pairs

2 3

Public Key

Inserted by Amazon into each EC2 instance that

you launch

Private Key

Downloaded and stored by you

Instance key pairs

EC2Instance

Comms securedwith private key

Private keys are not stored by AWS

Standard SSH RSA Key pair

Public/Private Keys

Public key provided by AWS to EC2 instance for secure, personalized, initial, non-generic access

Supports NIST and other security standards for providing non-default user access

Sign up

1Key pairs

2 3 4 5

AWS generated keysSelect your region

Create keys

Give them a name

Private key is generated and downloaded by your browser immediately

Create 1 key pair for all resources or as many as you like (e.g 1 per server type)

Import your own keys

You supply only the public key to AWS

Sign up

1Key pairs

2 3 4 5

ssh –I eu-west.pemec2-user@publicdns.amazonaws.com

1. Linux Launch (First Boot)

1. Instance initialization scripts insert publickey into ~/.ssh/authorized_keys

2. User connects with SSH using their PrivateKey

Sign up

1Key pairs

2 3 4 5

ssh –I eu-west.pemec2-user@publicdns.amazonaws.com

2. User connects with SSH using their PrivateKey

You can’t log into a Linuxinstance without key

3 4 5Sign up

1Key pairs

2

1. Linux Launch (First Boot)

1. Instance initialization scripts insert publickey into ~/.ssh/authorized_keys

ssh –I eu-west.pemec2-user@publicdns.amazonaws.com

Don’t lose it

3 4 5Sign up

1Key pairs

2

1. Linux Launch (First Boot)

1. Instance initialization scripts insert publickey into ~/.ssh/authorized_keys

2. User connects with SSH using their PrivateKey

1. Windows Launch (First Boot Sequence)

2. Instance initialization scripts:

a) Creates a random Administrator password

b) Encrypts random password with Public Key

c) Reports encrypted password to Windows System Log

3. User retrieves the encrypted password and decrypts it with their Private Key (usingAWS Console or API Call)

Sign up

1Key pairs

2 3 4 5

Choose key pair when launching instance

4 5Sign up

1Key pairs

2 3

Keepsecure

Do notshare

Rotate Need toknow

4 5Sign up

1Key pairs

2 3

Sign up

1Key pairs

2 3 4 5

Allowing accessto the instance

Sign up

1Key pairs

2Access

3 4 5

sudo yum -y install httpd

chkconfig httpd onsudo

sudo /etc/init.d/httpd start

Let’s install something

Install apache web server

Set it to run as a service

Start the web server

Sign up

1Key pairs

2Access

3 4 5

Security groups

Security Group

EC2 Classic EC2 VPC (virtual private cloud)

Inbound only Inbound and outbound

TCP, UDP, ICMP only Any protocol

Assigned at launch Assigned at launch or when running

Modify anytime Modify anytime

instance

Port 80(HTTP)

Port 22(SSH)

Name Description Protocol Port rangeIP Address, range, or another security group

Sign up

1Key pairs

2Access

3 4 5

Security groups

Added port 80to group

Open our security group

Sign up

1Key pairs

2Access

3 4 5

Test it by hitting the public DNS name of the instance

Sign up

1Key pairs

2Access

3 4 5

Sign up

1Key pairs

2Access

3 4 5

Reuse your instance!

Sign up

1Key pairs

2Access

3Image

4 5

Makes a snapshot of the instance

Creates an image that is private to you

Saves time in deployments and system setup

Sign up

1Key pairs

2Access

3Image

4 5

Create image

Sign up

1Key pairs

2Access

3Image

4 5

Name it and

create

Sign up

1Key pairs

2Access

3Image

4 5

Your AMI

Sign up

1Key pairs

2Access

3Image

4 5

…and launch a

new instance from the

AMI

Sign up

1Key pairs

2Access

3Image

4 5

Sign up

1Key pairs

2Access

3Image

4 5

Who can startan instance?

Sign up

1Key pairs

2Access

3Image

4IAM users

5

Identity and Access Management:

Securely control access to AWS services and resources for your

users

Sign up

1Key pairs

2Access

3Image

4IAM users

5

Account ownerAccess to all subscribed services Access to billing reportsAccess to console, REST and SOAP APIs

IAM users/groups

Access to specific servicesAccess to console and/or REST APIs and/or SOAP APIs

Sign up

1Key pairs

2Access

3Image

4IAM users

5

Account ownerAccess to all subscribed services Access to billing reportsAccess to console, REST and SOAP APIs

Regular usersIAM users/groups

Access to specific servicesAccess to console and/or REST APIs and/or SOAP APIs

Master user account – owns payment method

Sign up

1Key pairs

2IAM users

5Access

3Image

4

Account

Administrators Developers Applications

Bob

Kevin

Tomcat

Jim Brad

Mark

Susan

Reporting

Console

Sign up

1Key pairs

2Access

3Image

4IAM users

5

Account

Administrators Developers Applications

Bob

KevinMulti-factor authentication

Tomcat

Jim Brad

Mark

Susan

Reporting

Console

Groups

Sign up

1Key pairs

2Access

3Image

4IAM users

5

KevinAWS system entitlements

RolesAccount

Administrators Developers Applications

Bob

Tomcat

Jim Brad

Mark

Susan

Reporting

Console

Sign up

1Key pairs

2Access

3Image

4IAM users

5

{"Statement": [{

"Effect": "Allow", "Action": ["elasticbeanstalk:*", "ec2:*", "elasticloadbalancing:*", "autoscaling:*", "cloudwatch:*","s3:*","sns:*"

],"Resource": "*"

}]

}

Policy drivenDeclarative definition of rights for groups

Policies control access to AWS APIs

Sign up

1Key pairs

2IAM users

5Access

3Image

4

Sign up

1Key pairs

2Access

3Image

4IAM users

5

Next Steps

Auto ScalingAutomatic re-sizing of compute clusters based upon demand

Elastic Load BalancingCreate highly scalable applicationsDistribute load across EC2 instances in multiple availability zones

Relational Database ServiceDatabase-as-a-ServiceNo need to install or manage database instances Scalable and fault tolerant configurations

Next Steps

aws.amazon.comget started with the free tier

Thanks!Q & A?

For more info, please visit:

http://engineering.ysfhq.com

http://aws.amazon.com/