© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Zero to Sixty: AWS CloudFormation Chetan Dandekar, Senior Product Manager – AWS CloudFormation Capen Brinkley, Software Developer – Intuit November 13, 2013

AWS CloudFormation

creation order? how long do I pause? what errors can I recover from?

Instruction Manual

Instruction Manual

Instruction Manual

Provisioning Script(s)

what environment config and utilities does my script depend on?

can my script be faster? will this script work again?

Model Click Done

AWS CloudFormation

Dev

Test

Staging

Prod

Demos

Regions

Version Control Replicate

Standardization

Service Catalog

AWS CloudFormation Automate

CloudFormation

Build Pipeline

Config

Template Snippets

Code

API

API

SNS

Monitor Progress

Intuit’s CloudFormation Story

Key Takeaways

• How we use CloudFormation to manage large scale applications

• Methodologies and tools you can use to follow a similar path

Infrastructure Design

Template Management

Stack Management

Bootstrapping

Live Community Traffic April 15

Feb. 1

Amazon EC2 Amazon S3

Amazon RDS Elastic Load Balancing

Infrastructure as Code

Auto Scaling Group

Web App

Server

App Tier

Web App

Server

Amazon Route 53

Amazon EC2

Amazon S3

Amazon RDS

Amazon ElastiCache

Amazon CloudFront

Amazon CloudWatch

AWS CloudFormation

AWS IAM

Amazon SQS

Amazon SES Amazon SNS

Service Oriented Architecture

SQS Queue App Tier

Auto Scaling Group

Web App

Server

Web App

Server

Multiple Templates, Loosely Coupled

Multiple Templates, Loosely Coupled

Easy To Reason About

Reusable

Stack Management

Simple Deploy https://github.com/intuit/simple_deploy

attributes clone create deploy destroy environments events execute instances

list outputs parameters protect resources status template update

Simple Deploy Commands attributes clone create deploy destroy environments events execute instances

list outputs parameters protect resources status template update

elb-1

Auto Scaling Group

app-1 (v1.0.0) Blue

Auto Scaling Group

app-2 (v1.1.0) Green

$ simple_deploy environments

Default

lc_preprod_us_west_1

lc_preprod_us_west_2

lc_preprod_us_east_1

PROD_lc_prod_us_west_1_PROD

PROD_lc_prod_us_west_2_PROD

PROD_lc_prod_us_east_1_PROD

$ simple_deploy list \

–-environment lc_preprod_us_west_1

lc-dev-elb-1

lc-dev-app-1

lc-dev-db-master-1

lc-dev-db-parameter-group

simple_deploy create \

–-environment lc_preprod_us_west_1 \

–-name lc-dev-app-2 \

–-template app.json \

–-input-stack lc-dev-elb-1 \

–-input-stack lc-dev-db-master-1 \

–-attribute chef_repo=3f57f9f \

–-attribute app=bcb68de

simple_deploy clone

--environment lc_preprod_us_west_1 \

--source-stack lc-dev-1-app-1 \

--name lc-dev-1-app-2 \

--attribute app=afdac509b \

--attribute chef_repo=a4531e5ff6

simple_deploy destroy

--environment lc_preprod_us_west_1 \

--name lc-dev-1-app-1

Code / CI / Artifact

Simple Deploy

CloudFormation

Autoscaling

Userdata

CloudFormation::Init

Chef

Bootstrapping

> GET http://169.254.169.254/latest/user-data

#!/bin/bash

yum update -y aws-cfn-bootstrap

/opt/aws/bin/cfn-init –-stack lc-app-stack

-–verbose --resource InstanceLaunchConfig

--region=us-west-2 –-configsets bootstrap

"UserData": { "Fn::Base64": { "Fn::Join": ["", [

"#!/bin/bash\n",

"yum update -y aws-cfn-bootstrap\n",

"/opt/aws/bin/cfn-init --stack “,

{ "Ref": "AWS::StackName" },

" --verbose"

" --resource InstanceLaunchConfig",

" --region=", { "Ref": "AWS::Region" },

" -configsets bootstrap", "\n”

CloudFormation Instance User Data

Configsets Commands

Files Groups

Packages Services Sources Users

CloudFormation::Init Resources

Configsets Commands

Files Groups

Packages Services Sources Users

"configSets”: {

"bootstrap”: [ "create_files",

"install_packages",

"run_chef",

"clean_up” ]

}

"create_files": {

"files": {

"/etc/chef/ohai/hints/ec2.json": {

"content": "{}",

"mode": "000400",

"owner": "root",

"group": "root"

}

}

}

"install_packages": {

"packages": {

"yum”: {

"chef”: [ "11.6.2-1" ]

}

}

}

"run_chef": {

"commands": {

"1_download_chef_repo": { ... },

"2_decrypt_chef_repo": { ... },

"3_extract_chef_repo": { ... },

"4_run_chef": { ... }

}

}

"run_chef": {

"commands": {

"1_download_chef_repo": { ... },

"2_decrypt_chef_repo": { ... },

"3_extract_chef_repo": { ... },

"4_run_chef": { ... }

}

}

"run_chef": {

"commands": {

"run_chef": {

"command": "/usr/bin/chef-solo \

–c /var/chef/config/solo.rb \

–o ", { "Ref", "Role" }

}

}

}

"clean_up" : {

"commands": {

"1_cleanup_files" : {

"command": "rm –rf /var/tmp/chef_repo.tar.gz \

/var/tmp/chef_repo.tar.gz.gpg"

}

}

}

The Climb

What’s New in AWS CloudFormation

Let’s take an example

Scalable Reliable Highly Available

Two Types of Tasks

Develop

Parallel stack processing

Richer template language

Operate Fail-safe stack management

Updates without downtime

Federation and IAM roles

Parallel Stack Processing

Parallel Stack Processing

Richer Template Language

Conditions

Prod

Dev

Conditions "Parameters" : { "Environment" : { "Description" : "Specifies if this a Dev QA or Prod Environment", "Type" : "String", "Default" : "Dev", "AllowedValues" : [ "Dev", "QA", "Prod"] }, }, ... "Conditions" : { "ProdEnvironment" : { "Fn::Equals" : [ { "Ref" : "Environment" }, "Prod" ]} },

Conditions "DBInstance" : { "Type": "AWS::RDS::DBInstance", "Properties": { "DBName" : { "Ref" : "DBName" }, "Engine" : "MySQL", "MultiAZ" : { "Fn::If" : [ "ProdEnvironment", "true", "false" ] }, "DBSnapshotIdentifier" : { "Fn::If" : [ "ProdEnvironment", { "Ref" : "DBName" }, { "Ref" : "AWS::NoValue" } ] }, ... } },

Conditions "DBStorageAlarm" : { "Condition" : "ProdEnvironment", "Type" : "AWS::CloudWatch::Alarm", "Properties" : { "AlarmDescription" : "Alarm if db size grows beyond a threshold", "Namespace" : "AWS/RDS", "MetricName" : "FreeStorageSpace", ... } },

Conditions

• Fn::If • Fn::Equals • Fn::Not • Fn::And • Fn::Or

"Conditions" : { ... "ProdOrLoadTestingEnv" : { "Fn::Or" : [ { "Condition" : "ProdEnvironment"}, { "Fn::Equals" : [ ... ]} ] } }

"Fn::If": [{condition}, {value_if_true}, {value_if_false}]

User-Defined Resource Names By default,

• AWS CloudFormation

generates unique resource names

• “prodstack20131113-DBStorageAlarm-19BL0MOXL0TPI”

In addition,

• Flexibility to use custom names and still keep them unique

• “SalesDataStorageAlarm”

Develop

Parallel stack processing

Richer template language

Operate Fail-safe stack management

Updates without downtime

Federation and IAM roles

Fail-Safe Stack Management

Stack Protection

CloudFormation Prod

Dev1

Dev2

Dev3

{ "Effect" : "Allow", "Action" : [ "cloudformation:*" ], "Resource" : "arn:aws:cloudformation:us-west- 2:123456789012:stack/Dev*" }

{ "Effect" : "Allow", "Action" : [ "cloudformation:*" ], "Resource" : "*" }

Stack Protection { "Effect" : "Deny", "Action" : [ "cloudformation:DeleteStack", "cloudformation:UpdateStack" ], "Resource" : "arn:aws:cloudformation:us-west-2:123456789012:stack/productionstack/*" }

Stack Protection "Resources" : { "StackProtectionPolicy" : { "Type" : "AWS::IAM::Policy", "Properties" : { "PolicyName" : "StackProtectionPolicy", "Groups" : [ { "Ref" : "DenyGrp" } ], "PolicyDocument" : { "Statement" : [ { "Effect" : "Deny", "Action" : [ "cloudformation:DeleteStack", "cloudformation:UpdateStack" ], "Resource" : { "Ref" : “AWS::StackId" } }

Resource Protection { "Effect" : "Deny", "Action" : [ "ec2:TerminateInstances" ], "Condition": { "Null": { "ec2:ResourceTag/*cloudformation*" : "true" } }, "Resource" : "*" }

Preventing Updates { "Statement" : [ { "Effect" : "Deny", "Action" : "Update:*", "Principal" : "*", "Resource" : "ResourceType/AWS::RDS::DBInstance" }, { "Effect" : "Allow", "Action" : "Update:*", "Principal" : "*", "Resource" : "*" } ] }

Stack Policy Document

Preventing Updates > aws cloudformation create-stack -–template-url ... --stack-policy-url ...

{ "Statement" : [ { "Effect" : "Deny", "Action" : "Update:Replace", "Principal" : "*", "Resource" : "LogicalResourceId/MyInstance" }, { "Effect" : "Allow", "Action" : "Update:*", "Principal" : "*", "Resource" : "*" ...

Fine Grained Stack Policy Setting Stack Policy

Update without Downtime "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "LaunchConfigurationName" : { "Ref" : "LaunchConfig" }, ... }, "UpdatePolicy" : { "AutoScalingRollingUpdate" : { "MinInstancesInService" : “2", "MaxBatchSize" : “3", "PauseTime" : "PT20M" } } },

Using AWS CloudFormation with Federated Identities

Corporate identity store

1

User accesses broker

2 User authenticated

Identity broker

3 AWS Management

Console

User redirected to console

Temporary security credentials obtained

4

AWS Security Token Service

User accesses APIs

CloudFormation API and other AWS APIs

4

Network Architects

Application Developers

DB Admins

Calling AWS CloudFormation using IAM Roles

EC2 Instance

IAM Role

1. The IAM role has permissions to call

AWS CloudFormation and provision

underlying resources AWS CloudFormation

2. User or script on the EC2 instance calls CloudFormation to

provision a stack

3. AWS CloudFormation provisions the stack using a

template hosted in an S3 bucket inside the VPC

CloudFormation Stack

Related Resources • http://aws.amazon.com/cloudformation/

• "Fundamentals of CloudFormation" lab in the Self Paced Lab Lounge

• DMG303 - AWS CloudFormation under the Hood

• ARC203 - How Adobe Deploys: Refreshing the Entire Stack Every Time

• DMG209 - Enterprise Management for the AWS Cloud

• Multiple other sessions are presenting CloudFormation samples

Please give us your feedback on this presentation

As a thank you, we will select prize winners daily for completed surveys!

DMG201