Deep Dive: Infrastructure as Code
-
Upload
amazon-web-services -
Category
Technology
-
view
469 -
download
1
Transcript of Deep Dive: Infrastructure as Code
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Infrastructure as Code
with AWS CloudFormationChris Whitaker, Senior Software Development Manager, AWS
You are on board …
Continuous delivery
• services and applications
DevOps
• culture, automation, measurement, sharing
Cloud
• infrastructure-as-code
Business needs to experiment, innovate, reduce risk
AWS CloudFormation
AWS CloudFormation
• Create templates of the infrastructure and applications you want to run on AWS
• Have the CloudFormation service automatically provision the required AWS resources and their relationships from the templates
• Easily version control, replicate or update the infrastructure and applications using the templates
• Integrates with other development, CI/CD, and management tools
Basic workflow
design
create infrastructure
templates
write application
code
create stacks
iterate
depends on
Design: Building a food ordering service
food catalog website
ordering website
customer DB service
inventory service
recommendations service
analytics servicefulfillment
service
payment
service
Create template for the food catalog website
security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
customer DB service
inventory service
recommendations service
Software pkgs,
config, & data
CloudWatch
alarms
ElastiCache
Create template: Resources
security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
Software pkgs,
config, & data
CloudWatch
alarms
"Resources" : {"SecurityGroup" : {},"WebServerGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup","Properties" : {
"MinSize" : "1","MaxSize" : "3","LoadBalancerNames" : [ { "Ref" :
"LoadBalancer" } ],...
}},"LoadBalancer" : {},"CacheCluster" : {},"Alarm" : {}
},
CloudFormation Template
ElastiCache
Create template: Parameters
Auto Scaling group
EC2
instance
recommendations Service
inventory service
customer DB service
info to customize stack at creation
examples: instance type, app pkg version
"Parameters" : {"CustomerDBServiceEndPoint" : {
"Description" : "URL of the Customer DB Service","Type" : "String"
},"CustomerDBServiceKey" : {
"Description" : "API key for the Customer DB Service",
"Type" : "String","NoEcho" : "true"
},"InstanceType" : {
"Description" : "WebServer EC2 instance type","Type" : "String","Default" : "m3.medium","AllowedValues" :
["m3.medium","m3.large","m3.xlarge"],"ConstraintDescription" : "Must be a valid
instance type"
CloudFormation Template
Create template: Outputs
Elastic Load
Balancing
"Resources" : {
"LoadBalancer" : {},
...
},
"Outputs" : {
"WebsiteDNSName" : {
"Description" : "The DNS name of the website",
"Value" : {
"Fn::GetAtt" : [ "LoadBalancer", "DNSName" ]
}
}
}
CloudFormation Template
Create template: Deploy and configure software
Auto Scaling group
EC2
instance
Software pkgs,
config, & data
"AWS::CloudFormation::Init": {
"webapp-config": {
"packages" : {}, "sources" : {}, "files" : {},
"groups" : {}, "users" : {},
"commands" : {}, "services" : {}
},
"chef-config" : {}
}
CloudFormation Template
declarative
debug-able
updatable
highly secure
BIOT™ (Bring In
Other Tools)
Create template: Language features
Create stack
Operate stack
Use a wide range of AWS services
Auto Scaling
Amazon CloudFront
AWS CloudTrail
Amazon CloudWatch
AWS Data Pipeline (new)
Amazon DynamoDB
Amazon EC2
Amazon EC2 Container Service (new)
Amazon ElastiCache
AWS Elastic Beanstalk
Elastic Load Balancing
Managed policies in IAM (new)
Amazon Kinesis
AWS Lambda (new)
AWS OpsWorks
Amazon RDS
Amazon Redshift
Amazon Route 53
Amazon S3
Amazon SimpleDB
Amazon SNS
Amazon SQS
Amazon VPC
and more …
Basic workflow
design
create infrastructure
templates
write application
code
create stacks
iterate
Infrastructure-as-code workflow
codeversion control
code review
integrate
“It’s all software”
“It’s all software” – Organize like it’s software
front-end services
• consumer website, seller website, mobile back end
back-end services
• search, payments, reviews, recommendations
shared services
• CRM DBs, common monitoring /alarms, subnets, security groups
base Network
• VPCs, Internet gateways, VPNs, NATs
identity • IAM users, groups, roles
“It’s all software” – Build and operate like it’s
software
application software
source code
package
loader/interpreter
desired application state in memory
infrastructure software
JSON templates / JSON template generators
JSON templates
AWS CloudFormation
desired infrastructure in the cloud
Iterate on infrastructure
Update stack
in-place blue-green
faster
cost-efficient
simpler state and
data migration
working stack
not touched
Extending AWS CloudFormation
Extend with custom resources
security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
Software pkgs,
config, & data
CloudWatch
alarms
Web Analytics
ServiceAWS
CloudFormation
provision
AWS resources
"Resources" : {
"WebAnalyticsTrackingID" : {
"Type" : "Custom::WebAnalyticsService::TrackingID",
"Properties" : {
"ServiceToken" : "arn:aws:sns:...",
"Target" : {"Fn::GetAtt" : ["LoadBalancer", "DNSName"]},
"Plan" : "Gold"
}
},
...
“Success” + Metadata
“Create, update, roll back, or delete”
+ metadata
ElastiCache
Lambda-powered custom resources
security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
Software pkgs,
config, & data
CloudWatch
alarms
your AWS CloudFormation stack
// Implement custom logic here
look up an AMI ID
your AWS Lambda functions
look up a VPC ID and a subnet ID
reverse an IP address
Lambda-powered
custom resources
ElastiCache
Application deployment as code
Infrastructure provisioning
EC2
SQS, SNS, Kinesis, etc.
databases
VPC
IAM
Application deployment
download packages, install software, configure apps,
bootstrap apps, update software, restart apps,
etc.
CloudFormation
• templatize
• replicate
• automate
Application deployment as code
inside a CloudFormation template
Amazon Machine Images
CloudFormation::Init
Chef, Puppet, CodeDeploy
OpsWorks
Chef
Metadata
AWS::CloudFormation::Init
AWS::CloudFormation::Init
Declarative
Reusable
Grouping & Ordering
Debug-able
Updatable
Highly Secure
BIOT™ (Bring In Other Tools)
ow.ly/DiNCm
AWS::CloudFormation::Init
"AWS::CloudFormation::Init": {
"webapp-config": {
"packages" : {}, "sources" : {}, "files" : {},
"groups" : {}, "users" : {},
"commands" : {}, "services" : {}
Declarative
AWS::CloudFormation::Init
Debuggable
AWS::CloudFormation::Init
Supports updates
"packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {},"commands" : {}, "services" : {}
AWS::CloudFormation::Init
"install_chef" : {},
"install_wordpress" : {
"commands" : {
"01_get_cookbook" : {}, ...,
"05_configure_node_run_list" : {
"command" : "knife node run_list add -z `knife node list -z` recipe[wordpress]",
"cwd" : "/var/chef/chef-repo",
"env" : { "HOME" : "/var/chef" }
Flexibility to bring in other tools such as AWS CodeDeploy and Chef
ow.ly/DiNkz
AWS::CloudFormation::Init
"YourInstance": {
"Metadata": {
"AWS::CloudFormation::Authentication": {
"S3AccessCreds": {
"type": "S3",
"roleName": { "Ref" : "InstanceRole"},
"buckets" : ["your-bucket"]
}
},
"AWS::CloudFormation::Init": {}
Supports role-based authentication
Securely download
Choose auth type.
IAM role is
recommended.
ow.ly/DqkrB
Use AWS::CloudFormation::Init
"UserData": {
"# Get the latest CloudFormation helper scripts package\n","yum update -y aws-cfn-bootstrap\n",
"# Trigger CloudFormation::Init configuration \n","/opt/aws/bin/cfn-init --stack ", {"Ref": "AWS::StackId"},
" --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n",
"# Signal completion\n","/opt/aws/bin/cfn-signal –e $? --stack ", {"Ref": "AWS::StackId"},
" --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n"
Use Amazon CloudWatch Logs for debugging
"install_logs": {
"packages" : { ... "awslogs" ... },
"services" : { ... "awslogs" ... }
"files": {
"/tmp/cwlogs/cfn-logs.conf": {}
file = /var/log/cfn-init.loglog_stream_name = {instance_id}/cfn-init.log
file = /var/log/cfn-hup.loglog_stream_name = {instance_id}/cfn-hup.log
ow.ly/E0zO3
Bake AMIs for faster booting and
for maintaining golden images
dev/test stacks bake AMI staging/prod stacks
tracking
Using CloudFormation and OpsWorks
together
Infrastructure provisioning
EC2
SQS, SNS, Kinesis, etc.
databases
VPC
IAM
Application deployment
download packages, install software, configure apps,
bootstrap apps, update software, restart apps,
etc.
CloudFormation
• templatize
• replicate
• automate
OpsWorks
• built-in application lifecycle
• interactive application console
OpsWorks and CloudFormation side by side
OpsWorks
• built-in application lifecycle
• interactive application console
Infrastructure provisioning
EC2
SQS, SNS, Kinesis, etc.
databases
VPC
IAM
Application deployment
download packages, install software, configure apps,
bootstrap apps, update software, restart apps,
etc.
CloudFormation
• templatize
• replicate
• automate
OpsWorks “inside” CloudFormation
Infrastructure-as-code in a CI/CD pipeline
CloudFormation in a CI/CD pipeline
AWS CloudFormationIssue Tracker
app developers
DevOps engineers,infrastructure developers,
systems engineers
dev env code repo
app pkgs, CloudFormationtemplates, etc.
CI server
test
staging
prodcode review
“infrastructure-as-code"
app code & templates
Templatize existing resources
CloudFormer: Templatize existing resources
1. Launch a CloudFormer
application stack.
2. Walk through the
CloudFormer UI and select
resources to templatize.
4. Customize.
For example: parameterize
resource properties.
5. Create a new stack.
Practitioners of infrastructure-as-code
• Developers/DevOps teams value CloudFormation for its ability to treat infrastructure as code, allowing them to apply software engineering principles, such as SOA, revision control, code reviews, and integration testing to infrastructure.
• IT admins and MSPs value CloudFormation as a platform to enable standardization, managed consumption, and role-specialization.
• ISVs value CloudFormation for its ability to support scaling out of multi-tenant SaaS products by quickly replicating or updating stacks. ISVs also value CloudFormation as a way to package and deploy their software in their customer accounts on AWS.
New York