Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven...
Transcript of Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven...
![Page 1: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/1.jpg)
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS
Raj Wilkhu – Principal Engineer, JUST EAT Bruce Jackson – CTO, Myriad Group AG
![Page 2: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/2.jpg)
You are on-board …
needs to experiment, innovate, reduce risk Business
of services and applications Continuous
Delivery
culture, automation, measurement, sharing DevOps
infrastructure-as-code Cloud
![Page 3: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/3.jpg)
AWS CloudFormation • Create templates of the infrastructure and
applications you want to run on AWS
• Have the CloudFormation service automatically provision the required AWS resources and their relationships from the templates
• Easily version control, replicate or update the infrastructure and applications using the templates
• Integrates with other development, CI/CD, and management tools.
![Page 4: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/4.jpg)
Create template – Example web application
security group
Auto Scaling group
EC2 instance
Elastic Load Balancing
DB Service
Inventory Service
Recommendations Service
ElastiCache memcached cluster
Software pkgs, config, & data CloudWatch
alarms
![Page 5: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/5.jpg)
Create template – Resources
security group
Auto Scaling group
EC2 instance
Elastic Load Balancing
ElastiCache memcached cluster
Software pkgs, config, & data CloudWatch
alarms
"Resources" : { "SecurityGroup" : {}, "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "MinSize" : "1", "MaxSize" : "3", "LoadBalancerNames" : [ { "Ref" : "LoadBalancer" } ], ... } }, "LoadBalancer" : {}, "CacheCluster" : {}, "Alarm" : {} },
CloudFormation Template
![Page 6: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/6.jpg)
Create template – Parameters
Auto Scaling group
EC2 instance
Recommendations Service Inventory
Service Customer DB
Service
Info to Customize Stack at Creation. Examples: Instance Type, App Pkg Version
"Parameters" : { "CustomerDBServiceEndPoint" : { "Description" : "URL of the Customer DB Service", "Type" : "String" }, "CustomerDBServiceKey" : { "Description" : "API key for the Customer DB Service", "Type" : "String", "NoEcho" : "true" }, "InstanceType" : { "Description" : "WebServer EC2 instance type", "Type" : "String", "Default" : "m3.medium", "AllowedValues" : ["m3.medium","m3.large","m3.xlarge"], "ConstraintDescription" : "Must be a valid instance type"
CloudFormation Template
![Page 7: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/7.jpg)
Create template – Outputs
Elastic Load Balancing
"Resources" : { "LoadBalancer" : {}, ... }, "Outputs" : { "WebsiteDNSName" : { "Description" : "The DNS name of the website", "Value" : { "Fn::GetAtt" : [ "LoadBalancer", "DNSName" ] } } }
CloudFormation Template
![Page 8: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/8.jpg)
Create template – Deploy and configure software
Auto Scaling group
EC2 instance
Software pkgs, config, & data
"AWS::CloudFormation::Init": { "webapp-‐config": { "packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {}
}, "chef-‐config" : {} }
CloudFormation Template ü Declarative ü Debug-able ü Updatable ü Highly Secure ü BIOT™ Bring In
Other Tools
![Page 9: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/9.jpg)
Create template – Language features
![Page 10: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/10.jpg)
Use a wide range of AWS services
ü Auto Scaling ü Amazon CloudFront ü AWS CloudTrail ü AWS CloudWatch ü Amazon DynamoDB ü Amazon EC2 ü Amazon ElastiCache ü AWS Elastic Beanstalk ü AWS Elastic Load Balancing ü Amazon Kinesis
ü AWS Identity and Access Mgmt
ü AWS OpsWorks ü Amazon RDS ü Amazon Redshift ü Amazon Route 53 ü Amazon S3 ü Amazon SimpleDB ü Amazon SNS ü Amazon SQS ü Amazon VPC
and more … As of April 2015
![Page 11: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/11.jpg)
Nested CloudFormation Stacks
"myStackWithParams" : { "Type" : "AWS::CloudFormation::Stack", "Properties" : { "TemplateURL" : "https://s3.amazonaws.com/template-bucket/mystack.template", ”Parameters" : { "InstanceType" : "t1.micro",
"KeyName" : "mykey"
}
} }
Pass parameters to nested CloudFormation Stacks
![Page 12: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/12.jpg)
Iterate on infrastructure
![Page 13: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/13.jpg)
Update stack In-place Blue-Green
Faster
Cost-efficient
Simpler state and data migration
Working stack not touched
![Page 14: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/14.jpg)
Extending AWS CloudFormation
![Page 15: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/15.jpg)
Extend with Custom Resources
security group
Auto Scaling group
EC2 instance
Elastic Load Balancing
ElastiCache memcached cluster
Software pkgs, config, & data CloudWatch
alarms Web Analytics
Service AWS
CloudFormation
Provision AWS Resources
"Resources" : { "WebAnalyticsTrackingID" : { "Type" : "Custom::WebAnalyticsService::TrackingID", "Properties" : { "ServiceToken" : "arn:aws:sns:...", "Target" : {"Fn::GetAtt" : ["LoadBalancer", "DNSName"]}, "Plan" : "Gold" } }, ...
“Success” + Metadata
“Create, Update, Rollback, or Delete” + Metadata
![Page 16: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/16.jpg)
Lambda-powered custom resources
security group
Auto Scaling group
EC2 instance
Elastic Load Balancing
ElastiCache memcached cluster
Software pkgs, config, & data CloudWatch
alarms
// Implement Custom Logic Here
Lookup an AMI ID
Lookup VPC ID and Subnet ID
Reverse an IP Address
![Page 17: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/17.jpg)
Application Lifecycle
![Page 18: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/18.jpg)
Infrastructure Lifecycle
EC2
SQS, SNS, Kinesis, etc.
Databases
VPC
IAM
Application Lifecycle
Download Packages, Install Software, Configure Apps, Bootstrap Apps,
Update Software, Restart Apps,
etc.
CloudFormation
• Templatize • Replicate • Automate
![Page 19: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/19.jpg)
AWS::CloudFormation::Init
"AWS::CloudFormation::Init": {
"webapp-config": {
"packages" : {}, "sources" : {}, "files" : {},
"groups" : {}, "users" : {},
"commands" : {}, "services" : {}
Declarative
![Page 20: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/20.jpg)
AWS::CloudFormation::Init Supports updates
"packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {}
![Page 21: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/21.jpg)
AWS::CloudFormation::Init
"install_chef" : {},
"install_wordpress" : {
"commands" : {
"01_get_cookbook" : {}, ...,
"05_configure_node_run_list" : {
"command" : "knife node run_list add -z `knife node list -z` recipe[wordpress]",
"cwd" : "/var/chef/chef-repo",
"env" : { "HOME" : "/var/chef" }
Flexibility to bring in other tools such as AWS CodeDeploy and Chef
ow.ly/DiNkz
![Page 22: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/22.jpg)
Use AWS::CloudFormation::Init "UserData": { "# Get the latest CloudFormation helper scripts package\n", "yum update -y aws-cfn-bootstrap\n", "# Trigger CloudFormation::Init configuration \n", "/opt/aws/bin/cfn-init --stack ", {"Ref": "AWS::StackId"}, " --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n", "# Signal completion\n", "/opt/aws/bin/cfn-signal –e $? --stack ", {"Ref": "AWS::StackId"}, " --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n"
![Page 23: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/23.jpg)
Use CloudWatch Logs for debugging
"install_logs": {
"packages" : { ... "awslogs" ... },
"services" : { ... "awslogs" ... }
"files": {
"/tmp/cwlogs/cfn-logs.conf": {}
file = /var/log/cfn-init.log log_stream_name = {instance_id}/cfn-init.log file = /var/log/cfn-hup.log log_stream_name = {instance_id}/cfn-hup.log
ow.ly/E0zO3
![Page 24: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/24.jpg)
Use CloudWatch Logs for debugging
ow.ly/E0zO3
![Page 25: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/25.jpg)
Wait Conditions & DependsOn
![Page 26: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/26.jpg)
Example Wait for EC2 instance to signal success
"Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "UserData" : { "Fn::Base64" : {"Ref" : "myWaitHandle"}},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]} } },
![Page 27: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/27.jpg)
Example
"myWaitHandle" : { "Type" : "AWS::CloudFormation::WaitConditionHandle",
}, "myWaitCondition" : {
"Type" : "AWS::CloudFormation::WaitCondition", "DependsOn" : "Ec2Instance", "Properties" : { "Handle" : { "Ref" : "myWaitHandle" }, "Timeout" : "4500" }
}
Wait for EC2 instance to signal success
![Page 28: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/28.jpg)
CI/CD Pipeline
![Page 29: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/29.jpg)
CloudFormation in a CI/CD Pipeline
AWS CloudFormation Issue Tracker
App Developers
DevOps Engineers, Infrastructure Developers,
Systems Engineers
Dev Env Code Repo
App Pkgs, CloudFormation Templates, Etc.
CI Server
Test
Staging
Prod Code Review
"Infra-‐as-‐Code"
App Code & Templates
![Page 30: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/30.jpg)
LONDON
![Page 31: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/31.jpg)
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Continuous Integration and Deployment
Raj Wilkhu – Principal Engineer, JUST EAT
![Page 32: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/32.jpg)
JUST EAT Platform
• High volume e-commerce platform • 45-60 minutes cycle • Real time message delivery and confirmation network • Hardware in 22,500 restaurants in the UK, 40,800 worldwide • Peak traffic is 10,000% of normal daytime traffic • 3 cities • Over 130 releases a month • Primarily Windows
![Page 33: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/33.jpg)
Building reliable and high quality software
• Test-Driven Development, Continuous Deployment and Immutable Infrastructure
• Every part of the infrastructure replaced several times a day • Incentive to continuously speed up deployment process
![Page 34: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/34.jpg)
Deploying components
• Platform decomposed into components • Each component defined by single AWS Cloudformation
template • Single Build Artifact • Instances boot and bootstrap themselves from
• pre-baked AMI • Content in S3 based on Cloudformation::Init metadata
![Page 35: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/35.jpg)
Pre-baked AMI vs Dynamic config
• System dependencies (eg OS features, .NET, ruby, gems, etc)
• Static configurations
• Continuous Deployment (latest code) • Environment specific configuration • Automation to deploy feature
Increasing Boot time
• Unzip package • Install and configure dependencies • Execute deployment script(s) • Warm up feature • Tagged by deployment scripts
• Baked via CI pipeline • Tested and tagged
Pre-baked AMI CloudFormation::Init
![Page 36: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/36.jpg)
Cloudformation template generation
• Single json from multiple yaml and erb files • Similar to HTML templating engines • Organized by component • Convention over configuration
"<%= @feature_name %>_pre_deploy": { "commands": { <% cmd_prefix = 250 services_to_disable = %w(aelookupsvc bits scdeviceenum trustedinstaller wuauserv) services_to_disable.each do |service_name| %> "<%= cmd_prefix %>_<%= @feature_name %>_stop_<%= service_name %>_service_because_unnecessary": { "waitAfterCompletion": "0", "command": "powershell -noninteractive -noprofile -command \"& { stop-service -Name <%= service_name %> -verbose }\"", "ignoreErrors": true }, "<%= cmd_prefix + 1 %>_<%= @feature_name %>_disable_<%= service_name %>_service_because_unnecessary": { "waitAfterCompletion": "0", "command": "powershell -noninteractive -noprofile -command \"& { set-service -Name <%= service_name %> -StartupType Disabled -verbose }\"", "ignoreErrors": true }, <% cmd_prefix += 2 end %>
--- :feature_config: :ami_id: ami-c99544ef :elb: true :use_logging_client: true :elb_dns: true :parameters: :ElbTarget: "HTTP:80/status" :AsgDesiredSizeDuringPeakTime: 10 :InstanceType: c4.xlarge
Example yaml: Example json.erb template:
{ Master
![Page 37: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/37.jpg)
Immutable infrastructure - Cloudformation
Engineers
configs in yaml, template extensions in
json.erb + master
Tests in QA environment
Engineers
Component code
Continuous Integration Server (TeamCity Agent Cloud)
Build & Test Pipeline
Artifact
JustDeploy
Production/QA
EC2 Web/worker roles
Route 53
Rooms
Artifact
Git Git
![Page 38: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/38.jpg)
Immutable infrastructure - Baking AMIs
Engineers
AMI Bakery
Continuous Integration Server (TeamCity Agenet Cloud)
EC2 base image
configurations and resources
Shared AMI
Latest Amazon OS image
Git
Automated Tests
config.yaml updated
![Page 39: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/39.jpg)
Future
• Automatically select last tested AMI image • Use lambda to trigger AMI builds based on AMI release
notifications via SNS • Failover to secondary region • Publish more tools as OSS
– https://github.com/justeat
![Page 40: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/40.jpg)
@justeat_tech + @rajwilkhu http://tech.just-eat.com
Yes, we’re recruiting too. http://tech.just-eat.com/jobs
![Page 41: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/41.jpg)
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Deployment on the fly Bruce Jackson – CTO, Myriad Group AG
![Page 42: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/42.jpg)
From IMPS to msngr
database
app server
web server web server web server
...x16
• Move from a mid-2000 carrier platform to web scale
• …with 38M users on 16 stacks
• …in 6 months • …with a team of 4
![Page 43: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/43.jpg)
Basic foundations of our approach
Rules • OSGi used for all
application stacks • Make use of as much
managed infrastructure as possible
• Avoid any environment configuration
• Make deployment as familiar as coding
Rationale • Rapid creation of services
against interfaces • We don’t have any devops
so outsource management where possible
• No files/scripts that only one person understands
• The team is made up of developers
![Page 44: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/44.jpg)
What did we do? • Created utility libraries
that use code annotations and AWS tags for configuration
• Use Eclipse features to define bundles to deploy on systems
![Page 45: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/45.jpg)
What did we do (2) ?
• Created Eclipse tooling to manage deployment & code/feature checkout
![Page 46: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/46.jpg)
A quick demonstration…
![Page 47: Deep Dive: Infrastructure as Code - AWS - Amazon S3and...Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS Raj Wilkhu ... AWS CloudTrail ! AWS CloudWatch](https://reader033.fdocuments.in/reader033/viewer/2022042600/5abd13e77f8b9a76038eb072/html5/thumbnails/47.jpg)
LONDON