AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

37
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amjad Hussain Senior Manager, Amazon EC2 Tim Nicholas Lead Automation Architect, Xero 12/01/2016 How I learned to embrace DevOps and configure infrastructure at scale Amazon EC2 Systems Manager

Transcript of AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Page 1: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Amjad Hussain

Senior Manager, Amazon EC2

Tim Nicholas

Lead Automation Architect, Xero

12/01/2016

How I learned to embrace DevOps and configure

infrastructure at scale

Amazon EC2 Systems Manager

Page 2: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

What to Expect from the Session

• Introduction to EC2 Systems Manager

• Learn about Run Command, State Manager, and

Parameter Store

• How Xero uses Run Command

• Demo!

• FAQs and best-practices

Page 3: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

What we heard from you

• Traditional IT tools not built for the

cloud

• Managing resources at scale is difficult

• Lack of visibility into configuration,

granular control

• Multiple vendors; complex licensing

Page 4: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Introducing EC2 Systems Manager

A set of capabilities that enable automated configuration and

ongoing management of systems at scale, across all your

Windows and Linux workloads, running in Amazon EC2 or

on-premises

Page 5: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Systems Manager Capabilities

Run Command Maintenance

Window

Inventory

State Manager Parameter Store

Patch Manager

Automation

Deploy, Configure,

and Administer

Track and

UpdateShared

Capabilities

Page 6: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Run Command

Page 7: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Run Command

• Execution of administrative tasks

• Improve security posture – no need to SSH or RDP

• Delegated access control

• Customizable and flexible

• Get notified on the status of your commands

• Control the rate at which you send commands for scale

Page 8: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Sending a command

aws ssm send-command

--document-name AWS-RunPowerShellScript

--instance-id i-1234567

--parameters commands=“mkdir C:\Demo”

--service-role-arn <my-service-role>

-- notification-config NotificationArn=<my-topic-

arn>,NotificationEvents=“Success”,NotificationType=“Command”

Remotely create a directory on an instance and notify via

SNS when it completes

Page 9: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Run Command – Getting started

• Instance: Setup agent, AWS Identity & Access

Management (IAM) role on your instance

• Document: Author your intent

• Command and Command Invocation

• Plugins: In-guest actions that perform tasks

• Status and output: Granular results

Page 10: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Run Command – Scale

• Send a command based on a tag query

• Velocity control and error handling

aws ssm send-command --document-name <value> --targets

“Key=tag:ServerRole;Values=WebFrontEnd” […]

aws ssm send-command --max-concurrency 10 …

aws ssm send-command --max-errors 10 …

Page 11: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Setting up your instances

• Single light-weight agent, cross-platform

• SSM agent is open source, written in Go

• Health status via DescribeInstanceInformation

• On-demand agent update

• Hybrid support

Page 12: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Finding out which instances are heartbeating

D:\Users\amjadhu>aws ssm describe-instance-information

{

"InstanceInformationList": [

{

"IsLatestVersion": false,

"PingStatus": "Online",

"InstanceId": "i-c6d69773",

"ResourceType": "EC2Instance",

"AgentVersion": "3.17.1032",

"PlatformVersion": "6.2.9200",

"PlatformName": "Windows Server 2012 Standard",

"PlatformType": "Windows",

"LastPingDateTime": 1477203028.78

},

Page 13: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Setting up the agent for on-premises

• One-time setup to register

on-premises servers

• Consistent experience

• Identified by mi-*

• One-time setup to register

on-premises servers

• Consistent experience

• Identified by mi-*

Page 14: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Customizing commands

• Documents: A common way of authoring across EC2

Systems Manager

• Parameters: Allows passing in run-time values

• JSON schema, allows editing and versioning

• Sharing with accounts

• Amazon published documents (begin with AWS-*)

Page 15: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Document content

{

"schemaVersion": "2.0",

"description": “Simple script execution",

"mainSteps": [

{

"action": "aws:runPowerShellScript",

"name": “runMyScript",

"inputs": {

"runCommand": “.\myScript.ps1",

"workingDirectory": "C:\Temp",

"timeoutSeconds": "10"

}

}

]

}

Page 16: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Sharing Documents

• Share with other accounts

• Cross-account scenarios

Page 17: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

How Xero uses Run Command

Page 18: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Xero – Run Command

Beautiful cloud-

based

accounting software

Connecting people with the right numbers

anytime, anywhere, on any device

Beautiful accounting software

Page 19: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Xero – Run Command

1,500+Staff globally

862kSubscribers globally

$303msub revenue FY16

All figures shown are in NZD

2 yearsAWS design and build

6 monthsAWS service migration

Page 20: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Xero – Operational Challenges

• Host discovery

• Dynamic, disposable servers

• Increase in host count

• Integration with pipeline tools

• CI/CD tooling

• Chat bots

• Lambda

• Network isolation

• Production Servers vs Pipeline tools (git, CI/CD)

• Multiple AWS accounts

• Production Servers vs operations/development team workstations

Page 21: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

prox

yWAF

prox

yWAF

internetcustomer

sadmin

Prod TPZServices

TPZ

Prod App VPC Services App VPC

Xero – Network Separation

Page 22: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Xero – Run Command Use Cases

• Validation of .Net application configuration - From CI

• Reloading application pools - via CI

• Enabling services on a sample of machines in an

ASG via AWS Lambda

• PowerShell modules for interactive investigation

Page 23: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Xero – Demo!

PowerShell modules for interactive

investigation

Page 24: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

State Manager

Page 25: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

State Manager

• Maintain consistent state of instances

• Reapply to keep instances from drifting

• Easily view status of configuration changes

• Define schedule – ad hoc, periodic

• Track aggregate status for your fleet

Page 26: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

State Manager – Getting started

• Document: Author your intent

• Target: Instances or tag queries

• Association: Binding between a document and a

target

• Schedule: When to apply your association

• Status: Check the state of your association at an

aggregate or instance level

Page 27: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Creating an Association

aws ssm create-association

--document-name WebServerDocument

--document-version \$DEFAULT

--schedule-expression cron(0 */30 * * * ? *)

--targets “Key=tag:Name;Values=WebServer”

--output-location "{ \"S3Location\": { \"OutputS3Region\": \“us-east-1\",

\"OutputS3BucketName\": \“MyBucket\", \"OutputS3KeyPrefix\": \“MyPrefix\" } }“

Configures all instances that match the tag query and reapplies every

30 minutes

Page 28: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Parameter Store

Page 29: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Parameter Store

• Centrally store and find config data

• Repeatable, automatable management (e.g. SQL

connection strings)

• Granular access control – view, use and edit values

• Encrypt sensitive data using your own AWS KMS keys

Page 30: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Parameter Store – Getting started

• Parameter: Key-value pair

• Secure Strings: Encrypt sensitive parameters with your

own KMS or default account encryption key

• Reuse: In Documents and easily reference at runtime

across EC2 Systems Manager using {{ssm:parameter-

name}}

• Access Control: Create an IAM policy to control access

to specific parameter

Page 31: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Creating and using a parameter

aws ssm put-parameter

--name mycommand

--type string

--value “dir C:\Users”

aws ssm send-command

--name AWS-RunPowerShellScript

--parameters commands=[“echo {{ssm:mycommand}}”]

--target Key=tag:Name,Values=WebServer

Page 32: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Demo!

Page 33: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Best-practices and FAQs

• What OS platforms are supported?

• Update your SSM agent today to get started!

• What ports or network access do my instances need?

• Is there anything different to set up on-premises servers?

• Use notifications, velocity control

• For disruptive actions, use Run Command with Maintenance

Window

• Fine-grained access control through IAM policies on resources (e.g.

documents)

• Customize configuration with idempotent scripts for State Manager

Page 34: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Your Feedback is Important!

• These services are available today

• Learn more at https://aws.amazon.com/ec2/run-

command/

• Technical documentation at

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/

run-command.html

• Please send your feedback, improvements, requests to

[email protected]

Page 35: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Thank you!

Page 36: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Remember to complete

your evaluations!

Page 37: AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Related Sessions

WIN401 - How to Manage Inventory, Patching, and System

Images for Your Hybrid Cloud with AWS Management

Capabilities