AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

Post on 06-Jan-2017

122 views 2 download

Transcript of AWS re:Invent 2016: How I learned to embrace DevOps and Configure Infrastructure at Scale (WIN402)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Amjad Hussain

Senior Manager, Amazon EC2

Tim Nicholas

Lead Automation Architect, Xero

12/01/2016

How I learned to embrace DevOps and configure

infrastructure at scale

Amazon EC2 Systems Manager

What to Expect from the Session

• Introduction to EC2 Systems Manager

• Learn about Run Command, State Manager, and

Parameter Store

• How Xero uses Run Command

• Demo!

• FAQs and best-practices

What we heard from you

• Traditional IT tools not built for the

cloud

• Managing resources at scale is difficult

• Lack of visibility into configuration,

granular control

• Multiple vendors; complex licensing

Introducing EC2 Systems Manager

A set of capabilities that enable automated configuration and

ongoing management of systems at scale, across all your

Windows and Linux workloads, running in Amazon EC2 or

on-premises

Systems Manager Capabilities

Run Command Maintenance

Window

Inventory

State Manager Parameter Store

Patch Manager

Automation

Deploy, Configure,

and Administer

Track and

UpdateShared

Capabilities

Run Command

Run Command

• Execution of administrative tasks

• Improve security posture – no need to SSH or RDP

• Delegated access control

• Customizable and flexible

• Get notified on the status of your commands

• Control the rate at which you send commands for scale

Sending a command

aws ssm send-command

--document-name AWS-RunPowerShellScript

--instance-id i-1234567

--parameters commands=“mkdir C:\Demo”

--service-role-arn <my-service-role>

-- notification-config NotificationArn=<my-topic-

arn>,NotificationEvents=“Success”,NotificationType=“Command”

Remotely create a directory on an instance and notify via

SNS when it completes

Run Command – Getting started

• Instance: Setup agent, AWS Identity & Access

Management (IAM) role on your instance

• Document: Author your intent

• Command and Command Invocation

• Plugins: In-guest actions that perform tasks

• Status and output: Granular results

Run Command – Scale

• Send a command based on a tag query

• Velocity control and error handling

aws ssm send-command --document-name <value> --targets

“Key=tag:ServerRole;Values=WebFrontEnd” […]

aws ssm send-command --max-concurrency 10 …

aws ssm send-command --max-errors 10 …

Setting up your instances

• Single light-weight agent, cross-platform

• SSM agent is open source, written in Go

• Health status via DescribeInstanceInformation

• On-demand agent update

• Hybrid support

Finding out which instances are heartbeating

D:\Users\amjadhu>aws ssm describe-instance-information

{

"InstanceInformationList": [

{

"IsLatestVersion": false,

"PingStatus": "Online",

"InstanceId": "i-c6d69773",

"ResourceType": "EC2Instance",

"AgentVersion": "3.17.1032",

"PlatformVersion": "6.2.9200",

"PlatformName": "Windows Server 2012 Standard",

"PlatformType": "Windows",

"LastPingDateTime": 1477203028.78

},

Setting up the agent for on-premises

• One-time setup to register

on-premises servers

• Consistent experience

• Identified by mi-*

• One-time setup to register

on-premises servers

• Consistent experience

• Identified by mi-*

Customizing commands

• Documents: A common way of authoring across EC2

Systems Manager

• Parameters: Allows passing in run-time values

• JSON schema, allows editing and versioning

• Sharing with accounts

• Amazon published documents (begin with AWS-*)

Document content

{

"schemaVersion": "2.0",

"description": “Simple script execution",

"mainSteps": [

{

"action": "aws:runPowerShellScript",

"name": “runMyScript",

"inputs": {

"runCommand": “.\myScript.ps1",

"workingDirectory": "C:\Temp",

"timeoutSeconds": "10"

}

}

]

}

Sharing Documents

• Share with other accounts

• Cross-account scenarios

How Xero uses Run Command

Xero – Run Command

Beautiful cloud-

based

accounting software

Connecting people with the right numbers

anytime, anywhere, on any device

Beautiful accounting software

Xero – Run Command

1,500+Staff globally

862kSubscribers globally

$303msub revenue FY16

All figures shown are in NZD

2 yearsAWS design and build

6 monthsAWS service migration

Xero – Operational Challenges

• Host discovery

• Dynamic, disposable servers

• Increase in host count

• Integration with pipeline tools

• CI/CD tooling

• Chat bots

• Lambda

• Network isolation

• Production Servers vs Pipeline tools (git, CI/CD)

• Multiple AWS accounts

• Production Servers vs operations/development team workstations

prox

yWAF

prox

yWAF

internetcustomer

sadmin

Prod TPZServices

TPZ

Prod App VPC Services App VPC

Xero – Network Separation

Xero – Run Command Use Cases

• Validation of .Net application configuration - From CI

• Reloading application pools - via CI

• Enabling services on a sample of machines in an

ASG via AWS Lambda

• PowerShell modules for interactive investigation

Xero – Demo!

PowerShell modules for interactive

investigation

State Manager

State Manager

• Maintain consistent state of instances

• Reapply to keep instances from drifting

• Easily view status of configuration changes

• Define schedule – ad hoc, periodic

• Track aggregate status for your fleet

State Manager – Getting started

• Document: Author your intent

• Target: Instances or tag queries

• Association: Binding between a document and a

target

• Schedule: When to apply your association

• Status: Check the state of your association at an

aggregate or instance level

Creating an Association

aws ssm create-association

--document-name WebServerDocument

--document-version \$DEFAULT

--schedule-expression cron(0 */30 * * * ? *)

--targets “Key=tag:Name;Values=WebServer”

--output-location "{ \"S3Location\": { \"OutputS3Region\": \“us-east-1\",

\"OutputS3BucketName\": \“MyBucket\", \"OutputS3KeyPrefix\": \“MyPrefix\" } }“

Configures all instances that match the tag query and reapplies every

30 minutes

Parameter Store

Parameter Store

• Centrally store and find config data

• Repeatable, automatable management (e.g. SQL

connection strings)

• Granular access control – view, use and edit values

• Encrypt sensitive data using your own AWS KMS keys

Parameter Store – Getting started

• Parameter: Key-value pair

• Secure Strings: Encrypt sensitive parameters with your

own KMS or default account encryption key

• Reuse: In Documents and easily reference at runtime

across EC2 Systems Manager using {{ssm:parameter-

name}}

• Access Control: Create an IAM policy to control access

to specific parameter

Creating and using a parameter

aws ssm put-parameter

--name mycommand

--type string

--value “dir C:\Users”

aws ssm send-command

--name AWS-RunPowerShellScript

--parameters commands=[“echo {{ssm:mycommand}}”]

--target Key=tag:Name,Values=WebServer

Demo!

Best-practices and FAQs

• What OS platforms are supported?

• Update your SSM agent today to get started!

• What ports or network access do my instances need?

• Is there anything different to set up on-premises servers?

• Use notifications, velocity control

• For disruptive actions, use Run Command with Maintenance

Window

• Fine-grained access control through IAM policies on resources (e.g.

documents)

• Customize configuration with idempotent scripts for State Manager

Your Feedback is Important!

• These services are available today

• Learn more at https://aws.amazon.com/ec2/run-

command/

• Technical documentation at

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/

run-command.html

• Please send your feedback, improvements, requests to

ec2-ssm-feedback@amazon.com

Thank you!

Remember to complete

your evaluations!

Related Sessions

WIN401 - How to Manage Inventory, Patching, and System

Images for Your Hybrid Cloud with AWS Management

Capabilities