Download - (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Transcript
Page 1: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

November 12, 2014 | Las Vegas, NV

Thomas Lobinger and Jonathan Weiss, Amazon Web Services

Page 2: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 3: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 4: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 5: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

OpsWorks

Backend

OpsWorks

Agent

Amazon EC2, Amazon EBS, EIP,

Amazon VPC, Elastic Load Balancing….

Auto-Scaling, Auto-Healing….

On-instance execution via

Chef client/zero

Command

JSON

Command

Log+Status

Page 6: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Chef Client

&

Chef Server

Page 7: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 8: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 9: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 10: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 11: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 12: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 13: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 14: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 15: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

AWS OpsWorks

recipes

Yourcustomrecipes

Page 16: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 17: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 18: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

search(:node, “name:web1”)

search(:node, “name:web*”)

Page 19: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

appserver = search(:node, "role:php-app").first

Chef::Log.info(”Private IP: #{appserver[:private_ip]}")

Page 20: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

appserver = search(:node, "role:php-app").first

Chef::Log.info(”Private IP: #{appserver[:private_ip]}")

Page 21: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

{"opsworks": {

"data_bags": {"bag_name1": {

"item_name1: {"key1" : “value1”,"key2" : “value2”,...

}},"bag_name2": {

"item_name1": {"key1" : “value1”,"key2" : “value2”,...

}},...

}}

}

Page 22: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

{ "opsworks": {

"data_bags": {"myapp": {"mysql": {

"username": "default-user","password": "default-pass"

}}

}}

}

mything = data_bag_item("myapp", "mysql")Chef::Log.info("username: #{mything['username']}")

Page 23: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 24: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

user = ENV[“payment_provider_user“]password = ENV[“payment_provider_password“]

PaymentGateway.new(user, password)

Page 25: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 26: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 27: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 28: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 29: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 30: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 31: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Change

cookbook

Push

change

Stop an

instance

Update

cookbooks

Start an

instance

Trigger

event or

runlist

Check logs

Page 32: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 33: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

list_commands

get_json

run_command

show_log

Page 34: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

list

# list event history (up to 10 events)

# opsworks-agent-cli list

2014-11-06T10:26:58 setup

2014-11-06T10:55:13 configure

2014-11-06T14:59:05 execute_recipes

2014-11-07T18:25:53 deploy

2014-11-07T18:34:11 configure

2014-11-08T10:05:23 configure

2014-11-08T14:43:43 configure

Page 35: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

get

# show JSON events (latest or by type/timestamp)

# opsworks-agent-cli get

{

"ssh_users": {

"2066": {

"name": "thomas",

"public_key": "ssh-rsa AAAAB3NzaC1...",

"sudoer": true

...

Page 36: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

# list current agent folder and cookbooks

# ls /opt/aws/opsworks/current

cookbooks # OpsWorks cookbooks

berkshelf-cookbooks # Cookbooks from Berkshelf

site-cookbooks # Your cookbooks

merged-cookbooks # Resulting cookbooks for run

...

Page 37: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

run

# run last setup event (changed cookbooks, same JSON)

# opsworks-agent-cli run setup

# or select specific date

# opsworks-agent-cli run 2014-11-08T14:43:43

[2014-11-12 13:06:41] INFO [opsworks-agent(17817)]: About to re-run

'setup' from 2014-11-08T14:43:43

Waiting for process 17817

...

Finished Chef run with exitcode 0

Page 38: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

# change recipe

# cd /opt/aws/opsworks/current/site-cookbooks

# vim mycookbooks/recipes/default.rb

# confirm success

# opsworks-agent-cli run setup

# commit back to your git repo

# git commit -am “can develop on instance“

# git push

Page 39: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 40: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 41: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Destroy Create Converge Setup Verify Destroy

Page 42: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

.kitchen.yml

driver:

name: opsworks

stack_id: vvvvvvvv-wwww-xxxx-yyyy-zzzzzzzzzzzz

provisioner:

name: shell

busser:

ruby_bindir: /opt/aws/opsworks/local/bin

Page 43: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

.kitchen.yml

platforms:

- name: Amazon Linux 2014.09

- name: Ubuntu 12.04 LTS

- name: Ubuntu 14.04 LTS

suites:

- name: default

run_list:

- recipe[demo::default]

Page 44: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

# destroy, create, setup, converge, verify, destroy

kitchen test

kitchen verify [NAME|REGEXP|all] # run all 'tests'

kitchen verify ubuntu # only ubuntus

kitchen verify client # only client tests

kitchen verify -c NUMBER # concurrency

# Talk by Fletcher Nichol http://youtu.be/YzlCHAbJ7KM

Page 45: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

‣ integration

.kitchen.yml

metadata.rb

Page 46: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

▾ integration

‣ default <-- test suite name

.kitchen.yml

metadata.rb

Page 47: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

▾ integration

▾ default <-- test suite name

‣ bats <-- test framework

.kitchen.yml

metadata.rb

Page 48: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

▾ integration

▾ default <-- test suite name

▾ bats <-- test framework

htop_installed.bats <-- the actual test

.kitchen.yml

metadata.rb

Page 49: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Test file htop_installed.bats

#!/usr/bin/env bats

@test "htop binary is found in PATH" {

run which htop

[ "$status" -eq 0 ]

}

Page 50: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 51: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 52: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 53: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

5

commit

Stack

on

EC2

Page 54: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 55: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 56: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

$./opsrun.rb 92ca56ec-1672-4db0-8c6e-a8566e7db671 'echo $SHELL'ruby: warning: RUBY_HEAP_MIN_SLOTS is obsolete. UseRUBY_GC_HEAP_INIT_SLOTS instead.About to run a script on stack 92ca56ec-1672-4db0-8c6e-a8566e7db671 on 2 online instances......jenkins2:/bin/bashphp-app1:/bin/bash$

Page 57: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 58: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 59: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 60: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 61: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 62: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

http://bit.ly/awsevals