Chef & OpenStack: OSCON 2014

49

description

State of integrations between Chef and OpenStack July 2014. Covers Chef for OpenStack project, knife-openstack, Test Kitchen and Chef Metal.

Transcript of Chef & OpenStack: OSCON 2014

Page 1: Chef & OpenStack: OSCON 2014
Page 2: Chef & OpenStack: OSCON 2014

Chef for OpenStackMatt Ray & JJ Asghar OSCON July 24, 2014

Page 3: Chef & OpenStack: OSCON 2014

Introductions• Matt Ray

• Director of Partner Engineering

[email protected]

• mattray GitHub|IRC|Twitter

Page 4: Chef & OpenStack: OSCON 2014

Introductions• JJ Asghar

• Sr. Partner Engineer

[email protected]

• jjasghar GitHub and Twitter

• j^2 on IRC

Page 5: Chef & OpenStack: OSCON 2014

Chef•Open Source configuration management and systems automation framework •Infrastructure as Code, written in Ruby •Abstractions of Resources on the systems •Client/server model over REST APIs •Huge, vibrant community of contributors

Page 6: Chef & OpenStack: OSCON 2014

OpenStack Controls Compute, Storage & Networking

Page 7: Chef & OpenStack: OSCON 2014

Architecture

Page 8: Chef & OpenStack: OSCON 2014

Architecture

Page 9: Chef & OpenStack: OSCON 2014

Overview & Current Status of Chef for OpenStack

Page 10: Chef & OpenStack: OSCON 2014

Chef for OpenStack: Project• Community around the automated deployment and management of OpenStack

• Reduce fragmentation and increase collaboration

• Deploying OpenStack is not "Secret Sauce"

• Project, not a 'Product'

• Apache 2 License

Page 11: Chef & OpenStack: OSCON 2014

Community• #openstack-chef on irc.freenode.net

• groups.google.com/group/opscode-chef-openstack

• Google Hangout weekly status meetings (Monday 11am EST)

• @chefopenstack

• Stackalytics (stackforge->chef-group)

Page 12: Chef & OpenStack: OSCON 2014

Who's Involved?• AT&T

• Bluebox

• Chef

• Dell

• DreamHost

• HP

• IBM

• Korea Telecom

• Rackspace

• SUSE

• and many more

Page 13: Chef & OpenStack: OSCON 2014
Page 14: Chef & OpenStack: OSCON 2014

Chef Requirements• Chef 11

• Ruby 1.9.x

• Foodcritic, Rubocop, ChefSpec for testing

• attribute-driven by Environments

• platform logic in attributes

• currently packages-only installation

Page 15: Chef & OpenStack: OSCON 2014

StackForge: Cookbooks• "Official" OpenStack repositories

• github.com/stackforge/cookbook-openstack-*

• gated by review.openstack.org

• OpenStack services for Grizzly, Havana and Icehouse cookbooks

• block-storage, common, compute, dashboard, data-processing, database, identity, image, network, object-storage, orchestration, telemetry

• Operational support cookbooks

• ceph, client, integration-testing, ops-database, ops-messaging

Page 16: Chef & OpenStack: OSCON 2014

Launchpad• Blueprints

• https://blueprints.launchpad.net/openstack-chef

• Bugs

• https://bugs.launchpad.net/openstack-chef

Page 17: Chef & OpenStack: OSCON 2014

StackForge: Deployment• Chef repository for deploying Grizzly, Havana, or Icehouse

• example Environments and Roles

• example "All-in-One" Vagrant deployment

• github.com/stackforge/openstack-chef-repo

• Gated by review.openstack.org

• More single and multi-node testing coming

Page 18: Chef & OpenStack: OSCON 2014

Reference Implementation• Deployment examples in documentation

• All-in-One Compute

• Single Controller + N Compute

• more coming

• Will provide example HA configurations

• Operations outside of scope of core repository

• logging, monitoring, provisioning

Page 19: Chef & OpenStack: OSCON 2014

docs.getchef.com/openstack.html

Page 20: Chef & OpenStack: OSCON 2014

Documentation• docs.getchef.com/openstack.html

• Architecture

• Deployment Prerequisites

• Installation

• Development

• Cookbooks and Repositories

• Example Deployments

• github.com/opscode/chef-docs

• Creative Commons, no CLA required

Page 21: Chef & OpenStack: OSCON 2014

Example Deployments• Vagrant "All-in-One" for development/testing

• nova-network or Neutron

• Ubuntu 12.04 or CentOS 6.5

• Developer lab deployment "1+N"

• Single controller, N compute boxes

• 5 boxes, consumer-grade hardware

Page 22: Chef & OpenStack: OSCON 2014

StackForge: Grizzly Status• branch ‘stable/grizzly’

• Operating Systems: Ubuntu 12.04, SLES 11 SP2

• Databases: MySQL, SQLite (testing)

• Messaging: RabbitMQ

• Compute: KVM, LXC, Qemu

• Network: Nova + Quantum (Open vSwitch)

• Block Storage: LVM

• Object Storage: Swift

• Dashboard: Apache or Nginx

Page 23: Chef & OpenStack: OSCON 2014

StackForge: Havana Status• branch ‘stable/havana’

• Operating Systems: Ubuntu 12.04, SLES 11 SP3, RHEL 6

• Databases: DB2, MySQL, SQLite (testing)

• Messaging: Qpid, RabbitMQ

• Compute: KVM, LXC, QEMU

• Network: Nova, Neutron (Open vSwitch)

• Block Storage: LVM, NetApp, others

• Object Storage: Swift

• Dashboard: Apache or Nginx

Page 24: Chef & OpenStack: OSCON 2014

StackForge: Icehouse Status• branch ‘master’

• Operating Systems: Ubuntu 12.04/14.04, SLES 11 SP3, RHEL 6

• Databases: DB2, MySQL Postgres, SQLite (testing)

• Messaging: Qpid, RabbitMQ

• Compute: KVM, LXC, QEMU

• Network: Nova, Neutron (Open vSwitch)

• Block Storage: LVM, NetApp, others

• Object Storage: Swift

• Dashboard: Apache or Nginx

Page 25: Chef & OpenStack: OSCON 2014

StackForge: Roadmap• branching for Juno (tomorrow?)

• 'master' move to ‘stable/icehouse’

• Havana available in ‘stable/havana'

• Grizzly available in ‘stable/grizzly'

• More contributors!

Page 26: Chef & OpenStack: OSCON 2014

StackForge: Roadmap• Operating Systems: Debian

• Compute: Baremetal, Docker, ESX, Hyper-V, Xen

• Network: NSX, OpenDaylight

• Block Storage: Ceph

• Object Storage: Ceph

• Source builds via Omnibus

Page 27: Chef & OpenStack: OSCON 2014

OpenStack Ops Mid-Cycle Meetup• Operators mini summit

• August 25-26, San Antonio TX

• http://www.eventbrite.com/e/openstack-ops-mid-cycle-meetup-tickets-12149171499

Page 28: Chef & OpenStack: OSCON 2014

Chef Community Summit• Developer/Community unconference

• October 2 & 3 in Seattle, WA

Page 29: Chef & OpenStack: OSCON 2014

OpenStack Summit

Page 30: Chef & OpenStack: OSCON 2014

knife-openstack

Page 31: Chef & OpenStack: OSCON 2014

Available openstack subcommands: (for details, knife SUB-COMMAND --help)!

!** OPENSTACK COMMANDS **!

knife openstack flavor list (options)!

knife openstack group list (options)!

knife openstack image list (options)!

knife openstack network list (options)!

knife openstack server create (options)!

knife openstack server delete INSTANCEID [INSTANCEID] (options)!

knife openstack server list (options)!

knife openstack server show (options)

$ knife openstack

Page 32: Chef & OpenStack: OSCON 2014

Name ID Virtual CPUs RAM Disk !m1.large 4 4 8192 MB 80 GB !m1.medium 3 2 4096 MB 40 GB !m1.small 2 1 2048 MB 20 GB !m1.tiny 1 1 512 MB 10 GB !m1.xlarge 5 8 16384 MB 160 GB

$ knife openstack flavor list

Page 33: Chef & OpenStack: OSCON 2014

Name Protocol From To CIDR Description!

default tcp 22 22 0.0.0.0/0 default!

default icmp -1 -1 0.0.0.0/0 default!

haproxy tcp 22002 22002 0.0.0.0/0 22022!

$ knife openstack group list

Page 34: Chef & OpenStack: OSCON 2014

Name ID Snapshot!

centos-6.5 68555833-8497-4d14-88ca-c9062e25f14b no !

coreos 83d37ea5-d9ae-44cd-9110-d4d39ad997ce no !

fedora-19 9add7e14-25e3-41d8-963a-ca744d081f2e no !

fedora-20 acb6eba5-226a-4ed5-8db6-33a6fd8cf20d no !

freebsd-10.0 0e270df7-1a02-4e91-9fc3-6f5311c58193 no !

ubuntu-12.04 ce268db5-ceda-4a90-93c8-3b987ac3705f no !

ubuntu-14.04 4a4f85bf-f164-4e54-83d8-8b2e7d0712b2 no !

Windows Server 2012 64e7cba7-7a50-443f-8fa6-a065406e0b04 no !

$ knife openstack image list

Page 35: Chef & OpenStack: OSCON 2014

Name ID Tenant Shared!

external 06dc9d5a-f55a-410d-a7fd-4c7cb34ad927 5da25cc3853f4c54850898f9614c20bb true !

internal ba0fdd03-72b5-41eb-bb67-fef437fd6cb4 5da25cc3853f4c54850898f9614c20bb true

$ knife openstack network list

Page 36: Chef & OpenStack: OSCON 2014

Name Instance ID Public IP Private IP Flavor Image Keypair State!

os-45539345723309377 08f2d9f7-eeb0-45e7-8562-63aed8f096cc 50.56.12.229 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active!

os-9924426691020416 43c6bbf5-b397-4986-8aec-392d955ce5b1 50.56.12.232 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active!

os-553425714287088 c1b9e3df-e566-4378-8a52-ed998b516608 50.56.12.230 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active!

os-07459550287500682 f3edc5da-ef99-4acb-a141-d957e09809e3 50.56.12.231 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active

$ knife openstack server list

Page 37: Chef & OpenStack: OSCON 2014

$ knife openstack server create -a -f 2 -I 737969f8-6091-4896-ba9c-f3cf63bd25c5 -S rs-demo -i ~/.ssh/rs-demo.pem -x ubuntu -r "role[base]"!

Instance Name: os-45539345723309377!

Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096cc!

!Waiting for server.........!

Flavor: 2!

Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5!

SSH Identity File: /Users/mray/.ssh/rs-demo.pem!

SSH Keypair: rs-demo!

Public IP Address: 10.241.0.12!

Floating IP Address: 50.56.12.229!

Waiting for sshd.....done!

Bootstrapping Chef on 50.56.12.229!

Instance Name: os-45539345723309377!

Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096cc!

Flavor: 2!

Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5!

SSH Keypair: rs-demo!

Public IP Address: 50.56.12.229!

Environment: _default!

Run List: role[base]!

knife openstack server create

Page 38: Chef & OpenStack: OSCON 2014

knife openstack server create

Page 39: Chef & OpenStack: OSCON 2014

knife openstack Compatibility• Uses the OpenStack API

• Diablo, Essex, Folsom, Grizzly, Havana, Icehouse, trunk

• BlueBox

• Cloudscaling

• Crowbar

• DreamHost

• IBM

• MetaCloud

• Nebula

• Piston

• Rackspace Private Cloud

Page 40: Chef & OpenStack: OSCON 2014

knife openstack Resources• knife openstack --help

• docs.getchef.com/plugin_knife_openstack.html

• github.com/opscode/knife-openstack

• github.com/opscode/knife-openstack/issues

Page 41: Chef & OpenStack: OSCON 2014

knife openstack Roadmap• Continuous Integration for Chef-supported knife plugins soon

• Testing against multiple deployments

• 1.0.0: common knife-cloud base class

• 1.1.0: Network enhancements

• 1.2.0 guid cleanup, other niceties

Page 42: Chef & OpenStack: OSCON 2014

Test Kitchen

Page 43: Chef & OpenStack: OSCON 2014

Test Kitchen• Integration tool for developing and testing infrastructure code and software on isolated target platforms

• Integration test platform for your cookbooks on all the supported platforms with virtual machines

• https://github.com/test-kitchen/kitchen-openstack

Page 44: Chef & OpenStack: OSCON 2014

driver:!

name: openstack!

openstack_username: [YOUR OPENSTACK USERNAME]!

openstack_api_key: [YOUR OPENSTACK API KEY]!

openstack_auth_url: [YOUR OPENSTACK AUTH URL]!

require_chef_omnibus: latest!

image_ref: [SERVER IMAGE ID]!

flavor_ref: [SERVER FLAVOR ID]

Test Kitchen: kitchen.yml

Page 45: Chef & OpenStack: OSCON 2014
Page 46: Chef & OpenStack: OSCON 2014

Chef Metal• Chef recipes for deploying infrastructure

• Libraries for repeatably creating machines and deployments with Chef primitives

• Bootstrappers for many infrastructure types

Page 47: Chef & OpenStack: OSCON 2014

Chef Metal: Providers• Cloud

• Digital Ocean, EC2, Fog, OpenStack

• Virtualization

• Vagrant (VirtualBox, Fusion), VSphere

• Containers

• Docker & LXC

• SSH

• PXE in progress

Page 48: Chef & OpenStack: OSCON 2014

Chef Metal: Example Recipe

Page 49: Chef & OpenStack: OSCON 2014

Thanks! Any questions?Matt Ray [email protected] @mattray