Chef and OpenStack Workshop from ChefConf 2013

Chef and OpenStackMatt RayApril 24, 2013

Agenda

• OpenStack Overview• Chef for OpenStack Overview• Knife OpenStack• Lunch 12:15• Chef for OpenStack Grizzly Roadmap• Related Technologies• Code Walkthroughs

Introductions

• Matt Ray• [email protected]• mattray IRC, GitHub• @mattray

http://upload.wikimedia.org/wikipedia/commons/2/27/Hi_How_Are_You_Austin_2005.jpg

Presenter notes: Austin, San Antonio, Santa Clara, Boston, San Francisco, San Diego, PortlandBexar and onMercado Libre, Dell, Rackspace, HP, DreamHost

mailto:[email protected]




Agenda


"To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to

implement and massively scalable."

Mission Statement

Why OpenStack?

‣ Control. Open source, no vendor lock in. Apache 2 license.

‣ Flexibility. Modular design integrates legacy and third party technologies.

‣ Emerging Industry Standard. More than 180 technology industry leaders backing it and major public clouds built on OpenStack.

‣ Proven. Originally built for scale and redundancy at NASA and Rackspace. More than 200 large-scale deployments worldwide.

‣ Compatible and Connected. Enables portability.

Control. Open source means you’re never locked to a proprietary vendor. You always have visibility and the ability to directly influence the roadmap through the open design process.

Flexibility. Modular design can integrate with legacy systems and third-‐party technologies, so you don't have to rip-‐and-‐replace your exisAng infrastructure.

Emerging Industry Standard. More than 170 leading technology companies across the globe are developing and building tools for OpenStack, including AT&T, Cisco, Dell, HP, Intel, IBM, MicrosoP and Red Hat, and new OpenStack clouds are coming online daily.

Proven and Scalable. OpenStack was built for scale and redundancy. You can run the same soPware that today powers some of the world's largest public and private clouds.

Compa<ble and Connected. CompaAbility with public OpenStack clouds means enterprises are prepared for the future—making it easy to migrate data and applicaAons to public clouds when condiAons are right.

OpenStack Controls Compute, Storage & Networking

Architecture

Architecture: Nova

Compute: Nova‣ Virtual Machines

‣ Provision and manage large pools of on-demand computing resources (hypervisors & instances)

‣ KVM

‣ Xen

‣ LXC

‣ Hyper-V

‣ VMware

‣ Bare-metal

Compute (codenamed "Nova") provides virtual servers upon demand. Rackspace and HP provide commercial compute services built on Nova and it is used internally at companies like Mercado Libre, AT&T and NASA (where it originated).

Architecture: Cinder

Block Storage: Cinder‣ Virtual Block Storage Devices

‣ Volumes on commodity storage gear

‣ Drivers for more advanced systems like NetApp, Solidfire, Ceph and Nexenta

‣ Released in Folsom Fall 2012

Block Storage (codenamed "Cinder") provides persistent block storage to guest VMs. This project was born from code originally in Nova (the nova-volume service described below). Please note that this is block storage (or volumes) not filesystems like NFS or CIFS share.Cinder is new for the Folsom release.

Architecture: Quantum

Networking as a Service: Quantum‣ Virtualized Networking

‣ Software Defined Networking (SDN)

‣ Automation of hardware & software

‣ Define network connectivity & addressing used by devices from other services

‣ Drivers for Ryu, Floodlight, Nicira, Midokura, Cisco and many more

‣ Released in Folsom Fall 2012

Network (codenamed "Quantum") provides "network connectivity as a service" between interface devices managed by other OpenStack services (most likely Nova). The service works by allowing users to create their own networks and then attach interfaces to them. Quantum has a pluggable architecture to support many popular networking vendors and technologies. Quantum is new in the Folsom release.

Architecture: Glance

Image Registry: Glance‣ Multi-format virtual disk image registry & catalog‣ Delivery of images to Nova Compute‣ Allows uploads of private and public images in a wide variety of formats

‣ Machine (kernel/ramdisk outside of image, a.k.a. AMI)

‣ qcow2 (Qemu/KVM)

‣ VMDK (VMWare)

‣ OVF (VMWare, others)

‣ And more

Image (codenamed "Glance") provides a catalog and repository for virtual disk images. These disk images are mostly commonly used in OpenStack Compute. While this service is technically optional, any cloud of size will require it.

Architecture: Keystone

Identity: Keystone‣ Unifies all core projects with common authentication system‣ Provides authorization for multiple log-in credentials

‣ Username/password

‣ Token-based

‣ AWS-style logins

‣ Integrate with existing systems

Identity (codenamed "Keystone") provides authentication and authorization for all the OpenStack services. It also provides a service catalog of services within a particular OpenStack cloud.

Architecture: Swift

Object Storage: Swift‣ Redundant, resilient, horizontally scalable object storage‣ Petabytes of reliable storage on standard gear‣ Examples include virtual machine images, photo storage, email storage

and backup archiving‣ Rackspace Cloud Files

Object Store (codenamed "Swift") allows you to store or retrieve files (but not mount directories like a fileserver). Several companies provide commercial storage services based on Swift. These include KT, Rackspace (from which Swift originated) and Internap. Swift is also used internally at many large companies to store their data.

Architecture: Horizon

Web Dashboard: Horizon‣ Self-service, role-based web interface for users and administrators‣ Provision cloud-based resources through a self-service portal‣ Create and manage projects and users, defining resources available to

them‣ Extensible design makes it easy to plug in and expose third party

products and services‣ Django application that consumes APIs

Dashboard (codenamed "Horizon") provides a modular web-based user interface for all the OpenStack services. With this web GUI, you can perform most operations on your cloud like launching an instance, assigning IP addresses and setting access controls.

OpenStack Community

‣ OpenStack Summits (Spring & Fall)‣ IRC (irc.freenode.net)

‣ #openstack, #openstack-meetings, #openstack-chef, many more

‣ Mailing Lists (lists.openstack.org)‣ OpenStack.org

‣ Blog, Docs, Wiki

‣ Twitter @OpenStack

OpenStack Code

‣ GitHub‣ https://github.com/openstack/

‣ Gerrit‣ https://review.openstack.org

‣ Jenkins‣ https://jenkins.openstack.org/

‣ DevStack.org

https://github.com/openstack/

https://github.com/openstack/

https://jenkins.openstack.org/




OpenStack Foundation

Led by Executive Director, Jonathan Bryce, the Foundation is hiring 10-12 employees who, under the strategic direction of the Board, will help carry out the OpenStack mission. Specific responsibilities include coordinating the project's infrastructure, such as systems for testing the software at scale, community building activities, and managing the OpenStack trademark, which was transferred from Rackspace following the first board meeting.

OpenStack Distributions

‣ Linux Distributions‣ Debian

‣ Fedora

‣ Red Hat

‣ SUSE

‣ Ubuntu

‣ Commercial Offerings‣ CloudScaling

‣ Mirantis

‣ Nebula

‣ Piston

‣ Rackspace

‣ ...many more

Grizzly Release‣ 550 Developers‣ 2500 conference attendees‣ Red Hat, IBM, Rackspace‣ Incubating:

‣ Commons: Oslo

‣ Metering: Ceilometer

‣ Orchestration: Heat

• Ceilometer is a metering project. The project offers metering information and the ability to code more ways to know what has happened on an OpenStack cloud. While it provides metering, it is not a billing project. A full billing solution requires metering, rating, and billing. Metering lets you know what actions have taken place, rating enables pricing and line items, and billing gathers the line items to create a bill to send to the consumer and collect payment. Ceilometer is available as a preview.

• Heat provides a REST API to orchestrate multiple cloud applications implementing standards such as AWS CloudFormation.

Agenda


Why Chef?

I'm here to tell you

Chef is Infrastructure as Code

http://www.flickr.com/photos/louisb/4555295187/

• Programmatically provision and configure

• Treat like any other code base

• Reconstruct business from code repository, data backup, and bare metal resources.

When dealing with Chef, need to literally “think outside the box”, by shifting your thinking about configuration away from a single system, to that of an Application Infrastructure. The concept of an Infrastructure is an abstract one with a specific technical meaning. When we talk about Infrastructure, we mean..



Declarative Interface to Resources

• Define policy

• Say what, not how

• Pull not Push

http://www.flickr.com/photos/bixentro/2591838509/

Chef gives you declarative interfaces into the Resources on those Nodes.Being declarative means that you say what you want to do, instead of how to do it.For example, you declare that package foobar-1.2.3 should be installed, or that the directory /var/log/foobar should exist.Chef pulls down policy from the chef-server, ensuring that a node down for maintenance will receive its policy update when it comes back online.



Ruby!

extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend

Because we use a 3GL for the recipe config files, we can use features of ruby like case statements and iterative loops.Sysadmins don’t need to be afraid of Ruby, they’ve been dealing with sub-standard programming languages like configuration files for years. They’re also not limited to what the language tells them they can do.

Recipes and Cookbooks

• Recipes are collections of Resources

• Cookbooks contain recipes, templates, files, custom resources, etc

• Code re-use and modularity

• Hundreds already on Community.opscode.com

http://www.flickr.com/photos/shutterhacks/4474421855/



The Chef Community

• Apache License, Version 2.0

• 1300+ Individual contributors

• 200+ Corporate contributors

• Dell, DreamHost, HP, Rackspace, VMware, SUSE and many more

• 900+ cookbooks

• http://community.opscode.com

Chef is hackable! Permissive Apache2 license, vibrant community of awesome folks.Community is very important to us.That's why we're here.

http://community.opscode.com

http://community.opscode.com

http://www.flickr.com/photos/16339684@N00/2681435235/

Chef and OpenStack?

That's great and all, but tell me about

OpenStack!

And a basic fact about Infrastructure -- it EVOLVES.

Deploying OpenStack

• Chef ties it all together automatically

• Scaling changes how we deploy

• Interchangeable components

• Configurations shared, supported & documented

• Licensing makes it available to everyone

We've learned a lot of things.

Overview & Status of Chef for OpenStack

Chef for OpenStack: Who

• Arista

• AT&T

• Baremetal Cloud

• Calxeda

• Dell

• DreamHost

• HP

• HubSpot

• IBM

• Intel

• Internap

• Mercado Libre

• Mirantis

• NTT

• Nebula

• Nicira

• Piston Cloud

• Rackspace

• SUSE

• TryStack.org

• Voxel

• ...and more

These companies are currently involved to some extent. Some are paying customers that we've done engagements with.

Chef for OpenStack: Why

• Community for the automated deployment and management of OpenStack

• Reduce fragmentation and encourage collaboration

• Deploying OpenStack is not "secret sauce"

• Project not a product

• Apache 2 license

Chef for OpenStack: What

• Chef Repository for Deploying OpenStack

• Documentation for Chef for OpenStack

• Cookbooks• Keystone

• Glance

• Nova

• Horizon

• Swift

• Quantum

• Cinder

• knife-openstack

Chef for OpenStack: Where

• #openstack-chef on irc.freenode.net

• github.com/opscode/openstack-chef-repo

• github.com/opscode-cookbooks/

• keystone, glance, nova, horizon, swift,quantum,cinder

• github.com/opscode/knife-openstack

• github.com/mattray/openstack-chef-docs

• groups.google.com/group/opscode-chef-openstack

• @chefopenstack

• Chef repo for Essex/Grizzly

• Operating Systems (Ubuntu 12.04)

• Hypervisors (KVM, LXC)

• Databases (MySQL)

• Nova network FlatDHCP HA & VLAN

• Quantum Nicira plugin available

• Test Kitchen integration

Chef for OpenStack: When (Today)

Chef for OpenStack: When (Tomorrow)

• Grizzly sprint scheduled in 2 weeks

• Merging AT&T, DreamHost, HubSpot and Rackspace code

• Documentation (docs.opscode.com)

• Build packages from source

• Continuous integration

• Hypervisors (Hyper-V, bare metal)

• Databases (PostgreSQL)

• Cinder (Ceph)

• Quantum (Midokura)

• Operating Systems (RHEL, Debian, SUSE)

• Documentation (docs.opscode.com)

• HA Configurations

Chef for OpenStack: When (Roadmap)

ResourcesChef for OpenStackEcosystem

Mercado LibreHPRackspaceDellTrystackVoxelOpenStack

AT&T

• github.com/att-cloud/

• primary Folsom merge source

• openstack-common

• cinder (lvm, netapp, rbd)

• lots of support cookbooks

http://github.com/att-cloud/


Crowbar

• Hardware provisioning and application management platform

• crowbar.github.com

• Dell, SUSE, others

• likely Swift cookbook source combined with Rackspace's



DreamHost

• github.com/dreamhost

• ceph

• ceilometer

• quantum

https://github.com/dreamhost

https://github.com/dreamhost

• Nicira NVP cookbook

• Open vSwitch cookbook

• Development in progress by Opscode

• github.com/gmiranda23/nvp-cookbook

Nicira

https://github.com/gmiranda23/nvp-cookbook

https://github.com/gmiranda23/nvp-cookbook

Rackspace Private Cloud

• www.rackspace.com/cloud/private/

• github.com/rcbops/chef-cookbooks

• primary Essex merge source

• likely Red Hat source

http://www.rackspace.com/cloud/private/


• Cookbooks reusable outside of OpenStack

• Test Kitchen

• knife-rackspace/hp

• Crowbar, pxe_dust & Razor

• Arista EOS cookbook

• Berkshelf & Librarian

• Spiceweasel & Sputnik

Chef for OpenStack "Halo Effect"

Why the Cloud?Why OpenStack?

The solution to this perceived impediment to resources

• Instant infrastructure

• Unlimited capacity

• Autoscaling

• No commitment

• Immediate replacement

Why the Cloud?

Enforces good architecture No long term commitment Cloud benefits – instant infrastructure Immediate replacement, no sparing etc. Unlimited storage, snapshots (1 TB volume limit) Provisioning APIs, autoscaling, EU storage, geodistPublic, private, hybrid

• Real Open Source

• Anyone can play

• Choice of features

• Features achieving parity/accelerating ahead

Why OpenStack?

Know our escape plan for every infrastructure

provider

"Drink the cloud Kool-aid, but only drink our Kool-aid"If there are problems that you have with your cloud provider...

Not just the cloud

Chef for Infrastructure Portability

• knife ec2

• knife rackspace

• knife hp

• knife google

• knife azure

• knife cloudstack

• knife openstack

• knife vcloud

• ... and many others

From EC2 to Rackspace, HP or any other OpenStack provider

• Vagrant

• VMware

• CloudStack

• Eucalyptus

• OpenStack

• bare metal

• AWS

• Rackspace

• HP

• Google

• Azure

• many others

Desktop, Virtualization, Private & Public Clouds

Chef is hackable! Permissive Apache2 license, vibrant community of awesome folks.More than 360 individual contributors, over 70 corporate contributors. Community is very important to us.That's why we're here.

• Vagrant

• VMware

• CloudStack

• Eucalyptus

• OpenStack

• bare metal

Desktop, Virtualization, Private & Public Clouds

• AWS

• Rackspace

• HP

• Google

• Azure

• many others

Chef is hackable! Permissive Apache2 license, vibrant community of awesome folks.More than 360 individual contributors, over 70 corporate contributors. Community is very important to us.That's why we're here.

Chef for OpenStack TL;DL

• Project, not a product

• Lots of contributors with real deployments in a vibrant ecosystem

• Essex works, Grizzly soon

• Features driven by demand

• Documentation with examples

• Do real work with OpenStack

From EC2 to Rackspace, HP or any other OpenStack provider

Agenda


knife openstack

Knife is our command line tool, literally a swiss army knife of cloud APIsIt talks to the Chef server to manage your infrastructure, but it also talks to APIs like the OpenStack one

So even if you're not managing your OpenStack layer, you have Chef to manage the components on top of it.

knife openstack

$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)

** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack group list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)

This is a supported knife plugin for Chef, so we have ticket tracking and everything for it.It has the basics, server creation, deletion and listing available images and servers

knife openstack flavor list

$ knife openstack flavor listID Name Virtual CPUs RAM Disk 1 m1.tiny 1 512 MB 0 GB 2 m1.small 1 2048 MB 10 GB3 m1.medium 2 4096 MB 10 GB4 m1.large 4 8192 MB 10 GB5 m1.xlarge 8 16384 MB 10 GB

update your knife.rb with

### Note: If you are not proxying HTTPS to the OpenStack EC2 API port, the scheme should be HTTP, and the PORT is 8773.

you can get these from "knife node show blahblah -a nova"

knife openstack group list

$ knife openstack group listName Protocol From To CIDR Descriptiondefault tcp 22 22 0.0.0.0/0 default default icmp -1 -1 0.0.0.0/0 default haproxy tcp 22002 22002 0.0.0.0/0 22022

knife openstack image list

$ knife openstack image listID Name 4a197431-503d-4b85-b61e-84af21ca8654 cirros-image f8ebb842-c0c0-4be3-8c4c-f72f48edec50 precise-image




knife openstack server create -h

knife openstack server create (options) --bootstrap-version VERSION The version of Chef to install -N, --node-name NAME The Chef node name for your new node -s, --server-url URL Chef Server URL -k, --key KEY API Client Key --[no-]color Use colored output, defaults to enabled -c, --config CONFIG The configuration file to use --defaults Accept default values for all questions --disable-editing Do not open EDITOR, just accept the data as is -d, --distro DISTRO Bootstrap a distro using a template; default is 'chef-full' -e, --editor EDITOR Set the editor to use for interactive commands -E, --environment ENVIRONMENT Set the Chef environment -f, --flavor FLAVOR_ID The flavor ID of server (m1.small, m1.medium, etc) -a, --floating-ip [IP] Request to associate a floating IP address to the new OpenStack node. Assumes IPs have been allocated to the project. Specific IP is optional. -F, --format FORMAT Which format to use for output --[no-]host-key-verify Verify host key, enabled by default -i IDENTITY_FILE, The SSH identity file used for authentication --identity-file -I, --image IMAGE_ID The image ID for the server -u, --user USER API Client Username --openstack-api-endpoint ENDPOINT Your OpenStack API endpoint --insecure Ignore SSL certificate on the Auth URL -K, --openstack-password SECRET Your OpenStack Password -T, --openstack-tenant NAME Your OpenStack Tenant NAME -A, --openstack-username KEY Your OpenStack Username --prerelease Install the pre-release chef gems --print-after Show the data after a destructive operation --private-network Use the private IP for bootstrapping rather than the public IP -r, --run-list RUN_LIST Comma separated list of roles/recipes to apply -G, --groups X,Y,Z The security groups for this server -S, --ssh-key KEY The OpenStack SSH keypair id -P, --ssh-password PASSWORD The ssh password -x, --ssh-user USERNAME The ssh username --template-file TEMPLATE Full path to location of template to use -V, --verbose More verbose output. Use twice for max verbosity -v, --version Show chef version -y, --yes Say yes to all prompts for confirmation -h, --help Show this message




$ knife openstack server listInstance ID Name Public IP Private IP Flavor Image Keypair State08f2d9f7-eeb0-45e7-8562-63aed8f096cc os-45539345723309377 50.56.12.229 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active43c6bbf5-b397-4986-8aec-392d955ce5b1 os-9924426691020416 50.56.12.232 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo activec1b9e3df-e566-4378-8a52-ed998b516608 os-553425714287088 50.56.12.230 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo activef3edc5da-ef99-4acb-a141-d957e09809e3 os-07459550287500682 50.56.12.231 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active

knife openstack server list

How did we get to the point where we can build a multi-tiered, monitored infrastructure?

knife openstack server create -a -f 2 -I 737969f8-6091-4896-ba9c-f3cf63bd25c5 -S rs-demo -i ~/.ssh/rs-demo.pem -x ubuntu -r "role[base]"

knife openstack server create


knife openstack server create

Instance Name: os-45539345723309377Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096cc

Waiting for server.........Flavor: 2Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5SSH Identity File: /Users/mray/.ssh/rs-demo.pemSSH Keypair: rs-demoPublic IP Address: 10.241.0.12Floating IP Address: 50.56.12.229

Waiting for sshd.....doneBootstrapping Chef on 50.56.12.229

Instance Name: os-45539345723309377Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096ccFlavor: 2Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5SSH Keypair: rs-demoPublic IP Address: 50.56.12.229Environment: _defaultRun List: role[base]




knife-openstack compatibility

• Uses the OpenStack API

• Diablo, Essex, Folsom, Grizzly

• Cloudscaling

• Crowbar

• DreamHost

• Nebula

• Piston

• Rackspace Private Cloud

knife-openstack Roadmap


• docs.opscode.com/plugin_knife_openstack.html

• tickets.opscode.com/browse/KNIFE/component/

• windows support

• floating IP address management

• quantum network management

http://docs.opscode.com/plugin_knife_openstack.html


http://tickets.opscode.com/browse/KNIFE/component/11094


Agenda


Who was there

• AT&T

• Dell

• DreamHost

• HubSpot

• KT

• Midokura

• Opscode

• Rackspace

• SUSE

There had been several days of conversations, these companies were all represented in the meeting. Missing: eNovance, HP, IBM, KIO, Mirantis

Where

• #openstack-chef on irc.freenode.net

• groups.google.com/group/opscode-chef-openstack

• github.com/mattray/openstack-chef-docs

• @chefopenstack

The resources we're using

Licensing

• Apache 2

• Opscode CLA/CCLA required

• http://wiki.opscode.com/display/chef/How+to+Contribute

• http://wiki.opscode.com/display/chef/Approved+Contributors

Attendees were all covered already

http://wiki.opscode.com/display/chef/How+to+Contribute




http://wiki.opscode.com/display/chef/Approved+Contributors




Where on GitHub

• http://github.com/osops

• chef-repo/

• berkshelf, not git submodules

• cookbooks all end in "-cookbook"

• ie. "nova-cookbook"

• "operations" cookbooks outside scope

• ie. logging, monitoring, provisioning

Move to community GitHub repo, not Opscode's. Opscode will upstream from this repo.

https://github.com/osops

https://github.com/osops

Cookbooks

• cinder

• glance

• horizon

• keystone

• nova

• quantum

• swift

• ceilometer & heat eventually

The core OpenStack services

Goal of incorporating into OpenStack

• Get on StackForge

• will provide CI

• which everyone will probably slave

• Gerrit for code reviews

• we'll sort out reviewers once we start

• Testing with TestKitchen initially

• Grenade? Kong? SmokeStack?

We want to go into "mainline" OpenStack

• support alternative package sources

• source-built coming ("VanillaStack")

• packaging recipes before configuration

• ie. "nova/recipes/nova-compute-packages"

Packages

Use distro packages were applicable, but not everyone wants to use them. Build from source will come in eventually.

Chef Style Guide

• Chef 11 target release

• partial search

• partial templates

• Full-stack Chef-client compatible

• Ruby 1.9.x

• Upstream community cookbooks

• Foodcritic as much as possible

Table stakes

Chef Style Guide

• openstack-common instead of osops-utils

• Attribute injection

• attributes may short-circuit search

• few, if any, attributes in roles

• environment-driven attributes

• Chef Solo not actively supported

• platform logic in attributes files

Already using these patterns

• May release "2013.1.0"

• Chef repo for Grizzly

• Operating Systems (Ubuntu 12.04)

• Databases (MySQL)

• Hypervisors (KVM, LXC)

• Nova network FlatDHCP HA & VLAN

Initial osops release

Opscode employee Matt Ray and Chris McClimans are getting together after ChefConf to work on cleaning up Grizzly. Sources will be AT&T, Dell, HubSpot and Rackspace primarily.

• Operating Systems (RHEL, SUSE)

• Databases (Postgres)

• Hypervisors (Xen, bare metal)

• Cinder (Ceph, LVM, NetApp)

• Quantum (Bridge, Midokura, Nicira, OVS)

• Folsom backport

• HA Configurations may be stretch goal because of differing implementations

Grizzly Roadmap

SUSE: SLES, OpenSUSE, PostgresKT: XenHubSpot: bare metalDreamHost: CephAT&T: LVMRackspace: Bridge, NetApp, OVS, RHELOpscode: NiciraMidokura: Folsom, MidoNetAT&T, SUSE, Rackspace different HA setups

knife-openstack v0.7.0

$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)

** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack group list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)

Currently supported features.

knife-openstack compatibility

• Uses the OpenStack API

• Diablo, Essex, Folsom, Grizzly

• Cloudscaling

• Crowbar

• DreamHost

• Nebula

• Piston

• Rackspace Private Cloud

Continue to test for compatibility, will build out CI testing for Opscode-supported knife plugins.

knife-openstack Roadmap


• docs.opscode.com/plugin_knife_openstack.html

• tickets.opscode.com/browse/KNIFE/component/

• Continues to be managed by Opscode

• Test against multiple OpenStack deployments for compatibility

• next major release v0.8.0 (May)

• floating IP address management

• network assignment on server creates

More features will undoubtedly show up








• Submit talk "Chef for OpenStack Fall 2013 Overview & Status"

• Review this deck

• Report progress

• Submit for developer track session as well

• See you in Hong Kong!

Fall 2013 OpenStack Summit

We'll see what actually happens in November.

Agenda


Berkshelf

• http://berkshelf.com/• tool for managing your Chef Cookbooks and their dependencies

• Community site, Git, local development• Berksfile managed in version control

Spiceweasel

• https://github.com/mattray/spiceweasel• manages your Chef repositories and creating reproducible infrastructure

• nodes, cookbooks, roles, data bags & environments with a version controlled manifest

• validates dependencies • allows extraction and creation of infrastructure• lightweight orchestration and cluster management

• Sputnik Cloud Launcher• fills gap between the documentation and deployment of your Chef repository & infrastructure


• http://www.rackspace.com/cloud/private/• https://github.com/rcbops-cookbooks/• Session: Deploying OpenStack with Chef and Operational Tooling

https://github.com/rcbops-cookbooks/


Test Kitchen

• kitchen-openstack• https://github.com/RoboticCheese/kitchen-openstack• Session: Test Kitchen: Multi-Platform Integration Testing for the Masses

https://github.com/RoboticCheese/kitchen-openstack

https://github.com/RoboticCheese/kitchen-openstack

pxe_dust

• Provisioning solution for hardware• Initially developed by Matt Ray• https://github.com/opscode-cookbooks/pxe_dust• pxe_dust::bootstrap_template• pxe_dust::installers• pxe_dust::server

https://github.com/opscode-cookbooks/pxe_dust

https://github.com/opscode-cookbooks/pxe_dust

Crowbar

• Provisioning solution for hardware• Initially developed by Dell• crowbar.github.com• Barclamps!• 1.0 vs. 2.0

Razor

• Provisioning solution for hardware• Initially developed by EMC and Puppet• Open sourced as a Puppet Labs project• Install using Puppet, Chef, or manual• Auto-Discovered Real-Time Inventory Data• Dynamic Image Selection• Model-Based Provisioning• APIs and Plug-in Architecture• Metal-to-Cloud Application Lifecycle Management• Session: Harnessing the Power of Bare Metal with Razor and Chef Server

OpenStack Baremetal

• https://wiki.openstack.org/wiki/Baremetal• driver to allow OpenStack Compute to manage hardware directly (Grizzly)• provisioned via PXE and managed via IPMI• OpenStack Compute manages them via the Dashboard, CLI and API• OpenStack on OpenStack (aka "Triple-O")• authentication, authorization, quotas, a dashboard and an API provided by OpenStack• roadmap has device discovery, network management and additional hardware features

Agenda


currently Folsom release (v3.0.1)

open source Chef 11 server embedded


https://github.com/rcbops









github.com/rcbops/chef-cookbooks/tree/v3.0.1

roles/

environments/

cookbooks/


https://github.com/rcbops/chef-cookbooks/tree/v3.0.1








cookbooks/apache2/apt/aws/build-essential/cinder/collectd/collectd-graphite/collectd-plugins/corosync/database/dsh/erlang/exerstack/glance/graphite/haproxy/hardware/horizon/keepalived/keystone/kong/memcached/modules/

monit/monitoring/mysql/mysql-openstack/nova/nova-network/ntp/openssh/openssl/openstack-ha/osops-utils/postgresql/rabbitmq/rabbitmq-openstack/rsyslog/runit/selinux/sosreport/swift/sysctl/xfs/yum/

environments/example.json{ "name": "example_environment", "description": "Chef-server example environment", "cookbook_versions": { }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { }, "override_attributes": { "developer_mode": false, "monitoring" : { "metric_provider" : "collectd", "procmon_provider" : "monit" }, "glance": { "image_upload": true, "images": ["cirros", "precise"] },

environments/example.json "nova": { "ratelimit" : { "api" : { "enabled" : true }, "volume" : { "enabled" : true } }, "libvirt": { "virt_type": "qemu" }, "networks": [ { "label": "public", "ipv4_cidr": "10.10.100.0/24", "num_networks": "1", "network_size": "255", "bridge": "br100", "bridge_dev": "eth0.100", "dns1": "8.8.8.8", "dns2": "8.8.4.4" }, { "label": "private", "ipv4_cidr": "172.16.101.0/24", "num_networks": "1", "network_size": "255", "bridge": "br101", "bridge_dev": "eth0.101", "dns1": "8.8.8.8", "dns2": "8.8.4.4" } ] },

environments/example.json

"mysql": { "allow_remote_root": true, "root_network_acl": "%" }, "osops_networks": { "nova": "192.168.1.0/24", "public": "192.168.1.0/24", "management": "192.168.1.0/24" }, "package_component": "folsom" }}

roles/allinone.rbbase.rbcinder-all.rbcinder-api.rbcinder-scheduler.rbcinder-setup.rbcinder-volume.rbcollectd-client.rbcollectd-server.rbglance-api.rbglance-registry.rbglance-setup.rbglance.rbgraphite.rbha-controller1.rbha-controller2.rbhaproxy.rbhorizon-server.rbjenkins-allinone.rbjenkins-compute.rbjenkins-controller.rbjenkins-glance.rbjenkins-mysql-master.rbjenkins-nova-api.rbkeystone-api.rbkeystone.rbmysql-master.rb

nova-api-ec2.rbnova-api-os-compute.rbnova-api.rbnova-cert.rbnova-controller.rbnova-misc-services.rbnova-network-compute.rbnova-network-controller.rbnova-network.rbnova-scheduler.rbnova-setup.rbnova-vncproxy.rbnova-volume.rbopenstack-ha.rbquantum-network-manager.rbrabbitmq-server.rbrsyslog-client.rbrsyslog-server.rbsingle-compute.rbsingle-controller.rbswift-account-server.rbswift-all-in-one.rbswift-container-server.rbswift-management-server.rbswift-object-server.rbswift-proxy-server.rbswift-setup.rb

roles/allinone.rbname "allinone"description "This will create an all-in-one Openstack cluster"run_list( "role[single-controller]", "role[single-compute]")

expanded run list

osops-utils::packages, openssh, ntp, sosreport, rsyslog::default, hardware, osops-utils::default, mysql-openstack::server, erlang::default, rabbitmq-openstack::server, keystone::server, keystone::keystone-api, glance::setup, glance::registry, glance::api, nova::nova-setup, nova-network::nova-controller, nova::scheduler, nova::api-ec2, nova::api-os-compute, nova::volume, nova::nova-cert, nova::vncproxy, mysql::client, mysql::ruby, horizon::server, nova-network::nova-compute, nova::compute

roles/single-controllername "single-controller"description "Nova Controller (non-HA)"run_list( "role[base]", "role[mysql-master]", "role[rabbitmq-server]", "role[keystone]", "role[glance-setup]", "role[glance-registry]", "role[glance-api]", "role[nova-setup]", "role[nova-network-controller]", "role[nova-scheduler]", "role[nova-api-ec2]", "role[nova-api-os-compute]", "role[nova-volume]", "role[nova-cert]", "role[nova-vncproxy]", "role[horizon-server]")

roles/basename "base"description "Base role for a server"run_list( "recipe[osops-utils::packages]", "recipe[openssh]", "recipe[ntp]", "recipe[sosreport]", "recipe[rsyslog::default]", "recipe[hardware]", "recipe[osops-utils::default]")default_attributes( "ntp" => { "servers" => ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org"] })

roles/mysql-mastername "mysql-master"description "Installs mysql and sets up replication (if 2 nodes with role)"run_list( "role[base]", "recipe[mysql-openstack::server]")

roles/rabbitmq-servername "rabbitmq-server"description "RabbitMQ Server (non-ha)"run_list( "role[base]", "recipe[erlang::default]", "recipe[rabbitmq-openstack::server]")

roles/keystonename "keystone"description "Keystone server"run_list( "role[base]", "recipe[keystone::server]", "role[keystone-api]")

roles/glance-setupname "glance-setup"description "sets up glance registry db and passwords"run_list( "role[base]", "recipe[glance::setup]")

roles/glance-registryname "glance-registry"description "Glance Registry server"run_list( "role[base]", "recipe[glance::registry]")

roles/glance-apiname "glance-api"description "Glance API server"run_list( "role[base]", "recipe[glance::api]")

roles/nova-setupname "nova-setup"description "Where the setup operations for nova get run"run_list( "recipe[nova::nova-setup]")

/nova-network-controller

name "nova-network-controller"description "Setup nova-networking for controller node"run_list( "recipe[nova-network::nova-controller]")

roles/nova-schedulername "nova-scheduler"description "Nova scheduler"run_list( "role[base]", "recipe[nova::scheduler]")

roles/nova-api-ec2name "nova-api-ec2"description "Nova API EC2"run_list( "role[base]", "recipe[nova::api-ec2]")

/nova-api-os-computename "nova-api-os-compute"description "Nova API for Compute"run_list( "role[base]", "recipe[nova::api-os-compute]")

roles/nova-volumename "nova-volume"description "Nova Volume Service"run_list( "role[base]", "recipe[nova::volume]")

roles/nova-certname "nova-cert"description "Nova Certificate Service"run_list( "role[base]", "recipe[nova::nova-cert]")

roles/nova-vncproxyname "nova-vncproxy"description "Nova VNC Proxy"run_list( "role[base]", "recipe[nova::vncproxy]")

roles/horizon-servername "horizon-server"description "Horizon server"run_list( "role[base]", "recipe[mysql::client]", "recipe[mysql::ruby]", "recipe[horizon::server]")

roles/single-computename "single-compute"description "Nova compute (with non-HA Controller)"run_list( "role[base]", "role[nova-network-compute]", "recipe[nova::compute]")

Thanks!Matt [email protected]@mattray



Chef and OpenStack Workshop from ChefConf 2013

Technology

Transcript of Chef and OpenStack Workshop from ChefConf 2013