Chef for Openstack

Mohit Sethimohit@sethis.in

Whoami?

Mohit Sethi

Developer, Technical Lead

Senior Engineer at HP R&D

You?

• Developers?

• System Administrators?

• Architects?

Journey so far?

• 2010-11: • CFEngine,

• Puppet

• Chef

• 2011 - Present• Contributed to Chef core,

• Contributed to Knife cloud plugins such as ec2, azure, hp, openstack, rackspace, google, cloudstack, vsphere, vcloud

• Written extensions for automation tools such as Vagrant, vagrant-hp, vagrant-vsphere

Goal for today

• Configuration Management Framework – Opscode Chef,• Principles,

• Automation Constructs

What is Chef?

Chef is a systems integration framework, built to bring the benefits of configuration management to you entire infrastructure.

Wait, What?

What is Chef?

Chef is a tool that allows you to define the state your servers(local or cloud) should be in and then enforces that state on your servers.

An API for your entire Infrastructure.

A service that exposes data about the state of your infrastructure

Why should I use Chef?

You have Servers.

You need to configure them.

Why should I use Chef?

But I’ve my AWESOME bash scripts, which already does most of ‘those’ things,

Why else?

We are developers, we write multi-tier applications

Why else?

We like to make things interesting,

Why else?

Application grows,

Why else?

Why else?Meanwhile 6 months later,

• How did I do that?• Who changed that?• Why did I do it what way?

• Then It dies,• I have to rebuild it• Did I forget anything• How did I do it

• And you will be >>

Why else?

And you will be…

Why Chef?Provides:

Architecture(1000’ view)

Chef Client runs on your servers

Client talks to a Chef Server

Clients authenticate with RSA keys

Knife is used to talk to Chef-Server & initiate convergence of a server.

Principles

• Idempotent

• Provision Often

• Infrastructure As Code

• Data-Driven

• Thick Clients, Thin Server

#1 Idempotent

#1 Chef rule: Recipes/ Infrastructure code should be Idempotent.

The number of Chef runs should not affect the state of the server. The server should converge on the first run. And unless previously defined state changes, additional runs should not change anything.

Say “what to do” not “how”

#2 Provision Often

If your recipes are not idempotent refer rule #1.

If they are, you should consider provisioning your servers often. Possibly every 5 minutes. Seriously.

#3 Infrastructure As Code

Infrastructure should be represented as code,

Server configuration, packages installed, relationships with other servers, should be modeled with code to be automated.

#4 Data-Driven• Separate of policy & data (implemented using Attributes &

DataBags)

• Infrastructure code should not have sensitive data. Though it can have sane defaults.

• Sensitive data should be remain in a secured store, and should only be shared with authorized clients.

#5 Thick Clients, Thin Servers

As much as possible much work is done by Chef-Client(Nodes)

Pull not Push. Chef-client runs on each node & will interact with server when it needs to.

Server is designed to distribute data to each node, including cookbooks, recipes, templates, files and so on.

Server also retains a copy of state of node at the conclusion of every chef-client.

Okay! let’s write some infrastructure code…

Vocabulary

• Nodes == Servers

• Attributes ≈ Variables

• Roles can define a Node’s attributes and what Recipes are applied to that Node

• Clients == Anything that uses the API

• Resources are the basic building blocks to define state

• Related Resources are grouped into Recipes

• Related Recipes are grouped into Cookbooks

Do I need to know Ruby?

A little

It’s a simple syntax

Chef-solo

Chef can also run stand-alone

Nodes == Servers

Nodes have Attributes

Attributes == Variables

Attributes are Searchable

$ knife search node ‘platform:centos’

search(:node, ‘platform:centos’)

Attributes

Attributes == Variables

Attributes are Searchable

$ knife search node ‘platform:centos’

search(:node, ‘platform:centos’)

Nodes have RunList

A RunList defines:

What Roles or Recipes to apply in Order.

$ knife node show ks.ms.openstack.com –r {

“run_list”: [

“role[os-base]”,

“role[os-identity]”,

]

}

Nodes have Roles

Role: What describes a node• webserver

• dbserver

• glance-server

• keystone-server

• …etc

Roles have RunList

Roles

• Roles have Run-List

Roles

• Can have other roles!

Roles

• Can override default attributes

Roles

• Roles are Searchable

• To find all roles where attribute: max_children takes value 50.

$ knife search role ‘max_children:50’

search(:role, ‘max_children:50’)

Chef manages Resources on Nodes

• Resource: Declare a description of the state a part of node should be in.

• Have a type

• Have a name

• Have parameters

• Take action to put the resource in the declared state

• Can send notification to otherresources.

Providers

• Resource take action through providers.

• Know how to actually perform the actions specified by a resource,

• Multiple providers per resources type

• Eg. Resource “package” have providers apt, yum, rubygems, portage, macports, FreeBSD ports, etc

Resources

Platform

Provider

Recipes

• Recipes are list of Resources

• Apply resources in the order they are specified

• Recipes are `import` other recipes,

Recipes are just Ruby!

Cookbooks

• Cookbooks are packages for recipes,

• Distributable

• Versioned controlled.

• Can have dependency over other Cookbooks

Cookbook Structure

• Attributes

• Assets(Files/Templates)

• Providers

• Resources

• Recipes

• Metadata

Cookbook Metadata• Declares:

• Platform support

• Dependencies

• Recipes

DataBags

• A data bag is a global variable that is stored as JSON data and is accessible from a server.

• Create a data bag using knife

$ knife data bag create DATA_BAG_NAME (DATA_BAG_ITEM)

users = Chef::DataBag.new

• Can be encrypted

• Data values can be fetched from Recipes

Community Cookbooks

• 1000+ cookbooks for everything- databases, applications, CMS, package management, Hadoop,Cloud deployments

• http://community.opscode.com

• https://launchpad.net/openstack-chef

Platform Support

• Debian

• Ubuntu

• RHEL

• Centos

• OS X

• Windows

• FreeBSD

• SUSE Enterprise

• Solaris

• SUSE

• IBM AIX

Chef Flavors

• Opscode Hosted-Chef • http://manage.opscode.com

• Hosted SaaS version of Chef.

• Opscode Enterprise/Private Chef • Private deployments of Opscode Chef Server

• On-Premise deployments

• Open Source Chef• Installation

Cloud support

• Cloud support by Knife• EC2, Rackspace, HP, Google, Azure, CloudStack, OpenStack, vSphere, vCloud, Joyent, etc

• Extensible • Implement own Resources & Providers,

Questions??

self.intro do |mohit|

mohit.twitter = @mohitsethi, @openstackindia

mohit.email = mohit@sethis.in

end