Automating your infrastructure with Chef
-
Upload
john-ewart -
Category
Technology
-
view
1.334 -
download
6
description
Transcript of Automating your infrastructure with Chef
![Page 1: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/1.jpg)
AUTOMATING YOUR INFRASTRUCTURE WITH CHEF
John Ewart Sr. Software Engineer @ Chef
!@soysamurai :: http://johnewart.net
![Page 2: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/2.jpg)
I hope that this talk will…
- help you see how chef’s pieces fit together
- show you some of the things chef can do
- give you the tools to explore chef on your own
- get you thinking about advanced uses
- make you want to learn more about chef!
![Page 3: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/3.jpg)
outline- 10K foot view - code examples - provisioning a server - configuring software - managing local environments - some advanced uses / ideas - resources - Q & A
![Page 4: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/4.jpg)
10,000 foot view
![Page 5: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/5.jpg)
what is chef?
chef is a tool for automating your infrastructure
![Page 6: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/6.jpg)
Chef helps solve the problem of managing systems at scale by bringing the power of code to your infrastructure. You can easily manage complex
infrastructure on your own hardware or in the cloud. !
WHY USE CHEF?
![Page 7: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/7.jpg)
‣ facebook ‣ Riot games ‣ ancestry.com ‣ Splunk ‣ nordstrom
Chef is popular among organizations large and small, from small startups to large digital companies like Facebook and Riot Games and even brick-and-mortar companies such as Nordstrom. !The community around Chef is quite large and growing by the day. In the Chef Supermarket, there are over 1,500 recipes and almost 60,000 registered users.
WHO USES CHEF? (among others)
![Page 8: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/8.jpg)
what can chef do for you?‣ provision ‣ configure ‣ deploy ‣ orchestrate
You can use Chef to provision new hosts. - knife allows you to manually provision a new
host - chef-metal can provision large numbers of
hosts in parallel !With Chef you can construct new hosts and automatically configure them with one command.
![Page 9: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/9.jpg)
‣ provision ‣ configure ‣ deploy ‣ orchestrate
Using stored configuration and recipes, you can automate configuration of hosts. !- Install software - Provision users - Configure firewalls - Update kernels - Much more!
what can chef do for you?
![Page 10: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/10.jpg)
‣ provision ‣ configure ‣ deploy ‣ orchestrate
Deploy application code from GitHub or other source control system. - Fully automate your release cycle - Only perform operations if they are needed
what can chef do for you?
![Page 11: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/11.jpg)
‣ provision ‣ configure ‣ deploy ‣ orchestrate
Chef knows about your infrastructure - How it’s organized - What your configuration looks like - Health status of your systems !Leverage centralized configuration data to make informed decisions about your systems.
what can chef do for you?
![Page 12: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/12.jpg)
- Chef uses recipes that contain a set of managed resources - The Chef engine takes a run list, which contains a list of recipes to
be executed on the node - A run list is acted on using providers for each resource, which
contain the logic needed to achieve the desired effect for a given platform (update a file, create a user, install a package)
- Using this combination of logic and configuration data, the node will converge upon the desired end-state
chef in a nutshell
![Page 13: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/13.jpg)
- Recipes: Ruby scripts that, when executed, perform a desired behavior such as install software, provision users, and so on.
- Cookbooks: Collections of recipes that focus on a specific theme such as managing PostgreSQL or deploying a custom web application
key terms
![Page 14: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/14.jpg)
- Resources: Abstract Chef primitives that represent real-world components such as: files, templates, directories, packages, users or services.
- Providers: Concrete implementations of a resource. For example, the package resource has providers for FreeBSD, Debian, RedHat, OSX and other platforms.
key terms
![Page 15: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/15.jpg)
- Node: A system that is managed by Chef, this can be any system capable of running the Chef client (and possibly other things in the future)
- Role: A collection of recipes and configuration data that, when run together, allow a node to perform a specific role; examples might include a database server or web server but are free-form.
key terms
![Page 16: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/16.jpg)
- Attributes: Pieces of data that are used to configure your infrastructure.
- Environments: Provide a way to maintain similar but different configuration data. For example you might have a production and a test environment with the same roles but different nodes and configuration data
key terms
![Page 17: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/17.jpg)
- Run list: A list of recipes to execute during a client run. This is generated based on the roles and recipes put into the run list.
- Data bags: A generic collection of data stored in Chef that is not specific to any recipe, node or other entity in the system.
key terms
![Page 18: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/18.jpg)
A resource…- Has a type such as: file, user, package, or
service (among others) - Describes an entity on the system - Is abstract and does not provide
implementation details - Contains properties that will vary by resource - Provides a building-block for writing recipes - Requires a provider to perform the actual
work
file "/tmp/hello.txt" do action :create end !
user ‘samwise’ do
uid 500
gid 1000
end
![Page 19: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/19.jpg)
A recipe…- Uses the Chef recipe DSL - Is comprised of a one or more resources - Provides step-by-step instructions for
achieving a desired end-result - Might describe how to: install a MySQL
server, create users, or setup HAproxy !
package "haproxy" !directory node['haproxy']['conf_dir'] !template "/etc/init.d/haproxy" do source "haproxy-init.erb" owner "root" group "root" mode 00755 variables( :hostname => node['hostname'], :conf_dir => node[‘haproxy__dir'], :prefix => "/usr" ) end
![Page 20: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/20.jpg)
file "/tmp/hello.txt" do end
a simple recipe
Here we use the file resource
![Page 21: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/21.jpg)
file "/tmp/hello.txt" do action :create end
a simple recipe
We specify that the action to take on this resource is to create it
![Page 22: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/22.jpg)
file "/tmp/hello.txt" do action :create content "Greetings, Earthling!" end
a simple recipe
And we can specify the contents of the file
![Page 23: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/23.jpg)
recipe development- Recipes can contain as few as one resource - A recipe may contain hundreds of resources - Each recipe should have a specific end-goal - A recipe might:
- installing a service, - provisioning users, or - deploy an application
- Recipes that are related go together inside of a cookbook - There are many existing cookbooks for common activities
![Page 24: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/24.jpg)
case node['platform_family'] when "rhel", "fedora", "suse" include_recipe "postgresql::server_redhat" when "debian" include_recipe "postgresql::server_debian" end
One powerful feature of Chef is the ability to write recipes that support multiple distributions or even multiple operating systems
Supporting multiple platforms
![Page 25: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/25.jpg)
what else can you manage?
![Page 26: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/26.jpg)
["src", "logs", "conf"].each do |child_dir| directory “/opt/webapp/#{child_dir}” do owner node[:webapp][:user] group node[:webapp][:group] mode "0755" action :create recursive true end end !
create directories
![Page 27: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/27.jpg)
home_dir = "/home/#{u['id']}"
! user u['id'] do
uid u['uid']
gid u['gid']
shell u['shell']
comment u['comment']
supports :manage_home => true
home home_dir
end
create users
![Page 28: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/28.jpg)
git "/opt/app/webapp" do repository "https://github.com/johnewart/simplewebpy_app.git" action :sync branch 'master' user node[:webapp][:user] end
deploy code
![Page 29: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/29.jpg)
app_root = node[:webapp][:install_path] !python_interpreter = node[:webapp][:python] virtualenv = "#{app_root}/python" !python_virtualenv "#{virtualenv}" do owner node[:webapp][:user] group node[:webapp][:group] action :create interpreter "#{python_interpreter}" end
manage python virtualenvs
![Page 30: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/30.jpg)
postgresql_connection_info = { :host => 'localhost', :port => node[:postgresql][:config][:port], :username => 'postgres', :password => node[:postgresql][:password][:postgres] } !postgresql_database 'webapp' do connection postgresql_connection_info action :create end
create databases
![Page 31: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/31.jpg)
postgresql_connection_info = { :host => 'localhost', :port => node[:postgresql][:config][:port], :username => 'postgres', :password => node[:postgresql][:password][:postgres] } !postgresql_database_user node[:webapp][:dbuser] do connection postgresql_connection_info password node[:webapp][:dbpass] action :create end
create database users
![Page 32: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/32.jpg)
- If you can manage it manually, you can most likely build a custom provider and resource to manage it with Chef
- The community of Chef users is quite large, you can probably find it out there on the Internet
- If you can’t find it, you can easily build it; there are lots of great examples and resources out there to help
<insert activity here>
![Page 33: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/33.jpg)
from zero to database server in just a few minutes
![Page 34: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/34.jpg)
- Here we will take a quick tour through how to use Chef to install PostgreSQL with just a few commands
- This will use the default configuration provided by the cookbook - A good cookbook provides sane default values for this reason - You can always override the attributes by defining them in the
Chef server
Let’s see how this works
![Page 35: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/35.jpg)
- Chef’s knife command provides the equivalent of a command-line swiss-army knife
- Allows you to interact with the Chef server to manage configuration data, upload cookbooks, as well as manage remote systems
a chef’s tool
![Page 36: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/36.jpg)
1. Creating a new host with a cloud provider 2. Installing the Chef client onto the host and registering it 3. Downloading cookbooks from GitHub 4. Uploading our cookbooks to the Chef server 5. Configuring a PostgreSQL server role 6. Assigning that role to our new host 7. Running chef-client on the host in order to converge it 8. There is no step 8!
what we’ll be doing
![Page 37: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/37.jpg)
Creating a new droplet with Digital Ocean; this could be any cloud service, or you can use your own physical systems
![Page 38: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/38.jpg)
Now that’s it’s created, we have a host!
![Page 39: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/39.jpg)
Using knife, we can bootstrap our new node.
![Page 40: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/40.jpg)
Bootstrapping will:
1. Install the Chef client 2. Register itself with the Chef server 3. Make an initial run of the Chef client
![Page 41: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/41.jpg)
Let’s make sure that the Chef server knows about our new node
![Page 42: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/42.jpg)
Now that we have a host, we need some cookbooks!
![Page 43: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/43.jpg)
Cookbooks are uploaded to the server for us to use
![Page 44: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/44.jpg)
Once again, we can verify that the server know about our newly created objects.
![Page 45: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/45.jpg)
knife role edit postgresql_server
![Page 46: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/46.jpg)
Let’s take a quick look at our handiwork
![Page 47: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/47.jpg)
Once we have a role, we need to assign it to our node.
![Page 48: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/48.jpg)
Here’s what our node looks like now
![Page 49: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/49.jpg)
knife can do lots of things, one of those things is remote execution of commands.
![Page 50: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/50.jpg)
This also means we can execute the Chef client remotely using knife!
![Page 51: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/51.jpg)
The Chef client running on our node
![Page 52: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/52.jpg)
When it’s complete, Chef tells you how many things were changed and how long it took to finish the run
![Page 53: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/53.jpg)
Look, PostgreSQL is now running!
![Page 54: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/54.jpg)
- A key tenet of Chef is that configuration should be repeatable and consistent
- Running the client multiple times in a row, with no configuration changes will result in the system being untouched (idempotence)
- Resource providers determine if there is any work to be done and if there is no work to do, then the resource is left untouched
repeatability
![Page 55: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/55.jpg)
configuring software
![Page 56: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/56.jpg)
- Installing software is not that hard - Package management tools have made that a lot easier - Most folks can apt-get install over SSH (or equivalent) - Configuring software is much harder - Chef can help you keep your configuration consistent
software configuration
![Page 57: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/57.jpg)
- Easy enough to apt-get install postgresql-server - Consider having 100 database nodes - Harder to keep them consistently configured - Chef knows all about your database servers - Use that info to build a database connection pool server - Let’s look at how we might configure pgpool using Chef
example: webapp with pgpool
![Page 58: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/58.jpg)
# configure the backends backend_hostname0 = 'db01' backend_port0 = 5432 backend_weight0 = 1 backend_hostname1 = 'db02' backend_port1 = 5432 backend_weight1 = 1 backend_hostname2 = 'db03' backend_port2 = 5432 backend_weight2 = 1
Configuring pgpool - by hand
![Page 59: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/59.jpg)
db_servers = search(:node, “role:postgresql_server") !template "/etc/pgpool.conf" do variables(:db_servers => db_servers) source "pgpool2.conf.erb" owner "root" group "root" mode "644" end
Configuring pgpool
The configuration portion of a pgpool recipe
![Page 60: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/60.jpg)
!# configure the backends <% @db_servers.each_with_index do |server, index| -%> backend_hostname<%= index %> = ‘<%= server[‘fqdn’] %>’ backend_port<%= index %> = 5432 backend_weight<%= index %> = 1 <% end -%>
Configuring pgpool - chef
A portion of a pgpool.conf template
![Page 61: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/61.jpg)
- This example is very simple but can be expanded - For example you could use some information about the hosts to:
- Dynamically change the DB weight - Change how many child processes to run - Alter the max connections
- Configuration templates should be simple, like the views in MVC
Advanced configuration
![Page 62: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/62.jpg)
- With this you could create a role, pg_pool_server - By applying that to a node, you now have a pgpool instance - New nodes with the postgresql_server role are added dynamically - We have enough information to automatically configure an app
using your pool
![Page 63: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/63.jpg)
pg_pools = search(:node, “role:pg_pool_server”) !template "#{src_dir}/config.py" do source "config.py.erb" variables({ :dbname => node[:webapp][:dbname], :dbuser => node[:webapp][:dbuser], :dbpass => node[:webapp][:dbpass], # first pgpool host as endpoint (for simplicity) :dbhost => pg_pools[0][‘fqdn’] }) end
Configuring your app with chef
A small portion of a deployment recipe
![Page 64: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/64.jpg)
import web db_params = { 'dbn': 'postgres', 'db': '<%= @dbname %>', 'user': '<%= @dbuser %>', 'pw': '<%= @dbpass %>', 'host' : '<%= @dbhost %>' } DB = web.database(**db_params) cache = False
Configuring your app with chef
A simple web.py configuration file template
![Page 65: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/65.jpg)
Consistent configuration- Chef can do more than simply install software - You can also configure software using the data stored in Chef - Configuring software this way guarantees that you have one
source of truth that you can rely on - When your model gets updated with new data so do your
configurations
![Page 66: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/66.jpg)
provisioning local environments
![Page 67: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/67.jpg)
local environments- Chef server provides the functionality required to configure fleets - The core engine of Chef can be used for small-scale configuration - Good for single hosts, application deployments, and more - An ideal tool for development environments to reduce friction !
![Page 68: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/68.jpg)
what is chef solo?- Chef solo is designed for small-scale management - Chef solo does not use a central server - It is not designed for large scale infrastructure management - Typical use cases include:
- developers managing virtual machines - test installations - small-scale infrastructure management - local application configuration
![Page 69: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/69.jpg)
how do I use it?- Installation of Chef solo requires installing a single Ruby gem - Chef-solo is installed as a part of the chef gem - Cookbooks, recipes, and configuration data is stored locally - Can be used with self-provisioned virtual machines - Combine with Vagrant to automate provisioning of virtual hosts
![Page 70: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/70.jpg)
limitations- The following features are not supported with Chef solo:
- Node data storage - Search indexes - Centralized distribution of cookbooks - A centralized API - Authentication or authorization - Persistent attributes
![Page 71: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/71.jpg)
using chef-solo- Using chef-solo is as simple as:
- Creating some recipes - Generating a run list - Writing a solo configuration file - Executing chef-solo
![Page 72: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/72.jpg)
We create a simple recipe
![Page 73: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/73.jpg)
Our recipe creates a file with some contents in $HOME/example.txt
![Page 74: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/74.jpg)
Manually create our run list to contain our recipe
![Page 75: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/75.jpg)
Configure chef-solo
![Page 76: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/76.jpg)
Run chef-solo
![Page 77: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/77.jpg)
Check out our handiwork!
![Page 78: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/78.jpg)
Prove that Chef runs are idempotent
![Page 79: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/79.jpg)
Update our recipe
![Page 80: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/80.jpg)
Do it again!
![Page 81: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/81.jpg)
managing single hosts- As you can see, you can use the same techniques to manage a
single system - A large number of existing cookbooks will work “as-is” - Some recipes will require updates if they rely on server-specific
mechanisms !!!
![Page 82: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/82.jpg)
configuring software- Recipes and configuration files can be delivered with your app - Chef solo can be used to configure the software - Chef’s omnibus installer uses Chef solo to configure itself - Repeatability of configuration means a more consistent experience - Provides a way of programmatically testing your installation - A good experience = happy customers - Happier customers make everyone happy!
![Page 83: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/83.jpg)
chef and vagrant- Vagrant and Chef-solo is a very powerful combination - Distribute cookbooks and a Vagrantfile to your users - Developers can be up and running in no time - No need for developers to install services or tools manually - Repeatable, version controlled and easily distributed
![Page 84: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/84.jpg)
chef and vagrantVagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "chef_solo" do |chef| chef.cookbooks_path = "cookbooks" chef.roles_path = "roles" chef.data_bags_path = "data_bags" chef.add_recipe "postgresql::server" end end
Here we tell Vagrant to use chef solo and that we want to use the server recipe from the postgresql cookbook.
![Page 85: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/85.jpg)
chef and vagrant… chef.add_recipe "apt"
chef.add_recipe "nodejs"
chef.add_recipe "ruby_build"
chef.add_recipe "rbenv::user"
chef.add_recipe "rbenv::vagrant"
chef.add_recipe "vim"
chef.add_recipe "mysql::server"
chef.add_recipe "mysql::client" ! chef.json = {
rbenv: {
user_installs: [{
user: 'vagrant',
rubies: ["2.1.2"],
global: "2.1.2",
gems: {
"2.1.2" => [
{ name: "bundler" }
]
}
}]
},
mysql: {
server_root_password: ''
}
} !
- Provide JSON configuration data to Chef - Forward ports with Vagrant - Use recipes to:
- Install tools for development - Setup database servers - Configure passwords - Install gems - Anything you can think of!
![Page 86: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/86.jpg)
things you might do…- Provide an easy-to-use single-command way of building a
developer’s environment (very useful for getting new developers bootstrapped working with services)
- Configure your shipped software through version controlled JSON files (OmniTI’s Circonus does this)
- Manage your local VMs with Vagrant and Chef solo to provide various services or applications
![Page 87: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/87.jpg)
mad science!
![Page 88: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/88.jpg)
advanced uses for chef- Integrate existing tools with Chef - Test your recipes and infrastructure - Fully automate your deployments - Scale your application capacity with a single command - Provision hundreds of new systems in parallel
![Page 89: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/89.jpg)
integrate with Chef- Chef makes it easy to interact with your configuration data - Data is exposed through an HTTP API - By using the API you can extend your existing tools - For ideas on how you might do that, take a look at these:
- Ruby gem for talking to Chef’s API - https://github.com/sethvargo/chef-api
- Capistrano plugin for working with Chef - https://github.com/gofullstack/capistrano-chef
![Page 90: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/90.jpg)
test your infrastructure- Because your configuration is code, you can test it - Testing can range from simple unit tests on recipes to full
integration test suites - Chef gives you the ability to test across multiple platforms both at
the recipe level inside unit tests as well as integration tests - Test Kitchen (https://github.com/test-kitchen/test-kitchen) is
designed for integration testing - ChefSpec (https://github.com/sethvargo/chefspec) is useful for
unit testing
![Page 91: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/91.jpg)
test kitchen---
driver:
name: vagrant
!provisioner:
name: chef_solo
!platforms:
- name: ubuntu-12.04
- name: centos-6.4
!suites:
- name: default
run_list:
- recipe[testfile::default]
attributes:
Test on multiple platforms using Vagrant
![Page 92: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/92.jpg)
Chefspecrequire 'chefspec' !describe 'mycookbook::default' do let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) } ! it 'creates a file' do expect(chef_run).to create_file('/tmp/myfile.txt') end end
ChefSpec makes it easy to verify your recipes’ functionality.
![Page 93: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/93.jpg)
Chefspec let(:chef_run) do runner = ChefSpec::ChefRunner.new( 'platform' => 'windows', 'version' => '2012' ) runner.converge('mysql::server') end ! it 'should include the correct Windows server recipe' do chef_run.should include_recipe 'mysql::server_windows' end
Even unit test your recipe’s execution on different platforms
![Page 94: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/94.jpg)
fully automated deployments- You can use Chef to completely automate deployments - Model two environments, each configured for a different branch - Write a recipe to deploy your application - Run chef-client on a schedule using cron or another tool - Push code to the test environment branch to deploy to testing - When you are ready, push changes to the production branch - With some automated tests and elbow grease you can automate
that step too!
![Page 95: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/95.jpg)
scaling your infrastructure- Sometimes your needs outpace your hardware growth - Chef can help you adapt quickly and with ease - Using Chef and cloud services you can expand your capacity
Provision cloud server with knife
Deploy application with Chef
Register host with AWS ELB
More capacity!
![Page 96: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/96.jpg)
scaling your infrastructureaws_elastic_lb “#{node.myapp.website.load_balancer}" do aws_access_key "#{node.myapp.aws.access_key}" aws_secret_access_key "#{node.myapp.aws.secret_key}" action :register end
With the ELB resource, Chef can add your newly configured host to an AWS ELB instance.
![Page 97: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/97.jpg)
scaling your infrastructureknife ec2 server create -d ubuntu13.10 -I ami-c6402cf6 \ -f m3.large -Z us-west-2a -S ssh_key -N hostname \ -r “role['base_server'],role['webapp']" \ -g 'sg-e20fddcb,sg-359223ab'
With Chef, provisioning a new EC2 instance, deploying your app and registering it with the load balancer is as easy as one command.
![Page 98: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/98.jpg)
auto-scaling- With a combination of cloud providers, metrics and Chef you
could build a system that automatically grows or shrinks your infrastructure as needed
Metrics Chef Cloud provider
Scaling Engine
Feeds data into scaling engine
Determines a needfor more capacity
Provisions a newnode with knife
Brings up new node,bootstraps, registers
with load balancer, etc.
![Page 99: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/99.jpg)
chef-metal- chef-metal is designed to allow you to provision systems from
a set of configuration files - Use your configuration to provision hundreds of Vagrant
instances or EC2 instances - chef-metal is under active development num_webservers = 100
!1.upto(num_webservers) do |i| machine "web#{i}" do recipe 'apache' recipe 'webapp' end end
![Page 100: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/100.jpg)
resources
![Page 101: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/101.jpg)
- Learn Chef - http://learn.getchef.com/ - A great combination of getting started material for new Chef users
- Chef Supermarket - http://supermarket.getchef.com - Contains cookbooks, tools and extensions to Chef. Community
maintained - Mailing list
- http://lists.opscode.com/sympa/info/chef - An active mailing list for users of Chef to communicate
resources
![Page 102: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/102.jpg)
- Written by me - Instant Chef Starter - Managing Windows Servers with Chef - Chef Essentials (coming soon)
- Written by others - Learning Chef (coming soon) - Test-Driven Infrastructure with Chef, 2nd Edition
books
![Page 103: Automating your infrastructure with Chef](https://reader034.fdocuments.in/reader034/viewer/2022042614/558ec5a81a28ab80628b4651/html5/thumbnails/103.jpg)
I hope that this talk has…
- helped you see how chef’s pieces fit together
- shown you some of the things chef can do
- given you the tools to explore chef on your own
- gotten you thinking about advanced uses
- made you want to learn more about chef!