Luke Kaniesluke@reductivelabs.comFounder, Reductive Labs

Nashville, TennesseeUSA

Using Puppet

Why it exists, how it works, and why it works this way

Introductions

Image from http://www.flickr.com/photos/tafa/27605946/sizes/o/

* How many of you would consider yourselves sysadmins?* How many of you manage sysadmins?* Frightening: How many of you are managed by sysadmins?* How many of you have heard of Puppet?* How many of you use Puppet?

Puppet is a system for centrally specifying and

managing your infrastructure

Written in Ruby

Three major pieces

• A declarative, resource-oriented language

Three major pieces

• A declarative, resource-oriented language

• An execution engine

Three major pieces

• A declarative, resource-oriented language

• An execution engine

• A bunch of daemons and tools

Three major pieces

1) A resource-oriented language

Image from http://www.flickr.com/photos/heather/147388906/sizes/l/

Your infrastructure is code

Image from http://www.flickr.com/photos/echo_29/25829591/sizes/o/

And code is great because we have tooling like version control, and decades of experience with it

Resource Abstraction Layer (RAL)

Do you really care how rpm works?

rpm/apt/etc becomes packages/etc/hosts becomes hostsuseradd/netinfo, etc. becomes users/etc/init.d/smf/launchd/daemontools becomes services

Resource Providers

23 package types

Users in NetInfo, useradd, pw

Support for Debian, Ubuntu, Red Hat, Solaris, OS X, Gentoo, SuSE, FreeBSD, and more

Windows is being actively worked on

2) An engine for applying

that specification repeatably

Image from http://www.flickr.com/photos/jurvetson/480227362/sizes/l/

2) An engine for applying

that specification repeatably

Image from http://www.flickr.com/photos/jurvetson/480227362/sizes/l/

Specifications are idempotent

Idempotency is what allows us to manage a machine through its whole lifecycle

Specifications are idempotent

Idempotency is what allows us to manage a machine through its whole lifecycle

3) Daemons and executables that

determine Puppet’s architecture

client/server, cert mgmt, etc.

Centralized Management

puppetd in the cloud

Puppetmasterd

puppetdOS X puppetd

Linux

Code

Uses SSL, and provides a Certificate Authority

Image from http://www.flickr.com/photos/piet_musterd/2307596484/sizes/l/

* Every connection is encrypted, and the only connection that isn’t authenticated is the one that asks for a signed cert* Client certs* Autosign, manual sign, manual certificate generation* You don’t even have to use it

ralsh - a thin API wrapper

This uses the same model as the rest of puppet -- it chooses the appropriate provider for the local system. You can edit resources, and it even works over the network.

Who’s using Puppet?

“...at Google we're currently using Puppet to

manage close to 6,000 Macs, and it's likely our deployment will expand

dramatically beyond that....”

Testimonials

. . .at Google we're currently using it to manage close to 6,000 Macs, and it's likely our deployment will expand dramatically beyond that. . . Nigel KerstenMacOps

Thank You Puppet!! . . .We had a guy on staff who was doing only these things almost full time, so puppet's allowed him to do bigger and better things (like take Christmas break off :) Eugene Ventimiglia Director of Systems GridApp Systemsd

User Community Who Volunteer Info(http://reductivelabs.com/trac/puppet/wiki/WhosUsingPuppet)

The Ohio State University Department of Mathematics- Primarily used for: Mac OS X workstation and GNU/Linux server management.

Wordtracker- We introduced Puppet gradually by managing a handful of office workstations, and are now developing recipes for building servers with it

Imperial College London HEP group- We use puppet to manage our HPC clusters, our desktops and some of our servers

Bashton Ltd- Country: UK- We use Puppet to manage our own cluster of servers, as well as creating and supporting configurations for our customers

Fab:IT ApS- We use puppet to administrate our own servers placed in the local datacenter, and our customers placed both in the same datacenter, and scattered around the country. Puppet is ideal

because of it's ease of use, and especially because the puppet server compiles the configuration specifically for the individual host, so that no irrelevant information goes to the wrong nodes. Thanks, Luke, for a great tool!

- Country: Denmark

Geni Inc. (www.geni.com)- We use puppet to manage the configurations of all our Linux servers. With puppet my server build time is roughly 12-15 minutes to get a server ready for production.

Institute of Cognitive Science at the University of Osnabrueck, Germany- After having some headaches with cfengine, we started managing our linux workstations with puppet.- Shortly after Linux, we also started managing our Mac OS X workstations.

Why does Puppet exist?

Somebody has to do something, and it's just incredibly pathetic that it has to be us.

-- Jerry Garcia

Community

• 1000+ people on the mailing lists

• 200+ people in IRC all day

• More than 50 contributors

Programming SysAdmin

Low-level, non-portable

Assembly commands and files

Abstract, portable C* Resources

An Analogy

* For small values of abstract

* The assembly programmers fought the adoption of C* Fear for your career if you’re a bit too fond of assembly* It’s not about fewer people, it’s about higher quality and productivity* Are there more or fewer programmers today than in the days of assembly?

Image from http://flickr.com/photos/kenskritters/2128853769/

Speciation

These are house finches, reminding one of the finches Darwin observed in the Galapagos. I want to cause sysadmin speciation.

Sysadmins do too much

Image from http://flickr.com/photos/shirleytwofeathers/2068713495/

Firefighter? Architect? Developer? Tape-changer? All of the above?

Let’s talk about you

Do you have any computers?

Some questions

Some questions

• How many of you have written software to manage computers?

Some questions

• How many of you have written software to manage computers?

• How many have published this software?

Developer Sysadmin

Developer Sysadmin

Three people

Puppet allows you to bridge that gap

Image from http://www.flickr.com/photos/patrick-smith-photography/2969769911/sizes/o/

Either you can manage many machines with little effort

Either you can manage many machines with little effort

Or you can’t

How to use Puppet

The strategical version

Seek the pain

Image from http://www.flickr.com/photos/pagedooley/2147718252/sizes/l/

Solve the simple problems

Add the infrastructure features you always

wanted

How to use Puppet

The tactical version

Think like Puppet thinks

Think like Puppet thinks

• Resources, not text snippets or lines added to files

Think like Puppet thinks

• Resources, not text snippets or lines added to files

• What resources are you managing?

Think like Puppet thinks

• Resources, not text snippets or lines added to files

• What resources are you managing?

• How are they related to each other?

Replace Shell Scripts with ResourcesThis:

Replace Shell Scripts with Resources

Becomes:

This:

Replace Shell Scripts with Resources

Becomes:

This:

Replace Shell Scripts with Resources

Becomes:

This:

Replace Shell Scripts with Resources

Becomes:

This:

Relationships matter but are often implicit

Relationships matter but are often implicit

Package

Relationships matter but are often implicit

Package

Configuration

Configuration should get modifed after package

installation

Relationships matter but are often implicit

Package

Service

Service should restart whenconfiguration changesConfiguration

Configuration should get modifed after package

installation

Explicit Relationships

We’ll come back to abstraction

Relationships provide ordering and notification

"Exec[createrepo-PM-RHEL5-noarch]"

"Yumrepo[PM-RHEL5-x86_64]"

"Yumrepo[PM-RHEL5-noarch]"

"Package[postgresql-server]"

"Package[thttpd]"

"File[/var/www/thttpd/html/yum-PM-RHEL5-noarch]""File[/var/www/thttpd/html/yum-PM-RHEL5-x86_64]"

"Exec[rsync-rpmdir-PM-RHEL5-x86_64]"

"Exec[createrepo-PM-RHEL5-x86_64]"

"Postgres::Role[puppet]"

"Exec[rsync-rpmdir-PM-RHEL5-noarch]"

Classes document Intent

This is shareable, releasable code.Classes are analogous with tags

Organize files into modules

Provide platform abstraction

Debian

We’re doing the same thing with different commands on different platforms

Provide platform abstraction

Debian

Red Hat

We’re doing the same thing with different commands on different platforms

Provide platform abstraction

Debian

Red Hat

We’re doing the same thing with different commands on different platforms

Portability and Naming

Class Membership

Class Membership

Puppet in the cloud

Heavily used in EC2, VMWare, Xen, etc

Puppet allows you to avoid the foil ball

Image from http://www.flickr.com/photos/fungep/2516767121/sizes/l_

The Gory Details

Status and Roadmap

• In production usage for years

• About to release a new version with significant internal refactoring

• Adding simple graphical tools this year

Puppet and Capistrano

or ControlTier

Puppet vs. Cfengine

It scales like HTTPS

because it is https

And you don’t even need to centralize it.

Logs go to syslog (by default)

The people behind Puppet

This is all I do

Bad product,hungry Luke

Employs three people and still growing

Support, Paid features, Consulting, Training

Questions?