Code but now as we know it - Devoxx edition
111
Infrastructure as Code Patrick Debois http://www.tumblr.com/tagged/star-trek-quotes Thursday 15 November 12
-
Upload
patrick-debois -
Category
Technology
-
view
3.120 -
download
1
description
Comparing config management systems with 'regular' code. Contains a lot of tools and hightlights known and some more obscure existing tools in the space.
Transcript of Code but now as we know it - Devoxx edition
Infrastructure as Code Patrick Deboishttp://www.tumblr.com/tagged/star-trek-quotes
Thursday 15 November 12
Blog: http://jedi.be/blog - Twitter : @patrickdebois - #devops
Technical ReviewerMonitoring Chapter Co-Author Veewee / Sahara / Mccloud
Organizer Since 2009 Europe Organizer 2010
Engineer
First Europe Training 2010
Libvirt - Fog
Speaker
Freelance consultant http://github.com/jedi4ever
Thursday 15 November 12
CONFIGURATION MANAGEMENT SYSTEMS
http://ansible.cc/
http://opscode.com/chef/
http://cfengine.com/http://palletops.com/http://puppetlabs.com/
Thursday 15 November 12
http://www.nerdnirvana.org/wp-content/uploads/2010/12/apology-form.jpg
Chef & Puppet focusedbased on my experience
Thursday 15 November 12
SERVICE
FILE2
FILE1PACKAGE
DECLARATIVE LANGUAGE
Thursday 15 November 12
“KNOWN STATE”
STATE X STATE YCODE RUN
FILE1SERVICE
FILE2
FILE1PACKAGE
convergence
Thursday 15 November 12
“IDEMPOTENCE”
STATE Y STATE YCODE RUN
SERVICE
FILE2
FILE1PACKAGE
SERVICE
FILE2
FILE1PACKAGE
Thursday 15 November 12
GIVE ME THE CODE:“LANGUAGE”
Thursday 15 November 12
Puppet Chef
class myapache {
package “apache2”
service “apache2”: ensure => “running”, require => Package[“apache2”]
}
package “apache2”
service “apache2 do action [:enable , :start]end
EXAMPLE SYNTAX
Thursday 15 November 12
TERMINOLOGY
Java Puppet Chef
*.javamanifests
*.pprecipes
*.rb
package module cookbook
Thursday 15 November 12
TERMINOLOGY (2)
Java Puppet Chef
singletonparametrized
classesrecipes with attributes
classdefines, classes
definitions, libraries
Thursday 15 November 12
SYNTAXPuppet Chef
Limited DSL(by design) DSL + full ruby
Extended by Definitions
Light Weight Resource Providers /
DefinitionsCustom Resource providers
Providers, Light Weight Resource
ProvidersERB & Custom Libraries Libraries
Thursday 15 November 12
LOOPING CONSTRUCTS
Puppet Chef
pass array of elementserb template for loops
create_resource (:type, hash)
do while , repeat etc. available through
native ruby
Thursday 15 November 12
VARIABLES
Puppet Chef
special syntax$bla = “aa”
string interpolation$bla_string = “${bla}”
native rubybla = “aa”
string interpolationbla_string = #{bla}
Thursday 15 November 12
SINGLE CLASSINHERITANCE
Puppet Chef
override via ‘module path’
override via ‘cookbook’ path
+> operator “cookbook” attribute
Thursday 15 November 12
Puppet Module Chef Cookbook
files/lib/
README.rdocmanifests/templates/
attributes/definitions/files/libraries/metadata.rbproviders/README.rdocrecipes/resources/templates/
PROJECT STRUCTURE
Thursday 15 November 12
EXECUTION MODEL
Puppet Chef
Compile catalog(directed graph
model)Then Execute
Top Down execution
Thursday 15 November 12
PLUGINSPuppet Chef
FACES KNIFE PLUGINS
REPORTS REPORT HANDLERS
http://puppetlabs.com/blog/puppet-faces-what-the-heck-are-faces/http://wiki.opscode.com/display/chef/Knife+Plugins
Thursday 15 November 12
CODE EDITOR
Thursday 15 November 12
https://github.com/rodjek/puppet-pygments-lexerhttp://redmine.djagios.org/projects/puppet-lexer
LEXERS
Thursday 15 November 12
VIM SYNTAX HIGHLIGHTIING
https://github.com/rodjek/vim-puppethttps://github.com/t9md/vim-chef
https://github.com/vim-scripts/Cfengine-version-3-syntax
Thursday 15 November 12
http://cloudsmith.github.com/geppetto/https://github.com/cloudsmith/geppetto
GEPPETTO - ECLIPSE IDE
Thursday 15 November 12
JETBRAINS RUBYMINE (PUPPET)http://youtrack.jetbrains.com/issue/RUBY-10832
REACTOR8http://beta.r8network.com/
COMING SOON
https://github.com/janschumann/intellij-lang-puppet
Thursday 15 November 12
REVERSE ENGINEERING
https://github.com/devstructure/blueprint
Existing Server -> Manifest/Recipes
Thursday 15 November 12
CODING STYLE
Thursday 15 November 12
Puppet Chef
http://puppet-lint.com/ http://acrmp.github.com/foodcritic/
• Must use two-space soft tabs.• Must not use literal tab characters.• Must not contain trailing white space.• Should not exceed an 80 character line width
An exception has been made for source => 'puppet://...' lines as splitting these over multiple lines decreases the readability of the manifests.
• Should align arrows (=>) within blocks of attributes.• ....
•
• FC001: Use strings in preference to symbols to access node attributes• FC002: Avoid string interpolation where not required• FC003: Check whether you are running with chef server before using server-
specific features• FC004: Use a service resource to start and stop services• FC005: Avoid repetition of resource declarations• FC006: Mode should be quoted or fully specified when setting file
permissions• FC007: Ensure recipe dependencies are reflected in cookbook metadata• FC008: Generated cookbook metadata needs updating• FC009: Resource attribute not recognised• FC010: Invalid search syntax• FC011: Missing README in markdown format• FC012: Use Markdown for README rather than RDoc• FC013: Use file_cache_path rather than hard-coding tmp paths•
http://docs.puppetlabs.com/guides/style_guide.htmlhttps://github.com/ampledata/cookbook-style-guide
Thursday 15 November 12
http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-puppet-camp
Thursday 15 November 12
http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-puppet-camp
http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-puppet-camp
CODE VS CONFIG(THINK “PROFILES”)
http://wiki.opscode.com/display/chef/Data+Bags
http://projects.puppetlabs.com/projects/hierahttps://github.com/crayfishx/hiera-gpg
Thursday 15 November 12
https://github.com/jedi4ever/stop-the-forkhttp://realityforge.org/code/2012/11/12/reusable-cookbooks-revisited.htmlhttp://devopsanywhere.blogspot.it/2012/11/how-to-write-reusable-chef-cookbooks.html
“STOP THE FORK(ING)”
Thursday 15 November 12
PRE-COMMIT CHECKS
Thursday 15 November 12
DEPENDENCY MGMT
GitSubmodules
Subtree à la Pom File
Thursday 15 November 12
UPSTREAM REPOS
http://forge.puppetlabs.com/http://community.opscode.com/
Thursday 15 November 12
Puppet Chef
https://github.com/rodjek/librarian-puppet
https://github.com/applicationsonline/librarian
http://berkshelf.com/
site "http://community.opscode.com/api/v1"
cookbook "ntp"cookbook "timezone", "0.0.1"
cookbook "rvm", :git => "https://github.com/fnichol/chef-‐rvm", :ref => "v0.7.1"
cookbook "cloudera", :path => "vendor/cookbooks/cloudera-‐cookbook"
Thursday 15 November 12
CHEF METADATA
Thursday 15 November 12
http://bitfieldconsulting.com/puppet-dependency-graphs
Dependency Graphs
https://github.com/miketheman/knife-role-spaghettiThursday 15 November 12
http://wham.cloudsmith.com
Online Service
Thursday 15 November 12
DEBUGGING
Thursday 15 November 12
LOGGING WITH DIFFERENT LEVELS
Info only, no exceptionChef::Log.info('Some useful info')
Fatal, raising exceptionChef::Log.fatal('Something bad')
Thursday 15 November 12
REPL - STYLE(READ EVAL PRINT)
http://wiki.opscode.com/display/chef/Shefhttps://github.com/pry/pry
Thursday 15 November 12
https://github.com/vimeo/simple-black-boxThursday 15 November 12
DEV ENVIRONMENT
Thursday 15 November 12
SERVICE
FILE2
FILE1PACKAGE
Virtual Machine (1)
Just Enough Operating System+ Config Management Tool
Virtual Machine ... N
Your laptop (Linux, Wind
virtualization
Thursday 15 November 12
SERVICE
FILE2
FILE1PACKAGE
Virtual Machine (1)
Just Enough Operating System+ Config Management Tool
Your Laptopvirtualization
http://vagrantup.com/
Virtualbox + (Fusion Soon)
Linux & WindowsThursday 15 November 12
https://github.com/BIAINC/vagrant-windows
https://github.com/dotless-de/vagrant-vbguesthttps://github.com/folken-laeneck/vagrant-bindfs
https://github.com/BerlinVagrant/vagrant-dnshttps://github.com/dwt/vagrant-hosts
USEFULVAGRANT PLUGINS
Thursday 15 November 12
http://marketplace.eclipse.org/node/519961#.UKISB-Oe-d4
VAGRANT ECLIPSE INTEGRATION
Thursday 15 November 12
https://github.com/rtyler/blimpy
https://github.com/jedi4ever/mccloud/
Vagrant ‘clones’
EC2, KVM, FUSION,HOSTS, Vagrant
EC2
https://github.com/neerolyte/vagueanthttps://github.com/lstoll/tenderloin FUSION
LXC
Thursday 15 November 12
VAGRANT - CLOJURE
https://github.com/tbatchelli/vmfestThursday 15 November 12
VAGRANT - JAVA
https://github.com/guigarage/vagrant-bindingThursday 15 November 12
JEOS CREATION
Just Enough Operating System+ Config Management Tool
Your Laptopvirtualization
Virtualbox , FusionKVM, Parallels
Linux & Windows
https://github.com/jedi4ever/veewee/
Centos, Ubuntu, Debian, Solaris, Gentoo, Suse, Oracle Linux, ....
And windows now too!
Thursday 15 November 12
https://github.com/opscode/test-kitchenhttps://github.com/opscode/bento
uses #veewee
CREATING BASEBOX ‘EN MASS’
https://github.com/atkm/seisan-linehttps://github.com/atkm/origami
Thursday 15 November 12
http://www.vagrantbox.es/
http://www.morethanseven.net/2011/05/08/Vagrant-plugin-for-interacting-with-vagrantboxes/
http://vagrantfil.es/
http://www.nodescription.net/ideation-into-the-void-vagrant-plugin-for-vag
SHARE YOUR SETUP
Thursday 15 November 12
UNIT TESTING
Thursday 15 November 12
https://github.com/nistude/cucumber-puppetThursday 15 November 12
https://github.com/rodjek/rspec-puppet
https://github.com/calavera/rspec-chefhttps://github.com/acrmp/chefspec
Thursday 15 November 12
DON’T TEST YOUR CONFIG MANAGEMENT
TEST YOUR LOGIC
NOT
Was the package installed
YES
If (OS=’X’ ) and (file exists) ...
did it give an error?
Thursday 15 November 12
FAKE THE FACTSThursday 15 November 12
https://github.com/customink/fauxhaiThursday 15 November 12
AUTO-TRIGGER TESTS RUN
https://github.com/guard/guard-chefhttps://github.com/alister/guard-puppet-linthttps://github.com/johnbintz/guard-puppet
https://github.com/guard/guard-rspec
CODE EDIT
AUTO RUNFEEDBACK
Thursday 15 November 12
LOCAL VM TESTING
http://www.cucumber-chef.org/
https://github.com/exceedhl/toft
Faster testing via LXC
spinning up a vm is expensiveunless in container
EC2+LXC
Vagrant+LXCThursday 15 November 12
CI INTEGRATIONThursday 15 November 12
https://github.com/nicksieger/ci_reporter
Junit processor for Ruby
Thursday 15 November 12
SYNTAX CHECKSCOMPILE CHECKS
UNIT TESTS
SMOKE TESTS
NO VM NEEDED
REAL VM
Thursday 15 November 12
SMOKE TESTING(ON REAL MACHINE)
Code Applied
CLEAN VM
STATE A STATE BCode Run Run TestsCreate VM
SNAPSHOT
Thursday 15 November 12
http://auxesis.github.com/cucumber-nagios/Thursday 15 November 12
RE-USABLE CUCUMBER STEPS
https://github.com/hedgehog/cukenhttps://github.com/cucumber/aruba
http://www.cucumber-chef.org/
Thursday 15 November 12
https://github.com/calavera/minitest-chef-handlerhttps://github.com/jedi4ever/puppet-assert
Validate “inside” the provision-run cycle
Thursday 15 November 12
VM CREATION IS EXPENSIVE
https://github.com/jedi4ever/saharahttps://github.com/t9md/vagrant-snap
Code Applied
CLEAN VM
STATE A STATE BCode Run Run TestsCreate VM
SNAPSHOT
ROLLBACK
Thursday 15 November 12
https://github.com/rtyler/vagrant-pluginThursday 15 November 12
https://travis-ci.org/
Uses vagrant & veewee
Thursday 15 November 12
https://github.com/tknerr/bills-kitchen
Windows
Thursday 15 November 12
TARGETED TESTINGIMPACT OF CHANGE
https://github.com/jedi4ever/puppet-cichttps://github.com/jonlives/knife-preflight
ROLE1
ROLE2
ROLEN
MODULEA
MODULEB
MODULEC
MODULE...
Real VM testingis expensive
Only test what’s needed
Thursday 15 November 12
PERFORMANCE/METRICS
Thursday 15 November 12
https://github.com/rodjek/puppet-profilerThursday 15 November 12
https://github.com/joemiller/puppet-graphite_eventhttps://github.com/imeyer/chef-handler-graphite
https://github.com/krux/puppet-module-graphite-report
execution timenumber of classes
Thursday 15 November 12
Infrastructure as Code Patrick Deboishttp://www.tumblr.com/tagged/star-trek-quotes
Thursday 15 November 12
Thursday 15 November 12
THANKYOU!
Thursday 15 November 12
QUESTIONS?
Thursday 15 November 12
http://foodfightshow.org/
Thursday 15 November 12
Thursday 15 November 12
Thursday 15 November 12
http://wiki.opscode.com/display/chef/Whyrun+TestingThursday 15 November 12
https://github.com/lak/puppet-memcachedThursday 15 November 12
https://github.com/lak/puppet-static-compilerThursday 15 November 12
http://puppetlabs.com/blog/what-is-user-experience-in-puppet/
Thursday 15 November 12
AUDITING
http://www.normation.com/en/solutions/rudderhttp://www.fusioninventory.org/
Thursday 15 November 12
http://comodit.github.com/synapse-agent/Thursday 15 November 12
https://github.com/tobami/littlechefThursday 15 November 12
https://github.com/dwt/vagrant-hostsThursday 15 November 12
https://github.com/opscode/omnibusThursday 15 November 12
https://github.com/puppetlabs/puppet-module-toolThursday 15 November 12
http://wiki.opscode.com/display/chef/Recipes#Recipes-ExceptionsandLogging
Thursday 15 November 12
http://projects.puppetlabs.com/projects/puppet/wiki/Anchor_Pattern
Thursday 15 November 12
VENDORING
Thursday 15 November 12
https://github.com/railsmachine/rumpThursday 15 November 12
http://www.example42.com/?q=Puppi_A_Puppet_module_for_Deployment_AutomationThursday 15 November 12
http://www.sonian.com/cloud-monitoring-sensu/Thursday 15 November 12
http://docs.puppetlabs.com/guides/parameterized_classes.htmlThursday 15 November 12
https://github.com/rerun/rerunThursday 15 November 12
AUTHENTICATION
Thursday 15 November 12
DASHBOARDS
Thursday 15 November 12
http://puppetlabs.com/puppet/related-projects/dashboard/Thursday 15 November 12
http://blog.mornati.net/2012/01/29/kermit-a-webui-for-mcollective/
Thursday 15 November 12
http://www.opscode.com/hosted-chef/Thursday 15 November 12
https://github.com/chapmanb/cloudbiolinuxThursday 15 November 12
https://github.com/aetherical/nimblestratusThursday 15 November 12
https://github.com/fiddyspence/puppet-mconotify
Thursday 15 November 12
Thursday 15 November 12
