Puppet: From 0 to 100 in 30 minutes
-
Upload
alessandro-franceschi -
Category
Internet
-
view
515 -
download
1
Transcript of Puppet: From 0 to 100 in 30 minutes
Puppet.From 0 to 100in 30 minutes
(intermediary and next steps are on you)
Alessandro Franceschi @alvagante
From 0Puppet essentials
What is Puppet?
• Puppet is a Configuration Management Tool
• It manages applications, systems' components, network and cloud resources.
• It provides a language that describes the managed IT resources abstracting the implementation details.
• It implements paradigms like Infrastructure as Code and Infrastructure as Data
Puppet components• On managed nodes are executed Puppet agent and Facter.
• On the central Master runs the Puppet server, here public modules and our Puppet code and data are deployed with r10k.
• Puppet generated data is stored to PuppetDB (it uses PostgreSQL for persistence ;)
• Our data to configure classes and manage Infrastructure can be defined using Hiera.
• It's also possible to define what classes to use in which nodes, with External Node Classifiers (ENC) like Puppet Enterprise Console or The Foreman.
A Puppet run in essence• Puppet agent use facter to collect facts about the
system and sends them to the Puppet Master
• Puppet Master, using nodes' facts and our Puppet code and data builds a catalog of resources to manage
• The catalog is sent back to the client. The included resources are applied locally: packages are installed, files changed, services started, databases initialised...
• Whatever the number of runs, at the end the system should be at the desired configuration state (idempotency)
• A report of what happens on the client is sent back to the Puppet Master and typically stored on PuppetDB
Puppet language essentials• Puppet represents managed resources on the systems
via a declarative Domain Specific Language (DSL)
• Puppet language is written in files called manifests, which have a .pp extension
• The single unit of configurations are resource types
• Puppet resources are grouped in classes or defines
• Classes, user defined types (defines), along with relevant configuration files, are shipped in modules
• Public modules are released on Puppet Forge or GitHub
(Resource) types• Puppet types manage resources abstracting the
underlying OS, elements of the system. Examples:package { 'postgresql': ensure => present, }
service { 'postgresql': ensure => running, enable => true, }
file { '/var/lib/pgsql/data/pg_hba.conf': ensure => present, source => puppet:///modules/postgres/pg_hba.conf',}
• Common native resources: package, service, file, user, group, host, cron, exec, host, mount...
Extra resources from modules• Additional resources can be provided by dedicated
modules. Examples from puppetlabs-postgresql module:postgresql::server::db { 'mydatabasename': user => 'mydatabaseuser', password => postgresql_password('mydatabaseuser', 'mypassword'), }
postgresql::server::role { 'marmot': password_hash => postgresql_password('marmot', 'mypasswd'), }
postgresql::server::database_grant { 'test1': privilege => 'ALL', db => 'test1', role => 'marmot', }
postgresql::server::table_grant { 'my_table of test2': privilege => 'ALL', table => 'my_table', db => 'test2', role => 'marmot', }
postgresql::server::pg_hba_rule { 'allow application network to access app database': description => "Open up PostgreSQL for access from 200.1.2.0/24", type => 'host', database => 'app', user => 'app', address => '200.1.2.0/24', auth_method => 'md5', }
Extra resources examples• There are modules to manage almost everything:
- Specific applications (nginx, postgresql...) - Systems' features (iptables, sysctl, network...) - Network equipment (cisco, arista, cumulus... ) - Storage equipment (netapp, emc...)- Cloud resources (aws, azure, digitalocean...) - Cloud infrastructures (openstack, opennebula...) - Containers (docker, mesos, rkt...)
• Each module may provide specific classes and resources to manage with Puppet DSL specific components
Classes• Puppet classes expose parameters which define
what and how are the managed resourcesclass postgresql::server ( $postgres_password = undef, $package_ensure = $postgresql::params::package_ensure, $service_ensure = $postgresql::params::service_ensure, $service_enable = $postgresql::params::service_enable, ) inherits postgresql::params { [ ... ] package { 'postgresql': ensure => $package_ensure, # [...] } service { 'postgresql': ensure => $service_ensure, # [...] enable => $service_enable, # [...] } }
• Class parameters can be set via Hiera, for example using the yaml backendpostgresql::server::postgres_password: 'my_cleartext_password'postgresql::server::package_ensure: absent# Secrets can be encrypted using the hiera-eyaml backend. They look like: postgresql::server::postgres_password: 'ENC[PKCS7,MIIISA ...]
Modules• Modules are distributable directories containing manifests,
extensions, templates and files to manage.
• They have a standard structure:mysql/ # Main module directorymysql/manifests/ # Manifests directory. Puppet code stays here.mysql/lib/ # Plugins directory. Ruby code that extends # Puppet (types, providers, facts...) is here.mysql/templates/ # ERB and EPP Templates directorymysql/files/ # Static files directory
include mysql# Main mysql class is placed in: $modulepath/mysql/manifests/init.pp
include mysql::server# This class is defined in: $modulepath/mysql/manifests/server.pp
mysql::db { ...}# This define is defined in: $modulepath/mysql/manifests/db.ppfile { '/etc/motd': content => template('motd/motd.conf.erb'), } # Template is in: $modulepath/motd/templates/motd.conf.erb
• This allows some conventions:
Hiera• Hiera is a key-value lookup tool based on a
configurable hierarchy
• It can have different backends to store data in different places (yaml, eyaml, json, redis, mysql...)
• Hiera is configured in hiera.yaml--- :backends: - yaml
:hierarchy: - "hostname/%{::trusted.certname}" - "role/%{::role}-%{::env}" - "role/%{::role}" - common
:yaml: :datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"
Roles and profiles• A popular pattern to manage what resources
should be manages on which nodes
• A role is assigned to nodes (using different methods: custom facts, ENC...)
• A role represents what a node does and can include one or more profiles
• In profiles resources from modules are used to configure local systems in the desired way
Puppet environments• A Puppet environment is directory under /etc/puppetlabs/code/environments/ where are is placed our Puppet manifests, modules and Hiera data
• Puppet environments can can be distributed as git control-repos. They contain: manifest directory for common manifests modules directory for moduleshieradata directory for Hiera data, if used Puppetfile to configure r10k for public modules
to 100Introducing example42 Puppet control repo
example42 control-repo• The result of several years of Puppet experience.
A git repository with the content of a Puppet 4 environment featuring:
• Data and code for a state of the art modern Puppet setup
• Fabric integration for common operations
• Puppet 4 optimised code following updated design principles
• Customisable Vagrant environments to locally test code
• Docker integrations to test code or build images
• Classification based on Roles and Profiles pattern
• In green fields: whatever is needed to start a new Puppet project from scratch
• In brown fields: source of inspiration
• ... and, yes, a lot more.
Setup• Download the repo from GitHub
• Setup environment via a shell script ...
• Or via Fabric:
• Install git hooks for pre-commit syntax checks
git clone https://github.com/example42/control-repocd control-repo
bin/setup.sh
fab puppet.setup
fab git.install_hooks
Explore• Give a look to the following files and directories: # The first manifest parsed by Puppet server manifests/site.pp
# r10k Puppetfile and directory for public modules Puppetfilemodules/
# Directory containing local site modulessite/
# Sample Hiera configuration file and data directoryhiera.yamlhieradata/
# Directories for Vagrant and Docker operationsvagrant/docker/
# Blueprint directory for new Puppet 4 modules skeleton/
Test local code with VagrantDifferent Vagrant environments available: fab vagrant.all_status
Single roles can be tested in relevant VMs: fab vagrant.up:vm=dev-local-log-01fab vagrant.provision:vm=dev-local-log-01
# All Linux servers use this class of common resources:# site/profile/manifests/base/linux.pp# Common settings are in: # hieradata/common.yaml# For role "log" specific Hiera data is in# hieradata/role/log.yaml
Vagrant environments can be customised: # vi vagrant/environments/$vagrant_environment/config.yamlvi vagrant/environments/puppetinfra/config.yaml
Test local code with DockerBuild multiOS Docker images: fab docker.build_multios
# Images are buid based on the data in:# hieradata/role/docker_multios_build.yaml
Test a role on a given OS: fab docker.test_role:log,ubuntu-14.04
# Available images: ubuntu-12.04, ubuntu-14.04, ubuntu-14.06 # centos-7, debian-7, debian-8, alpine-3.3
Build an image based on a role (WIP): fab docker.build_role:log,ubuntu-14.04
Build multiple OS images based on a role (WIP): fab docker.build_role_multios:log
next stepslearn Puppet and customise your control-repo
Learning PuppetCheck the Official docs http://docs.puppet.com Ask and get involved in the Community https://puppet.com/community
Look for modules of Puppet Forge http://forge.puppet.com
Give a look to Tiny Puppet (used in example42/control-repo) http://www.tiny-puppet.com
Customise• The control-repo is just the starting point for a
greenfield modern Puppet setup
• Define a way to set your nodes' roles
• Select the public modules to use and add them to Puppetfile
• Write local profiles in site/profile/manifests
• Review hiera.yaml logic and customise data in hieradata/
• Customise your Vagrant environment
• Customise the skeleton to use for custom modules
Contribute• example42's control repo has a lot of room for
enhancements
• Provide profiles to manage applications
• Send bug and features reports
• Use and improve our code
• Spread the word!
#example42
From 0 to 100in 30 minutes?
Thank You(feel free to ask me anything)
Alessandro Franceschi @alvagante