debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . ....

54
debops-playbooks Documentation Release latest Jul 18, 2017

Transcript of debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . ....

Page 1: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks DocumentationRelease latest

Jul 18, 2017

Page 2: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...
Page 3: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

Contents

1 Introduction 1

2 Playbook layout 32.1 List of playbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Custom DebOps features 53.1 The project directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2 Common playbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.3 Host group namespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.4 Flattened lists in inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.5 File, template and task hooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.6 LDAP integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Playbook environment variables 74.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

5 Guides 95.1 Getting Started with DebOps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95.2 Using Linux containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.3 Creating a local apt server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.4 Custom services and their default ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

6 Copyright 21

7 Changelog 237.1 debops-playbooks master - unreleased . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237.2 debops-playbooks v0.2.9 - 2016-07-07 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267.3 debops-playbooks v0.2.8 - 2016-02-07 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277.4 debops-playbooks v0.2.7 - 2015-10-15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.5 debops-playbooks v0.2.6 - 2015-07-14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307.6 debops-playbooks v0.2.5 - 2015-04-01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317.7 debops-playbooks v0.2.4 - 2015-03-26 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317.8 debops-playbooks v0.2.3 - 2015-03-05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.9 debops-playbooks v0.2.2 - 2015-02-25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.10 debops-playbooks v0.2.1 - 2015-02-24 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.11 debops-playbooks v0.2.0 - 2015-02-22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337.12 debops-playbooks v0.1.0 - 2015-02-16 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

i

Page 4: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

7.13 2015-02-12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337.14 2015-02-06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337.15 2015-02-05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347.16 2015-02-04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347.17 2015-02-01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357.18 2015-01-31 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367.19 2015-01-28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367.20 2015-01-21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377.21 2015-01-20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377.22 2015-01-18 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377.23 2015-01-13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387.24 2014-12-23 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.25 2014-12-05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.26 2014-12-03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.27 2014-12-02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.28 2014-12-01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.29 2014-11-27 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.30 2014-11-26 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.31 2014-11-25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.32 2014-11-24 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.33 2014-11-22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427.34 2014-11-20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427.35 2014-11-19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427.36 2014-11-13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437.37 2014-11-07 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437.38 2014-11-05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447.39 2014-11-04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447.40 2014-11-02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447.41 2014-10-31 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457.42 2014-10-28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457.43 2014-10-17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457.44 2014-10-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467.45 2014-10-09 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467.46 2014-10-07 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467.47 2014-10-05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467.48 2014-10-02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477.49 2014-09-29 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477.50 2014-09-22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477.51 2014-09-21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487.52 2014-09-19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487.53 2014-09-18 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487.54 2014-09-17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497.55 2014-09-14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

ii

Page 5: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 1

Introduction

To use Ansible roles, you need playbooks which define what roles are run on which hosts. debops-playbooksrepository contains a set of such playbooks which let you use roles without the need to write them yourself.

Apart from the playbooks, this repository contains a set of Ansible modules, various plugins and custom fact scriptsused through the project. Some of them are required by the whole project, some just by a few selected roles.

1

Page 6: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

2 Chapter 1. Introduction

Page 7: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 2

Playbook layout

DebOps playbooks is split into several files to allow partial usage possible:

site.yml| ,---- <- core.yml|-- <- common.yml`-,

|-- <- systems.yml|-- <- environments.yml|-- <- networking.yml|-- <- services.yml|-- <- applications.yml|-- <- virtualization.yml`-- <- hardware.yml

When you run debops script or ansible-playbook, you can either run the main site.yml playbook, orspecify name of the playbook you want to use to narrow the set of roles; this makes the Ansible runs shorter andlowers the startup time.

Order of the playbooks apart from the common ones (common.yml, core.yml) is not significant, although runningroles that are used many times as role dependencies first might make the whole playbook run faster.

List of playbooks

Playbooks which are common for all hosts:

site.yml Main playbook, run by DebOps scripts by default, includes all other playbooks.

common.yml Playbook which runs on all hosts included in Ansible inventory. It executes a set of common roleswhich configure base services like SMTP service, a set of user accounts, ip(6)tables firewall, APT reposi-tories, and so on.

core.yml This is a playbook required on all hosts that use DebOps roles, regardless if you are using the playbooksor not. It’s included by the common.yml playbook. It will set up custom Ansible facts required by some of the

3

Page 8: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

roles, like root paths for several directory types, host UUID, installation of scripts that generate facts on the fly,and so on.

It also gathers the IP address of the Ansible Controller, or IP of the closest router which leads to it, to allowconnections from that IP address through the firewall.

Playbooks which have only roles that are activated by specific Ansible host groups:

systems.yml This playbook includes roles that configure services and resources that might be required by otherroles, such as user and group accounts, authentication services like LDAP, network filesystems like NFS. Any-thing that is expected to be used by other roles further down the playbook, but it’s not common enough to beincluded in the common.yml playbook, should be added here.

environments.yml This is a playbook focused on programming language environments, like Ruby, PHP, Java,NodeJS. Since these might be used by multiple roles further down the playbook, they are grouped here to be runfirst so that other roles might be executed faster.

networking.yml Playbook which focuses on roles that manage various network-related services, like DHCP,DNS, creating subnetworks or tunnels.

services.yml This playbook manages separate services like a webserver, various databases, file servers and oth-ers. These are usually standalone services which might be used by other roles down the line.

applications.yml This playbook manages either end-user applications which might use multiple services (usu-ally web applications like GitLab or phpIPAM) or end-point applications which can be used by other hosts inthe cluster, like iPXE, or rsnapshot.

virtualization.yml This playbook focuses on virtualization and hypervisors, like OpenVZ, KVM/libvirt orLXC.

hardware.yml At the end are roles which directly manage resources and services related to hardware, for exampleRAID health monitoring and notification.

4 Chapter 2. Playbook layout

Page 9: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 3

Custom DebOps features

To make integration of DebOps roles with your own infrastructure easier, DebOps playbooks include a set of Ansibleplugins and introduce several new concepts to Ansible best practices.

The project directory

By default, Ansible is written to use /etc/ansible/ directory and its contents in its daily use. In contrastto this, DebOps playbooks are designed to be used from a custom local directory, which you can initialize usingdebops-init command. By using Ansible this way, it’s much easier to create multiple, separate environments withdistinct inventories and configuration. To change the environment you are working in, you just need to switch to adifferent directory - there’s no need to use separate Ansible host groups, custom variables and so on.

The official playbooks and roles are installed in central, fixed location (~/.local/share/debops/debops-playbooks/ on Linux systems), and the debops script generates ansible.cfg configuration fileto provide correct paths for ansible-playbook command to use them indirectly from the project directory.

You can store your custom playbooks and roles in the project directory, in playbooks/ and roles/ subdirectories.

Common playbooks

In many Ansible environments a popular practice is to have a “common role” that contains tasks that are expected tobe run on any and all hosts managed by Ansible.

In DebOps, there is an entire playbook dedicated to this, located in playbooks/common.yml. It includes multipleroles that prepare a host from an unknown to a known state - for example, a ferm-based firewall will be installedand configured on a given host, unless disabled, some common, useful packages will be installed, and so on. OtherDebOps roles not included in the common.yml playbook are designed for hosts that were configured by it - theymight work outside of that environment, but it’s not guaranteed.

5

Page 10: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Host group namespace

To make host configuration in Ansible inventory more explicit, DebOps uses a set of Ansible host groups. All of theofficial groups are set in the [debops_*] namespace, so you are free to use other names without any possibility ofa collision.

Common DebOps playbook, as well as some other service playbooks that are included in it, use[debops_all_hosts] group. This is a base group of the project and all hosts managed by DebOps should beincluded in it.

Service playbooks use the [debops_service_*] group namespace in Ansible inventory (for example, de-bops.nginx_ role is activated on hosts in [debops_service_nginx] group). Some service playbooks use addi-tional groups for various purposes; you are advised to check the role documentation to see what is their intended usecase.

Flattened lists in inventory

Some DebOps roles use sets of default variables (usually lists) to allow you to define different settings for all hosts ininventory, a group of hosts, or even specific hosts. For example, using debops.sshd_ role you can whitelist a certainsubnet for all hosts in your inventory, add another subnet for a particular group of hosts, and so on. You can alsooverride more general list on specific hosts if needed.

File, template and task hooks

DebOps project introduces a set of Ansible lookup plugins which allow you to override certain aspects of public An-sible roles without modifying them directly. This allows for easier updates or customization of the files and templatesaccording to your specific needs.

Certain roles use file_src or template_src to calculate path to files or templates used by a role. You canoverride these paths using .debops.cfg configuration file and provide your own versions of files and templatesstored in DebOps project directory.

Some roles provide “task hooks” at the beginning and end of task lists, which are empty files in a specific subdirecto-ries. Using task_src lookup plugin and settings defined in .debops.cfg configuration file you can “inject” yourown tasks at the beginning or end of these roles, which gives you more control over the configuration.

By combining above techniques, you can very easily extend DebOps roles without losing the ability to update themusing git without merge conflicts.

LDAP integration

Certain DebOps roles can access LDAP server to create or update data as needed. Custom modules are provided forLDAP entry and attribute management, deeper integration is planned in the future.

6 Chapter 3. Custom DebOps features

Page 11: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 4

Playbook environment variables

In certain situations, for example on a network where direct Internet access is not allowed and users are required to usea HTTP proxy, you might need to define a custom set of environment variables for Ansible to execute playbooks. TheDebOps playbooks allow you to do that using a set of Ansible inventory variables which should be defined as YAMLdictionaries:

inventory__environment This variable is meant to set environment variables on all hosts in Ansible inventory.

inventory__group_environment This variable is meant to be used on a group of hosts in Ansible inventory.Only one group is supported.

inventory__host_environment This variable is meant to set environment variables on specific hosts in An-sible inventory.

The configured environment variables will be active in all of the DebOps playbooks included in this repository. Themore specific variables override the more general ones, just as normal Ansible variables.

The environment variables defined using these YAML dictionaries have only effect during the ansible-playbookrun. Normal ansible commands as well as commands/services executed on remote hosts will not use them. Toconfigure desired environment variables on remote hosts, you might wan to check the debops.resources_ Ansiblerole.

Examples

To configure a HTTP proxy which should be used by Ansible roles on all hosts, add in the ansible/inventory/group_vars/all/inventory.yml file:

inventory__environment:http_proxy: 'http://proxy.{{ ansible_domain }}:3128'

To add support for these variables in your own playbooks, make sure that they contain the following code:

- name: Configure a custom servicehosts: [ 'debops_service_custom' ]become: True

7

Page 12: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

environment: '{{ inventory__environment | d({})| combine(inventory__group_environment | d({}))| combine(inventory__host_environment | d({})) }}'

roles:

- role: custom-roletags: [ 'role::custom' ]

8 Chapter 4. Playbook environment variables

Page 13: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 5

Guides

Getting Started with DebOps

Welcome to DebOps! You have installed Ansible and DebOps scripts, downloaded the roles and playbooks, and arewondering where to go next? Here you can read about creating your first DebOps project, and managing remote hosts.

• An example environment

• Your first project

• Important inventory variables

– ansible_user

– bootstrap__domain

– sshd__whitelist

– ntp__timezone

– nullmailer__relayhost

– apt__default_mirrors_lookup, apt__default_sources_lookup

• Bootstrap a new host

• Configure the remote host

• Example application - DokuWiki

• Where to go from here

9

Page 14: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

An example environment

Ansible and DebOps are installed on your workstation or laptop, a so called “Ansible Controller” - this machine isused to control Ansible and issue commands. The machine you will configure using DebOps is known as a “remotehost”.

DebOps is designed to manage a host from the ground up. A good base installation is a Debian Stable netinst, withonly SSH server enabled and configured. Everything else will be installed as needed.

Note: If you are using Debian Jessie or other distributions based on it as the base install, by default OpenSSHserver configured by the installer will disallow password authentication on the root account. You can either enableit manually in the /etc/ssh/sshd_config file, or configure a separate admin account and use that to bootstrapthe host.

An important part of the environment is correctly configured DNS. Some of the DebOps roles expect a configureddomain - it doesn’t need to be a real, global domain, but it should be resolvable by the host. A good way to check if aremote host has a correctly configured domain is to use the hostname --fqdn command. If the output has at least1 dot, you should be good to go.

Do not use the domain apex (example.com) as the host name - this will confuse Ansible and leave you with abroken configuration. Instead, create separate hostnames inside the domain (server.example.com), this will beused by Ansible correctly. It’s good to use only one subdomain level per subnet, this will make wildcard certificateswork without issues. If you want, you can create separate subdomains per subnet.

If your host does not have a domain configured, you will be able to do that during the bootstrapping process.

In this guide, we will manage an example host called server.example.com. This host is a virtual machine, andwe can connect to it using commands like:

alice@laptop:~$ ssh [email protected]@laptop:~$ ssh root@server

The root account requires a password, SSH keys are not installed yet and there’s no administrator account.

Ansible commands are executed on the Ansible Controller from an unprivileged account alice. This user has anSSH key pair stored in ~/.ssh/id_rsa or has its SSH key available in the SSH Agent. An administrator accountwith the same name will be created on the remote host during the bootstrap process.

Your first project

Begin by creating a “DebOps project”. It’s a directory which contains all of the data related to a given environment- Ansible inventory, passwords and other secrets, custom playbooks and roles. To do this, use the debops-initcommand:

alice@laptop:~$ debops-init ~/myproject

This will create a new directory called myproject and populate it with some example directories and files. You willperform most of the commands from the main project directory, so let’s cd into it:

alice@laptop:~$ cd ~/myproject

Ansible uses a hosts file to identify hosts that are under its control. In the project directory this file is located inansible/inventory/hosts. Open it in your favorite text editor and add the remote host in the main DebOpshost group:

10 Chapter 5. Guides

Page 15: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

[debops_all_hosts]server ansible_ssh_host=server.example.com

Using a short inventory name allows you to run Ansible commands without specifying the fully qualified domain nameof the host.

Important inventory variables

Some of the configuration used by DebOps cannot be auto-detected - examples include IP addresses or networksubnets that can connect to a SSH service remotely, the administrator e-mail account which should receive importantnotifications, and so on. Here you can find a list of the most important variables which, when set correctly in inventory,can save you a trip to the data center.

To make sure that these variables apply to all hosts in your environment, you can include them in ansible/inventory/group_vars/all/ directory. A common practice is to name the files inside inventory directoriesafter variable prefixes, separately for each Ansible role. For example, variables related to debops.sshd_ role are storedin ansible/inventory/group_vars/all/sshd.yml, variables used by the debops.postfix_ role are writ-ten in ansible/inventory/group_vars/all/postfix.yml, and so on. The same scheme can be used inother inventory groups or for separate hosts.

ansible_user

This is an internal Ansible variable which is used to determine what remote user account will be used to login to theserver. If it’s not explicitly set, Ansible depends on SSH defaults which conventionally use the name of the currentuser as the remote username. It’s customary to specify this variable directly in the hosts file, that way it can beunique for each host:

[debops_all_hosts]server ansible_ssh_host=server.example.com ansible_user=ansible-admin

In DebOps this variable can be used to change the name of the default administrator account, it’s also used as a primaryuser account for various tasks, like database and application administrative accounts.

On a specific platforms you can set this variable to an automatically created username to make the remote host admin-istration easier:

• Ubuntu-based hosts usually use the ubuntu username;

• Raspberry Pi / Pi 2 Linux distributions use the pi user account for this purpose;

However, it is advisable to not use the default user accounts, and instead either create ones based on your own username(the default behavior) or create completely separate Ansible accounts with administrative access. If you configure theansible_user variable before bootstrapping the host, the specified username will be used to create an administratoraccount.

bootstrap__domain

If hosts that you want to manage don’t have a DNS domain set, or it’s incorrect (for example your VPS provider’sdomain instead of your own), the debops.bootstrap_ role can be used to easily fix that and configure your owndomain. By setting this variable to, for example:

---bootstrap__domain: 'example.com'

5.1. Getting Started with DebOps 11

Page 16: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

By running the debops bootstrap command (see further down), your domain will be configured in the remotehosts’ /etc/hosts file. Additionally, the hostname will be changed to the one you specified in the Ansible inventory.After that is done, it’s best to reboot the machine to make sure all of the changed settings are applied and are persistent.

This variable won’t have any effect on hosts that are not “bootstrapped”, and are instead configured using Debianpreseeding or LXC templates - these hosts will presumably get the needed information like hostname and domainfrom your own DHCP server.

You can check other debops.bootstrap variables for some more useful configuration, like the name of the administratoraccount.

sshd__whitelist

Protection of the SSH service is very important. Hosts configured by DebOps use a firewall and TCP Wrappers torestrict what hosts can connect to it and automatically block repeated offenders for certain amount of time.

To not block the Ansible Controller, DebOps tries to detect the IP address from which the connection is made. Forthe most part it should work as expected, but if you still are getting blocked, or to be sure that remote access won’tbe interrupted, you can define a list of IP addresses or CIDR subnets that will be allowed to connect to SSH withoutrestrictions.

To do that, in ansible/inventory/group_vars/all/sshd.yml add:

---sshd__whitelist: [ '192.0.2.0/24', '2001:db8::/32' ]

This will configure the debops.ferm_ and debops.tcpwrappers_ roles to allow connections to the ssh service fromspecified networks.

The debops.sshd_ role has many more variables, you can checkout to see the default configuration used by DebOpsand what can be changed as needed.

ntp__timezone

By default, DebOps does not try to change the remote host timezone and tries to use the detected one in rolesthat need that information for the configuration. If you need to change the timezone, you can do it by setting thentp__timezone variable like this:

---ntp__timezone: 'America/New_York'

For UTC timezone, use this format:

---ntp__timezone: 'Etc/UTC'

nullmailer__relayhost

The default SMTP server used by DebOps is nullmailer. It’s a simple, forward-only Mail Transport Agent whichsends all mail to another SMTP server for processing. It does not provide support for local mail accounts.

By default, nullmailer will send mail messages to the smtp.<your-domain> host (it does not support MXrecord lookups). If this host doesn’t exist, or your local SMTP server has a different address, you can change it bysetting the variable:

12 Chapter 5. Guides

Page 17: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

---nullmailer__relayhost: 'internal-mx.{{ ansible_domain }}'

Only one relayhost is supported at a time. The specified host should accept messages from hosts controlled by Ansiblefor this to work correctly. The SMTP connections will be encrypted using STARTTLS command, therefore the SMTPshould use a set of X.509 certificates which are trusted by the host.

The nullmailer service can be configured to a large extent using the debops.nullmailer role variables - you can usethem to configure SMTP authentication, use multiple relay servers, and so on.

If you need a more powerful SMTP server, DebOps includes support for Postfix as well - check the debops.postfix_Ansible role.

apt__default_mirrors_lookup, apt__default_sources_lookup

DebOps tries to detect the operating system a given host is using and configure it accordingly. Currently selectedDebian and Ubuntu releases are recognized and the package sources for these operating systems should be configuredwithout issues.

The Raspbian operating system is a little difficult to detect, because Ansible currently classifies it as “Debian”, howeverits package repositories are completely different. To avoid issues with incompatible package sources on your Rasp-berry Pi/Pi2, you should change the default debops.apt_ configuration manually to use the Raspbian repositories. Todo that, add these values in relevant inventory files:

---apt__default_mirrors_lookup: 'raspbian'apt__default_sources_lookup: 'raspbian'

Bootstrap a new host

Warning: Bootstrapping a host without a configured bootstrap_domain will result in a broken host config-uration.

At this point you most likely have to connect to that host using the root account and specifying a password. To makethat easier, you can use a special “bootstrap” Ansible playbook to prepare a host for easier management. To do this,execute the command:

alice@laptop:~/myproject$ debops bootstrap --limit server --user root --ask-pass

Or, for short:

alice@laptop:~/myproject$ debops bootstrap -l server -u root -k

This command will execute the debops.bootstrap_ role and use it to install a base set of packages needed by Ansiblelike python and sudo, prepare a new administrator account named after your system user (alice in our example)and allow that account full access to the root account using sudo. Your SSH keys will be installed on both theroot and administrator accounts.

Note: Bootstrapping a host this way is not needed if you already have an administrator account that can usesudo without a password. This includes hosts configured using Debian Preseed provided by DebOps as well asOpenVZ/LXC containers configured using provided templates.

5.1. Getting Started with DebOps 13

Page 18: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

When the bootstrap playbook has finished and there are no errors, you can check if you are able to connect to theserver on the administrator account without a password:

alice@laptop:~/myproject$ ssh server

After logging in, check if you can run commands using sudo without a password:

alice@server:~$ sudo -l

Configure the remote host

When a new remote host has been prepared for Ansible management, you can start the configuration:

alice@laptop:~/myproject$ debops -l server

This will start the ansible-playbook command with the main DebOps playbook. This by default includes thecommon playbook with a default set of roles, and any additional playbooks, if they have been enabled.

The initial configuration might take 5-10 minutes on a reasonably fast machine. There are some steps, like Diffie-Hellman parameter generation, which might take significantly more time to complete.

When the playbook run has been finished, your remote host should be configured with:

• a correct set of APT repositories for your operating system release;

• automatic updates of the installed packages with related e-mail messages sent to your admin account;

• a set of Diffie-Hellman parameters and SSL certificates ready to use by different services (encrypted TLS/SSLconnections out of the box);

• configured iptables/ip6tables firewall and TCP Wrappers;

• enabled network time synchronization as needed;

• a set of useful management software installed on the host (htop, mtr-tiny, mc, vim, among other things);

Example application - DokuWiki

Each host configured by common DebOps playbook should have the same set of base services. After a host is config-ured, you can enable additional Ansible roles to install and configure software and applications of your choice.

We will use DokuWiki as an example application. The role that manages the installation is called debops.dokuwiki_, ituses debops.nginx_ and debops.php5 roles to configure a webserver and PHP5 environment. The debops.nginx_role calls some additional roles, such as debops.ferm_ to configure needed services.

To install DokuWiki on your new remote host, you need to enable the respective role in Ansible inventory. This isdone by creating a new host group, [debops_service_dokuwiki] in the hosts file, and adding the desiredhosts to it:

[debops_all_hosts]server ansible_ssh_host=server.example.com

[debops_service_dokuwiki]server

As you can see, you don’t need to copy the whole host entry, only the short name is enough.

The debops.dokuwiki_ has many default variables you can use to customize the installation. One of the more usefulones is dokuwiki_main_domain; it’s a list which specifies what DNS subdomains are used to access the wiki

14 Chapter 5. Guides

Page 19: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

(each application in the DebOps set of roles is configured on a separate subdomain). By default DokuWiki willbe accessible on the wiki.{{ ansible_domain }} subdomain, if you want to change it, you can do so bycreating the ansible/inventory/host_vars/server/dokuwiki.yml configuration file and specifyingthe subdomain(s) in it:

---dokuwiki__main_domain: 'wiki.{{ ansible_domain }}'

Remember that the chosen subdomain (wiki. or your own) needs to be configured in your DNS server to point tothe specified remote host.

When everything is configured, you can execute the debops script to apply new configuration on the host:

alice@laptop:~/myproject$ debops -l server

This will apply the whole playbook with all the configuration on the specified server. However, to make this processfaster, DebOps provides separate “service playbooks” for each of the roles. To use these playbooks, you can specifythem as the first argument to the debops command:

alice@laptop:~/myproject$ debops service/dokuwiki -l server

This will tell the script to look for the playbook in several places:

• playbooks/ and ansible/playbooks/ subdirectories in the project directory;

• debops-playbooks/playbooks/ subdirectory of the project directory, if DebOps playbooks and rolesare installed inside of it;

• ~/.local/share/debops/debops-playbooks/playbooks/ directory (default install location);

The first one found will be executed. You can use this to your advantage by adding custom playbooks in playbooks/or ansible/playbooks/ directories, they need the be named with .yml extension. Custom roles can be placedin the roles/ or ansible/roles/ subdirectories located in the project directory.

After Ansible finishes the configuration, you will need to go to the https://wiki.<domain>/install.phppage to complete the installation process.

At this time you might find that the web browser you are using does not recognize the CA certificates served bythe host. This happens when server uses certificates signed by internal DebOps Certificate Authority instead of the“regular” ones. To fix that, consult the debops.pki_ role documentation (when it’s available).

Where to go from here

You can add more hosts to the Ansible inventory and configure them in a cluster. Hosts should automatically trusteach other using an internal Certificate Authority, so encrypted connections between them should work out of the box.

DebOps contains multiple Ansible roles that allow you to install and configure useful software, like GitLab, phpIPAM,ownCloud and others. You should check the documentation of the respective roles to see some example configurationsand useful tips. Note that parts of the documentation are currently outdated - if a given role has only one page, youshould check the role files directly.

You can check the DebOps Changelog for updates related to roles and playbooks (there’s also an Atom feed availablefor your feed reader).

Using Linux containers

• Host requirements

5.2. Using Linux containers 15

Page 20: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Configuring a host to make it capable of storing containers

• LXC cheatsheet to help you manage the containers

• Interacting with a container

Host requirements

Your host must be Debian based

It can be Ubuntu 14.x, Debian Wheezy/Jessie, etc.

If you’re using a Mac or a different Linux distro then you’ll want to setup a virtual machine to act as the containerhost. You can do this with Vagrant or some other virtualization software.

SSH key pair

You will also need an SSH key pair on your host. You probably have one setup but if you don’t you can runssh-keygen -t and follow the instructions. DebOps expects the RSA keys to be in ~/.ssh.

Configuring a host

Add it to your inventory

The paths are relative to where you debops-init a new project.

ansible/inventory/hosts

[debops_service_lxc]yourhostname

Decide on which network adapter you’re using

If you plan to make your main OS an LXC host then you’ll want to configure the host to use the NAT adapter bydefault. DNS is configured through NAT using dnsmasq.

Basically this means you don’t have to forward ports and DNS will work.

ansible/inventory/host_vars/yourhostname.yml

lxc_configuration_default: 'nat'

If you plan to use the bridged adapter through a VM then you do not have to set anything but keep in mind you willneed to connect through an IP address unless you have configured DNS yourself.

Make the host an LXC host by running DebOps

Run this from your terminal: debops -l debops_service_lxc.

If you are running Debian Wheezy you will have to reboot your LXC host due to a kernal update. Other operatingsystems like Ubuntu 14.x and Debian Jessie do not require the reboot.

LXC cheatsheet

# Create a new containersudo lxc-create -n mycontainer -t debops

# Return back a list of containers and basic information about themsudo lxc-ls -f

16 Chapter 5. Guides

Page 21: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

# Start a container, the -d flag runs it as a daemonsudo lxc-start -n mycontainer -d

# Stop a containersudo lxc-stop -n mycontainer

# Destroy a container, the -f flag does a stop before destroying itsudo lxc-destroy -n mycontainer -f

# There are many more commands like snapshotting, freezing, info, etc.# Check the LXC manpages for more informationsudo lxc-[tab complete]

Interacting with a container

Once it has been created and it’s running you can SSH to it, just run:

ssh containername if you have DNS setup, otherwise use the IP address. At this point you have a bare containerready to do whatever you want.

Setting it up with common DebOps services

If you plan to use containers for development then you’ll probably want to group your containers together in yourinventory.

ansible/inventory/hosts

[local_containers]mycontainer

Now you could create ansible/inventory/group_vars/local_containers.yml and start doing thingsthat would apply to all local containers.

Perhaps you want to install emacs or use your own dotfiles, etc..

Transferring files

To transfer files to/from the container you have 2 options.

1. SCP or some other file transfer utility that works through SSH

# To the containerscp somefile mycontainer:/tmp/somefile

# From a containerscp mycontainer:/tmp/somefile somefile

The second option requires knowing the dirty details about where the container has its configuration and file systemstored.

On the LXC host, navigate to /var/lib/lxc, then go into your container’s directory. You can find its file systemthere among other things. You can simply cp directly if your LXC host is local to your main OS.

Creating a local apt server

• What are some benefits of doing it this way?

5.3. Creating a local apt server 17

Page 22: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Pick a server

• Configure a throw away build server

• Configure the local APT server

• Make your hosts aware

• Use your shiny new package

Certain roles such as Ruby and Golang offer the ability to use a backported version of the package so it’s more up todate. The backports are built off of Debian testing without having to actually use the testing apt source.

What are some benefits of doing it this way?

A lot of other roles will compile from source but that’s time demanding and error prone. A backported version ofRuby 2.1.x will apt install in about 5 seconds once you setup your local APT server once.

Compile it once into a proper package and use it as many times as you want.

It also future proofs your role because you wouldn’t have to change anything once the next Debian version is officiallyreleased. From the role’s point of view it’s just installing an apt package using Ansible’s apt module. It does not carewhere the apt server is located.

Pick a server

The first step is to decide where you want this server. It doesn’t need to be literally local to your workstation. It’s localin the context of it not being an official APT server to the world.

Popular options could be your Ansible controller inside of a container or a micro-size instance on the cloud dependingon your requirements for availability.

Configure a throw away build server

You could use your apt server but it’s best to use a temporary host. I would just spin up a container.

In this example we’re going to build Ruby 2.1.x. You will have to do this if you plan to use GitLab so it’s a good ideato learn!

# inventory/hosts

[debops_service_ruby]yourbuildserver

# inventory/host_vars/yourbuildserver.yml

ruby_version: 'backport'

That tells the Ruby role to use the Backporter role as a dependency and that will kick off the entire build process foryou.

Then run:

debops -l yourbuildserver

Expect it to take 5 to 15 minutes depending on how fast your server is. You only need to do this once.

18 Chapter 5. Guides

Page 23: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Where are the packages

Good question, they have been transferred to your Ansible controller in the secret/reprepro/includedeb/wheezy-backports/ directory.

At this point you can delete your build server.

Configure the local APT server

Next up, we need to tell our server that it is an APT server.

# inventory/host_vars/youraptserver.yml

apt: 'youraptserver.{{ ansible_domain }}'

You must use your apt server’s fully qualified domain name. Run hostname -f on the server to check its fullyqualified domain name.

We’re just about done, now you need to transfer the packages to your apt server:

debops -l youraptserver -t apt

Make your hosts aware

The last step is to make your hosts aware of the server.

Below I’m just assuming you want to make it aware to all of your containers and you have your containers inside of a[containers] group.

# inventory/group_vars/containers.yml

apt: 'youraptserver.{{ ansible_domain }}'

Then run:

debops -l containers

Use your shiny new package

Well, this part is easy. Just use the Ruby role on any host that is aware of your local apt server and it will install Ruby2.1.x in about 5 seconds.

You do not need to set ruby_version: 'backport' on the hosts themselves. It will just use the defaultsetting which is the apt package and now since your local apt server is setup and your host is aware, it will use the newbackported version.

Custom services and their default ports

In various DebOps roles you can find named ports. They are defined in /etc/services using debops.etc_servicesrole which manages them using Ansible’s assemble module. To avoid collisions between various services we listhere custom ports that are set for applications and services that don’t have specified system ports by default.

You can find a list of ports used throughout the DebOps project by running command:

5.4. Custom services and their default ports 19

Page 24: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

debops-defaults | grep '_port:'

This should output list of all variables that define port numbers in various roles and are available in role defaults, andthus can be overridden by Ansible inventory.

Service Port Default bindapt-cacher-ng 3142 all interfaceselasticsearch 9200-9400 localhostetherpad 9000 localhostredis-server 6379 localhostredis-sentinel 26379 localhostrails apps 3000 socketgitlab-ci 18083 localhost

Standard ports

Run cat /etc/services to obtain a list of standard ports.

20 Chapter 5. Guides

Page 25: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 6

Copyright

debops-playbooks - Set of Ansible playbooks for DebOps Project

Copyright (C) 2013-2017 Maciej Delmanowski <[email protected]>Copyright (C) 2015-2017 Robin Schneider <[email protected]>Copyright (C) 2014-2017 DebOps https://debops.org/

This repository is part of DebOps.

DebOps is free software; you can redistribute it and/or modifyit under the terms of the GNU General Public License version 3, aspublished by the Free Software Foundation.

DebOps is distributed in the hope that it will be useful,but WITHOUT ANY WARRANTY; without even the implied warranty ofMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See theGNU General Public License for more details.

You should have received a copy of the GNU General Public Licensealong with DebOps. If not, see https://www.gnu.org/licenses/.

21

Page 26: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

22 Chapter 6. Copyright

Page 27: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

CHAPTER 7

Changelog

debops-playbooks

This project adheres to Semantic Versioning and human-readable changelog.

The current playbook maintainer is drybjed_.

debops-playbooks master - unreleased

Added

• Add support for custom Ansible environment variables in all playbooks. Environment can be configured usingAnsible inventory variables. [drybjed_]

• Add debops.php_ role and its corresponding playbook. [drybjed_]

• Add debops.environment_ role with it’s own playbook, and include it in the common.yml playbook. [dryb-jed_]

• Add debops.authorized_keys_ role with its own playbook, and include it in the common.yml playbook. Thisrole replaces the debops.sshkeys role and is backwards-compatible with it. [drybjed_]

• Add debops.debops_api_ role and its corresponding playbook. [ypid_]

• Add debops.hashicorp_ role and its corresponding playbook. [drybjed_]

• Add Core Infrastructure Initiative (CII) Best Practices badge to the repository README. [ypid_]

• Add debops.nullmailer_ role and its playbook. [drybjed_]

• Add debops_service_bootstrap Ansible host group to allow to run debops.bootstrap_ against hostsnot in the debops_all_hosts host group. [ypid_]

• Add debops.debops_fact_ role with corresponding playbook and include it in the common.yml playbook.[drybjed_]

• Add debops.sysctl_ role with corresponding playbook and include it in the common.yml playbook. [ypid_]

23

Page 28: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Add debops.apt_listchanges_ and debops.apt_proxy_ roles with their corresponding playbooks. Both rolesare included in the common.yml playbook. [drybjed_]

• Add debops.apache_ role and its corresponding playbook. [ypid_]

• Add debops.gunicorn_ role and its corresponding playbook. [drybjed_]

• Add debops.netbox_ role and its corresponding playbook. [drybjed_]

• Add debops.cron_ role and its playbook, and include it in the common.yml playbook. [drybjed_]

• Add debops.persistent_paths_ role and its corresponding playbook. [ypid_]

• Add alternative debops.dnsmasq_ playbook for use together with the debops.persistent_paths_ role. [ypid_]

• Add alternative debops.tinc_ playbook for use together with the debops.persistent_paths_ role. [ypid_]

• Add commonly used set of Jinja2 macros to the repository under ./templates/debops__tpl_macros.j2 to have acentral place where the file can be maintained and from where the latest version can be acquired. [ypid_]

• Add the debops.root_account_ role, included in the common.yml playbook. This role contains functionalitypreviously present in the debops.console_ role. This change will result in updated root account passwords,due to the changed location of the password files in the secret/ directory. [drybjed_]

• Add flattened Jinja filter to debops__tpl_macros.js_. [ypid_]

• Add the debops.nfs_server Ansible role with a corresponding playbook. [drybjed_]

• Add the debops.nsswitch Ansible role with its corresponding playbook. The role is included in thecommon.yml playbook. [drybjed_]

• Add the debops.avahi Ansible role with its corresponding playbook. [drybjed_]

• Add the debops.mosquitto Ansible role and its corresponding playbooks. [drybjed_]

• Add the debops.elastic_co Ansible role and its corresponding playbook. [drybjed_]

• Add the debops.kibana Ansible role and its corresponding playbook. [drybjed_]

• Add the debops.libvirtd_qemu role and playbook, update the debops.libvirtd playbook. [dryb-jed_]

• Add the debops.rabbitmq_server role and playbook. [drybjed_]

• Add the debops.rabbitmq_management role and playbook. [drybjed_]

• Add the debops.etc_aliases role and playbook. [drybjed_]

Changed

• Update debops.dokuwiki_ playbook. [drybjed_]

• Update debops.bootstrap_ playbook. [drybjed_]

• Update playbooks which use debops.php_ role to include the debops.apt_preferences_ role dependency. [dry-bjed_]

• Remove the debops.postfix_ role from the common.yml playbook. The Postfix server is replaced bynullmailer on new installations.

On existing installations, debops.nullmailer_ will detect Postfix and disable itself automatically. This meansthat the DebOps playbook won’t manage the existing Postfix installations anymore using the common.ymlplaybook. To enable Postfix management, add the host to [debops_service_postfix] inventory group.[drybjed_]

• Update the debops.owncloud_ playbook and add a playbook for debops.apache_ support. [ypid_]

24 Chapter 7. Changelog

Page 29: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Update debops.librenms_ playbook. [drybjed_]

• Update the ipaddr() filter plugin with the version from Ansible devel branch. [drybjed_]

• Update the debops.lxc_ playbook to the new version. [ypid_]

• Update the debops.redis_ playbook to the new version. [drybjed_]

• Update the debops.dnsmasq_ playbook to the new version. [ypid_]

• Update the ldap_addr and ldap_entry Ansible modules to versions created by Jiri Tyr and proposed forinclusion in the Ansible modules library. These versions should work better with the latest Ansible 2.x releases.[drybjed_]

• Update debops.ifupdown_ playbook to support new features in the role. You will need to update your inventory,check the role documentation for more details. [drybjed_]

• Make sure that debops.tcpwrappers_ role is included in the debops.dnsmasq_ playbook because the DNSmasqrole notifies the debops.tcpwrappers_ handler in case of any changes. [drybjed_]

• Update the ldap_entry Ansible module to the new version. This is a non-backwards compatible change,roles that use ldap_entry will need to be updated to use the new module atributes parameter instead ofspecifying LDAP entry parameters directly as module parameters. [drybjed_]

• Update the debops.swapfile_ playbook to the new version. [ypid_]

• Treat the debops.core_ role like a normal service role by moving it to playbooks/service/core.yml.This change is part of my effort to make DebOps usable on Qubes OS where you might only want to run aminimum set of roles. [ypid_]

• Update the debops.owncloud_ with Apache playbook to the new version. [ypid_]

• Update the debops.cryptsetup_ playbook to the new version. [ypid_]

• Update the debops.gitlab_ playbook to use role dependencies. [drybjed_]

• The order of the NFS server/client roles has been moved to be earlier in the main playbook to allow for mountingof the NFS shares early. [drybjed_]

• Update all playbooks that use the debops.java_ role with new debops.apt_preferences_ role dependency. [dry-bjed_]

• Update debops.salt_ playbook with new role dependencies. [drybjed_]

• Support dist upgrade from Debian Jessie to Debian Stretch using tools/dist-upgrade.yml. [ypid_]

• Update the debops.elasticsearch_ playbook with new release of the role. [drybjed_]

Deprecated

• Support for Ansible 2.0 is deprecated due to the changes in ldap_attr and ldap_entry Ansible moduleswhich require Ansible 2.1. Roles that don’t use these Ansible modules should still work correctly on Ansible2.0. [drybjed_]

Removed

• Remove the debops.sshkeys role, which is replaced by debops.authorized_keys_ role. [drybjed_]

• Remove the debops.ifupdown_ role form the common.yml playbook and turn it into a regular service. Thedebops.ifupdown_ role is activated when hosts are placed in the [debops_service_ifupdown] Ansiblehost group. [drybjed_]

7.1. debops-playbooks master - unreleased 25

Page 30: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Remove the debops.subnetwork role, its functionality has been merged with debops.ifupdown_ role.[drybjed_]

Fixed

• The ldap_attr and ldap_entry modules should now work correctly on Ansible 2.1. This is a non-backwards compatible change and it breaks compatibility with Ansible 2.0. [anzil]

Security

• Terminate playbook execution as soon as possible if a vulnerable Ansible version is used. The minimum Ansibleversion without known vulnerabilities is Ansible 2.1.4. The check is run as part of the common.yml playbookfile in a separate playbook which has gather_facts explicitly turned of and the task being delegated to theAnsible controller to avoid possible connection attempts to remote hosts before the check had the opportunityto terminate a vulnerable Ansible instance. This playbook is run with no limitation on remote hosts meaningit will also run the check even if the current Ansible run is limited to a host which is not even managed byDebOps. This check became necessary because some distributions only provide Ansible versions with knownvulnerabilities and some users are unaware. Note that you will need v2.1.5 or v2.2.2 because some requiredfixes for advanced templating features which DebOps uses have not made it into v2.1.4 (broke while fixing theCVEs). Refer to ‘Ansible Security‘_ for details. [ypid_]

debops-playbooks v0.2.9 - 2016-07-07

Added

• Add debops.snmpd_ role dependencies to service/snmpd.yml playbook. [drybjed_]

• Add debops.unattended_upgrades_ role to the common.yml playbook as well as its own service/unattended_upgrades.yml playbook. The support for unattended-upgrades package in de-bops.apt_ role will be removed. [drybjed_]

• Add debops.apt_cacher_ng_ role to it’s own service/apt_cacher_ng.yml playbook. The server-sidesupport for the apt-cacher-ng package in debops.apt_ role has be removed. Client-side support is beingreworked in debops.apt_. [ypid_]

• Add debops.gitlab_runner_ role and playbook. [drybjed_]

• Add debops.logrotate_ role and playbook, available as a standalone service as well as included in the common.yml playbook. [drybjed_]

• Add the debops.apt_install_ role with its own playbook as well as in the common.yml playbook. [drybjed_]

• Add the “role dependency” debops.apt_preferences_ to service/apt.yml. Especially with Debian Stretchenabled in sources.list(5), not running debops.apt_preferences_ together with debops.apt_ could riskinstalling new packages from Stretch. Using apt_preferences__preset_list can avoid that. [ypid_]

• Ensure that the galaxy/requirements* files are up-to-date by making it easy to regenerate them andcheck if they are up-to-date via Travis CI. [ypid_]

• Add debops.preseed_ role dependencies to service/preseed.yml playbook. [ypid_]

• Add back some of the roles that were removed previously from Ansible Galaxy requirements. They are stillpresent in the playbook, and will be removed at a later date. [drybjed_]

• Add debops.resources_ role and its playbook. [drybjed_]

26 Chapter 7. Changelog

Page 31: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Changed

• Update the service/iscsi.yml playbook to use roles that were previously hard dependencies of the de-bops.iscsi_ role. [drybjed_]

• Update the variables related to debops.sshd_ due to change in the variable naming scheme. You might need toupdate your Ansible inventory. [drybjed_]

• Update the service/tinc.yml playbook with support for new debops.tinc_ release. [drybjed_]

• Update the service/mailman.yml playbook with support for new debops.mailman_ release. [drybjed_]

• Moved roles dependencies from debops.subnetwork to subnetwork playbook. [ypid_]

• Update debops.apt_cacher_ng_ playbook and add support for nginx proxy for the cache. [drybjed_]

• Replace app/gitlab_ci*.yml playbooks with debops.gitlab_runner_ playbook. The old GitLab CI andGitLab CI Runner roles will no longer be active by default, since GitLab CI has been merged into GitLab itself.[drybjed_]

• Update the debops.rsyslog_ playbook as well as the common.yml playbook to support the rewritten de-bops.rsyslog_ role. [drybjed_]

• Update the debops.ntp_ playbook and common.yml playbook to use the new variable names. [drybjed_]

• Update debops.mariadb_server_ playbook to include its role dependencies. [drybjed_]

• Update the service/postgresql_server.yml playbook to use new namespaced variables. [drybjed_]

• Update the service/postgresql.yml playbook to use new namespaced variables. [drybjed_]

• Run debops.apt_ earlier in the common.yml playbook to setup things like APT proxy and sources.list(5) for other roles. [ypid_]

• Update service/ruby.yml playbook to use new role variables. [drybjed_]

• Update service/goland.yml playbook to use new role variables. [drybjed_]

• Update Changelog and other documentation to current project standards. [drybjed_]

Removed

• Remove debops.directories_ role and replace it with debops.resources_, which is included in the common.yml playbook. [drybjed_]

Fixed

• Fix compatibility issues of custom lookup plugins in Ansible v2.1+. [abadger]

debops-playbooks v0.2.8 - 2016-02-07

Added

• Add debops.swapfile_ role. [drybjed_]

• Add debops.atd_ role, included in the common.yml playbook by default.

The at and batch commands can be used to schedule delayed jobs using Ansible at module. [drybjed_]

7.3. debops-playbooks v0.2.8 - 2016-02-07 27

Page 32: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Add debops.dhparam_ role, included in the common.yml playbook by default. [drybjed_]

• Add debops.sshd_ configuration variables to debops.apt_preferences_, debops.ferm_ and de-bops.tcpwrappers_ configuration in common playbook. [drybjed_]

• Add set of common “service” playbooks that invoke Ansible roles that are used on all hosts. [drybjed_]

• Add playbook for debops.ntp_ role and update common.yml playbook with debops.ntp_ firewall configura-tion. [drybjed_]

• Add [debops_service_*] host groups to all relevant playbooks and clean up the hosts line in all play-books to use YAML lists instead of Ansible patterns. This should ensure that during transition from colons tocommas there shouldn’t be any issues. [drybjed_]

You can use sed --regexp-extended --in-place '/[debops_service_/! s/^[debops_(.+)]$/[debops_service_1]/' hosts to update your inventory file. [ypid_]

• Add debops.cryptsetup_ role, which can be used to manage filesystems encrypted using LUKS. Role wascreated by Robin Schneider. Thanks! [drybjed_]

• Add debops.apt_preferences_ role to service/postgresql.yml playbook. [drybjed_]

• Add missing ferm configuration for debops.postfix_ role. [drybjed_]

• Add missing playbooks for roles included in common.yml playbook so that they can be easily executed ontheir own. [drybjed_]

Changed

• All playbooks now use become: True instead of sudo: True to enable privileged operation. [dryb-jed_]

• Redesign common playbooks to only work with hosts that are in [debops_all_hosts] inventory group.This should improve support for non-DebOps managed hosts in Ansible inventory, but it requires modificationof existing inventories. [drybjed_]

• Update the Postfix, nginx and slapd playbooks to include the firewall and TCP Wrappers roles. OpenLDAPand nginx Ansible host groups have been renamed, you will need to update the inventory. Postfix playbook isprepared to manage Postfix as standalone service, not part of the common.yml playbook. [drybjed_]

• Move all role playbooks to service/ subdirectory and create symlinks in old locations. [drybjed_]

• Update service/docker.yml playbook and include additional roles required to configure the service prop-erly. [drybjed_]

• Update “Getting Started Guide” and parts of other documentation. [drybjed_]

• Moved roles dependencies from debops.owncloud_ to owncloud playbook. [ypid_]

• Moved roles dependencies from debops.tinc_ to tinc playbook. [le9i0nx_]

• Moved roles dependencies from debops.postgresql_server_ to postgresql_server playbook. [drybjed_]

• Update the common.yml playbook and service/pki.yml playbook with new debops.pki_ role require-ments. [drybjed_]

• Allow execution of debops.pki_ role with role::pki:secret tag so that it will create secret directories butnothing else. [drybjed_]

28 Chapter 7. Changelog

Page 33: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Removed

• Remove the default([]) alternatives from playbooks to make them work correctly on Ansible v1. [dryb-jed_]

• Remove legacy tags from role playbooks. [drybjed_]

• Remove d([]) from common.yml playbook to pass the variables correctly between roles. [drybjed_]

Fixed

• Fix support for no_log: True parameter in ldap_entry Ansible module, so that it works correctly onAnsible v2. [drybjed_]

debops-playbooks v0.2.7 - 2015-10-15

Added

• Add debops.lvm_ role. [drybjed_]

• Add debops.iscsi_ role. [drybjed_]

• Add debops.libvirt_ and debops.libvirtd_ roles. debops.kvm_ role is dropped, due to being replaced bydebops.libvirtd_. Hosts in [debops_kvm] host group will need to be moved to [debops_libvirtd],there might be some variable changes as well. [drybjed_]

• Add a context-based tags to common playbook as an experiment (libvirt(d) roles already use them). Con-text tags are inspired by debtags and will allow more fine-grained control over playbook tasks, when rolesstart to use them internally. Old-style tags will be phased out after some time. [drybjed_]

• Add debops.librenms_ role. [drybjed_]

• Add debops.core_ role as well as core.yml playbook and replace the root.yml playbook. This changemay affect any root_* variables set in the inventory. Check debops.core_ role documentation for new variablenames. [drybjed_]

• Add debops.fcgiwrap_ role. [drybjed_]

• Add new role tags in all playbooks. [drybjed_]

• Add debops.grub_ role, created by Patryk Sciborek (scibi_). Thanks! [drybjed_]

• Add debops.docker_ role. [drybjed_]

• Add globmatch() Ansible filter plugin. Using this filter, you can match strings or lists of strings against ashell glob pattern (a string or a list of patterns). This can be used to easily match one or more strings in a listusing * and ? characters. [drybjed_]

• Add debops.docker_gen_ role. [drybjed_]

• Add ldappassword filter. [scibi_]

• Add debops.postgresql_server_ role. [drybjed_]

7.4. debops-playbooks v0.2.7 - 2015-10-15 29

Page 34: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Changed

• Hosts in [debops_no_common] host group will no longer run a common playbook. [drybjed_]

• Lookup plugins task_src, template_src and file_src are updated using input from James Cammaratato work both in old Ansible 1.x series as well as in the new 2.x series. Thanks! [drybjed_]

• Split the environments.yml playbook into smaller plays included in the main playbook to see if this modelhas any issues. This change should make user of specific role plays easier on the command line and from otherplaybooks. [drybjed_]

• Split virtualization.yml playbook into separate plays. [drybjed_]

• All playbooks have been split into small plays. Playbook directories have shorter names, which are easier to usefrom the command line. [drybjed_]

Removed

• Remove root.yml playbook and its additional files - its functionality has been moved to debops.core_ Ansi-ble role. [drybjed_]

debops-playbooks v0.2.6 - 2015-07-14

Added

• Add debops.fail2ban_ role. [drybjed_]

• Add debops.preseed_ role. [drybjed_]

• Add debops.ipxe_ role. [drybjed_]

• Add debops.tftpd_ role. [drybjed_]

• Add debops.tgt_ role. [drybjed_]

• Add variables to set admin home directory group and permissions in bootstrap.yml playbook. [drybjed_]

• Add debops.rstudio_server_ role. [drybjed_]

• Add debops.tinc_ role. [drybjed_]

• Add ansible_local.timezone fact which returns currently set timezone in /etc/timezone. Factprovided by Ansible itself in ansible_date_time.tz is not suitable to use in application configurationfiles. [drybjed_]

• Add debops.snmpd_ role. [drybjed_]

• Add debops.memcached_ role. [drybjed_]

• Add MariaDB server and client roles. [drybjed_]

Changed

• Ansible will now try and read the remote host UUID using dmidecode and prefer that over using a randomlygenerated UUID if possible. This works on hardware hosts and virtual machines, but shouldn’t in containers.[drybjed_]

30 Chapter 7. Changelog

Page 35: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• During root.yml playbook, grab only the last line of dmidecode output in case that it decides to emitcomments about not supporting older releases in STDOUT. [drybjed_]

• Update of the bootstrap.yml playbook; there are now more variables that define the administrator account,admin account will be now a “system” account by default (UID < 1000). Playbook checks if an account with agiven name already exists and does not change its parameters if it does. Admin account will be in more groupsby default (admins (passwordless sudo access), staff and adm). [drybjed_]

• Move the Changelog references to the end of the file and remove duplicates, so that Sphinx does not complainabout them. [drybjed_]

• Replace old headers in Changelog to use current header order. [drybjed_]

• Move relevant documentation to debops-playbooks repository. [drybjed_]

Removed

• Remove debops.ansible_ role from requirements, you should switch to creating an ansible Debian packageand installing it on remote servers using local APT repository. [drybjed_]

• Remove debops.encfs_ role from requirements, it’s not used anymore and is ill designed to be used on serversat this point. [drybjed_]

• Remove debops.safekeep_ role from requirements, SafeKeep is not in official Debian repositories thereforeinstallation requires manual steps, debops.rsnapshot_ is a better alternative. [drybjed_]

• Remove debops.debug_ role from requirements, tools/debug.yml playbook should be a better alternativeand it’s easier to use. [drybjed_]

• Convert bootstrap.yml playbook to an Ansible role. [drybjed_]

debops-playbooks v0.2.5 - 2015-04-01

Added

• Add debops.dokuwiki_ role. [drybjed_]

• Add a “testing channel” Galaxy requirements file, to be used to download Ansible roles with “testing” branchinstead of “master”. [drybjed_]

• Reto Gantenbein created a ‘Dovecot‘_ role which has been added to the DebOps project. Thanks! de-bops.dovecot_ can be used to manage IMAP/POP3 service which will let you access your mail remotely over asecure connection. [ganto_, drybjed_]

debops-playbooks v0.2.4 - 2015-03-26

Added

• Add separate “root fact” directory where applications are installed, by default the same as the path for servicehome directories. [drybjed_]

• Install python-pip during bootstrapping. [htgoebel_]

• Add a way to install custom packages during bootstrapping. [drybjed_]

• Add debops.stunnel_ role. [drybjed_]

7.6. debops-playbooks v0.2.5 - 2015-04-01 31

Page 36: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Changed

• Reorder networking.yml playbook to run network-related roles before main services and applications. Thisshould make sure that networking is correctly set up when it’s needed. [drybjed_]

debops-playbooks v0.2.3 - 2015-03-05

Added

• Added new lookup plugins, file_src and template_src which allow custom template and file searchpaths in roles. [rchady]

• You can set global “root flags” on hosts using root.yml playbook. Ansible roles can check for their presenceor absence and automatically change their behavior. [drybjed_]

Changed

• Roles in common.yml playbook are rearranged to better support LDAP integration and avoid possible SSHlockdown if host was not prepared using bootstrap.yml playbook or preseeding. [drybjed_]

• Scripts which provide custom facts will be installed on the first run of the root.yml playbook. First such scriptprovides a list of currently enabled Linux capabilities, in ansible_local.cap12s fact tree. [htgoebel_,drybjed_]

• bootstrap.yml playbook will check if it can change the hostname before doing it using Linux capabilities.[htgoebel_, drybjed_]

debops-playbooks v0.2.2 - 2015-02-25

Added

• Add support for STARTTLS in ldap_attr and ldap_entry modules [psagers]

Changed

• Fix issue with ldap_entry not handling no_log: True in argument list properly [drybjed_]

debops-playbooks v0.2.1 - 2015-02-24

Changed

• Move library/ directory into correct place and sort modules in subdirectories mirroring the official layout.[drybjed_]

32 Chapter 7. Changelog

Page 37: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

debops-playbooks v0.2.0 - 2015-02-22

Added

• New role: debops.rsnapshot_ [drybjed_]

Changed

• Variables from bootstrap.yml playbook can now be customized using inventory. [drybjed_]

• Bootstrap variable names have been changed to be similar to what is used in other DebOps roles. Variable thatspecifies SSH key to install is now a normal Ansible list. [drybjed_]

debops-playbooks v0.1.0 - 2015-02-16

Added

• New paths have been added to root.yml service paths. [drybjed_]

Changed

• Format of the Changelog is modified to reflect new versioning. Old entries are preserved. [drybjed_]

• ansible_local.root.home default path has been changed from /var/lib to /var/local to movehome directories out of the way of the system packages. [drybjed_]

• root.yml service paths that are already configured on remote host as facts will override playbook or inventorychanges to protect already installed services from future changes. [drybjed_]

2015-02-12

Playbook updates

Due to practical reasons, role updates will be written in roles themselves from now on, in CHANGES.rst files.

New “root variable” has been added to root.yml playbook, ansible_local.root.uuid. It will contain arandom UUID generated on first DebOps run. It can be used to uniquely identify an instance of a particular host.

2015-02-06

Role updates

OpenLDAP server managed by debops.slapd_ role has gained support for TLS out of the box, using certificatesmanaged by debops.pki_ role. By default, slapd server listens for normal plain text connections, which can beprotected by the client requesting a StartTLS session, as well as for encrypted SSL/TLS connections. This also marksthe removal of Beta status from debops.slapd_ role.

7.11. debops-playbooks v0.2.0 - 2015-02-22 33

Page 38: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

To stay on the safe side, debops.auth_ role, which configures /etc/ldap/ldap.conf, will automatically setencrypted connections to OpenLDAP server using ldaps:// protocol. You can of course change that using roledefault variables.

Playbook updates

To make LDAP use easier within Ansible playbooks, I’ve included two Ansible LDAP modules created by PeterSagerson in the main DebOps playbook library/ directory, which makes them available anywhere within DebOpsproject directories (in playbooks and roles). You can use ldap_entry and ldap_attr modules to manipulateyour LDAP database, look in each module source code for examples.

2015-02-05

Role updates

debops.mysql_ role can now configure a MySQL server with SSL support enabled by default, using PKI infrastructuremanaged by debops.pki_ role.

debops.nginx_ role gained support for setting server-wide (as in, per domain) allow/deny rules, which is moresecure than just per-location (which was available previously). You can use Ansible lists to specify which hosts ornetworks have access to the server.

You can now configure HTTP Basic Authentication in debops.nginx_ role. It works on a server level (restrictedaccess to individual servers), as well as on the host level (restricted access to all nginx servers configured on this host).debops.nginx_ has a built-in support for htpasswd files - you specify a list of user accounts to configure in Ansibleinventory, and passwords themselves are stored in secret/ directory, managed by debops.secret_ role.

2015-02-04

Role updates

I have found out that some applications do not support SSL/TLS certificate chains correctly. Because of that, I haveadded a separate PKI realm, /etc/pki/service/, with corresponding Root Certificate Authority, which will signcertificates directly. It is meant for internal use only, each host in a cluster has its own certificate shared by all serviceson this host, private key is accessible for users belonging to ssl-cert system group.

For reference, Debian Bug #630625 which indicates that MySQL does not support certificate chains out of the box. Ifother such services are found, they will now use service PKI realm by default.

2015-02-03

Role updates

debops.nginx_ role will now track HTTP and HTTPS default_server configuration option separately, whichshould make it even more roboust and hard to break accidentally. Code which selected default_server wasmoved out of the server template and into separate Ansible tasks.

Nginx role has exposed two variables using local Ansible facts:

34 Chapter 7. Changelog

Page 39: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• ansible_local.nginx.user is the default system user (www-data) which is used to run the webserver.Some of the roles need to give read-only or read-write access to his user for specific files. To have it workproperly, debops.nginx_ role needs to be run before your own role, or you need to have it in your role’s depen-dencies.

• ansible_local.nginx.www is the default directory for web-accessible files (/srv/www). Most of thetime you will use it by creating separate subdirectory for a specific system user. Nginx role uses a specificstructure based on this path to automatically generate root configuration parameters;

Playbook updates

New playbook, tools/dist-upgrade.yml has been added. It should help with upgrading to next version of yourfavorite OS, currently supported upgrade paths are from Debian Wheezy to Debian Jessie and from Ubuntu Trusty toUbuntu Utopic.

To use the new playbook on a selected host, run command:

debops tools/dist-upgrade --limit hostname

Playbook is idempotent and it shouldn’t perform an upgrade on already upgraded hosts. After an upgrade is performedyou should receive email message with the log of the procedure for review. After that you might want to re-run at leastDebOps common playbook to make sure that any changes are accounted for and reboot the host.

Just a reminder, that at this time Debian Jessie is still a Testing distribution and you shouldn’t run the upgrade playbookon your production systems, unless you know what you are doing. DebOps playbooks and roles should work correctlyinstalled on either Wheezy or Jessie (if not, post an issue), but they are not tested against an upgrade from onedistribution to another.

I’ve created a separate dist-upgrade label for issues related to upgrade procedure. You should check it out beforeupgrading. If you find any issues regarding DebOps roles after performing an upgrade, please post them in ‘DebOpsPlaybooks‘_ repository so that they can be tracked in one place.

2015-02-01

Role updates

Small updates in debops.pki_ role:

• previously Diffie-Hellman parameter regeneration meant that on each Ansible run contents of /etc/pki/directory would change. Because role creates a snapshot of /etc/pki/ directory on any changes and sends itto Ansible Controller, if you keep your inventory and secrets in a git repository, it meant that your repositorywould constantly grow. Now debops.pki_ role will archive DH parameter files only the first time the snapshotfile is created; subsequent snapshots will ignore them, and thus no changes will be recorded and snapshot filewill not need to be archived, unless something else changes, for example certificates are added or updated.

• you can now disable or change the frequency of Diffie-Hellman parameter regeneration using inventory vari-ables. Default frequency has been changed from daily to weekly.

7.17. 2015-02-01 35

Page 40: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

2015-01-31

Playbook updates

New playbook, root.yml has been added and part of the common.yml playbook has been moved there. Thisplaybook is meant to prepare the system for the rest of the DebOps roles by creating a set of base directories:

• a root directory for service home directories, by default /var/lib

• a root directory for local data managed by the host, /srv

• a root directory for backups, both automated and manual, /var/backups

Paths to these directories are saved in Ansible local facts. Other DebOps roles can then access them usingansible_local.root hierarchy, for example:

role_home: '{{ ansible_local.root.home + "/role" }}'role_data: '{{ ansible_local.root.data + "/role" }}'role_backup: '{{ ansible_local.root.backup + "/role" }}'

Because of the way that Ansible manages dict variables, ansible_local.root.* local facts will be requiredon all hosts managed by DebOps playbooks and roles - otherwise you need to specifically check for existence ofansible_local and ansible_local.root variables before using them to avoid errors about missing vari-ables.

If you use DebOps playbooks, this should be handled for you automatically. If you use DebOps roles separately,you can add an include of root.yml playbook to your set of playbooks and these facts should be created for youautomatically. root.yml does not need to be included in all your playbooks, just in the first one at the beginning.

At the moment those variables are not used in any DebOps roles, that will change over time after a period of testing.

2015-01-28

Role updates

debops.reprepro_ role is no longer a dependency of debops.apt_. Instead it’s configured like any other service,by adding a host to [debops_reprepro] host group. This allows you to create separate hosts with differentrepositories if needed.

Default configuration of debops.reprepro_ role has 3 repositories:

• a backport repository configured for your installed release (for example on Debian Wheezy it will managepackages for wheezy-backports). You can upload to this repository directly;

• a “staging” repository for your organization, <release>-<domain>-staging. You can upload to thisrepository directly;

• a “production” repository for your organization, <release>-<domain>-prod, this repository is currentlymanaged manually from the reprepro user account. You can promote packages to it from -staging repos-itory using reprepro pull command;

You can also enable mirrors of selected distributions as needed, which allows you to use local APT mirror as a bufferbetween official repositories and your servers, if you need it. To upload packages to repositories you can use dputcommand to upload *.changes files over HTTPS.

debops.reprepro_ role automatically manages its GnuPG repository keys and makes snapshots of current keyringstate which are then uploaded to Ansible Controller’s secret/ directory. In case of a reinstall, role will reusealready existing GnuPG keys if they are found on Ansible Controller.

36 Chapter 7. Changelog

Page 41: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

There are many more configuration options prepared in debops.reprepro_, I suggest that you read its defaults/main.yml file to see what’s available.

Because of above changes, you need to separately add your local repositories in debops.apt_ configuration variables.To make it easier, there is now a separate list variable for APT key definitions (apt_keys, as well as a way toadd APT keys and repositories in a “delayed” way - instead of configuring your own repository immediately on firstinstall, which could result in an error if repository is not yet set up, you can add configuration in separate set ofapt_{keys,sources}_delayed variables which will be used only after debops.apt_ role had configured a hostonce.

Another small change in debops.apt_ is modification of conditional package installations - instead of separate aptmodule calls, packages are enabled dynamically during Ansible run using set_fact module. debops.apt_ willnow also correctly distinguish Debian and Ubuntu firmware packages which are named differently between those twodistributions.

2015-01-21

Role updates

Webserver status page has been enabled by default in debops.nginx_, it’s accessible on /nginx_status location,initially only from localhost addresses (from the webserver itself). You can add additional IP addresses or CIDRranges using separate list, nginx_status.

Fix for CVE-2013-4547 has been removed from the server template, since the issue has already been mitigated inDebian.

2015-01-20

Role updates

debops.gitlab_ci_ role has been updated to support ‘GitLab CI‘_ 5.4, with GitLab 7.7 providing authorization basedon OAuth. Due to the changes in GitLab CI itself, some configuration variables have been changed - check the roledefaults for new ones (mainly, you can define only 1 GitLab instance to connect to).

debops.users_ role has been slightly cleaned up and root-proofed - it shouldn’t make an error if you are connectingto your hosts directly as root account anymore. Role uses default(omit) filter in its tasks, which means thatDebOps now requires Ansible >= 1.8 for correct operation.

New playbook plugins

Hartmut Goebel created a small lookup plugin, with_lists, which allows you to use lists of items as “items”themselves, see an example in the with_lists plugin. Thanks!

2015-01-18

Role updates

debops.gitlab_ role has been finally rewritten. Lots of important changes:

• support for ‘GitLab‘_ 7.7 out of the box, even before official release ;)

7.20. 2015-01-21 37

Page 42: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• new home directory, /var/local/git/, you might want to reinstall your GitLab instance from scratch ortake care with moving your old instance files to new location;

• role does not depend on configuration file hashes anymore, updates should be much easier to perform andsupport for new versions should be included in a more timely manner;

• debops.gitlab_ will configure a daily backup of the application data to /var/backups/gitlab/, backupfiles older than a week should be automatically cleaned up;

• new GitLab install uses a random password stored in the DebOps secret/ directory instead of the officialpassword. Default admin account will have an email address in your domain instead of [email protected], so random bounced mails shouldn’t be a problem anymore;

Playbook updates

bootstrap.yml playbook gained new tasks which can be used to set hostname and domain on a given host. Youcan define bootstrap_hostname or bootstrap_domain variables in inventory and Ansible will try to enforcethese settings on a given host as well as in /etc/hosts. This functionality makes the tools/fqdn.yml playbookredundant, so it’s removed.

2015-01-13

Happy New Year 2015!

PKI rewrite

I’ve worked on debops.pki_ role since December, holiday season delayed it slightly, but finally it is here. :-)

New PKI infrastructure in DebOps is designed around creating and managing Certificate Authorities on the AnsibleController, inside secret/ directory managed by debops.secret_, signing Certificate Requests generated by remotehosts and sending back certificates. There’s 1 Root CA certificate you need to import into your browser or hostcertificate store and after that, all other servers should show up in your browser as accepted automatically.

You can also very easily copy your own certificates signed by an external CA, with private keys if needed, to yourservers using a set of directories in the secret/ directory.

Several roles which depended on the old debops.pki_ role have been now updated as well and take advantage offunctionality present in the new PKI infrastructure. These roles are:

• debops.nginx_

• debops.postfix_

• debops.postgresql_

• debops.boxbackup_

If you use any of these roles in your infrastructure, take care to make sure that your certificates are moved into newdirectory structure and configuration is updated as needed.

If there are any questions regarding new PKI and how to use it, feel free to ask them on the IRC channel or on themailing list.

38 Chapter 7. Changelog

Page 43: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

2014-12-23

Role updates

debops.users_ role can now set or update user passwords (by default no passwords are set).

debops.ntp_ role has gained support for ntpd daemon, thanks to ‘RedRampage‘_. Because of the issues withrole dependency variables and Jinja, access to NTP service through firewall is now controlled by a separate variable,ntp_firewall_access. By default, remote access is disabled.

2014-12-05

New roles

• debops.salt_ role allows you to install and configure ‘Salt‘_ Master service. You can use this to create Saltcontrol host to which other hosts (Salt Minions) can connect to. At the moment configuration is very basic, Saltmaster will automatically listen to IPv6 connections and firewall will be configured to accept connections ondefault ports.

Role updates

Salt Minion preseeding has been added in debops.apt_ (current Debian Preseed configuration is there, will be movedin the future to separate role), debops.lxc_ and debops.openvz_ roles. Automatic minion installation is disabled bydefault and can be enabled separately for each “mode” - Debian Preseed postinst script in case of physical hosts orKVM virtual machines, LXC template script for LXC containers, OpenVZ bootstrap script for OpenVZ containers.After installation, salt-minion will try to connect to salt host, so make sure that it’s present in your DNSconfiguration for best results.

2014-12-03

Role updates

Continuing the ‘GitLab‘_ revamp, debops.gitlab_ci_runner_ role has also been refactored and is unfortunately notcompatible with the previous version, reinstall of the nost is recommended.

Runner home directory has been moved to /var/local/ directory, most of role dependencies have been droppedand role now needs less upkeep than before. You can read about changes in latest commit.

2014-12-02

‘DebOps mailing list‘_ has been moved to groups.io_.

Role updates

debops.gitlab_ci_ role has been significantly refactored. Due to bug in GitLab CI 5.0 at the moment this versioncannot be installed, so I decided to use this opportunity to make some deep changes in the role. GitLab CI home has

7.24. 2014-12-23 39

Page 44: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

been moved to /var/local/gitlab-ci/ directory, and various tasks related to updating the application havebeen streamlined. You can read more information about various changes in the commit message.

2014-12-01

Hartmut Goebel has joined DebOps team and wrote an excellent guide for using DebOps scripts and playbooks withVagrant on single and multiple hosts. It’s available in ‘DebOps Examples‘_ repository.

Role updates

All DebOps roles again use Ansible devel branch on Travis CI for tests.

debops.debops_ role has been rewritten and updated to support current project installation method. By default onlyDebOps scripts will be installed system wide, but you can also install playbooks and roles to /usr/local by setting avariable. Dependency on debops.ansible_ role has been dropped and that role will be removed in the future. You caninstall Ansible from a Debian repository or by providing your own .deb package.

‘RedRampage‘_ has provided a failover code for debops.dhcpd_ role which should help set up failover DHCPservers. Thanks!

Several DebOps roles had a small fixes related to ansible-playbook --check command, which can now beused to check for possible changes before applying them on the remote hosts. Due to bugs in older Ansible versionsthis functionality works correctly on Ansible 1.8+ or current devel branch.

2014-11-27

Role updates

Support for management of SSH host fingerprints in /etc/ssh/ssh_known_hosts (via debops.sshd_ role) and/root/.ssh/known_hosts on OpenVZ hosts (via debops.openvz_ role) has been redesigned and no longer usesassemble Ansible module. Instead, Ansible checks already present fingerprints and adds new ones if they are notpresent in the files. This helps better obfuscate scanned hosts, which previously could be inferred from filenames ofparts assembled earlier.

Instances of with_items using multiple lists in a few roles have been replaced with with_flattened whichworks better in new release of Ansible, 1.8+.

debops.openvz_ role has been slightly updated and redundant configuration of ferm and sysctl, already configuredby debops.ferm_ role, has been dropped to prevent duplication.

2014-11-26

Role updates

debops.nginx_ role will now preserve the status for default_server of a particular configuration file in case thatanother instance of the role is added in the Ansible run. Saved local fact about which server is the default one will takeprecedence over automatically calculated setting.

If nginx role notices that Ansible local facts are missing, it will remove all files and symlinks from /etc/nginx/sites-enabled/ directory. This should happen in two instances - either nginx is configured for the first time, or

40 Chapter 7. Changelog

Page 45: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

/etc/ansible/facts.d/nginx.fact file has been removed. In that case all active config symlinks will beremoved to prevent accidental errors from some old, not regenerated configuration files.

2014-11-25

New roles

• debops.hwraid_ is a role that configures access to ‘HWRaid‘_ package repository and installs packages forrecognized RAID storage arrays connected to your hosts. It can be used to quickly and easily setup basicmonitoring for your storage - many packages contain automated scripts which send mail to root account incase of issues with RAID.

Role updates

debops.auth_ role will now manage /etc/ldap/ldap.conf configuration file. By default, LDAP server on localdomain is set up (currently without any encryption, so treat this as experimental feature and don’t use it in production)with local domain specified as BaseDN. you can change this in role default variables.

DebOps will automatically configure hidepid=2 option in /proc filesystem on selected hosts (hardware serversand fully virtualized VMs), using debops.console_ role. This functionality hides other users’ process information forunprivileged accounts. A separate system group, procadmins has been reserved for monitoring services and usersthat need full access to the /proc filesystem.

2014-11-24

New roles

• debops.slapd_ role manages OpenLDAP server, slapd. At the moment role is in beta stage - currently there isno SSL encryption available, no backup/restore scripts and no replication. But role installs a few useful scriptsand slapd management is done using custom Ansible modules. Deeper integration between OpenLDAP andother DebOps services will be created in the future.

Role updates

Because of recent changes in the debops.tcpwrappers_ role I decided to make the ferm rules for SSH access morestrict. From now on, iptables will check new SSH connections over period of 1 hour, if more than 3 new connec-tions from 1 IP address are attempted during that time, and address is not in the whitelist, it will be blocked for 2 hours,with each new connection attempt resetting the timer. All this is now configurable in debops.sshd_ and debops.ferm_roles.

Thanks to htgoebel’s suggestion I was able to refactor Postfix hash tables management. They are now generated fromall *.in files in current directory, which means that other Ansible roles or even other scripts can put their own filesin /etc/postfix/hash_*/ directories and if they are named with *.in extension, their corresponding *.dbfiles will be created automatically. Thanks to that, debops.postfix_ role now generates tables from templates usingwith_fileglob instead of static lists of templates, which makes the process of adding new tables if necessarymuch easier.

7.31. 2014-11-25 41

Page 46: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

2014-11-22

Role updates

You can now specify default value for entries in debops.tcpwrappers_ role, using item.default key. If this keyis specified, and item.clients is not present or is empty, default value will be used instead. Specify 'ALL' toallow connections from any host.

Consequently, debops.sshd_ role now will allow connections from any host by default in /etc/hosts.allow. Ifyou previously used a list of hosts using sshd_*_allow, your configuration shouldn’t be affected.

2014-11-20

Role updates

debops.ifupdown_ will now check if previous network configuration in /etc/network/interfaces was usingstatic IP addresses, which indicates that DHCP is not available on the network. In that case, a basic static IPv4interface configuration will be used with information gathered by Ansible to setup a default network interface. Thisshould prevent sudden loss of communication in cases where hosts are configured statically.

Playbook updates

tools/hostname.yml playbook has been renamed to tools/fqdn.yml and can get the new hostname anddomain from fqdn variable defined in inventory, which is less awkward to use than renaming the host in inventoryfile directly.

2014-11-19

Role updates

Network forwarding configuration in iptables has been moved from debops.kvm_, debops.lxc_ and debops.subnetwork roles into debops.ferm_ to avoid duplication. This will also result in forwarded network interfacesbeing able to accept Router Advertisements and configure their IPv6 addresses using SLAAC. In short, easier networkconfiguration.

Hartmut Goebel has provided a set of ‘Raspbian‘_ APT repositories for debops.apt_ role, thanks! Unfortunately, atthe moment Ansible does not correctly recognize Raspian as a separate distribution which prevents automatic sourceselection, but there are workarounds.

Because of the recent Debian Jessie freeze, DebOps project is starting preparations for full Jessie support, both as astandalone install, as well as an upgrade from Wheezy.

All debops.ferm_ configuration files had changed ownership from root:root to root:adm which is the defaultin Debian. This change should prevent back-and-forth changes of ownership after system has been upgraded, whichforces ferm files to change ownership to root:adm.

Some APT configuration files in debops.apt_ role have been renamed to avoid conflicts with existing files during theupgrade, this should prevent debconf questions about replacing modified configuration files.

42 Chapter 7. Changelog

Page 47: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Both debops.apt_ and debops.lxc_ roles now support ansible_distribution_release in 'release/sidformat, which lets DebOps function correctly on Jessie during the freeze. There might be other roles which need to beupdated to support this syntax, they will be fixed later.

debops.auth_ role now uses full templates instead of lineinfile module to configure sudo and su admin access.This should prevent debconf asking about modifications in /etc/pam.d/su (which is now diverted), and letssudo have more configuration options for admins group.

Playbook updates

New playbook, tools/hostname.yml can be used to change the hostname and FQDN of a host to those definedin Ansible inventory (and yes, you can do multiple hosts at once). It’s advised to not do it after services have beenconfigured, since some of them may rely on the correct FQDN defined in DNS. If you use DHCP to automaticallyconfigure DNS (for example with dnsmasq, rebooting the host after changing the hostname should ensure that thenew FQDN is correct.

2014-11-13

Role updates

debops.postfix_ role will now correctly work on hosts without FQDN configured. On these hosts, Postfix will au-tomatically override its configured capabilities and enable local mail delivery, mail will be originating from the hostinstead of the domain. Postfix role will also no longer modify /etc/hosts to rewrite IPv6 localhost address, itseems that the annoying warning in the mail log about unknown connection source has been fixed.

debops.dnsmasq_ role has been completely rewritten and now supports multiple network interfaces and IPv6, amongother things. It requires ipaddr() filter plugin to work, but thanks to that it can automatically configure servicesbased on IP addresses configured on specified interface - no more separate IP subnet configuration is needed. Rolenow also creates more fine-grained CNAME records and has more configuration options. And it’s out of beta! :-)

Playbook updates

Old ‘debops.nat’ role has been obsoleted by debops.subnetwork and removed from ansible-galaxy re-quirements file. It will also be removed from GitHub and Ansible Galaxy in the future. Also, debops.radvd_ hasbeen added to the requirements.

Virtualization playbook has been modified and roles that previously automatically configured internal network andDNS services have been removed from KVM and LXC plays (yes, this will change installation procedures in the docs,which are not yet updated). New playbook, ‘networking.yml’ has been added where you will find all network-relatedplays, like subnet creation and management (via debops.subnetwork and DHCP/DNS management.

2014-11-07

New roles

debops.subnetwork is a replacement for old debops.nat_ role, with many improvements. You can create abridge interface with local network behind it for virtual machines, or even switch to a real Ethernet interface for yourphysical hosts. You can create both an IPv4 network, which will be automatically configured behind NAT, and an IPv6network (with multiple prefixes). debops.subnetwork is not yet part of the main playbook, it will replace the oldNAT role when dnsmasq role is updated to support it.

7.36. 2014-11-13 43

Page 48: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Role updates

Because of the changes related to new networking, some code in debops.lxc_, debops.kvm_ and debops.nat_ had tobe moved around. Specifically, parts of the firewall and sysctl settings related to the LAN interface were moved intodebops.subnetwork role and parts of the forwarding configuration to external and internal networks were addedrespectively to LXC and KVM roles.

2014-11-05

New playbooks

New playbook has been added, net/ipv6/6to4.yml. This playbook configures ‘6to4 tunnel‘_ interface on ahost with public IPv4 address and allows you to easily connect to IPv6 network. To do that, you need to put a host in[debops_6to4] group. Afterwards, you can run the playbook using debops script:

debops net/ipv6/6to4 -l host

This is first step towards transition to playbooks placed in subdirectories. These playbooks will probably work cor-rectly only with debops script, which automatically generates ansible.cfg with correct configuration parame-ters. To use these playbooks standalone, you will need to create your own ansible.cfg and include in it paths toDebOps roles and plugins.

Role updates

You can now configure custom ‘ferm‘_ rules using a custom template in debops.ferm_. New ferm_*_rulesvariables allow you to create rules in /etc/ferm/ferm.d/ directory which can configure tables and chains otherthan INPUT.

2014-11-04

New roles

Finally, it’s time to start bringing out new toys. :-) For starters, debops.radvd_ role, which installs and lets youconfigure radvd, IPv6 Router Advertisement daemon. It will be used in future IPv6 router roles.

Playbook updates

ipaddr() filter has been rewritten again and it works now correctly with lists of values. Filter was completelyrefactored internally and its output should be now consistent with expectations. Hopefully for the last time.

2014-11-02

Playbook updates

More fixes in filters! split() filter will now handle incorrect input values gracefully and return them in a list, sinceoutput is usually expected to be a list. If a string cannot be split by specified separator, whole string will be returnedin a list.

44 Chapter 7. Changelog

Page 49: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

ipaddr('6to4') filter has been updated to not convert private IPv4 addresses, since their behavior is unspecified,this way Ansible can easily determine if a given IPv4 address can be used in 6to4 tunnel.

6to4 query will also now return proper ::/48 subnet instead of a single IPv6 address, this way a subnet can befurther manipulated to for example split it into smaller ::/64 subnets.

New ipaddr() query type has been added - you can now specify positive or negative numbers in a query, forexample {{ '192.168.0.1/24' | ipaddr('-1') }} will return last IPv4 address from a specified subnet.It’s an easy way to define DHCP dynamic ranges in dnsmasq configuration.

New filter, ipsubnet() has been added. It lets you manipulate IPv4 and IPv6 subnets; given a subnet and CIDRprefix you can check the number of subnets that it can be divided into, adding an index number to the query lets youget a specific subnet. You can also check the biggest subnet an address can be in by specifying the smallest prefixyou’re interested in.

You can now pass a list to ipaddr() filter and it will return only items that pass specified criteria, for example returnsonly list of IP addresses and subnets by default, or only IPv6 addresses and subnets, etc. It’s not yet 100% correct allthe time and not all queries work (or make sense in this context).

2014-10-31

Playbook updates

New filter, split() has been added into filter plugins. It lets you split strings into a list on a specified separator (bydefault, space). I’m amazed it hasn’t been included yet in core Ansible. :-) split() filter has been written by TimRaasveld and is included with his blessing, thanks!

ipaddr() filter will from now on correctly handle false values like False and "" by returning False whenencountered. It also gained new query type, '6to4' which lets you convert public IPv4 addresses into ‘6to4‘_ IPv6addresses or check if a specified IPv6 address/network is in 2002::/16 address range.

2014-10-28

Role updates

APT repository management in debops.apt_ role has been rewritten. Now role supports multiple APT mirrors, aswell as custom lists of repositories dependent on the current distribution (repository lists for Debian and Ubuntuare included). Configuration of default APT repositories has been moved from a separate config file in /etc/apt/sources.list.d/ directly to /etc/apt/sources.list, original configuration file is preserved us-ing dpkg-divert. Additionally, if debops.apt_ cannot recognize current distribution, it won’t modify the defaultsources.list file, this can also be enforced manually if needed.

2014-10-17

Role updates

Many more roles have now partial or full tests on Travis-CI, more to come.

Default version of ‘Etherpad‘_ installed by debops.etherpad_ role has been changed from 1.4.0 to develop,because current stable release does not recognize new npm installed in Debian. It will be switched to the next stablerelease when it’s available.

7.41. 2014-10-31 45

Page 50: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Because of the recent IPv6 changes in debops.nginx_, management of nginx configuration and daemon had to bechanged slightly. Role will try to automatically pick a sane server as the “default server”, if none are marked as one,due to ipv6only=off parameter tied to default_server parameter. Another added functionality is full nginxserver restart when configuration symlinks in /etc/nginx/sites-enabled/ directory are added or removed -this should help with requirement to restart the service on interface changes.

Default admin username and SSH keys are now exposed as defaults/ variables in debops.openvz_ role; SSH keysare also sourced from ssh-agent instead of directly from the ~/.ssh/id_rsa.pub file.

2014-10-10

Playbook updates

Maciej Delmanowski wrote a set of custom filter plugins for Ansible which let you manipulate IPv4 and IPv6 ad-dresses. You can test if a string is a valid IP address or convert them between various formats.

2014-10-09

Role updates

IPv6 firewall has been enabled by default in debops.ferm_ after all roles that configure ferm directly had theirconfiguration files fixed to support both iptables and ip6tables commands.

debops.boxbackup_ has been finally converted from a “common” role (run from common.yml playbook) to a group-based role. First host in debops_boxbackup will be configured as the BoxBackup server and the rest will be setup as its clients.

2014-10-07

Role updates

debops.ferm_ role is now IPv6-aware and can generate rules for iptables and ip6tables at the same time. Theway you use the role as a dependency hasn’t changed at all, so if you use dependent variables in your roles, you shouldbe fine. However, because some roles are managing their firewall rules by themselves, IPv6 support is disabled bydefault - this will change when all roles are updated to be IPv6-aware.

debops.nginx_ also gained support for IPv6 and will now listen for connections on both types of networks by default.If you have an already running nginx server, it will require manual restart for the new configuration to take effect.

2014-10-05

All role README files have been converted to reStructuredText format. Unfortunately, ‘Ansible Galaxy‘_ does notsupport README.rst files at this time, so role information cannot be updated there.

46 Chapter 7. Changelog

Page 51: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

2014-10-02

Role updates

debops.nginx_ role has been updated. Most changes are either cleanup (change names of some internal role files,remove unused redundant variables, etc.).

/etc/nginx/http-default.d/ directory has been renamed to /etc/nginx/site-default.d/ whichhopefully better shows the purpose of this directory in relation to nginx server configuration. Old directories haven’tbeen removed; if you use it, you will need to move the configuration files manually.

Support for map { } configuration sections has been added. It works similarly to upstreams and servers, that meansyou can define your maps in hashes and enable them using nginx_maps list. More information about ‘nginx mapmodule‘_ can be found at the nginx website.

You can now remove configuration of servers, upstreams and maps from hosts by adding delete: True to theconfiguration hashes.

Old remnants of the fastcgi_params configuration files are now automatically removed by the nginx role. Thisis the second step of the switch from custom to stock configuration file. Task which removes these old files will beremoved in the future.

2014-09-29

“{{ lookup(‘file’,’~/.ssh/id_rsa.pub) }}” considered harmful

The lookup above is common through Ansible playbooks and examples, and it is used as a prime method of accessingSSH public keys of current account on Ansible Controller host to, for example, install them on remote hosts usingauthorized_key Ansible module.

However, this is by no means a portable solution. Users can have public SSH key files with completely different names,or don’t even have them at all and instead use other means of SSH authentication, like GPG keys or smartcards.

Because of that, I’m changing the way that SSH public keys will be accessed by default in DebOps. For now, onlyplaybooks/bootstrap.yml playbook will be updated (this playbook is used to bootstrap new hosts and getthem ready for Ansible management), changes in other roles will come later. I hope that authors of other roles willfollow suit.

New way of accessing SSH keys will use SSH agent (or its alternatives): instead of accessing the keys directly,Ansible will request a list of currently enabled public keys from the SSH agent using "{{ lookup('pipe','ssh-add -L') }}" lookup. Because that lookup can return an empty value which will not create an error, youwant to safeguard against that in a key configuration task using failed_when: condition. Look in playbooks/bootstrap.yml to see how it’s used with authorized_key task.

2014-09-22

inventory.secret is renamed to secret

If you use DebOps, or at least some roles from it, you probably are familiar with debops.secret_ role, which makeshandling sensitive and confidential data easier within Ansible playbooks and roles. I’m mentioning this becausesecret variable is used through the DebOps project and this change will be significant - that’s why I want to do itright away instead of changing the role suddenly some time down the line.

7.48. 2014-10-02 47

Page 52: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

Previously debops.secret_ role created directory for secrets adjacent to the Ansible inventory directory. Because it wasassumed that inventories are kept in the same directory, debops.secret_ automatically took the name of the inventorydirectory and appended .secret suffix to it, making the resulting directory inventory.secret/.

Now, because each DebOps project lives in its own directory, this feature is no longer needed. Additionally in thecurrent state secret directory is kind of a show stopper, interfering for example with <Tab>-completion. Because ofthat, I’m changing the “formula” to instead just use the secret/ directory by default. It will be still created besidethe inventory/ directory.

All DebOps scripts will be updated at the same time, and should work with new directory name. However, existingdirectories will need to be renamed manually, otherwise DebOps might create new certificates, passwords, etc.

inventory.secret directory becomes secret.

If you use debops-padlock script, then .encfs.inventory.secret directory becomes .encfs.secret.

2014-09-21

Role updates

• debops.postfix_ has been cleaned up, all Ansible tasks have been rewritten from “inline” syntax to YAMLsyntax. Task conditions have been rearranged, now almost all of them can be found in tasks/main.yml fileinstead of in the file that are included.

• The way that ‘Postfix‘_ configuration files (main.cf and master.cf) are created by Ansible has beenchanged - instead of templating individual pieces on the remote servers and assembling them to finished files,configuration file templates are generated on Ansible Controller from parts included by Jinja and then templatedon the servers as a whole. This makes the process much faster and easier to manage.

• Postfix role has gained a new capability, archive. If it’s enabled, each mail that passes through the SMTPserver is blind carbon-copied to a separate archive mail account on local or remote SMTP server. This functionis configured automatically by the role, but can be modified using inventory variables. Archive account and/orarchive server need to be configured separately by the system administrator.

2014-09-19

Role updates

• debops.postfix_ role has gained support for SMTP client SASL authentication, in other words the ability tosend mail through remote relay MX hosts with client authentication, like public or commercial SMTP servers.You can either configure one username/password pair for a specified relayhost, or enable sender dependentauthentication and specify relayhost, user and password for each sender mail address separately. Passwords arenever stored in the inventory; instead Postfix role uses debops.secret_ role to store user passwords securely.

2014-09-18

Role updates

• debops.kvm_ role has been cleaned up from old and unused code, tasks were put in order and list of adminis-trator accounts that should have access to libvirt group changed name from auth_admin_accounts tokvm_admins (Ansible account is enabled automatically).

48 Chapter 7. Changelog

Page 53: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• debops.lxc_ role has been updated with changes to the LXC 1.0.5 package from Debian Jessie (some packagedependencies and build requirements were changed). You can read more in the lxc package changelog.

2014-09-17

Playbook updates

• You can now disable early APT cache update using apt_update_cache_early variable from debops.apt_role. This is useful in rare case when your APT mirror suddenly catches fire, and you need to switch to a differentone using Ansible.

Role updates

• debops.ferm_ role has gained new list variable, ferm_ansible_controllers, which can be used toconfigure CIDR hostnames or networks that shouldn’t be blocked by ssh recent filter in the firewall. This isuseful in case you don’t use DebOps playbook itself, which does that automatically. In addition, debops.ferm_saves list of known Ansible Controllers using local Ansible facts, and uses it to enforce current configuration.

• similar changes as above are now included in debops.tcpwrappers_ role, you can specify a list of AnsibleControllers in tcpwrappers_ansible_controllers list variable.

• Debian bug #718639 has been fixed which results in changes to several configuration files, including /etc/nginx/fastcgi_params and inclusion of a new configuration file /etc/nginx/fastcgi.conf. de-bops.nginx_ role will now check the version of installed nginx server and select correct file to include inPHP5-based server configuration.

2014-09-14

• Start of a new, separate changelog for DebOps_ playbooks and roles. This is a continuation of previousChangelog from ‘ginas‘_ project.

• all DebOps roles have been moved to ‘Ansible Galaxy‘_ and are now available via ansible-galaxy utilitydirectly. You can also browse them on the ‘DebOps Galaxy page‘_

New roles

• debops.elasticsearch_ is a role written to manage ‘Elasticsearch‘_ clusters, either standalone or on multiplehosts separated and configured using Ansible groups. Author: Nick Janetakis.

• debops.golang_ role can be used to install and manage ‘Go language‘_ environment. By default it will installpackages present in the distribution, but on Debian Wheezy a backport of golang package from Debian Jessiecan be automatically created and installed.

Role updates

• debops.ruby_ role has changed the way how different Ruby versions can be selected for installation. By de-fault, ruby_version: 'apt' variable tells the role to install any Ruby packages available via APT (bydefault 1.9.3 version will be installed on most distributions). If you change the value of ruby_version to'backport', a backported Ruby 2.1 packages will be created if not yet available, and installed.

7.54. 2014-09-17 49

Page 54: debops-playbooks Documentation · 7.13 2015-02-12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 7.14 2015-02-06 ...

debops-playbooks Documentation, Release latest

• Also in debops.ruby_, rubygems-integration package is installed separately from other packages andcan be disabled using ruby_gems_integration: False variable (this option was required for back-wards compatibility with ‘Ubuntu 12.04 LTS (Precise Pangolin)‘_ distribution).

50 Chapter 7. Changelog