CloudOpen 2014 - Extending Cloud Automation, When OpenStack Meets Ansible

COPYRIGHT © 2012 ALCATEL-LUCENT. ALL RIGHTS RESERVED.

1

Benjamin Zores, Alcatel-Lucent Enterprise CloudOpen 2014 – 14th October 2014 – Dusseldorf, Germany

Extending Cloud Automation: When OpenStack Meets Ansible

2


Extending Cloud Automation: When OpenStack Meets Ansible A Few Things About Me …

ALCATEL LUCENT

ENTERPRISE

TECHNICAL DIRECTOR, PERSONAL CLOUD SOLUTIONS • Cloud Architecture and Application/Infrastructure Design • R&D Development, Automation, Operations …

OPEN SOURCE

PROJECT FOUNDER, LEADER AND/OR CONTRIBUTOR FOR: • OpenBricks & GeeXboX : Embedded Linux cross-build tool and distribution. • uShare UPnP A/V & DLNA Media Server, FFMpeg, MPlayer …

LINUX FOUNDATION CONFERENCES

REGULAR LINUX FOUNDATION’S EVENTS SPEAKER • Various talks on:

• Linux Embedded Systems at Embedded Linux Conference (Europe) • Android Architecture and Device Porting at Android Builder Summit

GNU/LINUX MAGAZINE

FRANCE

RECURRENT TECHNICAL WRITER • Various publications on:

• Android Architecture Internals • Cloud (OpenStack, Ansible …)

3


Extending Cloud Automation: When OpenStack Meets Ansible Self-Promotion Time !

Series of articles published in GNU/Linux Magazine France

Android 4: Fondements Internes Benjamin Zores, Ed. Diamond – Sept. 2014

4


5


A Good Developer Is A Lazy Developer !

Don’t do over and over

again things that someone else

can do for you.

(especially if that someone is a bot)

Extending Cloud Automation: When OpenStack Meets Ansible Why we’re here ?

6


Extending Cloud Automation: When OpenStack Meets Ansible How one feels after deploying its servers through shell scripts …

#!/bin/sh echo “net.core.rmem_default=16384" | sudo tee -a /etc/sysctl.conf echo “deb http://nwps.ws/pub/mariadb/repo/5.5/debian wheezy main” | \

sudo tee –a /etc/apt/conf.d/mariadb.conf sudo apt-get -y install mariadb-server […]

AWESOME !!

7


Extending Cloud Automation: When OpenStack Meets Ansible Shell Scripts + Money = ? ( © John Lynch, http://goo.gl/gkmKGN )

“Model-driven orchestration frameworks for complex infrastructure

management and automation”

8


9


Extending Cloud Automation: When OpenStack Meets Ansible State of the Art

Original Rockstars !

- Great Tools - Field Pioneers

But just incredibly complex to start with, even for simple cases.

Not everyone is managing 500 servers in the cloud after all …

10


11


Extending Cloud Automation: When OpenStack Meets Ansible Introducing Ansible …

A fictional machine

capable of instantaneous or

superluminal communication (© Wikipedia).

Design Goals: 1.  Minimal in nature: Python based with no dependencies on the environment. 2.  Consistent. 3.  Secure: relies on OpenSSH only, with no vulnerable remote agents. 4.  Highly-Reliable: N re-deployments provide the same result. 5.  Low Learning Curve.

-  SW to manage and configure computers. -  Python + Jinja2 + YAML + SSH (that’s it !). -  Manages nodes over SSH. -  Does not require additional remote

dependencies. -  First Release: February 20th 2012.

12


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Architecture

13


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Default Model: Push to Nodes

© Julien Ponge, http://goo.gl/CB5f8a

14


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Alternative Model: Pull From Server

© Julien Ponge, http://goo.gl/CB5f8a

15


Extending Cloud Automation: When OpenStack Meets Ansible Introduction to Ansible

[lan1] 192.168.0.1 192.168.0.2

# ansible -i inventory.txt lan1 -m shell -a "/bin/echo Hello World" 192.168.0.1 | success | rc=0 >> Hello World 192.168.0.2 | success | rc=0 >> Hello World # ansible -i inventory.txt lan1 -u ben --sudo -m shell -a "/bin/echo Hello World"

Ansible Inventory File

Module Arguments Module

Name

16


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Inventory

[europe] foo.domain.com [usa] 192.168.0.1 [world:children] europe usa [webservers] www[01:50].domain.com [databases] db-[a:f].domain.com

Hosts can be described by FQDN or IP

With support for infinite depth inheritance

[targets] localhost ansible_connection=local web1.domain.com ansible_connection=ssh ansible_ssh_user=user1 web2.domain.com ansible_connection=ssh ansible_ssh_user=user2

With support for wildcards

And per-host tuning variables.

17


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Variables

[lan1] 192.168.0.1 msg="Hey !" 192.168.0.2 msg= "What’s up ?"

# ansible -i inventory.txt lan1 -m shell \ -a "/bin/echo {{msg}}"

192.168.0.1 | success | rc=0 >> Hey ! 192.168.0.2 | success | rc=0 >> What’s up ?

Inventory.txt: [europe:vars] domain=my.domain.eu [usa:vars] domain=my.domain.com

./group_vars/europe : -- domain: my.domain.eu ./group_vars/usa : -- domain: my.domain.com

YAML File

Format

18


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Modules (235 in Ansible v1.7.1)

# ansible-doc -l acl Sets and retrieves file ACL information. add_host Add a host (and alternatively a group) to the ansible-playbook alternatives Manages alternative programs for common commands apache2_module Enables/disables a module of the Apache2 webserver apt Manages apt-packages apt_key Add or remove an apt key apt_repository Add and remove APT repositories [...] xattr Set/retrieve extended attributes yum Manages packages with the `yum' package manager zfs Manage zfs zypper Manage packages on SuSE and openSuSE

19


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Module How-To

# ansible-doc shell > SHELL The [shell] module takes the command name followed by a list of space-delimited arguments. It is almost exactly like the [command] module but runs the command through a shell (`/bin/sh') on the remote node. Options (= is mandatory): - chdir cd into this directory before running the command - creates a filename, when it already exists, this step will *not* be run. - executable change the shell used to execute the command. Should be an absolute path to the executable. = free_form The shell module takes a free form command to run, as a string. There's not an actual option named "free form". See the examples! - removes a filename, when it does not exist, this step will *not* be run. Notes: If you want to execute a command securely and predictably, it may be better to use the [command] module instead. Best practices when writing playbooks will follow the trend of using [command] unless [shell] is explicitly required. When running ad-hoc commands, use your best judgement. # Execute the command in remote shell; stdout goes to the specified # file on the remote - shell: somescript.sh >> somelog.txt

20


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Playbooks

As appealing as they are, they’ve got nothing to do

with this !

21



--- - hosts: lan1 remote_user: ben sudo: no vars: name: ”My First Playbook” tasks: - name: verify servers activity ping: - name: say hello shell: /bin/echo "{{name}} {{msg}}” notify: - we are done handlers: - name: we are done shell: /bin/echo ”That’s it !"

YAML File

Format

22



# ansible-playbook -i inventory.txt playbook.yml -v PLAY [lan1] *********************************************************** GATHERING FACTS ***************************************************** ok: [192.168.0.1] ok: [192.168.0.2] TASK: [verify servers activity] ********************************************* ok: [192.168.0.1] => {"changed": false, "ping": "pong”} ok: [192.168.0.2] => {"changed": false, "ping": "pong"} TASK: [say hello] ******************************************************* changed: [192.168.0.1] => {"changed": true, "cmd": "/bin/echo \”My First Playbook Hey !\" ", "delta": "0:00:00.005264", "end": "2014-07-06 16:42:54.115860", "rc": 0, "start": "2014-07-06 16:42:54.110596", "stderr": "", "stdout": ”My First Playbook Hey !"} changed: [192.168.0.2] => {"changed": true, "cmd": "/bin/echo \”My First Playbook What’s up ?\" ", "delta": "0:00:00.002732", "end": "2014-07-06 16:42:54.078013", "rc": 0, "start": "2014-07-06 16:42:54.075281", "stderr": "", "stdout": ”My First Playbook What’s up ?"} […]

23



NOTIFIED: [we are done] **************************************** changed: [192.168.0.1] => {"changed": true, "cmd": "/bin/echo \"That’s it !\" ", "delta": "0:00:00.005559", "end": "2014-07-06 16:42:54.312184", "rc": 0, "start": "2014-07-06 16:42:54.306625", "stderr": "", "stdout": "That’s it !"} changed: [192.168.0.2] => {"changed": true, "cmd": "/bin/echo \"That’s it !\" ", "delta": "0:00:00.002824", "end": "2014-07-06 16:42:54.306878", "rc": 0, "start": "2014-07-06 16:42:54.304054", "stderr": "", "stdout": "That’s it !"} PLAY RECAP ************************************************** 192.168.0.1 : ok=4 changed=2 unreachable=0 failed=0 192.168.0.2 : ok=4 changed=2 unreachable=0 failed=0

24


Extending Cloud Automation: When OpenStack Meets Ansible Ansible Playbook

vars_file: - “vars/common.yml” - “vars/production.yml”

tasks: - include: tasks/nginx.yml - include: tasks/php-fpm.yml - include: tasks/mariadb.yml

25


Extending Cloud Automation: When OpenStack Meets Ansible Example: (Parts of) MariaDB Cluster Automation

[mariadb] 192.168.0.1 name=my-db-maria1 192.168.0.2 name=my-db-maria2 192.168.0.3 name=my-db-maria3

tasks/hosts.yml: - name: declare hosts lineinfile: dest=/etc/hosts regexp='.*{{ item }} .*$' line="{{item}} {{ hostvars[item]['name'] }} {{ hostvars[item]['name'] }}.{{domain}}" state=present with_items: groups['{{hosts_list}}']

- hosts: mariadb vars: domain : domain.com hosts_list: mariadb tasks: - include: tasks/hosts.yml

Targets /etc/hosts: 192.168.0.1 my-db-maria1 my-db-maria1.domain.com 192.168.0.2 my-db-maria2 my-db-maria2.domain.com 192.168.0.3 my-db-maria3 my-db-maria3.domain.com

26



tasks/mariadb-galera.yml: - name: Add MariaDB APT key apt_key: url=http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xcbcb082a1bb943db - name: Add MariaDB APT repository apt_repository: repo='deb http://ftp.igh.cnrs.fr/pub/mariadb/repo/5.5/debian wheezy main' - name: APT pinning for MariaDB action: copy src=files/mariadb.pref dest=/etc/apt/preferences.d/mariadb.pref \

owner=root group=root mode=0644

vars: mariadb_debian_password: my_debian_password mariadb_root_password: my_root_password mariadb_cluster_name: my_cluster mariadb_cluster_list: "{{ groups['mariadb'] }}” mariadb_gcache_size: 4G mariadb_extra_cfg: skip-external-locking: ~ skip-name-resolve: ~ # Force no DNS resolution tasks: - include: tasks/mariadb-galera.yml

27



tasks/mariadb-galera.yml: - name: install mariadb-galera-server action: apt name=mariadb-galera-server update_cache=yes - name: copy mysql_secure_installation credentials when: mariadb_root_password is defined action: template src=templates/mariadb_secure_installation.j2 \

dest=/tmp/mariadb_secure_installation owner=root group=root mode=0600

templates/mariadb_secure_installation.j2: {{ mariadb_root_password }} {{ mariadb_root_password }}

28



tasks/mariadb-galera.yml: - name: write conf.d/mariadb.cnf action: template src=templates/mariadb.cnf.j2 dest=/etc/mysql/conf.d/mariadb.cnf

templates/mariadb.cnf.j2: [mysqld] wsrep_provider=/usr/lib/galera/libgalera_smm.so {% if mariadb_cluster_name is defined %} wsrep_cluster_name='{{ mariadb_cluster_name }}’ {% endif %} {% if mariadb_cluster_list is defined %} wsrep_cluster_address=gcomm://{{ mariadb_cluster_list[0] }}{% for node in mariadb_cluster_list[1:] %},{{ node }}{% endfor %}{% endif %}

Targets /etc/mysql/mariadb.cnf: wsrep_cluster_name=‘my_cluster’ wsrep_cluster_address=gcomm://192.168.0.1,192.168.0.2,192.168.0.3

Jinja2 Syntax

29



templates/mariadb.cnf.j2: [mysqld] wsrep_node_address={{ ansible_eth0.ipv4.address }} wsrep_node_name='{{ ansible_hostname }}’ {% if mariadb_extra_cfg is defined %} {% for key, value in mariadb_extra_cfg.iteritems() %} {{ key }}{% if value is not none %}={{ value }}{% endif %} {% endfor %} {% endif %}

Target /etc/mysql/mariadb.cnf: wsrep_node_address=192.168.0.1 wsrep_node_name=my-db-maria1 skip-external-locking skip-name-resolve

30


31


Extending Cloud Automation: When OpenStack Meets Ansible OpenStack in a Nutshell

- #1 (most widely adopted) Open-Source IaaS project. - Awesome REST Management API. - Perfect for instant spawning of new Virtual Machines (VMs) - But VMs yet to be configured ...

What if I could connect

OpenStack VMs with Ansible

for nightly CI ?

32


Extending Cloud Automation: When OpenStack Meets Ansible (One of my) Cloud application: OpenTouch TeamShare

-  Online collaboration tool for SMBs. -  Provides multi-projects file storage and sharing for enterprises. -  With project management, chat and collaboration capabilities.

33


Extending Cloud Automation: When OpenStack Meets Ansible (One of my) Cloud application network topology

-  2x HAProxy Load-Balancers -  2x NGINX frontal Web Servers -  2x NGINX frontal Web File Servers -  2x PHP Backends -  1x SMTP Server -  3x MariaDB Master-Master Galera Cluster + 2x Galera Arbiters -  2x MongoDB Master-Slave Cluster + 1x MongoDB Arbiter -  3x RabbitMQ Master-Master Clusters -  1x LibreOffice Server -  1x NFS Server

Now let’s say that I want to test both my application and my infrastructure every single night for non-regression !

34


35


Extending Cloud Automation: When OpenStack Meets Ansible AvOID: Ansible OpenStack Instances Deployer

-  Open Source software by Alcatel-Lucent Enterprise. -  Written in Python with dependencies to OpenStack Nova/Neutron APIs. -  Relies on Ansible with IaaS bindings:

-  Currently OpenStack only -  But wide open to support many much more … who knows …

-  Comes as a library with both CLI and Web clients.

How it works: 1.  Describe your infrastructure topology in a YAML file once and for all. 2.  Run avoid-cli. Grab a coffee, that’s it.

How it (internally) works: 1.  Parses your topology file. 2.  Optionally terminates (all) OpenStack VMs and spawn new ones and build Ansible inventory file. 3.  Creates VMs dependency graph for parallelized post-configuration by Ansible. 4.  Post-configure VMs through Ansible playbooks (continuous-integration style).

36


Extending Cloud Automation: When OpenStack Meets Ansible AvOID: Ansible OpenStack Instances Deployer

Check it out on GitHub: https://github.com/OpenTouch/AvOID

37


Extending Cloud Automation: When OpenStack Meets Ansible AvOID Topology File

- globals: env: dev ssh_key: /path/to/ssh_private_key.pem ssh_user: remote_user os_user: openstack_user os_passwd: openstack_password os_tenant: openstack_tenant os_auth_url: http://my.private.cloud.com:5000/v2.0 os_image: Debian – Wheezy os_network: My OpenStack Tenant Network os_ssh_key: My OpenStack Tenant SSH Key Name ansible_inventory_template: /path/to/ansible/inventory_template.txt ansible_playbooks_directory: /path/to/ansible/playbooks

38


Extending Cloud Automation: When OpenStack Meets Ansible AvOID Topology File

nodes: - node: name: web-server-1 flavor: m1.small ansible_config_keys: webserver playbook: webserver additional_network: LAN Network 2, LAN Network 3 security: http, https depends: file-server-1 floating_ips: 1.2.3.4, 5.6.7.8, 10.20.30.40 vips: 10.0.1.2, 10.0.2.2, 10.0.3.2 - node: name: file-server-1 flavor: m3.medium ansible_config_keys: fileserver playbook: fileserver volumes: - { name: web-volume1, size: 1 } - { name: web-volume2, size: 10 }

39


Extending Cloud Automation: When OpenStack Meets Ansible AvOID command-line

# avoid-cli Usage: avoid-cli <topology_file.yml> <command> [opts] where <command> is: - status: list topology status - redeploy <list of playbooks or VM names>:

terminate VM, spawn a new one and restart playbook - redeployall: redeploy all nodes - runplaybook <list of playbooks>: restart playbook as it - runallplaybooks: restart all playbooks - geninventory:

generate Ansible inventory file based on topology.yml

40


Extending Cloud Automation: When OpenStack Meets Ansible AvOID Web UI

# avoid-web my-topo-dev.yml Read of my-dev.yml done: 32 VMs and 22 playbooks Now go to http://localhost:8888/ !

41


32 VMs fully deployed in 19mn40s (5mn40s for OpenStack and 14mn and

for Ansible post-configuration)

42


43


Extending Cloud Automation: When OpenStack Meets Ansible Let’s keep in touch …

benjaminzores

@gxben

#Benjamin Zores

CloudOpen 2014 - Extending Cloud Automation, When OpenStack Meets Ansible

Technology

Transcript of CloudOpen 2014 - Extending Cloud Automation, When OpenStack Meets Ansible