Managing Infrastructure with github.com/timtadh Python ... › pdfs ›...
Transcript of Managing Infrastructure with github.com/timtadh Python ... › pdfs ›...
![Page 1: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/1.jpg)
Managing Infrastructure with Python, Fabric and Ansible
By Tim [email protected] hackthology.com
github.com/timtadh
![Page 2: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/2.jpg)
part 00
![Page 3: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/3.jpg)
death of a sys-admin
![Page 4: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/4.jpg)
there are too many machines
![Page 5: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/5.jpg)
now, we have become root
![Page 6: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/6.jpg)
the world is changing
![Page 7: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/7.jpg)
operations are a programming problem
![Page 8: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/8.jpg)
A programmer's delight?
Server management should be:
1. Simple2. Well documented3. Repeatable4. Testable5. Auditable
![Page 9: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/9.jpg)
part 01
![Page 10: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/10.jpg)
- step 0: - desc: never type commands on a server reasons: - hard to keep servers in sync - hard to audit - hard to know server state- step 1: - desc: use a tool reasons: - easy to review changes - server state is known- step 2: - desc: standardize all processes reason: no special servers
![Page 11: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/11.jpg)
definitions
you
your servers
your programs
![Page 12: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/12.jpg)
definitions
you
![Page 13: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/13.jpg)
a single service
web tier
app tier
database tier
![Page 14: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/14.jpg)
your “app”
web
app
db
web
app
web
app
db
db
![Page 15: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/15.jpg)
web
app
db
web
app
web
app
db
db
we provision individual “network services” onto individual servers.
![Page 16: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/16.jpg)
web
app
db
web
app
web
app
db
db
but we also need to be able to control whole “logical services” and connect them together
![Page 17: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/17.jpg)
lifecycle of a network service
Provision
Bootstrap
Configure Deploy
Migrate?
![Page 18: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/18.jpg)
part 10
![Page 19: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/19.jpg)
Enough PhilosophizingSHOW ME THE CODE
![Page 20: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/20.jpg)
fabric
from fabric import api
def who():api.run('echo $(whoami) on $(uname -s)')
$ mkdir ops-example && cd ops-example$ virtualenv --no-site-packages env$ source ./env/bin/activate && pip install fabric$ cat >fabfile.py <<EOF
> EOF$ fab -H localhost who[localhost] Executing task 'who'[localhost] run: echo $(whoami) on $(uname -s)[localhost] out: hendersont on Linux[localhost] out:Done.Disconnecting from localhost... done.
>>>>>
![Page 21: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/21.jpg)
provision - config - deploy
from fabric import api
def provision(): api.sudo(‘apt-get install python python-dev python-pip’) api.sudo(‘pip install virtualenv’) api.run(‘git clone git@host:group/app’) api.run(‘virtualenv --no-site-packages app/env’)
def config(): api.put(‘startup.sh’, ‘startup.sh’) api.sudo(‘mv startup.sh /etc/init.d/app’) api.put(‘app-config.ini’, ‘app/config.ini’)
def deploy(): api.run(‘git -C app/ pull’) api.run(‘source app/env/bin/activate && ’ ‘pip install -U -r requirements’) api.sudo(‘service app restart’)
![Page 22: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/22.jpg)
simple, but brittle
![Page 23: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/23.jpg)
enter ansible, a higher level tool
![Page 24: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/24.jpg)
ansible
yourhost
$ mkdir ops-example && cd ops-example$ virtualenv --no-site-packages env$ source ./env/bin/activate && pip install ansible$ cat >HOSTS <<EOF
> EOF$ ansible all -i HOSTS -m shell \> -a 'echo $(whoami) on $(uname -s)'yourhost | SUCCESS | rc=0 >>hendersont on Linux$ cat >playbook.yml <<EOF
>
- hosts: all tasks:
- shell: echo $(whoami) on $(uname -s)
>>>> EOF
![Page 25: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/25.jpg)
ansible$ cat >playbook.yml <<EOF- hosts: all tasks:
- shell: echo $(whoami) on $(uname -s)
>>>> EOF$ ansible-playbook -v -i HOSTS playbook.ymlPLAY [all]
TASK [setup] ************************************************************ok: [yourhost]
TASK [command] ************************************************************changed: [yourhost] => {"changed": true, "stdout_lines": ["hendersont on Linux"]}
PLAY RECAP ************************************************************yourhost : ok=2 changed=1 unreachable=0 failed=0
![Page 26: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/26.jpg)
so far just a complicated ssh
![Page 27: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/27.jpg)
modules, roles, and templatesmake ansible worthwhile
![Page 28: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/28.jpg)
a motivating module: git$ cat >gitplay.yml <<EOF- hosts: all tasks:
- name: update code for app repo git: repo: https://host.com/group/app.git dest: /opt/app/ accept_hostkey: True force: yes update: yes version: stable-branch
>>>>>>>>>>> EOF$ ansible-playbook -i HOSTS gitplay.yml...TASK [command] ************************************************************changed: [yourhost] => {"after": "9c46a49a73103de9a929718c223326149cb9accd", "before": null, "changed": true}...
![Page 29: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/29.jpg)
roles add structure to your code$ tree -L 3 ..├── HOSTS├── playbook.yml└── roles
├── base│ ├── files/│ ├── handlers/│ └── tasks/├── go│ ├── tasks/│ └── templates/├── hadoop:base│ ├── files/│ ├── handlers/│ ├── tasks/│ ├── templates/│ └── vars/├── hdfs:datanode│ ├── meta/
│ ├── tasks/│ ├── templates/│ └── vars/├── hdfs:namenode│ ├── handlers/│ ├── meta/│ ├── tasks/│ ├── templates/│ └── vars/
![Page 30: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/30.jpg)
roles are directories, with a specified structure$ tree roles/hdfs:namenode/roles/hdfs:namenode/├── handlers│ └── main.yml├── meta│ └── main.yml├── tasks│ ├── config.yml│ ├── main.yml│ └── provision.yml├── templates│ ├── hdfs-namenode.conf│ ├── hdfs-nfs.conf│ └── hdfs-portmap.conf└── vars
└── main.yml
![Page 31: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/31.jpg)
tasks/main.yml is the execution entry point$ cat roles/hdfs:namenode/tasks/main.yml- include: "{{mode}}.yml"
![Page 32: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/32.jpg)
the mode variable is set in the calling playbook$ cat roles/hdfs:namenode/tasks/main.yml- include: "{{mode}}.yml"
$ cat provision.yml- hosts: hdfs_namenodes user: admin vars:
mode: provision roles:
- role: hdfs:namenode tags: - hdfs - hdfs:namenode
![Page 33: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/33.jpg)
which causes main.yml to load the correct tasks$ cat roles/hdfs:namenode/tasks/main.yml- include: "{{mode}}.yml"
$ cat provision.yml- hosts: hdfs_namenodes user: admin vars:
mode: provision roles:
- role: hdfs:namenode tags: - hdfs - hdfs:namenode
$ tree roles/hdfs:namenode/roles/hdfs:namenode/├── handlers│ └── main.yml├── meta│ └── main.yml├── tasks│ ├── config.yml│ ├── main.yml│ └── provision.yml├── templates│ ├── hdfs-namenode.conf│ ├── hdfs-nfs.conf│ └── hdfs-portmap.conf└── vars
└── main.yml
![Page 34: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/34.jpg)
implementing lifecycle phases at the service level$ cat roles/hdfs:namenode/tasks/main.yml- include: "{{mode}}.yml"
$ cat provision.yml- hosts: hdfs_namenodes user: admin vars:
mode: provision roles:
- role: hdfs:namenode tags: - hdfs - hdfs:namenode
$ tree roles/hdfs:namenode/roles/hdfs:namenode/├── handlers│ └── main.yml├── meta│ └── main.yml├── tasks│ ├── config.yml│ ├── main.yml│ └── provision.yml├── templates│ ├── hdfs-namenode.conf│ ├── hdfs-nfs.conf│ └── hdfs-portmap.conf└── vars
└── main.yml
![Page 35: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/35.jpg)
5 top level playbooks are used to run tasks
$ ls *.ymlbootstrap.ymlprovision.ymlconfig.ymldeploy.ymlmigrate.yml
Provision
Bootstrap
Configure Deploy
Migrate?
$ ansible-playbook -i HOSTS lifecycle.ymlPLAY [hdfs_namenodes] *********************************************TASK [setup] *********************************************ok: [host] ....
![Page 36: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/36.jpg)
each playbook lifecycle calls the appropriate roles$ cat config.yml- hosts: hdfs_namenodes user: admin vars:
mode: config roles:
- role: hdfs:namenode tags: - hdfs - hdfs:namenode
- hosts: zookeeper user: admin vars:
mode: config user: admin roles:
- role: zookeeper tags: - zookeeper
![Page 37: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/37.jpg)
templates enable shared configuration between files$ cat roles/hadoop\:base/templates/core-site.xml<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration> <property>
<name>fs.defaultFS</name> <value> hdfs://{{hostvars[groups['hdfs_namenodes'][0]] ['ansible_fqdn']}}:{{hdfs_namenode_port}} </value> </property> <property>
<name>io.file.buffer.size</name><value>131072</value>
</property> ...</configuration>
![Page 38: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/38.jpg)
let’s stop here
![Page 39: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/39.jpg)
takeaways
High level
1. Devs increasingly are increasingly performing operational roles2. Ops folk are increasingly performing developer roles3. Operations now looks more like programming than before
Practical
1. Don’t run commands on servers2. Use a tool3. Follow a standardized process for everything
![Page 40: Managing Infrastructure with github.com/timtadh Python ... › pdfs › managing-infrastructure-with-python.pdf · roles add structure to your code $ tree -L 3 .. ├── HOSTS](https://reader030.fdocuments.in/reader030/viewer/2022040411/5ed8f2356714ca7f4768dd81/html5/thumbnails/40.jpg)
resources
● https://docs.ansible.com/○ https://docs.ansible.
com/ansible/playbooks_best_practices.html○ https://github.com/ansible/ansible-examples
● http://docs.fabfile.org/○ http://docs.fabfile.org/en/1.12/tutorial.html
● https://www.vagrantup.com/● https://www.docker.com/