© 2014 VMware Inc. All rights reserved.© 2014 VMware Inc. All rights reserved.

22/02/2016

Ansible Module Development 101Yves FauserTechnical Product Manager @ VMware

Who is standing in front of you?

• I’m working with VMware’s network virtualization product called NSX• I’m the co-organizer for this Meetup & the OpenStack Munich Meetup group• I’m living in Munich since 14 years• I’ve spend 3 years working at VMware as Systems Engineer & Solution Architect,

7 years as a Systems Engineer at Cisco, and I was a networking / OS consultant and developer before

• Topics I love to discuss and work with: Configuration Management, Automation, Containers / ‘Cloud’, OpenStack, Networking, ...

Yves FauserTechnical Product Manager @ VMware

Why developing your own Modules

• There are tons of great build-in core and extra modules for Ansible – So why build your own?• Generally if you only work with Files, Packages and Services on standard OSs you are

covered, and there is no need to develop your own stuff• But sometimes you want to do ‘strange stuff’, and then making this idempotent is hard, e.g.

• You might want to interact with APIs of products and ‘cloud services’

• You might want to use commands / CLIs that were not build to work well with configuration management (e.g. that don’t implement granular return codes but give you lots of stdout garbage)

• I had to do both, interact with an API and use CLI tools with a lot of stdout that I needed to interpret

• Working with the VMware NSX API

• Using VMware’s OVFTool to deploy VMs into vCenter *

* We will show the use the OVFTool Module as an example later

Why using Python to develop Modules• Ansible generally supports writing Modules in any language, e.g Python, C, Ruby, Shell, Go, …• The only requirements are:

• The language needs to be able to implement File I/O

• The language needs to be able write JSON formatted output to STDOUT

• Why I am using Python:• First and foremost, I’m a Pythonist – I really like this language

• Ansible itself and all its core Modules are written in Python – You have a lot to read and learn from!

• There are very easy and ready to use ‘boilerplates’ and ‘helper functions’ for Ansible written for Python

• Besides Python, the second most often used language is shell script

Getting started – a few important bits of knowledge• When writing a custom module

you call it from a Play like this:• Alternatively you can use ‘action’• In this case ‘test_module’ can be

test_module.py, or test_module.sh, test_module.<you_name_it> in thelibrary path

• Ansible searches for modules the path specified by ANSIBLE_LIBRARY or the --module-path command line option

• The directory ”./library”, alongside your top level playbooks, is also automatically added as a search directory

• I personally always use the “./library” option when developing modules

Getting started – The ‘boilerplate’ code of every module

• You always start with a boilerplate code like this:

Uuuhh … that’s ugly, an import statement at the bottom of the file, and

importing *, that’s against all python style guides!

Why’s that?

Ansible code creation• You module code is not the final code that gets executed• Ansible takes the code and constructs a single Python script that gets copied to the remote

system and then executed. The statement ‘from ansible.module_utils.basic import *’ imports ~1600 lines of code into this final script

Ansible code creation

Here’s the place were it inserts the Ansible Module

code after your code

And the function main() is called right at the end of the

code

Getting started – The Module argument_spec dictionarymandatory variable

mandatory variable, do no log (e.g. passwords)

mandatory variable, with fixed possible values

optional variable taking any value

optional variables checked for existence later

variable having a default value

variable with a strict type

variable with an alias

checks if at least one variable in the list is present

ensures that only one of two variables are set

Getting started – The Module argument_spec dictionary• You can access the modules parameters dictionary with ‘module.params[‘key’]’

Exiting the module• Exiting the module is done using

‘module.exit_json’ for successful returns, and ‘module.fail_json’ to fail the task

Exit the module with the changed flag set and with additional variables returned to the Play

Exit the module successfully, but signaling that there was no change

Exit the module with a failure

Putting it to work – Example Module• I love to go running, but not when it’s too cold. I will let Ansible decide for me

You can find the code here: https://gist.github.com/yfauser/90c0955827c74d83d6a2

If the temperature returned by Open Weather Map in my city is bellow my threshold –I’ll stay at home (keep my current state)

If the temperature returned by Open Weather Map in my city is above my threshold – I’ll leave the house (change my state)

Putting it to work – The If / Elif / Else way

Putting it to work – Catching errors• OK, but what about failed states?

Let’s enhance our code a bit

Using try/except as well as http status codes as indications of failed conditions

Putting it to work – Catching errors Here we are passing the module

to the sub-function to callmodule.fail_json inside of it

Use OS commands with Python based Modules• The python Ansible base code has a great method to interact with the OS

• The difference from running this inside of your python based module instead of ‘command’ in your Play is that you can interpret the output returned by the OS command more flexibly

• The response object will be a list with two items:• [0] holds the return code of the command as an integer, so you can react based on the return code

• [1] holds the stdout of the command as a string – So you can do things like:

• Pass it to parsers like ‘json.loads’ to convert it to dictionaries• Search in it with ‘.find’• Count, sort, pass it through regexp, etc.

Use OS commands with Python based Modules• Lets change our example to

use OS commands (curl)

format the owm url with all mandatory url parameters

use the OS command curl to send the request

check if curl gave a positive return code

check if we received a positive ret. code

convert curl stdout to a python dict

extract temperature value from dict

You can find the code here:https://gist.github.com/yfauser/b2377eee842a1cd2a899

Use OS commands with Python based Modules• Here’s another real life example of using the OS command option

• This code usesmodule.run_commandto invoke the ovftool to deploya VM intovCenter

You can find the code here:https://github.com/vmware/nsxansible/blob/master/library/nsx_deploy_ova.py

Debugging your module• What is the #1 tool for debugging Python code? => ‘print’ ;-)• Unfortunately sending non-JSON

formatted output to stdout will not result in any output on the console when running your Play:

• Modules must also not output anything on standard error, because the system will merge standard out with standard error and prevent the JSON from parsing. Capturing standard error and returning it as a variable in the JSON on standard out is fine, and is, in fact, how the command module is implemented

• In short => Never use ‘print’

Debugging your module code• Instead of printing to stdout, send your debug data into a variable (e.g. a list) and

pass it to the module.exit_json or module.fail_json method:

What’s this ‘supports_check_mode’ about?

• Ansible Playbooks can run in check-modeusing the --check option

• If ‘supports_check_mode’ is set to:• ‘True’: It’s up to you to make sure your module doesn’t

apply changes, but only checks if it would have madechanges if Ansible runs in check mode

• ‘False’ or unset: The task using your module willbe skipped if Ansible runs in check mode

Inline Documentation

• Ansible supports the generation of moduledocumentation on their Web-Site frominline Documentation in the module code

• You can read about all the details here:http://docs.ansible.com/ansible/developing_modules.html#documenting-your-module

Developing Ansible Modules on a MAC• When developing a python based module on a MAC you might run into this:

• When you run the module it fails to import libraries because it doesn’t use the right interpreter and path

• Solution 1, use the ‘#!/usr/bin/env python’ shebang instead of ‘#!/usr/bin/python’:• Don’t use this! As it’s an anti-pattern if you want to get this into ansible-modules-extra or core

Solution 2, define the interpreter path in the hosts file This is the proper solution as this solves it on a per host level for those systems that need it

Questions?

Check out some more of my stuff here:https://github.com/vmware/nsxansible/