The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and...
Transcript of The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and...
![Page 1: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/1.jpg)
The journey to 1000 modules begins with a single contribution
Zim Kalinowski, Senior Software EngineerPC Chan, Senior Program Manager
![Page 2: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/2.jpg)
ZIM KALINOWSKISenior Software EngineerMicrosoft Corporation
• Home: Shanghai, China• Working on Open Source, previously involved
mostly in Embedded Systems• Started with Ansible in 2017• Favorite thing to do with Ansible: new modules• Big fan of American BBQ and draft beer
• Twitter: @ZimOnAzure• LinkedIn:
https://www.linkedin.com/in/smarterphone/
![Page 3: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/3.jpg)
PUI CHEE (PC) CHANSenior Program ManagerMicrosoft Corporation
• Home: Bellevue, Washington. Originally from Malaysia
• Started with open source in 2017 • New product owner of Ansible on
Azure• Favorite thing to do with Ansible: help
remove friction when using Ansible with Azure
• LinkedIn: https://www.linkedin.com/in/pui-chee-chan-pmp-7783411/
![Page 4: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/4.jpg)
Agenda
• Our shared journey • Tips and tricks when creating a modules• Future improvement/experimental ideas• Workarounds for lack of native support• Ansible Azure collections• Future roadmap
![Page 5: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/5.jpg)
Our shared journey - Ansible and Microsoft Azure
![Page 6: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/6.jpg)
Some lessons learned
• Coverage vs. end to end• Inconsistent implementation• Dependencies to Python SDK• Integration testing• Slow review process• Long release cycle
![Page 7: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/7.jpg)
Guidelines/tips and tricks
• Standardize how you create modules to accelerate the review process
• Introduce guidelines:Naming conventionSynchronization of test and samples/playbooksStandardization of facts/info module
• How to ensure Idempotency (and check mode)
![Page 8: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/8.jpg)
What is Ansible Module?
• Parameters transformationmostly renaming / flattening
• Additional validation• Idempotency check
NOTE: Most could be done in lower layers
Ansible Module
Azure Python SDK
Azure REST API
![Page 9: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/9.jpg)
Idempotency – How to do it properly?
GET – retrieve current resource state
APPLY parameters to retrieved resource state
CHECK whether anything changed
PUT – update resource state if change detected
![Page 10: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/10.jpg)
Idempotency – additional things to consider
• Write-only properties:e.g., keys and passwords
• Non-updatable Properties• Lists:
How to remove / add single element?• “Generic” modules:
Idempotency check and unflattening parameters
![Page 11: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/11.jpg)
Shrinking implementation (experimental)
• Common component:Generic idempotenceGeneric UX
• Simplify custom implementation• Benefits:
Can be used in CloudShellEasier implementationEasy to include in collections
Ansible Module
Python SDK
Azure REST API
Common
component
![Page 12: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/12.jpg)
Module auto-generation (experimental)
Autogeneration
Toolchain
Azure REST API Specs
Ansible Modules
Azure CLI
Swagger Integration
Tests
Examples:
- Azure CLI
- Python
- Ansible
![Page 13: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/13.jpg)
When creating playbook
What’s your overall scenario?
What do you need?
What’s available?native support (aka. modules)
azure_preview_modules
What’s not?
What do you do when there is a gap?
![Page 14: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/14.jpg)
Recommendation for workarounds
![Page 15: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/15.jpg)
There is more than one way to skin a cat…
• wrap Azure Resource Management (ARM) in Ansible
• wrap Azure REST API in Ansible• uri module to call Azure REST API• using run commands
(shell/command/raw/script):
shell module to call az cli
raw module to call PowerShell…
![Page 16: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/16.jpg)
Azure Resource Manager (ARM) template
• Use azure_rm_deployment to create or destroy ARM template via the Azure SDK for Python
• Idempotency is not guaranteed• Could be hard to debug
Recommendations:• Use if you want the latest and
greatest (preview feature)• One template for one resource
group• Use azure_rm_deployment_facts to
list information about created resources
• Quickstart template
# Create a simple Ubuntu VM
- name: Create Azure
azure_rm_deployment:
resource_group: "{{ resource_group }}"
location: "{{ location }}"
template_link:
'https://raw.githubusercontent.com/Azure/azure-quickstart-
templates/master/101-vm-simple-linux/azuredeploy.json’
parameters:
authenticationType:
value: "password"
adminUsername:
value: "{{ admin_user }}"
adminPasswordOrKey:
value: "{{ admin_password }}"
dnsLabelPrefix:
value: "{{ name }}"
ubuntuOSVersion:
value: "16.04.0-LTS"
![Page 17: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/17.jpg)
Azure REST API for update
• Use azure_rm_resource to create, update or delete any Azure resource using Azure REST API
• Refer to https://docs.microsoft.com/en-us/rest/api/
• Idempotency is not 100%If enabled, done by using GET method and compare with body
Recommendation:• Use for “patching”• Use azure_rm_resource_info to check
created resource IDs, list of all dependencies between resources etc.
using uri module to interact with webservice is not recommended
# Use REST API to create a network security group
- name: Call REST API
azure_rm_resource:
api_version: '2018-02-01'
resource_group: "{{ resource_group }}"
provider: network
resource_type: networksecuritygroups
resource_name: "{{ nsgname }}"
body:
location: "{{ location }}"
idempotency: yes
![Page 18: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/18.jpg)
Collections
• What is a collection?A new distribution formatAll modules will be moved from Ansible core to collections
• Our plan:By 2.9, 18 certified Azure modules will be moved to azure.azcollectionRemaining azure modules will move to same collection gradually
• What does this mean to you?More frequent updateNeed to install collectionNeed to specify the fully qualified collection name (FQCN)
![Page 19: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/19.jpg)
Collections (cont.)
• Use the keyword collections added in Ansible 2.8
- hosts: all
collections:
- my_namespace.my_collection
tasks:
- import_role:
name: role1
- mymodule:
option1: value
- debug:
msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')|
my_namespace.my_collection.filter1 }}'
![Page 20: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/20.jpg)
Future roadmap
• Moving all Azure modules to collection: azcollection• Ansible 2.9
VM/VMSS: shared image gallery; snapshot; specialized image supportManagement group; subscriptionAzure Firewall; IoT Hub/Devices; Azure Automation account
• Coming:VM/VMSS: ephemeral diskAzure Front DoorGovernance: Azure Policy; Azure Blueprints
• Refer to Azure Ansible Hub: https://aka.ms/ansiblesupport• Help us to make things better for you: https://aka.ms/ansiblefest2019
![Page 21: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/21.jpg)
Useful links
Developer Hub for Ansible:http://docs.microsoft.com/azure/ansible
Azure Modules:https://github.com/ansible/ansible/tree/latest/lib/ansible/modules/cloud/azure
Visual Studio Code Extension:https://marketplace.visualstudio.com/items?itemName=vscoss.vscode-ansible
Azure Preview Modules:https://galaxy.ansible.com/Azure/azure_preview_modules/
![Page 22: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/22.jpg)
Useful links
Azure Playbook Sampleshttps://github.com/Azure-Samples/ansible-playbooks
Azure Ansible Labshttps://github.com/microsoft/ansiblelabs
IRC#ansible-azure (Freenode)https://webchat.freenode.net/?channels=ansible-azure
Help us by giving us feedback: https://aka.ms/ansiblefest2019
![Page 23: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/23.jpg)
Q & A
![Page 24: The journey to 1000 modules begins with a single contribution ATL Slide... · Idempotency check and unflattening parameters. Shrinking implementation (experimental) • Common component:](https://reader033.fdocuments.in/reader033/viewer/2022060210/5f0481557e708231d40e4d70/html5/thumbnails/24.jpg)
Surveyhttps://aka.ms/ansiblefest2019