Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired...
Transcript of Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired...
Deploying Complex Stacks with AnsibleDevconf 2020 - 2020-01-24
Will Foster • @sadsfae github.com/sadsfae • https://hobo.house
● Client-less configuration management system● Written in Python● Uses SSH as a transport mechanism● Uses YAML for logic and tasks● Uses Jinja2 for templating
What is Ansible?
● Save time and resources so you can do other things● Significantly lower deployment time for apps / services● Reduce complexity and human error via automation
Why should I use Ansible?
● Strive for idempotency. ○ Ansible should do nothing if in desired state
● Template as many configuration files as possible● Break deployment pieces/objectives into logical parts● Make liberal use of configuration variables● Aim for an open-ended design and choice● Use Ansible provided modules wherever possible
Configuration Management Goals
How Baby Yoda Writes Ansible
● Deploy a full all-in-one ELK/EFK 6.8.x stack○ Elasticsearch (search engine, time-series datastore)○ Logstash (data collection, log parsing engine)
○ Kibana (analytics, visualization)
○ Nginx (web reverse proxy)
● We’ll use CentOS7
● Code here: github.com/sadsfae/ansible-elk
Complex Stack Example: ELK/EFK
● host-02 (client) →○ Send system logs via Filebeat to Logstash on host-01
● host-01 (server) → ○ Logstash accepts system logs over SSL/TLS○ Logstash filters logs and sends to Elasticsearch○ We visualize events in Kibana Web UI
DEMO: ELK/EFK Deployed via Ansible
You’ve had your snack, you’ve played with the buttonNow it’s time to put your jammies on.
Ansible Facts can Customize your Environment
● Example: automatically tune Elasticsearch JVM heapsize based on amount of physical memory detected
Use System Facts to Adjust Configuration
Handling Service Dependencies in Ansible
● When components depend on other components, check they are available before proceeding
● Can be done by HTTP return code, port, or web content
Make use of Service Dependency Checking
● wait_for, until and uri Ansible modules are useful for this
Make use of Service Dependency Checking
● Parent service availability checks during a playbook run
Make use of Service Dependency Checking
● Another example: checking raw output matches what we want before proceeding
Dependency Checking Example - Dell Racadm
Using Variables for Conditional Logic
● Complex stacks will inevitably grow to require more deploy options● Maximize the usage of conditional vars to provide choice● Expand options for deployment flexibility (see all.yml)
../install/group_vars/all.yml → → → →
vars_files Make your Playbook more Flexible
Organizing your Playbook and role Hierarchy
Common Design Hierarchy for Large Playbooks
Try to use one role per major component
● Use branches to support older versions/series of the stack● Use branches to support deployment to different environments● Gitlab.com offers free, private repositories each with 10Gb of space
Keep your Playbooks in an SCM (Git, etc).
Automate Client Operations When Possible
● Make it easy to automate client integration● e.g. SSL/TLS certificate retrieval, client applications / libraries
Make sure you’re automating client-side
CI / CD and Ansible Lint
● Use ansible-lint to test your playbooks
ansible-lint install/*.yml -v
ansible-lint install/roles/*/*/*.yml
Troubleshooting and Debugging Tips
● Useful to determine registers, variable output and stdout
Using Debug in a Playbook
● Debug can be useful for informational messages
Using Debug in a Playbook
Running ansible-playbook --check tells you what it would do
● Upgrade ELK Stack to 7.x+ (currently at 6.8.x)
● Support multi-node deployments
ansible-elk roadmap
We use ELK Stack for recording QUADS data
● Manages bare-metal systems and network switch automation based on schedules set in the future
● Powers automation within the Red Hat Scale Lab
● https://quads.dev● github.com/redhat-performance/quads
QUADS is an automation framework
● github.com/sadsfae
● https://hobo.house
● Twitter: @sadsfae
● Freenode IRC: sadsfae
Thank you for attending!
Questions, Comments, Discussion?