Back From The Brink - Can we

Refactor Cookbooks and Nodes?

Hemal VarambhiaAbelsonInfo

4th November 2014

Me

● @hnvarambhia● github.com/hemalvarambhia● RoR, Android developer and Chef at

AbelsonInfo

Who We Are

● AbelsonInfo are a live sports data company - we provide JSON and XML APIs

● We mainly develop RoR (Ruby on Rails) applications

● Infrastructure managed by Chef (Hosted Chef)

● Using Cookbooks from the community● Put together ‘just get something to work!’● Hard to read● Untested?

Where We Were

● We needed to provision a production-like environment - CentOS

Then Came The Change

● 29 ubuntu servers already managed by Chef (10.x)

● Too afraid to refactor - little idea of the consequences

● Infracode Smells

The Situation

● What did the servers really require?

Buy The Ingredients

● Created a new Chef organisation● The legacy cookbooks under the old

organisation were still ‘working’● Wanted to develop our own cookbooks

A Clean Chopping Board

● Created a new Chef organisation● The legacy cookbooks under the old

organisation were still ‘working’● Wanted to develop our own cookbooks● TDD from the start - refactoring is a very

important step● Infracode will feed back into the business

and vice versa

A Clean Chopping Board

● We made sure to keep our cookbook code AND tests clean

● Could we apply this to community cookbooks?

● consistent code style

Clean Infracode

● We bootstrapped a new node under a new organisation to make sure it was successful - ran serverspec on the node

● Performed a dry run of moving a node to the new organisation

● A real node refactoring - the server should still run the rails app! Only 28 more to go!

Refactoring Nodes

● Design decisions are as important in DevOps● Discovered the need for an xUnit pattern!● Value of unit testing is in confirming guard clauses

and configuration published in template resources● express the tests in the language of the tool●

What We Learned

"Habitability is the characteristic of source code that enables programmers, coders, bug-fixers, and people coming to the code later in its life to understand its construction and intentions and to change it comfortably and confidently." (Richard Gabriel http://c2.com/cgi/wiki?EndUserHabitability)

Cookbook Quality?

I will show some code samples demonstrating these points

Here’s What We Made Earlier

# Determine which apps require the 'app' tier to be set up on this node:

tier_apps = ai_stack.apps_for_tier(node, 'app')

# Initialize an array to store user names that will later be added to the

# "sshlogin" group:

sshlogin_users = []

# Perform configuration required by all apps:

unless tier_apps.empty?

# Ensure all dependencies necessary to build Ruby are present:

include_recipe 'build-essential'

# Install packages useful for apps that use the asset pipeline functionality.

include_recipe 'ai_stack::asset_pipeline'

…

Here’s What We Made Earlier

#

# Cookbook Name:: greyhounds

# Recipe:: default

#

# Copyright 2014, AbelsonInfo

#

# All rights reserved - Do Not Redistribute

#

include_recipe 'rails_app_server::default'

nginx

varnish

postgresql

install_ruby

Here’s What We Made Earlier

Clean Tests - Domain AssertionsRSpec::Matchers.define :permit do |ip_addresses|

match do |content|

escaped_url = Regexp.escape @url

permitted_ip_addresses = ip_addresses.select { |ip_address|

content=~/location #{escaped_url} \{[^}]+allow #{ip_address};/m

}

permitted_ip_addresses == ip_addresses

end

chain :access_to do |url|

@url = url

end

failure_message_for_should do |content|

"expected Nginx to permit #{ip_addresses.join(", ")} access to '#{@url}'"

end

end

describe "AI office" do

describe "/feed URL" do

its(:content) {

should permit(client_machines).access_to("/feed")

}

end

end

Clean Tests - Usage

● Refactoring Katas - Take a cookbook from the supermarket and treat it as a refactoring kata

● TDD ping pong (with a dev) - try a ruby refactoring.

● Next meetups - examples of exemplary code

Ideas From The Dev Community

● cleancoders.com - episodes 21 - 24 are about clean TDD

● https://github.com/emilybache - some nice refactoring katas here to get going

● Ruby Refactoring Book (co-authors by Martin Fowler)● POODR (Sandi Metz)● xUnit Test Patterns by Martin Mezaros

Useful Resources

Who’s For Dessert?

This man was whipping up awesome at atomic scales.

Now For Dessert

Thanks to: 1. Tim Mackinnon2. Engineering Team @ AbelsonInfo3. Alex G. Harvey

Acknowledgements