How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

59
How to choose tools for DevOps and Continuous Delivery Matthew Skelton, Skelton Thatcher Consulting DevOps Cardiff meetup group, 5 th November 2014, Cardiff, Wales #devopscardiff

description

With an ever-increasing array of tools and technologies claiming to 'enable DevOps', how do we know which tools to try or to choose? In-house, open source, or commercial? Ruby or shell? Dedicated or plugins? It transpires that highly collaborative practices such as DevOps and Continuous Delivery require new ways of assessing tools and technologies in order to avoid creating new silos. Matthew Skelton shares his recent experience of helping many different organisations to evaluate and select tools to facilitate DevOps; the recommendations may surprise you.

Transcript of How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Page 1: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

How to choose tools for DevOps and

Continuous Delivery

Matthew Skelton, Skelton Thatcher Consulting

DevOps Cardiff meetup group, 5th November 2014, Cardiff, Wales

#devopscardiff

Page 2: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Collaboration

Learning

Singleton tools

Conway’s Law

Page 3: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Matthew Skelton

•15 years building & operating software systems

•Cybernetics + Neuroscience• control engineering

• psychology

• ‘network’ interactions

@matthewpskelton

Page 4: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

PIPELINE Conference

London Continuous Delivery meetupgroup (#londoncd)

Page 5: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

We help you to transform your technology and teams

for ‘the cloud’

Interim CTO/Head of X, tech strategy, architecture, workshops, delivery

Page 6: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Recent clients

•Legal

•Donations

•Tourism

•Betting & gambling

•Travel booking

•Financial data & reporting

•Healthcare

Page 7: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Common themes•Online since ~1999

•Successful in their sector

•Large, central core database(s)

•Non-aligned goals

•Need to adopt DevOps and Continuous Delivery

Page 8: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Common needs

•Technology selection ticklists

•Team interactions

•Collaboration opportunities

•Tools as catalysts

Page 9: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 10: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Continuous Delivery

•A scientific approach to changing software systems (Dave Farley)

•Regular, rapid, repeatable, reliable changes

Page 11: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Continuous Delivery

•Quality

•Safety

•Reliability

•Psychology

•Effectiveness

Page 12: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 13: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

DevOps“Highly effective, daily collaboration between software developers and IT

operations people to produce relevant, working systems” *

*also QA/Testing, IT Service Desk, Programme Management,

Commercial, Marketing, etc.

Page 14: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Not DevOps

“Automation”

“Build & Release”

“Infrastructure Development”

“System Administration”

Page 15: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 16: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Collaboration

Page 17: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

1. Version Control

Page 18: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 19: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 20: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 21: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 22: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 23: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 24: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

2. Deployment Pipeline

Page 25: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

A large online retailer

•Travel sector

•Since ~1999

•UK market

•‘Non-core’ applications

Page 26: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Challenges were:

•Limited Git skills in Service team

•Manual deployments

•‘Snowflake’ servers

•No CI

•Risks: security, automation, etc

Page 27: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

http://bit.ly/thetrainline-weekly-deploy

Page 28: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

What we did•Built a walking skeleton pipeline

•Modelled security roles and stages

• Included manual steps (at first)

•Walked people through steps

•Finally: opened firewall so everyone could see the UI

Page 29: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Results

•Security: happy

•Service team: relieved (& happy)

•Developers: won over (& happy)

•Business: surprised (& happy)

Page 30: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

3. Log Aggregation

Page 31: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

LogStash

•ElasticSearch+ LogStash+ Kibana•ELK

•In Production, Pre-Prod, Test

•On developer machines!!!

Page 32: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

http://rashidkpc.github.io/Kibana/images/screenshots/searchss.png

Page 33: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 34: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Collaboration & tool choice

Value collaboration as a key criterion

Orthogonal to main purpose (?)

“How does [the use of] this tool help people to collaborate?”

Page 35: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Learning

Page 36: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 37: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 38: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 39: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 40: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Learning & tool choice

Bring people with you

Appreciate current skills

Prefer achievable gains now

Avoid fear of too-scary tools

Page 41: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Singleton tools(or the ‘Prize Bull’ approach)

Page 42: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 43: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Singleton tools

•Special database server

•Costly log aggregation

•Costly monitoring

•Server configuration

Page 44: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

http://www.walpapershddownload.com/highland-cattle-wallpapers/

Page 45: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

“Better features”?

Optimise globally across the teams that need to collaborate

Page 46: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Singleton tool

Breaks feedback (learning) loop from Production

Makes CI/CD more difficult

Underestimates value of collaboration and learning

Page 47: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Conway’s Law

Page 48: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Mel Conway, 1968

“organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations”

http://www.melconway.com/Home/Conways_Law.html

Page 49: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 50: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

DevOps Topologies

Page 51: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

DevOps Topologies

Page 52: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

DevOps Topologies

Page 53: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

DevOps Topologies

Page 54: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Conway & Tool Choice

See the organisation as a system

Separate tools for separate teams

Shared tools for collaborative teams

http://bit.ly/DevOpsTopologies

Page 55: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Conway’s Law•Allan Kelly - @allankellynet

•https://vimeo.com/channels/londoncd

Page 56: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff
Page 57: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

How to choose tools for DevOps

Value collaboration aspects

Avoid a learning mountain: evolve tooling

Avoid Production-only tools

Consider Conway’s Law

(this list is incomplete!)

Page 58: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Further readingBuild Quality In

buildqualityin.com70% of royalties donated to Code Club

Discount for @DevOpsCardiffpeople: http://leanpub.com/buildqualityin/c/DevOpsCardiff2014

Contributors include:- James Betteley- Marc Cluet- Anna Shipman- John Clapham

Page 59: How to choose tools for DevOps and Continuous Delivery - DevOps Cardiff

Thank youmatthewskelton.net / @matthewpskelton

skeltonthatcher.com

HT: @Squire_Matt, @alan_parkinson