I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14....
Transcript of I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14....
![Page 1: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/1.jpg)
I AM SYSADMIN(AND SO CAN YOU!)
Friday, November 14, 14
![Page 2: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/2.jpg)
April 2014
Friday, November 14, 14
![Page 3: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/3.jpg)
Musings: Rick Farrow“Like Elizabeth Zwicky writes in this issue, I
am not a system administrator. Oh, I can still fake it, by managing my own systems (DNS, SMTP, DHCP, and HTTP) and
occasionally even consulting locally. But I gave up on being a sysadmin when I found out that I wasn’t very interested in a very important aspect of it: building scripts and
systems to automate the work that needed to be done routinely.”
Friday, November 14, 14
![Page 4: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/4.jpg)
The Death of System AdministrationTodd Underwood
“We are in the final days of system administration.”
Friday, November 14, 14
![Page 5: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/5.jpg)
How to Be a Better System Administrator and Then Something ElseElizabeth Zwicky
“I once was a system administrator. Now I’m not.”
Friday, November 14, 14
![Page 6: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/6.jpg)
Friday, November 14, 14
![Page 7: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/7.jpg)
With these skills: You can enable:
Friday, November 14, 14
![Page 8: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/8.jpg)
Stigma?• SysAdmin’s apparent are:
• Grumpy grey-beards & BOFH
• Stuck in the dragon-slayer past & epic story tellers
• “Tape Swappers” who love non-automatable repetitive tasks
• Unable and/or Unwilling to code
• A junior level person that may become more one day
Friday, November 14, 14
![Page 9: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/9.jpg)
Friday, November 14, 14
![Page 10: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/10.jpg)
Contributing Factors• “Automate yourself out of a job” & “Keep it up”
are the oldest SA axiom, but tools to do this were poor, complex, or inefficient in the past
• We used products as designed
• We were (sometimes) trained by the vendor & utilized vendor support
• There was a “right way” and “wrong way”
• The server room was a walled garden, and we were the gardeners
Friday, November 14, 14
![Page 11: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/11.jpg)
Increasingly, these aren’t true anymore.
Friday, November 14, 14
![Page 12: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/12.jpg)
What SA’s Do
• System Admins:
• Build and maintain systems to facilitate the needs of a group of users (directly or indirectly)
• Support the users of the system to efficiently accomplish tasks
Friday, November 14, 14
![Page 13: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/13.jpg)
Word Games
• No-Ops
• Post-Ops
• DevOps
• SRE
These are philosophies of operation.
Friday, November 14, 14
![Page 14: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/14.jpg)
Stop the insanity!
Friday, November 14, 14
![Page 15: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/15.jpg)
What they mean• “Cloud” changes everything
• Most of the world is software defined, and software is malleable
• Automate all the things
• Write more code (software solves every problem)
• Every company should be as l337 as Google, Facebook, Netflix, etc.
Friday, November 14, 14
![Page 16: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/16.jpg)
Why didn’t I think of that?
Friday, November 14, 14
![Page 17: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/17.jpg)
The Dev Reality
• Devs Aren’t Helpless Anymore
• Services are the new Libraries (Dep Management)
• Ops can not bottleneck development anymore
Friday, November 14, 14
![Page 18: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/18.jpg)
The Ops Reality
• Most shops consist of 3 or less admins
• Daily stresses make project work nearly impossible (not enough time)
• Frequently IT & Ops are lumped together
• Ops technology moves simultaneously too fast and too slow (eg: Nagios & Ansible)
Friday, November 14, 14
![Page 19: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/19.jpg)
The Dilemma• Yes, Cloud changes the game.
• Yes, Software is more important than ever.
• Yes, I need to more with less and get out of developers way
• But, who builds the cloud(s)?
• But, who runs the cloud(s)?
• But, how do I keep up with the technology?
Friday, November 14, 14
![Page 20: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/20.jpg)
SysAdmin isn’t Dying.
It is Evolving.
Friday, November 14, 14
![Page 21: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/21.jpg)
Friday, November 14, 14
![Page 22: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/22.jpg)
Why So Hard?
SA’s are expected to not just operate the business but to also build the systems used to operate the business
Friday, November 14, 14
![Page 23: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/23.jpg)
Requirements for Change
• Freedom to innovate and evolve means embracing failure and learning... which means culture & management maters!
• Acquiring the skills takes time... which means support & education are important!
Give us an environment in which we can evolve.Friday, November 14, 14
![Page 24: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/24.jpg)
So how do we do that?
Friday, November 14, 14
![Page 25: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/25.jpg)
Divide & Conquer
• If you have legacy, make a concerted effort to eliminate it
• Adopt SaaS where you can
• Use the public clouds where you can
• Build & Run internal clouds (*aaS)
Friday, November 14, 14
![Page 26: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/26.jpg)
Development & Deployment are changing
• Vagrant is awesome (IaaS Model on a laptop)
• Docker is awesome (Repeatable Laptop to Production App Container)
• Config Management isn’t just for Ops
• Self-Service Infrastructure is a signup away (Cloud)
Friday, November 14, 14
![Page 27: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/27.jpg)
Small is the new Big• Micro-Services (neo-SOA) & Distributed
Architectures are awesome, particularly when containerized
• New Essential skills:
• RESTful APIs (curl & OO Ninja)
• Message Queues (RabbitMQ)
• Service Coordination & Discovery (ZooKeeper/etcd & Consul)
Friday, November 14, 14
![Page 28: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/28.jpg)
PERL & PHP
• I have seen PERL6... its Ruby
• Python & Ruby should be the standard languages of the admin today
• Go is the language for Ops
• Everyone should know a little JavaScript & how to use Node.js
• BASH & AWK still rule though
Friday, November 14, 14
![Page 29: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/29.jpg)
APIs are the new
CLI
Friday, November 14, 14
![Page 30: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/30.jpg)
HTML has changed
• Update on HTML5 & CSS3 Basics
• Try out client-side frameworks (Angular)
• Try out Node.js for back-end services
Friday, November 14, 14
![Page 31: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/31.jpg)
Metrics!
• Lots of great SaaS solutions:
• Circonus, Librato, DataDog, etc.
• Graphite is the go-to solution
• Watch this space!
Friday, November 14, 14
![Page 32: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/32.jpg)
Logging
• Splunk is still great
• JSON logging formats are here to stay
• ELK is.... high maintenance, but neat
• Logstash is your swiss army knife
• Great SaaS solutions available:
• ... but just use Sumologic
Friday, November 14, 14
![Page 33: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/33.jpg)
Discover KV Stores
• NoSQL makes little sense from a non-programatic viewpoint
• Play with MongoDB or LevelDB from JavaScript (Node.js)
• Discover the joy of caching with Redis!
Friday, November 14, 14
![Page 34: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/34.jpg)
Git
• Learn Git
• ... Learn Git better (Lynda.com)
• Use Github without shame!
Friday, November 14, 14
![Page 35: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/35.jpg)
Continuous Integration
• Play with Jenkins!
• (Its not as magical or complicated as you might think)
• Check out Travis for Github (travis-ci.org)
Friday, November 14, 14
![Page 36: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/36.jpg)
Standardization
• Document procedures
• ... then automate it down to a single step
• ... then remove that step
• You can’t improve what you don’t understand
• Iterate from current-state to future-state
Friday, November 14, 14
![Page 37: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/37.jpg)
Compliance Basics
• Know what controls, policy, and procedures are and how they fit together
• Learn about information classification (FIPS 199)
• Know the risk assessment basics (NIST SP 800-39)
Friday, November 14, 14
![Page 38: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/38.jpg)
Be a Leader
• Share knowledge
• Take the initiative
• Make the time to improve yourself & your infrastructure
• Empower your colleagues across the organization
Friday, November 14, 14
![Page 39: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/39.jpg)
Don’t be a hater
• Encourage peers to try new things!
• Share successes!
• Embrace failure and learn together!
• Assholes kill innovation
• Try & Appreciate everything
Friday, November 14, 14
![Page 40: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/40.jpg)
Sites that can help• Ops School (opsschool.org)
• Lynda.com
• Pluralsight.com
• SafariBooksOnline.com
• CodeSchool.com
• Ask questions on Google
• Watch presentations on YouTube & Vimeo
Friday, November 14, 14
![Page 41: I AM SYSADMIN - USENIX · Friday, November 14, 14. ... Now I’m not.” Friday, November 14, 14. Friday, ... • “Tape Swappers” who love non-automatable repetitive tasks](https://reader035.fdocuments.in/reader035/viewer/2022070610/5af8a6a27f8b9a19548ba93f/html5/thumbnails/41.jpg)
$SYSTEM admin for life.
Ben Rockwood@benrhttp://cuddletech.com
Friday, November 14, 14