Running Operations In Two Hours a Day
41
RUNNING OPERATIONS IN 2 HOURS Erno Aapa ernoaapa
Transcript of Running Operations In Two Hours a Day
RUNNING OPERATIONS IN 2 HOURS!
Erno Aapa " ernoaapa
201520102005
Continuous Delivery
Kanban DevOps DevOps Finland
Dockercontainers
Lean Startup Lean
Cloud
There must be a better way! Continuous
IntegrationConfiguration Management
Scrum
Infrastructure Automation
" ernoaapa
CleanCode
DevTernity.
Startup
MesosKubernetes
Terraform
Open DevOps Engineer position
Continuous Integration, Continuous delivery,
deployment automation, Agile, Scrum, Kanban, test
automation, Cloud, DevOps, Infrastructure automation,
Configuration Management, RDBMS, No-SQL, CDN,
network automation, Docker, containers, Hardware, ...
Responsibilities:
- Fullstack engineer
- Operations, network
- Infrastructure, sysadmin
- Team Manager,
- Be CIO, CTO, ...
132 new mail
KNOW EVERYTHING,
DO EVERYTHING!
" ernoaapa
I'm DevOps? NO!" ernoaapa
Devops!=
one man
two jobswithwith
ulture
utomation
Easure
haring
This DevOps means for me!
C
A
M
S
" ernoaapa
" ernoaapa
This is your product...
... and the crew is your team!
Make yourself replaceable!
" ernoaapa
Refer to putting one person or thing in place of another
replaceable /rɪˈpleɪs/
" ernoaapa
No time for learning
and innovation
Repeat same task over and
over again
No progress in your career
High risk to become
unemployed
You might undermine
whole company
Irreplaceable
" ernoaapa
Being IRREplaceable is not good to you
• No time for learning because you use all your time doing same task over and over again
• You don't develop because you don't have time for learning and innovation
• You don't progress in your career because you company cannot move you off from the task
• You become useless when product/service get changed and your tasks disappear
• You end up being unemployed with outdated knowledge of some legacy system what no one needs
" ernoaapa
IRREplaceable is not good for company
• Your business is very vulnerable - what if that guy dies? • That guy can use that to require high salary • Company knowledge doesn't grow and you end up having
outdated tools, practices and technologies • Company doesn't innovate and get stuck on single legacy
solution what new companies can outdo
" ernoaapa
Share your knowledge Share responsibility
Time for innovation
You become more productive
$Improve the process Teach others to solve the problems
" ernoaapa
" ernoaapa
StartupEnterprise %
" ernoaapa
MY PERFECT PLAN
" ernoaapa
MY PLAN OUR NEED
How I did cut off-80%
operation work?20%
" ernoaapa
- 20% Remove all unnecessary
-20%Remove legacy servers
Switch to cloud services
Remove all "I might need some day" -code
Clean up duplicate code
" ernoaapa
-30% Simplify, SIMPLIFY But don't over-Simplify!
-30%Remove rarely used features
Refactor codebase
Simplify technology stack
Remove legacy technologies
Split service to smaller pieces (micro services)
But DON'T force to single stack!
" ernoaapa
- 30% Make your self replaceable
-30%
Automate your work
Push responsibility to the team
Teach everything you know to everyone
Make sure team can work without you
" ernoaapa
Story of two companies
Intrepid
Tit
anic
From Ansible newsletter 9/15 / Matt Jaynes" ernoaapa
&&&&&&&&&&&&&&&&&&&& 20 servers
$60.000/Month
From Ansible newsletter 9/15 / Matt Jaynes
Revenue
Sysadmins
Servers
Operation costs
' $5M/Year
$$$ 3 Administrators
Titanic
" ernoaapa
''''''''''''''''''''''' $115M/Year
From Ansible newsletter 9/15 / Matt Jaynes
Revenue
Sysadmins
Servers
Operation costs
INTREPID
$ ¼ Developer
&&&& 4 Servers
$2.500/Month
" ernoaapa
From Ansible newsletter 9/15 / Matt Jaynes
Intrepid
Titanic
Operation costs per $1M revenue
$144 000$260
That's over 500 times more!
" ernoaapa
Buy as-a-Service all you can!
" ernoaapa
$100 Developer / day Costs for the company
5 Days To setup server
$500 Total costs To setting up costs
+$200 Costs / Month Server and maintenance
" ernoaapa
Self build and host cost
$100 Total costs To setting up costs
+$99 Costs / Month Grows by needs and usage
" ernoaapa
"Buy-as-a-service" cost
$100 Developer / day Costs for the company
1 Days To setup server
-400$!
-101$!
Saas vs self hostedCo
sts
Time
Self hosted
SaaSHuge upfront invest of time and money
to get started
Startup rebuilds the stack
here anyway!
" ernoaapa
Force team to not depend on you!
" ernoaapa
$Developer
!Production
$Test
Manager
$Release Manager
$Operations Engineer
Common enterprise deployment pipeline
Tests (manually) that new version
works as expected
Decides when change
can be deployed
Deploy new version to
production
Risk!
Risk!
Risk!
Risk that something goes wrong,
get misunderstood or work queue up!
" ernoaapa
$Developer
!Production
Provides environment where is easy to deploy,
get visibility and it's easy to maintain
$Fellow
Developer
Review code changes and give feedback
$Test
Manager
Maintain testing automation and help with tests
$Operations Engineer
Automatio
n!
Continuous delivery pipeline
" ernoaapa
" ernoaapa
My man, Hubro!Hubot: https://hubot.github.com/
My dad!
I'll deploy "frontweb" project branch "feature-searchbar" to production Contains commits:- Add new search API to backend- Implement searchbar to the front page
Reviewed: ( Approved: (
Chatops
hubro deploy frontweb/feature-searchbar
) Type to chat...
*Developer
Deployment of "frontweb" project branch "feature-searchbar" completed! +
Apply same practice for server
configuration changes too!
" ernoaapa
Hubro
Hubro
@team Alert: response time change +20%
Chatops
hubot show production cpu usage
) Type to chat...
*Anyone
Production CPU usage last 7h
" ernoaapa
Hubro
Hubro
How can I make team to own the monitoring?
" ernoaapa
Alert!
" ernoaapa
+ +MeasureCollect metrics from your code
easily with Statsd
DetectAdd Sensu checks into your project
repository in JSON file
AlertEveryone is in paging team to
wake up for alerts
" ernoaapa
frontend.git,
- sensu/checks. page_size.json
backend.git,
- sensu/checks. response_time.json
operations.git,
- sensu/checks. cpu_usage.json
- src/javascript - src/java - src/ansible
Team maintains their own checks and alerts in their
codebase like they do with code
No need for strange UI, take the settings to place where
the team is on their own!
" ernoaapa
1 Do I really need it?
2 Can I buy it?
3 Can machine do it?
4 Can somebody else do it?
5 Ok, I'll do it!
Ask from yourself before doing anything!
Build simple software what it easy to understand. Write tests so it's easy to
maintain and make sure that it's transparent and easy to operate.
MAKE YOUR SELF REPLACEABLE/
" ernoaapa
Make the infrastructure as simple as possible and use SaaS services when it's possible.
Automate all changes and actions and push responsibility to solve issues to the team.
MAKE YOUR SELF REPLACEABLE/
" ernoaapa
Please give feedback sayat.me/ernoaapa
Thank you!
CreditsBackgrounds: Mehmet Demiray, Turkey "two companies"-story: Matt Jaynes, Ansible newsletter 9/15
" ernoaapa
