devops pulse 2017 final - Logz.io · This report is the second annual DevOps Pulse, a study...
Transcript of devops pulse 2017 final - Logz.io · This report is the second annual DevOps Pulse, a study...
The 2017
DevOps Pulse
Who Took This SurveyWho Took This Survey Pages 4-5Pages 4-5
DevOps CultureDevOps Culture Pages 6-11Pages 6-11
DevOps SalariesDevOps Salaries Pages 12-13Pages 12-13
ArchitectureArchitecture Pages 14-16Pages 14-16
Code DeploymentCode Deployment Page 17Page 17
Monitoring and LoggingMonitoring and Logging Pages 18Pages 18
Security and Cost ManagementSecurity and Cost Management Pages 19-20Pages 19-20
This report is the second annual DevOps Pulse, a study published by Logz.io that researches and captures important trends in IT operations and DevOps engineering. This year, the report was created based on the data from a survey of 700 respondents throughout the world and includes a special additional section on DevOps culture specifically because that topic is one that is not researched often enough despite being a huge part of what DevOps is.
Whats is The DevOps Pulse?Whats is The DevOps Pulse?
DevOps is still considered a relatively new phenomenon. 50% of companies are still in the process of implementing DevOps or have done it within the past year.
The three hardest parts of establishing DevOps are shifting company incentives to change behaviors, maintaining open communication and transparency, and having flexible processes.
The top problems in DevOps implementation are a lack of time and a lack of expertise.
90% of respondents stated that DevOps and increased collaboration have had a positive effect on the quality of their products. 75% of respondents say that DevOps has shifted their perspectives on their roles and the roles of others on their teams.
Burnout remains a serious issue. 70% of respondents stated that they could see themselves burning out.
DevOps Pulse 2017 Summary & Key FindingsDevOps Pulse 2017 Summary & Key Findings
IntroductionIntroduction
Who are we?Who are we?Logz.io is an AI-powered log analytics company that offers the open source ELK Stack as an end-to-end cloud service powered by machine learning algorithms. We support hundreds of DevOps organizations worldwide. If you have questions, we invite you to e-mail [email protected].
@logzio@logziohttps://logz.io [email protected]
11.57%
34.57%
30.00%
18.57%
28.86%
19.43%
4.71%
0% 20% 40% 60% 80% 100%
Other
Developer
DevOps Engineer
Architect
SysAdmin
IT Manager
Site Reliability Engineer
What is the size of your company?What is your role in the company?
15.29%
7.43%
13.00%
8.00%
21.57%
19.00%
6.43%
3.57%
5.71%
0% 20% 40% 60% 80% 100%
10000+ employees
5000-9999 employees
1000-4999 employees
500-999 employees
100-499 employees
20-99 employees
10-19 employees
5-9 employees
1-4 employees
What industry does your company belong to? What is the size of your team?
10.43%
2.86%
7.43%
5.43%
6.00%
7.14%
4.57%
0.71%
2.86%
2.29%
41.86%
2.43%
6.00%
0% 20% 40% 60% 80% 100%
OtherManufacturing
Professional ServicesGovernment
EducationFinance
ConsumerMaterials
TransportationEnergy & Utilities
TechnologyNon-profitHealthcare
22.57%
35.71%
29.86%
11.86%
0% 20% 40% 60% 80% 100%
10 and above
5-9
2-4
It's just me!
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
What is your experience level? Where in the world do you work?
6.88%
15.47%
17.05%
60.60%
0% 20% 40% 60% 80% 100%
Up to 2 years
3-5 years
6-9 years
10 years and above
9.32%
1.46%
4.08%
10.33%
16.45%
58.37%
0% 20% 40% 60% 80% 100%
East Asia
South America
Russia and Eastern Europe
Africa and Middle East
Western Europe
North America
Out of the 700 people who participated in The Logz.io DevOps Pulse 2017, most were developers, DevOps engineers, or system administrators in North America who work in
companies in the technology industry with up to 500 employees. Most have ten or more years of experience and work in teams of nine or fewer people.
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
When did your company implement DevOps? How long did implementation take?
34.46%
15.54%
27.48%
9.01%
13.51%
0% 20% 40% 60% 80% 100%
In process
1 year ago
2-3 years ago
4 years ago
5 years ago or more
20.05%
22.00%
29.83%
28.12%
0% 20% 40% 60% 80% 100%
More than 2 years
1-2 years
6-12 months
Up to 6 months
Who helped ensure that implementation ran smoothly? What is your background?
Implementing DevOps
40.09%
5.53%
16.36%
27.42%
10.60%
0% 20% 40% 60% 80% 100%
No one specific
External consultant
Middle manager
Senior manager
CIO
6.62%
41.50%
3.97%
47.90%
0% 20% 40% 60% 80% 100%
Other
Operations
Testing
Development
DevOps is still considered a new phenomenon. 50% percent of respondents are still implementing DevOps or have done it within the past year.
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
What are the hardest parts of establishing DevOps? What are the key issues you’ve experienced in DevOps implementation?
Are you currently using ChatOps?
DevOps Difficulties
4.37%
24.03%
33.50%
36.41%
35.68%
30.10%
20.87%
31.55%
15.53%
39.08%
32.04%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Other
Gaining the support of the executive leadership
Instilling a problem-solving culture
Maintaining open communication and transparency
Having flexible processes
Putting developers on-call
Creating empathy among everyone
Making operations staff less fearful of failures
Eliminating feature guilt among developers
Shifting company incentives to change behaviors
Getting developer buy-in
4.66%
51.75%
41.26%
37.30%
51.28%
0% 20% 40% 60% 80% 100%
Other
Lack of expertise
Lack of resources
Lack of communication
Lack of time
83.93%
16.07%
0% 20% 40% 60% 80% 100%
No
Yes
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
The most difficult aspects of establishing DevOps including shifting company priorities, getting developer buy-in, and maintaining open communications. These three actions -- among others -- are often tied to the fact that it is very hard to escape the typical
silos that develop within companies and teams. Everyone and every team has a different way of working and a different set of priorities to address and goals to meet.
DevOps Difficulties
How effective are you at eliminating developer-operations silos?
13.88%
38.82%
29.41%
17.88%
0% 20% 40% 60% 80% 100%
Very effective
Somewhat effective
A little effective
No silos is possible only in theory
How effective are you at creating autonomous teams?
19.63%
47.66%
20.79%
11.92%
0% 20% 40% 60% 80% 100%
Very effective
Somewhat effective
A Little effective
Possible only in theory
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
DevOps Burnout
While a plurality of respondents indicated that they feel a moderate amount of pressure at work, 70% did report that they could see themselves burning out at some point.
On the day to day, how much pressure do you feel at work? (1 being not stressed at all, 5 being very stressed)
Could you see yourself burning out?
9.46%
24.77%
40.54%
18.92%
6.31%
0% 20% 40% 60% 80% 100%
5
4
3
2
1
31.76%
29.95%
38.29%
0% 20% 40% 60% 80% 100%
Maybe
No
Yes
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
Would you say DevOps has contributed to greater overall agility? How collaborative is your work environment? (1 being least collaborative, 5 being most)
Has DevOps shifted your perspective on your role and therole of others within your team?
DevOps Results
3.64%
20.96%
21.18%
5.24%
48.97%
0% 20% 40% 60% 80% 100%
Usually not
Sometimes
Usually
No
Yes
4.73%
10.59%
34.91%
38.74%
11.04%
0% 20% 40% 60% 80% 100%
1
2
3
4
5
29.13%
25.69%
45.18%
0% 20% 40% 60% 80% 100%
Somewhat
No
Yes
Do you and your team members (whether developers or operations staff) share common goals?
3.40%
27.89%
26.08%
4.76%
37.87%
0% 20% 40% 60% 80% 100%
Rarely
Usually
Sometimes
No
Yes
@logzio@logziohttps://logz.io [email protected]
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
Has DevOps adoption led to an increase in innovation? How much emphasis is placed on communication?
How has DevOps and increased collaboration affected thequality of products produced?
DevOps Results
Is speed or quality more important as a result of DevOps?
33.95%
23.09%
42.96%
0% 20% 40% 60% 80% 100%
Somewhat
No
Yes
4.35%
19.68%
46.68%
29.29%
0% 20% 40% 60% 80% 100%
None
A bit
A decent amount
A lot
40.23%
49.30%
10.47%
0% 20% 40% 60% 80% 100%
Significant impact
Small impact
No impact
7.59%
51.49%
28.05%
12.87%
0% 20% 40% 60% 80% 100%
Neither is more important
Both
Quality
Speed
75% stated that DevOps has at least somewhat changed their perspectives of their roles and those of others on their teams. Almost everyone reported that everyone on DevOps teams,
from developers to operations, shares common goals at least some of the time. 75% also responded that DevOps has led to an increase in innovation. 90% reported that DevOps and
increased collaboration has had a positive effect on the quality of their products.
@logzio@logziohttps://logz.io [email protected]
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
What is your annual salary? (North America) What is your annual salary? (Western Europe)
What is your annual salary? (Russia and Eastern Europe) What is your annual salary? (Africa and the Middle East)
23.74%
21.46%
21.97%
32.83%
0% 20% 40% 60% 80% 100%
$120,000 and above
$100,000-$120,000
$80,000-$100,000
$50,000-$80,000
6.80%
2.91%
9.71%
80.58%
0% 20% 40% 60% 80% 100%
120,000 EURO and above
100,000-120,000 EURO
80,000-100,000 EURO
50,000-80,000 EURO
0.00%
11.11%
3.70%
85.19%
0% 20% 40% 60% 80% 100%
100,000 EURO and above
80,000-100,000 EURO
60,000-80,000 EURO
40,000-60,000 EURO
41.18%
20.59%
17.65%
20.59%
0% 20% 40% 60% 80% 100%
100,000 USD and above
80,000-100,000 USD
60,000-80,000 USD
40,000-60,000 USD
@logzio@logziohttps://logz.io [email protected]
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
What is your annual salary? (East Asia) What is your annual salary? (South America)
In North America, 45% of DevOps engineers now earn at least $100,000 per year. More than 65% earn more than $80,000. In western Europe, 80% make between
€50,000 and €80,000 per year. In Russia and eastern Europe, 85% earn the equivalent of between €40,000 and €60,000 per year.
In Africa and the Middle East, the results were more uniform -- 58% reported incomes of below the equivalent of $100,000 per year and 41% reported higher salaries. In
eastern Asia, 61% make between the equivalent of $40,000 and $60,000 per year. In South America, every respondent reported salaries of between the equivalent of
$40,000 and $60,000 per year.
11.11%
11.11%
16.67%
61.11%
0% 20% 40% 60% 80% 100%
100,000 USD and above
80,000-100,000 USD
60,000-80,000 USD
40,000-60,000 USD
0.00%
0.00%
0.00%
100.00%
0% 20% 40% 60% 80% 100%
100,000 USD and above
80,000-100,000 USD
60,000-80,000 USD
40,000-60,000 USD
@logzio@logziohttps://logz.io [email protected]
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
What is the size of your infrastructure? (servers/instances) What architecture are you using to buildyour application?
Are you using serverless in your application logic? Do you use container services?
14.98%
10.77%
21.89%
45.62%
6.73%
0% 20% 40% 60% 80% 100%
More than 1000
500-1000
100-500
5-100
1-4
Microservices and Containers
65.99%
15.32%
18.69%
0% 20% 40% 60% 80% 100%
A mixture of both
Monolithic
Microservices
30.55%
69.45%
0% 20% 40% 60% 80% 100%
Yes
No
55.89%
44.11%
0% 20% 40% 60% 80% 100%
Yes
No
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
Do you use container orchestration services, and if so - which?
Microservices and Containers
8.06%
14.24%
1.37%
15.27%
3.09%
57.98%
0% 20% 40% 60% 80% 100%
Other
Yes, Docker Swarm
Yes, Nomad
Yes, Kubernetes
Yes, Mesos or DC/OS
No
90% reported that they are now using microservices to some degree. More than half are not using container services, but Kubernetes and Docker Swarm are leading the
pack among those who do.
@logzio@logziohttps://logz.io [email protected]
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
What percentage of your infrastructure is based in the public cloud?
Which cloud are you using? (you can choose more than one)
The Cloud
21.72%
11.95%
11.78%
54.55%
0% 20% 40% 60% 80% 100%
All of it
50-75%
25-50%
0-25%
13.32%
6.37%
4.05%
2.51%
3.67%
27.61%
18.73%
70.66%
0% 20% 40% 60% 80% 100%
Other
Digital Ocean
Oracle
Docker Datacenter
IBM Softlayer
Azure
Google Cloud Platform
AWS
One interesting story is the ever-increasing dominance of the Big Three public cloud companies -- AWS, Azure, and Google Cloud Platform. In The DevOps Pulse 2016, 28% stated that they used an “other” platform. This year, the number fell to 13% while the percentages of
people using the Big Three each increased. More people in general are using AWS, Azure, or Google Cloud Platform. As might not be surprising, AWS still leads the pack.
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
How frequently does your team deploy code to production? Do you have a Continuous Integration (CI) strategy in place?
Do you have a Continuous Deployment (CD) strategy in place? How do you do QA in your organization?
13.22%
32.17%
36.87%
17.74%
0% 20% 40% 60% 80% 100%
Once every couple of months
Once every couple of weeks
A few times a week
Multiple times a day
11.30%
17.57%
20.00%
51.13%
0% 20% 40% 60% 80% 100%
No, but we're thinking about it
No
We're in the process of implementing
Yes
13.39%
23.48%
28.35%
34.78%
0% 20% 40% 60% 80% 100%
No, but we're thinking about it
No
We're in the process of implementing
Yes
4.35%
68.70%
4.70%
22.26%
0% 20% 40% 60% 80% 100%
We don't do QA
Both manual and automated
100% automated
Manual
54% of of survey takers are deploying code to production between a few times a week and multiple times per day. 71% of people now have a continuous integration (CI)
strategy. 63% have a continuous deployment (CD) strategy. Almost 70% use a mix of manual and automated QA.
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
What’s your most common use case for log analytics? How do you distribute alerts in your organization?
As we found last year, the top three use cases for log analytics (in descending order) are troubleshooting, alerting, and security. Slack devotees might be shocked to hear
that 86% of people still prefer to receive alerts over old-fashioned e-mail.
2.93%
30.95%
25.64%
56.41%
79.49%
0% 20% 40% 60% 80% 100%
Other
Security and compliance
Business intelligence
Alerting
Troubleshooting and forensics
11.36%
4.58%
2.38%
0.92%
37.18%
17.03%
85.71%
0% 20% 40% 60% 80% 100%
Other
OpsGenie
VictorOps
Sense
Slack
Pagerduty
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
Who is responsible for security in your organization? Which tools do you use for security monitoring?
After the open source ELK Stack, the most popular security monitoring tools are Nagios, Nmap, Splunk, and a collection of other lesser-known or
custom-built platforms.
Security Operations
2.92%
27.10%
10.14%
36.84%
23.00%
0% 20% 40% 60% 80% 100%
Site reliability engineering
System administration
Development
Security operations
DevOps
28.08%
36.70%
15.76%
8.62%
3.45%
10.59%
21.43%
2.71%
32.51%
6.65%
2.96%
2.96%
0% 20% 40% 60% 80% 100%
Other
ELK
Splunk
Spiceworks
OSSIM
OSSEC
Nmap
NetFort
Nagios
LANSweeper
GlassWire
FireGen
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
Cost Management
In your organization, who monitors the cost of production? If you use a public cloud, what priority does cost management have?
If you do practice cloud cost management, what tools do you use? If you have specific initiatives around cost management, what are your goals?
While the initial rush to the cloud was in part to cut costs, the cloud still becomes expensive as companies’ underlying infrastructures grow with the business. As a
result, businesses are using popular cost management platforms such as Cloud Native, CloudHealth, and a batch of others including AWS itself and custom-built tools. The
most common reasons for cost management are to understand cloud usage and optimize environments.
12.47%
15.49%
20.72%
14.08%
37.22%
0% 20% 40% 60% 80% 100%
Other
VP Engineering
Head of DevOps
CFO
CTO
6.25%
50.00%
21.88%
21.88%
0% 20% 40% 60% 80% 100%
We worry about cost much more than stability
Price is an important factor
Price is a small factor
Stability comes first, so cost is not an issue
31.20%
35.47%
23.08%
9.83%
16.24%
0% 20% 40% 60% 80% 100%
Other
Cloud Native tools
CloudHealth
Cloudyn
Cloudability
5.54%
57.26%
24.80%
69.92%
0% 20% 40% 60% 80% 100%
Other
Understanding our usage
Buying reserved instances from public cloud providers
Optimizing our environment
Who Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud ManagementWho Took This Survey DevOps Culture DevOps Salaries Architecture Code Deployment Monitoring and Logging Security and Cloud Management
@logzio@logziohttps://logz.io [email protected]
Questions on The DevOps Pulse?Questions on The DevOps Pulse?Logz.io aims for this report to become more in-depth and comprehensive every year. So, we invite feedback from the community on what you would like to see in 2018.
Are some topics less relevant? Would you like more detail on a specific subject? Is there an area that we have not covered at All?
Please e-mail [email protected] with any questions about Logz.io as well as any thoughts and ideas, and we will incorporatethem into next year’s report.
@logzio@logziohttps://logz.io [email protected]