Post on 14-Feb-2017
© Continuity Software, 2015 2
Executive Summary
The goal of this benchmark survey is to provide IT executives with an understanding of how their peers
are using IT Operations Analytics to measure and meet their organization’s operational goals. The
results presented here are based on responses from over 200 IT professionals from a range of
industries and geographies collected through an online survey.
Some of the survey’s key findings include:
Uptime is the leading KPI for IT operations excellence, cited by 89% of the respondents. As the
leading KPI, uptime is tracked in real-time by 51% of the respondents, and daily by another 19%.
While 29% of the respondents consistently met or exceeded their KPI goals, the majority of the
organizations did not fare that well.
On average, organizations are able to detect and address just 57% of the critical IT issues
before they adversely impact the business.
Frequently tracking configuration consistency across more areas of IT operations correlates to
the ability of organizations to meet their KPI goals.
© Continuity Software, 2015 3
Over 80% of the organizations analyze configuration consistency across all IT domains with the
notable exception of the Cloud, where only 50% of the organizations analyze configuration
consistency.
47% of the large organizations surveyed use analytical tools to measure IT performance,
compared to only 32% of the smaller companies surveyed. Over three quarters (76%) of those
that use IT analytics tools find them helpful in early detection of critical IT issues.
© Continuity Software, 2015 4
Table of Contents
Key Performance Indicators for IT Operations Excellence ............................................................................................................... 5
Frequency of KPI Tracking ................................................................................................................................................................................ 6
Most Difficult KPIs to Meet ............................................................................................................................................................................... 7
Who Is Concerned with KPIs?.......................................................................................................................................................................... 8
Achieving KPI Goals .............................................................................................................................................................................................. 9
Use of Analytical Tools to Measure IT Performance .......................................................................................................................... 10
Where KPIs Are Monitored ............................................................................................................................................................................. 11
KPIs Measured across All or Most Domains .......................................................................................................................................... 12
Frequency of Analyzing Configuration Consistency across IT Operations Areas ................................................................ 13
Meeting KPI Goals vs. Frequency of Analyzing Configuration Consistency ............................................................................ 14
Measuring Replication, Failover, RPO and RTO Goals ....................................................................................................................... 15
Portion of Critical IT Issues Detected and Addressed before Impacting the Business .................................................... 16
How helpful are IT analytics in early detection of critical issues? ................................................................................................ 17
Transforming Insights into Improvements ............................................................................................................................................. 18
Transforming Insights into Improvements by Company Size ....................................................................................................... 19
Means to Achieve Operations Excellence ............................................................................................................................................... 20
Respondent Demographics ........................................................................................................................................................................... 21
Respondent Demographics ........................................................................................................................................................................... 22
© Continuity Software, 2015 5
Key Performance Indicators for IT Operations Excellence
Uptime is the leading KPI for IT
operations excellence, cited by
89% of the respondents, and
followed by performance (80%).
Other common KPIs for IT
operations include the number of
open issues (48%), average time to
fix (46%), security breaches (45%),
and data loss (43%).
Figure 1: KPIs for IT operations excellence
(respondents could select multiple options)
4%
31%
43%
45%
46%
48%
80%
89%
Other
Mean Time Between Failures (MTBF)
Data Loss
Security Breaches
Average Time to Fix
Number of Open Issues
Performance
Uptime
© Continuity Software, 2015 6
Frequency of KPI Tracking
The top two KPIs – uptime and
performance – are also the KPIs
most frequently tracked by IT
organizations. Uptime is tracked
in real-time by 51% of the
respondents, and daily by another
19%.
Performance and security
breaches are tracked in real-time
by 39% and 38% respectively.
Figure 2: Frequency of Tracking KPIs
51%
39% 38%
15% 14%6% 6%
19%
26%23%
32% 31%
23% 19%
9%14%
12%
16%26%
23%25%
15%11%
8%
13%
20%
23%18%
3%
2%11%
6%
3%
8%
7%
1% 1%
3%
1%
2%
1%
1%8%
7%15%
5%
15%23%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Uptime Performance SecurityBreaches
Data Loss Number ofOpen Issues
Average Timeto Fix
MTBF
real-time daily weekly monthly quarterly annually never
© Continuity Software, 2015 7
Most Difficult KPIs to Meet
According to survey respondents,
performance is the most difficult KPI
to meet (cited by 71%), followed by
uptime (51%) and data loss (40%).
Figure 3: Most difficult KPIs to meet
36%
17%9% 10% 8% 8%
21%
19%
12% 10% 12% 11%
15%
15%
19%13% 11%
9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Performance Uptime Data Loss SecurityBreaches
Number ofOpen Issues
MTBF
Most difficult 2nd most difficult 3rd most difficult
© Continuity Software, 2015 8
Who Is Concerned with KPIs?
In 77% of the organizations
surveyed the CIO is the executive
most concerned with IT
Operations KPIs, followed closely
by the Head of IT Infrastructure
(69%).
Figure 4: Concerned with KPIs
(respondents could select multiple options)
77%
69%
25% 25% 22% 20% 19%14%
7% 5%
© Continuity Software, 2015 9
Achieving KPI Goals
While 29% of the respondents
consistently met or exceeded their
goals, the majority of the organizations
did not fare that well.
46% met their goals most of the time,
and 13% were able to achieve only
some of their goals. 7% failed most of
the time and another 4% could not tell
whether they met their goals or not.
Figure 5: Achieving KPI goals
4%
2%
5%
13%
46%
24%
5%
We don't really know
Consistently failed to meet our goals
Did not meet our goals most of the time
Met some of our goals but not consistently
Met our goals most of the time
Consistently met our goals
Exceeded our goals
© Continuity Software, 2015 10
Use of Analytical Tools to Measure IT Performance
47% of the large organizations surveyed
use analytical tools to measure IT
performance, compared to only 32% of
the smaller companies surveyed.
Figure 6: Use of tools to measure IT performance by comapnay size
Number of employees in the organization
© Continuity Software, 2015 11
Where KPIs Are Monitored
Three quarters of the organizations
surveyed monitor their KPI’s over their
networks (79%) and databases (78%).
Other areas of IT operations where
KPIs are commonly monitored are
applications (72%), and storage (71%).
At the same time, KPI monitoring in the
Cloud environment is still lagging far
behind other areas of the
infrastructure (20%).
Figure 7: Monitoring IT operations areas for KPIs
(respondents could select multiple options)
79% 78%
72% 71%
45%
20%
Network Databases Applications Storage Clusters Cloud
© Continuity Software, 2015 12
KPIs Measured across All or Most Domains
As the leading KPI for IT operations
excellence, uptime is measured across
all or most domains in 85% of the
organizations surveyed.
Performance (71%) and number of open
issues (44%) are additional KPIs that are
commonly measured across all or most
domains.
Figure 8: KPIs measured across all or most domains
(respondents could select multiple options)
22%
36%
38%
42%
44%
71%
85%
MTBF
Average Time to Fix
Data Loss
Security Breaches
Number of Open Issues
Performance
Uptime
© Continuity Software, 2015 13
Frequency of Analyzing Configuration Consistency across IT
Operations Areas
Over 80% of the organizations analyze
configuration consistency across all
areas with the notable exception of the
Cloud, where only 50% of the
organizations analyze configuration
consistency.
Over 50% of the organizations analyze
configuration consistency across all
areas at least once a week, while only
30% do the same in the Cloud.
Figure 9: Frequency of analyzing configuration consistency
26% 25% 21% 20% 20%11%
21%15% 20%
17% 15%
12%
10%17% 17%
14% 18%
10%
10% 12% 10%
12% 10%
8%
6%7% 6%
9% 8%
5%
13% 8% 11% 11%10%
5%
14% 15% 16% 16% 19%
50%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Network Databases Storage Applications Clusters Cloud
real-time daily weekly monthly quarterly annually never
© Continuity Software, 2015 14
Meeting KPI Goals vs. Frequency of Analyzing Configuration
Consistency
Frequently tracking configuration
consistency across more areas of IT
operations correlates to the ability of
these organizations to meet their KPI
goals.
39% of the organizations that track
configuration consistency across 4-6 IT
operations areas in real-time or daily
are meeting or exceeding their goals,
compared to 28% of the organizations
that track only up to 3 areas.
Figure 10: Meeting KPI goals vs. frequency of analyzing
configuration consistency across IT operations areas
28%39%
46%
48%
17%
10%8%
3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0-3 4-6
Did not meet out goals
Met some of our goals but notconsistently
Met our goals most of the time
Consistently met or exceededour goals
Number of IT Areas Tracked in Real-Time or Daily
© Continuity Software, 2015 15
Measuring Replication, Failover, RPO and RTO Goals
60% of the respondents measure the
portion of critical data successfully
replicated either in real time (24%) or
daily (36%).
Virtualization, storage, and load-
balancing failover success are
measured either in real-time or daily
by over 40% of the respondents.
On the other side of the scale, only
29% of the organizations measure the
percentage of RPO and RTO objectives
met either in real time or daily.
Figure 11: Frequency of measuring goals
24% 22% 22% 21%
8% 7%
36%
22% 20% 22%
21% 22%
12%
11% 15% 13%
12% 12%
6%
7%
10%7%
12% 13%
3%
8%
8%
7%12% 12%
13%22%
14%21% 21% 21%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
real-time daily weekly monthly quarterly annually never
How these measurements were defined in the survey:
Replication: % of critical data successfully replicated
Storage: % of active-active storage operational
following a full site outage
Virtualization: % of critical VMs that fail-over
Load-balancing: % of applications operational
following a full site outage
RPO: % of goals met
RTO: % of goals met
© Continuity Software, 2015 16
Portion of Critical IT Issues Detected and Addressed before
Impacting the Business
On average, organizations are able to
detect and address 57% of the critical
IT issues before they impact the
business.
The best performers are those 22% of
the respondents that are able to
detect and address over 80% of the
critical issues before the impact the
business.
A third of the respondents (33%) are
able to address just 40% or less of
their critical issues before they impact
the business.
Figure 12: Portion of critical IT issues detected and addressed before
impacting the business
13%
20%18%
27%
22%
0-20% 21-40% 41-60% 61-80% 81-100%
Average 57%
© Continuity Software, 2015 17
How helpful are IT analytics in early detection of critical issues?
Over three quarters (76%) of those that
use IT analytics tools find them either
extremely helpful (24%) or helpful (52%)
in early detection of critical IT issues.
Figure 13: Meeting KPI goals vs. portion of detecting and addressing critical IT
issues before impacting the business
Extremely helpful, 24%
Helpful, 52%
Neutral, 16%
Slightly helpful7%
Not helpful at all2%
© Continuity Software, 2015 18
Transforming Insights into Improvements
48% of the organizations surveyed are
able to transform all or most of the
relevant insights from IT analytics into
actionable plans for improvement.
An equal number (48%) are able to
translate only some of the relevant
insights into improvements.
Figure 14: Transforming insights into improvements
4%
48%
34%
14%
Rarely do insights translate into improvements
Some relevant insights translate intoimprovements
Most relevant insights translate intoimprovements
Every relevant insight translates intoimprovement
© Continuity Software, 2015 19
Transforming Insights into Improvements by Company Size
While over half (56%) of the large
organizations are able to transform all
or most of the relevant insights from IT
analytics into actionable plans for
improvement, only 43% of the smaller
organizations are able to realize these
benefits.
Figure 15: Transforming insights into improvements by company size
8%17%
35%
39%
54%
39%
4% 6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
<10,000 > 10,000
Rarely do insights translateinto improvements
Some relevant insightstranslate into improvements
Most relevant insightstranslate into improvements
Every relevant insighttranslates into improvement
© Continuity Software, 2015 20
Means to Achieve Operations Excellence
According to survey respondents, the
most effective means for achieving IT
operations excellence are better tools
for measurement & analysis (28%),
enforcement of IT best practices (28%),
and detection of cross-domain IT
configuration issues (22%).
Figure 16: Means to achieve operations excellence
3%
19%
22%
28%
28%
Other
Automation of auditing/testing and qualityassurance
Tools to detect cross-domain IT configurationissues
Tools to enforce IT best practices
Better measurement and analysis tools
© Continuity Software, 2015 21
Respondent Demographics
48% of the survey respondents come
from organizations of 10,000 or more
employees.
56% of the respondents have more than
500 servers in their datacenter and 30%
have more than 2,500 servers.
Figure 17: Number of employees
Figure 18: Number of servers
11%14%
11%16%
48%
1-500 501-2,500 2,501-5,000 5,001-10,000 > 10,000
9%11%
23%26%
30%
< 50 51-100 101-500 501-2,500 > 2,500
© Continuity Software, 2015 22
Respondent Demographics
Survey respondents come from a
variety of industries: 18% represent
financial services organizations, while
13% come from manufacturing and
10% are in healthcare.
Figure 19: Respondent Industry
18%
13%
10%
5%9%
7%
7%
5%
4%
3%3%
15%Financial Services
Manufacturing
Healthcare
Telecommunication
Public Sector
Retail / Wholesale
Business Services
Education
Media / Publishing
Energy
High Tech
Other
© Continuity Software, 2015 23
Table of Figures
Figure 1: KPIs for IT operations excellence ............................................................................................................................................................. 5
Figure 2: Frequency of Tracking KPIs ....................................................................................................................................................................... 6
Figure 3: Most difficult KPIs to meet ........................................................................................................................................................................ 7
Figure 4: Concerned with KPIs .................................................................................................................................................................................. 8
Figure 5: Achieving KPI goals ..................................................................................................................................................................................... 9
Figure 6: Use of tools to measure IT performance by comapnay size ............................................................................................................. 10
Figure 7: Monitoring IT operations areas for KPIs ............................................................................................................................................... 11
Figure 8: KPIs measured across all or most domains ......................................................................................................................................... 12
Figure 9: Frequency of analyzing configuration consistency .............................................................................................................................. 13
Figure 10: Meeting KPI goals vs. frequency of analyzing configuration consistency across IT operations areas ..................................... 14
Figure 11: Frequency of measuring goals ............................................................................................................................................................. 15
Figure 12: Portion of critical IT issues detected and addressed before impacting the business ................................................................. 16
Figure 13: Meeting KPI goals vs. portion of detecting and addressing critical IT issues before impacting the business ......................... 17
Figure 14: Transforming insights into improvements ......................................................................................................................................... 18
Figure 15: Transforming insights into improvements by company size ........................................................................................................... 19
Figure 16: Means to achieve operations excellence ............................................................................................................................................ 20
Figure 17: Number of employees ........................................................................................................................................................................... 21
Figure 18: Number of servers ................................................................................................................................................................................. 21
Figure 19: Respondent Industry ............................................................................................................................................................................. 22
© Continuity Software, 2015 24
See how IT Operations Analytics can help you prevent
infrastructure outages
Sign up for a 30-minute demo and find out
how AvailabilityGuard can improve your ability
to meet critical KPI goals:
Detect single-points-of-failure and
misconfigurations that can cause downtime
or data loss
Validate compliance with vendor best
practices
Validates that HA systems are always fail-
over ready and that production and DR
systems are always in sync
Get clear visibility into RPO, RTO and other
operational metrics (actual vs. planned)
Present actionable recommendations that
help IT teams prevent outages before they
impact the business
Sign up today
About Continuity Software
Continuity Software helps the world’s leading
organizations prevent unplanned IT outages.
Our award-winning AvailabilityGuard software
enables IT teams to proactively identify and
eliminate single-points-of-failure and other
misconfigurations across the entire
infrastructure.
Using AvailabilityGuard’s advanced IT
operations analytics, organizations are able to
deliver the highest levels of IT service
availability while improving operational
efficiency.
Website: www.continuitysoftware.com
Email: info@continuitysoftware.com
Tel: 1-888-782-8170 or +1-646.216.8628