Bdcl3-A4-Bdcl3 a4 104 Adams Os

8/13/2019 Bdcl3-A4-Bdcl3 a4 104 Adams Os

1/34

To the Point: Measure Value Through

Metrics

IT Infrastructure, Operation and Data Center

Summit

April 9-10, 2013

Sheraton Sao Paulo WTC Hotel

Sao Paulo, Brazil

Patricia Adams

This presentation, including any supporting materials, is owned by Gartner,Inc. and/or its affiliates and is for the sole use of the intended Gartneraudience or other authorized recipients. This presentation may containinformation that is confidential, proprietary or otherwise legally protected, and

it may not be further copied, distributed or publicly displayed without theexpress written permission of Gartner, Inc. or its affiliates. 2013 Gartner, Inc. and/or its affiliates. All rights reserved.

Notes accompany this presentation. Please select Notes Page view.These materials can be reproduced only with written approval from Gartner.Such approvals must be requested via e-mail: [email protected] is a registered trademark of Gartner, Inc. or its affiliates.


2/34Page 1

Patricia Adams

BDCL3_104, 4/13

This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its affiliates and is for the sole use ofhe intended Gartner audience or other authorized recipients. This presentation may contain information that is confidential,proprietary or otherwise legally protected, and it may not be further copied, distributed or publicly displayed without theexpress written permission of Gartner, Inc. or its affiliates. 2013 Gartner, Inc. and/or its affiliates. All rights reserved.

To the Point: Measure Value Through Metrics


3/34Page 2

Patricia Adams

BDCL3_104, 4/13



Regardless of the function within infrastructure and operations (I&O), metrics inundate our days, as we try

to report and explain productivity in terms of numbers and measures that everyone can understand. But our

focus on providing metrics to prove how successful we are has left a void in the value message of what the

I&O is actually providing to the business. Although we believe the business should be impressed with how

effective the incident management process is and marvel at the mean time to restore service (MTRS),

reality shows us that most of the metrics we report on are of no interest to the business users, and don't

really tell the story of what value the I&O is providing to the business. If we can take the time and push our

existing approach to metrics aside, and follow some simple steps in how we report on our activity, we can

begin to use metrics to show how the I&O is a true valued business partner.


4/34Page 3

Patricia Adams

BDCL3_104, 4/13




5/34Page 4

Patricia Adams

BDCL3_104, 4/13




6/34Page 5

Patricia Adams

BDCL3_104, 4/13



Key Issue: How can I&O focus on using the right metrics?

When examining a metric, or set of metrics, to be used to determine how a service, process or person is

performing, we can use a simple balanced scorecard approach to ascertain at what level the metrics are

being used. The graphic on this slide depicts four elements that can be used to assess the maturity of the

metrics. The four dimensions are as follows:

ManageableCan the I&O make changes to how it operates based on the outcome of the metric?

Goal Aligned

Is the metric aligned to any specific goals of the organization? MultidimensionalIs the metric siloed, or does it span multiple groups and/or multiple dimensions of

assessment to ensure quality ?

Benchmarked Is the metric benchmarked, either internally or externally?

Determining the level of maturity at which your organization is using metrics requires assessment against

the balanced scorecard, with the ultimate goals of being able to answer yes across all four dimensions,

ensuring that the metrics being used can assist in showing value.

Action Item: Assess metrics being used against the scorecard to determine the maturity of those measures.


7/34Page 6

Patricia Adams

BDCL3_104, 4/13




Business metrics must align to each side of the triangle to understand the balance of efficiency, productivity

and satisfaction. In determining which side to align business-oriented metrics to, the metrics should not be

used twice to eliminate hyper-focus. Metrics may overlap, but having providing a balanced viewpoint

requires that organizations interpret metrics appropriately as they align to business objectives. For example

a metric such as MTRS can show how efficient a team is being, but it can also be an indicator of how

productive the team is as well. Deciding how to use the metric eliminates hyperfocus. This is also importanbecause steering performance toward the wrong metric can have un-intended consequences. For instance,

should you choose to use FCR as your main measure and the IT service desk staff knows that is how they

are being measured, they may begin to spend too much time on one particular issue. Although this may

certainly address that one issue, it can cause the backlog to become unmanageable, impact the MTRS, as

well as the resolution time distribution, and more importantly, cause a dissatisfaction of other customers

who may be waiting in the queue for assistance with their issue.

Action Item: Establish metrics triangles to ensure you are using small subsets of metrics that work in unison

to tell the story of performance for the I&O.


8/34Page 7

Patricia Adams

BDCL3_104, 4/13



The build on this chart shows that when you just look at single metrics such as FCR, it doesn't tell the story

of quality. By combining FCR along with the reopen rate of incidents you can better see if business users

are getting a quality interaction.


9/34Page 8

Patricia Adams

BDCL3_104, 4/13



The build on this chart shows that when you just look at single metrics such as FCR, it doesn't tell the story

of quality. By combining FCR along with the reopen rate of incidents you can better see if business users

are getting a quality interaction.


10/34Page 9

Patricia Adams

BDCL3_104, 4/13



The build on this chart shows that when you just look at single metrics such as number of incidents, you are

missing a balanced story. When you add in the MTRS you can better see that changes in the incident

volume may have an impact on the service being provided. Metrics should never be looked at in a vacuum.


11/34Page 10

Patricia Adams

BDCL3_104, 4/13



The build on this chart shows that when you just look at single metrics such as number of incidents, you are

missing a balanced story. When you add in the MTRS you can better see that changes in the incident

volume may have an impact on the service being provided. Metrics should never be looked at in a vacuum.


12/34Page 11

Patricia Adams

BDCL3_104, 4/13




To determine where to optimize investments in I&O, there must be goals and objectives linked to the

identification of critical success factors (CSFs), corresponding key performance indicators (KPIs) and

metrics that are explicitly mapped to the processes with the I&O. Without this alignment to IT and business

goals and objectives, the process managers and the IT organizations will be unable to determine appropriate

process investments and the achievement of improvements.

Together, the CSFs, KPIs and metrics serve as a definition of success. Too often, organizations rush to avariety of metrics without linking them to overall goals and CSFs. A CSF defines an action or result that

must occur for the incident management process to be characterized as successful. KPIs are used to

measure the achievement of a CSF and are selected to ensure efficiency and effectiveness. Without

alignment to CSFs and KPIs, a metric is just a measurement and lacks, by itself, the context to assess

achievement. It is important not to drown in a plethora of metrics.

Action Item: Review current metrics to determine whether they align to specific goals.


13/34Page 12

Patricia Adams

BDCL3_104, 4/13




The greatest challenge that I&O faces is how to define value in the services it provides. IT leadership will

spend countless hours creating colorful charts, reports, scorecards and dashboards targeted at showing how

much work is being performed by the team. The savvy team leader will pause before creating the barrage of

reports and ask them to what purpose do all these reports serve? Even more important, with all the potential

metrics available, how does one pick a set of metrics that will most accurately reflect how the team is

performing. The greatest challenge comes in deciding which of the myriad of measures that can be reportedup on and analyzed will more accurately reflect what business value the IT service desk is truly providing.

Herein lies the concern with how the I&O organization can determine what will show true value to the

organization. The answer, simply put, would be to ask the business how it would define value. This can be

accomplished through the establishment of an IT governance team with respect to the IT service desk. The

IT governance team, consisting of a variety of representatives from both the business and IT, enables the

I&O to determine what shared and agreed goals will best describe performance.

Action Item: Establish an IT governance team with the goal of establishing shared goals and CSFs for

which specific metrics can be aligned.


14/34Page 13

Patricia Adams

BDCL3_104, 4/13




I&O organizations often engage Gartner and ask the question of how their operational metrics compare with other

organizations, seeking some sort of benchmark. This is generally discouraged because one organization's incident

mean time to restore service (MTRS) is 2.5 hours, which does not necessarily mean your organization should also be

at 2.5 hours. This is due to a lack of standard methods by which a metric is produced. For instance, while everyone

may agree about what MTRS means, it would also have to be agreed on as to what issues within an organization were

classified as incidents and as well as an agreed on manner in how time is calculated. Because of that lack of

consistency, benchmarking against other organizations becomes difficult. Additionally, given that metrics must bealigned to goals specified by your business, aligning to benchmarked metrics of another organization is not ideal.

Ultimately, there are two methods by which an I&O might use benchmarking. If comparison with another

organization is required, then find an organization that is as identical to your organization as possible and endure that

metrics are defined and calculated in the same manner as yours. Alternately, choose to benchmark against your own

performance. Determine baselines for your metrics, establish improvement targets, and continue to measure against

your existing benchmark until reaching the target. If the target is tied to a CSF, then you will have not only improved

your performance as compared with the previous benchmark, but also achieved an established CSF.

Action Item: Use Gartner's IT key metrics data for metrics comparisons when available, but focus on establishing

benchmarks within your own organization with targeted improvements that align to established CSFs.


15/34Page 14

Patricia Adams

BDCL3_104, 4/13




16/34Page 15

Patricia Adams

BDCL3_104, 4/13




17/34Page 16

Patricia Adams

BDCL3_104, 4/13



This slide highlights that although I&O is always providing reporting as to its productivity, nowhere in our

research do we see that CIOs are looking for that type of information. This means we need to get away

from focusing on performance metrics.


18/34Page 17

Patricia Adams

BDCL3_104, 4/13




19/34Page 18

Patricia Adams

BDCL3_104, 4/13



This slide shows how a typical I&O department like the service desk tries to provide metrics with the

business not being interested, because none of the presented metrics actually show what is happening in

terms of business productivity.


20/34Page 19

Patricia Adams

BDCL3_104, 4/13



Key Issue: How can I&O show the value of services provided through metrics?

One of the key problems for the I&O is to prove that they are more than just a simple cost center to the business. The

focus on metrics often provide proof of the productivity for I&O, but lack evidence of what value to the business has

been gained. When presented at the executive level, a report showing how many incidents were handled and the

associated MTRS does not translate into a business justification. If the I&O can approach their metrics and reporting

form a slightly different perspective, then it can easily begin to paint a picture of true value provided to the business.

For example, rather than reporting on how many incidents and how long it took to response to them, turn the report

upside down and start to report on the impact of all those incidents. In our example, 100 incidents with an MTRS oftwo hours would equate to potential downtime of 200 hours. If, through effective incident management the I&O is

able to reduce the MTRS of those 100 incidents by 25% to 1.5 hours, then the potential downtime would be reduced

to 150 hours. In essence, the I&O has returned 50 hours of potential productivity to the business. When a cost basis is

applied to productivity, the value can be put into financial terms, which anybody at the executive meeting or board

level would appreciate. Furthermore, the I&O now has the ability to show that investments in the I&O group can yield

increased productivity for the business.

Action Item: Begin to look at the metrics being used and turn those metrics and reports into value statements that

clearly define how the I&O impact business productivity.


21/34Page 20

Patricia Adams

BDCL3_104, 4/13



Key Issue: How can I&O show the value of services provided through metrics?

Gartner's ITScore for I&O (ITSIO) indicates the overall I&O maturity and the maturity of each of the four critical

dimensions: people, process, technology and business management. The fact that metrics is a component of the I&O

maturity across three of these dimensions (in process, people, and business management) shows the importance of

mature definitions of metrics and robust measurement practices. ITSIO data available on metrics across these three

I&O maturity dimensions, shows the following (N = 277):

People = 1.94

Process = 1.68 Business Management = 2.31

The data shows the maturity of metrics across all the three dimensions are not more than 3, whereas, as per ITScore,

maturity starts from level 3. In process dimension, it is the lowest scoring attribute and the value is significantly lower

than 2, in the 1 to 5 scale. This analysis shows that, despite all the tools being deployed by several I&O teams to

monitor I&O metrics, these are still in a nascent stage when it comes to maturity. Although the majority of

organizations are very adept at collecting, reviewing and reporting metrics, metrics can provide very different

perspectives of the I&O when used at the appropriate maturity levels.

Action Item: Use Gartner's ITScore to assess the current maturity of your I&O and review research to help move up

the maturity stack.


22/34Page 21

Patricia Adams

BDCL3_104, 4/13



Business metrics must align to each side of the triangle to understand the balance of efficiency, productivity

and satisfaction. In determining which side to align business-oriented metrics to, the metrics should not be

used twice to eliminate hyperfocus. Metrics may overlap, but having providing a balanced viewpoint

requires that organizations interpret metrics appropriately as they align to business objectives. For example

a metric such as MTRS can show how efficient a team is being, but it can also be an indicator of how

productive the team is as well. Deciding how to use the metric eliminates hyper-focus. This is also

important because steering performance toward the wrong metric can have un-intended consequences. For

instance, should you choose to use FCR as your main measure and the IT service desk staff knows that is

how they are being measured, they may begin to spend too much time on one particular issue. Although this

may certainly address that one issue, it can cause the backlog to become unmanageable, impact the MTRS

as well as the resolution time distribution, and more importantly, cause a dissatisfaction of other customers

who may be waiting in the queue for assistance with their issue. Additionally, look to update existing

dashboards with business value dashboards and work to utilize metrics that tell a story about the value of

service, beyond just productivity and performance.


23/34Page 22

Patricia Adams

BDCL3_104, 4/13




24/34Page 23

Patricia Adams

BDCL3_104, 4/13



Tools such as ITSSM have criteria around metrics, reporting and analytics. But in the recent review of the

tools, there was a definite lack of the vendors ability to show they understand this. Rather, they still provide

hundreds of canned reports that focus solely on productivity.


25/34Page 24

Patricia Adams

BDCL3_104, 4/13




26/34Page 25

Patricia Adams

BDCL3_104, 4/13



This slide explains that when creating a dashboard, before putting information on it, you have to consider

the context around that information. This will help better define what information might be appropriate for

one dashboard versus another.


27/34Page 26

Patricia Adams

BDCL3_104, 4/13



Key Challenges

Executive dashboards focus on showing the operational and technical metrics (such as uptime) that

generally justify the I&O organization's existence.

Business value dashboards (BVDs) provide CIOs with a value-based view of how the I&O group impacts

not only business productivity, but also the key initiatives that transform and grow the business.

CIOs focus on reducing the cost of IT, because they can't clearly delineate the actual value of what IT

brings to the business.

I&O organizations that can't clearly demonstrate business value risk substantial budget reductions.

Recommendations

Take the current executive dashboard components, and transform them from productivity and

performance statements into value statements.

Gather the appropriate level of financial information needed to ensure that reporting that's focused on

revenue, cost savings and spending is as accurate as possible.

Understand the business goals from the CIO's perspective, and use the BVD to report on the metrics that

will appeal most to the CIO.


28/34Page 27

Patricia Adams

BDCL3_104, 4/13



This slide has animation to go through some sample visual dashboard to show what an organization might

want to present to different business leaders. It is very visual, and any item on it can be further explored to

get to the data behind the data.


29/34Page 28

Patricia Adams

BDCL3_104, 4/13







30/34Page 29

Patricia Adams

BDCL3_104, 4/13







31/34Page 30

Patricia Adams

BDCL3_104, 4/13







32/34Page 31

Patricia Adams

BDCL3_104, 4/13




33/34Page 32

Patricia Adams

BDCL3_104, 4/13




34/34


Bdcl3-A4-Bdcl3 a4 104 Adams Os

Documents

Transcript of Bdcl3-A4-Bdcl3 a4 104 Adams Os