Metrics & Reporting - A Failure in Communication

Metrics & Reporting

2

CONTENTS

Metrics and Reporting ............................................................................................................. 3

The Problem Measured ........................................................................................................... 4

Is This Important? ................................................................................................................... 5

‘Communication is What the Receiver Does’ .......................................................................... 6

What IT/Security is Doing ........................................................................................................ 7

The Danger in Poor Communication ..................................................................................... 10

What is IT/Security Doing About this Lack of Communication? ............................................ 10

What Should IT/Security Be Doing? ...................................................................................... 11 © 2014 Wisegate. All Rights Reserved. All information in this document is the property of Wisegate. This publication may not be reproduced or distributed in any form without Wisegate's prior written permission. There’s a good chance we’ll let you use it, but still: it’s nice to ask first.

a Failure in Communication

3

In June of 2014, Wisegate conducted a member-driven research initiative designed to assess the current state of security risks and controls in business today. Assessing IT Security Risks addresses many of the top takeaways from this survey. This document is the first in a series of reports designed to look more closely at four specific issues highlighted by that survey.

» Metrics and reporting

» Malware and data breaches

» Data-centric security

» Automation and orchestration

Metrics and Reporting This document might have just as easily been titled, ‘The Lack of Metrics’. It is highlighted in a simple conclusion reached in Assessing IT Security Risks:

“Overall, [security] teams were optimistic but not overwhelmingly confident.”

Metrics & Reporting

4

On the surface, this statement appears to hide a contradiction: how can someone be genuinely optimistic without being simultaneously confident? That apparent contradiction hides a potentially widespread problem in information security: CISOs are always improving their company security; there is little ability, however, to measure that success (or indeed, lack of it). Without having the metrics of success or failure, security teams can be optimistic in what they are doing—but cannot ultimately be confident in its effect. This problem is then compounded. Metrics form the basis of business-level reporting, and without those metrics IT struggles to effectively communicate security issues to Business.

The Problem Measured Participants in this survey were asked, ‘do you have metrics in place to track your top three risks?’ (see Figure 1). Overall, 50% do not have metrics.

…the real problem with security risk management in the enterprise isn’t of confidence—it’s of measurement; survey respondents don’t really have a good way of indicating the effectiveness (or lack thereof) of existing programs.

—Assessing and Managing IT Security Risks

Figure 1: Survey Question: Do you have a metric to measure the risk in your top three areas of concerns?

Source: Wisegate June 2014


5

The problem is that there is a general acceptance that all three top risks are growing—more than 80% of participants believe that major risks are increasing in their industry (see Figure 2). [Note: These three ‘top risks’ are non-specific—they are whatever the participant considered to be his or her personal top three risks. Overall, the top three risks are malware, data breaches and outsider threat.]

Figure 2: Survey Question: Which risks are growing for your specific company and industry?

Source: Wisegate June 2014

What this means, in effect, is that IT cannot accurately communicate an increasing security risk to Business; and Business cannot accurately understand that security risk and its possible impact on the business.

Is This Important? This lack of communication is very important, for three particular reasons:

» Real security cannot be achieved without full Business buy-in.

» Business is likely to become suddenly very keen on understanding security following the recent prosecution of FedEx in what can be seen as an extension of the ‘failure to prevent’ theory. “This bodes ill not only for corporations that fail to prevent criminal activity, but for corporate compliance officers whose programs,

Metrics & Reporting

6

when scrutinized under the glare of 20-20 hindsight, may be found deficient.”1 It is possible that within a relatively short period, individual board members could be held legally liable for security failures.

» Boards are being urged by the National Association of Corporate Directors to be more proactive in information security.

The reality is that possibly for the first time, corporate boardrooms are taking cyber security seriously. The continuous flow of news of major security breaches in major companies is having an effect. Boards are asking:

» How does our security stack up?

» How do we compare with other companies in our sector? Without adequate security metrics to answer those questions in the language that Business understands, IT/Security will miss a major opportunity.

‘Communication is What the Receiver Does’ It is a tenet of communication that you have to listen. There are signs that Business is ready to listen. In July 2014 the National Association of Corporate Directors published a new handbook for its members: Cyber-Risk Oversight2. Its advice to directors is organized around five key principles:

1. Directors need to understand and approach cyber security as an enterprise-wide risk management issue, not just an IT issue.

2. Directors should understand the legal implications of cyber-risks as they relate to their company's specific circumstances.

3. Boards should have adequate access to cyber security expertise, and discussions about cyber-risk management should be given regular and adequate time on the board meeting agenda.

4. Directors should set the expectation that management will establish an enterprise-wide, cyber-risk management framework with adequate staffing and budget.

5. Discussion of cyber-risks between boards and senior managers should include identification of which risks to avoid, accept, mitigate or transfer through insurance as well as specific plans associated with each approach.

1 The Rise of 'Failure to Prevent' Crimes and CCO Liability; New York Law Journal (27 October 2014): http://newyorklawjournal.com/id=1202674374593 2 Cyber-Risk Oversight Handbook (free to NACD members): http://www.nacdonline.org/cyber


7

That last point highlights the need for discussion between IT/Security and the board. When the handbook was first published, Internet Security Alliance President Larry Clinton commented, "Most business leaders do not spend a lot of time talking about ISO standards and NIST framework. They talk about things like profitability, growth, innovation product development, price-to-earnings ratios. This publication, perhaps for the first time, attempts to put cybersecurity squarely within that business context." But while Business might be ready to listen, there remains a difficulty for IT/Security to speak in a language that it understands.

What IT/Security is Doing IT/Security is taking a risk-based approach to defending systems; but it currently lacks the means to report the risk status to boards and internal business partners. “CISOs are measuring tactical things,” explains the Assessing IT Security Risks lead author, Bill Burns. “What metrics exist are events-driven: how much classified data was blocked from leaving the system; how many malware hits were stopped at the firewall or by the AV software. But there exists a huge disconnect between such activity-based metrics and rolling them up into ‘what is the impact of our security programs on the business’.” The problem, he suggests, is that there remains a tool-centric rather than risk-centric view of security—and the tools that are available rarely provide metrics that can be combined into an overall metrics-based company risk report suitable for delivery to the board. This leads to a failure of communication between IT/Security and Business—which is, says Burns, a major challenge for IT/Security. To a large degree this basic problem is a natural result of the security product market, which comprises a wide range of distinct point products. The natural desire to use a ‘best of breeds’ approach (that is, to use the best available solution for each separate risk) doesn’t lend itself to seamless security metrics. The extent of the problem can be seen in Figures 3 to 6, taken from the survey. The diversity of different products expected to be used in the next 3-5 years makes seamless and cohesive reporting across the whole security discipline difficult to achieve—and almost impossible in a format suitable to present to business management. This is unlikely to change within the next five years.

Metrics & Reporting

8

Figure 3: Survey Question: Which endpoint-targeted security controls will be a top-priority to you in the next 3-5 years (multiple selections allowed).

Source: Wisegate, June 2014

Figure 4: Survey Question: Which mobility / IoT security control will be most important to your company in the next 3-5 years?



9

Figure 5: Survey Question: Which of these Messaging, File/Doc Sharing controls will be a top priority to you in the next 3-5 years (multiple selections allowed).


Figure 6: Survey Question: Stack-rank these Infrastructure controls by which will be a top priority to you in the next 3-5 years.


Metrics & Reporting

10

This volume of different products makes communicating strengths and weaknesses in the corporate security profile in relation to business impact a difficult proposition. “Although this sounds harsh,” comments Burns, “it results in a failure of the security teams to communicate in business terms, and for business people to understand security. There’s a business gap—and it’s one of the biggest challenges I see for Security.”

The Danger in Poor Communication The two primary dangers of poor communications are:

» A continuing disconnect between Business and Security, leading to underfunding and weak policy implementation

» A Business concentration on the one set of industry-wide metrics already available: compliance checklists

Many security teams already believe they suffer from the first, and many more will increasingly come up against the latter. “I think we are finally at the point, with so many large scale breaches,” explains Burns, “that Business is taking Security seriously. Boards are ready to listen if we can learn their language to speak to them. What they want to know is, ‘are we doing everything we should be doing; and are we doing what our peers are doing?’” It is that latter point that leads Business to concentrate on compliance-based security. If the only metrics available are the compliance regulations, then conforming strictly to those requirements serves two purposes: firstly it provides a defense against any possible ‘failure to prevent’ legal challenges; and secondly it provides a likely ‘peer comparison’ point. Most security professionals do not believe that conforming to a compliance checklist provides the best possible security. However, unless Security can develop its own metrics and reporting, Business will inevitably increasingly rely on compliance instead—possibly to detriment of real security.

What is IT/Security Doing About this Lack of Communication? IT/Security readily acknowledges that communication is a problem. “People accept that this is a problem, and talk about it,” comments Burns. “But not one of the survey participants could say, ‘I cracked the nut—this is what you have to do to communicate successfully.’”


11

It is a subject that frequently occurs in Wisegate roundtable discussions. For example, in a recent Wisegate Live Research call, one CISO with a large financial firm noted:

“The higher you go, the more you need to be able to talk about business drivers in business language that business can understand. The thing that works best seems to be stories and analogies—they seem to be the best way to share information with the more senior individuals in your business.”

—“What are the soft skills required for a career in IT and security?” Roundtable

Talking, however, is not reporting, and stories are not metrics. The reality is that IT/Security mostly does little more than talk about the problem of metrics and reporting.

What Should IT/Security Be Doing? The survey shows that IT/Security suffers from a lack of adequate metrics. This translates into poor communication between IT/Security and Business. In the short term this can be improved by IT/Security aggregating security point solutions to provide a seamless holistic risk rating; and then creating the metrics to demonstrate the impact of security on business.

In the longer term, the problem provides an opportunity for security users and security vendors. As the move towards the adoption of security as a service (SaaS) solutions gathers pace, security teams can start to insist on the provision of usable metrics as part of the partner agreement.

Metrics & Reporting

12

PHONE 512.763.0555

EMAIL [email protected]

www.wisegate i t .com

Would you like to join us? Go to wisegateit.com/request-invite/ to learn more and to submit your request for membership.

Metrics & Reporting - A Failure in Communication

Technology

Transcript of Metrics & Reporting - A Failure in Communication