INFORMATION SECURITY BENCHMARKING USING THE CORE DATA SERVICE

May 2014

Today’s Speakers

Cathy Bates, Assoc. VC & CIO, Appalachian State University

Joshua Beeman, Chief Information Security Officer, University of Pennsylvania

Stephen C. Gay, Information Security Officer, Kennesaw State University

Joanna Lyn Grama, Director of DRA Operations, IT GRC and Cybersecurity Programs, EDUCAUSE

Agenda

IT Security Metrics Core Data Service Benchmarking IT

Security Using CDS Panel Discussion

IT Security Metrics

IT Security Metrics - Defining

Measurement + Analysis = Metrics

IT Security Metrics:• Demonstrate the degree to which security 

goals are being met• Drive actions to improve security

IT Security Metrics - Examples

Example IT Security Metric: The change in number of vulnerabilities rated as “high”

on the IT department’s servers in FY 2011, as compared to the baseline established in FY 2010.

Other Security metrics we already use (and should we?) Responsive requests Risk (assessments) Vulnerability and incident statistics Acronyms: ALE, TCO, ROI, etc.

IT Security Metrics - Varieties

Qualitative vs. Quantitative Religious argument Best approach depends on your audience Best approach contains elements of each type Best approach is also SMART:

Specific ☻ Measurable ☻ AttainableRelevant ☻ Time-based

IT Security Metrics - Considerations

Why is it collected? What decisions will it be used to support?

IT Security Metrics – NIST modelSecurity Program Maturity Most Effective Metric Category

Stage 1: Few policies, procedures and controls; little measurement data available 

N/A  ‐Should focus first on clear definition of security program goals and objectives

Stage 2: Some policies, procedures, and controls implemented; some measurement data collected 

Implementation metrics 

Stage 3: Well‐established policies, procedures, and controls; measurement data readily available

Efficiency/effectiveness metrics

Stage 4:  Policies, procedures, and controls are well‐integrated within the security program and with other institutional programs; measurement data collected as a by‐product of business processes

Impact metrics

NIST SP 800‐55: http://csrc.nist.gov/publications/nistpubs/800‐55‐Rev1/SP800‐55‐rev1.pdf

IT Security Metrics – Audience Matters

– Campus Executives– Business Leaders– IT Groups– Peers– Other Institutions

For all audiences, it’s important to:• Establish proper context• Be transparent about how metric is derived• Communicate long‐term vision

IT Security Metrics – Audience Example

For campus executives and business leaders, you also must: Link security posture to the needs of the

institution Tie in long-term strategy and mission/vision Communicate operational credibility Protect brand reputation Demonstrate compliance

That’s a lot for a metric program to do!

What is CDS?

A benchmarking service used by colleges and universities since 2002 to inform their IT strategic planning and management.

FREEBENCHMARKING

SERVICE

IT financials, staffing, and services

Step 1: Complete the survey

Administration and Management of ITIT Support ServicesEducational Technology ServicesResearch Computing ServicesData Center services

Communications InfrastructureEnterprise Infrastructure and Services

Information SecurityIdentity Management

Information Systems and Applications

CDS Update Newsletter

CDS Reporting is powered by

Step 2: Access the data

CDS Executive Summary ReportCDS AlmanacsECAR analysis of Core Data (accessible to ECAR subscribing institutions)

Step 3: Gain additional insight

Information Security in CDS

Percentage of Total Central IT Expenditures

Information Security Reporting

Primarily central IT, 90%

Shared between central

IT and other unit(s), 7%

Primarily other admin or academic unit(s), 1%

Primarily system or

district office, 1%

Primarily outsourced, 1%

No organizational

unit responsible, 0%

Information Security Reporting by Class

90% 95% 94% 88% 90% 91% 88% 87%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AA BAPriv

BAPub

MAPriv

MAPub

DRPriv

DRPub

INTL

Primarily central IT

Shared betweencentral IT and otheradmin or academicunit(s)Primarily other adminor academic unit(s)

Primarily system ordistrict office

Primarily outsourced

Not applicable - noorganizational unitresponsible

Central IT InfoSec Responsibility

91% 90% 90%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2011 2012 2013

Primarily central IT

Shared between centralIT and other admin oracademic unit(s)Primarily other admin oracademic unit(s)

Primarily system ordistrict office

Primarily outsourced

Not applicable - noorganizational unitresponsible

Central IT InfoSec Responsibility (Top Three Activities 2012 vs. 2013)

96% 95% 91% 91% 91% 90%

0%10%20%30%40%50%60%70%80%90%

100%

2012 2013 2012 2013 2012 2013

Networksegmentation

Firewall operationand management

Network accesscontrol

Primarily central IT

Shared between centralIT and other admin oracademic unit(s)Primarily other admin oracademic unit(s)

Primarily system ordistrict office

Primarily outsourced

Not applicable - noorganizational unitresponsible

Shared InfoSec Responsibility

7% 7% 8%0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2011 2012 2013

Primarily central IT

Shared between centralIT and other admin oracademic unit(s)Primarily other admin oracademic unit(s)

Primarily system ordistrict office

Primarily outsourced

Not applicable - noorganizational unitresponsible

Shared InfoSec Responsibility (Top Three Activities 2012 vs. 2013)

64% 60%55% 56%

46% 40%

0%10%20%30%40%50%60%70%80%90%

100%

2012 2013 2012 2013 2012 2013

Information securityand privacyregulatory

compliance (e.g.,HIPAA, FISMA,ITAR, PCI DSS)

PCI (payment cardindustry)

compliance

Information riskmanagement

Primarily central IT

Shared between centralIT and other admin oracademic unit(s)

Primarily other admin oracademic unit(s)

Primarily system ordistrict office

Primarily outsourced

Not applicable - noorganizational unitresponsible

Outsourced InfoSec Activities

0% 0% 1%0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2011 2012 2013

Primarily central IT

Shared between central ITand other admin oracademic unit(s)Primarily other admin oracademic unit(s)

Primarily system or districtoffice

Primarily outsourced

Not applicable ‐ noorganizational unitresponsible

Outsourced InfoSec Responsibility (Top Four Activities 2012 vs. 2013)

12%15%

6%6%

5% 6%8% 5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2012 2013 2012 2013 2012 2013 2012 2013

Penetrationtesting

Scanning ofweb

applications for

Scanning thenetwork for

vulnerabilities

Forensicanalysis

Primarily central IT

Shared betweencentral IT and otheradmin or academicunit(s)Primarily other adminor academic unit(s)

Primarily system ordistrict office

Primarily outsourced

Not applicable - noorganizational unitresponsible

Risk Assessments by Area

Central IT systems and infrastructure

Central administrative

systems and data

Medical center systems and data

Research systems and data

Instructional systems and data

No risk assessments

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2011 2012 2013

Panel Discussion

Cathy Bates, Assoc. VC & CIO, Appalachian State University

Joshua Beeman, Chief Information Security Officer, University of Pennsylvania

Stephen C. Gay, Information Security Officer, Kennesaw State University

Joanna Lyn Grama, Director of DRA Operations, IT GRC and Cybersecurity Programs, EDUCAUSE