Security Metrics [2008]
-
Upload
phil-huggins -
Category
Business
-
view
212 -
download
2
description
Transcript of Security Metrics [2008]
14/07/2008
Security MetricsPhil Huggins
Security Metrics14/07/2008Page 2
Core Text
Security Metrics : Replacing Fear, Uncertainty and Doubt
Andy Jaquith, 2007
0-321-34998-9
Security Metrics14/07/2008Page 3
Recommended Texts
Security Metrics14/07/2008Page 4
Growing field
► Areas of interest► Software security ► Modelling► Benchmarking► Return on investment► Breach data
► Standards► ISO / IEC 27004► NIST SP800-55
► Communities► Securitymetrics.org► Metricsexchange.org► Cybersecurity KTN
Security Metrics14/07/2008Page 5
Securitymetrics.org
► Open mailing list and wiki► Active community► Established by Andy Jaquith► Runs the US based Metricon and MiniMetricon events
each year
Security Metrics14/07/2008Page 6
Metricsexchange.org
► New open community group► Established by Elizabeth Nichols► Sharing metrics definitions, learning and data ► Early days, big ideas
Security Metrics14/07/2008Page 7
Cybersecurity KTN – Metrics SIG
► UK Knowledge Trading Networks established by DTI► Promoting collaboration between industry, academia and
government► Metrics Special Interest Group has focused on the
delivery of the Internet Threat Exposure (ITE) Index► Threat and Countermeasure focused metric of exposure► Appears to be aimed at less sophisticated security
practitioners► Risk assessment-lite ?► Currently being developed in an open group
Security Metrics14/07/2008Page 8
Standards
► NIST SP800 – 55► Exhaustive list of possible security metrics to measure► 99 pages► No real sense of what is a useful metric► Defines useful characteristics to describe a metric
► Performance Goal, Performance Objective, Metric, Purpose, Implementation Evidence, Frequency, Formula, Data Source, Indicator
► ISO/IEC 27004► Currently in draft / closed group► Metrics covering the performance of an ISMS as defined in 27001
and 27002
Security Metrics14/07/2008Page 9
Types of Security Metrics
► Risk Metrics► Compliance Metrics► Operational Metrics► Quality Metrics► Management Metrics► Business Metrics
► Confusion among practitioners
Security Metrics14/07/2008Page 10
Focus problems
► Technical Focus► “What do we count?”
► Business Focus► “What do we need to do and why?”
► Counting is the mechanical foundation► The business wants the story the numbers tell
► Metrics are not the answer to funding problems
Security Metrics14/07/2008Page 11
Other common problems
► Managing to the metric► No longer focused on the result
► Measuring emerging threats► Measuring last years breaches
Security Metrics14/07/2008Page 12
Questions from the board
► Am I safe?► Can I take responsibility for the actions of my company?► Who handles my data?► Who am I doing business with?► Are they accountable?
Security Metrics14/07/2008Page 13
Metricon practitioners top 10 metrics
► Data volumes transmitted to competition► Coverage metrics► Availability of business systems► End user perception of security► Legal fees paid out► Total cost of information security► Information asset value► Count of events on systems► Security control success rate► Cost of security monitoring and reporting
Security Metrics14/07/2008Page 14
Balanced Security Scorecards
► Complete:► People, Process, Technology, Budgeting, Innovation,
Organisational Planning, Operations
► Traditionally include four primary perspectives:► Financial► Customer► Internal Processes► Learning and Growth
► Jaquith has a comprehensive chapter on balanced security scorecards in his book
Security Metrics14/07/2008Page 15
Geer’s Scorecard
► Finance► Cost of data security per transaction► Downtimes lost to attack by attack class► Data flow per transaction and source► Budget correlation with risk measures
► Process► % of critical systems under a DR plan► % of critical systems obeying the security policy► MTBF & MTTR for security incidents► Frequency of security team internal consultations► Latency to obey security change orders by department
Security Metrics14/07/2008Page 16
Geer’s Scorecard
► Learning and growth► % of job reviews involving security► % of security workers with training► Ratio of B.U. security staff to central security staff► Timely new system security consultations► % of programs with budgeted security
► Customer► % of SLAs with security standards► % of tested external facing applications► Number of non-employees with access► % of data secure by default► % of customer data outside the data centre
Security Metrics14/07/2008Page 17
GE Global experience
► Metrics to drive behaviour► Scorecard approach► Business unit drill down and comparison views► Communication plan was key► Built a custom system piecemeal over several years► Started with manual data, automated over time► Now moving to a common platform► Monolithic vs Composite data sources► Centralised vs Business unit data sources
Security Metrics14/07/2008Page 18
Dept of Veterans Affairs’ experience
► Didn’t have common definitions of:► What IT Security was► What better IT security looked like► The value of security
► Identified the security events that drove perception of security
► Focused on the frequency and impact of those events► Did not ignore uncertainty!► Results-focused
Security Metrics14/07/2008Page 19
Intel’s experience
► Developed predictive model for future security incidents► Used to provide ROI on ‘reduce the occurrence’ controls
NOT ‘reduce the effect’ controls► Needed to gather current state data first in order to
identify ‘Annual Rate of Occurrence’► 2 years of data from 20+ global locations
► Needed to estimate ‘Single Loss Expectancy’ value for target environment
► Identified limited target groups to pilot controls in first to measure results
► Needed a LOT of data► 87% accurate predictions over a 12 month period
Security Metrics14/07/2008Page 20
Verizon 2008 Data Breach Investigations Report
► 500 Investigations over 4 Years► 18% of breaches were the result of an unpatched system► 90% of unpatched breaches had had patches publicly
available for 6 months or more► No more would have been prevented by a patch cycle
shorter than a month► There is a lot of useful data in this report
Security Metrics14/07/2008Page 21
Dan Geer’s counterpoint
► We are losing► The bad guys are in it for the money► Attackers costs are continually falling► Need to start measuring ‘attack metrics’► Focus on increasing their cost of attack► More cost effective to redirect than to resist
Security Metrics14/07/2008Page 22
Marcus Ranum’s counterpoint
► Statistics only work where:► Population is large► Problems are common and widely shared► Aggressors act consistently
► The only scores that matter are 0% and 100%
► Security is not ‘risk management’ it is ‘complexity management’
14/07/2008
Thank [email protected]