Metricon ‘06

Leading Indicators inInformation security

John NyeAugust 1, 2006

2Symantec Security Services

Leading Indicators

In Medicine Body temperature

• Elevated values indicate probable illness andseverity

• Temperature alone can not diagnose the illness

Characteristics Inexpensive to collect Accurately diagnose the presence of the

condition May or may not reveal the nature of the

condition

3Symantec Security Services

Leading Indicators in Information Security

Are there easily measured system attributes that predictan insecure configuration?For example, does having a large number of open portscorrelate to having an insecure environment?

Application

Evaluate an environment for its degree ofvulnerability/risk to determine if additional investment iswarranted (for example conducting a full vulnerabilityassessment)

4Symantec Security Services

Symantec Attack Center

5Symantec Security Services

SYMC Attack Center – The Data Set

Scans conducted between April, 2005 and July, 2006 Adoption of the tool has been increasing Most scan results are relatively recent

449 Scans ConductedMostly External Penetration TestsNessusSet Selection – We Eliminated: Suspected test scans (i.e. we were testing the AC, not a client) Scans that weren’t used to produce a report

6Symantec Security Services

Methodology - Identifying Leading Indicators

Performed initial analysis using scans as the setVulnerability Score = sum of vulnerability severitiesdivided by host count (calculated for each scan)Scans ranked into quartiles based on vulnerabilityscoresVulnerability Saturation = count of instances of aparticular vulnerability divided by host count(calculated for each quartile)Plotted each vulnerability’s saturation from quartile toquartile and examined the results

7Symantec Security Services

Eliminating Vulnerabilities as Potential Leading Indicators

Vulnerability eliminated from consideration if: Highest quartile saturation did not exceed 2% Saturation didn’t increase with environment’s vulnerability Particular to a type of environment, not generic to most

environments (i.e. Web vulnerabilities)

Real Problems with the Data Set – 11th hour

Internal Network Scans Had to eliminate most vulnerable quartile completely from the

analysis because it contained multiple (and not-easily identified)scans conducted from within an enterprise perimeter

Probably eliminated several of the most vulnerable externalscans in doing so

8Symantec Security Services

Findings (By Nessus Vuln ID)

All non-Web scanner findings with a final saturation > 2% identified during remote penetration tests.

Potential Leading Indicators

0

0.05

0.1

0.15

0.2

0.25

0.3

1 2 3

Quartile

Vu

lnera

bilit

y S

atu

rati

on

11951

11935

10092

10263

11002

11618

10114

11936

9Symantec Security Services

Top General IndicatorsLeading Indicators (Preliminary Study)

0

0.05

0.1

0.15

0.2

0.25

0.3

1 2 3

Quartile

Vu

lne

rab

ilit

y S

atu

rati

on

Host Responds to

Syn/Fin

ICMP Timestamp

Request

OS Identified

10Symantec Security Services

Top Web Indicators

Leading Web Indicators (Preliminary Study)

0

0.1

0.2

0.3

0.4

0.5

0.6

1 2 3

Quartile

Vu

lne

rab

ilit

y S

atu

rati

on

SSL2.0

Web Mirror

Possible missing IIS

Service Pack

HTTP Trace Enabled

HTTP: Does not

reply with 404

HTTP Directory

Enumeration

HTTP Server Type

and Version

HTTP Server Type

and Version

11Symantec Security Services

Correlation: Scans vs. Project ReportsLeading Indicators (Small Data Set)

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

1 2 3 4

Quartile

Vu

lne

rab

ilit

y S

atu

rati

on

FTP Banner (10092)

HTTP Server Type

and Version (10107)

ICMP Timestamp

Request (10114)

HTTP Directory

Enumeration (11032)

HTTP Trace Enabled

(11213)

Possible Missing IIS

Service Pack (11874)

•All data is from external penetration testsSmall sample spaceTop 8 general and top 8 Web vulnerabilities depicted (only 6 of the 16 were presentin this data set.

12Symantec Security Services

Next Steps

Clean up the data set Quartile ranking of project reports doesn’t match that of Scans Mix of internal and external scan data Small sample set of project reports

Upgrade the math Statistical regression Multi-vulnerability analysis

Repeat analysis for different types of environment Internal vs. External, Web vs. Generic, etc.

Implement the analysis directly in the Attack Center

13Symantec Security Services

Dangers with Leading Indicators

The leading indicator itself can not be used as adiagnosisGaming the system Administrators may attempt to resolve only those

vulnerabilities that are used as leading indicators.

14Symantec Security Services

Questions?

John NyeConsulting Services Technical Lead

T. 617-768-2737M. 617-501-3248

john_nye@symantec.com

Thank You.