CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for...

Security Reputation Metrics for Hosting Providers

@CSET’1510 Aug. 2015

Arman Noroozian, Maciej Korzcyński,

Samaneh Tajalizadehkhoob, Michel van Eeten

… to make and Interpret properly

Reputation Metrics are Hard !

2

Why Metrics ? The “Lemons Market” Problem

Information Asymmetry

Consumer / Policy maker/ Law enforcement officer Which provider is better/worse in security?

The provider (intermediary) itself doesn’t know either!

Erodes incentives to invest in security3

Hosting Providers Legitimate hosting provider types

Bulletproof Hosting!

(M3AAWG, Anti-Abuse Best Common Practices for Hosting and Cloud Service Providers. Technical Report March 2015.)

4

Concentration of Abuse

Attractive Pressure points Remediation

Policy making

Source: McAfee Threats Report Q2 2012 5

Source: http://krebsonsecurity.com/2010/03/naming-and-shaming-bad-isps/

Concentrations of Abuse (Cont.)

6

Source: http://hostexploit.com/downloads/world_hosts_report_201403.pdf

Hoster Size Matters

7

Measures of SizeAdvertised IP Space Hosted 2nd level domains

8

Indicators of Abuse

Indicators Why Challenge

Occurrence of Abuse(How often abused?)

Signals network hygiene and vulnerability

Hard to isolate provider efforts from other factors

Uptime of abuse(How long abused?)

Signals effectiveness of abuse handing

Hard to measure at scale

9

Sensitivity of Metrics Choice of abuse data

Biases and errors in abuse data

Errors in mapping abuse data

Biases and errors in size estimation data

10

Dutch Police:“Who are the worst hosting providers in our jurisdiction?”

A Dutch Case Study

11

Data Sources Abuse

StopBadware

Shadowserver Compromised servers

Outbound malware connections

Zeustracker C&Cs (Abuse.ch)

Mutual Legal Assistance Treaty (MLAT) requests

Dutch child pornography hotline

Phishtank

Anti-Phishing Working Group

IP Routing Data Python pyasn library

Passive DNS (pDNS) DNSDB from Farsight Security

750 million unique 2LDs

93 million unique IPv4 Addresses

12

Our Methodology

13

Abuse Feeds

p-DNS / IP Routing

• Shadow Server Compromise• Shadow Server Sandbox URL• Zeustracker C&Cs• MLAT requests• PhishTank• APWG• Child Pornography Hotline

• # Advertised IPs• # IPs in p‐DNS• # Domains Hosted

Abuse Mapping

Size Mapping

• Farsight Security p-DNS Data

• Internet IP Routing Data

• # Unique Abuse / AS

Abuse MapsAbuse Maps

PhishTankAS#1 100 AS#2 200

MLATAS#1 50AS#2 73

Size MapsSize Maps

Advertised IPsAS#1 256AS#2 1024

Domains HostedAS#1 23AS#2 1232

Step 1+2: Mapping

14

Abuse MapsAbuse Maps

PhishTankAS#1 100 AS#2 200

MLATAS#1 50AS#2 73

Size MapsSize Maps

Advertised IPsAS#1 256AS#2 1024

Domains HostedAS#1 23AS#2 1232

Normalized AbuseNormalized Abuse

PhishTank / Advrt. IPsAS#1 0.39AS#2 0.19

PhishTank / Domains HostedAS#1 4.34AS#2 0.16

MLAT / Advrt. IPsAS#1 0.19AS#2 0.07

MLAT / Domains HostedAS#1 2.17AS#2 0.05

Normalization

• # Abuse / Size

Step 3: Normalization

15

Normalized AbuseNormalized Abuse

PhishTank / Advrt. IPsAS#1 0.39AS#2 0.19

PhishTank / Domains HostedAS#1 4.34AS#2 0.16

MLAT / Advrt. IPsAS#1 0.19AS#2 0.07

MLAT / Domains HostedAS#1 2.17AS#2 0.05

Abuse RankingAbuse Ranking

PhishTank Ranking 1AS#1 834AS#2 833


MLAT Ranking 1AS#1 235AS#2 234


Rank

Sort Rank High Low

Step 4: Ranking

16







Overall RankingAS#1 1AS#2 0.92AS#3 0.87AS#4 0.86

Combine Ranks

Borda Count

Step 5: Aggregation

17

Security Reputation Metrics20 worst Dutch hosting providers Abuse Rate vs Cleanup Rate

18

Abuse Metrics are Hard How to measure abuse and remediation

What abuse can be observed What does it tell us about remediation efforts

How to associate it with hosting providers What is a hosting provider How to identify them at scale

How to control for differences among providers and interpret metric(s) How to take size into account How to take different business models into account

How to aggregate indicators into a comprehensive metric (set of metrics) ?

19

Towards better metrics How to measure abuse and remediation

Increase coverage, add different global abuse feeds Add uptime data (e.g. phishing)

How to associate it with hosting providers Identify hosting providers from IP ownership data (WHOIS) instead of AS-level routing

data (BGP)

How to control for differences among providers and interpret metric(s) Extract ‘profiles’ from pDNS data (size, shared hosting, dedicated, non-webdomain)

How to aggregate indicators into a comprehensive metric (set of metrics) ? More sensitivity analysis of aggregation methods

20

Questions?

Thank you for attention21

CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for...

Documents

Transcript of CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for...