Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of...

23
Aggregation and Aggregation and Correlation of Correlation of Intrusion-Detection Intrusion-Detection Alerts Alerts Hervé Debar Hervé Debar France Télécom R&D France Télécom R&D Andreas Wespi Andreas Wespi IBM Zurich Research Laboratory IBM Zurich Research Laboratory

Transcript of Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of...

Page 1: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Aggregation and Aggregation and Correlation of Correlation of

Intrusion-Detection Intrusion-Detection AlertsAlerts

Hervé DebarHervé DebarFrance Télécom R&DFrance Télécom R&D

Andreas WespiAndreas WespiIBM Zurich Research LaboratoryIBM Zurich Research Laboratory

Page 2: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

IntroductionArchitectureRepresentation of alertsAggregation and correlation of alertsFuture work

OverviewOverview

Page 3: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Aggregation and correlation of ID alerts generated by a diversity of sensorsFlooding: operator gets overloaded by the large number of ID alerts

Context: grouping of related alertsElimination of false alerts

Scalability

Issues addressedIssues addressed

Page 4: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

What's special about our project?What's special about our project?Started about 3 years agoInfluenced the IDWG work on IDMEFReused existing framework to integrate the ID sensors and their alerts: Tivoli Enterprise ConsoleIs available as a product: Tivoli Risk Manager

Page 5: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

IntroductionArchitectureRepresentation of alertsAggregation and correlation of alertsFuture work

OverviewOverview

Page 6: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

ArchitectureArchitecture

Probe(ISS RealSecure)

Probe(Cisco Secure IDS)

Probe(WebIDS)

Aggregation &Correlation

(TEC)

Aggregation &Correlation

(TEC)

Alert emissionDiagnostic

Alert analysisAlert acquisition

Report emissionReport generationAlert accumulation

Alert emissionDiagnostic

Data acquisitionData formatting

Data analysis

Aggregation &Correlation

(TEC)

Probe(ISS RealSecure)

Page 7: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Tivoli Enterprise ConsoleTivoli Enterprise Console

Probe(ISS RealSecure)

Probe(Cisco Secure IDS)

Probe(WebIDS)

Pre-adapter(ISS RealSecure)

Pre-adapter(Cisco Secure IDS)

TEC_tasksAlert reaction

TEC_receptionAlert reception

TEC_rulesAlert correlation

Tivo

li En

terp

rise

Con

sole

TEC_

disp

lay

Ale

rt d

ispla

y

Page 8: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

IntroductionA generic, scalable ID ArchitectureRepresentation of alertsAggregation and correlation of alertsFuture work

OverviewOverview

Page 9: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Data-driven approachrepresent in a structured way all information as provided by the probesuse same data structure for all types of probes (host- and network-based, knowledge- and behavior-based, ...)

Unified data model:independence of the correlation algorithms and the actual alerts generated by the probeseasy integration of any probe in the correlation framework

Alert class hierarchy (1)Alert class hierarchy (1)

Page 10: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Data model corresponds to an early draft of the IDWG proposal for a standard message exchange formatAbstract classes:

generic part of alertsImplementation classes:

inherit from abstract classescarry specific alert information generated by specific probes

Alert class hierarchy (2)Alert class hierarchy (2)

Page 11: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Sensor layer

Target layer

Source layer

Detailed targetlayer

Layered data modelLayered data model

ID Probeproduct

hostnameIP address

ID Alerttimename

severityconfidence

ICMPCodeType

TCP/UDPSrc PortDest Port

Single HostsDest IP

Multiple HostsNumber of hosts

Real OriginSrc IP

Spoofed OriginSpoofed Src IP

HTTPURL

SNMPCommunity

Page 12: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

IntroductionArchitectureRepresentation of alertsAggregation and correlationConclusions and future work

OverviewOverview

Page 13: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Purpose:error checkingunify information provided by different probes

IP address (network-based IDS) vs. hostname (host-based IDS)Port number (network-based IDS) vs. service name (host-based IDS)Validity of timestamp is checked and adjusted if needed

Alert prepocessingAlert prepocessing

Page 14: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Different sensors see the same attack, e.g.:duplicate_event('NR_WWW_bat_File', 'RS_HTTP_IE_BAT', [source, dest, url, time], 4.0).If source, dest, and url of the two events are the same and the timestamps are close, the two events are linked and further on treated as a single event.

Duplicates (1)Duplicates (1)

Page 15: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Duplicates (2)Duplicates (2)

Two events by the same sensor are related, e.g.:duplicate_event('WW_Suspicious_Cgi', 'WW_Success', [ids, req_id], 25.0).Duplicate gets a high severity value due to a successful cgi attack.

Page 16: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

A consequence chain is a set of alerts linked in a given order, where the link must occur within a given time interval, e.g.:consequence('RS_Http_Phf, 'rs.probe.org', 'WW_InsecCgiPhf', 'web.probe.org', 30, 10.0).If the WebIDS probe does not report the phf attack within 30 seconds, an internal event is generated with severity 10.0

This feature may be used to check whether a probe is still operational.

ConsequencesConsequences

Page 17: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Single alerts may not be significant, but the aggregation of single alerts may reveal the attack.A situation is a set of alerts that have certain characteristics in commonThree aggregation axes: source, target, and attack categorySystematic combination of aggregation axes results in seven situations

Situations (1)Situations (1)

Page 18: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

1 attack/source/dest attack source dest

2-1 attack/source attack source *

2-2 attack/dest attack * dest

2-3 source/dest * source dest

3-1 attack attack * *

3-2 source * source *

3-3 dest * * dest

Situations (2)Situations (2)

Page 19: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Situations are evaluated in parallel.Thresholds for different warning levels have to be specified, e.g.threshold('situation1',warning,minor,critical,source,dest,attack).

Generic as well as specific situations, e.g.:threshold('situation1', 10.0, 20.0, 30.0, _, _, _).threshold('situation1', 5.0, 10.0, 20.0, 'susp.host.org, _, _).

Different sizes of time windows are considered.

Situations (3)Situations (3)

Page 20: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

IntroductionArchitectureRepresentation of alertsCorrelation and aggregationFuture work

OverviewOverview

Page 21: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Improved correlation rulesAnalysis of real dataRobust correlation rules

Improved performance by outsourcing certain correlation rules from TEC's prolog engine

Zurich correlation engineConsider information about network topology and machine configurationsIntegration of new sensors

Future workFuture work

Page 22: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Operational issuesOperational issuesConfiguration

Severity and confidence value per alertDefault vs. site-specific values

PerformanceOne alert per second

Page 23: Aggregation and Correlation of Correlation of Intrusion ... · Correlation of Correlation of Intrusion-Detection Intrusion-Detection AlertsAlerts Hervé DebarHervé Debar France Télécom

Correlation is needed to reduce the number of alerts a system administrator has to handle.A standard message format will help to easily integrate new sensors in our framework.

ConclusionsConclusions