Thomas Lukaseder, Jessika Fiedler,Frank Kargl

June 27th, 2018

Performance Evaluation inHigh-Speed Networks by theExample of IDS

2 Performance Evaluation in High-Speed Networks by the Example of IDS

bwNET100G+

3 Performance Evaluation in High-Speed Networks by the Example of IDS

bwNET100G+ – Ulm: Security Concepts

Security of and through SDNSecurity issues of SDN networks – how can SDN Networks beattacked?How can SDN be used to secure a network?

Mitigation of DDoS AttacksDetection of attacks within the network through traffic analysisand potential victim surveillance.DDoS attack vs. flash crowd effect.Identification of attackers – how can attackers bedifferentiated from benign clients?

Intrusion Detection Systems in high-throughput Networks.

4 Performance Evaluation in High-Speed Networks by the Example of IDS

bwNET100G+ – Ulm: Security Concepts

Security of and through SDNSecurity issues of SDN networks – how can SDN Networks beattacked?How can SDN be used to secure a network?

Mitigation of DDoS AttacksDetection of attacks within the network through traffic analysisand potential victim surveillance.DDoS attack vs. flash crowd effect.Identification of attackers – how can attackers bedifferentiated from benign clients?

Intrusion Detection Systems in high-throughput Networks.

5 Performance Evaluation in High-Speed Networks by the Example of IDS

Motivation

Ever increasing bandwidth requirements whilecomputational power increases slower.

IDS necessary to detect attackers in the network. Perimetersecurity in form of firewalls can only protect against someattacks.

Budget constrains: is a cheap solution viable?

6 Performance Evaluation in High-Speed Networks by the Example of IDS

Motivation – Cheap solution?

No licensing costs: Open Source IDS.

Affordable Hardware.

7 Performance Evaluation in High-Speed Networks by the Example of IDS

Setup – Hardware

IDSSender

Receiver

home networkexternal network

8 Performance Evaluation in High-Speed Networks by the Example of IDS

Setup – Hardware

9 Performance Evaluation in High-Speed Networks by the Example of IDS

Setup – Hardware

4 CPU cores with 3.1 GHz, 6 GB of memory

10 Gbps SFP+ connection

10 Performance Evaluation in High-Speed Networks by the Example of IDS

Setup – Software

Snort Suricata

11 Performance Evaluation in High-Speed Networks by the Example of IDS

Snort

First introduced 1998 by Martin Roesch.

Developed by Sourcefire.

Sourcefire was bought by Cisco in 2013.

3 modes: sniffer, packet logger, IDS.

Current stable version single threaded.

12 Performance Evaluation in High-Speed Networks by the Example of IDS

Suricata

Developed by the Open Information Security Foundation.

First Beta in 2009.

First stable release in 2010.

Multi-threaded.

Features GPU-Acceleration.

13 Performance Evaluation in High-Speed Networks by the Example of IDS

Setup – Software: Attacks

Benign traffic: iperf3

Attack type Tool usedsuccessful SSH brute force Metasploit frameworkunsuccessful SSH brute force Metasploit frameworkTCP connect flood npingTCP SYN flood hping3UDP flood hping3SYN scan nmap -sSSYN OS-scan nmap -sS -OUDP scan nmap -sUUser enumeration nmap

14 Performance Evaluation in High-Speed Networks by the Example of IDS

Attack Traffic Generation Scheme

15 Performance Evaluation in High-Speed Networks by the Example of IDS

Setup – Software: Rule Selection

Suricata accepts rule sets written in Snort’s config fileformat.

The Snort community offers a community rule set.

Small changes to ensure detection of our attacks.

Identical rule sets for both IDS.

16 Performance Evaluation in High-Speed Networks by the Example of IDS

Evaluation

Tests at different bandwidths (1,2,3,4,5,6, and 7 Gbps).

Tests at different attack strengths per attack (between 10 and35 attacks per minute per attack).

17 Performance Evaluation in High-Speed Networks by the Example of IDS

Evaluation

Value Meaning ArithmeticTP Correct logged messages sample sumFP Logged but not expected sample sumFN Expected but not logged sample sumTPR Attack detection rate (Sensitivity) TP/(TP + FN)Precision Rate of correct alerts among alerts TP/(TP + FP)CPU CPU usage of IDS sample averageMemory Memory usage of IDS sample averageRP Packets analyzed by IDS average over timeDR Packets dropped by the IDS average over timeSP Actual send packets average over time

18 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Snort CPU Utilization (in %)

19 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Suricata CPU Utilization (in %)

20 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Snort Drop Rate (in %)

21 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Suricata Drop Rate (in %)

22 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Snort Precision (TP/(TP + FP))

23 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Suricata Precision (TP/(TP + FP))

24 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Snort Sensitivity (TP/(TP + FN))

25 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Suricata Sensitivity (TP/(TP + FN))

26 Performance Evaluation in High-Speed Networks by the Example of IDS

Results

Memory usage is fixed; bandwidth and number of attackshave no influence.

CPU utilization depends on bandwidth (Snort) or settings(Suricata); no correlation with number of attacks in thenetwork.

Even with a higher drop rate, Suricata achieves higherprecision and sensitivity than Snort.

27 Performance Evaluation in High-Speed Networks by the Example of IDS

Future Work

Current stable release of Snort is single threaded, betaversion (Snort 3) is multi-threaded. Evaluation of this isplanned.

Suricata offers GPU acceleration. How does this performcompared to CPU only?

Experimental integration of GPU acceleration was done forSnort ten years ago. We are currently working on integratingthis again in Snort 3.

Publish the test attack traffic combined with a networktesting environment.

28 Performance Evaluation in High-Speed Networks by the Example of IDS

Thank you

29 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Droprate Snort

30 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Droprate Suricata

31 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – False alarms @ 7 Gbps

32 Performance Evaluation in High-Speed Networks by the Example of IDS

Results – Sensitivity @ 7 Gbps