RIPE71 - Bucharest, Romania · PDF fileScalable high-speed packet capture Using OpenFlow and...
Transcript of RIPE71 - Bucharest, Romania · PDF fileScalable high-speed packet capture Using OpenFlow and...
Scalable high-speed packetcaptureUsing OpenFlow and Intel DPDK
Wouter de Vries
RIPE71 - Bucharest, Romania
Who am I?
Wouter de VriesPh.D. student
Design and Analysis ofCommunication SystemsUniversity of Twente
Scalable high-speed packet capture November 13, 2015 2 / 24
DACSDesign and Analysis of Communication Systems
Introduction
We want to capture large-scale DDoS attacks withoutsignificant packet loss, why?
I Mitigation is hardI In-depth analysis could provide valuable insights
Other uses of high-speed packet capture:I Intrusion detectionI Monitoring (start your own NSA!)
Scalable high-speed packet capture November 13, 2015 3 / 24
The Problem
The total bandwidth of The InternetTM is ever increasing.
Table: Cisco Visual Networking Index 2015
Year 2014 2015 2016 2017 2018 2019PB per Month 59,8 72,4 88,4 109,0 135,5 168,0
Scalable high-speed packet capture November 13, 2015 4 / 24
The Problem
The total bandwidth of The InternetTM is ever increasing.
Table: Cisco Visual Networking Index 2015
Year 2014 2015 2016 2017 2018 2019PB per Month 59,8 72,4 88,4 109,0 135,5 168,0
Scalable high-speed packet capture November 13, 2015 4 / 24
The Problem
In order to analyze real-world traffic, the capture methods needto evolve.At speeds in excess of 10 Gbit/s things start to get difficult:
I ≥ 14.8 million packets per secondI Only a few clockcycles per packetI Storing ≥1.25 Gigabytes per second
Scalable high-speed packet capture November 13, 2015 5 / 24
The Problem
In order to analyze real-world traffic, the capture methods needto evolve.At speeds in excess of 10 Gbit/s things start to get difficult:
I ≥ 14.8 million packets per second
I Only a few clockcycles per packetI Storing ≥1.25 Gigabytes per second
Scalable high-speed packet capture November 13, 2015 5 / 24
The Problem
In order to analyze real-world traffic, the capture methods needto evolve.At speeds in excess of 10 Gbit/s things start to get difficult:
I ≥ 14.8 million packets per secondI Only a few clockcycles per packetI Storing ≥1.25 Gigabytes per second
Scalable high-speed packet capture November 13, 2015 5 / 24
The Problem
In order to analyze real-world traffic, the capture methods needto evolve.At speeds in excess of 10 Gbit/s things start to get difficult:
I ≥ 14.8 million packets per secondI Only a few clockcycles per packetI Storing ≥1.25 Gigabytes per second
Scalable high-speed packet capture November 13, 2015 5 / 24
Goal
A scalable system that is able to capture and generatepackets at high speed (e.g. ≥40 Gbit/s)
Scalable high-speed packet capture November 13, 2015 6 / 24
Proposal
I Use DPDK (Data Plane Development Kit) to maximizesingle machine performance.
I Use OpenFlow-switches to distribute traffic over multiplemachines
Scalable high-speed packet capture November 13, 2015 7 / 24
Proposal
I Use DPDK (Data Plane Development Kit) to maximizesingle machine performance.
I Use OpenFlow-switches to distribute traffic over multiplemachines
Scalable high-speed packet capture November 13, 2015 7 / 24
Proposal - Overview
Openflow Switch
Capture machine
Capturemachine
Capturemachine
Capturemachine
40 Gb/s 10 Gb/s
10 Gb/s
10 G
b/s
Recombine (offline)
pcap-file
splits traffic
Scalable high-speed packet capture November 13, 2015 8 / 24
Implementation - What is DPDK?
The Data Plane Development Kit is a library for fast packetprocessing
Main features:I Zero-CopyI Fast buffersI Designed for
multicore
Zero-copy allows the networkhardware to directly copy data tomemory buffers using DMA
Scalable high-speed packet capture November 13, 2015 9 / 24
Implementation - What is DPDK?
The Data Plane Development Kit is a library for fast packetprocessing
Main features:I Zero-CopyI Fast buffersI Designed for
multicore
Fast and thread-safe implementationsof (ring) buffers making developmentof multithreaded applications mucheasier
Scalable high-speed packet capture November 13, 2015 9 / 24
Implementation - What is DPDK?
The Data Plane Development Kit is a library for fast packetprocessing
Main features:I Zero-CopyI Fast buffersI Designed for
multicore
Has been designed from the groundup to support multiple cores, eachthread runs on its own core
Scalable high-speed packet capture November 13, 2015 9 / 24
Implementation - Using DPDK
Capture 64-byte packets at 10 Gbit/s (1.25 GB/s or 1 DVDevery 4 seconds) in PCAP-format on commodity hardware.What to do with all this data?
Scalable high-speed packet capture November 13, 2015 10 / 24
Implementation - Adding compression
gzip bzip2 lzma xz lzo
Compression time
Method
Tim
e (s
)
010
2030
4050
Compressing the linux kernel to a ram disk.Source: http://catchchallenger.first-world.info
Scalable high-speed packet capture November 13, 2015 11 / 24
Intermediate results
I Using compression specially crafted 64-byte packets canbe captured at line-rate on a single conventional HDDusing 3 cores
I Generating packets at line-rate (10 Gbit/s) is possibleusing a single core
Scalable high-speed packet capture November 13, 2015 12 / 24
Implementation - What is OpenFlow?
”OpenFlow allows direct access to and manipulationof the forwarding plane of network devices such asswitches and routers”
— Open Networking Foundation
Scalable high-speed packet capture November 13, 2015 13 / 24
Implementation - What is OpenFlow?
Controller
Openflowswitch
SW
HW
securechannel
Flow table
Scalable high-speed packet capture November 13, 2015 14 / 24
Implementation - OpenFlow
We need to define something that we split the traffic on.Possible candidates:
I Source port for TCP/UDP (allows mask on Open vSwitch)I IP-address (allows mask)I Equal-Cost Multi-Path (ECMP) routing algorithms
Scalable high-speed packet capture November 13, 2015 15 / 24
Implementation - UDP
0
1000
2000
3000
4000
5000
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Stan
dard
Dev
iatio
n
Rela
tive
Stan
dard
Dev
iatio
n
Modulo
Splitting traffic by UDP source port (2167847 packets)
SD RSD
Data: Random DRDoS attack PCAP from simpleweb.org
Scalable high-speed packet capture November 13, 2015 16 / 24
Implementation - Example flow table
Bitmask on last two bits of UDP source port
Scalable high-speed packet capture November 13, 2015 17 / 24
Implementation - IP address
4096
8192
16384
32768
65536
131072
262144
524288
1.04858x106
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Stan
dard
Dev
iatio
n
Rela
tive
Stan
dard
Dev
iatio
n
Modulo
Splitting traffic by last octet of IP address (2167847 packets)
SD RSD
Data: Random DRDoS attack PCAP from simpleweb.org
Scalable high-speed packet capture November 13, 2015 18 / 24
Implementation - IP address
4096
8192
16384
32768
65536
131072
262144
524288
1.04858x106
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0
0.01
0.02
0.03
0.04
0.05
Stan
dard
Dev
iatio
n
Rela
tive
Stan
dard
Dev
iatio
n
Modulo
Splitting traffic by MD5 hash of IP address (2167847 packets)
SD RSD
Data: Random DRDoS attack PCAP from simpleweb.org
Scalable high-speed packet capture November 13, 2015 19 / 24
Implementation - ECMP
Equal-Cost Multi-Path routing is used to balance traffic overmultiple links that have the same cost.
I ECMP Algorithm is not defined by OpenFlowI Result: ECMP implementation varies by vendor
The definition of ECMP is a great match to our problem
Scalable high-speed packet capture November 13, 2015 20 / 24
Current state
I For some types of traffic splitting is easier than othersI On-going work to find a generic way to balance flowsI ECMP is promising, depending on the implementation by
the vendor
Scalable high-speed packet capture November 13, 2015 21 / 24
Conclusion
I Using DPDK allows line-rate packet capture on 10 Gbit/sI Using OpenFlow-compatible switches has the potential to
scale the capture speed horizontallyI Combined, these two technologies allow us to capture
≥40 Gbit/s
Scalable high-speed packet capture November 13, 2015 22 / 24
Open-source
I DPDK-based packet capture tool (DPDKcap):https://github.com/woutifier/dpdkcap
TRY THIS AT HOME
Scalable high-speed packet capture November 13, 2015 23 / 24