Next-Gen DDoS Detection:Leveraging the Power of Big Data Analytics

Jim Frey, VP Product, Kentik Technologies

February 24, 2016

• Context: DDoS Landscape Today

• DDoS Defense Equation: Detection + Mitigation

• Case Example: DDoS Detection

• Big Data Analytics: Key to Advanced Detection

• Kentik’s Approach: NextGen DDoS Detection• Wrap-Up / Q&A

Agenda

2

3

DDoS LandscapeA Clear and Present Danger

3

DDoS Landscape Today (1/6)Who is Being Targeted?

Companies surveyed were attackedin 2014 or early 2015

Of those attacked were hitrepeatedly.

Source: Neustar DDoS Attacks & Protection Report: North America & EMEA, October 2015

Being attacked at least monthly Attacks lasted > 24 hours

4

DDoS Landscape Today (2/6)

Goal: Take down target with sheer massive volume of requests or activity. Can be aimed at network or server resource exhaustion.

Examples:• TCP SYN Floods• UDP Floods (NTP, DNS, SSDP)• UDP Fragments• NTP Amplification• ICMP Flood

VolumetricGoal:

Starve target’s resources by making normal exchanges…. Take.... Way.... Longer.

Examples:• Slow Loris

• Sockstress• Slow HTTP GET

• Slow HTTP POST

Low and SlowGoal: Exploit specific Layer 7 protocol and application flaws to prevent normal function

Examples:• HTTP Flood• HTTPS Flood• DNS Amplification• RegEx• Hash Collision

Application Layer

Attack Types?

5

DDoS Landscape Today (3/6)Mix is broad, and heavily infrastructure-focused

Source: Akamai State of the Internet (Security) report,Q3 2015

6

DDoS Landscape Today (4/6)Size/Frequency Ramping

Increased attack frequency Quarter over Quarter

Increased average attack sizeQuarter over Quarter

Source: Verisign Distributed Denial of Service Trends Report, Q3 2015

Average attack size in Gbps 1 in 5 Attacks > 10 Gbps

7

DDoS Landscape Today (5/6)Sources Vary…

Source: Akamai State of the Internet (Security) report, Q3 20158

DDoS Landscape Today (6/6)Reflection Attacks on the Rise

Source: Akamai State of the Internet (Security) report, Q3 2015

9

10

DDoS DefenseA Two-Part Challenge: Detect + Mitigate

10

DDoS Defense Architecture: Requirements

- Real-time / sub-minute

- Accurate (no false positives, no false negatives)

- Flexible (can work with multiple mitigation strategies)

- Supportive of automation/integration

- Cost Effective

Detection

- Easy to configure

- Adaptable (can support new types of attacks)

- Automated

- Deployment options (in band vs. out of band, always on vs. on demand)

- Cost Effective

Mitigation

11

DDoS Defense Architecture: Tech Options

Data Source

- Stateful Packet Inspection- Flow Monitoring (NetFlow, sFlow,

IPFIX)

Platform

- Appliances

- Downloadable Software- SaaS

Detection

- BGP RTBH

- Router ACL- BGP FlowSpec

- OpenFlow

- Cloud Scrubbing Service- On-Premises Scrubbing Appliances

- No Action

Mitigation

12

End to End DDoS Protection: Attack Begins

Target Servers

Internet

Detector

Attack traffic

Legit traffic

Flow data 13

End to End DDoS Protection: Direct Trigger to Edge

Internet

Detector

Attack traffic

Legit traffic

ACL, Flowspec, RTBH

Flow data 14

Operator Action or automated

script/programAlert

Target Servers

End to End DDoS Protection: On-Prem Scrubber

Internet

Detector

Attack traffic

Legit traffic

Redirect to Mitigation

Flow data 15

DDoS Scrubber

Target Servers

End to End DDoS Protection: Cloud Mitigation

Internet

Detector

Attack traffic

Legit traffic

Redirect to Mitigation

Flow data

Cloud Mitigation

Service

16

Target Servers

17

DDoS DetectionThe Common Thread

17

18

Case Example: DDoS AttackThings you may find when doing forensic DDoS analysis…

18

19

Seemingly Normal Variations over Several Days….?

Starting Point: Total Traffic

19

20

Looking at only SRC=CN (China)

Sorting by Source Geo

20

21

Zooming in time range on Second Spike

Drilling Deeper

21

22

Number of Unique Source IP Addresses

Checking another Dimension

22

23

Flip to: Destination Addresses

Where is the Traffic Going?

23

24

Looking at all inbound traffic to the target victim Dest IP

Pulling Back to Gauge the Situation

24

25

Attack details by protocol

Narrowing in on the Actual Attack

25

26

Multiple simultaneous vectors at hand

The Finding: Multi-Layered Attack

26

27

Finding the Necessary Details for Setting Filter Policies

The Mitigation Plan

27

28

- Unusual traffic patterns from suspect Geo- Turned out to be DNS Amplification targeting a specific dest IP- But main attack was hiding other attacks/exploits- Data harvested for mitigation

- Time required to complete this analysis: 3 minutes!- How is this possible???

Case Example: Summary

28

29

Big Data Analytics for DDoSKey to Advanced DDoS Detection and Forensics

29

DDoS Detection Tooling – Major Decision Points1. Packet-based or Flow-based?

• Packet-based requires in-line inspection, usu. via appliances ($$)

• Flow-based can be local/appliance or SaaS

2. Fully Integrated with Mitigation, or Best of Breed?

• Fully Integrated only works when mitigation is “always on”

• Independent detection ensures mitigation flexibility

3. Next-Gen Data Architecture, or Legacy?

30

DDoS Detection Tooling – Data ArchitectureKey Question

“To Summarize or Not to Summarize??”

Advantages of Summarization

- More compact long term data store

- Faster (?) searches against history

Disadvantages of Summarization- Major Loss of essential detail!!

Only Viable Answer: NO SUMMARIZATION 31

Big Data for Next-Gen DDoS DetectionWhy Big Data??Network Monitoring Data IS Big Data

• Meets Volume/Variety/Velocity Test

• Billions of records/day (millions/second)Big Data architectures:

• Mature, viable for hyper-scale, real-time data sets – SCALABLE, RELIABLE

• Capable of performance at scale for analyzing ALL data – not just summaries/metadata –RESULTS IN SECONDS

Big Data Analytics: The DDoS Detection PayoffWhat Do I Get by Going With Big Data?

• Accuracy

• Having ALL raw data available, not just what was pre-defined

• Essential for answering key questions like: Is this Friend or Foe?

• Flexibility

• Don’t have to wait for vendor to support new attack profiles

• Easy to add more data types/sets to enrich the story

• Can export data quickly/easily to other systems

Kentik’s ApproachNext Gen Big Data NetFlow Analytics for DDoS Detection…. And more

34

Kentik Detect: the first and only SaaS SolutionFor Network Ops Management & Visibility at Terabit Scale

CLOUD- BAS ED REAL- T I M E MULT I - TENANT OP EN GLOBAL

Analyze & Take Action

Big Data NetworkTelemetry Platform

in the Cloud

The Network is the Sensor

Web Portal

Real-time & historical queries

NetFlow/sFlow/IPFIX

SNMPBGP

Alerts: DDoS, Ops

E-mail / Syslog / JSON

Open API

SQL / RESTful

Kentik Data Engine

35

Multi-tiered/Clustered Big Data Architecture for Scale / Load Balancing / HA

What’s Behind Kentik Detect : The Kentik (big) Data Engine

POSTGRESSERVERS

SQL

DATA STORAGE CLUSTER

NetFlowSNMPBGP

INGEST CLUSTER

CLIENTS

N M

Optimized for Massive Data Ingest & Rapid Query Response36

NextGen NetFlow Analytics: Full Detail, Fast Navigation, Infinite Granularity

37

NextGen NetFlow Analytics: Dashboards in Seconds

38

Key Takeaways

What NextGen DDoS Detection Can (Should) Do for You: - Deliver true live monitoring & alerting

- Quickly recognize / analyze attacks

- Operate on a full data set, not just summaries or pre-defined rules

- Support multiple mitigation options

- Enable automation

39

Network Intelligence at Exabit Scale

Thank You!

Jim FreyVP Product

Kentik Technologiesjfrey@kentik.com

@jfrey80