Next-Gen DDoS Detection:Leveraging the Power of Big Data Analytics
Jim Frey, VP Product, Kentik Technologies
February 24, 2016
• Context: DDoS Landscape Today
• DDoS Defense Equation: Detection + Mitigation
• Case Example: DDoS Detection
• Big Data Analytics: Key to Advanced Detection
• Kentik’s Approach: NextGen DDoS Detection• Wrap-Up / Q&A
Agenda
2
3
DDoS LandscapeA Clear and Present Danger
3
DDoS Landscape Today (1/6)Who is Being Targeted?
Companies surveyed were attackedin 2014 or early 2015
Of those attacked were hitrepeatedly.
Source: Neustar DDoS Attacks & Protection Report: North America & EMEA, October 2015
Being attacked at least monthly Attacks lasted > 24 hours
4
DDoS Landscape Today (2/6)
Goal: Take down target with sheer massive volume of requests or activity. Can be aimed at network or server resource exhaustion.
Examples:• TCP SYN Floods• UDP Floods (NTP, DNS, SSDP)• UDP Fragments• NTP Amplification• ICMP Flood
VolumetricGoal:
Starve target’s resources by making normal exchanges…. Take.... Way.... Longer.
Examples:• Slow Loris
• Sockstress• Slow HTTP GET
• Slow HTTP POST
Low and SlowGoal: Exploit specific Layer 7 protocol and application flaws to prevent normal function
Examples:• HTTP Flood• HTTPS Flood• DNS Amplification• RegEx• Hash Collision
Application Layer
Attack Types?
5
DDoS Landscape Today (3/6)Mix is broad, and heavily infrastructure-focused
Source: Akamai State of the Internet (Security) report,Q3 2015
6
DDoS Landscape Today (4/6)Size/Frequency Ramping
Increased attack frequency Quarter over Quarter
Increased average attack sizeQuarter over Quarter
Source: Verisign Distributed Denial of Service Trends Report, Q3 2015
Average attack size in Gbps 1 in 5 Attacks > 10 Gbps
7
DDoS Landscape Today (5/6)Sources Vary…
Source: Akamai State of the Internet (Security) report, Q3 20158
DDoS Landscape Today (6/6)Reflection Attacks on the Rise
Source: Akamai State of the Internet (Security) report, Q3 2015
9
10
DDoS DefenseA Two-Part Challenge: Detect + Mitigate
10
DDoS Defense Architecture: Requirements
- Real-time / sub-minute
- Accurate (no false positives, no false negatives)
- Flexible (can work with multiple mitigation strategies)
- Supportive of automation/integration
- Cost Effective
Detection
- Easy to configure
- Adaptable (can support new types of attacks)
- Automated
- Deployment options (in band vs. out of band, always on vs. on demand)
- Cost Effective
Mitigation
11
DDoS Defense Architecture: Tech Options
Data Source
- Stateful Packet Inspection- Flow Monitoring (NetFlow, sFlow,
IPFIX)
Platform
- Appliances
- Downloadable Software- SaaS
Detection
- BGP RTBH
- Router ACL- BGP FlowSpec
- OpenFlow
- Cloud Scrubbing Service- On-Premises Scrubbing Appliances
- No Action
Mitigation
12
End to End DDoS Protection: Attack Begins
Target Servers
Internet
Detector
Attack traffic
Legit traffic
Flow data 13
End to End DDoS Protection: Direct Trigger to Edge
Internet
Detector
Attack traffic
Legit traffic
ACL, Flowspec, RTBH
Flow data 14
Operator Action or automated
script/programAlert
Target Servers
End to End DDoS Protection: On-Prem Scrubber
Internet
Detector
Attack traffic
Legit traffic
Redirect to Mitigation
Flow data 15
DDoS Scrubber
Target Servers
End to End DDoS Protection: Cloud Mitigation
Internet
Detector
Attack traffic
Legit traffic
Redirect to Mitigation
Flow data
Cloud Mitigation
Service
16
Target Servers
17
DDoS DetectionThe Common Thread
17
18
Case Example: DDoS AttackThings you may find when doing forensic DDoS analysis…
18
19
Seemingly Normal Variations over Several Days….?
Starting Point: Total Traffic
19
20
Looking at only SRC=CN (China)
Sorting by Source Geo
20
21
Zooming in time range on Second Spike
Drilling Deeper
21
22
Number of Unique Source IP Addresses
Checking another Dimension
22
23
Flip to: Destination Addresses
Where is the Traffic Going?
23
24
Looking at all inbound traffic to the target victim Dest IP
Pulling Back to Gauge the Situation
24
25
Attack details by protocol
Narrowing in on the Actual Attack
25
26
Multiple simultaneous vectors at hand
The Finding: Multi-Layered Attack
26
27
Finding the Necessary Details for Setting Filter Policies
The Mitigation Plan
27
28
- Unusual traffic patterns from suspect Geo- Turned out to be DNS Amplification targeting a specific dest IP- But main attack was hiding other attacks/exploits- Data harvested for mitigation
- Time required to complete this analysis: 3 minutes!- How is this possible???
Case Example: Summary
28
29
Big Data Analytics for DDoSKey to Advanced DDoS Detection and Forensics
29
DDoS Detection Tooling – Major Decision Points1. Packet-based or Flow-based?
• Packet-based requires in-line inspection, usu. via appliances ($$)
• Flow-based can be local/appliance or SaaS
2. Fully Integrated with Mitigation, or Best of Breed?
• Fully Integrated only works when mitigation is “always on”
• Independent detection ensures mitigation flexibility
3. Next-Gen Data Architecture, or Legacy?
30
DDoS Detection Tooling – Data ArchitectureKey Question
“To Summarize or Not to Summarize??”
Advantages of Summarization
- More compact long term data store
- Faster (?) searches against history
Disadvantages of Summarization- Major Loss of essential detail!!
Only Viable Answer: NO SUMMARIZATION 31
Big Data for Next-Gen DDoS DetectionWhy Big Data??Network Monitoring Data IS Big Data
• Meets Volume/Variety/Velocity Test
• Billions of records/day (millions/second)Big Data architectures:
• Mature, viable for hyper-scale, real-time data sets – SCALABLE, RELIABLE
• Capable of performance at scale for analyzing ALL data – not just summaries/metadata –RESULTS IN SECONDS
Big Data Analytics: The DDoS Detection PayoffWhat Do I Get by Going With Big Data?
• Accuracy
• Having ALL raw data available, not just what was pre-defined
• Essential for answering key questions like: Is this Friend or Foe?
• Flexibility
• Don’t have to wait for vendor to support new attack profiles
• Easy to add more data types/sets to enrich the story
• Can export data quickly/easily to other systems
Kentik’s ApproachNext Gen Big Data NetFlow Analytics for DDoS Detection…. And more
34
Kentik Detect: the first and only SaaS SolutionFor Network Ops Management & Visibility at Terabit Scale
CLOUD- BAS ED REAL- T I M E MULT I - TENANT OP EN GLOBAL
Analyze & Take Action
Big Data NetworkTelemetry Platform
in the Cloud
The Network is the Sensor
Web Portal
Real-time & historical queries
NetFlow/sFlow/IPFIX
SNMPBGP
Alerts: DDoS, Ops
E-mail / Syslog / JSON
Open API
SQL / RESTful
Kentik Data Engine
35
Multi-tiered/Clustered Big Data Architecture for Scale / Load Balancing / HA
What’s Behind Kentik Detect : The Kentik (big) Data Engine
POSTGRESSERVERS
SQL
DATA STORAGE CLUSTER
NetFlowSNMPBGP
INGEST CLUSTER
CLIENTS
N M
Optimized for Massive Data Ingest & Rapid Query Response36
NextGen NetFlow Analytics: Full Detail, Fast Navigation, Infinite Granularity
37
NextGen NetFlow Analytics: Dashboards in Seconds
38
Key Takeaways
What NextGen DDoS Detection Can (Should) Do for You: - Deliver true live monitoring & alerting
- Quickly recognize / analyze attacks
- Operate on a full data set, not just summaries or pre-defined rules
- Support multiple mitigation options
- Enable automation
39
Network Intelligence at Exabit Scale
Thank You!
Jim FreyVP Product
Kentik [email protected]
@jfrey80
Top Related