February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of...
Transcript of February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of...
![Page 1: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/1.jpg)
February 10, 2009
Xin Hu, Matthew Knysz, Kang G. Shin{huxin, mknysz, kgshin}@eecs.umich.edu
Computer Science & Engineering, University of Michigan, Ann Arbor
![Page 2: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/2.jpg)
OutlineMotivation of RB‐SeekerSystem ArchitectureOverview of subsystemsEvaluation of resultsConclusion
2/10/2009 2
![Page 3: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/3.jpg)
Motivation: the botnet problem
Financial Incentive◦
Underground market
Common uses of botnets:◦
Redirection/Proxy, Spam, ID theft, DDoS, phishing
Can cause A LOT of damage◦
Can bring down entire systems or nations
2/10/2009 3
![Page 4: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/4.jpg)
Motivation: botnet appeal
Modular and AdaptableEvolve to overcome defenses
Distributed natureDifficult to find/stop botmaster
DiscreetPropagation, infection, and occupation
2/10/2009 4
![Page 5: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/5.jpg)
Motivation: Redirection/Proxy BotnetRedirect users to malicious servers◦
Additional layer of misdirection◦
Protect mothership servers◦
Evade URL based detection or IP based black list
Redirect tohttp://server2
Server1 (redirection bot)
Server2 (redirection bot)
Follow linkhttp://server2
Redirect tohttp://final_Server
Final destination
“mothership”
Real nefarious
content
Forwarding servers
Forwarding servers
Issue HTTP
request
Real nefarious
content
Real nefarious
content
Server2 (proxy bot)
2/10/2009 5
![Page 6: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/6.jpg)
Motivation: RB‐SeekerBotnet is an ideal source for redirection/proxy servers
Botnets used for multiple purposes/scams
Previous research: detection of C&C channel
2/10/2009 6
![Page 7: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/7.jpg)
Overview: RB‐SeekerAutomatic detection of redirection/proxy botnetsUtilizes 3 cooperating subsystemsBehavior‐based detection
Quick identification of aggressive botnets (FP < 0.01%)Advertise many IPs per queryChange IPs very often (short TTL)
Accurate identification of stealthy botnetsAdvertise few IPs per queryChange IPs more slowly (very small TTL, closely monitored)2/10/2009 7
![Page 8: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/8.jpg)
System Architecture
DNS
logs
Content
Analysis
…..Redirection
domain db
URL probing Engine
Spam url db
…….
Web Server nWeb Server 1
Spam source ( Spam trap, open relay, personal
junk mailbox)
DNS probing engine
Report
& alert
University
Core Router
Correlation
engine
Redirection
server IPs
DNS query
history
redirection
domains
Spam Source Subsystem (SSS)
NetFlow Analysis Subsystem (NAS)
Active DNS Anomaly Detection Subsystem (a-DADs)
Local DNS server
NetFlow
Exports
DNS
query
db
RBnet
classification
engine
2/10/2009 8
![Page 9: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/9.jpg)
SSS: Spam Source Subsystem
2/10/2009 9
![Page 10: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/10.jpg)
SSS: Spam Source Subsystem
1. Extract embedded URLs from message bodies
2.
Probe extracted URLs to identify redirection URL links3.
Domains added to redirection domain database
Content
Analysis
…..Redirection
domain db
URL probing Engine
Spam url db
Spam source ( Spam trap, open relay, personal
junk mailbox)
Spam Source Subsystem (SSS)
…….
Web Server nWeb Server 1
redirection
domains
2/10/2009 10
![Page 11: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/11.jpg)
System Architecture
DNS
logs
Content
Analysis
…..Redirection
domain db
URL probing Engine
Spam url db
…….
Web Server nWeb Server 1
Spam source ( Spam trap, open relay, personal
junk mailbox)
DNS probing engine
Report
& alert
University
Core Router
Correlation
engine
Redirection
server IPs
DNS query
history
redirection
domains
Spam Source Subsystem (SSS)
NetFlow Analysis Subsystem (NAS)
Active DNS Anomaly Detection Subsystem (a-DADs)
Local DNS server
NetFlow
Exports
DNS
query
db
RBnet
classification
engine
2/10/2009 11
![Page 12: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/12.jpg)
NAS: NetFlow Analysis SubsystemUse NetFlow because:
Inspecting packet contents incurs too much overheadPrivacy concerns
Spammers send image‐ or PDF‐based emailsEvade content‐based filtering
User redirected to RBnet by clicking on malicious webpageInspecting each email not always possible
Privacy concerns/laws
2/10/2009 12
![Page 13: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/13.jpg)
NAS: NetFlow Analysis SubsystemNetFlow: core router on campusLooks for suspicious redirection attempts
Without analyzing packet contents
University
Core Router
NetFlow Analysis Subsystem (NAS)
2/10/2009 13
![Page 14: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/14.jpg)
NAS: NetFlow Analysis SubsystemSequential Hypothesis testing on:
Flow size, inter‐flow duration, and flow duration
University
Core Router
NetFlow Analysis Subsystem (NAS)
NetFlow
Exports
2/10/2009 14
![Page 15: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/15.jpg)
NAS: NetFlow Analysis SubsystemIdentifies IPs participating in redirection
Correlation engine uses DNS logs to add domains participating in redirection to redirection domain db
DNS
logs
…..Redirection
domain db
University
Core Router
Correlation
engine
Redirection
server IPs
DNS query
history
redirection
domains
NetFlow Analysis Subsystem (NAS)
Local DNS server
NetFlow
Exports
2/10/2009 15
![Page 16: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/16.jpg)
Accept H1
PendingAccept H0
PendingAccept H0
NAS: NetFlow Analysis Subsystem
Redirection: obtained from SSS, servers identified as redirection
Normal: normal web browsing over 2 days (removing redirection)
Start
No
H.T. on
inter‐
flow
Sort all flows
chronologically
Yes
Accept H1
H.T on
flow
size
Optional H.T. on flow
duration
Inter‐flow >
threshold
H0
: NormalH1
: Redirection
H.T.
history
database
Normal
Accept H1
Redirection
Size H.T.
history
database
2/10/2009 16
Redirection Normal
![Page 17: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/17.jpg)
System Architecture
DNS
logs
Content
Analysis
…..Redirection
domain db
URL probing Engine
Spam url db
…….
Web Server nWeb Server 1
Spam source ( Spam trap, open relay, personal
junk mailbox)
DNS probing engine
Report
& alert
University
Core Router
Correlation
engine
Redirection
server IPs
DNS query
history
redirection
domains
Spam Source Subsystem (SSS)
NetFlow Analysis Subsystem (NAS)
Active DNS Anomaly Detection Subsystem (a-DADs)
Local DNS server
NetFlow
Exports
DNS
query
db
RBnet
classification
engine
2/10/2009 17
![Page 18: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/18.jpg)
a‐DADS: active DNS Anomaly Detection Subsystem
Actively performs DNS queries on domains in redirection domain dbUses CDN Filter to remove Content Delivery Networks
CDNs behave similarly to redirection/proxy botnetsRecursively removes CDNs
…..Redirection
domain db
DNS probing engine
Report
& alert
Active DNS Anomaly Detection Subsystem (a-DADs)
Local DNS server
DNS
query
db
RBnet
classification
engine
2/10/2009 18
![Page 19: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/19.jpg)
a‐DADS: active DNS Anomaly Detection Subsystem
IP Usage:RBnets will accrue more unique IPs over timeRBnets will have more unique IPs per valid query
Reverse DNS names with “bad words”e.g., broadband, cable, comcast, charter, etc…
AS countNumber of different ASes the IPs belong toRBnets consist of home computers scattered geographically
2/10/2009 19
![Page 20: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/20.jpg)
a‐DADS: active DNS Anomaly Detection Subsystem
Applies 2‐tier linear SVM on remaining domainsTrained: 124 valid, 18 aggressive, 10 stealth10‐fold cross validation on multiple classifiers
knn, decision tree, naïve Bayesian, various SVMs and kernel functions
2/10/2009 20
![Page 21: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/21.jpg)
a‐DADS: active DNS Anomaly Detection Subsystem
SVM‐1: detects Aggressive RBnets based on 2 valid queriesunique IPs, num ASes, DNS “bad words”
2/10/2009 21
![Page 22: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/22.jpg)
a‐DADs: SVM‐1 Aggressive RBnets
2/10/2009 22
![Page 23: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/23.jpg)
a‐DADS: active DNS Anomaly Detection Subsystem
SVM‐2: detects Stealth RBnets using a week of DNS queries unique IPs, num ASes
2/10/2009 23
![Page 24: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/24.jpg)
a‐DADs: SVM‐2 Stealth RBnets
2/10/2009 24
![Page 25: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/25.jpg)
Evaluation of ResultsSSS and NAS identified 91,600+ suspicious domains over 2 month perioda‐DADS CDN Filter
Removed 5,005 CDN domainsRecursion 16.8% increase in identified CDN domains (13.1% in IPs)Similar technique for valid domains reduced this to 35,000+ domains to be monitored
2/10/2009 25
![Page 26: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/26.jpg)
Evaluation of Results
SVM‐1: Experienced 1 FP (< 0.008%)
2/10/2009 26
![Page 27: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/27.jpg)
Aggressive RBnets: Redirection vs. Proxy Botnets
2/10/2009 27
48.8% 51.2%
![Page 28: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/28.jpg)
Stealth RBnets
2/10/2009 28
![Page 29: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/29.jpg)
Evaluation of ResultsFFSN detector:
Detected 124 of the 125 Aggressive RBnets1 FP: same as ours (mozilla.org)Missed all the Stealth RBnets
2/10/2009 29
![Page 30: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/30.jpg)
ConclusionDesigned and implemented system for detecting redirection/proxy botnetsUses network detection techniques
multiple data sources readily available to enterprise network environments
Behavior‐based detection works despite use of C&C protocol or structureCapable of detecting Aggressive and Stealthy RBnetsAutomatic detection with low false positives (< 0.01%)
2/10/2009 30
![Page 31: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/31.jpg)
Questions?
2/10/2009 31
![Page 32: February 10, 2009 - NDSS Symposium · Motivation of RB‐Seeker y System Architecture y Overview of subsystems y Evaluation of results y Conclusion 2/10/2009 2. Motivation: the botnet](https://reader035.fdocuments.in/reader035/viewer/2022071019/5fd28bee28d6470ec658bd3e/html5/thumbnails/32.jpg)
Evaluation of Results
2/10/2009 32