Network Security Professor Adeel Akram. Network Security Architecture.
Network Security Today: Finding Complex Attacks at 100Gb/s · Sabotage. Network Security Today...
Transcript of Network Security Today: Finding Complex Attacks at 100Gb/s · Sabotage. Network Security Today...
Network Security Today
Robin Sommer!International Computer Science Institute, &!
Lawrence Berkeley National Laboratory
[email protected] http://www.icir.org/robin
Network Security Today: Finding Complex Attacks at 100Gb/s
Network Security Today
The Old Days …
2
Border Traffic!Lawrence Berkeley National Lab (Today)!
10GE upstream, 4,000 user, 12,000 hosts
Total connections
Network Security Today
The Old Days …
2
Border Traffic!Lawrence Berkeley National Lab (Today)!
10GE upstream, 4,000 user, 12,000 hosts
Attempted connectionsSuccessful connectionsTotal connections
Network Security Today
The Old Days …
2
Border Traffic!Lawrence Berkeley National Lab (Today)!
10GE upstream, 4,000 user, 12,000 hosts
Attempted connectionsSuccessful connectionsTotal connections
Network Security Today
Today’s Threats
3
Network Security Today
Today’s Threats
3
Trend 1: Commercialization of attacks!Thriving underground economy (“Crime-as-a-Service”).!Bear Race: Attack is good enough if it pays.
Source: Gary Larson
Network Security Today
Today’s Threats
3
Trend 1: Commercialization of attacks!Thriving underground economy (“Crime-as-a-Service”).!Bear Race: Attack is good enough if it pays.
Trend 2: High-skill / high-resource attacks.!Activist Hacking.!Advanced Persistent Threats / Nation-states.
Source: Wikimedia CommonsSource: Computer Security Articles Source: EFF
Network Security Today
Today’s Threats
3
Trend 1: Commercialization of attacks!Thriving underground economy (“Crime-as-a-Service”).!Bear Race: Attack is good enough if it pays.
Trend 2: High-skill / high-resource attacks.!Activist Hacking.!Advanced Persistent Threats / Nation-states.
Trend 3: Insider Attacks!Exfiltration !Sabotage
Network Security Today
Defender Challenges
Varying threat models.!No ring rules them all.
4
Network Security Today
Defender Challenges
Varying threat models.!No ring rules them all.
Semantic complexity.!The action is really at the application-layer.
4
Network Security Today
Defender Challenges
Varying threat models.!No ring rules them all.
Semantic complexity.!The action is really at the application-layer.
Volume and variability.!Network traffic is an enormous haystack.
4
Network Security Today
Deep Packet Inspection at High Speed
5
Network Security Today
Analyzing Semantics
6
Network Security Today
Analyzing Semantics
6
Tap
Internet Internal
Network
IDS
Example: Finding downloads of known malware. !
Network Security Today
Analyzing Semantics
6
Tap
Internet Internal
Network
IDS
1. Find and parse all Web traffic.!2. Find and extract binaries.!3. Compute hash and compare with database.!4. Report, and potentially kill, if found.
Example: Finding downloads of known malware. !
Network Security Today
Back in 2005 …
7
Data: Leibniz-Rechenzentrum, München
020
4060
80
TByt
es/m
onth
1997 1998 1999 2000 2001 2002 2003 2004 2005
Total bytesIncoming bytes
Total upstream bytesIncoming bytes
Munich Scientific Network (2005)!3 major universities, 1 GE upstream!~100,000 Users!~50,000 Hosts
Network Security Today
Back in 2005 …
8
Data: Leibniz-Rechenzentrum, München
050
010
0015
00
TByt
es/m
onth
1996 1998 2000 2002 2004 2006 2008 2010 2012
Total bytesIncoming bytes
Oct 2005
Total upstream bytesIncoming bytes
Munich Scientific Network (Today)!3 major universities, 2x10GE upstream!~100,000 Users!~65,000 Hosts
Network Security Today
Traditional Gap: Research vs. Operations
Conceptually simple tasks can be hard in practice.!Academic research often neglects operational constraints.!Operations cannot leverage academic results. !
We focus on working with operations.!Close collaborations with several large sites.!Extremely fruitful for both sides.
9
Network Security Today
Research Platform: Bro
10
Network Security Today
Research Platform: Bro
10
Originally developed by Vern Paxson in 1996.!
Open-source, BSD-license, maintained at ICSI and NCSA.!
In operational use since the beginning. !
Conceptually very different from other IDS.
http://www.bro.org
Network Security Today
Architecture
11
Network
Packets
Network Security Today
Architecture
11
Network
Event EngineProtocol Decoding
Events
Packets
Network Security Today
Architecture
11
Network
Event EngineProtocol Decoding
Script InterpreterAnalysis Logic
Logs
Events
Packets
Notification
Network Security Today
Architecture
11
Network
Event EngineProtocol Decoding
Script InterpreterAnalysis Logic
Logs
Events
Packets
Notification“User Interface”
Network Security Today
Script Example: Matching URLs
12
Task: Report all Web requests for a file “passwd”
Network Security Today
Script Example: Matching URLs
12
Task: Report all Web requests for a file “passwd”
!event http_request(c: connection, # Connection.! method: string, # HTTP method.! original_URI: string, # Requested URL.! unescaped_URI: string, # Decoded URL.! version: string) # HTTP version.!{! if ( method == "GET" && unescaped_URI == /.*passwd/! )! NOTICE(...); # Alarm.!}
Network Security Today
Script Example: Scan Detector
13
Task: Count failed connection attempts per source address.
Network Security Today
Script Example: Scan Detector
13
Task: Count failed connection attempts per source address.
global attempts: table[addr] of count &default=0;!!event connection_rejected(c: connection)!{! local orig = c$id$orig_h; # Get originator address.!! local n = ++attempts[orig]; # Increase counter.! ! if ( n == SOME_THRESHOLD ) # Check for threshold.! NOTICE(...); # Alarm.!}
Network Security Today
“Who’s Using It?”
14
Diverse Deployment Base Universities
Research Labs Supercomputer Centers
Government Organizations Fortune 20 Enterprises
Recent User Meetings Bro Workshops 2011/13 at NCSA
Bro Exchange 2012 at NCAR
Attended by about 50-80 operators from from 30-40 organizations
Examples Lawrence Berkeley National Lab
National Center for Supercomputing Applications National Center for Atmospheric Research
Indiana University !
... and many more sites
Fully integrated into Security Onion Popular security-oriented Linux distribution
Network Security Today
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011
Bro History
1995 20101996 2012
Vern writes 1st line of code!
2013
Network Security Today
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011
Bro History
1995 20101996 2012
Vern writes 1st line of code!
2013
Bro SDCI!
v2.0!New Scripts
v0.2!1st CHANGES!
entry!
v0.6!RegExps!
Login analysis!!
v0.8aX/0.9aXSSL/SMB!
STABLE releases!BroLite
v1.1/v1.2!when Stmt!Resource
tuning!Broccoli!
DPD!
v1.5!BroControl!
v0.7a90!Profiling!
State Mgmt
v1.4!DHCP/BitTorrent!
HTTP entities!NetFlow!
Bro Lite Deprecated!
v1.0!BinPAC!
IRC/RPC analyzers!64-bit support!Sane version
numbers!
v0.4 HTTP analysis!Scan detector!IP fragmentsLinux support!
v0.7a175/0.8aX !Signatures!
SMTP!IPv6 support!User manual!!
v0.7a48!Consistent CHANGES
v1.3!Ctor expressions!
GeoIP!Conn Compressor
0.8a37!Communication!
Persistence!Namespaces!Log Rotation
LBNL starts using Bro!
operationally
v2.1!IPv6!
Input Framew.
v2.2!File Analysis!
Summary Stat.
Bro Center!
Network Security Today
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011
Bro History
1995
USENIX Paper!Stepping Stone
Detector!
AnonymizerActive Mapping!Context Signat.!
TRWState Mgmt.!
Independ. State!
Host Context!Time Machine!
Enterprise Traffic
BinPAC!DPD!
2nd Path
Bro ClusterShunt
Autotuning
Parallel Prototype
20101996
Academic Publications
Input Framework
2012
Vern writes 1st line of code!
2013
Bro SDCI!
v2.0!New Scripts
v0.2!1st CHANGES!
entry!
v0.6!RegExps!
Login analysis!!
v0.8aX/0.9aXSSL/SMB!
STABLE releases!BroLite
v1.1/v1.2!when Stmt!Resource
tuning!Broccoli!
DPD!
v1.5!BroControl!
v0.7a90!Profiling!
State Mgmt
v1.4!DHCP/BitTorrent!
HTTP entities!NetFlow!
Bro Lite Deprecated!
v1.0!BinPAC!
IRC/RPC analyzers!64-bit support!Sane version
numbers!
v0.4 HTTP analysis!Scan detector!IP fragmentsLinux support!
v0.7a175/0.8aX !Signatures!
SMTP!IPv6 support!User manual!!
v0.7a48!Consistent CHANGES
v1.3!Ctor expressions!
GeoIP!Conn Compressor
0.8a37!Communication!
Persistence!Namespaces!Log Rotation
LBNL starts using Bro!
operationally
v2.1!IPv6!
Input Framew.
v2.2!File Analysis!
Summary Stat.
Bro Center!
Network Security Today
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011
Bro History
1995
USENIX Paper!Stepping Stone
Detector!
AnonymizerActive Mapping!Context Signat.!
TRWState Mgmt.!
Independ. State!
Host Context!Time Machine!
Enterprise Traffic
BinPAC!DPD!
2nd Path
Bro ClusterShunt
Autotuning
Parallel Prototype
20101996
Academic Publications
Input Framework
2012
Vern writes 1st line of code!
2013
Bro SDCI!
v2.0!New Scripts
v0.2!1st CHANGES!
entry!
v0.6!RegExps!
Login analysis!!
v0.8aX/0.9aXSSL/SMB!
STABLE releases!BroLite
v1.1/v1.2!when Stmt!Resource
tuning!Broccoli!
DPD!
v1.5!BroControl!
v0.7a90!Profiling!
State Mgmt
v1.4!DHCP/BitTorrent!
HTTP entities!NetFlow!
Bro Lite Deprecated!
v1.0!BinPAC!
IRC/RPC analyzers!64-bit support!Sane version
numbers!
v0.4 HTTP analysis!Scan detector!IP fragmentsLinux support!
v0.7a175/0.8aX !Signatures!
SMTP!IPv6 support!User manual!!
v0.7a48!Consistent CHANGES
v1.3!Ctor expressions!
GeoIP!Conn Compressor
0.8a37!Communication!
Persistence!Namespaces!Log Rotation
LBNL starts using Bro!
operationally
v2.1!IPv6!
Input Framew.
v2.2!File Analysis!
Summary Stat.
Bro Center!
Example: Processing performance!LBNL operations had trouble keeping up.!Research question: How can Bro scale up?
Network Security Today
Load-balancing Architecture
16
Network Security Today
Load-balancing Architecture
16
Detection LogicPacket Analysis
NIDS
10G
Network Security Today
Load-balancing Architecture
16
10G
Exte
rnal
Pac
ket L
oad-
Bala
ncer!
Flows
Detection Logic
Packet Analysis
NIDS 2
Detection Logic
Packet Analysis
NIDS 1
Detection Logic
Packet Analysis
NIDS 3
1G
1G
1G
Network Security Today
Load-balancing Architecture
16
10G
Exte
rnal
Pac
ket L
oad-
Bala
ncer!
Flows
Detection Logic
Packet Analysis
NIDS 2
Detection Logic
Packet Analysis
NIDS 1
Detection Logic
Packet Analysis
NIDS 3
Communication
Communication
1G
1G
1G
Network Security Today
Load-balancing Architecture
16
10G
Exte
rnal
Pac
ket L
oad-
Bala
ncer!
Flows
“Bro Cluster”
Detection Logic
Packet Analysis
NIDS 2
Detection Logic
Packet Analysis
NIDS 1
Detection Logic
Packet Analysis
NIDS 3
Communication
Communication
1G
1G
1G
Network Security Today
A Production Load-Balancer
1717
Network Security Today
A Production Load-Balancer
1717
cFlow: 10GE line-rate, stand-alone load-balancer
10 Gb/s in/out!Web & CLI!
Filtering capabilities!!
Network Security Today
A Production Load-Balancer
1717
cFlow: 10GE line-rate, stand-alone load-balancer
10 Gb/s in/out!Web & CLI!
Filtering capabilities!!
Network Security Today
Next Stop: 100 Gb/s
18
Source: ESNet
Now these sites need a monitoring solution ... Working with cPacket on a 100GE load-balancer!
DOE/ESNet !100G Advanced Networking Initiative
2011
Source: ESNet
Network Security Today
Next Stop: 100 Gb/s
19
Source: ESNet
2014
Network Security Today
On Deck: 400G Connectivity
20
Computational Research and Theory Building.
Oakland Scientific Facility.
100G
2 x 100G
File System Links
Inter-site Traffic
100G WAN 100G WAN
Berkeley National Laboratory
Sources: ESNet/LBNL/NERSC
Network Security Today
10G 10G10G
Science DMZ
21
Campus LAN
Internet
Network Security Today
100G 100G100G
Science DMZ
21
Campus LAN
Internet
Network Security Today
10G 10G
Science DMZ
21
Campus LAN
100GInternet
Network Security Today
10G 10G
Science DMZ
21
Campus LAN
100G
100G
Transfer/Storage Nodes
100G
Science DMZ Switch
Internet
Network Security Today
10G 10G
Science DMZ
21
Campus LAN
100G
Clean, high-bandwith path
Low-bandwidth!campus access
100G
Transfer/Storage Nodes
100G
Science DMZ Switch
Internet
Network Security Today
10G 10G10G
100G
Science DMZ
22
Campus LAN
100G
Transfer/Storage Nodes
100G
Science DMZ Switch
100GInternet
Network Security Today
100G
10G 10G10G
100G
Science DMZ
22
Campus LAN
100G
Transfer/Storage Nodes
100G
Science DMZ Switch
100GInternet
Network Security Today
100G Bro Cluster
23
100G
Science DMZ Switch
Network Security Today
100G Bro Cluster
23
100G Load-balancer
100G
Science DMZ Switch
Network Security Today
100G Bro Cluster
23
100G Load-balancer
10G
100G
Science DMZ Switch
Network Security Today
100G Bro Cluster
23
100G Load-balancer
10G
Bro Cluster
100G
Science DMZ Switch
Network Security Today
100G Bro Cluster
23
100G Load-balancer
10G
Bro Cluster
API
Con
trol
100G
Science DMZ Switch
Network Security Today
100G Bro Cluster
23
100G Load-balancer
10G
Bro Cluster
API
Con
trol
100G
Science DMZ Switch
Con
trol
API
Network Security Today
100G Bro Cluster
23
100G Load-balancer
10G
Bro Cluster
API
Con
trol
100G
Science DMZ Switch
Con
trol
API
Network Security Today
Parallelizing DPI on Multi-core Systems
24
Network Security Today
Going Multi-Core …
Bro is single-threaded!Cluster backends have muitple cores, mostly idle.!Work-around: “Cluster in a box”!
We really want multi-threading, though.!Needs to scale well with increasing numbers of cores.!Needs to be transparent to the operator.!
For some IDS, that’s not so hard.!For others, it is ...
25
Network Security Today
Concurrent Analysis
26
Network
Event EngineProtocol Decoding
Script InterpreterAnalysis Logic
Logs
Events
Packets
Notification
Network Security Today
Concurrent Analysis
26
Single Thread
Network
Event EngineProtocol Decoding
Script InterpreterAnalysis Logic
Logs
Events
Packets
Notification
Network Security Today
Concurrent Analysis
27
Event Engine
Network
Packets
Events
Notification
Script ThreadsScripting Language
Event Engine! ThreadsPacket Analysis
Detection Logic
Dispatcher Kernel or NIC
Network Security Today
Concurrent Analysis
27
Event Engine
Network
Packets
Events
Notification
Script ThreadsScripting Language
Event Engine! Threads
“Cluster in a Box”
Packet Analysis
Detection Logic
Dispatcher Kernel or NIC
Network Security Today
Concurrent Analysis
27
Event Engine
Network
Packets
Events
Notification
Script ThreadsScripting Language
Event Engine! Threads
“Cluster in a Box”
Packet Analysis
Detection Logic
Dispatcher Kernel or NIC
How to parallelize!a scripting language?
Network Security Today
How to Parallelize Event Handlers?
28
Simple: State-less Analysis
Network Security Today
How to Parallelize Event Handlers?
28
Simple: State-less Analysis
!event http_request(c: connection, # Connection.! method: string, # HTTP method.! original_URI: string, # Requested URL.! unescaped_URI: string, # Decoded URL.! version: string) # HTTP version.!{! if ( method == "GET" && unescaped_URI == /.*passwd/! )! NOTICE(...); # Alarm.!}
Network Security Today
How to Parallelize Event Handlers? (2)
29
Challenging: Analysis that keeps global state.
Network Security Today
How to Parallelize Event Handlers? (2)
29
Challenging: Analysis that keeps global state.
global attempts: table[addr] of count &default=0;!!event connection_rejected(c: connection)!{! local orig = c$id$orig_h; # Get originator address.!! local n = ++attempts[orig]; # Increase counter.! ! if ( n == SOME_THRESHOLD ) # Check for threshold.! NOTICE(...); # Alarm.!}
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
LOCK(attempts)!
++attempts[s]!
UNLOCK(attempts)!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
LOCK(attempts)!
++attempts[s]!
UNLOCK(attempts)!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
LOCK(attempts)!
++attempts[s]!
UNLOCK(attempts)!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
attempts_1
attempts_2
attempts_3
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator!
!
++attempts_1[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_3[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_2[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
attempts_1
attempts_2
attempts_3
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator!
!
++attempts_1[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_3[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_2[s]!
!
hash(addr)1
1
2
2
3
3
hash: addr -> {1, 2 ,3}
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
attempts_1
attempts_2
attempts_3
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator!
!
++attempts_1[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_3[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_2[s]!
!
hash(addr)1
1
2
2
3
3
hash: addr -> {1, 2 ,3}
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
attempts_1
attempts_2
attempts_3
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator!
!
++attempts_1[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_3[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_2[s]!
!
hash(addr)1
1
2
2
3
3
hash: addr -> {1, 2 ,3}
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
attempts_1
attempts_2
attempts_3
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator!
!
++attempts_1[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_3[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_2[s]!
!
hash(addr)1
1
2
2
3
3
hash: addr -> {1, 2 ,3}
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
Thread 1’s attempts
Thread 2’s attempts
Thread 3’s attempts
Network Security Today
!!
connection_rejected(c):!
!
s = c.originator !
! ++attempts[s]!
!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
! ++attempts[s]!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator!
!
++attempts_1[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_3[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_2[s]!
!
hash(addr)1
1
2
2
3
3
hash: addr -> {1, 2 ,3}
Thread 1!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread 3!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread 2!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts_(hash(s))[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 134.96.7.179 !
s = c.originator !
! ++attempts[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 131.159.15.49 !
s = c.originator !
!
++attempts[s]!
!
Thread hash(s)!!
connection_rejected(c):!
# 192.150.187.12!
s = c.originator !
!
++attempts[s]!
!
Parallelizing Event Execution
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
30
attempts[addr] of count
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
addr count131.234.142.33 12
134.96.7.179 32
141.142.192.147 71
192.150.187.12 8
128.3.41.105 555
131.159.15.49 1
Thread 1’s attempts
Thread 2’s attempts
Thread 3’s attempts
Network Security Today
Parallel Event Scheduling
31
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
http_request
Conn X
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
http_request
Conn X
http_request
Conn Y
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
http_request
Conn X
http_reply
Conn
http_request
Conn Y
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
http_request
Conn X
http_reply
Conn
http_request
Conn Y
http_reply
Conn Y
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
http_request
Conn X
http_reply
Conn
http_request
Conn Y
http_reply
Conn Y
conn_rejected
Orig A
conn_rejected
Orig A
Network Security Today
Parallel Event Scheduling
31
Thread!1
Thread!2
Thread!3
Thread!4 … Thread!
n
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
Que
ue
conn_rejected
Orig A
conn_rejected
Orig B
http_request
Conn X
http_reply
Conn
http_request
Conn Y
http_reply
Conn Y
conn_rejected
Orig A
conn_rejected
Orig A
Challenge: Implementing this …
Network Security Today
New Platform: Abstract Machine
32
A High-Level Intermediary Language for Traffic Inspection
Network Security Today
New Platform: Abstract Machine
32
First-class networking types
built-in
Containers with state management
support
Platform for building high-level, reusable
functionality onDomain-specific
concurrency modelWell-defined,
contained execution environment
Domain-specific Data Types
Robust/Secure Execution
Concurrent Analysis
High-level Standard
Components
State Management
Timers can drive execution
Real-time Performance
Support for incremental processing
Extensive optimization
potential
Scalability through parallelization
Static type-system, and robust error
handlingCompilation to
native code
A High-Level Intermediary Language for Traffic Inspection
Network Security Today
New Platform: Abstract Machine
32
First-class networking types
built-in
Containers with state management
support
Platform for building high-level, reusable
functionality onDomain-specific
concurrency modelWell-defined,
contained execution environment
Domain-specific Data Types
Robust/Secure Execution
Concurrent Analysis
High-level Standard
Components
State Management
Timers can drive execution
Real-time Performance
Support for incremental processing
Extensive optimization
potential
Scalability through parallelization
Static type-system, and robust error
handlingCompilation to
native code
A High-Level Intermediary Language for Traffic Inspection
Network Security Today
Summary
33
Network Security Today
Conclusions
Threats have changed.!Detection requires deep, flexible, semantic analysis.!
Working to push the limits. !Leverage capabilities of modern network hardware.!Exploit parallelism inherent in network traffic analysis.!
Bro is an ideal platform for such work.!Operationally deployed across the country.!Bridges traditional gap between academia and operations. !
34
Network Security Today
Robin Sommer!International Computer Science Institute, &!
Lawrence Berkeley National Laboratory
[email protected] http://www.icir.org/robin
Thanks for you attention!
35