ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK...
Transcript of ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK...
![Page 1: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/1.jpg)
Soo-Jin MoonJeffrey Helt, Yifei Yuan, Yves Bieri,
Sujata Banerjee, Vyas Sekar, Wenfei Wu, Mihalis Yannakakis, Ying Zhang
Carnegie Mellon Univ., Princeton Univ., Intentionet, ETH Zurich, VMware Research, Tsinghua Univ., Columbia Univ., Facebook, Inc.
Contributions by Soo-Jin Moon were made in-part during a former internship at Hewlett Packard Labs. Other contributors from former employees at Hewlett Packard Labs include Sujata Banerjee, Ying Zhang and Wenfei Wu.
ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS
![Page 2: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/2.jpg)
Stateful Network Functions (NFs) in Modern Networks
Firewalls and NATs
Modern networks contain a wide range of complex stateful network functions from many vendors
IDS/IPSsLoad balancers
2
![Page 3: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/3.jpg)
Motivating Example: Stateful Firewall (FW)
FW
If a connection is ESTABLISHED from the LAN, allow TCP traffic from the WAN else DROP
LAN WAN
Host A Host B
SYNSYN①
Connection Map:
3
![Page 4: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/4.jpg)
Motivating Example: Stateful Firewall (FW)
FW
If a connection is ESTABLISHED from the LAN, allow TCP traffic from the WAN else DROP
LAN WAN
Host A Host B
SYNSYN①SA SA ②
Connection Map:
3
![Page 5: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/5.jpg)
Motivating Example: Stateful Firewall (FW)
FW
If a connection is ESTABLISHED from the LAN, allow TCP traffic from the WAN else DROP
LAN WAN
Host A Host B
SYNSYN①SA SA ②
Connection Map:
ACK ACK③
3
![Page 6: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/6.jpg)
Motivating Example: Stateful Firewall (FW)
FW
If a connection is ESTABLISHED from the LAN, allow TCP traffic from the WAN else DROP
LAN WAN
Host A Host B
SYNSYN①SA SA ②
Connection Map:
ACK ACK③ DATA DATA ④
A → B == ESTABLISHED
3
![Page 7: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/7.jpg)
Motivating Example: Stateful Firewall (FW)
FW
If a connection is ESTABLISHED from the LAN, allow TCP traffic from the WAN else DROP
LAN WAN
Host A Host B
SYNSYN①SA SA ②
ACK ACK③ DATA DATA ④
Connection Map: A → B == ESTABLISHED
SYN
Host C
3
![Page 8: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/8.jpg)
Network Testing and Verification
Stateful NF
LAN WAN
Host A Host B
? • Is the policy implemented correctly?• Can we check before on-boarding?
We need network testing/verification tools (e.g.,VMN , SYMNET, BUZZ…)
Operator If a connection is ESTABLISHED from the LAN, allow TCP traffic from the WAN else DROP
4
![Page 9: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/9.jpg)
Today: Need NF Models for Testing and Verification
StatefulNFLAN WAN
Host A Host B
Model (e.g., finite state machine)
TestingVerificationOn-boarding
Today, these NF models are handwritten based on manual investigation
Config with intended policy
5
![Page 10: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/10.jpg)
Limitation of Handwritten Model: InaccuracyNetwork testing tool e.g., BUZZ [NSDI 16]
Test traffic(from BUZZ)
Intended Policy
Real FW
Handwritten Model
SYN
SA
SYN
SYN
SA
SYN
SA
SYN
SA
SYN
Error! ≠
Handwritten FW model
6
![Page 11: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/11.jpg)
Limitation of Handwritten Model: InaccuracyNetwork testing tool e.g., BUZZ [NSDI 16]
Test traffic(from BUZZ)
Intended Policy
Real FW
Handwritten Model
SYN
SA
SYN
SYN
SA
SYN
SA
SYN
SA
SYN
≠
Handwritten FW model
6
![Page 12: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/12.jpg)
Limitation of Handwritten Model: InaccuracyNetwork testing tool e.g., BUZZ [NSDI 16]
≠
Handwritten FW model
Real FW implementation
/NULL
SYN
SA /
/ACK
SYN SENT
SASENT
Else/
Else/
. . . .
*/
SA
SYN
ACK
EST
Handwritten FW model (BUZZ, NSDI 16)
/NULL
SYN
SA /
SYN SENT
SASENT
Else/
…
*/
SA
SYN
=6
![Page 13: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/13.jpg)
Limitation of Handwritten Model: InaccuracyNetwork testing tool e.g., BUZZ [NSDI 16]
≠
Handwritten FW model
Real FW implementation
/NULL
SYN
SA /
/ACK
SYN SENT
SASENT
Else/
Else/
. . . .
*/
SA
SYN
ACK
EST
Handwritten FW model (BUZZ, NSDI 16)
/NULL
SYN
SA /
SYN SENT
SASENT
Else/
…
*/
SA
SYN
=6
![Page 14: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/14.jpg)
Limitation of Handwritten Model: InaccuracyNetwork testing tool e.g., BUZZ [NSDI 16]
≠
Handwritten FW model
Real FW implementation
/NULL
SYN
SA /
/ACK
SYN SENT
SASENT
Else/
Else/
. . . .
*/
SA
SYN
ACK
EST
Handwritten FW model (BUZZ, NSDI 16)
/NULL
SYN
SA /
SYN SENT
SASENT
Else/
…
*/
SA
SYN
=6
![Page 15: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/15.jpg)
Test traffic(from BUZZ)
Untangle FW
PropNFFW
SYN
SA
SYN
SYN
SA
SYN
ACK
Vendor-specific differences
Limitation of Handwritten Model: Vendor Diversity
SYN
SA
SYN
Vendors have different implementations!
7
![Page 16: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/16.jpg)
Our Work: Alembic
StatefulNF
Config
Customers: 1) BUZZ [NSDI16]2) SYMNET [SIGCOMM16]3) VMN [NSDI17]
Automatically infer a behavioral model of the NF for a configuration
Finite State Machine (FSM)Model = NF(config)}
8
![Page 17: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/17.jpg)
Talk Outline
• Motivation and Goal
• Challenges and Insights
• Overall Workflow
• Evaluation
9
![Page 18: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/18.jpg)
High-Level Challenges
StatefulNF
Inferring NF behavior
Large configuration space
. . .
Config N
Rule 1 Rule 2
… Rule 1000Config 1
10
![Page 19: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/19.jpg)
Challenges on Large Configuration Space
• Configuration ! many rules
• Rule ! IP/port fields take large sets of values (e.g., 232 for IPs)
• Rule ! IP/port fields can be ranges (e.g., /16 for IP prefixes)
11
![Page 20: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/20.jpg)
Rule1
Rule2
. . .
RuleN
Rule1
Rule2
. . .
RuleN
Insight 1: We Can Compose Models of Individual Rules
StatefulNF
Rule1
Rule2
. . . RuleN
. . .
Model
Naive solution
12
![Page 21: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/21.jpg)
Rule1
Rule2
. . .
RuleN
Rule1
Rule2
. . .
RuleN
Insight 1: We Can Compose Models of Individual Rules
StatefulNF
Rule1
Rule2
. . . RuleN
. . .
Model2
Model1
ModelN
12
![Page 22: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/22.jpg)
Rule1
Rule2
. . .
RuleN
Rule1
Rule2
. . .
RuleN
Insight 1: We Can Compose Models of Individual Rules
StatefulNF
Rule1
Rule2
. . . RuleN
. . .
ProcessOrder
“compose” per rule models
Model2
Model1
ModelN
12
![Page 23: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/23.jpg)
Challenges on Large Configuration Space
• Configuration ! many rules
• Rule ! IP/port fields take large sets of values (e.g., 232 for IPs)
• Rule ! IP/port fields can be ranges (e.g., /16 for IP prefixes)
13
![Page 24: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/24.jpg)
Insight 2: Use Symbolic Models to represent Large SetsRule 1: SRC IP:10… DST IP:15
S0 S1
10→15
15→10
Else 10→15
15→10
Rule 2: SRC IP:12… DST IP:15
12→15Else
12→15
S0 S1
15→1215→12
14
![Page 25: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/25.jpg)
Insight 2: Use Symbolic Models to represent Large SetsRule 1: SRC IP:10… DST IP:15
S0 S1
10→15
15→10
Else 10→15
15→10
Rule 2: SRC IP:12… DST IP:15
12→15Else
12→15
S0 S1
15→1215→12
SRC IP:A… DST IP:B
M(A,B) = S0 S1
A→B
B→A
Else A→B
B→A
14
![Page 26: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/26.jpg)
Insight 2: Use Symbolic Models to represent Large SetsRule 1: SRC IP:10… DST IP:15
S0 S1
10→15
15→10
Else 10→15
15→10
Rule 2: SRC IP:12… DST IP:15
12→15Else
12→15
S0 S1
M(A,B) where A = 13, B = 16SRC IP:13… DST IP:16If we get a new config:
15→1215→12
SRC IP:A… DST IP:B
M(A,B) = S0 S1
A→B
B→A
Else A→B
B→A
14
![Page 27: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/27.jpg)
Challenges on Large Configuration Space
• Configuration ! many rules
• Rule ! IP/port fields take large sets of values (e.g., 232 for IPs)
• Rule ! IP/port fields can be ranges (e.g., /16 for IP prefixes)
15
![Page 28: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/28.jpg)
Insight 3: Exploit Independence to Create an Ensemble of FSMsSRC IP:10.1.1.0/16…DST IP:15.1.1.0/16
16
![Page 29: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/29.jpg)
Insight 3: Exploit Independence to Create an Ensemble of FSMsSRC IP:10.1.1.0/16…DST IP:15.1.1.0/16
Independent packet processing per connectionPer-connection
Conn 1 : 10.1.1.1 → 15.1.1.1
Conn 2 : 10.1.1.2 → 15.1.1.2States do not interfere}
1616
![Page 30: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/30.jpg)
Insight 3: Exploit Independence to Create an Ensemble of FSMsSRC IP:10.1.1.0/16…DST IP:15.1.1.0/16
An ensemble of concrete FSMs can represent a rule with IP/port ranges
Per-connection
(symbolic model from insight 2)
Learn M(A, B)
[10.1.1.1→15.1.1.1]
S0 S1
10.1.1.1…
15.1.1.1…
Else
10.1.1.1…
15.1.1.1…
Instantiate
at runtime
Ensemble of FSMs
Independent packet processing per connection
16
![Page 31: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/31.jpg)
Summary of Insights to Address Large Configuration Space
A configuration is composed of many number of rules
Compositional Model
A rule contains IP/port fields which take large sets of values and ranges.
An Ensemble of FSMsSymbolic ModelInstantiation
17
![Page 32: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/32.jpg)
Back to High-Level Challenges
StatefulNF
Inferring NF behavior
Large configuration space
. . .
Config N
Rule 1 Rule 2
… Rule 1000Config 1
18
![Page 33: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/33.jpg)
• Inferring the symbolic FSM
• Inferring the state granularity
• Handling dynamic header modification
Challenges on Inferring NF Behavior
19
![Page 34: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/34.jpg)
Insight: Leverage L* Algorithm to Infer a Symbolic FSM
NF
Config
Alembic
Blackbox L* algorithm
≡ FSM representing the blackbox
We can use the L* algorithm!20
![Page 35: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/35.jpg)
Background on L* for Black-box FSM Inference
Input Alphabet (Σ = {a,b})
• Generates sequences (e.g., aa, aba) and probes the blackbox
• Builds a hypothesis FSM with input-output pairs seen so far
• Queries an Equivalence Oracle (EO) for counterexamples
Blackbox
L* Equivalence Oracle
21
![Page 36: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/36.jpg)
Practical Challenges of Applying L* for an NF
• Generate input alphabet
• Classify output of an NF
• Build an Equivalence Oracle
22
![Page 37: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/37.jpg)
Generating Input Alphabet to handle Large Traffic Space
StatefulNF
Rule1: SRC IP:A…DST IP:B Naive solutions:
1. Exhaustively generating packets
2. Randomly generating packets
Infeasible
Does not explore the relevant state spaceLAN WAN
23
![Page 38: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/38.jpg)
Generating Input Alphabet to handle Large Traffic Space
To exercise the rule, we generate packets with IP/ports in the rule
StatefulNF
Rule1: SRC IP:A…DST IP:B Naive solutions:
1. Exhaustively generating packets
2. Randomly generating packets
Infeasible
Does not explore the relevant state space
1) Find IP/port fields that appear in the rule Generate the packet for for all interfaces using A and B A→B
B→A
A→B
B→A
LAN WAN
23
![Page 39: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/39.jpg)
Generating Input Alphabet to handle Large Traffic Space
To exercise the rule, we generate packets with IP/ports in the rule
StatefulNF
Rule1: SRC IP:A…DST IP:B Naive solutions:
1. Exhaustively generating packets
2. Randomly generating packets
Infeasible
Does not explore the relevant state space
1) Find IP/port fields that appear in the rule Generate the packet for for all interfaces using A and B A→B
B→A
LAN WAN 2) (Optional) Prune based on reachability
23
![Page 40: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/40.jpg)
Generating Input Alphabet to handle Large Traffic Space
To exercise the rule, we generate packets with IP/ports in the rule
StatefulNF
Rule1: SRC IP:A…DST IP:B Naive solutions:
1. Exhaustively generating packets
2. Randomly generating packets
Infeasible
Does not explore the relevant state space
1) Find IP/port fields that appear in the rule Generate the packet for for all interfaces using A and B
LAN WAN 2) (Optional) Prune based on reachability
3) Plug in “packet types”
SYN, A→B
SYNACK, A→B
SYN, B→A
SA, B→A
… …
23
![Page 41: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/41.jpg)
• Generate input alphabet
• Classify output of an NF
• Configure the “timeout” to classify output
• Translating to/from symbolic and concrete packets
• Build an Equivalence Oracle
Practical Challenges of Applying L* for an NF
24
![Page 42: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/42.jpg)
Challenges on Inferring NF Behavior
• Inferring the symbolic model (FSM)
• Inferring the state granularity
• Handling dynamic header modification
25
![Page 43: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/43.jpg)
Different Types of State Granularity
Cross-connection One FSM for all connections
Per-source One FSM for each srcip
Per-destination One FSM for each dstip
Per-connection One FSM for every IP-port pair
State Granularity: the state variables (IP/ports) that the NF uses to keep state
This is like a “key” mapping to the FSM 26
![Page 44: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/44.jpg)
Learning the State Granularity
A
A’
B
B’
DST IPSRC IPconn1
conn2 Cross-connection
Do these affect the “same” FSM?
No
AB
B’
conn1
conn2Per-source
Do these affect the “same” FSM?
No . . .
27
![Page 45: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/45.jpg)
Learning the State Granularity
A
A’
B
B’
DST IPSRC IPconn1
conn2 Cross-connection
Do these affect the “same” FSM?
No
AB
B’
conn1
conn2Per-source
Do these affect the “same” FSM?
No . . .Construct test cases for independence across connections
27
![Page 46: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/46.jpg)
Alembic Workflow: Offline
Library of symbolic models
RuleType i :(Keyi, SymFSMi)
KeyLearning
VendorDoc
NF
PacketTypes
RuleTypeGen
RuleTypei
FSMInference (Extended L*)
Runs once per NF
Distributed Learning
28
![Page 47: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/47.jpg)
Alembic Workflow: OnlineRuns for every config
Concrete config
Instantiate(Rule1)
. . .
Instantiate(Rule2)
Instantiate(RuleN)
Rule1 Rule2 . . .
RuleN
RuleType i :(Keyi, SymFSMi)
If packet p match Rule1:
Ensemble(Rule1)
Elif packet p match Rule2:
Ensemble(Rule2)
. . .
29
![Page 48: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/48.jpg)
Evaluation Summary
• Alembic-generated models are accurate
• Case Studies: Alembic finds differences across NF implementations
• Alembic workflow is scalable
• Alembic-generated models improve the accuracy of network testing/verification tools
30
![Page 49: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/49.jpg)
Evaluation Setup• Validated Alembic using Click-based NFs where we know the ground truth• Real NFs we modeled :
• PfSense (FW, static NAT, random NAT, LB) • Proprietary NF (FW, static NAT)• Untangle (FW) • HAproxy (LB)
• Packet types used: • Correct-Seq: {SYNC, SYN-ACKC, ACKC, FIN-ACKC, RST-ACKC} • Combined-Seq: extend the correct-seq set with incorrect seq and ack,
{SYN-ACKI, ACKI, FIN-ACKI, RST-ACKI}
31
![Page 50: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/50.jpg)
Accuracy Evaluation
1) Iperf testing: 100% across all settings for all NFs 2) Random Packet testing (randomly choosing IP/port): 99.8% to 100% across all settings for all NFs
3) Rule Activation testing (choosing IP/port to activate one rule): 94.8% to 100% across all settings for all NFs
• Config generation: 1 to 100 rules in a configuration• Packet generation: 20 to 300 packets in a sequence
Since we do not have the ground-truth, we designed complementary testing methodology to test the accuracy of our models
32
![Page 51: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/51.jpg)
Evaluation Summary
• Alembic-generated models are accurate
• Case Studies: Alembic finds differences across NF implementations
• Alembic workflow is scalable
• Alembic-generated models improve the accuracy of network testing/verification tools
33
![Page 52: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/52.jpg)
Firewall Case StudyPfSense ProprietaryNF
Packet sequence before the FW allows TCP traffic from an external host (B ) to an internal host (A)
Number of states 3 79
Default behavior Default Drop Default Drop
SYN, A→B
SYN, A→B
SA, B→A
34
![Page 53: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/53.jpg)
Firewall Case StudyPfSense ProprietaryNF
Packet sequence before the FW allows TCP traffic from an external host (B ) to an internal host (A)
Number of states 3 79
Default behavior Default Drop Default Drop
SYN, A→B
SYN, A→B
SA, B→A
34
![Page 54: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/54.jpg)
Firewall Case StudyPfSense ProprietaryNF
Packet sequence before the FW allows TCP traffic from an external host (B ) to an internal host (A)
Number of states 3 79
Default behavior Default Drop Default Drop
SYN, A→B
SYN, A→B
SA, B→A
34
![Page 55: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/55.jpg)
• Implements “default allow”• Connection-terminating
Firewall Case Study: Untangle Firewall
0
*
SYN
/* /
SYN /1
SYN
ACK
//
2SA/SA ACK
SYN / SA
SA / ACK
ACK /
35
![Page 56: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/56.jpg)
• Implements “default allow”• Connection-terminating
Firewall Case Study: Untangle Firewall
0
*
SYN
/* /
SYN /1
SYN
ACK
//
2SA/SA ACK
SYN / SA
SA / ACK
ACK /
When B responds with SA, the FW preemptively responds with ACK
35
![Page 57: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/57.jpg)
• Implements “default allow”• Connection-terminating
Firewall Case Study: Untangle Firewall
0
*
SYN
/* /
SYN /1
SYN
ACK
//
2SA/SA ACK
SYN / SA
SA / ACK
ACK /
When A replies with ACK, theFW drops to prevent duplicates
35
![Page 58: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/58.jpg)
• Implements “default allow”• Connection-terminating
Firewall Case Study: Untangle Firewall
0
*
SYN
/* /
SYN /1
SYN
ACK
//
2SA/SA ACK
SYN / SA
SA / ACK
ACK /
Takeaways: 1) Vendor diversity (no common practice) 2) The real FSMs are complex and are infeasible for humans to manually generate
35
![Page 59: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/59.jpg)
Other Findings
• FW: models with incorrect seq ! large FSM (257 states for PfSense)
• FW: many do not correctly handle out-of-window packets
• LB: HAproxy (connection-terminating) vs. PfSense (destination NAT)
. . .
36
![Page 60: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/60.jpg)
Evaluation Summary
• Alembic-generated models are accurate
• Case Studies: Alembic finds differences across NF implementations
• Alembic workflow is scalable
• Alembic-generated models improve the accuracy of network testing/verification tools
37
![Page 61: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/61.jpg)
Scalability of Alembic Online
Number of Rules Runtime
10 0.075 s
100 0.6 s
1,000 5 s
Alembic can generate concrete models in a few seconds for a large config
38
![Page 62: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/62.jpg)
Limitations and Future WorkAssumption on configurations: • Assume at most one rule is applied• States across different state granularities (i.e., keys) are independent• Assume that IP/port fields are treated homogeneously such that we can pick one
representative sample and infer a model
Assumption on NF actions: • Focused on modeling TCP-relevant behavior where actions are restricted to
dropping and forwarding, possibly with IP/port modifications• Do not explicitly model temporal effects• Support the following state granularity types: per-connection, per-source, per-
destination, cross-connection, and stateless
Future work: • Dealing with more complex NFs (e.g., rate-limiting NF, modeling temporal effects)
39
![Page 63: ALEMBIC: AUTOMATED MODEL INFERENCE FOR ......ALEMBIC: AUTOMATED MODEL INFERENCE FOR STATEFUL NETWORK FUNCTIONS. Soo-Jin Moon. Jeffrey Helt, Yifei Yuan, Yves Bieri, Sujata Banerjee,](https://reader036.fdocuments.in/reader036/viewer/2022071512/61332276dfd10f4dd73ae471/html5/thumbnails/63.jpg)
Conclusions: Alembic can accurately model stateful NFs
• Network testing and verification today need NF models
• Handwritten models: tedious, error-prone, and inaccurate
• Alembic: infers a high-fidelity NF model given a configuration
• Our evaluations show:
• Alembic finds implementation-specific behavior of NFs
• Alembic-generated models increase the accuracy of testing/verification
• Alembic is scalable and accurate
Soo-Jin Moon: [email protected]