Post on 10-Jan-2016
description
Language-Based Generation and Evaluation of NIDS Signatures
Shai Rubin
Somesh Jha Barton P. Miller
University of Wisconsin, Madison
Rubin, Jha, Miller 2
Attacker
“TYPE A \n CWD <long arg>\n”
NetworkNIDS
Signature database
Misuse Network Intrusion Detection System (NIDS)
Problem: A single attack might have many forms: – Ptacek and Newsham, 1988– Handley and Paxson, 2001– Marty, 2002– Mutz, Vigna, and Kemmerer, 2003– Vigna, Robertson, and Balzarotti, 2004– Rubin, Jha, Miller, 2004 – And others...
“TYPE A \n (.)* CWD <long arg>”
TYPE A \n LIST \n CWD ...
Rubin, Jha, Miller 3
Attacker NetworkNIDS
Signature database
Problem: Accurate Signatures
• Today, we construct signatures in an ad-hoc manner• Challenges: complex protocols, redundancy• Questions:
– Can we systematically construct an accurate signature?– Can we systematically evaluate a signature?– Can we systematically compare signatures?
“TYPE A \n (.)* CWD <long arg>”
TYPE A \n LIST \n CWD ...
Rubin, Jha, Miller 4
Contributions
• Practical: provide signature writers a methodology and a tool that enables them to systematically construct a signature, evaluate its accuracy, and compare it to other signatures
• Conceptual: – a session signature, – a semantic model for an attack protocol, – a language-base approach for signature construction
Rubin, Jha, Miller 5
A NIDS Signature
• Attack: a set of TCP streams• Signature: a set of TCP streams
TCP Streams
ASig
Rubin, Jha, Miller 6
A NIDS Signature
• Attack: a set of TCP streams• Signature: a set of TCP streams• A prefect signature: Sig=A
TCP Streams
ASigSig=A
Rubin, Jha, Miller 7
A NIDS Signature
• Attack: a set of TCP streams• Signature: a set of TCP streams• A prefect signature: Sig=A • Problem: most of the time A is
unknown. Difficult to: – construct accurate a signature– evaluate changes to the
signature– compare signatures
TCP Streams
ASig
Rubin, Jha, Miller 8
A NIDS Signature
TCP Streams
ASig
• Attack: a set of TCP streams• Signature: a set of TCP streams• A prefect signature: Sig=A • Problem: most of the time A is
unknown. Difficult to: – construct accurate a signature– evaluate changes to the
signature– compare signatures
Rubin, Jha, Miller 9
Language-Based Approach
TCP Streams• Attack: the language Aghost
• Signature: the language Lsig
• Goal: compare the language• Problem: difficult to determine
containment Aghost.
• Ideas: 1. Abstraction: over-approximate
Aghost, such that it is easy to determine containment
2. Automation: Use an automatic tool to compare Lsig and Ainv
Lsig
Aghost
Ainv
Rubin, Jha, Miller 10
Language-Based Signature Construction
TCP Streams
Lsig
Aghost
Ainv
Conclusion Action
fp
fn
Rubin, Jha, Miller 11
Language-Based Signature Construction
TCP Streams
Lsig
Aghost
Ainv
Conclusion Action
LsigAinvA false positive
Shrink signature
fp
fn
Rubin, Jha, Miller 12
Language-Based Signature Construction
TCP Streams
Lsig
Aghost
Ainv
fp
Conclusion Action
LsigAinvA false positive
Shrink signature
LsigAinv
Ainv fn
Rubin, Jha, Miller 13
Language-Based Signature Construction
TCP Streams
Lsig
Aghost
Ainv
fp
Conclusion Action
LsigAinvA false positive
Shrink signature
LsigAinv
A false negative
Expand signature
Ainv fn
Rubin, Jha, Miller 14
Language-Based Signature Construction
TCP Streams
Lsig
Aghost
Ainv
fp
Conclusion Action
LsigAinvA false positive
Shrink signature
LsigAinv
A false negative
Expand signature
A spurious sequence
Refine Ainv
Ainv fn
sp
Rubin, Jha, Miller 15
Language-Based Signature Construction
TCP Streams
Lsig
Aghost
Ainv
fp
Conclusion Action
LsigAinvA false positive
Shrink signature
LsigAinv
A false negative
Expand signature
A spurious sequence
Refine Ainv
LsigAinv
Discussion in the paperLsigAinv
Ainv fn
sp
Rubin, Jha, Miller 16
Outline
• Goal: develop methodology to construct and evaluate signatures
• Main idea: use a formal language to approximate Aghost and automatically compare this language to Lsig
• The languages• The signature construction process
Rubin, Jha, Miller 17
Lsig: A Syntactic Representation of the Attack
• Our signature is a regular language• Alphabet: application-level events. For example,
FTP commands• A session signature: a string in the language
represents the entire attack.• Each signature is a concatenation of three
languages: preparation (Lpre), exploitation (Lexp), and confirmation (Lconf)
Rubin, Jha, Miller 18
ftp-cwd [CAN-2002-0126]
• Preparation: FTP login
loginL
logoutQ
QL
Token Description
L Login confirmation
Q Connection termination
Rubin, Jha, Miller 19
ftp-cwd [CAN-2002-0126]
• Preparation: FTP login
• Exploitation: A CWD command with a long argument
loginL
logout
Q
QL
attack
A such that (length>100 && data (.)*/bin/sh(.)*
Clogin
Token Description
L Login confirmation
Q Connection termination
C CWD command
A CWD argument
Rubin, Jha, Miller 20
Lftp-cwd: ftp-cwd Session Signature
• Non-recursive hierarchical state machine• Constructed automatically• Can be analyzed
intrusion
logout 2
1 attack
A,IR,L
IRA,L
C
IR ,LC
A,C,IR,Q
Q
QCQL
A
accept
start
reject
Rubin, Jha, Miller 21
Lftp-cwd: Vs. Snort
• Non-recursive hierarchical state machine• Constructed automatically• Can be analyzed
intrusion
logout 2
1 attack
A,IR,L
IRA,L
C
IR ,LC
A,C,IR,Q
Q
QCQL
A
accept
start
reject
Rubin, Jha, Miller 22
Language-Based Signature Construction
TCP Streams
Session Signature
Aghost
Ainv
fp
Conclusion Action
LsigAinvA false positive
Shrink signature
LsigAinv
A false negative
Expand signature
A spurious sequence
Refine Ainv
LsigAinv
Discussion in the paperLsigAinv
Ainv fn
sp
Rubin, Jha, Miller 23
Ainv: Semantic Representation of the Attack
• Another regular language• Models semantics properties:
– “Requires FTP login”– “Requires ASCII FTP mode”– “Requires HTTP 1.1”
• Using an FSM we model the semantics of the application-level protocol that the attack uses
Rubin, Jha, Miller 24
FTP Semantic Model
Variable Description Values
X1 User logged in {0,1}
X2 FTP transfer mode {‘A’,’B’,0}
Name Token Description Precond. Postcond.
SLOGIN L Victim indicates successful login - X1=1,X2=‘A’
BINARY B Attacker issues TYPE B command X1=1 X2=‘B’
ASCII A Attacker issues TYPE A command X1=1 X2=‘A’
VQUIT Q1 Victim terminates connection - Xi=0
UQUIT Q2 Attacker terminates connection - Xi=0
FTP State variables
FTP Transitions
Rubin, Jha, Miller 25
Language-Based Signature Construction
TCP Streams
Session Signature
Aghost
Semantic model
fp
fn
Semantic Model
Signature
Spin
String/NULL
SPFN or FP
Manual refinement (currently) Automatic comparison
Rubin, Jha, Miller 26
TCP Streams
Constructing a Signature for ftp-cwd
login=1
Lpre Lexp False Positive
L1 (.)* CWD <long arg>
Semantic ModelSignature
Spin
String/NULL
Rubin, Jha, Miller 27
TCP Streams
Constructing a Signature for ftp-cwd
login=1
FP1
L1
Lpre Lexp False Positive
L1 (.)* CWD <long arg> FP1=“CWD <long arg>”
Semantic ModelSignature
Spin
String FP1
Rubin, Jha, Miller 28
TCP Streams
Constructing a Signature for ftp-cwd
login=1
FP1
L1
Lpre Lexp False Positive
L1 (.)* CWD <long arg> FP1=“CWD <long arg>”
L2 L(.)* CWD <long arg>
Semantic ModelSignature
Spin
String/NULL
Rubin, Jha, Miller 29
TCP Streams
Constructing a Signature for ftp-cwd
L1
login=1
FP1 FP2
L2
Lpre Lexp False Positive
L1 (.)* CWD <long arg> FP1=“CWD <long arg>”
L2 L(.)* CWD <long arg> FP2=“LUQUITCWD <long arg>”
Semantic ModelSignature
Spin
StringFP2
Rubin, Jha, Miller 30
TCP Streams
Constructing a Signature for ftp-cwd
login=1
FP1
L1
FP2
L2
Lpre Lexp False Positive
L1 (.)* CWD <long arg> FP1=“CWD <long arg>”
L2 L(.)* CWD <long arg> FP2=“LUQUITCWD <long arg>”
L3 L(UQ)* CWD <long arg>
Semantic ModelSignature
Spin
String/NULL
Rubin, Jha, Miller 31
TCP Streams
Constructing a Signature for ftp-cwd
login=1
Lpre Lexp False Positive
L1 (.)* CWD <long arg> FP1=“CWD <long arg>”
L2 L(.)* CWD <long arg> FP2=“LUQUITCWD <long arg>”
L3 L(UQ)* CWD <long arg> FP3=“LVQUITCWD <long arg>”
FP1
L1
FP2
L2
FP3
L3
Semantic ModelSignature
Spin
String FP3
Rubin, Jha, Miller 32
TCP Streams
Constructing a Signature for ftp-cwd
login=1
Lpre Lexp False Positive
L1 (.)* CWD <long arg> FP1=“CWD <long arg>”
L2 L(.)* CWD <long arg> FP2=“LUQUITCWD <long arg>”
L3 L(UQ)* CWD <long arg> FP3=“LVQUITCWD <long arg>”
FP1
L1
FP2
L2
FP3
L3
Semantic ModelSignature
Spin
NULL
Rubin, Jha, Miller 33
Constructing a Signature for ftp-cwd
TCP Streams
login=1
FP1
L1
FP2
L2
FP3
L3
L1L2L3L4
L4
More false positives Less false positives
• Comparing signature:
• It is possible to show that L4 does not miss more attacks than L1 (under certain assumptions)
Rubin, Jha, Miller 34
Constructing a Signature for pro-ftpd
Session Signature (simplified) False Negative/SpuriousL TYPEA ST RET RET
TCP Streams
login=1TYPE=‘A’
Rubin, Jha, Miller 35
Constructing a Signature for pro-ftpd
Session Signature (simplified) False NegativeL TYPEA ST RET RET FN1=L ST RET RET
TCP Streams
login=1TYPE=‘A’
FN1
Two signatures based on the configuration of the FTP server
Rubin, Jha, Miller 36
Lessons to Take Home
• A methodology to construct and evaluate signatures
• Able to detect loopholes in signatures, loopholes that we did not anticipate
• The accuracy of the signature depends of the accuracy of the semantic model
TCP Streams
Session Signature
Aghost
Ainv
fp
Ainv fn
sp