Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who...

29
Analysis of Kullback-Leibler Divergence for Masquerade Detection Geetha Ranjini Viswanathan * Richard M. Low Mark Stamp Abstract A masquerader is an attacker who gains access to a legitimate user’s credentials and pretends to be that user in an effort to avoid detection. Several statistical tech- niques have been applied to the masquerade detection problem, including hidden Markov models (HMM) and one class na¨ ıve Bayes (OCNB). In addition, Kullback- Leibler (KL) divergence has been used in an effort to improve detection rates. In this project, we develop and analyze masquerade detection techniques that employ KL divergence, HMMs, and OCNB. Detailed statistical analysis is provided to show that our results outperform previous related research. 1 Introduction An intruder is an attacker who gains unauthorized access to a system. A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to be a legitimate user [24]. A masquerade detection system is an intrusion detection system that is specifically designed to detect such an intruder. The research presented here represents an anomaly-based approach to masquerade detection. Specifically, we analyze UNIX commands for anomalous behavior. There is a vast amount of prior research on this particular problem; representative examples include [6, 8, 9, 10, 12, 17, 19, 24, 26, 27]. The survey paper [2] lists more than 40 relevant publications that appeared prior to 2009. In this research, we employ hidden Markov models (HMM), which can be viewed as a machine learning technique [27]. We train an HMM on each legitimate user’s UNIX commands. These models are then used to determine the likelihood that a given set of commands is from the specified user or not. Prior research has shown that HMMs can be an effective technique for masquerade detection [10]. We also use one class na¨ ıve Bayes (OCNB). In OCNB, elementary statistical analysis is applied to a user’s input data as a means to detect masqueraders. As * Department of Computer Science, San Jose State University Department of Mathematics, San Jose State University Department of Computer Science, San Jose State University: [email protected] 1

Transcript of Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who...

Page 1: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Analysis of Kullback-Leibler Divergence forMasquerade Detection

Geetha Ranjini Viswanathan∗ Richard M. Low† Mark Stamp‡

Abstract

A masquerader is an attacker who gains access to a legitimate user’s credentialsand pretends to be that user in an effort to avoid detection. Several statistical tech-niques have been applied to the masquerade detection problem, including hiddenMarkov models (HMM) and one class naıve Bayes (OCNB). In addition, Kullback-Leibler (KL) divergence has been used in an effort to improve detection rates. In thisproject, we develop and analyze masquerade detection techniques that employ KLdivergence, HMMs, and OCNB. Detailed statistical analysis is provided to show thatour results outperform previous related research.

1 Introduction

An intruder is an attacker who gains unauthorized access to a system. A masqueraderis an intruder who carries out some malicious activity and attempts to avoid detectionby pretending to be a legitimate user [24]. A masquerade detection system is anintrusion detection system that is specifically designed to detect such an intruder.

The research presented here represents an anomaly-based approach to masqueradedetection. Specifically, we analyze UNIX commands for anomalous behavior. Thereis a vast amount of prior research on this particular problem; representative examplesinclude [6, 8, 9, 10, 12, 17, 19, 24, 26, 27]. The survey paper [2] lists more than 40relevant publications that appeared prior to 2009.

In this research, we employ hidden Markov models (HMM), which can be viewedas a machine learning technique [27]. We train an HMM on each legitimate user’sUNIX commands. These models are then used to determine the likelihood that agiven set of commands is from the specified user or not. Prior research has shownthat HMMs can be an effective technique for masquerade detection [10].

We also use one class naıve Bayes (OCNB). In OCNB, elementary statisticalanalysis is applied to a user’s input data as a means to detect masqueraders. As

∗Department of Computer Science, San Jose State University†Department of Mathematics, San Jose State University‡Department of Computer Science, San Jose State University: [email protected]

1

Page 2: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

with HMMs, OCNB has been successfully applied to the masquerade detection prob-lem [10, 24].

Kullback-Leibler (KL) divergence is a statistical method that can be used toseparate data from different distributions [11]. This technique has previously beenstudied in the context of masquerade detection as a means for separating masqueradedata (i.e., data intended to mimic the legitimate user) from attack data [24]. Ifproperly separated, we can obtain stronger scores.

We have implemented and tested HMM-based and OCNB-based masquerade de-tection techniques. The contribution of this paper is a rigorous analysis of the effec-tiveness of KL divergence on detection rates for these two approaches. We show thatthe results from our research offer a significant improvement over the results obtainedin previous research.

Section 2 introduces intrusion detection and the performance measures used inthis project. Section 3 discusses the implementation details for HMM, OCNB, KLdivergence as used in our research. Section 4 contains various test results. Finally,Section 5 concludes the paper and provides some possible directions for future work.

2 Background

In this section, we provide brief introductions to several relevant background topics.Specifically, we discuss IDS, the dataset used in this research, and our performancemeasure.

2.1 Intrusion detection

An intrusion is said to occur when an attacker gains access to a system. The goal of anintrusion detection system (IDS) is to identify whether current activity is legitimateor part of attack. IDS research has employed hidden Markov models [6, 8, 10, 27],one class naıve Bayes [10, 19, 24, 26], and support vector machines [9], among many,many other techniques. In broad terms, we can classify an IDS as signature-based oranomaly-based [23].

An attack signature is a fixed pattern that represents a known attack [23]. Signature-based intrusion detection is relatively simple, efficient, and effective against attacksthat have been previously observed and analyzed. The weakness of such an approachis that the attack must be known in advance and a useful signature must have beenextracted. Consequently, previously unknown attacks are unlikely to be detected.

A anomaly-based IDS depends on the behavior and the assumption that there willbe behavioral difference between attacks and legitimate use [14]. An anomaly-basedIDS operates in two phases: training and detection [7]. In masquerade detection, thegoal of the training phase is to determine a model that fits a given user’s behavior.Then during the detection phase, we use the model to detect significant deviationsfrom the user’s expected behavior.

2

Page 3: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Unlike signature-based systems, anomaly-based systems have the potential to de-tect previously unknown attacks. However, the false positive rate of an anomaly-based system is likely to be much higher, and the training phase may be complex andcostly [7].

2.2 Schonlau dataset

The Schonlau dataset [18] consists of truncated user-issued UNIX commands collectedfrom 50 users. The data file contain 15,000 UNIX commands for each user, composedof 150 blocks of 100-commands each. For each user, the first 50 blocks (i.e., 5000commands) are the training data, while the remaining 100 blocks (i.e., 10,000 com-mands) are the test set. The training set consists entirely of commands generated bythe specified user, while some blocks in the test set are user commands, while othersare “attack” commands, where the attacks are simply blocks selected from other userprofiles. Figure 1 shows a diagrammatic representation of the Schonlau dataset.

The dataset also includes a map file, in the form of a 0-1 matrix of size 100× 50,which indicates which of the test blocks are attacks and which are not. In this matrix,a 0 represents that the corresponding user test block is attack-free while a 1 representsthat the block contains attack data. Figure 2 depicts this mapping file. The attack-free blocks (i.e., the blocks that belong to the specified user) are referred to as “self”blocks and the attack blocks are “non-self” blocks [24].

2.3 Performance measures

A detection result can be classified in one of four ways. If the data is an attack andit is identified as such by the IDS, this is a true positive. If the data is not an attackbut it is identified as an attack, then a false positive has occurred. If an attack isnot identified as such, we have a false negative, and, finally, if legitimate (attack-free) data is identified as legitimate, it is a true negative. These four outcomes areillustrated in the form of a confusion matrix in Figure 3.

Performance measures such as the true positive rate (TPR), false positive rate(FPR), and accuracy can be used to determine the effectiveness of a detection system.These measures are computed as

TPR =TP

TP + FN

FPR =FP

FP + TN

Accuracy =TP + TN

TP + FN + FP + TN(1)

Receiver operating characteristic (ROC) curves provide us with a visual repre-sentation for comparing the effectiveness of detection strategies. For the researchpresented here, ROC curves are obtained by plotting the FPR on the x-axis versus

3

Page 4: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

the TPR on the y-axis, where the threshold is varied over the range of scores [4].Once the ROC plots are obtained, we can then compute the area under the ROCcurve (AUC). Ideal detection will result in an AUC of 1, while classification basedon coin flipping will yield an AUC of approximately 0.5. The AUC is useful, since itprovides a single, meaningful number that we can used to compare the performanceof different detection strategies [3].

3 Detection techniques

In this section, we discuss the various technique that are employed in the masqueradedetection research considered in this paper. Specifically, we discuss hidden Markovmodels (HMM), one class naıve Bayes, Kullback-Leibler divergence, and our scoringmethods that employ these techniques.

3.1 Hidden Markov model

A hidden Markov model (HMM) is a machine-learning technique that can be used tobuild a model based on a given sequence of input data. HMMs have been successfullyapplied to a wide variety of problems, including malware detection [16, 21]. In addi-tion, HMMs have been extensively studied in the context of anomaly-based intrusiondetection. Previous research has shown that when applying HMMs to the Schonlaudataset, attacks of length 30 or more are effectively detected [10].

An HMM is denoted is denoted as λ = (A,B, π), where

1. A is the state transition matrix of the underlying Markov process,

2. B is the observation probability distribution matrix, and

3. π is the initial state distribution matrix.

All three of these matrices are row-stochastic. The following notation is commonlyused for HMMs [22]:

T = length of the observation sequence

N = number of states in the Markov model

M = number of unique observation symbols

Q = {q0, q1, . . . , qN−1} = the unique states of the Markov process

V = {0, 1, . . . ,M − 1} = the set of possible observations

O = (O0,O1, . . . ,OT−1) = observation sequence.

Figure 4 illustrates the concept behind an HMM. The hidden states of the under-lying Markov process are represented by X0, X1, X2, . . . , XT−1. The A matrix drivesthe Markov process, while the B matrix probabilistically relates the hidden states tothe observations. The region above the dashed line represents the “hidden” part ofthe HMM.

4

Page 5: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Here, we train 50 HMMs, one for each user in the Schonlau dataset. Using thesetrained models, we compute scores over entire test data. After obtaining these base-line results, we experiment with Kullback-Leibler divergence to determine how bestto improve on the baseline results.

For more details on HMMs, see, for example [22].

3.2 One class naıve Bayes

One class naıve Bayes (OCNB) is another learning algorithm that has been suc-cessfully applied to the masquerade detection problem. OCNB is a straightforwardclassifier derived from Bayes rule. Using OCNB, we can compute the probability thatan instance x belongs to a class y. By using

P (y|x) =P (y)

P (x)P (y|x) =

P (y)

P (x)

m∏i=1

P (xi|y)

where x = (x1, x2, . . . , xm), we can determine the class y that maximizes the proba-bility of a given observation x [24].

In this paper, OCNB is used to compare the probability of a command occurringin the test data to the probability of the same command occurring in a given user’straining data. We can use these results to classify whether a series of commandscorresponds to an attack or not. Previous research involving the Schonlau datasethas shown this technique to be effective at detecting attack sequences of length 50 ormore [10].

For the Schonlau dataset, every block B of 100 commands can be represented bya vector [n1(B), n2(B), . . . , nm(B)], where ni(B) is the number of times that com-mand ci appears in the block B and m is the number of distinct commands in theblock B [24]. For OCNB, the probability that the block B belongs to user y iscomputed as

P (y|B) = P (y)

m∏i=1

P (ci|y)ni(B) (2)

where, since the priors are unknown, we let P (y) = 1. The values P (ci|y) in (2) arederived from the training set for user y as

P (ci|y) =

∑B∈T (y) ni(B) + α

|B| · |T (y)|+ α ·m(3)

where T (y) is the training set for user y (i.e., the self data). The parameter α isused to ensure that all commands have a non-zero probability. Following previouswork [12, 26], here we have selected α = 0.01.

Note that in (2), the value ni(B) corresponds the block B that is to be testedagainst the training set. In contrast, in (3), the value of ni(B) is derived from thetraining set.

5

Page 6: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

We can use (2) to compute a score as [24]

score(B) = − logP (y|B) = −m∑i=1

ni(B) logP (ci|y) (4)

The higher the score, the more anomalous the test block and hence the more likelyit will be classified as an attack. On the other hand, a score closer to 0 is indicativeof a self block.

3.3 Kullback-Leibler divergence

For two discrete probability distributions P and Q, the Kullback-Leibler (KL) diver-gence of Q from P is given by [24]

DKL(P ||Q) =∑i

P (i) logP (i)

Q(i)

= −∑i

P (i) logQ(i) +∑i

P (i) logP (i)

KL divergence provides a means for distinguishing observations from differentdistributions [11]. KL divergence has been used to identifying anomalies in wirelesssignals [1], biomedical data [15], and network traffic [5, 13], among many other appli-cations. In [24], KL divergence is applied in an attempt to improve on OCNB-basedmasquerade detection. Next, we consider the technique in [24] in more detail.

3.3.1 Probabilistic padding identification

In [24], a probabilistic padding identification (PPI) algorithm is used to distinguishpadding commands from attack commands. This algorithm makes use of the KLdivergence.

Let A and P represent the attack and padding portions , respectively, for a givenblock B. Let M represent the trained model corresponding to the padding P . Then,in general, P should match M more closely than A matches M .

In a PPI algorithm, we attempt to find subsets P , A ⊆ B, with P ∪ A = B andP ∩ A = ∅, such that DKL(P ||M) is “low” and DKL(A||M) is “high”. That is, wewant to partition B into P and A so that P matches M , while A does not.

For any putative choice of A and P , we compute a score for the partition as

|Dp −Da| =

∣∣∣∣∣∑i

|P | log|P ||M |−∑i

|A| log|A||M |

∣∣∣∣∣where the notation |X| represents the number of elements in the set X.

We have implemented and analyzed two versions of the PPI algorithm, namely,an attack-first and a padding-first version. These names refer to the initialization

6

Page 7: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

process—in the attack-first case, we initialize A = B and P = ∅, while in the padding-first case, we initialize A = ∅ and P = B. The attack-first strategy is analyzed in [24],while the padding-first approach appears to be new. We have implemented both andwe compare the results in Section 4.

As given in [24], for the attack-first PPI we determine A an P using Algorithm 1which, in turn, uses Algorithm 2. For the padding-first PPI we use the obviousmodification; see Algorithm 3.

Algorithm 1 Attack-first PPI

Input: Block B, Model MOutput: Boolean vector C(i) = true if B(i) is padding

1: Initially C(i)← false for all i2: for i = 1 to |B| do3: d = DiffKL(C,B,M)4: C(i)← true5: d = DiffKL(C,B,M)6: if (d ≤ d) then7: C(i)← false8: end if9: end for

10: return P = commands B(i) such that C(i) is true

Algorithm 2 DiffKL

Input: Boolean vector C, block B, model MOutput: Difference of KL divergences

1: A← PDF of those B(i) such that C(i) is false2: P ← PDF of those B(i) such that C(i) is true3: Da ← DKL(A||M)4: Dp ← DKL(P ||M)5: return |Dp −Da|

Finally, given A and P , we compute a score as

scorePPI(B) = −score(P )− β · score(A) (5)

where score(·) is defined in (4) and β ≥ 1 is a weight. Note that in [24], differentvalues of β are selected for different users so as to improve detection rates. Here, forsimplicity we use a fixed value of β = 2 for all users.

7

Page 8: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Algorithm 3 Padding-first PPI

Input: Block B, Model MOutput: Boolean vector C(i) = true if B(i) is padding

1: Initially C(i)← false for all i2: for all B(i) in M , C(i)← true3: d = DiffKL(C,B,M)4: for i = 1 to |B| do5: C(i)← false6: d = DiffKL(C,B,M)7: if (d < d) then8: C(i)← true9: end if

10: end for11: return P = commands B(i) such that C(i) is true

3.4 Masquerade attacks

As in [24], we make the following assumptions.

1. Perfect knowledge: The masquerader has full knowledge of the detection algo-rithm, including all relevant parameters and the trained model.

2. Non-poisoned detector: The masquerade detection system is trained on attack-free data.

3. Attack padding: The attack sequence must be contained within a single 100-command block. Also, each attack command can be placed at any locationwithin the block.

Figure 5 illustrates an example of a masquerade, or mimicry, attack. The boxedcommands represent the attack and the unboxed commands are the padding.

To test the performance of the HMM and OCNB-based systems analyzed here,masquerade attacks were generated for each user. We have used commands fromother users as attack data. For each user, 50 attack blocks were generated for each ofthe attack lengths 10, 20, 30, . . . , 100, with the attack commands randomly distributedthroughout the block. Therefore, the performance of each technique was tested on atotal of 500 attack blocks.

4 Experimental results

In this section, we present detailed test results for the performance of our HMMand OCNB based masquerade detection systems (MDS). Both the HMM and OCNB

8

Page 9: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

approach are evaluated for each of the following three cases:

• No probabilistic padding identification (PPI)

• Padding-first PPI

• Attack-first PPI

For all experiments, our performance measure is the area under the curve (AUC) ofROC curves, as discussed in Section 2.3.

We generate attack blocks as described in Section 3.4. That is, for each user, 50attack blocks were generated for each attack length |A| = 10, 20, 30, . . . , 100. Recallthat all blocks B are of length 100, and hence for each experiment, the padding Psatisfies |P | = 100− |A|.

To test the performance in the “no PPI” case, the attack blocks were directlyscored against the appropriate trained model (HMM and OCNB). These results pro-vide a baseline against which we can compare the performance of the PPI-basedapproaches. Recall that the PPI techniques employ Kullback-Leibler divergence.

To analyze the performance of padding-first and attack-first PPI, the attack blockswere first put through the appropriate PPI, which yields putative attack and paddingsubsequences for each block. These sequences were then scored using the score in (5).But, before we turn our attention to the MDS results, we first consider the successof the PPI approaches at separating the padding and attack commands from a givenmasquerade attack block.

4.1 PPI comparison

The graph in Figure 6 provides a direct comparison of the accuracy of padding-firstPPI and attack-first PPI. These results are based on the identification of attack dataand padding data in blocks, with accuracy determined using the formula in (1). Inthe Schonlau dataset, it is known that some users’ behavior is easier to model thanothers—the “hard,” “easy,” and “intermediate” cases in Figure 6 represent users thathave proven hard, easy, and average to model. Specifically, users 47, 49, and 1 wereused for the hard, easy, and intermediate cases, respectively, with the attacks selectedfrom user 2 data in each case. Also, 10 attacks were tested for each case (easy, hard,intermediate) for each attack of length 10, 20, 30, 40, 50, 60, 70, 80, 90, and 100.

From Figure 6 we see that for all cases tested, and all attack lengths, padding-firstPPI provides a significant improvement in accuracy, as compared to attack-first PPI.Consequently, we expect an MDS that employs padding-first PPI to outperform asimilar MDS that employs attack-first PPI. Below, we show that this is indeed thecase for an HMM-based detector and for an OCNB-based detector. Furthermore, weshow that the advantage of padding-first PPI is most pronounced for shorter attacklengths, which are the cases where PPI is most relevant.

9

Page 10: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

4.2 HMM results

In this section we consider an HMM-based MDS. We compare results obtained whenno PPI is used to those obtained using the padding-first PPI and attack-first PPI.

We first score the attack blocks for each user and each attack length using HMMstrained for the appropriate user. The resulting scores are compared to scores obtainedon self blocks.

These experiments were then repeated with padding-first PPI applied to eachmasquerade attack block, with the scores computed using (5). Finally, we repeatedthis latter experiment using the attack-first PPI.

In Figure 7 we present ROC curves comparing these three different HMM-basedapproaches (no PPI, padding-first PPI and attack-first PPI) for attack length 10.Similarly, Figure 8 gives the ROC comparison for attack length 30. These resultsindicate that the none of the three methods succeeds at detecting attacks of length 10at a rate significantly better than guessing. On the other hand, the results for attacklength 30 show that no PPI and padding-first PPI offer some improvement over theattack-first approach.

Figure 9 shows the AUC comparison for the no PPI, padding-first PPI and attack-first PPI cases for each attack length tested. Figure 10 shows the partial AUC (AUCp)for a 5% false positive rate, for the same set of experiments. From these results, wesee that for the HMM-based detector under consideration, padding-first PPI showsimproved detection rates for attacks of length about 25 or more. In contrast, theattack-first PPI-based HMM detection rates are lower than the rates obtained usingno PPI for all attack lengths. See [25] for additional HMM results.

4.3 OCNB results

In this section we consider an OCNB-based MDS. As in the previous section, wecompare results obtained when no PPI is used with results for the padding-first PPIand attack-first PPI.

Figure 11 shows the ROC comparison for attacks of length 10. As in the HMMcase, detection rates are essentially equivalent to flipping a coin. However, in Fig-ure 12 we can infer that for attacks of length 30, padding-first PPI achieves abouta 50% improvement over the no PPI detection rate.

Figure 13 shows the AUC comparison for no PPI, padding-first PPI, and attack-first PPI using OCNB, and Figure 14 gives the AUCp results (FPR 5%) for the sameexperiments. For attack lengths of 30 or more, the padding-first PPI shows significantimprovement in detection rate, whereas the detection rates of attack-first PPI for anyattack length is lower than OCNB with no PPI. See [25] for additional OCNB results.

10

Page 11: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

4.4 The 1vs49 case

We provide detailed analysis of the “1vs49” test case for both the HMM and OCNB-based masquerade detectors, using both padding-first and attack-first PPI. That is,for each experiment, we test the user 1 model with the attack data selected from eachof the other 49 users. These cases cover all possible masquerade attacks on user 1for the specified |A|. Tables 1 and 2 contain AUC results of these 1vs49 tests forthe HMM detector using padding-first and attack-first PPI, respectively. Similarly,Tables 3 and 4 contain the AUC results of 1vs49 tests for OCNB, using padding-firstand attack-first PPI, respectively. The corresponding AUCp results, for 5% FPR, aregiven in Tables 5 through 8.

4.5 Discussion

The results in Figure 9 show that for our HMM-based MDS, the padding-first PPIoffers a marginal improvement over no PPI for attack lengths greater than 10. Inaddition, for the HMM case, the attack-first PPI is worse than using no PPI. FromFigure 13, we see that for our OCNB-based MDS, the padding-first PPI is againsuperior, and in this case, the improvement is significant. For OCNB, the attack-firstPPI performs poorly in all cases. Also, a comparison of Figures 9 and 13 shows thatthe HMM yields far superior results to the OCNB in every case, and the differenceis particularly pronounced for the attack-first PPI.

5 Conclusions and future work

In this paper, we considered masquerade detection systems based on hidden Markovmodel (HMM) analysis and one class naıve Bayes (OCNB). For both, we experimentedwith two types of probabilistic padding identification (PPI), namely, attack-first andpadding-first. The purpose of the PPI is to separate attack commands from paddingcommands, and thereby improve the scoring results. These PPI approaches bothrely on Kullback-Leibler (KL) divergence. The attack-first PPI has been analyzedin previous research [24], while the padding-first PPI is appears to be new. Ourexperiments were conducted using the Schonlau dataset of UNIX commands, whichhas formed the basis for a large volume of published masquerade detection research.

For both PPI algorithms, we measured the detection rates of the HMM andOCNB-based masquerade detection systems for attack lengths ranging between 10%to 100% of the block length. For the padding-first PPI we were able to correctlyidentify a significantly higher percentage of the padding in these masquerade attackblocks, as compared to the attack-first PPI. Given this results, it is not surpris-ing that both the HMM and OCNB detectors performed much better when usingthe padding-first PPI, as compared to the attack-first PPI. In fact, the HMM-basedattack-first PPI-based masquerade detection recorded no improvement over an HMM

11

Page 12: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

with no PPI, while for OCNB, the attack-first PPI results were actually worse thanan OCNB detector with no PPI.

The research in this paper was limited to HMM and OCNB-based masqueradedetection. This work could easily be extended to other related techniques, such assupport vector machines [20]. The attack-first PPI and padding-first PPI algorithmsconsidered here are the extreme cases. It is possible that better results would beobtained by an initialization scheme that includes an appropriate mixture of attackand padding commands. Also, the test case considered here was UNIX commands,but the same techniques could be applied to other examples of user behavior-baseddata, such as keyboard dynamics, mouse movements, and so on. More generally, thetechniques considered here could be applied to intrusion detection problems otherthan masquerade detection.

Our main objective was to obtain improved detection rates and this objective wasachieved. However, the true value of any masquerade detection system is measuredby its robustness, that is its ability to detect sophisticated attacks aimed at defeatingthe particular detection strategy. Further research needs to be conducted to searchfor attacks that might evade the systems analyzed here. If such attack strategies arefound, those results could, in turn, be used to strengthen statistical-based masqueradedetectors.

References

[1] M. Afgani, Anomaly detection using the Kullback-Leibler divergence metric,Applied Sciences on Biomedical and Communication Technologies. ISABEL ’08,First International Symposium, 1–5, 2008

[2] M. Bertacchini and P. I. Fierens, A Survey on Masquerader Detection Ap-proaches, CIBSI 2009,http://www.criptored.upm.es/cibsi/cibsi2009

/docs/Papers/CIBSI-Dia2-Sesion5(2).pdf

[3] A. P. Bradley, The use of the area under the roc curve in the evaluation ofmachine learning algorithms, Pattern Recognition, 30(7), 1145–1159, 1997

[4] T. Fawcett, An introduction to ROC analysis. Pattern Recognition Letters, 27(8),861–874, 2006

[5] Y. Gu, A. McCallum, and D. Towsley, Detecting anomalies in network traf-fic using maximum entropy estimation, IMC ’05 Proceedings of the 5th ACMSIGCOMM conference on Internet Measurement, 32–37, 2005

[6] L. Huang and M. Stamp, Masquerade detection using profile hidden Markovmodels, Computers and Security, 30(8), 732–747, 2011

[7] N. Idika and A. Mathur, A survey of malware detection techniques, Technicalreport, Software Engineering Research Center, 2007http://www.serc.net/system/files/SERC-TR-286.pdf

12

Page 13: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

[8] R. Khanna and H. Liu, Control theoretic approach to intrusion detection usinga distributed hidden Markov model, IEEE Wireless Communications, 15(4), 24–33, 2008

[9] H. Kim and S. Cha, Empirical evaluation of svm-based masquerade detectionusing unix commands. Computers and Security, 24(2), 160–168, 2005

[10] A. Kothari, Defeating masquerade detection, Master’s Projects, Paper 239, 2012http://scholarworks.sjsu.edu/etd_projects/239

[11] S. Kullback and R. A. Leibler, On information and sufficiency, The Annals ofMathematical Statistics, 22(1), 79–86, 1951

[12] R. Maxion and T. Townsend, Masquerade detection using truncated commandlines, Dependable Systems and Networks, DSN 2002, 219–228, 2002

[13] G. J. Mun, B. N. Noh and Y. M. Kim, Enhanced stochastic learning for featureselection in intrusion classification, International Journal of Innovative Comput-ing, Information and Control, 5(11A), 3625–3635, 2009

[14] A. Murali and M. Rao, A survey on intrusion detection approaches, Informationand Communication Technologies, ICICT 2005, 233–240, 2005

[15] J. H. Oh, J. Gao, and K. Rosenblatt, Biological data outlier detection based onKullback-Leibler divergence, Bioinformatics and Biomedicine, BIBM ’08, 249–254, 2008

[16] N. Runwal, R. M. Low, and M. Stamp, Opcode graph similarity and metamor-phic detection, Journal in Computer Virology, 8(1–2), 37–52, 2012

[17] M. Schonlau and M. Theus, Detecting masquerades in intrusion detection basedon unpopular commands, Information Processing Letters, 76(1–2), 33–38, 2000

[18] Schonlau, M. Masquerading user data. Masquerade data. Retrieved from:http://www.schonlau.net/intrusion.html

[19] A. Sharma and K. Paliwal, Detecting masquerades using a combination of naıveBayes and weighted RBF approach, Journal in Computer Virology, 3(3), 237–245, 2007

[20] S. Shetty, S. K. Mukkavilli, and L. H. Keel, An integrated machine learning andcontrol theoretic model for mining concept-drifting data streams, Technologiesfor Homeland Security (HST), 75–80, 2011

[21] S. M. Sridhara and M. Stamp, Metamorphic worm that carries its own morphingengine, Journal of Computer Virology and Hacking Techniques, 9(2), 49–58, 2013

[22] M. Stamp, A revealing introduction to hidden Markov model, 2012http://www.cs.sjsu.edu/~stamp/RUA/HMM.pdf

[23] M. Stamp. Information Security: Principles and Practice, 2nd edition, Wiley,2011

13

Page 14: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

[24] J. Tapiador and J. Clark, Masquerade mimicry attack detection: A randomisedapproach, Computers and Security, 30(5), 297–310, 2011

[25] G. R. Viswanathan, Analysis of Kullback-Leibler Divergence for MasqueradeDetection, Master’s report, Department of Computer Science, San Jose StateUniversity, 2013

[26] K. Wang and S. Stolfo, One class training for masquerade detection, 3rd IEEEConference Data Mining Workshop on Data Mining for Computer Security, 2003http://cs.columbia.edu/~kewang/paper/DMSEC-camera.pdf

[27] Q. Yin, et al, Intrusion detection based on hidden Markov model, MachineLearning and Cybernetics, 5, 3115–3118, 2003

14

Page 15: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Figure 1: Schonlau dataset representation [6]

Figure 2: Map file for Schonlau dataset [6]

15

Page 16: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Predicted Class

P N

Act

ual

Cla

ss P TP FN

N FP TN

Figure 3: Confusion matrix

Markov process: X0 X1 X2 · · · XT−1-A -A -A -A

?

B

?

B

?

B

?

B

Observations: O0 O1 O2 · · · OT−1

Figure 4: Hidden Markov process [22]

Figure 5: Example of mimicry attack [24]

16

Page 17: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Figure 6: Padding-First PPI vs Attack-First PPI

Figure 7: HMM: ROC comparison for |A| = 10

17

Page 18: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Figure 8: HMM: ROC comparison for |A| = 30

Figure 9: HMM: AUC comparison

18

Page 19: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Figure 10: HMM: AUCp comparison (FPR 5%)

Figure 11: OCNB: ROC comparison for |A| = 10

19

Page 20: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Figure 12: OCNB: ROC comparison for |A| = 30

Figure 13: OCNB: AUC comparison

20

Page 21: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Figure 14: OCNB: AUCp comparison (FPR 5%)

21

Page 22: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 1: 1vs49 AUC comparison of HMM without PPI and padding-first PPI (bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.47253 / 0.44335 0.54527 / 0.55755 0.61294 / 0.65351 0.67918 / 0.72714 0.74433 / 0.784412 0.69906 / 0.75661 0.82637 / 0.83229 0.89029 / 0.88853 0.92082 / 0.92073 0.93539 / 0.938163 0.72673 / 0.78743 0.82069 / 0.83918 0.88029 / 0.88331 0.91384 / 0.91106 0.93535 / 0.933314 0.59527 / 0.59682 0.67929 / 0.68882 0.76090 / 0.76449 0.80571 / 0.79384 0.84110 / 0.826535 0.70722 / 0.78510 0.81300 / 0.85314 0.86008 / 0.87751 0.86718 / 0.88457 0.88033 / 0.895766 0.79580 / 0.81518 0.87016 / 0.87992 0.90927 / 0.94208 0.93286 / 0.96869 0.95531 / 0.974577 0.48163 / 0.61053 0.56722 / 0.68743 0.64527 / 0.74363 0.71853 / 0.78702 0.77347 / 0.825598 0.82498 / 0.81465 0.85939 / 0.86335 0.89384 / 0.90237 0.91673 / 0.92498 0.93559 / 0.941719 0.28616 / 0.56878 0.44735 / 0.67624 0.58482 / 0.74416 0.69118 / 0.79200 0.76845 / 0.8370210 0.70367 / 0.71098 0.73718 / 0.75147 0.78424 / 0.79494 0.84339 / 0.84449 0.89163 / 0.8917611 0.65645 / 0.71853 0.74359 / 0.80602 0.80827 / 0.86155 0.85265 / 0.89547 0.87737 / 0.9217612 0.56767 / 0.59698 0.63024 / 0.64592 0.68012 / 0.68767 0.72208 / 0.72710 0.74376 / 0.7507313 0.60482 / 0.55984 0.68400 / 0.67073 0.74690 / 0.74106 0.78884 / 0.78971 0.81890 / 0.8151414 0.36710 / 0.55420 0.46314 / 0.66061 0.56147 / 0.75020 0.66780 / 0.81196 0.76935 / 0.8560015 0.41894 / 0.50882 0.55359 / 0.60771 0.63482 / 0.67947 0.69367 / 0.73037 0.72653 / 0.7668216 0.51151 / 0.51469 0.57884 / 0.56657 0.63955 / 0.61959 0.69184 / 0.67135 0.73363 / 0.7160017 0.49592 / 0.51490 0.57388 / 0.60649 0.65163 / 0.66988 0.71494 / 0.72559 0.76718 / 0.7708618 0.75612 / 0.73016 0.80433 / 0.80176 0.82478 / 0.82980 0.85829 / 0.86180 0.88029 / 0.8789019 0.55012 / 0.64498 0.68641 / 0.75208 0.78106 / 0.82731 0.82751 / 0.86208 0.85192 / 0.8822920 0.61143 / 0.51600 0.67563 / 0.62269 0.72147 / 0.68694 0.75947 / 0.73412 0.78429 / 0.7612221 0.78776 / 0.82837 0.90763 / 0.90155 0.92690 / 0.92604 0.93755 / 0.93467 0.94400 / 0.9437622 0.67469 / 0.75359 0.77192 / 0.82441 0.82049 / 0.87314 0.85082 / 0.89592 0.86396 / 0.9108223 0.32506 / 0.41412 0.43990 / 0.48939 0.51653 / 0.55535 0.57608 / 0.61188 0.63384 / 0.6633124 0.56527 / 0.67057 0.69102 / 0.79057 0.79437 / 0.86322 0.87237 / 0.90241 0.90853 / 0.9266125 0.51686 / 0.64514 0.62971 / 0.71922 0.71649 / 0.77878 0.79020 / 0.82473 0.84351 / 0.8657626 0.66208 / 0.62306 0.75816 / 0.76531 0.83261 / 0.83735 0.87245 / 0.88237 0.89457 / 0.9113927 0.35935 / 0.56629 0.53363 / 0.67457 0.66612 / 0.75527 0.76306 / 0.81065 0.82692 / 0.8513928 0.73147 / 0.78710 0.86729 / 0.88167 0.92073 / 0.92241 0.93653 / 0.94482 0.94196 / 0.9516329 0.51629 / 0.56257 0.63029 / 0.68894 0.73302 / 0.76751 0.79792 / 0.81829 0.83673 / 0.8463330 1.00000 / 1.00000 1.00000 / 1.00000 1.00000 / 1.00000 1.00000 / 1.00000 1.00000 / 1.0000031 0.56210 / 0.74155 0.70951 / 0.85331 0.79935 / 0.89437 0.85116 / 0.91314 0.88771 / 0.9323132 0.78694 / 0.78522 0.78931 / 0.78735 0.79441 / 0.79282 0.80078 / 0.79880 0.80861 / 0.8071433 0.65543 / 0.68914 0.72388 / 0.75865 0.78633 / 0.81020 0.83693 / 0.84849 0.89041 / 0.8879234 0.67816 / 0.64710 0.78318 / 0.74902 0.84641 / 0.82984 0.87547 / 0.86494 0.89522 / 0.8916735 0.45704 / 0.62588 0.68669 / 0.75278 0.84710 / 0.86976 0.94424 / 0.94792 0.98151 / 0.9826136 0.76331 / 0.79559 0.84527 / 0.85661 0.89429 / 0.90706 0.93020 / 0.93914 0.95282 / 0.9550637 0.54204 / 0.59445 0.63004 / 0.69759 0.71327 / 0.76600 0.78090 / 0.82539 0.82584 / 0.8612238 0.66180 / 0.74445 0.77886 / 0.82216 0.83286 / 0.86539 0.86898 / 0.89424 0.88906 / 0.9130239 0.45853 / 0.60057 0.60890 / 0.70257 0.70339 / 0.76265 0.75608 / 0.79967 0.78571 / 0.8261240 0.60294 / 0.70041 0.71682 / 0.79490 0.79988 / 0.85057 0.83841 / 0.87988 0.86690 / 0.9007841 0.66233 / 0.78633 0.72718 / 0.84159 0.78943 / 0.86731 0.83041 / 0.88037 0.85518 / 0.9036342 0.62329 / 0.63971 0.70006 / 0.71151 0.74678 / 0.75498 0.77776 / 0.78131 0.80257 / 0.8144143 0.72269 / 0.71710 0.75820 / 0.76865 0.79424 / 0.80367 0.82694 / 0.83204 0.85355 / 0.8643344 0.62727 / 0.64951 0.69016 / 0.71122 0.74812 / 0.75833 0.78829 / 0.79294 0.81869 / 0.8184945 0.55522 / 0.65822 0.66949 / 0.75371 0.75724 / 0.80147 0.81514 / 0.83596 0.84735 / 0.8640046 0.66204 / 0.74237 0.77937 / 0.80955 0.85592 / 0.86873 0.90441 / 0.91278 0.93494 / 0.9403747 0.58045 / 0.62229 0.70592 / 0.76608 0.82576 / 0.85686 0.90049 / 0.92163 0.94396 / 0.9456748 0.68894 / 0.66204 0.74082 / 0.72722 0.77510 / 0.76727 0.81249 / 0.79751 0.82659 / 0.8166549 0.32482 / 0.48229 0.46784 / 0.57265 0.57367 / 0.63951 0.63906 / 0.68008 0.68318 / 0.7142050 0.50657 / 0.62990 0.62502 / 0.72498 0.71031 / 0.78671 0.75245 / 0.81963 0.78502 / 0.84555

22

Page 23: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 2: 1vs49 AUC comparison of HMM without PPI and attack-first PPI (bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.47253 / 0.47522 0.54527 / 0.53678 0.61294 / 0.59429 0.67918 / 0.65604 0.74433 / 0.717472 0.69906 / 0.70384 0.82637 / 0.82767 0.89029 / 0.89604 0.92082 / 0.92710 0.93539 / 0.946533 0.72673 / 0.72694 0.82069 / 0.81690 0.88029 / 0.86727 0.91384 / 0.89976 0.93535 / 0.919514 0.59527 / 0.61604 0.67929 / 0.67947 0.76090 / 0.74690 0.80571 / 0.78710 0.84110 / 0.822005 0.70722 / 0.71849 0.81300 / 0.80620 0.86008 / 0.84543 0.86718 / 0.85698 0.88033 / 0.866826 0.79580 / 0.79073 0.87016 / 0.87224 0.90927 / 0.90486 0.93286 / 0.92473 0.95531 / 0.945477 0.48163 / 0.46812 0.56722 / 0.55024 0.64527 / 0.62559 0.71853 / 0.69061 0.77347 / 0.747028 0.82498 / 0.82310 0.85939 / 0.84763 0.89384 / 0.87396 0.91673 / 0.89045 0.93559 / 0.906619 0.28616 / 0.28016 0.44735 / 0.43024 0.58482 / 0.56159 0.69118 / 0.67298 0.76845 / 0.7423310 0.70367 / 0.71873 0.73718 / 0.74678 0.78424 / 0.78059 0.84339 / 0.82008 0.89163 / 0.8577611 0.65645 / 0.67494 0.74359 / 0.74976 0.80827 / 0.80453 0.85265 / 0.84461 0.87737 / 0.8633912 0.56767 / 0.59159 0.63024 / 0.65433 0.68012 / 0.69257 0.72208 / 0.72686 0.74376 / 0.7473913 0.60482 / 0.58024 0.68400 / 0.65800 0.74690 / 0.71559 0.78884 / 0.76531 0.81890 / 0.8054314 0.36710 / 0.34404 0.46314 / 0.43053 0.56147 / 0.51486 0.66780 / 0.60812 0.76935 / 0.7043315 0.41894 / 0.38886 0.55359 / 0.54918 0.63482 / 0.62271 0.69367 / 0.67555 0.72653 / 0.7032716 0.51151 / 0.48686 0.57884 / 0.56224 0.63955 / 0.61139 0.69184 / 0.66163 0.73363 / 0.7021217 0.49592 / 0.49551 0.57388 / 0.58161 0.65163 / 0.65649 0.71494 / 0.72384 0.76718 / 0.7787318 0.75612 / 0.75253 0.80433 / 0.80233 0.82478 / 0.82449 0.85829 / 0.85531 0.88029 / 0.8798419 0.55012 / 0.60127 0.68641 / 0.70661 0.78106 / 0.78371 0.82751 / 0.81780 0.85192 / 0.8300820 0.61143 / 0.62510 0.67563 / 0.67739 0.72147 / 0.72200 0.75947 / 0.75449 0.78429 / 0.7745721 0.78776 / 0.80665 0.90763 / 0.90539 0.92690 / 0.92531 0.93755 / 0.93282 0.94400 / 0.9386122 0.67469 / 0.62759 0.77192 / 0.71318 0.82049 / 0.76710 0.85082 / 0.80041 0.86396 / 0.8128623 0.32506 / 0.35514 0.43990 / 0.46894 0.51653 / 0.54331 0.57608 / 0.60147 0.63384 / 0.6479624 0.56527 / 0.57641 0.69102 / 0.68498 0.79437 / 0.77722 0.87237 / 0.85576 0.90853 / 0.8868625 0.51686 / 0.52245 0.62971 / 0.62237 0.71649 / 0.70429 0.79020 / 0.77673 0.84351 / 0.8222426 0.66208 / 0.69053 0.75816 / 0.77698 0.83261 / 0.84069 0.87245 / 0.87408 0.89457 / 0.8864527 0.35935 / 0.35359 0.53363 / 0.50110 0.66612 / 0.62482 0.76306 / 0.70890 0.82692 / 0.7746328 0.73147 / 0.73727 0.86729 / 0.85882 0.92073 / 0.90831 0.93653 / 0.93265 0.94196 / 0.9316729 0.51629 / 0.53253 0.63029 / 0.65935 0.73302 / 0.74082 0.79792 / 0.80078 0.83673 / 0.8335930 1.00000 / 1.00000 1.00000 / 1.00000 1.00000 / 1.00000 1.00000 / 1.00000 1.00000 / 1.0000031 0.56210 / 0.54233 0.70951 / 0.69282 0.79935 / 0.77976 0.85116 / 0.82694 0.88771 / 0.8635932 0.78694 / 0.83143 0.78931 / 0.84335 0.79441 / 0.85041 0.80078 / 0.86073 0.80861 / 0.8683733 0.65543 / 0.66616 0.72388 / 0.72127 0.78633 / 0.76886 0.83693 / 0.81159 0.89041 / 0.8484134 0.67816 / 0.67324 0.78318 / 0.76961 0.84641 / 0.82029 0.87547 / 0.84408 0.89522 / 0.8622435 0.45704 / 0.41910 0.68669 / 0.64951 0.84710 / 0.81494 0.94424 / 0.90224 0.98151 / 0.9513136 0.76331 / 0.76453 0.84527 / 0.83935 0.89429 / 0.88445 0.93020 / 0.91780 0.95282 / 0.9393937 0.54204 / 0.53653 0.63004 / 0.61833 0.71327 / 0.68759 0.78090 / 0.75757 0.82584 / 0.7953538 0.66180 / 0.63845 0.77886 / 0.75069 0.83286 / 0.81490 0.86898 / 0.83967 0.88906 / 0.8607339 0.45853 / 0.42482 0.60890 / 0.57539 0.70339 / 0.67045 0.75608 / 0.73861 0.78571 / 0.7610240 0.60294 / 0.51510 0.71682 / 0.65543 0.79988 / 0.75527 0.83841 / 0.79139 0.86690 / 0.8287341 0.66233 / 0.66359 0.72718 / 0.72841 0.78943 / 0.78531 0.83041 / 0.82710 0.85518 / 0.8513542 0.62329 / 0.59447 0.70006 / 0.70196 0.74678 / 0.74984 0.77776 / 0.77567 0.80257 / 0.7968643 0.72269 / 0.72478 0.75820 / 0.75331 0.79424 / 0.78331 0.82694 / 0.80741 0.85355 / 0.8271844 0.62727 / 0.88163 0.69016 / 0.88833 0.74812 / 0.89286 0.78829 / 0.89494 0.81869 / 0.8975145 0.55522 / 0.54216 0.66949 / 0.64951 0.75724 / 0.73290 0.81514 / 0.78639 0.84735 / 0.8188846 0.66204 / 0.65441 0.77937 / 0.76135 0.85592 / 0.83416 0.90441 / 0.88420 0.93494 / 0.9151447 0.58045 / 0.56608 0.70592 / 0.68265 0.82576 / 0.79347 0.90049 / 0.86714 0.94396 / 0.9146548 0.68894 / 0.67073 0.74082 / 0.72602 0.77510 / 0.75767 0.81249 / 0.79086 0.82659 / 0.8016749 0.32482 / 0.33486 0.46784 / 0.46173 0.57367 / 0.56086 0.63906 / 0.61114 0.68318 / 0.6504750 0.50657 / 0.43265 0.62502 / 0.54616 0.71031 / 0.64759 0.75245 / 0.68445 0.78502 / 0.71563

23

Page 24: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 3: 1vs49 AUC comparison of OCNB without PPI and padding-first PPI (bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.39180 / 0.32543 0.42886 / 0.42935 0.46963 / 0.52278 0.50669 / 0.59139 0.55633 / 0.641962 0.58671 / 0.77935 0.72710 / 0.79502 0.82110 / 0.83592 0.87902 / 0.88351 0.91367 / 0.912843 0.54414 / 0.74929 0.64453 / 0.78331 0.71804 / 0.81543 0.77414 / 0.83976 0.81747 / 0.861474 0.55069 / 0.55486 0.58604 / 0.62514 0.62873 / 0.68233 0.66449 / 0.70700 0.69555 / 0.730905 0.31080 / 0.69347 0.40378 / 0.78645 0.49510 / 0.81037 0.53986 / 0.80918 0.59437 / 0.827806 0.46018 / 0.75108 0.68688 / 0.83755 0.77114 / 0.89143 0.81380 / 0.93476 0.84998 / 0.945677 0.36631 / 0.57082 0.39557 / 0.62882 0.42592 / 0.66637 0.45992 / 0.69535 0.49886 / 0.723108 0.77849 / 0.72645 0.79690 / 0.78939 0.81478 / 0.82184 0.82984 / 0.84143 0.84394 / 0.861499 0.08755 / 0.53447 0.13090 / 0.60253 0.17931 / 0.63837 0.23635 / 0.66545 0.29398 / 0.6973510 0.68645 / 0.70086 0.70016 / 0.72273 0.71682 / 0.74220 0.73102 / 0.75598 0.75108 / 0.7708611 0.46392 / 0.66120 0.48847 / 0.74857 0.51757 / 0.79294 0.56216 / 0.82520 0.58416 / 0.8510212 0.39263 / 0.53604 0.42394 / 0.56976 0.44537 / 0.58865 0.46580 / 0.60657 0.48218 / 0.6175913 0.48539 / 0.45496 0.53316 / 0.55914 0.57949 / 0.62573 0.61794 / 0.66906 0.65904 / 0.6931014 0.27931 / 0.56588 0.31910 / 0.62106 0.37118 / 0.67673 0.43335 / 0.72496 0.50788 / 0.7579215 0.17222 / 0.46049 0.23145 / 0.54518 0.29731 / 0.59167 0.35973 / 0.62792 0.41404 / 0.6555516 0.39569 / 0.43229 0.46661 / 0.46004 0.51914 / 0.48696 0.56757 / 0.52049 0.60624 / 0.5495117 0.41604 / 0.48722 0.47406 / 0.57257 0.53743 / 0.62459 0.58329 / 0.66359 0.62461 / 0.6962018 0.72098 / 0.51869 0.79082 / 0.66541 0.81931 / 0.78057 0.84604 / 0.83449 0.87392 / 0.8624919 0.20863 / 0.56412 0.27386 / 0.66576 0.34416 / 0.72894 0.38943 / 0.75869 0.44141 / 0.7865720 0.54045 / 0.35184 0.59861 / 0.47816 0.64253 / 0.55902 0.68649 / 0.61959 0.71404 / 0.6471821 0.65176 / 0.78355 0.75841 / 0.81196 0.84488 / 0.85039 0.90347 / 0.87951 0.94639 / 0.9176322 0.22455 / 0.64108 0.28769 / 0.73710 0.33216 / 0.79249 0.37339 / 0.82927 0.41394 / 0.8468223 0.13800 / 0.40824 0.23747 / 0.47504 0.34712 / 0.53131 0.44518 / 0.57808 0.51533 / 0.6301224 0.39204 / 0.60465 0.44365 / 0.72053 0.49927 / 0.78665 0.55841 / 0.82302 0.59663 / 0.8523725 0.39592 / 0.58353 0.45551 / 0.64845 0.50927 / 0.68498 0.57373 / 0.72657 0.62627 / 0.7601226 0.43053 / 0.46057 0.48971 / 0.58069 0.54708 / 0.63669 0.59753 / 0.68629 0.62631 / 0.7159027 0.14571 / 0.50424 0.20302 / 0.55551 0.26857 / 0.59482 0.35384 / 0.64049 0.43024 / 0.6798228 0.11571 / 0.67784 0.17492 / 0.78661 0.26343 / 0.82216 0.36710 / 0.84290 0.43435 / 0.8505329 0.32955 / 0.43694 0.39922 / 0.52235 0.46749 / 0.58392 0.54078 / 0.64539 0.60945 / 0.6993330 0.47543 / 1.00000 0.71273 / 1.00000 0.84384 / 1.00000 0.90571 / 1.00000 0.93367 / 1.0000031 0.24931 / 0.68131 0.32982 / 0.81069 0.41700 / 0.84173 0.48876 / 0.85735 0.55406 / 0.8793932 0.84555 / 0.84378 0.85722 / 0.85482 0.86359 / 0.86143 0.86882 / 0.86514 0.87131 / 0.8677133 0.59376 / 0.62200 0.62424 / 0.67143 0.66082 / 0.70280 0.69463 / 0.72865 0.72616 / 0.7432234 0.25406 / 0.55269 0.35408 / 0.63651 0.45449 / 0.68763 0.51084 / 0.71571 0.55094 / 0.7409435 0.31878 / 0.57906 0.46327 / 0.62629 0.61514 / 0.69453 0.74829 / 0.78204 0.83890 / 0.8517636 0.63878 / 0.77404 0.73155 / 0.81245 0.79622 / 0.84510 0.84004 / 0.87837 0.87310 / 0.9065337 0.40857 / 0.48192 0.44771 / 0.56649 0.48245 / 0.61629 0.52690 / 0.67780 0.56984 / 0.7203538 0.36318 / 0.65922 0.44973 / 0.74784 0.51545 / 0.78367 0.57563 / 0.81282 0.61404 / 0.8411039 0.18147 / 0.56431 0.19978 / 0.64500 0.22086 / 0.68655 0.25976 / 0.70812 0.29057 / 0.7240640 0.31082 / 0.58802 0.38571 / 0.65492 0.45608 / 0.69469 0.51759 / 0.72710 0.57755 / 0.7525541 0.46555 / 0.72910 0.51424 / 0.79488 0.55653 / 0.82031 0.59159 / 0.82763 0.61755 / 0.8501242 0.38588 / 0.58461 0.55343 / 0.66333 0.64139 / 0.71188 0.69173 / 0.74669 0.72171 / 0.7832743 0.66947 / 0.69882 0.67969 / 0.74196 0.68833 / 0.76127 0.69824 / 0.77563 0.70490 / 0.7948244 0.50137 / 0.59918 0.54551 / 0.64669 0.57773 / 0.68441 0.60771 / 0.71102 0.63637 / 0.7364145 0.34120 / 0.57841 0.41663 / 0.67041 0.48159 / 0.70339 0.53429 / 0.72947 0.57341 / 0.7588646 0.50398 / 0.72298 0.61294 / 0.78049 0.69967 / 0.81118 0.75673 / 0.84151 0.81196 / 0.8618047 0.49816 / 0.49188 0.57555 / 0.66649 0.65365 / 0.75280 0.74335 / 0.82963 0.81967 / 0.8816348 0.52190 / 0.58698 0.53647 / 0.62392 0.53935 / 0.64094 0.55410 / 0.65214 0.56320 / 0.6573549 0.11159 / 0.43196 0.13790 / 0.47565 0.17947 / 0.50861 0.21957 / 0.52216 0.25916 / 0.5406350 0.20076 / 0.54067 0.23582 / 0.62576 0.27535 / 0.66980 0.31796 / 0.69657 0.33696 / 0.71769

24

Page 25: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 4: 1vs49 AUC comparison of OCNB without PPI and attack-first PPI (bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.39180 / 0.40222 0.42886 / 0.42655 0.46963 / 0.44347 0.50669 / 0.46516 0.55633 / 0.495082 0.58671 / 0.64653 0.72710 / 0.72688 0.82110 / 0.77053 0.87902 / 0.77741 0.91367 / 0.812043 0.54414 / 0.50737 0.64453 / 0.58886 0.71804 / 0.62016 0.77414 / 0.66441 0.81747 / 0.693554 0.55069 / 0.60849 0.58604 / 0.60845 0.62873 / 0.62004 0.66449 / 0.62384 0.69555 / 0.622415 0.31080 / 0.36135 0.40378 / 0.38206 0.49510 / 0.39402 0.53986 / 0.39961 0.59437 / 0.429786 0.46018 / 0.45392 0.68688 / 0.59445 0.77114 / 0.65931 0.81380 / 0.73400 0.84998 / 0.767047 0.36631 / 0.35196 0.39557 / 0.37037 0.42592 / 0.39649 0.45992 / 0.39780 0.49886 / 0.421248 0.77849 / 0.77408 0.79690 / 0.77947 0.81478 / 0.78686 0.82984 / 0.78114 0.84394 / 0.786209 0.08755 / 0.11555 0.13090 / 0.15118 0.17931 / 0.18049 0.23635 / 0.22298 0.29398 / 0.2446510 0.68645 / 0.68976 0.70016 / 0.68867 0.71682 / 0.68706 0.73102 / 0.66543 0.75108 / 0.6528611 0.46392 / 0.44824 0.48847 / 0.44684 0.51757 / 0.43141 0.56216 / 0.43716 0.58416 / 0.4314312 0.39263 / 0.40280 0.42394 / 0.42224 0.44537 / 0.42492 0.46580 / 0.42333 0.48218 / 0.4125313 0.48539 / 0.44216 0.53316 / 0.49447 0.57949 / 0.50612 0.61794 / 0.53204 0.65904 / 0.5684314 0.27931 / 0.26869 0.31910 / 0.29869 0.37118 / 0.32620 0.43335 / 0.35686 0.50788 / 0.4054315 0.17222 / 0.20024 0.23145 / 0.23657 0.29731 / 0.25310 0.35973 / 0.30669 0.41404 / 0.3500816 0.39569 / 0.32171 0.46661 / 0.36955 0.51914 / 0.39873 0.56757 / 0.44796 0.60624 / 0.4791417 0.41604 / 0.45663 0.47406 / 0.48767 0.53743 / 0.51608 0.58329 / 0.55433 0.62461 / 0.5884918 0.72098 / 0.63645 0.79082 / 0.71686 0.81931 / 0.75053 0.84604 / 0.80269 0.87392 / 0.8209019 0.20863 / 0.34202 0.27386 / 0.35878 0.34416 / 0.37027 0.38943 / 0.38163 0.44141 / 0.4155120 0.54045 / 0.56114 0.59861 / 0.59376 0.64253 / 0.59106 0.68649 / 0.59355 0.71404 / 0.5984921 0.65176 / 0.60890 0.75841 / 0.66496 0.84488 / 0.72569 0.90347 / 0.75445 0.94639 / 0.7697322 0.22455 / 0.22002 0.28769 / 0.21088 0.33216 / 0.21245 0.37339 / 0.20482 0.41394 / 0.2073523 0.13800 / 0.25086 0.23747 / 0.31704 0.34712 / 0.37459 0.44518 / 0.43110 0.51533 / 0.4703324 0.39204 / 0.41914 0.44365 / 0.43945 0.49927 / 0.47690 0.55841 / 0.51455 0.59663 / 0.5405125 0.39592 / 0.40465 0.45551 / 0.44455 0.50927 / 0.46361 0.57373 / 0.49939 0.62627 / 0.5273926 0.43053 / 0.50476 0.48971 / 0.51971 0.54708 / 0.52710 0.59753 / 0.52596 0.62631 / 0.5210227 0.14571 / 0.16257 0.20302 / 0.18716 0.26857 / 0.21122 0.35384 / 0.24347 0.43024 / 0.2934128 0.11571 / 0.21486 0.17492 / 0.23822 0.26343 / 0.26918 0.36710 / 0.29008 0.43435 / 0.3084929 0.32955 / 0.42380 0.39922 / 0.47557 0.46749 / 0.48963 0.54078 / 0.54488 0.60945 / 0.5864930 0.47543 / 0.60759 0.71273 / 0.89298 0.84384 / 0.92433 0.90571 / 0.95180 0.93367 / 0.9539231 0.24931 / 0.24733 0.32982 / 0.29916 0.41700 / 0.32829 0.48876 / 0.36600 0.55406 / 0.4149832 0.84555 / 0.92000 0.85722 / 0.92045 0.86359 / 0.90804 0.86882 / 0.89073 0.87131 / 0.8815533 0.59376 / 0.55314 0.62424 / 0.55486 0.66082 / 0.57033 0.69463 / 0.58335 0.72616 / 0.5985534 0.25406 / 0.32249 0.35408 / 0.35147 0.45449 / 0.37290 0.51084 / 0.38076 0.55094 / 0.4208835 0.31878 / 0.28649 0.46327 / 0.38927 0.61514 / 0.49824 0.74829 / 0.57147 0.83890 / 0.6602036 0.63878 / 0.59780 0.73155 / 0.66678 0.79622 / 0.70539 0.84004 / 0.72627 0.87310 / 0.7378837 0.40857 / 0.41049 0.44771 / 0.42549 0.48245 / 0.43469 0.52690 / 0.46649 0.56984 / 0.4824138 0.36318 / 0.34808 0.44973 / 0.38516 0.51545 / 0.39761 0.57563 / 0.43502 0.61404 / 0.4501439 0.18147 / 0.15973 0.19978 / 0.15390 0.22086 / 0.15516 0.25976 / 0.15192 0.29057 / 0.1618440 0.31082 / 0.23710 0.38571 / 0.26359 0.45608 / 0.30171 0.51759 / 0.31678 0.57755 / 0.3410841 0.46555 / 0.48649 0.51424 / 0.49618 0.55653 / 0.49682 0.59159 / 0.50273 0.61755 / 0.4994742 0.38588 / 0.34718 0.55343 / 0.47604 0.64139 / 0.54463 0.69173 / 0.61565 0.72171 / 0.6561043 0.66947 / 0.66988 0.67969 / 0.67396 0.68833 / 0.66935 0.69824 / 0.66082 0.70490 / 0.6540044 0.50137 / 0.46963 0.54551 / 0.49865 0.57773 / 0.52310 0.60771 / 0.54953 0.63637 / 0.5599645 0.34120 / 0.35490 0.41663 / 0.38543 0.48159 / 0.41378 0.53429 / 0.43347 0.57341 / 0.4557646 0.50398 / 0.46751 0.61294 / 0.55149 0.69967 / 0.62057 0.75673 / 0.66849 0.81196 / 0.7002447 0.49816 / 0.44773 0.57555 / 0.51755 0.65365 / 0.56727 0.74335 / 0.63114 0.81967 / 0.6672748 0.52190 / 0.46212 0.53647 / 0.48122 0.53935 / 0.48506 0.55410 / 0.48302 0.56320 / 0.4798449 0.11159 / 0.19082 0.13790 / 0.19671 0.17947 / 0.21143 0.21957 / 0.20743 0.25916 / 0.2150250 0.20076 / 0.17576 0.23582 / 0.18682 0.27535 / 0.21110 0.31796 / 0.20982 0.33696 / 0.21404

25

Page 26: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 5: 1vs49 AUCp (FPR 5%) comparison of HMM without PPI and padding-first PPI(bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.05714 / 0.02286 0.08490 / 0.06612 0.16327 / 0.09224 0.21959 / 0.12816 0.22531 / 0.243272 0.13388 / 0.50612 0.44653 / 0.51184 0.72082 / 0.70204 0.75265 / 0.77714 0.69143 / 0.751843 0.36735 / 0.47837 0.50041 / 0.56980 0.59347 / 0.64980 0.73388 / 0.74776 0.81224 / 0.807354 0.30041 / 0.21714 0.32327 / 0.24490 0.35265 / 0.27020 0.38612 / 0.27755 0.40000 / 0.309395 0.12653 / 0.18286 0.19592 / 0.22367 0.14857 / 0.17796 0.12000 / 0.15592 0.08816 / 0.141226 0.59020 / 0.58531 0.66367 / 0.65061 0.74939 / 0.76490 0.71918 / 0.79184 0.76980 / 0.823677 0.01714 / 0.03429 0.08000 / 0.07592 0.12408 / 0.21633 0.16245 / 0.18204 0.17224 / 0.251438 0.68000 / 0.58776 0.68980 / 0.59673 0.71837 / 0.62776 0.74449 / 0.65551 0.78122 / 0.706129 0.00980 / 0.00571 0.04571 / 0.04000 0.06531 / 0.07510 0.10204 / 0.15184 0.10286 / 0.2228610 0.62612 / 0.52898 0.64816 / 0.61306 0.66286 / 0.64898 0.66612 / 0.66204 0.67265 / 0.6946911 0.09959 / 0.07510 0.18857 / 0.16408 0.20735 / 0.31347 0.25796 / 0.39673 0.23265 / 0.5102012 0.17959 / 0.20000 0.21224 / 0.24000 0.23102 / 0.24653 0.27184 / 0.35102 0.20980 / 0.3379613 0.14857 / 0.09306 0.19755 / 0.19755 0.22449 / 0.22857 0.25714 / 0.26367 0.20735 / 0.2465314 0.02204 / 0.03347 0.05633 / 0.18041 0.15184 / 0.27265 0.17796 / 0.10204 0.24000 / 0.0522415 0.02531 / 0.02939 0.04735 / 0.04571 0.08082 / 0.08980 0.09143 / 0.07673 0.05551 / 0.1028616 0.21469 / 0.06857 0.24408 / 0.04000 0.30531 / 0.03918 0.32327 / 0.04898 0.30449 / 0.0857117 0.07510 / 0.06531 0.18939 / 0.19265 0.25959 / 0.24816 0.32082 / 0.27755 0.41061 / 0.3085718 0.50857 / 0.20327 0.66041 / 0.64245 0.68816 / 0.69388 0.70204 / 0.72000 0.71918 / 0.7249019 0.13959 / 0.15265 0.15918 / 0.17714 0.17959 / 0.19673 0.17388 / 0.17633 0.11184 / 0.1575520 0.17714 / 0.03592 0.27837 / 0.04735 0.26041 / 0.07184 0.28735 / 0.08735 0.24816 / 0.0832721 0.47673 / 0.42857 0.61633 / 0.60980 0.82857 / 0.82531 0.86286 / 0.86612 0.87837 / 0.8857122 0.08490 / 0.11673 0.12490 / 0.14286 0.16816 / 0.20490 0.18122 / 0.20980 0.13061 / 0.2179623 0.03837 / 0.05796 0.19592 / 0.21061 0.23592 / 0.26449 0.29551 / 0.31837 0.34286 / 0.3640824 0.12408 / 0.10041 0.16082 / 0.17469 0.20735 / 0.24653 0.26694 / 0.29469 0.31184 / 0.3306125 0.04898 / 0.12735 0.14367 / 0.13143 0.20408 / 0.20327 0.31429 / 0.27184 0.37551 / 0.3477626 0.10449 / 0.07184 0.14694 / 0.09959 0.16490 / 0.11837 0.16163 / 0.19918 0.24571 / 0.3102027 0.01143 / 0.02612 0.01959 / 0.06857 0.06286 / 0.14776 0.15755 / 0.27837 0.25714 / 0.3600028 0.16735 / 0.16490 0.23347 / 0.28653 0.23510 / 0.32082 0.23918 / 0.33143 0.30531 / 0.3665329 0.06041 / 0.13061 0.11429 / 0.18531 0.31837 / 0.30286 0.40898 / 0.44490 0.48163 / 0.5349030 0.97959 / 0.97959 0.97959 / 0.97959 0.97959 / 0.97959 0.97959 / 0.97959 0.97959 / 0.9795931 0.11510 / 0.12653 0.19673 / 0.22041 0.19592 / 0.22776 0.24000 / 0.29265 0.29061 / 0.3608232 0.76408 / 0.76408 0.76408 / 0.76408 0.76408 / 0.76408 0.76408 / 0.76408 0.76245 / 0.7640833 0.50531 / 0.46286 0.52245 / 0.51184 0.56000 / 0.59918 0.59673 / 0.62939 0.62531 / 0.6400034 0.21143 / 0.16571 0.22857 / 0.19510 0.25143 / 0.24571 0.18694 / 0.18694 0.19592 / 0.2375535 0.00408 / 0.37878 0.26122 / 0.40163 0.46531 / 0.56163 0.61633 / 0.64082 0.82367 / 0.8293936 0.34857 / 0.52163 0.54694 / 0.64000 0.71429 / 0.74204 0.77796 / 0.79347 0.79918 / 0.8228637 0.10776 / 0.13714 0.11837 / 0.13224 0.14204 / 0.11673 0.22122 / 0.16653 0.27837 / 0.2236738 0.11837 / 0.11265 0.12735 / 0.12735 0.15918 / 0.17633 0.21959 / 0.25306 0.21388 / 0.2751039 0.00000 / 0.00000 0.00408 / 0.01224 0.01796 / 0.05878 0.05878 / 0.06694 0.08327 / 0.2000040 0.02204 / 0.04408 0.05061 / 0.05714 0.07347 / 0.09061 0.09551 / 0.14531 0.11184 / 0.2538841 0.16980 / 0.13388 0.27592 / 0.26122 0.30939 / 0.26776 0.34776 / 0.29061 0.35755 / 0.3559242 0.28204 / 0.31755 0.44898 / 0.44816 0.52408 / 0.46204 0.55673 / 0.47102 0.58286 / 0.5110243 0.55102 / 0.45306 0.56163 / 0.46367 0.57959 / 0.48816 0.58776 / 0.50612 0.59918 / 0.5706144 0.30204 / 0.30041 0.40000 / 0.37878 0.45714 / 0.43755 0.48980 / 0.46367 0.52000 / 0.5061245 0.15673 / 0.15429 0.23673 / 0.21633 0.30531 / 0.24735 0.36327 / 0.26649 0.34041 / 0.3208246 0.22694 / 0.35429 0.47837 / 0.62939 0.61633 / 0.68735 0.67265 / 0.73796 0.70122 / 0.7257147 0.24408 / 0.20980 0.32898 / 0.34612 0.48571 / 0.56490 0.65878 / 0.78694 0.80816 / 0.8759248 0.10367 / 0.18776 0.17959 / 0.20980 0.19918 / 0.21061 0.21388 / 0.20816 0.18286 / 0.2440849 0.03592 / 0.02612 0.06286 / 0.05551 0.06367 / 0.04898 0.07837 / 0.06857 0.08571 / 0.0677650 0.04327 / 0.05143 0.08163 / 0.08735 0.09959 / 0.11673 0.07673 / 0.12000 0.06367 / 0.19592

26

Page 27: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 6: 1vs49 AUCp (FPR 5%) comparison of HMM without PPI and attack-first PPI(bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.05714 / 0.05796 0.08490 / 0.08653 0.16327 / 0.14939 0.21959 / 0.20490 0.22531 / 0.203272 0.13388 / 0.07265 0.44653 / 0.39592 0.72082 / 0.58612 0.75265 / 0.64000 0.69143 / 0.664493 0.36735 / 0.31347 0.50041 / 0.46939 0.59347 / 0.56163 0.73388 / 0.66776 0.81224 / 0.731434 0.30041 / 0.33714 0.32327 / 0.35837 0.35265 / 0.37633 0.38612 / 0.39265 0.40000 / 0.395105 0.12653 / 0.08082 0.19592 / 0.13878 0.14857 / 0.14857 0.12000 / 0.11429 0.08816 / 0.071846 0.59020 / 0.44735 0.66367 / 0.65469 0.74939 / 0.67510 0.71918 / 0.66694 0.76980 / 0.708577 0.01714 / 0.02694 0.08000 / 0.07020 0.12408 / 0.11592 0.16245 / 0.14939 0.17224 / 0.144498 0.68000 / 0.68163 0.68980 / 0.68490 0.71837 / 0.68653 0.74449 / 0.68898 0.78122 / 0.685719 0.00980 / 0.00908 0.04571 / 0.06694 0.06531 / 0.04816 0.10204 / 0.08816 0.10286 / 0.0775510 0.62612 / 0.62531 0.64816 / 0.63265 0.66286 / 0.66041 0.66612 / 0.66449 0.67265 / 0.6710211 0.09959 / 0.09061 0.18857 / 0.17714 0.20735 / 0.16653 0.25796 / 0.20408 0.23265 / 0.1600012 0.17959 / 0.16490 0.21224 / 0.18367 0.23102 / 0.17306 0.27184 / 0.18612 0.20980 / 0.1640813 0.14857 / 0.14776 0.19755 / 0.17551 0.22449 / 0.19755 0.25714 / 0.23673 0.20735 / 0.1918414 0.02204 / 0.00000 0.05633 / 0.04571 0.15184 / 0.05878 0.17796 / 0.17143 0.24000 / 0.2236715 0.02531 / 0.02857 0.04735 / 0.05633 0.08082 / 0.04653 0.09143 / 0.03510 0.05551 / 0.0114316 0.21469 / 0.07592 0.24408 / 0.15755 0.30531 / 0.20408 0.32327 / 0.25959 0.30449 / 0.2734717 0.07510 / 0.05143 0.18939 / 0.11673 0.25959 / 0.19510 0.32082 / 0.22367 0.41061 / 0.3061218 0.50857 / 0.50041 0.66041 / 0.64245 0.68816 / 0.67918 0.70204 / 0.69769 0.71918 / 0.7159219 0.13959 / 0.18939 0.15918 / 0.17306 0.17959 / 0.18857 0.17388 / 0.15429 0.11184 / 0.0987820 0.17714 / 0.16816 0.27837 / 0.21551 0.26041 / 0.21306 0.28735 / 0.23347 0.24816 / 0.2155121 0.47673 / 0.45061 0.61633 / 0.60653 0.82857 / 0.79020 0.86286 / 0.81714 0.87837 / 0.7942922 0.08490 / 0.10857 0.12490 / 0.10857 0.16816 / 0.12980 0.18122 / 0.10694 0.13061 / 0.0759223 0.03837 / 0.03429 0.19592 / 0.14694 0.23592 / 0.20816 0.29551 / 0.23837 0.34286 / 0.2318424 0.12408 / 0.11184 0.16082 / 0.15510 0.20735 / 0.18531 0.26694 / 0.26041 0.31184 / 0.2930625 0.04898 / 0.07429 0.14367 / 0.15102 0.20408 / 0.19673 0.31429 / 0.29306 0.37551 / 0.2938826 0.10449 / 0.16816 0.14694 / 0.22122 0.16490 / 0.18449 0.16163 / 0.14776 0.24571 / 0.1714327 0.01143 / 0.01633 0.01959 / 0.03265 0.06286 / 0.04327 0.15755 / 0.05878 0.25714 / 0.0677628 0.16735 / 0.16571 0.23347 / 0.24082 0.23510 / 0.22449 0.23918 / 0.27020 0.30531 / 0.2465329 0.06041 / 0.06449 0.11429 / 0.11755 0.31837 / 0.25633 0.40898 / 0.27918 0.48163 / 0.3412230 0.97959 / 0.97959 0.97959 / 0.97959 0.97959 / 0.97959 0.97959 / 0.97959 0.97959 / 0.9795931 0.11510 / 0.09061 0.19673 / 0.13878 0.19592 / 0.14857 0.24000 / 0.20735 0.29061 / 0.2236732 0.76408 / 0.76408 0.76408 / 0.76408 0.76408 / 0.76408 0.76408 / 0.76980 0.76245 / 0.7779633 0.50531 / 0.50449 0.52245 / 0.50776 0.56000 / 0.55102 0.59673 / 0.55184 0.62531 / 0.5510234 0.21143 / 0.18694 0.22857 / 0.21347 0.25143 / 0.20735 0.18694 / 0.13878 0.19592 / 0.1526535 0.00408 / 0.01959 0.26122 / 0.07510 0.46531 / 0.35429 0.61633 / 0.48000 0.82367 / 0.6008236 0.34857 / 0.30367 0.54694 / 0.49714 0.71429 / 0.65224 0.77796 / 0.71510 0.79918 / 0.7363337 0.10776 / 0.07510 0.11837 / 0.12163 0.14204 / 0.13714 0.22122 / 0.19184 0.27837 / 0.2171438 0.11837 / 0.12816 0.12735 / 0.13878 0.15918 / 0.14776 0.21959 / 0.15347 0.21388 / 0.0971439 0.00000 / 0.00000 0.00408 / 0.00898 0.01796 / 0.02041 0.05878 / 0.03020 0.08327 / 0.0383740 0.02204 / 0.01959 0.05061 / 0.03673 0.07347 / 0.09388 0.09551 / 0.11102 0.11184 / 0.0906141 0.16980 / 0.12327 0.27592 / 0.21061 0.30939 / 0.24163 0.34776 / 0.26939 0.35755 / 0.2857142 0.28204 / 0.13224 0.44898 / 0.30449 0.52408 / 0.45469 0.55673 / 0.50776 0.58286 / 0.5167343 0.55102 / 0.54776 0.56163 / 0.57224 0.57959 / 0.57061 0.58776 / 0.58939 0.59918 / 0.6000044 0.30204 / 0.73633 0.40000 / 0.73796 0.45714 / 0.73143 0.48980 / 0.72327 0.52000 / 0.7298045 0.15673 / 0.17959 0.23673 / 0.23510 0.30531 / 0.27347 0.36327 / 0.25633 0.34041 / 0.2146946 0.22694 / 0.22776 0.47837 / 0.45633 0.61633 / 0.60163 0.67265 / 0.65310 0.70122 / 0.6236747 0.24408 / 0.19592 0.32898 / 0.27429 0.48571 / 0.36653 0.65878 / 0.50531 0.80816 / 0.6269448 0.10367 / 0.11755 0.17959 / 0.19224 0.19918 / 0.21224 0.21388 / 0.20490 0.18286 / 0.1820449 0.03592 / 0.03918 0.06286 / 0.04245 0.06367 / 0.04653 0.07837 / 0.04245 0.08571 / 0.0506150 0.04327 / 0.06367 0.08163 / 0.09061 0.09959 / 0.10939 0.07673 / 0.08245 0.06367 / 0.06857

27

Page 28: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 7: 1vs49 AUCp (FPR 5%) comparison of OCNB without PPI and padding-first PPI(bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00000 0.00082 / 0.00082 0.00735 / 0.005712 0.07429 / 0.07837 0.08327 / 0.08327 0.09959 / 0.09714 0.12939 / 0.13551 0.17959 / 0.191843 0.00000 / 0.00000 0.00000 / 0.00000 0.00082 / 0.00000 0.00000 / 0.00000 0.00408 / 0.002454 0.07429 / 0.06816 0.07265 / 0.07510 0.07510 / 0.09388 0.10367 / 0.13714 0.13061 / 0.163275 0.00000 / 0.00000 0.00000 / 0.00408 0.00082 / 0.01388 0.01224 / 0.03265 0.01959 / 0.049806 0.00000 / 0.00163 0.00245 / 0.03429 0.01633 / 0.07429 0.02204 / 0.19496 0.02449 / 0.284087 0.00000 / 0.02122 0.00163 / 0.02367 0.00490 / 0.02694 0.01224 / 0.04653 0.02041 / 0.066128 0.00245 / 0.01633 0.00327 / 0.01878 0.01306 / 0.02776 0.01388 / 0.03102 0.03429 / 0.051849 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00000 0.00245 / 0.00163 0.00327 / 0.0024510 0.00000 / 0.00000 0.00082 / 0.00000 0.00898 / 0.00408 0.01388 / 0.00980 0.02694 / 0.0195911 0.00000 / 0.01388 0.00163 / 0.03673 0.00816 / 0.06041 0.01959 / 0.08245 0.02612 / 0.1069412 0.00000 / 0.00000 0.00163 / 0.00163 0.00245 / 0.00327 0.00980 / 0.00327 0.01551 / 0.0098013 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000014 0.06612 / 0.04735 0.07020 / 0.08816 0.07510 / 0.10694 0.07918 / 0.12082 0.09796 / 0.1673515 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00327 0.00490 / 0.0106116 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00245 / 0.00245 0.00653 / 0.0089817 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00163 0.00408 / 0.00327 0.01306 / 0.0098018 0.00000 / 0.00000 0.00082 / 0.00000 0.00735 / 0.00000 0.02367 / 0.00653 0.05143 / 0.0195919 0.00000 / 0.00000 0.00000 / 0.01796 0.00653 / 0.06612 0.00816 / 0.12653 0.01633 / 0.1804120 0.00000 / 0.00000 0.00000 / 0.00082 0.00000 / 0.00000 0.00163 / 0.00490 0.00735 / 0.0114321 0.31673 / 0.25061 0.34612 / 0.29714 0.42367 / 0.33551 0.49959 / 0.37551 0.61878 / 0.4310222 0.00408 / 0.00490 0.01959 / 0.02694 0.04082 / 0.06122 0.07673 / 0.10041 0.10286 / 0.1363323 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000024 0.00000 / 0.00245 0.00653 / 0.01224 0.01633 / 0.04245 0.02041 / 0.08082 0.03429 / 0.1461225 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00082 0.00000 / 0.00163 0.00245 / 0.0016326 0.00000 / 0.00082 0.00245 / 0.00490 0.00653 / 0.00653 0.01633 / 0.01714 0.01959 / 0.0187827 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.0000028 0.00000 / 0.00163 0.00000 / 0.00816 0.00980 / 0.02286 0.03673 / 0.05388 0.09959 / 0.0938829 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000030 0.13061 / 0.40000 0.24327 / 0.40000 0.31184 / 0.40000 0.35184 / 0.40000 0.36327 / 0.4000031 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00163 0.00408 / 0.00653 0.00490 / 0.0220432 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.0000033 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000034 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00163 0.02449 / 0.03184 0.05714 / 0.0653135 0.00000 / 0.00000 0.00163 / 0.00408 0.03020 / 0.02857 0.11184 / 0.11918 0.23673 / 0.2261236 0.00163 / 0.00000 0.01143 / 0.00735 0.01959 / 0.01633 0.02204 / 0.01714 0.03429 / 0.0310237 0.00000 / 0.00245 0.00735 / 0.02612 0.01143 / 0.02857 0.01714 / 0.04571 0.02694 / 0.0710238 0.00000 / 0.00082 0.00000 / 0.00490 0.00000 / 0.01551 0.00163 / 0.03592 0.01143 / 0.0604139 0.00000 / 0.02041 0.00000 / 0.01061 0.00000 / 0.01388 0.00245 / 0.01633 0.00408 / 0.0187840 0.00000 / 0.01061 0.00245 / 0.04653 0.00571 / 0.06367 0.01469 / 0.10122 0.04245 / 0.1306141 0.07429 / 0.06041 0.07102 / 0.06857 0.07510 / 0.06857 0.08816 / 0.07265 0.11020 / 0.0832742 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0008243 0.00000 / 0.00000 0.00000 / 0.00000 0.00735 / 0.00653 0.00980 / 0.00980 0.01306 / 0.0138844 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0016345 0.00082 / 0.00000 0.00980 / 0.02204 0.03510 / 0.05061 0.04898 / 0.06531 0.07837 / 0.1028646 0.00000 / 0.02857 0.01959 / 0.05551 0.03755 / 0.05796 0.04980 / 0.07102 0.07673 / 0.0906147 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00082 0.00327 / 0.00327 0.01224 / 0.0130648 0.00000 / 0.00000 0.00327 / 0.00327 0.00327 / 0.00408 0.01224 / 0.01143 0.02286 / 0.0171449 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00163 0.00327 / 0.0024550 0.00653 / 0.02041 0.02531 / 0.01796 0.04898 / 0.03673 0.07184 / 0.06041 0.09714 / 0.06531

28

Page 29: Analysis of Kullback-Leibler Divergence for Masquerade ... · A masquerader is an intruder who carries out some malicious activity and attempts to avoid detection by pretending to

Table 8: 1vs49 AUCp (FPR 5%) comparison of OCNB without PPI and attack-first PPI(bold face)

User |A| = 10 |A| = 20 |A| = 30 |A| = 40 |A| = 501 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00163 0.00082 / 0.00163 0.00735 / 0.008162 0.07429 / 0.04571 0.08327 / 0.05143 0.09959 / 0.05224 0.12939 / 0.05551 0.17959 / 0.099593 0.00000 / 0.00000 0.00000 / 0.00000 0.00082 / 0.00082 0.00000 / 0.00000 0.00408 / 0.000824 0.07429 / 0.01224 0.07265 / 0.01551 0.07510 / 0.03020 0.10367 / 0.04163 0.13061 / 0.055105 0.00000 / 0.00000 0.00000 / 0.00000 0.00082 / 0.00000 0.01224 / 0.00408 0.01959 / 0.006536 0.00000 / 0.00163 0.00245 / 0.01306 0.01633 / 0.02367 0.02204 / 0.03020 0.02449 / 0.032657 0.00000 / 0.00000 0.00163 / 0.00653 0.00490 / 0.01224 0.01224 / 0.00816 0.02041 / 0.013888 0.00245 / 0.00490 0.00327 / 0.00408 0.01306 / 0.00980 0.01388 / 0.01306 0.03429 / 0.022049 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00000 0.00245 / 0.00163 0.00327 / 0.0016310 0.00000 / 0.00000 0.00082 / 0.00000 0.00898 / 0.00816 0.01388 / 0.00327 0.02694 / 0.0106111 0.00000 / 0.00000 0.00163 / 0.00000 0.00816 / 0.00327 0.01959 / 0.00816 0.02612 / 0.0146912 0.00000 / 0.00000 0.00163 / 0.00000 0.00245 / 0.00163 0.00980 / 0.00082 0.01551 / 0.0057113 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00082 0.00000 / 0.0049014 0.06612 / 0.06041 0.07020 / 0.06857 0.07510 / 0.07429 0.07918 / 0.07102 0.09796 / 0.0800015 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00490 / 0.0000016 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00245 / 0.00327 0.00653 / 0.0032717 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00000 0.00408 / 0.00653 0.01306 / 0.0138818 0.00000 / 0.00000 0.00082 / 0.00082 0.00735 / 0.00327 0.02367 / 0.01143 0.05143 / 0.0383719 0.00000 / 0.00000 0.00000 / 0.00163 0.00653 / 0.01469 0.00816 / 0.02612 0.01633 / 0.0424520 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00163 0.00163 / 0.00163 0.00735 / 0.0106121 0.31673 / 0.11918 0.34612 / 0.09306 0.42367 / 0.10122 0.49959 / 0.10204 0.61878 / 0.1249022 0.00408 / 0.00082 0.01959 / 0.00082 0.04082 / 0.01143 0.07673 / 0.01551 0.10286 / 0.0163323 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000024 0.00000 / 0.00245 0.00653 / 0.00408 0.01633 / 0.01224 0.02041 / 0.02122 0.03429 / 0.0342925 0.00000 / 0.00082 0.00000 / 0.00245 0.00163 / 0.00327 0.00000 / 0.00653 0.00245 / 0.0146926 0.00000 / 0.02449 0.00245 / 0.04980 0.00653 / 0.06041 0.01633 / 0.07510 0.01959 / 0.0930627 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.0000028 0.00000 / 0.00000 0.00000 / 0.00000 0.00980 / 0.00163 0.03673 / 0.01714 0.09959 / 0.0367329 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000030 0.13061 / 0.00000 0.24327 / 0.00000 0.31184 / 0.00000 0.35184 / 0.00000 0.36327 / 0.0000031 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00408 / 0.00327 0.00490 / 0.0049032 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00163 0.00163 / 0.0008233 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.0000034 0.00000 / 0.00000 0.00000 / 0.00408 0.00000 / 0.01633 0.02449 / 0.03837 0.05714 / 0.0653135 0.00000 / 0.00163 0.00163 / 0.01633 0.03020 / 0.05061 0.11184 / 0.10367 0.23673 / 0.1616336 0.00163 / 0.03837 0.01143 / 0.03347 0.01959 / 0.03510 0.02204 / 0.02857 0.03429 / 0.0326537 0.00000 / 0.00000 0.00735 / 0.00490 0.01143 / 0.00490 0.01714 / 0.00735 0.02694 / 0.0114338 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00163 / 0.00163 0.01143 / 0.0081639 0.00000 / 0.00163 0.00000 / 0.00163 0.00000 / 0.00408 0.00245 / 0.00980 0.00408 / 0.0049040 0.00000 / 0.00000 0.00245 / 0.00163 0.00571 / 0.00408 0.01469 / 0.01551 0.04245 / 0.0212241 0.07429 / 0.07918 0.07102 / 0.07429 0.07510 / 0.08000 0.08816 / 0.07429 0.11020 / 0.0840842 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00082 0.00000 / 0.0008243 0.00000 / 0.00000 0.00000 / 0.00000 0.00735 / 0.00245 0.00980 / 0.00163 0.01306 / 0.0089844 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00163 0.00000 / 0.0008245 0.00082 / 0.00000 0.00980 / 0.00490 0.03510 / 0.01061 0.04898 / 0.02776 0.07837 / 0.0342946 0.00000 / 0.00000 0.01959 / 0.00980 0.03755 / 0.02531 0.04980 / 0.03429 0.07673 / 0.0481647 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00653 0.00327 / 0.01143 0.01224 / 0.0318448 0.00000 / 0.00000 0.00327 / 0.00082 0.00327 / 0.00753 0.01224 / 0.00898 0.02286 / 0.0130649 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00000 / 0.00000 0.00327 / 0.0000050 0.00653 / 0.00082 0.02531 / 0.01061 0.04898 / 0.01388 0.07184 / 0.01061 0.09714 / 0.02122

29