Knowledge Discovery from Web Usage Data: Research andDevelopment of Web Access Pattern Tree BasedSequential Pattern Mining Techniques: A Survey

G. Shivaprasad, N. V. Subbareddy and U. Dinesh AcharyaDepartment of Computer Science and Engineering,

Manipal Institute of Technology, Manipal University, Manipal, India

Abstract— Sequential pattern mining is the process of applying data mining techniques to a sequential database, to extractfrequent subsequences to discover correlation that exists among the ordered list of events. Web Usage mining (WUM) discoversand extracts interesting knowledge/patterns from Web logs is one of the applications of Sequential Pattern Mining. In this paper,we present a survey of the sequential pattern mining techniques based Web Access Pattern (WAP) tree on Web logs.

I. INTRODUCTION

Sequential pattern mining, which finds high-frequencypatterns with respect to time or other patterns, was firstintroduced by [3] as follows: given a sequence databasewhere each sequence is a list of transactions ordered bytransaction time and each transaction consists of a setof items, find all sequential patterns with a userspecified minimum support, where the support is thenumber of data sequences that contain the pattern.Since the access patterns from web log take on obvioustime sequence characteristic, it is natural to apply thetechnology of sequential pattern mining to web mining.Sequential pattern mining [4] discovers the existingfrequent sequences in a given database, when the datato be mined has some sequential nature. These patternsmay represent very valuable information. A sequentialpattern (as applied to Web usage mining) is defined asan ordered set of pages that satisfies a given supportand is maximal (i.e., it has no subsequence that is alsofrequent). Support is the percentage of the customerswho have the pattern. Since a user may have manysessions, it is possible that a sequential pattern shouldspan a lot of sessions. It also need not be contiguouslyaccessed pages.

There exists a great diversity of algorithms forsequential pattern mining. Sequential access patternmining techniques are mainly based on two approaches:Apriori-based mining algorithms and WAP tree basedmining algorithms. Most of the earlier algorithms forsequential pattern mining are based on the Aprioriproperty proposed in association rule mining [3]. Theapriori-like algorithms are not efficient when applied tolong sequential patterns, which is an importantdrawback when working with Web logs. In [5], Pei et alproposed another category of algorithm based on Webaccess pattern (WAP) tree. Several improvements ofWAP-tree algorithm, the Pre-order Linked WAP treeand Layer Coded Breadth-First WAP-tree were thenproposed. In the next section, we present a briefdiscussion of these algorithms.

In order to make it easier for us to compare thosealgorithms, we use the same sequential database, a weblog (shown in Table I). Suppose the minimum supportthreshold is set at 75%, which means an accesssequence, s should have a count of 3 out of 4 records inour example, to be considered frequent. The frequentsub-sequences are shown in the 3rd column of Table Iare used as input.

II. WAP TREE BASED MINING ALGORITHMS

A. WAP-Mine

The WAP-mine algorithm[5] based on WAP-tree isspecifically tailored for mining Web access patterns.The algorithm scans the access sequence databasetwice. The set of frequent events are determined in thefirst scan. Using these frequent events, a WAP-treewith count information is built in the next scan. TheWAP-tree stores the web log data in a prefix treeformat. Then WAP-mine recursively mine the WAP-tree using conditional search to find all Web accesspatterns. Conditional search narrows the search spaceby looking for patterns with the same suffix and countfrequent events in the set of prefixes with respect tocondition as suffix. The process of recursive mining ofa conditional suffix WAP tree ends when it has onlyone branch or is empty.

The WAP-tree algorithm first stores the frequentitems as header nodes for linking all the nodes of theirtype in the WAP-tree in the order the nodes areinserted.

Initially, a virtual root (Root) is first inserted in theWAP-tree. Then, for each frequent sequence in thetransaction, a branch from the Root to a leaf node of the

TABLE I SEQUENTIAL DATABASE ADAPTED FROM [5]

319

Downloaded 04 Jan 2011 to 59.145.105.130. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/about/rights_permissions

tree is constructed. Each event in a sequence is insertedas a node with count 1 from Root if that node type doesnot yet exist. If the node type already exists, then thecount of the node is increased by 1. Also, the head linkfor the inserted event is connected (in broken lines) tothe newly inserted node from the last node of its typethat was inserted or from the header node of its type ifit is the very first node of that event type inserted. Fig.1shows the complete WAP-tree for the sequential data ofTable I.

The basic structure of mining all Web access patternsin WAP-tree is as follows: If the WAP-tree has only onebranch, return all (ordered) combinations events in thebranch; Otherwise, for each frequent event ei in theWAP-tree by following the ei-queue started from headertable, an ei-conditional access sequence base isconstructed, denoted as PS|ei, which contains all andonly all prefix sequences of ei . Each prefix sequence inPS|ei carries its count from the WAP-tree. For eachprefix sequence of ei with count c, when it is inserted into PS| ei, all of its sub-prefix sequences of ei are insertedin to PS|ei with count –c, to prevent it contributingtwice. It is easy to show that by accumulating counts ofprefix sequences, a prefix sequence in PS| ei holds itsunsubsumed count. Then, the complete set of Webaccess patterns which are prefix sequence of ei can bemined by concatenating ei to all Web access patternsreturned from mining the conditional WAP-tree, and eiitself.

For the WAP-tree of Fig. 1, first the prefix sequenceof the base c (the lowest frequent event) or theconditional sequence base of c is computed as follows:

aba:2; ab:1; abca:1; ab:−1; baba:1; abac:1; aba:−1.The conditional sequence list of a suffix event isobtained by following the header link of the event andreading the path from the root to each node (excludingthe node). The count for each conditional base path is

the same as the count on the suffix node itself. The firstsequence in the list above, aba represents the path to thefirst c node in the WAP tree. When a conditionalsequence in a branch of a WAP-tree, has a prefix

subsequence that is also a conditional sequence of anode of the same base, the count of this newsubsequence is subtracted because it has contributedbefore. Thus, the conditional sequence list above hastwo sequences ab and aba with counts of −1. This isbecause, when the subsequence, abca is added to thelist, its subsequence ab was already in the list, from thefirst c on the same branch. Thus, the count of ab with−1 has to be added to prevent it from contributingtwice. To qualify as a conditional frequent event, oneevent must have a count of 3. Therefore, after countingthe events in sequences above, the conditional frequentevents are a(4) and b(4) and c with a count of 2, whichis less than the minimum support, is discarded. So, theconditional sequences based on c are:

aba:2; ab:1; aba:1; ab:−1; baba:1; aba:1; aba:−1Using these conditional sequences, a conditional WAPtree, WAP-tree|c, is built using the same method asshown in Fig. 1. The new conditional WAP-tree isshown in Fig. 2(a). Recursively, based on this WAP-tree, the next conditional sequence base for the next

suffix subsequence, bc is computed as a(3),

∅, ba(1).

Since a is the only frequent pattern in this base, a(4) isused to construct the next WAP tree for the frequentsequence base of bc as shown in Fig. 2(b). Now the re-construction of WAP trees that with suffix sequences |cand |bc is completed and the frequent patterns foundalong this line are c, bc and abc. Next, the recursioncontinues with the suffix path |c, |ac, |bac, |aac. Now,the conditional search of c is completed. The search forfrequent patterns that have the suffix of other headerfrequent events are also mined in the same way. Aftermining the whole tree, the discovered frequent patternset is: {c, aac, bac, abac, ac, abc, bc, b, ab, a, aa, ba,aba}.

Figure 1. WAP-tree for the Frequent Sequence in Table I Conditional WAP-tree|c Conditional WAP-tree|bc

(a) (b)

Figure 2. Conditional WAP-tree for Conditional Sequence Base c

320

Downloaded 04 Jan 2011 to 59.145.105.130. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/about/rights_permissions

WAP-tree algorithm scans the original database onlytwice and avoids the problem of generating explosivecandidate sets as in Apriori-like algorithms. So, theMining efficiency is improved sharply. The maindrawback of WAP-tree algorithm is that it requiresrecursive construction of large numbers of intermediateWAP-trees during mining. This necessitates storingintermediate patterns and hence consumes much timeand space.

B. Pre-order Linked WAP (PLWAP)

Pre-Order linked WAP tree algorithm [6] assignsunique binary position code to each node of the WAPtree. The header nodes are linked in pre-order fashion.Both the pre-order linkage and binary position codesavoids recursive re-construction of the intermediateWAP trees.

The position code for each node are assigned asfollows: the root has null code, and the leftmost childof any parent node has a code that appends ‘1’ to theposition code of its parent, while the position code ofany other node has ‘0’ appended to the position code ofits nearest left sibling. In general, for the nth leftmostchild, the position code is obtained by appending thebinary number 2n−1 to the parent’s code.

In PLWAP, first the common prefix sequences arefound. The main idea is to find a frequent pattern byprogressively finding its common frequentsubsequences starting with the first frequent event in afrequent pattern. For example, if pqrs is a frequentpattern to be discovered, the PLWAP technique, willfind the prefix event p first, then, using the suffix treesof node p, it will find the next prefix subsequence pqand continuing with the suffix tree of q, it will find thenext prefix subsequence pqr and finally, pqrs. Thebinary position codes are introduced for identifying theposition of every node in the WAP tree.

The PLWAP algorithm builds a prefix tree, PLWAPtree, by inserting the frequent sequence of eachtransaction in the tree in the same way as in WAP-Mine. The position codes are set by applying the above

mentioned position coding rule. Once the frequentsequence of the last database transaction is inserted inthe PLWAP-tree, the tree is traversed in pre-orderfashion to build the frequent header node linkages.Event-node queue links all the nodes in the tree withthe same label. The event-node queue with label ei isalso called ei-queue. There is one header table L for aPLWAP-tree, and the head of each event-node queue isregistered. Fig. 3 shows the completely constructedPLWAP tree with the pre-order linkages for thesequence database of Table I.

The algorithm starts by mining the tree in Fig. 3 forthe first element in the header linkage list, a followingthe a-link to find the first occurrences of a nodes ina:3:1 and a:1:101 of the suffix trees of the Root sincethis is the first time the whole tree is passed for mininga frequent 1-sequence. Now the list of mined frequentpatterns F is {a} since the count of event a in thiscurrent suffix trees is 4 (sum of a:3:1 and a:1:101counts), and more than the minimum support. Themining of frequent 2-sequences that start with event awould continue with the next suffix trees of a rooted at{b:3:11, b:1:1011} shown in Fig. 4 as un-shadowednodes. The objective here is to find if 2-sequences aa,ab and ac are frequent using these suffix trees. In orderto confirm aa frequent, we need to confirm event afrequent in the current suffix tree set, and similarly, toconfirm ab frequent, we should again follow the b linkto confirm event b frequent using this suffix tree set,same for ac. The process continues to obtain thefrequent sequence sets:

{a, aa, aac, ab, aba, abac, abc, ac, b, ba, bac, bc, c}The PLWAP algorithm eliminates the need to store

numerous intermediate WAP trees during mining,which drastically cuts off huge memory access costs.The pre-order linking of header nodes makes the searchprocess more efficient. Thus, it saves the processingtime and more so when the number of frequent patternsincreases and the minimum support threshold is low.

Figure 4. a| Suffix treeFigure 3. PLWAP Tree adapted from [6]

321

Downloaded 04 Jan 2011 to 59.145.105.130. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/about/rights_permissions

PLWAP’s performance seems to degrade with very longsequences having sequence length more than 20 becauseof the increase in the size of position code that it needsto build and process for very deep PLWAP tree

C. Layer Coded Breadth-First WAP-tree

The Layer Coded Breadth First WAP-tree algorithm [7]builds the frequent header node links of the originalWAP-tree in a Breadth-First fashion and uses the layercode of each node to identify the ancestor-descendantrelationships between nodes of the tree. It then, findseach frequent sequential pattern, through progressiveBreadth-First sequence search, starting with its firstBreadth-First subsequence event.

Suppose that any ancestor node in WAP tree has notmore than n (n<100) children node. Given a WAP-treewith some nodes, the layer code for each node isassigned as follows: the root has a layer code of 0 andthe leftmost child of the root has a code of 001, thesecond leftmost node has a code of 002, and so on. Sothe layer code of a node in WAP tree is its parent nodelayer code plus its position (from left to right) 01-99.Also, a node α is an ancestor of another node β if andonly if the layer code of α equals the leftmost n bitslayer code of β, n is the length of layer code a.

The Pre-Order event linkage [6] can’t reflectancestor-descendant relationship directly and clearly. InFig. 4, by following the linkage started in head table,the chain of event a is {a:3:1, a:2:111, a:1:11101,a:1:101, a:1:10111}. In this chain, a:2:111 appearsbefore a:1:101, but a:1:101 is the second level of rootand a:2:111 is the third level of root. So in order to findall first descendant of root of event a, we must scanentire chain. Hence, this algorithm employs theBreadth-First traversal mechanism. Breadth-Firsttraversal mechanism makes use of a Queue into whichthe nodes of the tree are inserted. First all child nodesof root enter the queue from left to right. Then, as perthe FIFO (first in first out) rule, the first element of thequeue is accessed. After adding it to the correspondingevent queue, all of its children nodes are inserted intothe queue and then it is deleted from the queue. Theremaining elements of the queue are dealt in the samemanner. Fig.5 shows the completely constructedBFWAP tree for the sequence database of Table I.

The algorithm starts mining the tree in Fig. 5 forfinding frequent 1-sequence by following the a link tofind the first occurrences of a nodes in a:3:001 anda:1:00201 of the suffix trees of the Root. Since thecount of event a, in this current suffix trees is 4 (sum ofa:3:001 and a:1:00201 counts), and more than theminimum support {a} is listed as frequent pattern.Next, we need to confirm whether the 2-sequences aa,ab and ac are frequent using the suffix trees rooted at ai.e at { b:3:00101, b:1:0020101}. For example, toconfirm aa is frequent, we need to confirm that event ais frequent in the current suffix tree set. The layer codes

of the nodes are used to identify which nodes of eachsuffix tree are descendants of the first event node ei inthe same subtree. To mine all the frequent events in thesuffix trees of a:3:001 and a:1:201 , which are rooted atb:3: 00101 and b:1: 0020101 respectively, we find thefirst occurrence of 'a' on each suffix tree, asa:2:0010101, a: 1:001010201 and a:1:002010101. Thetotal count of a is 4 making a, the next frequent eventin sequence. Thus, a is appended to the last list offrequent sequence 'a', forming the new frequentsequence 'aa'. This way, we continue to mine otherfrequent events in the suffix trees. Backtracking in theorder of previous conditional suffix BFWAP-treemined, we search for other frequent events the same asabove. Finally, we have the following frequentsequence set: { a, aa, aac, ab, aba, abac, abc, ac, b, ba, bac, bc, c}.

The BFWAP algorithm makes use of the WAP-treestructure for storing frequent sequential patterns to bemined. However, to improve the mining efficiency, thealgorithm finds layered patterns instead of the prefixpatterns as done by PLWAP. The Breadth-First linkageprovides a way to traverse the event queue withoutgoing backwards. The layer codes are used to identifythe level of nodes in the BFWAP tree. With these twomethods, the next frequent event in each suffix tree isfound without traversing the whole WAP-tree. Thus, itavoids re-constructing WAP-tree recursively. It hasbeen experimented in [7], that mining web log usingBFWAP algorithm is much more efficient than withWAP-tree and GSP algorithms, especially when theaverage frequent sequence becomes longer and theoriginal database becomes larger.

Figure 5. BFWAP-tree adapted from [7]

322

Downloaded 04 Jan 2011 to 59.145.105.130. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/about/rights_permissions

III. CONCLUSIONS

Sequential Pattern Mining techniques based on Apriorirequire multiple scans of the sequence database andgenerate huge number of candidate sets for long webaccess sequences. This problem makes these algorithmsineffective and inefficient in dealing with largesequence sets with long sequences, especially with weblogs. Experimental results on some limited data setscarried out by researchers indicated that the WAP-treealgorithm is ”an order of magnitude faster thanconventional Apriori methods”. Several modificationswere then proposed in order to further improve thespeed of the WAP-tree algorithm.

This paper presents a study of different sequentialpattern mining algorithms based on web access patterntree. It is found that the efficiency and effectiveness ofthese algorithms will deteriorate rapidly when appliedto larger log files with long clicking streams. Thisweakness will become even more serious as thenumbers of web pages and users still keep increasingquickly. Therefore, there is a real need for improvingthe current web usage mining algorithms or developingnew efficient ones.

REFERENCES

[1] R. Cooley, B. Mobasher, and J. Srivastava, Data Preparation forMining World Wide Web Browsing Patterns, In Journal ofKnowledge and Information Systems, Vol. 1,Issue 1, pp. 5-32,1999.

[2] J. Han and M. Kamber, Data Mining Concepts and Techniques,Morgan Kaufmann, 2000.

[3] R. Agrawal and R. Srikant, Fast Algorithms for MiningAssociation Rules, 20th International Conference on Very LargeDatabases (VLDB’94), pp. 487-499, 1994.

[4] R. Agrawal and R. Srikant, Mining sequential patterns, In 11th

International Conference of Data Engineering (ICDE’95), pp. 3-14, 1995.

[5] J. Pei, J. Han, B. Mortazavi-Asl, and H. Zhu., Mining accesspatterns efficiently from web logs, In Proceedings of thePacific-Asia Conference on Knowledge Discovery and DataMining, 2000.

[6] C. Ezeife & Y. Lu., Mining web log sequential patterns withposition coded pre-order linked wap-tree, International Journalof Data Mining and Knowledge Discovery (DMKD), SpringerScience + Business Media, Vol. 10, pp.5–38, 2005.

[7] Liu, L., & Liu, J., Mining maximal sequential patterns withlayer coded Breadth-First linked WAP-tree., Asia-PacificConference on Computational Intelligence and IndustrialApplications (PACIIA), IEEE, pp. 61-65, 2009.

323

Downloaded 04 Jan 2011 to 59.145.105.130. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/about/rights_permissions