Intelligent Database Systems Lab N.Y.U.S.T. I. M. Determining the best K for clustering...
-
Upload
anabel-skinner -
Category
Documents
-
view
218 -
download
0
Transcript of Intelligent Database Systems Lab N.Y.U.S.T. I. M. Determining the best K for clustering...
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
Determining the best K for clustering transactional datasets –
A coverage density-based approach
Presenter : Lin, Shu-Han
Authors : Hua Yan, Keke Chen, Ling Liu, Joonsoo Bae
Data & Knowledge Engineering (DKE) 68 (2009) 28–48
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
2
Outline
Motivation Objective Methodology Experiments Conclusion Comments
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Motivation
Cluster the transactional datasets – a kind of special categorical data
Time complexity: O(dmN2logN)
3
Name Buy
Jane Coke, Milk
Mary Coke, Pepsi
Tom Milk, Water
Denny Milk, Juice
TinaJuice, Red
Wine, Pepsi
Boolean values
Name Coke Milk Pepsi Water Juice Red Wine
Jane 1 1 0 0 0 0Mary 1 0 1 0 0 0Tom 0 1 0 1 0 0
Denny 0 1 0 0 1 0Tina 0 0 1 0 1 1
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Objectives
To design a method ACTD (Agglomerative Clustering algorithm with Transactional-cluster-modes Dissimilarity) especially for transactional data
Instead of ACE (Agglomerative Categorical clustering with Entropy criterion) Find best-K
More efficiently
4
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
ACE ACTD
Methodology – Overview of SCALE
5
(Sampling, Clustering structure Assessment, cLustering & domain-specfic Evaluation)
Agglomerative
BKPlot DMDI
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
Coverage Density
Transactional-cluster-mode A subset of items
Methodology – ACTDIntra-cluster similarity
6
9
7
33
7
Nk
Mk
1.c
2/3,b
2/3,a
.8,
in this case, only c is the transactional-cluster-mode
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
Transactional-cluster-mode dissimilarity
Time complexity: O(dmN2logN) O(MN2logN)
Methodology – ACTDInter-cluster similarity
7
032
33-1
5
2
10
6-1
52
33-1
2
1
12
6-1
62
33-1
[0, .5]
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – DMDI
8
Valleys、change dramatically
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments – Performance
9
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments – Quality
10
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments – Quality on sample dataset
11
With noise
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
12
Conclusions
The ACTD The Coverage Density-based method is promising for
transactional datasets Faster
More stable
than entropy-based method
The Agglomerative Hierarchical clustering algorithm and DMDI can help to find best-K
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
13
Comments
Advantage …
Drawback …
Application …