A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang...
-
date post
20-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang...
![Page 1: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/1.jpg)
A Unified View of Kernel k-means, Spectral Clustering and
Graph Cuts
Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis
![Page 2: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/2.jpg)
Outline
• (Kernel) kmean, weighted kernel kmean
• Spectral clustering algorithms
• The connect of kernel kmean and spectral clustering algorithms
• The Uniformed Problem and the ways to solve the problem
• Experiment results
![Page 3: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/3.jpg)
K means and Kernel K means
![Page 4: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/4.jpg)
Weighted Kernel k means
Matrix Form
Distance from ai to cluster c
![Page 5: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/5.jpg)
Spectral Methods
• Represent the data by a graph– Each data points corresponds to a node on
the graph– The weight of the edge between two nodes
represent the similarity between the two corresponding data points
– The similarity can be a kernel function, such as the RBF kernel
• Use spectral theory to find the cut for the graph: Spectral Clustering
![Page 6: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/6.jpg)
Spectral Methods
![Page 7: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/7.jpg)
Spectral Methods
Similar in the cluster
Difference between clusters
![Page 8: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/8.jpg)
Represented with Matrix
( , ) Tc c c clinks V V x Ax ( , \ ) T
c c c clinks V V V x Lx
| | Tc c cV x x ( ) T
c c cdegree V x Dx
L for Ncut
Ratio assoc
Ratio cut
Norm assoc
![Page 9: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/9.jpg)
Weighted Graph CutWeighted association
Weighted cut
![Page 10: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/10.jpg)
Conclusion
• Spectral Methods are special case of Kernel K means
![Page 11: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/11.jpg)
Solve the unified problem
• A standard result in linear algebra states that if we relax the trace maximizations, such that Y is an arbitrary orthonormal matrix, then the optimal Y is of the form Vk Q, where Vk consists of the leading k eigenvectors of W1/2KW1/2 and Q is an arbitrary k × k orthogonal matrix.
• As these eigenvectors are not indicator vectors, we must then perform postprocessing on the eigenvectors to obtain a discrete clustering of the point
![Page 12: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/12.jpg)
From Eigen Vector to Cluster Indicator
Normalized U with L2 norm equal to 1
2
1
![Page 13: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/13.jpg)
The Other Way
• Using k means to solve the graph cut problem: (random start points+ EM, local optimal).
• To make sure k mean converge, the kernel matrix must be positive definite.
• This is not true for arbitrary kernel matrix
![Page 14: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/14.jpg)
The effect of the regularizationai is in c
cai is not in
![Page 15: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/15.jpg)
Experiment results
![Page 16: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/16.jpg)
Results (ratio association)
![Page 17: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/17.jpg)
Results (normalized association)
![Page 18: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/18.jpg)
Image Segmentation
![Page 19: A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d445503460f94a216eb/html5/thumbnails/19.jpg)
Thank you. Any Question?