Clustering and the K-means algorithm - MIT...

24
Clustering and the K-means algorithm Yihui Saw 18.304 Seminar Talk I March 6, 2013 Saturday, March 16, 13

Transcript of Clustering and the K-means algorithm - MIT...

Page 1: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Clustering and the K-means algorithm

Yihui Saw18.304 Seminar Talk I

March 6, 2013

Saturday, March 16, 13

Page 2: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Clustering examples

• Customer purchase patterns

• Language family models

• Data compression

Saturday, March 16, 13

Page 3: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Original Image

Saturday, March 16, 13

Page 4: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

2 colors

Saturday, March 16, 13

Page 5: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

4 colors

Saturday, March 16, 13

Page 6: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

8 colors

Saturday, March 16, 13

Page 7: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

The clustering problem

Saturday, March 16, 13

Page 8: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Distance metric

Saturday, March 16, 13

Page 9: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

The cost of clustering

Saturday, March 16, 13

Page 10: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

K-means algorithm

Saturday, March 16, 13

Page 11: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

K-means algorithm

Saturday, March 16, 13

Page 12: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

K-means algorithm

Saturday, March 16, 13

Page 13: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

K-means algorithm

Saturday, March 16, 13

Page 14: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

K-means algorithm

Saturday, March 16, 13

Page 15: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

K-means algorithm

Saturday, March 16, 13

Page 16: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Proof of convergence• Each iterative step necessarily lowers the

cost - the cost monotonically decrease

Saturday, March 16, 13

Page 17: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Proof of convergence• Each iterative step necessarily lowers the

cost - the cost monotonically decrease

]

Saturday, March 16, 13

Page 18: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13

Page 19: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13

Page 20: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13

Page 21: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13

Page 22: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13

Page 23: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13

Page 24: Clustering and the K-means algorithm - MIT Mathematicsmath.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Yihui-Talk1.pdf · Clustering and the K-means algorithm Yihui Saw 18.304 Seminar

Convergence to local minimum

Saturday, March 16, 13


Clustering. 2 Outline Introduction K-means clustering Hierarchical clustering: COBWEB.

Clustering. 2 Outline Introduction K-means clustering Hierarchical clustering: COBWEB.

Passion For Fashion ZARA Presented By: Eva (Yihui Yan), Sylvia (Fei Gao), Emma(Hanul Yang)

Passion For Fashion ZARA Presented By: Eva (Yihui Yan), Sylvia (Fei Gao), Emma(Hanul Yang)

Clustering: Partition Clustering

Clustering: Partition Clustering

ASA Clustering within VMDC Architecture - Cisco€¦ · ASA Clustering within VMDC Architecture ASA Clustering Overview ASA Clustering Overview The clustering feature for the ASA

ASA Clustering within VMDC Architecture - Cisco€¦ · ASA Clustering within VMDC Architecture ASA Clustering Overview ASA Clustering Overview The clustering feature for the ASA

Affinity Clustering: Hierarchical Clustering at Scalepapers.nips.cc/paper/7262-affinity-clustering-hierarchical-clustering-at-scale.pdf · Affinity Clustering: Hierarchical Clustering

Affinity Clustering: Hierarchical Clustering at Scalepapers.nips.cc/paper/7262-affinity-clustering-hierarchical-clustering-at-scale.pdf · Affinity Clustering: Hierarchical Clustering

Channel Pruning and Other Methods for Compressing CNN · Channel Pruning and Other Methods for Compressing CNN Yihui He Xi’an Jiaotong Univ. October 12, 2017 Yihui He (Xi’an Jiaotong

Channel Pruning and Other Methods for Compressing CNN · Channel Pruning and Other Methods for Compressing CNN Yihui He Xi’an Jiaotong Univ. October 12, 2017 Yihui He (Xi’an Jiaotong

DATA MINING - CLUSTERING. Clustering 4 Clustering - unsupervised classification 4 Clustering - the process of grouping physical or abstract objects into.

DATA MINING - CLUSTERING. Clustering 4 Clustering - unsupervised classification 4 Clustering - the process of grouping physical or abstract objects into.

Climate prediction Ding Yihui, Liu Yanju, Liu Yiming, Shi Xueli National Climate Center, CMA.

Climate prediction Ding Yihui, Liu Yanju, Liu Yiming, Shi Xueli National Climate Center, CMA.

1 Microarray Clustering. 2 Outline Microarrays Hierarchical Clustering K-Means Clustering Corrupted Cliques Problem CAST Clustering Algorithm.

1 Microarray Clustering. 2 Outline Microarrays Hierarchical Clustering K-Means Clustering Corrupted Cliques Problem CAST Clustering Algorithm.

Segmentation and Clustering · – Divisive and hierarchical clustering – k-means clustering – Mean shift clustering – Graph cuts •Applications – Image processing, object

Segmentation and Clustering · – Divisive and hierarchical clustering – k-means clustering – Mean shift clustering – Graph cuts •Applications – Image processing, object

FUZZY CLUSTERING 2009/2010. 2 What is Data Clustering? Fuzzy C-Means Clustering Subtractive Clustering Data Clustering Using the Clustering GUI.

FUZZY CLUSTERING 2009/2010. 2 What is Data Clustering? Fuzzy C-Means Clustering Subtractive Clustering Data Clustering Using the Clustering GUI.

Clustering IV. Outline Impossibility theorem for clustering Density-based clustering and subspace clustering Bi-clustering or co-clustering.

Clustering IV. Outline Impossibility theorem for clustering Density-based clustering and subspace clustering Bi-clustering or co-clustering.

The Amazing Cephalopods! By: Matthew Dai and Yihui Liao.

The Amazing Cephalopods! By: Matthew Dai and Yihui Liao.

Yihui Optoelectronics Optical Measuring Instrument Catalog

Yihui Optoelectronics Optical Measuring Instrument Catalog

Bi-Clustering. 2 Data Mining: Clustering Where K-means clustering minimizes.

Bi-Clustering. 2 Data Mining: Clustering Where K-means clustering minimizes.

Boosting Singapore’s Productivity: New Evidence, …...Boosting Singapore’s Productivity: New Evidence, Innovative Approaches Catherine Jessica Yihui Lai Singapore’s productivity

Boosting Singapore’s Productivity: New Evidence, …...Boosting Singapore’s Productivity: New Evidence, Innovative Approaches Catherine Jessica Yihui Lai Singapore’s productivity

MICROWAVE SPECTRA OF DEUTERIUM ISOTOPOLOGUES OF cis-HEXATRIENE NORMAN C. CRAIG, VINCENT A. ALESSI, YIHUI…

MICROWAVE SPECTRA OF DEUTERIUM ISOTOPOLOGUES OF cis-HEXATRIENE NORMAN C. CRAIG, VINCENT A. ALESSI, YIHUI…

Cluster Ensembles Subspace Clustering Distributed Clustering

Cluster Ensembles Subspace Clustering Distributed Clustering

Minimum Spanning Trees - Massachusetts Institute of …math.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Varun-MST... · Definitions Common Algorithms Applications Kruskal’s Algorithm

Minimum Spanning Trees - Massachusetts Institute of …math.mit.edu/~rothvoss/18.304.1PM/Presentations/1-Varun-MST... · Definitions Common Algorithms Applications Kruskal’s Algorithm

All-Pairs Shortest Paths with Matrix Multiplicationmath.mit.edu/~rothvoss/18.304.1PM/Presentations/2-Chandler-slideslect2.pdf · All-Pairs Shortest Paths Problem To find the shortest

All-Pairs Shortest Paths with Matrix Multiplicationmath.mit.edu/~rothvoss/18.304.1PM/Presentations/2-Chandler-slideslect2.pdf · All-Pairs Shortest Paths Problem To find the shortest