1

Context-Aware Clustering

Junsong Yuan and Ying WuEECS Dept., Northwestern University

2

Contextual pattern and co-occurrences

?

Spatial contexts provide useful cues for clustering

3

K-means revisit

Assumption: data samples are independent

Binary label indicator

Limitation: contextual information of spatial dependency is not considered in clustering data samples

EM Update

4

Clustering higher-level patterns

• Regularized k-means

Distortion in original feature space

Distortion in hamming spacecharactering contextual patterns

Same as traditionalK-means clustering

Regularization term due tocontextual patterns

Not a smooth term!

5

Chicken and Egg Problem• Hamming distance in clustering contextual patterns

• Matrix form

• Cannot minimize J1 and J2 separately !

J1 is coupled with J2

6

Decoupling

Fix Update

Fix Update

7

Nested-EM solution

NestedE-step

M-stepUpdate and separately

the nested-EM algorithm can converge in finite steps.

Theorem of convergence

8

Simulation results (feature space)

9

Simulation results (spatial space)

10

11

K-m

eansInitialization

12

1st roundFinal Phrases

13

14

15

16

Multiple-feature clustering• Dataset: handwritten numerical (‘0’-‘9’) from UCI data set

– Each digit has three different types of features– Contextual pattern corresponds to compositional feature

• Different types of features serve as contexts of each other– Clustering each type of features into 10 “words”– Clustering 10 “phrases” based on a word-lexicon of size 3x10

17

Conclusion• A context-aware clustering formulation proposed

– Targets on higher-level compositional patterns in terms of co-occurrences

– Discovered contextual patterns can feed back to improve the primitive feature clustering

• An efficient nested-EM solution which is guaranteed to converge in finite steps

• Successful applications in image pattern discovery and multiple-feature clustering– Can be applied to other general clustering problems