Post on 29-Jan-2016
CSSE463: Image Recognition CSSE463: Image Recognition Day 23Day 23 Midterm behind us…Midterm behind us… Foundations of Image Recognition completed!Foundations of Image Recognition completed!
This week: This week: K-means: a method of image segmentationK-means: a method of image segmentation Sunday Sunday night: literature review duenight: literature review due
Questions?Questions?
An image to segment…
SegmentationSegmentation The process of breaking an The process of breaking an
image into regions.image into regions. Two types: Two types:
General-purposeGeneral-purpose ““One size fits all”One size fits all” Very difficult…Very difficult…
SpecializedSpecialized Intended for a specific domain Intended for a specific domain
(say fruit-, circle- or skin-(say fruit-, circle- or skin-finding)finding)
Can be successfulCan be successful One to right is created using the One to right is created using the
mean-shift algorithm mean-shift algorithm D. Comaniciu, P. Meer: D. Comaniciu, P. Meer:
Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Machine IntellIEEE Trans. Pattern Anal. Machine Intell, , 2424, , 603-619, 2002.603-619, 2002.
EDISON code downloadable at EDISON code downloadable at
http://www.caip.rutgers.edu/riul/http://www.caip.rutgers.edu/riul/research/robust.htmlresearch/robust.html
What properties can we use to What properties can we use to segment?segment?
Regions homogeneous wrt. color, texture, Regions homogeneous wrt. color, texture, etc.etc.
Adjacent regions different (else merge)Adjacent regions different (else merge)Smooth boundariesSmooth boundaries
ApproachesApproaches
1.1. ModelsModels Uses an expected shape, color, etc. (fruit- and Uses an expected shape, color, etc. (fruit- and
circle-finders)circle-finders) Can use probabilitiesCan use probabilities
2.2. ClusteringClustering An An unsupervised unsupervised machine learning techniquemachine learning technique
No class labels used in learning!No class labels used in learning!
Groups pixels “close” to each other by some metric.Groups pixels “close” to each other by some metric. Color distance, texture, intensity, spatial location, etc.Color distance, texture, intensity, spatial location, etc.
Regions are then found using connected Regions are then found using connected componentscomponents
Q1
K-means clusteringK-means clustering Goal:Goal: given K given K
clusters, assign each clusters, assign each pixel to one of the pixel to one of the clusters such that the clusters such that the total total distance from distance from each pixel to the each pixel to the center of its cluster is center of its cluster is minimized.minimized.
We control C, the assignment We control C, the assignment of pixels to each cluster. (We of pixels to each cluster. (We will actually do this by will actually do this by specifying the location of their specifying the location of their means)means)
D= total distanceD= total distance K = # of clustersK = # of clusters x are pixelsx are pixels CCkk is the set of pixels in is the set of pixels in
cluster cluster kk mmkk is the center of cluster is the center of cluster kk ||.|| is a distance||.|| is a distance
Q2
K
k Cxki
Cki
mxD1
2min
K-means clusteringK-means clusteringProblems: Problems:
What’s K?What’s K? How do we know How do we know
which pixel belongs to which pixel belongs to each cluster?each cluster?
K-means is an K-means is an answer to the second answer to the second question.question.
K = # of clustersK = # of clusters x are pixelsx are pixels CCkk is the set of pixels in is the set of pixels in
cluster cluster kk mmkk is the center of cluster is the center of cluster kk ||.|| is a distance||.|| is a distance
K
k Cxki
Cki
mxD1
2min
K-means clusteringK-means clustering
Iterative process to group into Iterative process to group into kk clusters. clusters. Algorithm Algorithm (Sonka, p 403; Forsyth&Ponce, p. 315; Shapiro, p. 282)(Sonka, p 403; Forsyth&Ponce, p. 315; Shapiro, p. 282)
Initialize K cluster meansInitialize K cluster means Repeat until convergence:Repeat until convergence:
For each pixel, find the closest mean and assign it For each pixel, find the closest mean and assign it to that cluster to that cluster
Re-compute the mean of all pixels assigned to the Re-compute the mean of all pixels assigned to the clustercluster
Label each pixel with its current clusterLabel each pixel with its current cluster Example on board using 2D spatial distanceExample on board using 2D spatial distance
Q3-5
K-means clusteringK-means clustering We are trying to find out where the clusters are and We are trying to find out where the clusters are and
which points are assigned to each cluster. We which points are assigned to each cluster. We iteratively solve half the problem. Notice the overall iteratively solve half the problem. Notice the overall structure:structure:
Repeat until convergence:Repeat until convergence:
Assume you know where the cluster centers are. For each For each pixel, find the closest mean and assign it to that cluster pixel, find the closest mean and assign it to that cluster
Assume you know which points belong to each cluster. Recompute the mean of all pixels assigned to the clusterRecompute the mean of all pixels assigned to the cluster
Label each pixel with its current clusterLabel each pixel with its current cluster
K-means clusteringK-means clustering Pros:Pros:
Easy to implementEasy to implement Finds local optimum (best we can hope for)Finds local optimum (best we can hope for)
Cons:Cons: The number of clusters, The number of clusters, KK, must be known in advance, must be known in advance Some clusters might have 0 pointsSome clusters might have 0 points Local optimum is not guaranteed to be global optimumLocal optimum is not guaranteed to be global optimum
Ideas:Ideas: Can re-run with several initializationsCan re-run with several initializations Can choose Can choose KK based on observation or statistical means based on observation or statistical means
Q6
K-means clusteringK-means clustering
K = # of clustersK = # of clusters x are pixelsx are pixels CCkk is the set of pixels in is the set of pixels in
cluster cluster kk mmkk is the center of cluster is the center of cluster kk ||.|| is a distance: could be 2D ||.|| is a distance: could be 2D
distance in image or distance in image or 3D 3D Euclidean distance between Euclidean distance between colorscolors (or combination of both) (or combination of both)
(On Lab: will produce disconnected regions)
Q7-8
K
k Cxki
Cki
mxD1
2min
K-means resultsK-means results
K=3Original (120x160)
K=5 K=7