PowerPoint

Superpixel methodsImage SegmentationSpeaker: Hsuan-Yi Ko1OutlineIntroduction to superpixelsGraph-based superpixel methods Ncut ERSGradient-based superpixel methods SLIC WaterpixelConclusionReference

22Whats the superpixel?A cluster of connected pixels with similar features (ex: colorbrightnesstexture...).It can be regarded as a result of over segmentation.The concept was proposed in 2003 but the results of some former methods also can be called superpixels. (ex: watershedmean shift)Watershed [1]

TurboPixel [2]

33Desirable Properties of SuperpixelsGood adherence to object boundariesRegular shape and similar sizeCompute fast and simple to use

44Advantage of SuperpixelsRegional informationHigh computational efficiency

55Superpixel MethodsGraph-based: Superpixel LatticeEfficient Graph-based segmentationNcutERSGradient-based: WatershedMeanShiftQuick-ShiftTurboPixelSLIC

66Graph-Based MethodsNormalized Cut (Ncut) [5]Entropy Rate Superpixel (ERS) [4]77Graph RepresentationDenote an undirected graph as G= where V is the vertex set and E is the edge set.Wij: the weight on the edge which connects node i and node j In an undirected graph, the edge weights are symmetric, that is Wij = Wji.

8undirected graph Example for an undirected graph 8Graph-Based SegmentationConsider an image as an undirected graph and each edge is assigned with a non-negative weight Treat each pixel as a node in a graph Edge weights are related to the similarity between neighboring pixels.Various techniques are formed based on this assumption and graph cut.

9Consider an image as an undirected graph 9Min CutThe cost of the cut is the sum of the weights on cut edges.

Min cut is a method of minimizing the cost of the cut, but it favors cutting small sets of isolated nodes in the graph.

10[5] Shi, Jianbo, and Jitendra Malik. "Normalized cuts and image segmentation." [5] Shi, Jianbo, and Jitendra Malik. "Normalized cuts and image segmentation.\10NcutAvoid the unnatural biasMinimize Ncut to segment images

the total connection from nodes in A to all nodes in the graph

11[5] Shi, Jianbo, and Jitendra Malik. "Normalized cuts and image segmentation." 11Simulation ResultsRegular and compact shapeBad adherence to objectboundariesHigh computational cost especially for images with large size

12[6] Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods."The average superpixel size in the upper left of each image is 100 pixels and is 300 in the lower right.[6] Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods."12ERS Objective function: (1)H(A): entropy rate of a random walk on a graph compact and homogeneous clusters superpixels overlapping only a single object

(2) B(A): balancing term on the cluster distribution clusters with similar sizes

1313Random walks on graphsLet X = { Xt|t T, Xt V } be a random walk on the graph G = (V, E) and the entropy rate can be written as

14Xt node Xt t+1 node Xt+1 random walkentropy rateWi i Wt Wij ij ()Pij i j (ij) paperself loopPij 14

The role of entropy rate in obtaining compact and homogeneous clustering.15[4] Liu, Ming-Yu, et al. "Entropy rate superpixel segmentation."[4] Liu, Ming-Yu, et al. "Entropy rate superpixel segmentation."15Balancing TermNA is the number of connected componentsLet the graph partitioning for the edge set A be SA = {S1, S2, ..., SNA}. Then the distribution of ZA is equal to

16The entropy H(ZA) favors clusters with similar sizes; whereas NA favors fewer number of clusters.

16

The role of the balancing function in obtaining clusters of similar sizes.17[4] Liu, Ming-Yu, et al. "Entropy rate superpixel segmentation."clusterEntropy rateMaximize the objective function17Simulation ResultsIrregular shape but similar sizeGood adherence to object boundariesFast

18[4] Liu, Ming-Yu, et al. "Entropy rate superpixel segmentation."[4]18Gradient-Based SegmentationStarting from rough initial clusters and then iteratively refine the clusters by gradient until some convergence criterion is met.Simple linear iterative clustering (SLIC) [6]Waterpixel [1] [8]

1919SLICCIELAB color spaceSet initial seeds: distance S and low gradient in 3x3 window Local clustering by k-means in 5D space

(a) standard k-means searches the entire image. (b) SLIC searches a limit region.(a)(b)20[6] Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods."N pixels, K superpixelsSedge3x3 windowgradientlabelLABDpixelm2Sx2SK-means 20Simulation ResultsRegular and compact superpixelFast and simple

21[6] Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods."The average superpixel size in the upper left of each image is 100 pixels and is 300 in the lower right.The average superpixel size in the upper left of each image is 100 pixels and is 300 in the lower right21Waterpixel

22spatially regularized gradient [11] Machairas, Vaa, et al. "Waterpixels."Image Processing, IEEE Transactions on24.11 (2015): 3707-3716.22Waterpixel

(h) k=0(i) k=4 (j) k=10The choice of k is application dependent: when k equals zero, no regularization of the gradient is applied; when k , we approach the regular grid.

23[7] Machairas, V., Etienne Decencre, and Thomas Walter. "Waterpixels: Superpixels based on the watershed transformation."Waterpixelksuperpixel: k=0gradient k , regular grid.

[7] Machairas, V., Etienne Decencre, and Thomas Walter. "Waterpixels: Superpixels based on the watershed transformation."23Simulation ResultsFast computationRegular shape and similar sizeBad adherence to object boundaries

24[11] Machairas, Vaa, et al. "Waterpixels."Image Processing, IEEE Transactions on24.11 (2015): 3707-3716.WaterpixelWaterpixelWaterpixel superpixel SLICWaterpixel SLIC24ConclusionSuperpixel methods extract the meaningful regions in the image, and improve the computation based on pixels.Superpixel methods can be categorized into two types: graph-based methods and gradient-based methods.Different superpixel methods have different advantages and drawbacks. We should choose the proper method according to the problem.

2525Reference[1] Machairas, Vaa, Etienne Decencire, and Thomas Walter. "Spatial Repulsion Between Markers Improves Watershed Performance."Mathematical Morphology and Its Applications to Signal and Image Processing. Springer International Publishing, 2015. 194-202.[2] Levinshtein, Alex, et al. "Turbopixels: Fast superpixels using geometric flows."Pattern Analysis and Machine Intelligence, IEEE Transactions on31.12 (2009): 2290-2297.[3] Yi, Faliu, and Inkyu Moon. "Image segmentation: A survey of graph-cut methods."Systems and Informatics (ICSAI), 2012 International Conference on. IEEE, 2012.[4] Liu, Ming-Yu, et al. "Entropy rate superpixel segmentation."Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011.[5] Shi, Jianbo, and Jitendra Malik. "Normalized cuts and image segmentation."Pattern Analysis and Machine Intelligence, IEEE Transactions on22.8 (2000): 888-905.[6] Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods."Pattern Analysis and Machine Intelligence, IEEE Transactions on34.11 (2012): 2274-2282.2626[7] Machairas, V., Etienne Decencre, and Thomas Walter. "Waterpixels: Superpixels based on the watershed transformation."Image Processing (ICIP), 2014 IEEE International Conference on. IEEE, 2014.[8] http://www.lunwen365.com/qitaleibie/lunwenzhidao/fanli/563192.html[9] http://m.blog.csdn.net/blog/Guzenyel/25769507[10] Peng, Bo, Lei Zhang, and David Zhang. "A survey of graph theoretical approaches to image segmentation."Pattern Recognition46.3 (2013): 1020-1038.[11] Machairas, Vaa, et al. "Waterpixels."Image Processing, IEEE Transactions on24.11 (2015): 3707-3716.

27