Unsupervised Segmentation in 3D Planar Object Maps Based on Fuzzy Clustering

Xin Liu, S. Y. Cheng, X. W. Zhang, X. R. Yang

School of Electromechanical Engineering Guangdong University of Technology

Guangzhou, China e-mail: liuxin@gdut.edu.cn

Thach Ba Nguyen, Sukhan Lee

School of Information and Communication Engineering Sungkyunkwan University

Suwon, South Korea e-mail: thach@skku.edu

Abstract—This paper investigates the problem of acquiring planar object maps of indoor household environments in particular kitchens. The objects modeled in these maps include tables, walls and ceilings. Our segmentation approach is based on 3D point cloud data representations. In order to solve the segmentation problem in complicated environment, a variable model is used in this paper. It is applied in 3D planar segmentation from point clouds. The segmentation algorithm was developed based on fuzzy K-means, which can automatically find the optimal number of clusters and self- organize the clusters based on Inter cluster Distance Index and Sample Distribution Index. After the 3D point clouds of each planar are clustered, we get the 3D planar candidates by extracting semantic information from the clusters based on functional reasoning module. The model has been evaluated on the real test scenes, which contain noisy point clouds. Empirical results show that our model can rationally interpret planar objects from the point clouds and shows the high performance and robust results.

Keywords-3D planar; segmenataion; fuzzy k-means; self-organizing

I. INTRODUCTION The paper proposes a new approach to acquire an

unsupervised planar segmentation of 3D point clouds based on self-organizing fuzzy K-means algorithm. After that we can acquire more detailed representation of an environment object model, more generally of human living environments. The acquired environment model is to serve as a resource for robotic assistants that are to perform household chores including setting the table, cleaning up, preparing meals, etc. Raw laser scanner point clouds require a great deal of processing before final object models can be derived. The capability of carrying out useful analyses is rather limited with unstructured 3D points. Thus, segmentation becomes an essential step whenever grouping of points with common attributes is required.

Clustering, and more specifically fuzzy clustering methods, are powerful tools to detect clusters in feature spaces. Since all the points lying on the same surface have similar features, i.e. curvature, they create a dense region in a feature space. Thus, clustering methods are a useful tool to extract surfaces from point clouds.

The paper reviews fuzzy clustering algorithms and show their ability to perform a robust segmentation for laser scanner point clouds. The rest of the paper is organized as

follows: Section II presents related work, followed by a description of our system overview, architecture in Section III. Section IV explains the self-organizing fuzzy K-means algorithm method used to categorize point clouds. Section V explains one of the technicalities of the two main modules, referred to as Functional Reasoning which extracts semantic information from clustering results, such as extracting the general categories of objects (tables, floor, and ceiling). We briefly reiterate our empirical results in Section VI and conclude with Section VII.

II. RELATED WORK The clustering applications in various areas have been

well documented [1] [2]. In these clustering methods, the hard c-means (or k-means) and fuzzy c-means (FCM) clustering algorithms are the most well-known methods, The FCM was proposed by Dunn and generalized by Bezdek [3].

The FCM has the following features: (1) the algorithm produces c vectors in the space where the data points lie. The c vectors are called cluster centers. (2) The algorithm produces c membership values for each data point which is a real number between zero and one that shows the similarity of the data point to each cluster center. (3) The algorithm partitions data points into c clusters according to the membership values. (4) FCM generates fuzzy clusters in an iterative fashion.

However, as Gath and Geva pointed out [4], there are three major difficulties in fuzzy clustering: (1) determining the optimal number of clusters to be created (most algorithms require the user to specify the number of clusters); (2) choosing the initial cluster centroid, The character and location of cluster centroid is not necessarily known a priori, and initial guesses have to be made; (3) handling data characterized by large variability in cluster shape, cluster density, and the number of points in different clusters.

The first of these difficulties is the number of clusters. Since most basic clustering algorithms assume that the number of clusters in a data set is a user-defined parameter (one that is difficult to set in practical applications), the common approach is an iterative trial-and-error process. The trial-and-error process performs the model selection according to the terms used by Jain [5]. We are interested in the problem of cluster validation in the context of partitioning-based clustering algorithms. In other words, the clustering algorithm is run with different initial values for the

2012 Eighth International Conference on Computational Intelligence and Security

978-0-7695-4896-8/12 $26.00 © 2012 IEEE

DOI 10.1109/CIS.2012.88

364

number of clusters and the results are compared in order to determine the most appropriate number of clusters. FCM is one of the most widely used clustering algorithms. Several validity indices have been proposed in the literatures [6][7] for use with the FCM clustering algorithm.

The paper proposes a new algorithm for clustering while automatically determining the number of clusters. The new algorithm improves the conventional model selection process by combining the splitting and merging strategies. Splitting strategies have been designed and combined with the basic clustering algorithm so that for each candidate number of clusters, the clustering process can be carried out starting with the previously obtained clusters. Merging strategies have been designed for solving the problem of controlling the merging of nearby clusters, which means that we can reduce the number of clusters in case of the inter-clusters distance meets the predefined criterion.

In the initialization of the iterative calculation, the initial cluster centroid must be specified. However, the precise locations of the initial centroids are not always known in advance. This can create problems in that the final fuzzy clusters may vary depending on the starting points selected. There is no general agreement regarding which initialization scheme gives the best results. There are three most popular initialization techniques as follows: (1) using c data points randomly selected from the set; (2) using the �rst c distinct data points in the set and (3) using c data points uniformly distributed across the data set. Our algorithms choose a random selection because such a selection is sure to converge by the iterative process.

Because the clustering results obtained using K-means and FCM are roughly spherical with similar volumes, many clustering algorithms such as the Gustafson–Kessel algorithm [8], the unsupervised fuzzy partition-optimal number of classes algorithm [4], etc. were proposed to accommodate elliptical clusters with different volumes. Our algorithm optimizes the results by analyzing two indices in a iterated procedure, since the used distance is based on Mahalanobis distance, we compute the mutual distance between two clusters with the difference co-variance of clusters and it shows the optimal results of merging nearby clusters.

An object based approach for cognitive maps is used to recognize objects and classify rooms in different categories in [9] [10]. The work presented in [11] provides a method for classifying different places in the environment into semantic classes like kitchens, corridors, rooms using simple geometric features extracted from laser data and information extracted from camera data.

III. SYSTEM OVERVIEW Given the scenario laid out in the introduction we are

investigating the following computational problem: from a set of point clouds, how to automatically compute for applications requiring the labeling of point clouds, surface extraction and classification into homogeneous areas. The computational process that solves this problem is organized as follows:

Step one: Segmentation algorithms can be classified based on surface growing algorithms or clustering algorithms. This paper presents an unsupervised robust clustering approach using fuzzy methods. As part of geometrical reasoning module, self-organizing fuzzy K-means algorithm is performed to get the classification of data points into surface categories. With the correct clustering on point clouds, we can archive a high recognition of 3D planes;

Step two: classification of points lying on planar features into different planes. The second module, functional reasoning module, extract semantic information from the environment map, based on 3D geometry and a set of assumptions about the world, by employing a set of algorithms, the following general categories of objects need to be exacted: (1) tables – as planar horizontal surfaces located roughly at hip height; (2) floors and ceilings – as planar horizontal surfaces located roughly at the lowest height and the highest height.

IV. SELF-ORGANIZING FUZZY K-MEANS ALGORITHM The general purpose of clustering analysis on 3D point

clouds is to organize data into right solution, which is based on their shape and compactness. The right solution means that the clustering results have the right number of clusters and best fits the data. The self-organizing fuzzy K-means algorithm is composed of the general model selection algorithm for automatically determining the number of clusters in data set.

A. Measurement of distance, membership matrix and cluster centers The algorithm is based on the following objective

function:

2

1 1Minimize ( , ) ( ) ( , )

n cm

m ki k ik i

J U V u d X V= =

=�� (1)

Where n is the total number of data vectors in a given data set, c is the number of clusters; X={X1, X1, … , Xn} ∈ RS and V={V1, V1, … , Vn} ∈ RS are the feature data and cluster centers; and U=(uji)n×c is a fuzzy partition matrix composed of the membership of each feature vector Xk in each cluster i.

uji should satisfy 1

1c

jii

u=

=� for 1, 2,j n= …, and 0jiu ≥ for

all 1, 2,i c= …, and 1,2,j n= …, . The exponent m is a parameter, usually called a fuzzifier.

To minimize U, the cluster centers Vi and the membership matrix U need to be computed according to the iterative formula below:

1 ( 1)2

1 ( 1)2

1

1 ( , )

1 ( , )

m

j iij K m

j kk

d X Vu

d X V

−

−

=

� �� �=� �� ��

(2)

365

(a) SDI with one cluster (b) SDI with two clusters

For 1,2,j n= …, , 1, 2, ,k K= … , the cluster centers Vi are computed according to formula below:

1

1

( )

( )

X jn m

ijj

i nm

ijj

uV

u

=

=

=�

� (3)

For the case of hyper-ellipsoidal clusters, as well as in the presence of variable cluster densities and unequal numbers of data points in each cluster, and “exponential” distance measure 2 ( , )j id X V is defined [4]:

1( ) ( )/22 ( , )

Tj i i j ii X V F X V

j ii

Fd X V e

P−− −= (4)

Where Fi is the fuzzy covariance matrix of the ith cluster, and Pi is the a priori probability of selecting the ith cluster.

1

1

( )( )N

Tji j i j i

ji N

jij

u X V X VF

u

=

=

− −=�

� (5)

( )1

1 N

i jj

P h i XN =

= � (6)

B. Validity indices Since the problems include both finding the minimum

number of cluster (the optimal one) and the data which are best fitted by clusters, in the results performance evaluation step, we take advantage of two types of indices:

1) Inter cluster Distance Index: The first index, called “Inter cluster Distance Index”

(IDI), which is for analyzing the separateness of clusters in the results, can be represented by our method of inter-clusters distance measurement. We use this index to see that if there are clusters which can be merged.

( ) 2 21, ( )i j ij jiIDC c c d dN

= + (7)

Where dij and dji are the distance from cluster ith to jth and from cluster jth to cluster ith, respectively. Inter cluster distance index is to look for the redundancy in clustering. Since the inter cluster distances are normalized by the same number of data point, we can find out the nearby clusters by the pre-defined threshold. If the IDI is larger than threshold, two clusters are separated; otherwise, two clusters are overlapping, and can be merged into one cluster.

2) Sample Distribution Index:

The second index is for analyzing the compactness of cluster, called “Sample Distribution Index (SDI)”. Here we analyze the ratio of filling of data point over the reference samples. This index tells us the quality of clusters, shows the wrong classification and discards those clusters of poor sample distribution for re-clustering.

1

( )1( )k

f

j j

N jSDI k

k N== � (8)

Where k is number of clusters, Nj is the number of points of cluster jth, and Nf (j) is the number of filled bins of cluster jth. The sample distribution index is to look for those clusters that are not compact to represent their samples. The cluster will be split when the value of SDI is less than threshold. The classification based on two kinds of cluster was evaluated by SDI as seen in Fig.1: the left SDI is 0.99721, the right SDI is 0.52057, which will be split based on pre-defined threshold.

Since indices we proposed focus on the different aspects of performance in clustering, each index is supposed to play a proper role at different stages of algorithm. The performance may be represented by larger inter cluster distances and each cluster is with high sample distribution and compactness.

Figure 1. Example of a computation results of SDI.

C. Flowchart of the clustering algorithm 1. Initialization

Initialize arbitrary number of cluster k and cluster centers vi

Compute uij for all i and j n_try = 0; 2. n_iter = 0; 3. While not satisfy saturated condition n_iter = n_iter + 1 4. Compute � iju (the degree of membership of all feature

vectors in all the clusters) and new centroids iv� again 5. If �max ij iij u v ε− <� n_try = n_try +1 If n_iter = MAX_ITER or n_try = MAX_TRY

Break; Else

Go to STEP 3 6. Perform IDI checking

If IDI between clusters < threshold1 k = k – 1 (decrease number of cluster)

366

Else if SDI < threshold2 k = k + 1 (increase number of cluster)

Go to STEP 2 7. Finish: Saving results.

V. FUNCTIONAL REASONING

A. Plane fitting and finding the normal vector of each group After generating the groups by clustering, we have to

deal with the important issue like interpreting a set of points in a plane.

A representation of a plane is defined:

0a x b y c z d⋅ + ⋅ + ⋅ + = (9)

Starting with the distance from a point to a plane, we wish to find a, b, c and d so that we can minimize

2

2 2 21

( , , , )N

i i i

i

ax by cz df a b c d

a b c=

+ + +=

+ +� (10)

If we set the partial derivative with respect to d equal to zero, we can solve for d to get:

0 0 0( )d ax by cz= − + + (11)

Where (x0, y0, z0) is the centroid of the data. This means that the least squares plane contains the centroid. If we substitute it back into the equation for the plane we get

(12)

We can rewrite f(a, b, c, d) like this

2

0 0 02 2 2

1

( ) ( ) ( )( , , )

N

i

a x x b y y c z zf a b c

a b c=

− + − + −=

+ +� (13)

Now we are going to switch over to a matrix representation. Let's define V and M as:

[ ]TV a b c= (14)

1 0 1 0 1 0

0 0 0n n n

x x y y z zM

x x y y z z

− − −=

− − −� � � (15)

Multiplying the matrices out, f(a, b, c) becomes

( )( )( )T T

T

V M MVf VV V

= (16)

Let's define A=MTM. f(V) is minimized by the eigenvector of A that corresponds to its smallest eigenvalue. We could compute the eigenvectors of A, but this is not needed. The SVD of M is M=USVT, where S is a diagonal matrix containing the singular values of M, the columns of V are its singular vectors, and U is an orthogonal matrix. Now

2( ) ( )T T T T TA M M USV USV VS V= = = (17)

This decomposition of A diagonalizes the matrix and provides eigenvector decomposition. It means that the eigenvalues of A are the squares of the singular values of M, and the eigenvectors of A are the singular vectors of M.

To conclude, the orthogonal least squares 3D plane contains the centroid of the data, and its normal vector is the singular vector of M corresponding to its smallest singular value.

Assuming singular values �0 � �1 � �2, it follows that the plane:

0( ) 0x x v− ⋅ = (18)

It minimizes the sum of squared distances to points within a cluster. Thus v0 approximates the surface normal nx at x, or in other words, v1 and v2 span the tangent plane. That �0 quantitatively describes the variation along the surface normal. That �1 and �2 describe the variation of the sampling distribution in the tangent plane.

B. labeling possible objects which have perpendicular normal vectors After getting the normal vector of each cluster which is

grouped by self-organized fuzzy K-mean algorithm, we can get the angles between each normal vector, which will be helpful for deciding which group is like walls, floors and ceilings. For two vectors, A and B:

cosA B A B θ⋅ = (19)

Where � is the angle between A and B, |A| is the length of A and |B| is the length of B. If we have ( , , )A a x a y a z= ⋅ ⋅ ⋅ and ( , , )B b x b y b z= ⋅ ⋅ ⋅ , then:

1cos ( )A BA B

θ− ⋅= (20)

If the angle between A and B approximately equal 90 degree, then A or B is possibly the objects belonging to wall, floor or ceiling.

VI. EXPERIMENTAL RESULTS We tested the computation model on two real indoor

scenes. As is shown in Fig. 2a, to highlight the computed planar shape regions in the segmentation result, the boundaries of these shape regions are displayed in green

367

(Fig. 2b), a table is classified clearly. The part of indoor kitchen 3D structures inferred for the planar objects in the two examples are shown in Fig. 3a and Fig. 3b with different view position. It is clear that the segmentation results are greatly improved and walls in kitchen are classified, since the 3D perception can help explain many ambiguities in the segmentation. Our algorithm is applied on the real 3D planar identification from point clouds. Due to the variance of light condition, texture of objects in indoor environment, the quality of point clouds are not always good; therefore, the result may contain error groups.

(a) (b)

Figure 2. Example of 3D point clouds and segmented table.

(a) (b)

Figure 3. Top-view of the groups and segmented objects.

VII. SUMMARY AND FUTURE WORK In this paper we propose a new unsupervised planar

segmentation computation model for extracting 3D structure from 3D point clouds. This model is unsupervised with performance checking by indices and thus can correct possible errors at lower levels to achieve a global optimum. Currently, we infer the 3D structure of objects individually without considering more spatial relation between them besides angles between normal vectors of each cluster. In reality, objects are organized in a scene with strong spatial

regularities, which provide more constraints to recover 3D scene. We will add this to the current model in the future.

ACKNOWLEDGMENT This work is done at the Intelligent Systems Research

Center, Sungkyunkwan University and is supported by Korea Science and Engineering Foundation (Grant No. R31-2008-000-10062-0) and China National Natural Science Foundation (Grant No. 5110578, No.50805025). The authors would like to thank Mr. Jaewoong Kim, for previous work, and Dr. Zhaojin Lu for extensive discussion.

REFERENCES [1] Duda, R.O., Hart, P.E., Pattern Classification and Scene analysis,

John Wiley and Sons, NY., 1973. [2] Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R.,

Advances in Knowledge Discovery and Data Mining, AAAI Press, 1996.

[3] J. C. Bezdek, C. Dunn, “optimal fuzzy patition: A heuristic for estimating the parameters in a mixture of normal distributions,” IEEE Trans. Comput., vol. C-23, pp. 835-838, 1975.

[4] I. Gath and A. B. Geva, “Unsupervised Optimal Fuzzy Clustering,” IEEE Transections on Pattern Analysis and Machine Intelligence, vol. 11, No. 7. July, 1989, pp. 773–779.

[5] A. K. Jain , R. C. Dubes, Algorithms for Clustering Data, Eaglewood Cliffs, NJ:Prentice-Hall, 1988.

[6] J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, New York: Plenum, 1981.

[7] X. L. Xie, G. Beni, A validity measure for fuzzy clustering, IEEE Trans. Pattern Anal. Mach. Intell., 1991, pp. 841-847,

[8] E. E. Gustafson, W. C. Kessel, "fuzzy clustering with a fuzzy covariance matrix," Proc. IEEE CDC, San Diego, CA., 1979, pp. 761-766.

[9] R.B.Rusu, Z. C. Marton, N. Blodow, M. Dolha, M. Beete, “Towards 3D Point cloud based object maps for house hold environments ”Robotics and Autonomous Systems, 2008, pp. 927-941

[10] S. Vasudevan, S. Gachter, V. Nguyen, R. Siegwart, “Cognitive maps for mobile robots-an object based approach,” Robotics and Autonomous Systems, 2007, pp. 359-371.

[11] O.M. Mozos, A. Rottmann, R. Triebel, P. Jensfelt, W. Burgard, “Semantic labeling of places using information extracted from laser and vision sensor data,” Proceedings of the IEEE/RSJ IROS Workshop: From Sensors to Human Spatial Concepts, Beijing, China, 2006.

368