S. Z. Selim and M. A. Ismail, K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.6, n.1, pp.81-86, 1984  Home/Search   Document Not in Database   Summary   Related Articles   Check

....starts with any feasible solution, and it repeatedly computes the neighborhood of each center and then moves the center to the centroid of its neighborhood, until some convergence criterion is satisfied. It can be shown that Lloyd s algorithm eventually converges to a locally optimal solution [38]. Computing nearest neighbors is the most expensive step in Lloyd s algorithm, but a number of practical implementations of this algorithm have been discovered recently [2, 24, 35, 36, 37] x y Data points Optimal centers Heuristic centers z Fig. 1: Lloyd s algorithm can produce an ....

S. Z. Selim and M. A. Ismail. K-means-type algorithms: A generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:81--87, 1984.

....is an iterative (EM type) process, which tries to nd data map D and its correspondent feature map F to maximize L(W j W (D; F ) Several approximation methods have been used in our algorithm to speed up the process. For a rigorous proof of convergence in 7 EM type algorithms, please refer to [Selim and Ismail, 1984; Ambroise and Govaert, 1998; Kaski, 2000] Our algorithm can be easily adapted to estimate the number of clusters instead of using K as an input parameter. We observed that the best number of clusters results the smallest average conditional entropy between clustering results obtained from ....

S. Z. Selim and M. A. Ismail, \K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality," IEEE Trans., PAMI-6(1), January 1984.

....is nearest to a given target item. This is often referred to as the closest point problem, or the nearest neighbour (NN) search problem. For example, we can find the NN search problem in algorithms for k nearest neighbour classification[39, 12] vector quantisation[63, 56] and k means clustering[106, 149, 86, 129, 100] where the bulk of the computational cost in each iteration is in mapping each element of the training set to its nearest cluster centre. Algorithms which require NN search are applied in a variety of problem domains, such as image compression[63, 122, 1] conceptual clustering[90] and machine ....

....of all data points belonging to that cluster. The k means algorithm is also known to researchers in vector quantisation as the generalised Lloyd Max algorithm, or the Linde BuzoGray (LBG) algorithm [110, 114, 106] A rigorous proof of the finite convergence of the k means algorithm can be found in [149]. It is well known, however, that the k means algorithm is very much dependent on the choice of the initial distribution of cluster centres, and often results in locally optimal solutions [94, 100, 148] When the algorithm is applied to create VQ codebooks, it converges to a locally optimal ....

[Article contains additional citation context not shown here]

Shokri Z. Selim and M.A. Ismail. k-means-type algorithms: A generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6(1):81--87, January 1984.

..... Such an increase or decrease happens smartly, trying to insert new codewords where the quantization error is higher and to eliminate them where the error is lower. A similar strategy is proposed by Fritzke in [30,32] for his competitive learning algorithms, while FACS is a K means type algorithm [39,47]. Besides, in FACS, insertions and deletions of codewords are regulated by a stochastic process; in [30, 32] they occur deterministically. Particular attention is paid so that the technique employed allows the convergence of the algorithm towards a good solution in a few iterations. The results ....

S. Z. Selim and M. A. Ismail, \K-means-type algorithms: A generalized convergence theorem and characterization of local optimality," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, no. 1, 1984.

..... Such an increase or decrease happens smartly, trying to insert new codewords where the quantization error is higher and to eliminate them where the error is lower. A similar strategy is proposed by Fritzke in [61,63] for his competitive learning algorithms, while FACS is a K means type algorithm [3, 88]. Besides, in FACS, insertions and deletions of codewords are regulated by a stochastic process; in [61, 63] they occur deterministically. Particular attention is paid so that the technique employed allows the convergence of the algorithm towards a good solution in a few iterations. The results ....

S. Z. Selim and M. A. Ismail, \K-means-type algorithms: A generalized convergence theorem and characterization of local optimality," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, no. 1, 1984.

....the so built rectangular regions as classifiers and finally evaluating the resulting clusterings by the r.u.m.length. The best so achieved classification was then compared against the results of AutoClass and KMeans . KMeans is known to converge to a local optimum of its quality measure [SI84] so it is reasonable to assume that the final result will be a near optimal clustering according to this measure. We initialized KMeans with the number of initial centroids ranging from 2 to 10 and accepted the best result (in terms of recall) on the datasets. For AutoClass we used the default ....

S.Z. Selim and M.A. Ismail. K-means-type algorithms: A generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(1):81--87, 1984.

....is required whose optimization would provide the nal clusters. An intuitively simple criterion is the within cluster spread, which, as in the K means algorithm, needs to be minimized for good clustering. However, unlike the K means algorithm which may get stuck at values which are not optimal [22], the proposed technique should be able to provide good results irrespective of the starting con guration. It is towards this goal that we have integrated the simplicity of the K means algorithm with the capability of GAs in avoiding local optima for developing a GA based clustering technique ....

....is the number of elements belonging to cluster . Step 4: If zH z , i 1, 2, 2 , K then terminate. Otherwise continue from step 2. Note that in case the process does not terminate at Step 4 normally, then it is executed for a maximum xed number of iterations. It has been shown in Ref. [22] that K means algorithm may converge to values that are not optimal. Also global solutions of large problems cannot be found with a reasonable amount of computation e ort [29] It is because of these factors that several approximate methods are developed to solve the underlying optimization ....

S.Z. Selim, M.A. Ismail, K-means type algorithms: a generalized convergence theorem and characterization of local optimality, IEEE Trans. Pattern Anal. Mach. Intell. 6 (1984) 81}87.

....In addition, a single document typically contains only a small fraction of the total number of words in the entire collection; hence, the document vectors are generally very sparse, i.e. contain a lot of zero entries. The k means algorithm is a popular method for clustering a set of data vectors [5, 2, 18]. The classical version of k means uses Euclidean distance, however this distance measure is often inappropriate for its application to clustering a collection of documents [21] An e ective measure of similarity between documents, and one that is often used in information retrieval, is cosine ....

S. Z. Selim and M. A. Ismail. K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. Pattern Analysis and Machine Intelligence, 6:81-87, 1984.

....4 terminates the algorithm. tu Theorem 3 The Algorithm in Figure 1 always converges to a local minimum in a nite number of iterations. Proof. The result follows since the algorithm monotonically decreases the objective function value, which is bounded from below (by zero) For more details, see [27]. tu We now show that the total Jensen Shannon divergence (which is constant for a given set of probability distributions) can be written as the sum of two terms, one of which is the objective function (14) that our algorithm minimizes. Theorem 4 Let p 1 ; pn be a set of probability ....

S. Z. Selim and M. A. Ismail. K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. Pattern Analysis and Machine Intelligence, 6:81-87, 1984.

....Although finding an optimal solution to the k Median problem is known to be NP hard, many useful heuristics, including k Means, have been proposed. The k Means algorithm has enjoyed considerable practical success [3] although the solution it produces is only guaranteed to be a local optimum [28]. On the other hand, in the algorithms literature, several approximation algorithms have 1 Although squared Euclidean distance is not a metric, it obeys a relaxed triangle inequality and therefore behaves much like a metric. 2 been proposed for k Median. A c approximation algorithm is ....

S.Z. Selim and M.A. Ismail. K--means type algorithms: A generalized convergence theorem and characterization of local optimality. IEEE Trans. PAMI, 6:81--86, 1984.

....The problem is that of assigning m points in R n , represented by the data set A : fx i g m i=1 , into k clusters. The clustering problem is first formulated as minimizing a piecewise linear concave function over a polyhedral set resulting in the k Median algorithm. The k Mean algorithm [147, 81, 63] computes a local solution to the problem of minimizing a nonconvex objective over a polyhedral set. We examine a novel approach, k Plane clustering, where clusters are characterized by planes in R n , obtainable by solving an eigenvalue problem for each cluster. The algorithms presented above ....

....are described by a centroid in R n . 80 3.1 Clustering to Centroids We consider the unsupervised assignment of elements of a given set to groups or clusters of like points. Many approaches to this problem include statistical [81, 63] machine learning [60] integer and mathematical programming [147, 3, 135]. We address the following explicit description of the clustering problem: given m points in n dimensional real space R n , and a fixed integer k of clusters, determine k centers in R n such that the sum of the distances of each point to the nearest cluster center is minimized. If the ....

[Article contains additional citation context not shown here]

S. Z. Selim and M. A. Ismail. K-Means-Type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6:81--87, 1984.

....with many clustering algorithms that work as a hill climbing strategy whose deterministic behaviour leads to a local minima dependent on initial solution and on instance order. Although there is no guarantee of achieving a global minima, at least the convergence of K Means algorithm is ensured [26]. 3 Milligan in [24] shows the strong dependence of the K Means algorithm on initial clustering and suggests that good final cluster structures can be obtained using Ward s hierarchical method [29] to provide the K Means algorithm with initial clusters. Fisher in [10] propose creating the ....

Selim, S.Z. and Ismail, M.A. (1984). K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 81-87.

....that of S minus a point, but the result would have been the same) Line 5 of the code deserves further explanation: in order to cluster the set of Fd i values, we can use any known algorithm. For instance, we could feed the fractal dimension values Fd i , and a value k to a K means implementation [23, 9]. Alternatively, we can let a hierarchical clustering algorithm (e.g. CURE [12] cluster the sequence of Fd i values. Although, in principle, any of the dimensions in the family described by Equation 1 can be used in line 4 of the initialization step, we have found that the best results are ....

....clustering methods in three varieties: 5.1 Model and Optimization Based Approaches Most of the initial approaches devised for clustering belong to this class. The methods usually start with an initial partition and use an iterative strategy to optimize a function. The K Means algorithm [23, 9] uses k initial prototypes of center of gravity of the clusters, and then iteratively assigns the data points to the nearest prototype and shifts the prototypes towards the mean of the clusters obtained. K means was initially devised as an in memory algorithm, but variants of it have been recently ....

S.Z. Selim and M.A. Ismail. K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6(1), 1984.

....Bibliometrics techniques can be thought of as local in nature because they typically consider only the local link properties between two documents. Of course, similarity metrics such as co citation and bibliographic coupling can be used along with classical clustering techniques, such as k means [9], to reduce the dimensionality of the document space, thus identifying documents in a community that is centered about a cluster centroid. More radical forms of dimensionality reduction have used this basic idea to cluster literature databases with over 150 thousand documents [10] However, ....

S. Selim and M. Ismail. K-Means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(1):81-87, 1984.

....For points in general position, the algorithm will eventually converge to a point that is a local minimum. This is true because any local minimum for this problem corresponds to a centroidal Voronoi configuration (see [15, 12] However, the result is not necessarily a global minimum. See [7, 30, 33, 35] for further discussion of its statistical and convergence properties. Because of its simplicity and flexibility it is a very popular algorithm and is widely used in statistical analysis, in spite of its apparent limitations. For example, it can be used in conjunction with another more global ....

S. Z. Selim and M. A. Ismail. K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. on Pattern Analysis and Machine Intelligence, 6:81--87, 1984.

....the nearest plane to m given points. They showed that the problem reduces to finding the least eigenvalue and associated eigenvector of a certain n n symmetric positive semidefinite matrix. The above two problems are of interest since they arise as subproblems in algorithms for data clustering [1, 3, 5]. In this note, we consider the general problem of finding the nearest q flat (i.e. q dimensional a#ne set [4, page 3] to m points in # n , with 0 # q # n 1 integer. We show that this problem reduces to finding the p = n q least eigenvalues and associated eigenvectors of the same ....

....page 3] to m points in # n , with 0 # q # n 1 integer. We show that this problem reduces to finding the p = n q least eigenvalues and associated eigenvectors of the same nn matrix considered by Bradley and Mangasarian (see Theorem 1) Accordingly, the k mean algorithm [3, 3.3. 2] [5] and the k plane algorithm of [1] can be extended by replacing the mean (q = 0) and the plane (q = n 1) with q flat. We first formulate the problem. Recall that the square of the distance from any x # # n to the q flat (0 # q # n 1 integer) y # # n : W # y = # # , 1) where ....

S. Z. Selim and M. A. Ismail, K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6 (1984), 81--87. 4

No context found.

Selim, S. Z.; Ismail, M. A. K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. on Pattern Analysis and Machine Intelligence. Vol 6, no. 1, pp. 81-86.

No context found.

S. Z. Selim and M. A. Ismail, K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.6, n.1, pp.81-86, 1984

No context found.

S. Z. Selim and M. A. Ismail, \K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality," IEEE Trans., PAMI-6(1), January 1984.

No context found.

S. Selim and M. A. Ismail. K-means type algorithms: A generalized convergence theorem and characterzation of local optimality. 6.

No context found.

S. Z. Selim and M. A. Ismail. K-Means-Type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6:81--87, 1984.

No context found.

S.Z. Selim and M.A. Ismail, k-means-type algorithm: generalized convergence theorem and characterization of local optimality, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(1984), 81-87.

No context found.

S. Z. Selim and M. A. Ismail. K-Means-Type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6:81--87, 1984.

No context found.

Selim, S. and Ismail, M. (1984). K-means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, No. 1 (pp. 81-87).

No context found.

S.Z. Selim and M.A. Ismail. K-Means-type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI, 6:81--87, 1984.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC