Results 1  10
of
37
Consistency of spectral clustering
, 2004
"... Consistency is a key property of statistical algorithms, when the data is drawn from some underlying probability distribution. Surprisingly, despite decades of work, little is known about consistency of most clustering algorithms. In this paper we investigate consistency of a popular family of spe ..."
Abstract

Cited by 572 (15 self)
 Add to MetaCart
Consistency is a key property of statistical algorithms, when the data is drawn from some underlying probability distribution. Surprisingly, despite decades of work, little is known about consistency of most clustering algorithms. In this paper we investigate consistency of a popular family of spectral clustering algorithms, which cluster the data with the help of eigenvectors of graph Laplacian matrices. We show that one of the two of major classes of spectral clustering (normalized clustering) converges under some very general conditions, while the other (unnormalized), is only consistent under strong additional assumptions, which, as we demonstrate, are not always satisfied in real data. We conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering in practical applications. We believe that methods used in our analysis will provide a basis for future exploration of Laplacianbased methods in a statistical setting.
Protovalue functions: A laplacian framework for learning representation and control in markov decision processes
 Journal of Machine Learning Research
, 2006
"... This paper introduces a novel spectral framework for solving Markov decision processes (MDPs) by jointly learning representations and optimal policies. The major components of the framework described in this paper include: (i) A general scheme for constructing representations or basis functions by d ..."
Abstract

Cited by 92 (10 self)
 Add to MetaCart
This paper introduces a novel spectral framework for solving Markov decision processes (MDPs) by jointly learning representations and optimal policies. The major components of the framework described in this paper include: (i) A general scheme for constructing representations or basis functions by diagonalizing symmetric diffusion operators (ii) A specific instantiation of this approach where global basis functions called protovalue functions (PVFs) are formed using the eigenvectors of the graph Laplacian on an undirected graph formed from state transitions induced by the MDP (iii) A threephased procedure called representation policy iteration comprising of a sample collection phase, a representation learning phase that constructs basis functions from samples, and a final parameter estimation phase that determines an (approximately) optimal policy within the (linear) subspace spanned by the (current) basis functions. (iv) A specific instantiation of the RPI framework using leastsquares policy iteration (LSPI) as the parameter estimation method (v) Several strategies for scaling the proposed approach to large discrete and continuous state spaces, including the Nyström extension for outofsample interpolation of eigenfunctions, and the use of Kronecker sum factorization to construct compact eigenfunctions in product spaces such as factored MDPs (vi) Finally, a series of illustrative discrete and continuous control tasks, which both illustrate the concepts and provide a benchmark for evaluating the proposed approach. Many challenges remain to be addressed in scaling the proposed framework to large MDPs, and several elaboration of the proposed framework are briefly summarized at the end.
Shape priors using Manifold Learning Techniques
 in &quot;11th IEEE International Conference on Computer Vision, Rio de Janeiro
, 2007
"... We introduce a nonlinear shape prior for the deformable model framework that we learn from a set of shape samples using recent manifold learning techniques. We model a category of shapes as a finite dimensional manifold which we approximate using Diffusion maps, that we call the shape prior manifol ..."
Abstract

Cited by 38 (2 self)
 Add to MetaCart
(Show Context)
We introduce a nonlinear shape prior for the deformable model framework that we learn from a set of shape samples using recent manifold learning techniques. We model a category of shapes as a finite dimensional manifold which we approximate using Diffusion maps, that we call the shape prior manifold. Our method computes a Delaunay triangulation of the reduced space, considered as Euclidean, and uses the resulting space partition to identify the closest neighbors of any given shape based on its Nyström extension. Our contribution lies in three aspects. First, we propose a solution to the preimage problem and define the projection of a shape onto the manifold. Based on closest neighbors for the Diffusion distance, we then describe a variational framework for manifold denoising. Finally, we introduce a shape prior term for the deformable framework through a nonlinear energy term designed to attract a shape towards the manifold at given constant embedding. Results on shapes of cars and ventricule nuclei are presented and demonstrate the potentials of our method.
Manifold denoising
 Advances in Neural Information Processing Systems (NIPS) 19
, 2006
"... We consider the problem of denoising a noisily sampled submanifold M in R d, where the submanifold M is a priori unknown and we are only given a noisy point sample. The presented denoising algorithm is based on a graphbased diffusion process of the point sample. We analyze this diffusion process us ..."
Abstract

Cited by 36 (1 self)
 Add to MetaCart
(Show Context)
We consider the problem of denoising a noisily sampled submanifold M in R d, where the submanifold M is a priori unknown and we are only given a noisy point sample. The presented denoising algorithm is based on a graphbased diffusion process of the point sample. We analyze this diffusion process using recent results about the convergence of graph Laplacians. In the experiments we show that our method is capable of dealing with nontrivial highdimensional noise. Moreover using the denoising algorithm as preprocessing method we can improve the results of a semisupervised learning algorithm. 1
Multiway spectral clustering with outofsample extensions through weighted kernel PCA
 IEEE Trans. Pattern Anal. Mach. Intell
, 2010
"... Abstract—A new formulation for multiway spectral clustering is proposed. This method corresponds to a weighted kernel principal component analysis (PCA) approach based on primaldual leastsquares support vector machine (LSSVM) formulations. The formulation allows the extension to outofsample poi ..."
Abstract

Cited by 18 (6 self)
 Add to MetaCart
(Show Context)
Abstract—A new formulation for multiway spectral clustering is proposed. This method corresponds to a weighted kernel principal component analysis (PCA) approach based on primaldual leastsquares support vector machine (LSSVM) formulations. The formulation allows the extension to outofsample points. In this way, the proposed clustering model can be trained, validated, and tested. The clustering information is contained on the eigendecomposition of a modified similarity matrix derived from the data. This eigenvalue problem corresponds to the dual solution of a primal optimization problem formulated in a highdimensional feature space. A model selection criterion called the Balanced Line Fit (BLF) is also proposed. This criterion is based on the outofsample extension and exploits the structure of the eigenvectors and the corresponding projections when the clusters are well formed. The BLF criterion can be used to obtain clustering parameters in a learning framework. Experimental results with difficult toy problems and image segmentation show improved performance in terms of generalization to new samples and computation times. Index Terms—Spectral clustering, kernel principal component analysis, outofsample extensions, model selection. Ç 1
An Analysis of the Convergence of Graph Laplacians
"... Existing approaches to analyzing the asymptotics of graph Laplacians typically assume a wellbehaved kernel function with smoothness assumptions. We remove the smoothness assumption and generalize the analysis of graph Laplacians to include previously unstudied graphs including kNN graphs. We also i ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
Existing approaches to analyzing the asymptotics of graph Laplacians typically assume a wellbehaved kernel function with smoothness assumptions. We remove the smoothness assumption and generalize the analysis of graph Laplacians to include previously unstudied graphs including kNN graphs. We also introduce a kernelfree framework to analyze graph constructions with shrinking neighborhoods in general and apply it to analyze locally linear embedding (LLE). We also describe how, for a given limit operator, desirable properties such as a convergent spectrum and sparseness can be achieved by choosing the appropriate graph construction. 1.
Coclustering for directed graphs; the stochastic coblockmodel and a spectral algorithm
, 2012
"... Communities of highly connected actors form an essential feature in the structure of several empirical directed and undirected networks. However, compared to the amount of research on clustering for undirected graphs, there is relatively little understanding of clustering in directed networks. Th ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
Communities of highly connected actors form an essential feature in the structure of several empirical directed and undirected networks. However, compared to the amount of research on clustering for undirected graphs, there is relatively little understanding of clustering in directed networks. This paper extends the spectral clustering algorithm to directed networks in a way that coclusters or biclusters the rows and columns of a graph Laplacian. Coclustering leverages the increased complexity of asymmetric relationships to gain new insight into the structure of the directed network. To understand this algorithm and to study its asymptotic properties in a canonical setting, we propose the Stochastic CoBlockmodel to encode coclustering structure. This is the first statistical model of coclustering and it is derived using the concept of stochastic equivalence that motivated the original Stochastic Blockmodel. Although directed spectral clustering is not derived from the Stochastic CoBlockmodel, we show that, asymptotically, the algorithm can estimate the blocks in a high dimensional asymptotic setting in which the number of blocks grows with the number of nodes. The algorithm, model, and asymptotic results can all be extended to bipartite graphs.
UNIFYING LOCAL AND NONLOCAL PROCESSING WITH PARTIAL DIFFERENCE OPERATORS ON WEIGHTED GRAPHS
 INTERNATIONAL WORKSHOP ON LOCAL AND NONLOCAL APPROXIMATION IN IMAGE PROCESSING, SUISSE
, 2008
"... In this paper, local and nonlocal image processing are unified, within the same framework, by defining discrete derivatives on weighted graphs. These discrete derivatives allow to transcribe continuous partial differential equations and energy functionals to partial difference equations and discrete ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
(Show Context)
In this paper, local and nonlocal image processing are unified, within the same framework, by defining discrete derivatives on weighted graphs. These discrete derivatives allow to transcribe continuous partial differential equations and energy functionals to partial difference equations and discrete functionals over weighted graphs. With this methodology, we consider two gradientbased problems: regularization and mathematical morphology. The gradientbased regularization framework allows to connect isotropic and anisotropic pLaplacians diffusions, as well as neighborhood filtering. Within the same discrete framework, we present morphological operations that allow to recover and to extend wellknown PDEsbased and algebraic operations to nonlocal configurations. Finally, experimental results show the ability and the flexibility of the proposed methodology in the context of image and unorganized data set processing.
An iterated graph laplacian approach for ranking on manifolds
 In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
, 2011
"... Ranking is one of the key problems in information retrieval. Recently, there has been significant interest in a class of ranking algorithms based on the assumption that data is sampled from a low dimensional manifold embedded in a higher dimensional Euclidean space. In this paper, we study a popular ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
Ranking is one of the key problems in information retrieval. Recently, there has been significant interest in a class of ranking algorithms based on the assumption that data is sampled from a low dimensional manifold embedded in a higher dimensional Euclidean space. In this paper, we study a popular graph Laplacian based ranking algorithm [23] using an analytical method, which provides theoretical insights into the ranking algorithm going beyond the intuitive idea of “diffusion. ” Our analysis shows that the algorithm is sensitive to a commonly used parameter due to the use of symmetric normalized graph Laplacian. We also show that the ranking function may diverge to infinity at the query point in the limit of infinite samples. To address these issues, we propose an improved ranking algorithm on manifolds using Green’s function of an iterated unnormalized graph Laplacian, which is more robust and density adaptive, as well as pointwise continuous in the limit of infinite samples. We also for the first time in the ranking literature empirically explore two variants from a family of twice normalized graph Laplacians. Experimental results on text and image data support our analysis, which also suggest the potential value of twice normalized graph Laplacians in practice.