Results 1  10
of
25
Nearoptimal joint object matching via convex relaxation. arxiv preprint arXiv:1402.1473
, 2014
"... Joint object matching aims at aggregating information from a large collection of similar instances (e.g. images, graphs, shapes) to improve the correspondences computed between pairs of objects, typically by exploiting global map compatibility. Despite some practical advances on this problem, fro ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Joint object matching aims at aggregating information from a large collection of similar instances (e.g. images, graphs, shapes) to improve the correspondences computed between pairs of objects, typically by exploiting global map compatibility. Despite some practical advances on this problem, from the theoretical point of view, the errorcorrection ability of existing algorithms are limited by a constant barrier — none of them can provably recover the correct solution when more than a constant fraction of input correspondences are corrupted. Moreover, prior approaches focus mostly on fully similar objects, while it is practically more demanding and realistic to match instances that are only partially similar to each other. In this paper, we propose an algorithm to jointly match multiple objects that exhibit only partial similarities, where the provided pairwise feature correspondences can be densely corrupted. By encoding a consistent partial map collection into a 01 semidefinite matrix, we attempt recovery via a twostep procedure, that is, a spectral technique followed by a parameterfree convex program called MatchLift. Under a natural randomized model, MatchLift exhibits nearoptimal errorcorrection ability, i.e. it guarantees the recovery of the groundtruth maps even when a dominant fraction of the inputs are randomly corrupted. We evaluate the proposed algorithm on various benchmark data sets including synthetic examples and realworld examples, all of which confirm the practical applicability of the proposed algorithm.
Breaking the Small Cluster Barrier of Graph Clustering Supplementary Material
"... In this supplementary material, we present proof details. 1 Notation and Conventions We use the following notation and conventions throughout the supplement. For a real n × n matrix M, we use the unadorned norm ‖M ‖ to denote its spectral norm. The notation ‖M‖F refers to the Frobenius norm, ‖M‖1 is ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
In this supplementary material, we present proof details. 1 Notation and Conventions We use the following notation and conventions throughout the supplement. For a real n × n matrix M, we use the unadorned norm ‖M ‖ to denote its spectral norm. The notation ‖M‖F refers to the Frobenius norm, ‖M‖1 is ∑ i,j M(i, j)  and ‖M‖ ∞ is maxij M(i, j). We will also study operators on the space of matrices. To distinguish them from the matrices studied in this work, we will simply call these objects “operators”, and will denote them using a calligraphic font, e.g. P. The norm ‖P ‖ of an operator is defined as where the supremum is over matrices M. ‖P ‖ = sup ‖PM‖F, M:‖M‖F =1 For a fixed, real n × n matrix M, we define the matrix linear subspace T (M) as follows: T (M): = {Y M + MX: X, Y ∈ R n×n}. In words, this subspace is the set of matrices spanned by matrices each row of which is in the row space of M, and matrices each column of which is in the column space of M. For any given subspace of matrices S ⊆ Rn×n, we let PS denote the orthogonal projection onto S with respect to the the inner product 〈X, Y 〉 = ∑n i,j=1 X(i, j)Y (i, j) = tr XtY. This means that for any matrix M, PSM = argminX∈S ‖M − X‖F. For a matrix M, we let Γ(M) denote the set of matrices supported on a subset of the support of M. Note that for any matrix X,
Robust and Computationally Feasible Community Detection in the Presence of Arbitrary Outlier Nodes
, 2014
"... Community detection, which aims to cluster N nodes in a given graph into r distinct groups based on the observed undirected edges, is an important problem in network data analysis. In this paper, the popular stochastic block model (SBM) is extended to the generalized stochastic block model (GSBM) th ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
Community detection, which aims to cluster N nodes in a given graph into r distinct groups based on the observed undirected edges, is an important problem in network data analysis. In this paper, the popular stochastic block model (SBM) is extended to the generalized stochastic block model (GSBM) that allows for adversarial outlier nodes, which are connected with other nodes in the graph in an arbitrary way. Under this model, we introduce a procedure using convex optimization followed by kmeans algorithm with k = r. Both theoretical and numerical properties of the method are analyzed. A theoretical guarantee is given for our methodology to accurately detect the communities with small misclassification rate under the setting where the number of clusters can grow with N. This theoretical result admits to the best known result in the literature of computationally feasible community detection. Numerical results show that our method is both computationally fast and robust to different kinds of outliers, while some popular computationally fast community detection algorithms, such as spectral clustering applied to adjacency matrices or graph Laplacians, may fail due to a very small portion of outliers. We apply a slight modification of our method to a political blogs data set, showing that our method is competent in practice, and comparable to existing computationally feasible methods in the literature. To the best of the authors ’ knowledge, our result is the first in the literature in terms of clustering communities with fast growing numbers under the generalized stochastic block model where a portion of arbitrary outlier nodes exist.
A Tensor Approach to Learning Mixed Membership Community Models
"... Community detection is the task of detecting hidden communities from observed interactions. Guaranteed community detection has so far been mostly limited to models with nonoverlapping communities such as the stochastic block model. In this paper, we remove this restriction, and provide guaranteed ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
Community detection is the task of detecting hidden communities from observed interactions. Guaranteed community detection has so far been mostly limited to models with nonoverlapping communities such as the stochastic block model. In this paper, we remove this restriction, and provide guaranteed community detection for a family of probabilistic network models with overlapping communities, termed as the mixed membership Dirichlet model, first introduced by Airoldi et al. (2008). This model allows for nodes to have fractional memberships in multiple communities and assumes that the community memberships are drawn from a Dirichlet distribution. Moreover, it contains the stochastic block model as a special case. We propose a unified approach to learning these models via a tensor spectral decomposition method. Our estimator is based on loworder moment tensor of the observed network, consisting of 3star counts. Our learning method is fast and is based on simple linear algebraic operations, e.g., singular value decomposition and tensor power iterations. We provide guaranteed recovery of community memberships and model parameters and present a careful finite sample analysis of our learning method. As an important special case, our results match the best known scaling requirements for the (homogeneous) stochastic block model.
Network crossvalidation for determining the number of communities in network data. Available at arXiv:1411.1715v1
, 2014
"... The stochastic block model and its variants have been a popular tool in analyzing large network data with community structures. Model selection for these network models, such as determining the number of communities, has been a challenging statistical inference task. In this paper we develop an effi ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
The stochastic block model and its variants have been a popular tool in analyzing large network data with community structures. Model selection for these network models, such as determining the number of communities, has been a challenging statistical inference task. In this paper we develop an efficient crossvalidation approach to determine the number of communities, as well as to choose between the regular stochastic block model and the degree corrected block model. Our method, called network crossvalidation, is based on a blockwise edge splitting technique, combined with an integrated step of community recovery using subblocks of the adjacency matrix. The solid performance of our method is supported by theoretical analysis of the subblock parameter estimation, and is demonstrated in extensive simulations and a data example. Extensions to more general network models are also discussed. 1
Provable Algorithms for Machine Learning Problems
, 2013
"... Modern machine learning algorithms can extract useful information from text, images and videos. All these applications involve solving NPhard problems in average case using heuristics. What properties of the input allow it to be solved efficiently? Theoretically analyzing the heuristics is often v ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Modern machine learning algorithms can extract useful information from text, images and videos. All these applications involve solving NPhard problems in average case using heuristics. What properties of the input allow it to be solved efficiently? Theoretically analyzing the heuristics is often very challenging. Few results were known. This thesis takes a different approach: we identify natural properties of the input, then design new algorithms that provably works assuming the input has these properties. We are able to give new, provable and sometimes practical algorithms for learning tasks related to text corpus, images and social networks. The first part of the thesis presents new algorithms for learning thematic structure in documents. We show under a reasonable assumption, it is possible to provably learn many topic models, including the famous Latent Dirichlet Allocation. Our algorithm is the first provable algorithms for topic modeling. An implementation runs 50 times faster than latest MCMC implementation and produces comparable results. The second part of the thesis provides ideas for provably learning deep, sparse representations. We start with sparse linear representations, and give the first algorithm for dictionary learning problem with provable guarantees. Then we apply similar ideas to deep learning: under reasonable assumptions our algorithms can learn a deep network built by denoising autoencoders. The final part of the thesis develops a framework for learning latent variable models. We demonstrate how various latent variable models can be reduced to orthogonal tensor decomposition, and then be solved using tensor power method. We give a tight perturbation analysis for tensor power method, which reduces the number of samples required to learn many latent variable models. In theory, the assumptions in this thesis help us understand why intractable problems in machine learning can often be solved; in practice, the results suggest inherently new approaches for machine learning. We hope the assumptions and algorithms inspire new research problems and learning algorithms. iii
Improved graph clustering
"... Graph clustering involves the task of partitioning nodes, so that the edge density is higher within partitions as opposed to across partitions. A natural, classic and popular statistical setting for evaluating solutions to this problem is the stochastic block model, also referred to as the planted p ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Graph clustering involves the task of partitioning nodes, so that the edge density is higher within partitions as opposed to across partitions. A natural, classic and popular statistical setting for evaluating solutions to this problem is the stochastic block model, also referred to as the planted partition model. In this paper we present a new algorithm a convexified version of Maximum Likelihood for graph clustering. We show that, in the classic stochastic block model setting, it outperforms all existing methods by polynomial
Fusion moves for correlation clustering
 In CVPR
"... Correlation clustering, or multicut partitioning, is widely used in image segmentation for partitioning an undirected graph or image with positive and negative edge weights such that the sum of cut edge weights is minimized. Due to its NPhardness, exact solvers do not scale and approximative solv ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Correlation clustering, or multicut partitioning, is widely used in image segmentation for partitioning an undirected graph or image with positive and negative edge weights such that the sum of cut edge weights is minimized. Due to its NPhardness, exact solvers do not scale and approximative solvers often give unsatisfactory results. We investigate scalable methods for correlation clustering. To this end we define fusion moves for the correlation clustering problem. Our algorithm iteratively fuses the current and a proposed partitioning which monotonously improves the partitioning and maintains a valid partitioning at all times. Furthermore, it scales to larger datasets, gives near optimal solutions, and at the same time shows a good anytime performance. 1.
Graph Clustering With Missing Data: Convex Algorithms and Analysis
"... We consider the problem of finding clusters in an unweighted graph, when the graph is partially observed. We analyze two programs, one which works for dense graphs and one which works for both sparse and dense graphs, but requires some a priori knowledge of the total cluster size, that are based on ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We consider the problem of finding clusters in an unweighted graph, when the graph is partially observed. We analyze two programs, one which works for dense graphs and one which works for both sparse and dense graphs, but requires some a priori knowledge of the total cluster size, that are based on the convex optimization approach for lowrank matrix recovery using nuclear norm minimization. For the commonly used Stochastic Block Model, we obtain explicit bounds on the parameters of the problem (size and sparsity of clusters, the amount of observed data) and the regularization parameter characterize the success and failure of the programs. We corroborate our theoretical findings through extensive simulations. We also run our algorithm on a real data set obtained from crowdsourcing an image classification task on the Amazon Mechanical Turk, and observe significant performance improvement over traditional methods such as kmeans. 1
Weighted Graph Clustering with NonUniform Uncertainties
"... We study the graph clustering problem where each observation (edge or noedge between a pair of nodes) may have a different level of confidence/uncertainty. We propose a clustering algorithm that is based on optimizing an appropriate weighted objective, where larger weights are given to observati ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We study the graph clustering problem where each observation (edge or noedge between a pair of nodes) may have a different level of confidence/uncertainty. We propose a clustering algorithm that is based on optimizing an appropriate weighted objective, where larger weights are given to observations with lower uncertainty. Our approach leads to a convex optimization problem that is efficiently solvable. We analyze our approach under a natural generative model, and establish theoretical guarantees for recovering the underlying clusters. Our main result is a general theorem that applies to any given weight and distribution for the uncertainty. By optimizing over the weights, we derive a provably optimal weighting scheme, which matches the information theoretic lower bound up to logarithmic factors and leads to strong performance bounds in several specific settings. By optimizing over the uncertainty distribution, we show that nonuniform uncertainties can actually help. In particular, if the graph is built by spending a limited amount of resource to take measurement on each node pair, then it is beneficial to allocate the resource in a nonuniform fashion to obtain accurate measurements on a few pairs of nodes, rather than obtaining inaccurate measurements on many pairs. We provide simulation results that validate our theoretical findings.