Results 1 - 10
of
43
Tensor decompositions for learning latent variable models
, 2014
"... This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their low-order observable mo ..."
Abstract
-
Cited by 83 (7 self)
- Add to MetaCart
This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthog-onal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin’s perturbation theorem for the singular vectors of matrices. This implies a ro-bust and computationally tractable estimation approach for several popular latent variable models.
Estimating and understanding exponential random graph models
, 2011
"... We introduce a new method for estimating the parameters of exponential random graph models. The method is based on a large-deviations approximation to the normalizing constant shown to be consistent using theory developed by Chatterjee and Varadhan [15]. The theory explains a host of difficulties e ..."
Abstract
-
Cited by 50 (1 self)
- Add to MetaCart
We introduce a new method for estimating the parameters of exponential random graph models. The method is based on a large-deviations approximation to the normalizing constant shown to be consistent using theory developed by Chatterjee and Varadhan [15]. The theory explains a host of difficulties encountered by applied workers: many distinct models have essentially the same MLE, rendering the problems “practically” ill-posed. We give the first rigorous proofs of “degeneracy” observed in these models. Here, almost all graphs have essentially no edges or are essentially complete. We supplement recent work of Bhamidi, Bresler and Sly [6] showing that for many models, the extra sufficient statistics are useless: most realizations look like the results of a simple Erdős–Rényi model. We also find classes of models where the limiting graphs differ from Erdős–Rényi graphs and begin to make the link to models where the natural parameters alternate in sign.
Matrix estimation by universal singular value thresholding
, 2012
"... Abstract. Consider the problem of estimating the entries of a large matrix, when the observed entries are noisy versions of a small ran-dom fraction of the original entries. This problem has received wide-spread attention in recent times, especially after the pioneering works of Emmanuel Candès and ..."
Abstract
-
Cited by 28 (0 self)
- Add to MetaCart
(Show Context)
Abstract. Consider the problem of estimating the entries of a large matrix, when the observed entries are noisy versions of a small ran-dom fraction of the original entries. This problem has received wide-spread attention in recent times, especially after the pioneering works of Emmanuel Candès and collaborators. This paper introduces a sim-ple estimation procedure, called Universal Singular Value Thresholding (USVT), that works for any matrix that has ‘a little bit of structure’. Surprisingly, this simple estimator achieves the minimax error rate up to a constant factor. The method is applied to solve problems related to low rank matrix estimation, blockmodels, distance matrix completion, latent space models, positive definite matrix completion, graphon esti-mation, and generalized Bradley–Terry models for pairwise comparison. 1.
More uses of exchangeability: Representations of complex random structures
- Probability and Mathematical Genetics: Papers in Honour of Sir
, 2010
"... We review old and new uses of exchangeability, emphasizing the general theme of exchangeable representations of complex random structures. Illustrations of this theme include processes of stochastic coalescence and fragmentation; continuum random trees; second-order limits of distances in random gra ..."
Abstract
-
Cited by 18 (2 self)
- Add to MetaCart
(Show Context)
We review old and new uses of exchangeability, emphasizing the general theme of exchangeable representations of complex random structures. Illustrations of this theme include processes of stochastic coalescence and fragmentation; continuum random trees; second-order limits of distances in random graphs; isometry classes of metric spaces with probability measures; limits of dense random graphs; and more sophisticated uses in finitary combinatorics.
Graphons, cut norm and distance, couplings and rearrangements
, 2010
"... We give a survey of basic results on the cut norm and cut metric for graphons (and sometimes more general kernels), with emphasis on the equivalence problem. The main results are not new, but we add various technical complements. We allow graphons on general probability spaces whenever possible. We ..."
Abstract
-
Cited by 13 (5 self)
- Add to MetaCart
We give a survey of basic results on the cut norm and cut metric for graphons (and sometimes more general kernels), with emphasis on the equivalence problem. The main results are not new, but we add various technical complements. We allow graphons on general probability spaces whenever possible. We also give some new results for {0,1}-valued graphons.
Deducing the density Hales-Jewett theorem from an infinitary removal lemma
- J. Theoret. Probab
"... removal lemma ..."
(Show Context)
Interval graph limits
, 2011
"... We work out the graph limit theory for dense interval graphs. The theory developed departs from the usual description of a graph limit as a symmetric function W (x, y) on the unit square, with x and y uniform on the interval (0, 1). Instead, we fix a W and change the underlying distribution of the c ..."
Abstract
-
Cited by 8 (5 self)
- Add to MetaCart
(Show Context)
We work out the graph limit theory for dense interval graphs. The theory developed departs from the usual description of a graph limit as a symmetric function W (x, y) on the unit square, with x and y uniform on the interval (0, 1). Instead, we fix a W and change the underlying distribution of the coordinates x and y. We find choices such that our limits are continuous. Connections to random interval graphs are given, including some examples. We also show a continuity result for the chromatic number and clique number of interval graphs. Some results on uniqueness of the limit description are given for general graph limits.
Exchangeability and Continuum Limits of Discrete Random Structures
"... Exchangeablerepresentationsofcomplexrandomstructuresareusefulinseveral ways, in particular providing a moderately general way to derive continuum limits of discrete random structures. I shall describe an old example (continuum random trees) and a more recent example (dense graph limits). Thinking th ..."
Abstract
-
Cited by 8 (1 self)
- Add to MetaCart
(Show Context)
Exchangeablerepresentationsofcomplexrandomstructuresareusefulinseveral ways, in particular providing a moderately general way to derive continuum limits of discrete random structures. I shall describe an old example (continuum random trees) and a more recent example (dense graph limits). Thinking this way about road routes suggests challenging new problems in the plane. Mathematics Subject Classification (2000). Primary 60G09; Secondary 60C05. This write-up follows the style of the ICM talk, presented as 5 episodes in the development of a topic over the last 80 years. • Exchangeability and de Finetti’s theorem (1930s- 50s) • Structure theory for partially exchangeable arrays (1980s) • A general program for continuum limits of discrete random structures, illustrated by trees (1990s) • 3 recent “pure math ” developments (2000s) • Road routes from this viewpoint (2010s) An expanded version of the material in sections 1-4 appears as a longer survey
