Results 1  10
of
17
Posterior contraction in sparse Bayesian factor models for massive covariance matrices
, 2012
"... ar ..."
(Show Context)
Infinite edge partition models for overlapping community detection and link prediction.
 In AISTATS,
, 2015
"... Abstract A hierarchical gamma process infinite edge partition model is proposed to factorize the binary adjacency matrix of an unweighted undirected relational network under a BernoulliPoisson link. The model describes both homophily and stochastic equivalence, and is scalable to big sparse networ ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
Abstract A hierarchical gamma process infinite edge partition model is proposed to factorize the binary adjacency matrix of an unweighted undirected relational network under a BernoulliPoisson link. The model describes both homophily and stochastic equivalence, and is scalable to big sparse networks by focusing its computation on pairs of linked nodes. It can not only discover overlapping communities and intercommunity interactions, but also predict missing edges. A simplified version omitting intercommunity interactions is also provided and we reveal its interesting connections to existing models. The number of communities is automatically inferred in a nonparametric Bayesian manner, and efficient inference via Gibbs sampling is derived using novel data augmentation techniques. Experimental results on four real networks demonstrate the models' scalability and stateoftheart performance.
Bayesian nonparametric poisson factorization for recommendation systems
 In AISTATS
, 2014
"... ..."
(Show Context)
DirichletLaplace priors for optimal shrinkage. arXiv preprint arXiv:1401.5398
, 2014
"... Penalized regression methods, such as L1 regularization, are routinely used in highdimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is routinely induced through twocomponent mixture priors having a pro ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
Penalized regression methods, such as L1 regularization, are routinely used in highdimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is routinely induced through twocomponent mixture priors having a probability mass at zero, but such priors encounter daunting computational problems in high dimensions. This has motivated an amazing variety of continuous shrinkage priors, which can be expressed as globallocal scale mixtures of Gaussians, facilitating computation. In sharp contrast to the frequentist literature, little is known about the properties of such priors and the convergence and concentration of the corresponding posterior distribution. In this article, we propose a new class of Dirichlet– Laplace (DL) priors, which possess optimal posterior concentration and lead to efficient posterior computation exploiting results from normalized random measure theory. Finite sample performance of Dirichlet–Laplace priors relative to alternatives is assessed in simulated and real data examples.
Priors for random count matrices derived from a family of negative binomial processes. arXiv:1404.3331v2, 2014. A Proof for Theorem 1 Proof. Let us consider the process XG, conditional on G, given by XG(A) = ∑ k nk 1(ωk ∈ A). Now it is easy to see that E[
 j=1 pij = 1. B Proof for Corollary 3 This follows directly from Bayes’ rule, since p(ziz−i, n, γ0, ρ) = p(zi,z−i,nγ0,ρ)p(z−i,nγ0,ρ) , where p(zi, z −i, nγ0, ρ) = n−1 p(z−i, n−1γ0, ρ) γ0 ∫ ∞ se−sρ(ds)1(zi = l−i + 1) + l−i∑ k=1 ∫∞ sn
"... We define a family of probability distributions for random count matrices with a potentially unbounded number of rows and columns. The three distributions we consider are derived from the gammaPoisson, gammanegative binomial (GNB), and betanegative binomial (BNB) processes, which we refer to gene ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
We define a family of probability distributions for random count matrices with a potentially unbounded number of rows and columns. The three distributions we consider are derived from the gammaPoisson, gammanegative binomial (GNB), and betanegative binomial (BNB) processes, which we refer to generically as a family of negativebinomial processes. Because the models lead to closedform update equations within the context of a Gibbs sampler, they are natural candidates for nonparametric Bayesian priors over count matrices. A key aspect of our analysis is the recognition that, although the random count matrices within the family are defined by a rowwise construction, their columns can be shown to be independent and identically distributed; this fact is used to derive explicit formulas for drawing all the columns at once. Moreover, by analyzing these matrices ’ combinatorial structure, we describe how to sequentially construct a columni.i.d. random count matrix one row at a time, and derive the predictive distribution of a new row count vector with previously unseen features. We describe the similarities and differences between the three priors, and argue that the greater flexibility of the GNB and BNB processes—especially their ability to model overdispersed, heavytailed count data—makes these well suited to a wide variety of realworld applications. As an example of our framework, we construct a naiveBayes text classifier to categorize a count vector to one of several existing random count matrices of different categories. The classifier supports an unbounded number of features, and unlike most existing methods, it does not require a predefined finite vocabulary to be shared by all the categories. Both the gamma and beta negative binomial processes are shown to significantly outperform the gammaPoisson process when applied to document categorization. The authors are with the Department of Information, Risk, and Operations Management
Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices.
 The 18th International Conference on Artificial Intelligence and Statistics (AISTATS2015), 38,
, 2015
"... Abstract A gamma process dynamic Poisson factor analysis model is proposed to factorize a dynamic count matrix, whose columns are sequentially observed count vectors. The model builds a novel Markov chain that sends the latent gamma random variables at time (t − 1) as the shape parameters of those ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
Abstract A gamma process dynamic Poisson factor analysis model is proposed to factorize a dynamic count matrix, whose columns are sequentially observed count vectors. The model builds a novel Markov chain that sends the latent gamma random variables at time (t − 1) as the shape parameters of those at time t, which are linked to observed or latent counts under the Poisson likelihood. The significant challenge of inferring the gamma shape parameters is fully addressed, using unique data augmentation and marginalization techniques for the negative binomial distribution. The same nonparametric Bayesian model also applies to the factorization of a dynamic binary matrix, via a BernoulliPoisson link that connects a binary observation to a latent count, with closedform conditional posteriors for the latent counts and efficient computation for sparse observations. We apply the model to text and music analysis, with stateoftheart results.
Betanegative binomial process and exchangeable random partitions for mixedmembership modeling.
 In NIPS,
, 2014
"... Abstract The betanegative binomial process (BNBP), an integervalued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
Abstract The betanegative binomial process (BNBP), an integervalued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been developed, current inference for the BNBP has to truncate the number of atoms of the beta process. This paper introduces an exchangeable partition probability function to explicitly describe how the BNBP clusters the data points of each group into a random number of exchangeable partitions, which are shared across all the groups. A fully collapsed Gibbs sampler is developed for the BNBP, leading to a novel nonparametric Bayesian topic model that is distinct from existing ones, with simple implementation, fast convergence, good mixing, and stateoftheart predictive performance.
Scalable Deep Poisson Factor Analysis for Topic Modeling
"... A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multilayer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of w ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multilayer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of which selects topics for use in a Poisson factor analysis model. Under this setting, topics live on the bottom layer of the model, while the deep specification serves as a flexible prior for revealing topic structure. Scalable inference algorithms are derived by applying Bayesian conditional density filtering algorithm, in addition to extending recently proposed work on stochastic gradient thermostats. Experimental results on several corpora show that the proposed approach readily handles very large collections of text documents, infers structured topic representations, and obtains superior test perplexities when compared with related models. 1.
Bayesian poisson tensor factorization for inferring multilateral relations from sparse dyadic event counts.
 In KDD,
, 2015
"... ABSTRACT We present a Bayesian tensor factorization model for inferring latent group structures from dynamic pairwise interaction patterns. For decades, political scientists have collected and analyzed records of the form "country i took action a toward country j at time t"known as dyadi ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
ABSTRACT We present a Bayesian tensor factorization model for inferring latent group structures from dynamic pairwise interaction patterns. For decades, political scientists have collected and analyzed records of the form "country i took action a toward country j at time t"known as dyadic eventsin order to form and test theories of international relations. We represent these event data as a tensor of counts and develop Bayesian Poisson tensor factorization to infer a lowdimensional, interpretable representation of their salient patterns. We demonstrate that our model's predictive performance is better than that of standard nonnegative tensor factorization methods. We also provide a comparison of our variational updates to their maximum likelihood counterparts. In doing so, we identify a better way to form point estimates of the latent factors than that typically used in Bayesian Poisson matrix factorization. Finally, we showcase our model as an exploratory analysis tool for political scientists. We show that the inferred latent factor matrices capture interpretable multilateral relations that both conform to and inform our knowledge of international affairs. Categories and Subject Descriptors Keywords Poisson tensor factorization, Bayesian inference, dyadic data, international relations Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.
Generalized Negative Binomial Processes and the Representation of Cluster Structures
, 2013
"... The paper introduces the concept of a cluster structure to define a joint distribution of the sample size and its exchangeable random partitions. The cluster structure allows the probability distribution of the random partitions of a subset of the sample to be dependent on the sample size, a feature ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
The paper introduces the concept of a cluster structure to define a joint distribution of the sample size and its exchangeable random partitions. The cluster structure allows the probability distribution of the random partitions of a subset of the sample to be dependent on the sample size, a feature not presented in a partition structure. A generalized negative binomial process countmixture model is proposed to generate a cluster structure, where in the prior the number of clusters is finite and Poisson distributed and the cluster sizes follow a truncated negative binomial distribution. The number and sizes of clusters can be controlled to exhibit distinct asymptotic behaviors. Unique model properties are illustrated with example clustering results using a generalized Pólya urn sampling scheme. The paper provides new methods to generate exchangeable random partitions and to control both the clusternumber and clustersize distributions.