Results 1  10
of
28
Negative Binomial Process Count and Mixture Modeling
, 2013
"... The seemingly disjoint problems of count and mixture modeling are united under the negative binomial (NB) process. A gamma process is employed to model the rate measure of a Poisson process, whose normalization provides a random probability measure for mixture modeling and whose marginalization lead ..."
Abstract

Cited by 17 (10 self)
 Add to MetaCart
The seemingly disjoint problems of count and mixture modeling are united under the negative binomial (NB) process. A gamma process is employed to model the rate measure of a Poisson process, whose normalization provides a random probability measure for mixture modeling and whose marginalization leads to an NB process for count modeling. A draw from the NB process consists of a Poisson distributed finite number of distinct atoms, each of which is associated with a logarithmic distributed number of data samples. We reveal relationships between various count and mixturemodeling distributions and construct a Poissonlogarithmic bivariate distribution that connects the NB and Chinese restaurant table distributions. Fundamental properties of the models are developed, and we derive efficient Bayesian inference. It is shown that with augmentation and normalization, the NB process and gammaNB process can be reduced to the Dirichlet process and hierarchical Dirichlet process, respectively. These relationships highlight theoretical, structural and computational advantages of the NB process. A variety of NB processes, including the betageometric, betaNB, markedbetaNB, markedgammaNB and zeroinflatedNB processes, with distinct sharing mechanisms, are also constructed. These models are applied to topic modeling, with connections made to existing algorithms under Poisson factor analysis. Example results show the importance of inferring both the NB dispersion and probability parameters.
Lognormal and Gamma Mixed Negative Binomial Regression
"... In regression analysis of counts, a lack of simple and efficient algorithms for posterior computation has made Bayesian approaches appear unattractive and thus underdeveloped. We propose a lognormal and gamma mixed negative binomial (NB) regression model for counts, and present efficient closedform ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
In regression analysis of counts, a lack of simple and efficient algorithms for posterior computation has made Bayesian approaches appear unattractive and thus underdeveloped. We propose a lognormal and gamma mixed negative binomial (NB) regression model for counts, and present efficient closedform Bayesian inference; unlike conventional Poisson models, the proposed approach has two free parameters to include two different kinds of random effects, and allows the incorporation of prior information, such as sparsity in the regression coefficients. By placing a gamma distribution prior on the NB dispersion parameter r, and connecting a lognormal distribution prior with the logit of the NB probability parameter p, efficient Gibbs sampling and variational Bayes inference are both developed. The closedform updates are obtained by exploiting conditional conjugacy via both a compound Poisson representation and a PolyaGamma distribution based data augmentation approach. The proposed Bayesian inference can be implemented routinely, while being easily generalizable to more complex settings involving multivariate dependence structures. The algorithms are illustrated using real examples. 1.
AugmentandConquer Negative Binomial Processes
"... By developing data augmentation methods unique to the negative binomial (NB) distribution, we unite seemingly disjoint count and mixture models under the NB process framework. We develop fundamental properties of the models and derive efficient Gibbs sampling inference. We show that the gammaNB pro ..."
Abstract

Cited by 10 (7 self)
 Add to MetaCart
(Show Context)
By developing data augmentation methods unique to the negative binomial (NB) distribution, we unite seemingly disjoint count and mixture models under the NB process framework. We develop fundamental properties of the models and derive efficient Gibbs sampling inference. We show that the gammaNB process can be reduced to the hierarchical Dirichlet process with normalization, highlighting its unique theoretical, structural and computational advantages. A variety of NB processes with distinct sharing mechanisms are constructed and applied to topic modeling, with connections to existing algorithms, showing the importance of inferring both the NB dispersion and probability parameters. 1
Infinite edge partition models for overlapping community detection and link prediction.
 In AISTATS,
, 2015
"... Abstract A hierarchical gamma process infinite edge partition model is proposed to factorize the binary adjacency matrix of an unweighted undirected relational network under a BernoulliPoisson link. The model describes both homophily and stochastic equivalence, and is scalable to big sparse networ ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
Abstract A hierarchical gamma process infinite edge partition model is proposed to factorize the binary adjacency matrix of an unweighted undirected relational network under a BernoulliPoisson link. The model describes both homophily and stochastic equivalence, and is scalable to big sparse networks by focusing its computation on pairs of linked nodes. It can not only discover overlapping communities and intercommunity interactions, but also predict missing edges. A simplified version omitting intercommunity interactions is also provided and we reveal its interesting connections to existing models. The number of communities is automatically inferred in a nonparametric Bayesian manner, and efficient inference via Gibbs sampling is derived using novel data augmentation techniques. Experimental results on four real networks demonstrate the models' scalability and stateoftheart performance.
Priors for random count matrices derived from a family of negative binomial processes. arXiv:1404.3331v2, 2014. A Proof for Theorem 1 Proof. Let us consider the process XG, conditional on G, given by XG(A) = ∑ k nk 1(ωk ∈ A). Now it is easy to see that E[
 j=1 pij = 1. B Proof for Corollary 3 This follows directly from Bayes’ rule, since p(ziz−i, n, γ0, ρ) = p(zi,z−i,nγ0,ρ)p(z−i,nγ0,ρ) , where p(zi, z −i, nγ0, ρ) = n−1 p(z−i, n−1γ0, ρ) γ0 ∫ ∞ se−sρ(ds)1(zi = l−i + 1) + l−i∑ k=1 ∫∞ sn
"... We define a family of probability distributions for random count matrices with a potentially unbounded number of rows and columns. The three distributions we consider are derived from the gammaPoisson, gammanegative binomial (GNB), and betanegative binomial (BNB) processes, which we refer to gene ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
(Show Context)
We define a family of probability distributions for random count matrices with a potentially unbounded number of rows and columns. The three distributions we consider are derived from the gammaPoisson, gammanegative binomial (GNB), and betanegative binomial (BNB) processes, which we refer to generically as a family of negativebinomial processes. Because the models lead to closedform update equations within the context of a Gibbs sampler, they are natural candidates for nonparametric Bayesian priors over count matrices. A key aspect of our analysis is the recognition that, although the random count matrices within the family are defined by a rowwise construction, their columns can be shown to be independent and identically distributed; this fact is used to derive explicit formulas for drawing all the columns at once. Moreover, by analyzing these matrices ’ combinatorial structure, we describe how to sequentially construct a columni.i.d. random count matrix one row at a time, and derive the predictive distribution of a new row count vector with previously unseen features. We describe the similarities and differences between the three priors, and argue that the greater flexibility of the GNB and BNB processes—especially their ability to model overdispersed, heavytailed count data—makes these well suited to a wide variety of realworld applications. As an example of our framework, we construct a naiveBayes text classifier to categorize a count vector to one of several existing random count matrices of different categories. The classifier supports an unbounded number of features, and unlike most existing methods, it does not require a predefined finite vocabulary to be shared by all the categories. Both the gamma and beta negative binomial processes are shown to significantly outperform the gammaPoisson process when applied to document categorization. The authors are with the Department of Information, Risk, and Operations Management
Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices.
 The 18th International Conference on Artificial Intelligence and Statistics (AISTATS2015), 38,
, 2015
"... Abstract A gamma process dynamic Poisson factor analysis model is proposed to factorize a dynamic count matrix, whose columns are sequentially observed count vectors. The model builds a novel Markov chain that sends the latent gamma random variables at time (t − 1) as the shape parameters of those ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
Abstract A gamma process dynamic Poisson factor analysis model is proposed to factorize a dynamic count matrix, whose columns are sequentially observed count vectors. The model builds a novel Markov chain that sends the latent gamma random variables at time (t − 1) as the shape parameters of those at time t, which are linked to observed or latent counts under the Poisson likelihood. The significant challenge of inferring the gamma shape parameters is fully addressed, using unique data augmentation and marginalization techniques for the negative binomial distribution. The same nonparametric Bayesian model also applies to the factorization of a dynamic binary matrix, via a BernoulliPoisson link that connects a binary observation to a latent count, with closedform conditional posteriors for the latent counts and efficient computation for sparse observations. We apply the model to text and music analysis, with stateoftheart results.
Betanegative binomial process and exchangeable random partitions for mixedmembership modeling.
 In NIPS,
, 2014
"... Abstract The betanegative binomial process (BNBP), an integervalued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
Abstract The betanegative binomial process (BNBP), an integervalued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been developed, current inference for the BNBP has to truncate the number of atoms of the beta process. This paper introduces an exchangeable partition probability function to explicitly describe how the BNBP clusters the data points of each group into a random number of exchangeable partitions, which are shared across all the groups. A fully collapsed Gibbs sampler is developed for the BNBP, leading to a novel nonparametric Bayesian topic model that is distinct from existing ones, with simple implementation, fast convergence, good mixing, and stateoftheart predictive performance.
A survey of nonexchangeable priors for Bayesian nonparametric models
, 2014
"... Dependent nonparametric processes extend distributions over measures, such as the Dirichlet process and the beta process, to give distributions over collections of measures, typically indexed by values in some covariate space. Such models are appropriate priors when exchangeability assumptions do ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Dependent nonparametric processes extend distributions over measures, such as the Dirichlet process and the beta process, to give distributions over collections of measures, typically indexed by values in some covariate space. Such models are appropriate priors when exchangeability assumptions do not hold, and instead we want our model to vary fluidly with some set of covariates. Since the concept of dependent nonparametric processes was formalized by MacEachern [1], there have been a number of models proposed and used in the statistics and machine learning literatures. Many of these models exhibit underlying similarities, an understanding of which, we hope, will help in selecting an appropriate prior, developing new models, and leveraging inference techniques.
A unifying representation for a class of dependent random measures
, 1211
"... We present a general construction for dependent random measures based on thinning Poisson processes on an augmented space. The framework is not restricted to dependent versions of a specific nonparametric model, but can be applied to all models that can be represented using completely random measure ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We present a general construction for dependent random measures based on thinning Poisson processes on an augmented space. The framework is not restricted to dependent versions of a specific nonparametric model, but can be applied to all models that can be represented using completely random measures. Several existing dependent random measures can be seen as specific cases of this framework. Interesting properties of the resulting measures are derived and the efficacy of the framework is demonstrated by constructing a covariatedependent latent feature model and topic model that obtain superior predictive performance. 1
Bayesian poisson tensor factorization for inferring multilateral relations from sparse dyadic event counts.
 In KDD,
, 2015
"... ABSTRACT We present a Bayesian tensor factorization model for inferring latent group structures from dynamic pairwise interaction patterns. For decades, political scientists have collected and analyzed records of the form "country i took action a toward country j at time t"known as dyadi ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
ABSTRACT We present a Bayesian tensor factorization model for inferring latent group structures from dynamic pairwise interaction patterns. For decades, political scientists have collected and analyzed records of the form "country i took action a toward country j at time t"known as dyadic eventsin order to form and test theories of international relations. We represent these event data as a tensor of counts and develop Bayesian Poisson tensor factorization to infer a lowdimensional, interpretable representation of their salient patterns. We demonstrate that our model's predictive performance is better than that of standard nonnegative tensor factorization methods. We also provide a comparison of our variational updates to their maximum likelihood counterparts. In doing so, we identify a better way to form point estimates of the latent factors than that typically used in Bayesian Poisson matrix factorization. Finally, we showcase our model as an exploratory analysis tool for political scientists. We show that the inferred latent factor matrices capture interpretable multilateral relations that both conform to and inform our knowledge of international affairs. Categories and Subject Descriptors Keywords Poisson tensor factorization, Bayesian inference, dyadic data, international relations Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.