Results 1  10
of
52
Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1
"... A dependent hierarchical beta process (dHBP) is developed as a prior for data that may be represented in terms of a sparse set of latent features, with covariatedependent feature usage. The dHBP is applicable to general covariates and data models, imposing that signals with similar covariates are l ..."
Abstract

Cited by 24 (11 self)
 Add to MetaCart
(Show Context)
A dependent hierarchical beta process (dHBP) is developed as a prior for data that may be represented in terms of a sparse set of latent features, with covariatedependent feature usage. The dHBP is applicable to general covariates and data models, imposing that signals with similar covariates are likely to be manifested in terms of similar features. Coupling the dHBP with the Bernoulli process, and upon marginalizing out the dHBP, the model may be interpreted as a covariatedependent hierarchical Indian buffet process. As applications, we consider interpolation and denoising of an image, with covariates defined by the location of image patches within an image. Two types of noise models are considered: (i) typical white Gaussian noise; and (ii) spiky noise of arbitrary amplitude, distributed uniformly at random. In these examples, the features correspond to the atoms of a dictionary, learned based upon the data under test (without a priori training data). Stateoftheart performance is demonstrated, with efficient inference using hybrid Gibbs, MetropolisHastings and slice sampling.
Bayesian gaussian copula factor models for mixed data. Arxiv Preprint arXiv:1111.0317
, 2011
"... Gaussian factor models have proven widely useful for parsimoniously characterizing dependence in multivariate data. There is a rich literature on their extension to mixed categorical and continuous variables, using latent Gaussian variables or through generalized latent trait models acommodating mea ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
Gaussian factor models have proven widely useful for parsimoniously characterizing dependence in multivariate data. There is a rich literature on their extension to mixed categorical and continuous variables, using latent Gaussian variables or through generalized latent trait models acommodating measurements in the exponential family. However, when generalizing to nonGaussian measured variables the latent variables typically influence both the dependence structure and the form of the marginal distributions, complicating interpretation and introducing artifacts. To address this problem we propose a novel class of Bayesian Gaussian copula factor models which decouple the latent factors from the marginal distributions. A semiparametric specification for the marginals based on the extended rank likelihood yields straightforward implementation and substantial computational gains. We provide new theoretical and empirical justifications for using this likelihood in Bayesian inference. We propose new default priors for the factor loadings and develop efficient parameterexpanded Gibbs sampling for posterior computation. The methods are evaluated through simulations and applied to a dataset in political science. The models in this paper are implemented in the R package bfa.
Bayesian Nonparametric Covariance Regression
, 1101
"... Summary. Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression mode ..."
Abstract

Cited by 11 (1 self)
 Add to MetaCart
Summary. Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression models, which allow an unknown p × p covariance matrix to change flexibly with predictors. The proposed modeling framework induces a prior on a collection of covariance matrices indexed by predictors through priors for predictordependent loadings matrices in a factor model. In particular, the predictordependent loadings are characterized as a sparse combination of a collection of unknown dictionary functions (e.g, Gaussian process random functions). The induced covariance is then a regularized quadratic function of these dictionary elements. Our proposed framework leads to a highlyflexible, but computationally tractable formulation with simple conjugate posterior updates that can readily handle missing data. Theoretical properties are discussed and the methods are illustrated through simulations studies and an application to the Google Flu Trends data.
A Sparse FactorAnalytic Probit Model for Congressional Voting Patterns
, 2010
"... This paper adapts sparse factor models for exploring covariation in multivariate binary data, with an application to measuring latent factors in U.S. Congressional rollcall voting patterns. We focus on the advantages of using formal probability models for inference in this context, drawing parallel ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
(Show Context)
This paper adapts sparse factor models for exploring covariation in multivariate binary data, with an application to measuring latent factors in U.S. Congressional rollcall voting patterns. We focus on the advantages of using formal probability models for inference in this context, drawing parallels with the seminal findings of Poole and Rosenthal (1991). Our methodological innovation is to introduce a sparsity prior on a latent covariance matrix that descibes common factors in binary and ordinal outcomes. We apply the method to analyze sixty years of rollcall votes from the United States Senate, focusing primarily on the interpretation of posterior summaries that arise from the model. We also explore two advantages of our approach over traditional factor analysis. First, patterns of sparsity in the factorloadings matrix often have natural subjectmatter interpretations. For the rollcall vote data, the sparsity prior enables one to conduct a formal hypothesis test about whether a given vote can be explained exclusively by partisanship. Moreover, the factor scores provide a novel way of ranking Senators by the partisanship of their voting patterns. Second, by introducing sparsity into existing factoranalytic probit models, we effect a favorable bias–variance tradeoff in estimating the latent covariance matrix. Our model can thus be used in situations where the number of variables is very large relative to the number of observations. Key words: covariance estimation; factor models; multivariate probit models; voting patterns 1 Corresponding author.
Scalable nonparametric multiway data analysis
 In International Conference on Artificial Intelligence and Statistics
, 2015
"... Abstract Multiway data analysis deals with multiway arrays, i.e., tensors, and the goal is twofold: predicting missing entries by modeling the interactions between array elements and discovering hidden patterns, such as clusters or communities in each mode. Despite the success of existing tensor fa ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract Multiway data analysis deals with multiway arrays, i.e., tensors, and the goal is twofold: predicting missing entries by modeling the interactions between array elements and discovering hidden patterns, such as clusters or communities in each mode. Despite the success of existing tensor factorization approaches, they are either unable to capture nonlinear interactions, or computationally expensive to handle massive data. In addition, most of the existing methods lack a principled way to discover latent clusters, which is important for better understanding of the data. To address these issues, we propose a scalable nonparametric tensor decomposition model. It employs Dirichlet process mixture (DPM) prior to model the latent clusters; it uses local Gaussian processes (GPs) to capture nonlinear relationships and to improve scalability. An efficient online variational Bayes ExpectationMaximization algorithm is proposed to learn the model. Experiments on both synthetic and realworld data show that the proposed model is able to discover latent clusters with higher prediction accuracy than competitive methods. Furthermore, the proposed model obtains significantly better predictive performance than the stateoftheart large scale tensor decomposition algorithm, GigaTensor, on two large datasets with billions of entries.
Bayesian factorizations of big sparse tensors
, 2013
"... It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of extensions to the tensor case in statistics. The most common low rank tensor factorization relies on par ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of extensions to the tensor case in statistics. The most common low rank tensor factorization relies on parallel factor analysis (PARAFAC), which expresses a rank k tensor as a sum of rank one tensors. When observations are only available for a tiny subset of the cells of a big tensor, the low rank assumption is not sufficient and PARAFAC has poor performance. We induce an additional layer of dimension reduction by allowing the effective rank to vary across dimensions of the table. For concreteness, we focus on a contingency table application. Taking a Bayesian approach, we place priors on terms in the factorization and develop an efficient Gibbs sampler for posterior computation. Theory is provided showing posterior concentration rates in highdimensional settings, and the methods are shown to have excellent performance in simulations and several real data applications.
Leveraging Features and Networks for Probabilistic Tensor Decomposition
"... We present a probabilistic model for tensor decomposition where one or more tensor modes may have sideinformation about the mode entities in form of their features and/or their adjacency network. We consider a Bayesian approach based on the Canonical PARAFAC (CP) decomposition and enrich this sing ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
We present a probabilistic model for tensor decomposition where one or more tensor modes may have sideinformation about the mode entities in form of their features and/or their adjacency network. We consider a Bayesian approach based on the Canonical PARAFAC (CP) decomposition and enrich this singlelayer decomposition approach with a twolayer decomposition. The second layer fits a factor model for each layerone factor matrix and models the factor matrix via the mode entities ’ features and/or the network between the mode entities. The secondlayer decomposition of each factor matrix also learns a binary latent representation for the entities of that mode, which can be useful in its own right. Our model can handle both continuous as well as binary tensor observations. Another appealing aspect of our model is the simplicity of the model inference, with easytosample Gibbs updates. We demonstrate the results of our model on several benchmarks datasets, consisting of both real and binary tensors.
Modeling Correlated Arrival Events with Latent SemiMarkov Processes
"... The analysis of correlated point process data has wide applications, ranging from biomedical research to network analysis. In this work, we model such data as generated by a latent collection of continuoustime binary semiMarkov processes, corresponding to external events appearing and disappear ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
The analysis of correlated point process data has wide applications, ranging from biomedical research to network analysis. In this work, we model such data as generated by a latent collection of continuoustime binary semiMarkov processes, corresponding to external events appearing and disappearing. A continuoustime modeling framework is more appropriate for multichannel point process data than a binning approach requiring time discretization, and we show connections between our model and recent ideas from the discretetime literature. We describe an efficient MCMC algorithm for posterior inference, and apply our ideas to both synthetic data and a realworld biometrics application. 1.
Dynamic rank factor model for text streams
 In Proc. of NIPS
, 2014
"... We propose a semiparametric and dynamic rank factor model for topic modeling, capable of (i) discovering topic prevalence over time, and (ii) learning contemporary multiscale dependence structures, providing topic and word correlations as a byproduct. The highdimensional and timeevolving ordi ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We propose a semiparametric and dynamic rank factor model for topic modeling, capable of (i) discovering topic prevalence over time, and (ii) learning contemporary multiscale dependence structures, providing topic and word correlations as a byproduct. The highdimensional and timeevolving ordinal/rank observations (such as word counts), after an arbitrary monotone transformation, are well accommodated through an underlying dynamic sparse factor model. The framework naturally admits heavytailed innovations, capable of inferring abrupt temporal jumps in the importance of topics. Posterior inference is performed through straightforward Gibbs sampling, based on the forwardfiltering backwardsampling algorithm. Moreover, an efficient data subsampling scheme is leveraged to speed up inference on massive datasets. The modeling framework is illustrated