Results 1 - 10
of
52
Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1
"... A dependent hierarchical beta process (dHBP) is developed as a prior for data that may be represented in terms of a sparse set of latent features, with covariate-dependent feature usage. The dHBP is applicable to general covariates and data models, imposing that signals with similar covariates are l ..."
Abstract
-
Cited by 24 (11 self)
- Add to MetaCart
(Show Context)
A dependent hierarchical beta process (dHBP) is developed as a prior for data that may be represented in terms of a sparse set of latent features, with covariate-dependent feature usage. The dHBP is applicable to general covariates and data models, imposing that signals with similar covariates are likely to be manifested in terms of similar features. Coupling the dHBP with the Bernoulli process, and upon marginalizing out the dHBP, the model may be interpreted as a covariatedependent hierarchical Indian buffet process. As applications, we consider interpolation and denoising of an image, with covariates defined by the location of image patches within an image. Two types of noise models are considered: (i) typical white Gaussian noise; and (ii) spiky noise of arbitrary amplitude, distributed uniformly at random. In these examples, the features correspond to the atoms of a dictionary, learned based upon the data under test (without a priori training data). State-of-the-art performance is demonstrated, with efficient inference using hybrid Gibbs, Metropolis-Hastings and slice sampling.
Bayesian gaussian copula factor models for mixed data. Arxiv Preprint arXiv:1111.0317
, 2011
"... Gaussian factor models have proven widely useful for parsimoniously characterizing dependence in multivariate data. There is a rich literature on their extension to mixed categorical and continuous variables, using latent Gaussian variables or through generalized latent trait models acommodating mea ..."
Abstract
-
Cited by 12 (3 self)
- Add to MetaCart
(Show Context)
Gaussian factor models have proven widely useful for parsimoniously characterizing dependence in multivariate data. There is a rich literature on their extension to mixed categorical and continuous variables, using latent Gaussian variables or through generalized latent trait models acommodating measurements in the exponential family. However, when generalizing to non-Gaussian measured variables the latent variables typically influence both the dependence structure and the form of the marginal distributions, complicating interpretation and introducing artifacts. To address this problem we propose a novel class of Bayesian Gaussian copula factor models which decouple the latent factors from the marginal distributions. A semiparametric specification for the marginals based on the extended rank likelihood yields straightforward implementation and substantial computational gains. We provide new theoretical and empirical justifications for using this likelihood in Bayesian inference. We propose new default priors for the factor loadings and develop efficient parameter-expanded Gibbs sampling for posterior computation. The methods are evaluated through simulations and applied to a dataset in political science. The models in this paper are implemented in the R package bfa.
Bayesian Nonparametric Covariance Regression
, 1101
"... Summary. Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression mode ..."
Abstract
-
Cited by 11 (1 self)
- Add to MetaCart
Summary. Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression models, which allow an unknown p × p covariance matrix to change flexibly with predictors. The proposed modeling framework induces a prior on a collection of covariance matrices indexed by predictors through priors for predictor-dependent loadings matrices in a factor model. In particular, the predictor-dependent loadings are characterized as a sparse combination of a collection of unknown dictionary functions (e.g, Gaussian process random functions). The induced covariance is then a regularized quadratic function of these dictionary elements. Our proposed framework leads to a highly-flexible, but computationally tractable formulation with simple conjugate posterior updates that can readily handle missing data. Theoretical properties are discussed and the methods are illustrated through simulations studies and an application to the Google Flu Trends data.
A Sparse Factor-Analytic Probit Model for Congressional Voting Patterns
, 2010
"... This paper adapts sparse factor models for exploring covariation in multivariate binary data, with an application to measuring latent factors in U.S. Congressional roll-call voting patterns. We focus on the advantages of using formal probability models for inference in this context, drawing parallel ..."
Abstract
-
Cited by 11 (0 self)
- Add to MetaCart
(Show Context)
This paper adapts sparse factor models for exploring covariation in multivariate binary data, with an application to measuring latent factors in U.S. Congressional roll-call voting patterns. We focus on the advantages of using formal probability models for inference in this context, drawing parallels with the seminal findings of Poole and Rosenthal (1991). Our methodological innovation is to introduce a sparsity prior on a latent covariance matrix that descibes common factors in binary and ordinal outcomes. We apply the method to analyze sixty years of roll-call votes from the United States Senate, focusing primarily on the interpretation of posterior summaries that arise from the model. We also explore two advantages of our approach over traditional factor analysis. First, patterns of sparsity in the factor-loadings matrix often have natural subject-matter interpretations. For the roll-call vote data, the sparsity prior enables one to conduct a formal hypothesis test about whether a given vote can be explained exclusively by partisanship. Moreover, the factor scores provide a novel way of ranking Senators by the partisanship of their voting patterns. Second, by introducing sparsity into existing factor-analytic probit models, we effect a favorable bias–variance tradeoff in estimating the latent covariance matrix. Our model can thus be used in situations where the number of variables is very large relative to the number of observations. Key words: covariance estimation; factor models; multivariate probit models; voting patterns 1 Corresponding author.
Scalable nonparametric multiway data analysis
- In International Conference on Artificial Intelligence and Statistics
, 2015
"... Abstract Multiway data analysis deals with multiway arrays, i.e., tensors, and the goal is twofold: predicting missing entries by modeling the interactions between array elements and discovering hidden patterns, such as clusters or communities in each mode. Despite the success of existing tensor fa ..."
Abstract
-
Cited by 5 (1 self)
- Add to MetaCart
(Show Context)
Abstract Multiway data analysis deals with multiway arrays, i.e., tensors, and the goal is twofold: predicting missing entries by modeling the interactions between array elements and discovering hidden patterns, such as clusters or communities in each mode. Despite the success of existing tensor factorization approaches, they are either unable to capture nonlinear interactions, or computationally expensive to handle massive data. In addition, most of the existing methods lack a principled way to discover latent clusters, which is important for better understanding of the data. To address these issues, we propose a scalable nonparametric tensor decomposition model. It employs Dirichlet process mixture (DPM) prior to model the latent clusters; it uses local Gaussian processes (GPs) to capture nonlinear relationships and to improve scalability. An efficient online variational Bayes Expectation-Maximization algorithm is proposed to learn the model. Experiments on both synthetic and real-world data show that the proposed model is able to discover latent clusters with higher prediction accuracy than competitive methods. Furthermore, the proposed model obtains significantly better predictive performance than the state-of-the-art large scale tensor decomposition algorithm, GigaTensor, on two large datasets with billions of entries.
Bayesian factorizations of big sparse tensors
, 2013
"... It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of exten-sions to the tensor case in statistics. The most common low rank tensor factorization relies on par ..."
Abstract
-
Cited by 5 (0 self)
- Add to MetaCart
It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of exten-sions to the tensor case in statistics. The most common low rank tensor factorization relies on parallel factor analysis (PARAFAC), which expresses a rank k tensor as a sum of rank one tensors. When observations are only available for a tiny subset of the cells of a big tensor, the low rank assumption is not sufficient and PARAFAC has poor performance. We induce an additional layer of dimension reduction by allowing the effective rank to vary across dimensions of the table. For concreteness, we focus on a contingency table application. Taking a Bayesian approach, we place priors on terms in the factorization and develop an efficient Gibbs sampler for posterior computation. Theory is provided showing posterior concentration rates in high-dimensional settings, and the methods are shown to have excellent performance in simulations and several real data applications.
Leveraging Features and Networks for Probabilistic Tensor Decomposition
"... We present a probabilistic model for tensor decomposi-tion where one or more tensor modes may have side-information about the mode entities in form of their features and/or their adjacency network. We consider a Bayesian approach based on the Canonical PARAFAC (CP) decomposition and enrich this sing ..."
Abstract
-
Cited by 4 (1 self)
- Add to MetaCart
(Show Context)
We present a probabilistic model for tensor decomposi-tion where one or more tensor modes may have side-information about the mode entities in form of their features and/or their adjacency network. We consider a Bayesian approach based on the Canonical PARAFAC (CP) decomposition and enrich this single-layer decom-position approach with a two-layer decomposition. The second layer fits a factor model for each layer-one fac-tor matrix and models the factor matrix via the mode entities ’ features and/or the network between the mode entities. The second-layer decomposition of each factor matrix also learns a binary latent representation for the entities of that mode, which can be useful in its own right. Our model can handle both continuous as well as binary tensor observations. Another appealing aspect of our model is the simplicity of the model inference, with easy-to-sample Gibbs updates. We demonstrate the re-sults of our model on several benchmarks datasets, con-sisting of both real and binary tensors.
Modeling Correlated Arrival Events with Latent Semi-Markov Processes
"... The analysis of correlated point process data has wide applications, ranging from biomedical re-search to network analysis. In this work, we model such data as generated by a latent col-lection of continuous-time binary semi-Markov processes, corresponding to external events ap-pearing and disappear ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
The analysis of correlated point process data has wide applications, ranging from biomedical re-search to network analysis. In this work, we model such data as generated by a latent col-lection of continuous-time binary semi-Markov processes, corresponding to external events ap-pearing and disappearing. A continuous-time modeling framework is more appropriate for multichannel point process data than a binning approach requiring time discretization, and we show connections between our model and recent ideas from the discrete-time literature. We de-scribe an efficient MCMC algorithm for posterior inference, and apply our ideas to both synthetic data and a real-world biometrics application. 1.
Dynamic rank factor model for text streams
- In Proc. of NIPS
, 2014
"... We propose a semi-parametric and dynamic rank factor model for topic model-ing, capable of (i) discovering topic prevalence over time, and (ii) learning con-temporary multi-scale dependence structures, providing topic and word correla-tions as a byproduct. The high-dimensional and time-evolving ordi ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
We propose a semi-parametric and dynamic rank factor model for topic model-ing, capable of (i) discovering topic prevalence over time, and (ii) learning con-temporary multi-scale dependence structures, providing topic and word correla-tions as a byproduct. The high-dimensional and time-evolving ordinal/rank ob-servations (such as word counts), after an arbitrary monotone transformation, are well accommodated through an underlying dynamic sparse factor model. The framework naturally admits heavy-tailed innovations, capable of inferring abrupt temporal jumps in the importance of topics. Posterior inference is performed through straightforward Gibbs sampling, based on the forward-filtering backward-sampling algorithm. Moreover, an efficient data subsampling scheme is leveraged to speed up inference on massive datasets. The modeling framework is illustrated
