Results 1  10
of
11
Bayesian Canonical Correlation Analysis
 JOURNAL OF MACHINE LEARNING RESEARCH 14 (2013) 9651003
, 2013
"... Canonical correlation analysis (CCA) is a classical method for seeking correlations between two multivariate data sets. During the last ten years, it has received more and more attention in the machine learning community in the form of novel computational formulations and a plethora of applications. ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
Canonical correlation analysis (CCA) is a classical method for seeking correlations between two multivariate data sets. During the last ten years, it has received more and more attention in the machine learning community in the form of novel computational formulations and a plethora of applications. We review recent developments in Bayesian models and inference methods for CCA which are attractive for their potential in hierarchical extensions and for coping with the combination of large dimensionalities and small sample sizes. The existing methods have not been particularly successful in fulfilling the promise yet; we introduce a novel efficient solution that imposes groupwise sparsity to estimate the posterior of an extended model which not only extracts the statistical dependencies (correlations) between data sets but also decomposes the data into shared and data setspecific components. In statistics literature the model is known as interbattery factor analysis (IBFA), for which we now provide a Bayesian treatment.
Groupsparse embeddings in collective matrix factorization,” arXiv preprint arXiv:1312.5921
, 2013
"... CMF is a technique for simultaneously learning lowrank representations based on a collection of matrices with shared entities. A typical example is the joint modeling of useritem, itemproperty, and userfeature matrices in a recommender system. The key idea in CMF is that the embeddings are sh ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
CMF is a technique for simultaneously learning lowrank representations based on a collection of matrices with shared entities. A typical example is the joint modeling of useritem, itemproperty, and userfeature matrices in a recommender system. The key idea in CMF is that the embeddings are shared across the matrices, which enables transferring information between them. The existing solutions, however, break down when the individual matrices have lowrank structure not shared with others. In this work we present a novel CMF solution that allows each of the matrices to have a separate lowrank structure that is independent of the other matrices, as well as structures that are shared only by a subset of them. We compare MAP and variational Bayesian solutions based on alternating optimization algorithms and show that the model automatically infers the nature of each factor using groupwise sparsity. Our approach supports in a principled way continuous, binary and count observations and is efficient for sparse matrices involving missing data. We illustrate the solution on a number of examples, focusing in particular on an interesting usecase of augmented multiview learning. 1.
Integrating Features and Similarities: Flexible Models for Heterogeneous Multiview Data
"... We present a probabilistic framework for learning with heterogeneous multiview data where some views are given as ordinal, binary, or realvalued feature matrices, and some views as similarity matrices. Our framework has the following distinguishing aspects: (i) a unified latent factor model for int ..."
Abstract
 Add to MetaCart
(Show Context)
We present a probabilistic framework for learning with heterogeneous multiview data where some views are given as ordinal, binary, or realvalued feature matrices, and some views as similarity matrices. Our framework has the following distinguishing aspects: (i) a unified latent factor model for integrating information from diverse feature (ordinal, binary, real) and similarity based views, and predicting the missing data in each view, leveraging view correlations; (ii) seamless adaptation to binary/multiclass classification where data consists of multiple feature and/or similaritybased views; and (iii) an efficient, variational inference algorithm which is especially flexible in modeling the views with ordinalvalued data (by learning the cutpoints for the ordinal data), and extends naturally to streaming data settings. Our framework subsumes methods such as multiview learning and multiple kernel learning as special cases. We demonstrate the effectiveness of our framework on several realworld and benchmarks datasets.
A Bayesian Framework for MultiModality Analysis of Mental Health
"... We develop statistical methods for multimodality assessment of mental health, based on four forms of data: (i) selfreported answers to a set of classical questionnaires, (ii) singlenucleotide polymorphism (SNP) data, (iii) fMRI data measured in response to visual stimuli, and (iv) scores for psy ..."
Abstract
 Add to MetaCart
(Show Context)
We develop statistical methods for multimodality assessment of mental health, based on four forms of data: (i) selfreported answers to a set of classical questionnaires, (ii) singlenucleotide polymorphism (SNP) data, (iii) fMRI data measured in response to visual stimuli, and (iv) scores for psychiatric disorders. The data were acquired from hundreds of college students. We utilize the data and model to ask a timely and novel clinical question: can one predict brain activity associated with risk for mental illness and treatment response based on knowledge of how the subject answers questionnaires, and using genetic (SNP) data? Also, in another direction: can one predict an individual’s fundamental propensity for psychopathology based on observed selfreport, SNP and fMRI data (separately or in combination)? The data are analyzed with a multimodality factor model, with sparsity imposed on the factor loadings, linked to the particular type of data modality. The analysis framework encompasses a wide range of problems, such as matrix completion and clustering, leveraging information in all the data sources. We use an efficient variational inference algorithm to fit the model, which is especially flexible in dealing with ordinalvalued views (selfreport answers and SNP data). The variational inference is validated with slower but rigorous sampling methods. We demonstrate the effectiveness of the model to perform accurate predictions for clinically relevant brain activity relative to baseline models, and to identify meaningful associations between data views.
Integrating Features and Similarities: Flexible Models for Heterogeneous Multiview Data
"... We present a probabilistic framework for learning with heterogeneous multiview data where some views are given as ordinal, binary, or realvalued feature matrices, and some views as similarity matrices. Our framework has the following distinguishing aspects: (i) a unified latent factor model for int ..."
Abstract
 Add to MetaCart
(Show Context)
We present a probabilistic framework for learning with heterogeneous multiview data where some views are given as ordinal, binary, or realvalued feature matrices, and some views as similarity matrices. Our framework has the following distinguishing aspects: (i) a unified latent factor model for integrating information from diverse feature (ordinal, binary, real) and similarity based views, and predicting the missing data in each view, leveraging view correlations; (ii) seamless adaptation to binary/multiclass classification where data consists of multiple feature and/or similaritybased views; and (iii) an efficient, variational inference algorithm which is especially flexible in modeling the views with ordinalvalued data (by learning the cutpoints for the ordinal data), and extends naturally to streaming data settings. Our framework subsumes methods such as multiview learning and multiple kernel learning as special cases. We demonstrate the effectiveness of our framework on several realworld and benchmarks datasets.
Bayesian Probabilistic CoSubspace Addition
"... For modeling data matrices, this paper introduces Probabilistic CoSubspace Addition (PCSA) model by simultaneously capturing the dependent structures among both rows and columns. Briefly, PCSA assumes that each entry of a matrix is generated by the additive combination of the linear mappings of two ..."
Abstract
 Add to MetaCart
(Show Context)
For modeling data matrices, this paper introduces Probabilistic CoSubspace Addition (PCSA) model by simultaneously capturing the dependent structures among both rows and columns. Briefly, PCSA assumes that each entry of a matrix is generated by the additive combination of the linear mappings of two lowdimensional features, which distribute in the rowwise and columnwise latent subspaces respectively. In consequence, PCSA captures the dependencies among entries intricately, and is able to handle nonGaussian and heteroscedastic densities. By formulating the posterior updating into the task of solving Sylvester equations, we propose an efficient variational inference algorithm. Furthermore, PCSA is extended to tackling and filling missing values, to adapting model sparseness, and to modelling tensor data. In comparison with several stateofart methods, experiments demonstrate the effectiveness and efficiency of Bayesian (sparse) PCSA on modeling matrix (tensor) data and filling missing values. 1
1Group Factor Analysis
"... Abstract—Factor analysis provides linear factors that describe relationships between individual variables of a data set. We extend this classical formulation into linear factors that describe relationships between groups of variables, where each group represents either a set of related variables o ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Factor analysis provides linear factors that describe relationships between individual variables of a data set. We extend this classical formulation into linear factors that describe relationships between groups of variables, where each group represents either a set of related variables or a data set. The model also naturally extends canonical correlation analysis to more than two sets, in a way that is more flexible than previous extensions. Our solution is formulated as variational inference of a latent variable model with structural sparsity, and it consists of two hierarchical levels: The higher level models the relationships between the groups, whereas the lower models the observed variables given the higher level. We show that the resulting solution solves the group factor analysis problem accurately, outperforming alternative factor analysis based solutions as well as more straightforward implementations of group factor analysis. The method is demonstrated on two life science data sets, one on brain activation and the other on systems biology, illustrating its applicability to the analysis of different types of highdimensional data sources. Index Terms—factor analysis, multiview learning, probabilistic algorithms, structured sparsity 1