Results 1  10
of
40
Sparse Bayesian methods for lowrank matrix estimation. arXiv:1102.5288v1 [stat.ML
, 2011
"... Abstract—Recovery of lowrank matrices has recently seen significant ..."
Abstract

Cited by 28 (11 self)
 Add to MetaCart
(Show Context)
Abstract—Recovery of lowrank matrices has recently seen significant
Recovery of lowrank plus compressed sparse matrices with application to unveiling traffic anomalies
 IEEE TRANS. INFO. THEORY
, 2013
"... Given the noiseless superposition of a lowrank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the lowrank and sparse components becomes possible. This fundamental identif ..."
Abstract

Cited by 21 (5 self)
 Add to MetaCart
Given the noiseless superposition of a lowrank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the lowrank and sparse components becomes possible. This fundamental identifiability issue arises with traffic anomaly detection in backbone networks, and subsumes compressed sensing as well as the timely lowrank plus sparse matrix recovery tasks encountered in matrix decomposition problems. Leveraging the ability of and nuclear norms to recover sparse and lowrank matrices, a convex program is formulated to estimate the unknowns. Analysis and simulations confirm that the said convex program can recover the unknowns for sufficiently lowrank and sparse enough components, along with a compression matrix possessing an isometry property when restricted to operate on sparse vectors. When the lowrank, sparse, and compression matrices are drawn from certain random ensembles, it is established that exact recovery is possible with high probability. Firstorder algorithms are developed to solve the nonsmooth convex optimization problem with provable iteration complexity guarantees. Insightful tests with synthetic and real network data corroborate the effectiveness of the novel approach in unveiling traffic anomalies across flows and time, and its ability to outperform existing alternatives.
Robust PCA as bilinear decomposition with outliersparsity regularization
 IEEE TRANS. SIGNAL PROCESS
, 2012
"... Principal component analysis (PCA) is widely used for dimensionality reduction, with welldocumented merits in various applications involving highdimensional data, including computer vision, preference measurement, and bioinformatics. In this context, the fresh look advocated here permeates benefit ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
Principal component analysis (PCA) is widely used for dimensionality reduction, with welldocumented merits in various applications involving highdimensional data, including computer vision, preference measurement, and bioinformatics. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify PCA against outliers. A leasttrimmed squares estimator of a lowrank bilinear factor analysis model is shown closely related to that obtained from an(pseudo)normregularized criterion encouraging sparsity in a matrix explicitly modeling the outliers. This connection suggests robust PCA schemes based on convex relaxation, which lead naturally to a family of robust estimators encompassing Huber’s optimal Mclass as a special case. Outliers are identified by tuning a regularization parameter, which amounts to controlling sparsity of the outlier matrix along the whole robustification path of (group) leastabsolute shrinkage and selection operator (Lasso) solutions. Beyond its ties to robust statistics, the developed outlieraware PCA framework is versatile to accommodate novel and scalable algorithms to: i) track the lowrank signal subspace robustly, as new data are acquired in real time; and ii) determine principal components robustly in (possibly) infinitedimensional feature spaces. Synthetic and real data tests corroborate the effectiveness of the proposed robust PCA schemes, when used to identify aberrant responses in personality assessment surveys, as well as unveil communities in social networks, and intruders from video surveillance data.
A Probabilistic Approach to Robust Matrix Factorization
"... Abstract. Matrix factorization underlies a large variety of computer vision applications. It is a particularly challenging problem for largescale applications and when there exist outliers and missing data. In this paper, we propose a novel probabilistic model called Probabilistic Robust Matrix Fac ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Matrix factorization underlies a large variety of computer vision applications. It is a particularly challenging problem for largescale applications and when there exist outliers and missing data. In this paper, we propose a novel probabilistic model called Probabilistic Robust Matrix Factorization (PRMF) to solve this problem. In particular, PRMF is formulated with a Laplace error and a Gaussian prior which correspond to an ℓ1 loss and an ℓ2 regularizer, respectively. For model learning, we devise a parallelizable expectationmaximization (EM) algorithm which can potentially be applied to largescale applications. We also propose an online extension of the algorithm for sequential data to offer further scalability. Experiments conducted on both synthetic data and some practical computer vision applications show that PRMF is comparable to other stateoftheart robust matrix factorization methods in terms of accuracy and outperforms them particularly for large data matrices. 1
Bilinear Generalized Approximate Message Passing
, 2013
"... Abstract—We extend the generalized approximate message passing (GAMP) approach, originally proposed for highdimensional generalizedlinear regression in the context of compressive sensing, to the generalizedbilinear case, which enables its application to matrix completion, robust PCA, dictionary l ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
(Show Context)
Abstract—We extend the generalized approximate message passing (GAMP) approach, originally proposed for highdimensional generalizedlinear regression in the context of compressive sensing, to the generalizedbilinear case, which enables its application to matrix completion, robust PCA, dictionary learning, and related matrixfactorization problems. In the first part of the paper, we derive our Bilinear GAMP (BiGAMP) algorithm as an approximation of the sumproduct belief propagation algorithm in the highdimensional limit, where centrallimit theorem arguments and Taylorseries approximations apply, and under the assumption of statistically independent matrix entries with known priors. In addition, we propose an adaptive damping mechanism that aids convergence under finite problem sizes, an expectationmaximization (EM)based method to automatically tune the parameters of the assumed priors, and two rankselection strategies. In the second part of the paper, we discuss the specializations of EMBiGAMP to the problems of matrix completion, robust PCA, and dictionary learning, and present the results of an extensive empirical study comparing EMBiGAMP to stateoftheart algorithms on each problem. Our numerical results, using both synthetic and realworld datasets, demonstrate that EMBiGAMP yields excellent reconstruction accuracy (often best in class) while maintaining competitive runtimes and avoiding the need to tune algorithmic parameters. I.
PanSharpening with a Bayesian Nonparametric Dictionary Learning Model
"... Pansharpening, a method for constructing high resolution images from low resolution observations, has recently been explored from the perspective of compressed sensing and sparse representation theory. We present a new pansharpening algorithm that uses a Bayesian nonparametric dictionary learnin ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Pansharpening, a method for constructing high resolution images from low resolution observations, has recently been explored from the perspective of compressed sensing and sparse representation theory. We present a new pansharpening algorithm that uses a Bayesian nonparametric dictionary learning model to give an underlying sparse representation for image reconstruction. In contrast to existing dictionary learning methods, the proposed method infers parameters such as dictionary size, patch sparsity and noise variances. In addition, our regularization includes image constraints such as a total variation penalization term and a new gradient penalization on the reconstructed PAN image. Our method does not require high resolution multiband images for dictionary learning, which are unavailable in practice, but rather the dictionary is learned directly on the reconstructed image as part of the inversion process. We present experiments on several images to validate our method and compare with several other wellknown approaches. 1
NonConvex Rank Minimization via an Empirical Bayesian Approach
"... In many applications that require matrix solutions of minimal rank, the underlying cost function is nonconvex leading to an intractable, NPhard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many pr ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
In many applications that require matrix solutions of minimal rank, the underlying cost function is nonconvex leading to an intractable, NPhard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios there is no longer any guarantee that we can correctly estimate generative lowrank matrices of interest, theoretical special cases notwithstanding. Consequently, this paper proposes an alternative empirical Bayesianprocedure build upon a variational approximation that, unlike the nuclear norm, retains the same globally minimizing point estimate as the rank function under many useful constraints. However, locally minimizing solutions are largely smoothed away via marginalization, allowing the algorithm to succeed when standard convex relaxations completely fail. While the proposed methodology is generally applicable to a wide range of lowrank applications, we focus our attention on the robust principal component analysis problem (RPCA), which involves estimating an unknown lowrank matrix with unknown sparse corruptions. Theoretical and empirical evidence are presented to show that our method is potentially superior to related MAPbased approaches, for which the convex principle componentpursuit(PCP)algorithm(Candès et al., 2011) can be viewed as a special case. 1
A Variational Approach for Sparse Component Estimation and LowRank Matrix Recovery
"... We propose a variational Bayesian based algorithm for the estimation of the sparse component of an outliercorrupted lowrank matrix, when linearly transformed composite data are observed. The model constitutes a generalization of robust principal component analysis. The problem considered herein is ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
We propose a variational Bayesian based algorithm for the estimation of the sparse component of an outliercorrupted lowrank matrix, when linearly transformed composite data are observed. The model constitutes a generalization of robust principal component analysis. The problem considered herein is applicable in various practical scenarios, such as foreground detection in blurred and noisy video sequences and detection of network anomalies among others. The proposed algorithm models the lowrank matrix and the sparse component using a hierarchical Bayesian framework, and employs a variational approach for inference of the unknowns. The effectiveness of the proposed algorithm is demonstrated using real life experiments, and its performance improvement over regularization based approaches is shown. Index Terms—Bayesian inference, variational approach, robust principal component analysis, foreground detection, network anomaly detection improvement of the proposed algorithm over its regularization based counterpart. This paper is organized as follows. In Section II we present the general data model and several areas of applications. A brief overview of the related work in each of these areas is also provided. In Section III we introduce the proposed hierarchical Bayesian model. Details of the variational inference procedure are provided in Section IV. Numerical examples are presented in Section V. Finally we draw conclusion remarks in Section VI. Notation: Matrices and vectors are denoted by uppercase and lowercase boldface letters, respectively. vec(·), diag(·) and Tr(·) are vectorization, diagonalization and trace operators, respectively. Given a matrix X, we denote as xi·, x·j and Xij its ith row, jth column and (i, j) th element, respectively.
Adaptive Sensing Techniques for Dynamic Target Tracking and Detection with Applications to Synthetic Aperture Radars
, 2013
"... ..."
(Show Context)
Bayesian Nonparametric Dictionary Learning for Compressed Sensing MRI
"... Abstract — We develop a Bayesian nonparametric model for reconstructing magnetic resonance images (MRIs) from highly undersampled kspace data. We perform dictionary learning as part of the image reconstruction process. To this end, we use the beta process as a nonparametric dictionary learning prio ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract — We develop a Bayesian nonparametric model for reconstructing magnetic resonance images (MRIs) from highly undersampled kspace data. We perform dictionary learning as part of the image reconstruction process. To this end, we use the beta process as a nonparametric dictionary learning prior for representing an image patch as a sparse combination of dictionary elements. The size of the dictionary and patchspecific sparsity pattern are inferred from the data, in addition to other dictionary learning variables. Dictionary learning is performed directly on the compressed image, and so is tailored to the MRI being considered. In addition, we investigate a total variation penalty term in combination with the dictionary learning model, and show how the denoising property of dictionary learning removes dependence on regularization parameters in the noisy setting. We derive a stochastic optimization algorithm based on Markov chain Monte Carlo for the Bayesian model, and use the alternating direction method of multipliers for efficiently performing total variation minimization. We present empirical results on several MRI, which show that the proposed regularization framework can improve reconstruction accuracy over other methods. Index Terms — Compressed sensing, magnetic resonance imaging, Bayesian nonparametrics, dictionary learning.