Results 1  10
of
42
Sparse Bayesian methods for lowrank matrix estimation. arXiv:1102.5288v1 [stat.ML
, 2011
"... Abstract—Recovery of lowrank matrices has recently seen significant ..."
Abstract

Cited by 28 (11 self)
 Add to MetaCart
(Show Context)
Abstract—Recovery of lowrank matrices has recently seen significant
Recovery of lowrank plus compressed sparse matrices with application to unveiling traffic anomalies
 IEEE TRANS. INFO. THEORY
, 2013
"... Given the noiseless superposition of a lowrank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the lowrank and sparse components becomes possible. This fundamental identif ..."
Abstract

Cited by 21 (5 self)
 Add to MetaCart
Given the noiseless superposition of a lowrank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the lowrank and sparse components becomes possible. This fundamental identifiability issue arises with traffic anomaly detection in backbone networks, and subsumes compressed sensing as well as the timely lowrank plus sparse matrix recovery tasks encountered in matrix decomposition problems. Leveraging the ability of and nuclear norms to recover sparse and lowrank matrices, a convex program is formulated to estimate the unknowns. Analysis and simulations confirm that the said convex program can recover the unknowns for sufficiently lowrank and sparse enough components, along with a compression matrix possessing an isometry property when restricted to operate on sparse vectors. When the lowrank, sparse, and compression matrices are drawn from certain random ensembles, it is established that exact recovery is possible with high probability. Firstorder algorithms are developed to solve the nonsmooth convex optimization problem with provable iteration complexity guarantees. Insightful tests with synthetic and real network data corroborate the effectiveness of the novel approach in unveiling traffic anomalies across flows and time, and its ability to outperform existing alternatives.
Robust PCA as bilinear decomposition with outliersparsity regularization
 IEEE TRANS. SIGNAL PROCESS
, 2012
"... Principal component analysis (PCA) is widely used for dimensionality reduction, with welldocumented merits in various applications involving highdimensional data, including computer vision, preference measurement, and bioinformatics. In this context, the fresh look advocated here permeates benefit ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
Principal component analysis (PCA) is widely used for dimensionality reduction, with welldocumented merits in various applications involving highdimensional data, including computer vision, preference measurement, and bioinformatics. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify PCA against outliers. A leasttrimmed squares estimator of a lowrank bilinear factor analysis model is shown closely related to that obtained from an(pseudo)normregularized criterion encouraging sparsity in a matrix explicitly modeling the outliers. This connection suggests robust PCA schemes based on convex relaxation, which lead naturally to a family of robust estimators encompassing Huber’s optimal Mclass as a special case. Outliers are identified by tuning a regularization parameter, which amounts to controlling sparsity of the outlier matrix along the whole robustification path of (group) leastabsolute shrinkage and selection operator (Lasso) solutions. Beyond its ties to robust statistics, the developed outlieraware PCA framework is versatile to accommodate novel and scalable algorithms to: i) track the lowrank signal subspace robustly, as new data are acquired in real time; and ii) determine principal components robustly in (possibly) infinitedimensional feature spaces. Synthetic and real data tests corroborate the effectiveness of the proposed robust PCA schemes, when used to identify aberrant responses in personality assessment surveys, as well as unveil communities in social networks, and intruders from video surveillance data.
A Probabilistic Approach to Robust Matrix Factorization
"... Abstract. Matrix factorization underlies a large variety of computer vision applications. It is a particularly challenging problem for largescale applications and when there exist outliers and missing data. In this paper, we propose a novel probabilistic model called Probabilistic Robust Matrix Fac ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Matrix factorization underlies a large variety of computer vision applications. It is a particularly challenging problem for largescale applications and when there exist outliers and missing data. In this paper, we propose a novel probabilistic model called Probabilistic Robust Matrix Factorization (PRMF) to solve this problem. In particular, PRMF is formulated with a Laplace error and a Gaussian prior which correspond to an ℓ1 loss and an ℓ2 regularizer, respectively. For model learning, we devise a parallelizable expectationmaximization (EM) algorithm which can potentially be applied to largescale applications. We also propose an online extension of the algorithm for sequential data to offer further scalability. Experiments conducted on both synthetic data and some practical computer vision applications show that PRMF is comparable to other stateoftheart robust matrix factorization methods in terms of accuracy and outperforms them particularly for large data matrices. 1
Bilinear Generalized Approximate Message Passing
, 2013
"... Abstract—We extend the generalized approximate message passing (GAMP) approach, originally proposed for highdimensional generalizedlinear regression in the context of compressive sensing, to the generalizedbilinear case, which enables its application to matrix completion, robust PCA, dictionary l ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
(Show Context)
Abstract—We extend the generalized approximate message passing (GAMP) approach, originally proposed for highdimensional generalizedlinear regression in the context of compressive sensing, to the generalizedbilinear case, which enables its application to matrix completion, robust PCA, dictionary learning, and related matrixfactorization problems. In the first part of the paper, we derive our Bilinear GAMP (BiGAMP) algorithm as an approximation of the sumproduct belief propagation algorithm in the highdimensional limit, where centrallimit theorem arguments and Taylorseries approximations apply, and under the assumption of statistically independent matrix entries with known priors. In addition, we propose an adaptive damping mechanism that aids convergence under finite problem sizes, an expectationmaximization (EM)based method to automatically tune the parameters of the assumed priors, and two rankselection strategies. In the second part of the paper, we discuss the specializations of EMBiGAMP to the problems of matrix completion, robust PCA, and dictionary learning, and present the results of an extensive empirical study comparing EMBiGAMP to stateoftheart algorithms on each problem. Our numerical results, using both synthetic and realworld datasets, demonstrate that EMBiGAMP yields excellent reconstruction accuracy (often best in class) while maintaining competitive runtimes and avoiding the need to tune algorithmic parameters. I.
NonConvex Rank Minimization via an Empirical Bayesian Approach
"... In many applications that require matrix solutions of minimal rank, the underlying cost function is nonconvex leading to an intractable, NPhard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many pr ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
In many applications that require matrix solutions of minimal rank, the underlying cost function is nonconvex leading to an intractable, NPhard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios there is no longer any guarantee that we can correctly estimate generative lowrank matrices of interest, theoretical special cases notwithstanding. Consequently, this paper proposes an alternative empirical Bayesianprocedure build upon a variational approximation that, unlike the nuclear norm, retains the same globally minimizing point estimate as the rank function under many useful constraints. However, locally minimizing solutions are largely smoothed away via marginalization, allowing the algorithm to succeed when standard convex relaxations completely fail. While the proposed methodology is generally applicable to a wide range of lowrank applications, we focus our attention on the robust principal component analysis problem (RPCA), which involves estimating an unknown lowrank matrix with unknown sparse corruptions. Theoretical and empirical evidence are presented to show that our method is potentially superior to related MAPbased approaches, for which the convex principle componentpursuit(PCP)algorithm(Candès et al., 2011) can be viewed as a special case. 1
PanSharpening with a Bayesian Nonparametric Dictionary Learning Model
"... Pansharpening, a method for constructing high resolution images from low resolution observations, has recently been explored from the perspective of compressed sensing and sparse representation theory. We present a new pansharpening algorithm that uses a Bayesian nonparametric dictionary learnin ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Pansharpening, a method for constructing high resolution images from low resolution observations, has recently been explored from the perspective of compressed sensing and sparse representation theory. We present a new pansharpening algorithm that uses a Bayesian nonparametric dictionary learning model to give an underlying sparse representation for image reconstruction. In contrast to existing dictionary learning methods, the proposed method infers parameters such as dictionary size, patch sparsity and noise variances. In addition, our regularization includes image constraints such as a total variation penalization term and a new gradient penalization on the reconstructed PAN image. Our method does not require high resolution multiband images for dictionary learning, which are unavailable in practice, but rather the dictionary is learned directly on the reconstructed image as part of the inversion process. We present experiments on several images to validate our method and compare with several other wellknown approaches. 1
Adaptive Sensing Techniques for Dynamic Target Tracking and Detection with Applications to Synthetic Aperture Radars
, 2013
"... ..."
(Show Context)
Fast algorithms for Gaussian noise invariant independent component analysis
 Advances in Neural Information Processing Systems 26
, 2013
"... The performance of standard algorithms for Independent Component Analysis quickly deteriorates under the addition of Gaussian noise. This is partially due to a common first step that typically consists of whitening, i.e., applying Principal Component Analysis (PCA) and rescaling the components to h ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
The performance of standard algorithms for Independent Component Analysis quickly deteriorates under the addition of Gaussian noise. This is partially due to a common first step that typically consists of whitening, i.e., applying Principal Component Analysis (PCA) and rescaling the components to have identity covariance, which is not invariant under Gaussian noise. In our paper we develop the first practical algorithm for Independent Component Analysis that is provably invariant under Gaussian noise. The two main contributions of this work are as follows: 1. We develop and implement an efficient, Gaussian noise invariant decorrelation (quasiorthogonalization) algorithm using Hessians of the cumulant functions. 2. We propose a very simple and efficient fixedpoint GIICA (Gradient Iteration ICA) algorithm, which is compatible with quasiorthogonalization, as well as with the usual PCAbased whitening in the noiseless case. The algorithm is based on a special form of gradient iteration (different from gradient descent). We provide an analysis of our algorithm demonstrating fast convergence following from the basic properties of cumulants. We also present a number of experimental comparisons with the existing methods, showing superior results on noisy data and very competitive performance in the noiseless case. 1
Bayesian Nonparametric Dictionary Learning for Compressed Sensing MRI
"... Abstract — We develop a Bayesian nonparametric model for reconstructing magnetic resonance images (MRIs) from highly undersampled kspace data. We perform dictionary learning as part of the image reconstruction process. To this end, we use the beta process as a nonparametric dictionary learning prio ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract — We develop a Bayesian nonparametric model for reconstructing magnetic resonance images (MRIs) from highly undersampled kspace data. We perform dictionary learning as part of the image reconstruction process. To this end, we use the beta process as a nonparametric dictionary learning prior for representing an image patch as a sparse combination of dictionary elements. The size of the dictionary and patchspecific sparsity pattern are inferred from the data, in addition to other dictionary learning variables. Dictionary learning is performed directly on the compressed image, and so is tailored to the MRI being considered. In addition, we investigate a total variation penalty term in combination with the dictionary learning model, and show how the denoising property of dictionary learning removes dependence on regularization parameters in the noisy setting. We derive a stochastic optimization algorithm based on Markov chain Monte Carlo for the Bayesian model, and use the alternating direction method of multipliers for efficiently performing total variation minimization. We present empirical results on several MRI, which show that the proposed regularization framework can improve reconstruction accuracy over other methods. Index Terms — Compressed sensing, magnetic resonance imaging, Bayesian nonparametrics, dictionary learning.