Results 1  10
of
61
Generalized Approximate Message Passing for Estimation with Random Linear Mixing
, 2012
"... We consider the estimation of an i.i.d. random vector observed through a linear transform followed by a componentwise, probabilistic (possibly nonlinear) measurement channel. A novel algorithm, called generalized approximate message passing (GAMP), is presented that provides computationally effici ..."
Abstract

Cited by 123 (18 self)
 Add to MetaCart
We consider the estimation of an i.i.d. random vector observed through a linear transform followed by a componentwise, probabilistic (possibly nonlinear) measurement channel. A novel algorithm, called generalized approximate message passing (GAMP), is presented that provides computationally efficient approximate implementations of maxsum and sumproblem loopy belief propagation for such problems. The algorithm extends earlier approximate message passing methods to incorporate arbitrary distributions on both the input and output of the transform and can be applied to a wide range of problems in nonlinear compressed sensing and learning. Extending an analysis by Bayati and Montanari, we argue that the asymptotic componentwise behavior of the GAMP method under large, i.i.d. Gaussian transforms is described by a simple set of state evolution (SE) equations. From the SE equations, one can exactly predict the asymptotic value of virtually any componentwise performance metric including meansquared error or detection accuracy. Moreover, the analysis is valid for arbitrary input and output distributions, even when the corresponding optimization problems are nonconvex. The results match predictions by Guo and Wang for relaxed belief propagation on large sparse matrices and, in certain instances, also agree with the optimal performance predicted by the replica method. The GAMP methodology thus provides a computationally efficient methodology, applicable to a large class of nonGaussian estimation problems with precise asymptotic performance guarantees.
the Spectral Algorithm.
"... Here we give a brief introduction to tensor algebra (for more details, see [2]). A tensor is a multidimensional array, and its order is the number of dimensions, also known as modes. In this paper, vectors (tensors of order one) are denoted by boldface lowercase letters, e.g., a. Matrices (tensors o ..."
Abstract

Cited by 53 (9 self)
 Add to MetaCart
Here we give a brief introduction to tensor algebra (for more details, see [2]). A tensor is a multidimensional array, and its order is the number of dimensions, also known as modes. In this paper, vectors (tensors of order one) are denoted by boldface lowercase letters, e.g., a. Matrices (tensors of order two) are denoted by boldface capital letters, e.g., A. Higherorder tensors (order three or higher) are denoted by boldface caligraphic letters, e.g., T. Scalars are denoted by lowercase letters, e.g., a. Subarrays of a tensor are formed when a subset of the indices is fixed. Particularly, a fiber is defined by fixing every index but one. Fibers are the higherorder analogue of matrix rows and columns. A colon is used to indicate all elements of a mode. Thus, the jth column of a matrix A is A(:, j), and the ith row of A is A(i,:). Analogously, the moden fiber of a Nth order tensor T is then denoted as T (i1, i2,..., in−1,:, in+1,..., iN). Tensors can be multiplied together. For matrices and vectors, we will use standard notation for their multiplications, e.g., Ba and AB. For tensors of higher order, we are particularly interested in multiplying a tensor by matrices and vectors. The nmode matrix product is the multiplication of a tensor with a matrix in mode n of the tensor. Let T ∈ R I1×I2×...×IN be an Nth order tensor and A ∈ R J×In be a matrix. Then T ′ = T ×n A ∈ R I1×...In−1×J×In+1×...×IN, (1) where the entries T ′ (i1,..., in−1, j, in+1,..., iN) are defined as ∑ In
Alternating direction algorithms for ℓ1problems in compressive sensing
, 2009
"... Abstract. In this paper, we propose and study the use of alternating direction algorithms for several ℓ1norm minimization problems arising from sparse solution recovery in compressive sensing, including the basis pursuit problem, the basispursuit denoising problems of both unconstrained and constr ..."
Abstract

Cited by 48 (5 self)
 Add to MetaCart
(Show Context)
Abstract. In this paper, we propose and study the use of alternating direction algorithms for several ℓ1norm minimization problems arising from sparse solution recovery in compressive sensing, including the basis pursuit problem, the basispursuit denoising problems of both unconstrained and constrained forms, as well as others. We present and investigate two classes of algorithms derived from either the primal or the dual forms of the ℓ1problems. The construction of the algorithms consists of two main steps: (1) to reformulate an ℓ1problem into one having partially separable objective functions by adding new variables and constraints; and (2) to apply an exact or inexact alternating direction method to the resulting problem. The derived alternating direction algorithms can be regarded as firstorder primaldual algorithms because both primal and dual variables are updated at each and every iteration. Convergence properties of these algorithms are established or restated when they already exist. Extensive numerical results in comparison with several stateoftheart algorithms are given to demonstrate that the proposed algorithms are efficient, stable and robust. Moreover, we present numerical results to emphasize two practically important but perhaps overlooked points. One point is that algorithm speed should always be evaluated relative to appropriate solution accuracy; another is that whenever erroneous measurements possibly exist, the ℓ1norm fidelity should be the fidelity of choice in compressive sensing. Key words. Sparse solution recovery, compressive sensing, ℓ1minimization, primal, dual, alternating direction method
Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming
, 2011
"... Abstract. Recently, we have proposed to combine the alternating direction method (ADM) with a Gaussian back substitution procedure for solving the convex minimization model with linear constraints and a general separable objective function, i.e., the objective function is the sum of many functions w ..."
Abstract

Cited by 37 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Recently, we have proposed to combine the alternating direction method (ADM) with a Gaussian back substitution procedure for solving the convex minimization model with linear constraints and a general separable objective function, i.e., the objective function is the sum of many functions without coupled variables. In this paper, we further study this topic and show that the decomposed subproblems in the ADM procedure can be substantially alleviated by linearizing the involved quadratic terms arising from the augmented Lagrangian penalty on the model’s linear constraints. When the resolvent operators of the separable functions in the objective have closedform representations, embedding the linearization into the ADM subproblems becomes necessary to yield easy subproblems with closedform solutions. We thus show theoretically that the blend of ADM, Gaussian back substitution and linearization works effectively for the separable convex minimization model under consideration.
Augmented Lagrangian alternating direction method for matrix separation based on lowrank factorization
, 2011
"... The matrix separation problem aims to separate a lowrank matrix and a sparse matrix from their sum. This problem has recently attracted considerable research attention due to its wide range of potential applications. Nuclearnorm minimization models have been proposed for matrix separation and prov ..."
Abstract

Cited by 29 (2 self)
 Add to MetaCart
(Show Context)
The matrix separation problem aims to separate a lowrank matrix and a sparse matrix from their sum. This problem has recently attracted considerable research attention due to its wide range of potential applications. Nuclearnorm minimization models have been proposed for matrix separation and proved to yield exact separations under suitable conditions. These models, however, typically require the calculation of a full or partial singular value decomposition (SVD) at every iteration that can become increasingly costly as matrix dimensions and rank grow. To improve scalability, in this paper we propose and investigate an alternative approach based on solving a nonconvex, lowrank factorization model by an augmented Lagrangian alternating direction method. Numerical studies indicate that the effectiveness of the proposed model is limited to problems where the sparse matrix does not dominate the lowrank one in magnitude, though this limitation can be alleviated by certain data preprocessing techniques. On the other hand, extensive numerical results show that, within its applicability range, the proposed method in general has a much faster solution speed than nuclearnorm minimization algorithms, and often provides better recoverability.
Consistent shape maps via semidefinite programming
 In Computer Graphics Forum
"... Recent advances in shape matching have shown that jointly optimizing the maps among the shapes in a collection can lead to significant improvements when compared to estimating maps between pairs of shapes in isolation. These methods typically invoke a cycleconsistency criterion — the fact that comp ..."
Abstract

Cited by 23 (3 self)
 Add to MetaCart
(Show Context)
Recent advances in shape matching have shown that jointly optimizing the maps among the shapes in a collection can lead to significant improvements when compared to estimating maps between pairs of shapes in isolation. These methods typically invoke a cycleconsistency criterion — the fact that compositions of maps along a cycle of shapes should approximate the identity map. This condition regularizes the network and allows for the correction of errors and imperfections in individual maps. In particular, it encourages the estimation of maps between dissimilar shapes by compositions of maps along a path of more similar shapes. In this paper, we introduce a novel approach for obtaining consistent shape maps in a collection that formulates the cycleconsistency constraint as the solution to a semidefinite program (SDP). The proposed approach is based on the observation that, if the ground truth maps between the shapes are cycleconsistent, then the matrix that stores all pairwise maps in blocks is lowrank and positive semidefinite. Motivated by recent advances in techniques for lowrank matrix recovery via semidefinite programming, we formulate the problem of estimating cycleconsistent maps as finding the closest positive semidefinite matrix to an input matrix that stores all the initial maps. By analyzing the KarushKuhnTucker (KKT) optimality condition of this program, we derive theoretical guarantees for the proposed algorithm, ensuring the correctness of the recovery when the errors in the inputs maps do not exceed certain thresholds. Besides this theoretical guarantee, experimental results on benchmark datasets show that the proposed approach outperforms stateoftheart multiple shape matching methods. 1
EXACT AND STABLE RECOVERY OF ROTATIONS FOR ROBUST SYNCHRONIZATION
, 1211
"... Abstract. The synchronization problem over the special orthogonal group SO(d) consists of estimating a set of unknown rotations R1, R2,..., Rn from noisy measurements of a subset of their pairwise ratios R −1 i Rj. The problem has found applications in computer vision, computer graphics, and sensor ..."
Abstract

Cited by 22 (9 self)
 Add to MetaCart
(Show Context)
Abstract. The synchronization problem over the special orthogonal group SO(d) consists of estimating a set of unknown rotations R1, R2,..., Rn from noisy measurements of a subset of their pairwise ratios R −1 i Rj. The problem has found applications in computer vision, computer graphics, and sensor network localization, among others. Its least squares solution can be approximated by either spectral relaxation or semidefinite programming followed by a rounding procedure, analogous to the approximation algorithms of MaxCut. The contribution of this paper is threefold: First, we introduce a robust penalty function involving the sum of unsquared deviations and derive a relaxation that leads to a convex optimization problem; Second, we apply the alternating direction method to minimize the penalty function; Finally, under a specific model of the measurement noise and the measurement graph, we prove that the rotations are exactly and stably recovered, exhibiting a phase transition behavior in terms of the proportion of noisy measurements. Numerical simulations confirm the phase transition behavior for our method as well as its improved accuracy compared to existing methods. Key words. Synchronization of rotations; least unsquared deviation; semidefinite relaxation; and alternating direction method 1. Introduction. The
Approximate message passing with consistent parameter estimation and applications to sparse learning,” arXiv:1207.3859 [cs.IT
, 2012
"... We consider the estimation of an i.i.d. vector x ∈ Rn from measurements y ∈ Rm obtained by a general cascade model consisting of a known linear transform followed by a probabilistic componentwise (possibly nonlinear) measurement channel. We present a method, called adaptive generalized approximate ..."
Abstract

Cited by 21 (4 self)
 Add to MetaCart
(Show Context)
We consider the estimation of an i.i.d. vector x ∈ Rn from measurements y ∈ Rm obtained by a general cascade model consisting of a known linear transform followed by a probabilistic componentwise (possibly nonlinear) measurement channel. We present a method, called adaptive generalized approximate message passing (Adaptive GAMP), that enables joint learning of the statistics of the prior and measurement channel along with estimation of the unknown vector x. Our method can be applied to a large class of learning problems including the learning of sparse priors in compressed sensing or identification of linearnonlinear cascade models in dynamical systems and neural spiking processes. We prove that for large i.i.d. Gaussian transform matrices the asymptotic componentwise behavior of the adaptive GAMP algorithm is predicted by a simple set of scalar state evolution equations. This analysis shows that the adaptive GAMP method can yield asymptotically consistent parameter estimates, which implies that the algorithm achieves a reconstruction quality equivalent to the oracle algorithm that knows the correct parameter values. The adaptive GAMP methodology thus provides a systematic, general and computationally efficient method applicable to a large range of complex linearnonlinear models with provable guarantees. 1
An Alternating Direction Algorithm for Matrix Completion with Nonnegative Factors
"... Abstract. This paper introduces a novel algorithm for the nonnegative matrix factorization and completion problem, which aims to find nonnegative matrices X and Y from a subset of entries of a nonnegative matrix M so that XY approximates M. This problem is closely related to the two existing problem ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
(Show Context)
Abstract. This paper introduces a novel algorithm for the nonnegative matrix factorization and completion problem, which aims to find nonnegative matrices X and Y from a subset of entries of a nonnegative matrix M so that XY approximates M. This problem is closely related to the two existing problems: nonnegative matrix factorization and lowrank matrix completion, in the sense that it kills two birds with one stone. As it takes advantages of both nonnegativity and low rank, its results can be superior than those of the two problems alone. Our algorithm is applied to minimizing a nonconvex constrained leastsquares formulation and is based on the classic alternating direction augmented Lagrangian method. Preliminary convergence properties and numerical simulation results are presented. Compared to a recent algorithm for nonnegative random matrix factorization, the proposed algorithm yields comparable factorization through accessing only half of the matrix entries. On tasks of recovering incomplete grayscale and hyperspectral images, the results of the proposed algorithm have overall better qualities than those of two recent algorithms for matrix completion.
An introduction to a class of matrix cone programming
"... In this paper, we define a class of linear conic programming (which we call matrix cone ..."
Abstract

Cited by 16 (5 self)
 Add to MetaCart
(Show Context)
In this paper, we define a class of linear conic programming (which we call matrix cone