Results 1  10
of
24
Compressive Phase Retrieval via Generalized Approximate Message Passing
"... Abstract—In this paper, we propose a novel approach to compressive phase retrieval based on loopy belief propagation and, in particular, on the generalized approximate message passing (GAMP) algorithm. Numerical results show that the proposed PRGAMP algorithm has excellent phasetransition behavior ..."
Abstract

Cited by 30 (8 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we propose a novel approach to compressive phase retrieval based on loopy belief propagation and, in particular, on the generalized approximate message passing (GAMP) algorithm. Numerical results show that the proposed PRGAMP algorithm has excellent phasetransition behavior, noise robustness, and runtime. In particular, for successful recovery of synthetic BernoullicircularGaussian signals, PRGAMP requires ≈ 4 times the number of measurements as a phaseoracle version of GAMP and, at moderate to large SNR, the NMSE of PRGAMP is only ≈ 3 dB worse than that of phaseoracle GAMP. A comparison to the recently proposed convexrelation approach known as “CPRL ” reveals PRGAMP’s superior phase transition and ordersofmagnitude faster runtimes, especially as the problem dimensions increase. When applied to the recovery of a 65kpixel grayscale image from 32k randomly masked magnitude measurements, numerical results show a median PRGAMP runtime of only 13.4 seconds. A. Phase retrieval I.
Phase Retrieval with Application to Optical Imaging
, 2015
"... The problem of phase retrieval, i.e., the recovery of a function given the magnitude of its ..."
Abstract

Cited by 18 (6 self)
 Add to MetaCart
The problem of phase retrieval, i.e., the recovery of a function given the magnitude of its
Exact and stable covariance estimation from quadratic sampling via convex programming. to appear
 IEEE Transactions on Information Theory
, 2015
"... Statistical inference and information processing of highdimensional data often require efficient and accurate estimation of their secondorder statistics. With rapidly changing data, limited processing power and storage at the sensor suite, it is desirable to extract the covariance structure from ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Statistical inference and information processing of highdimensional data often require efficient and accurate estimation of their secondorder statistics. With rapidly changing data, limited processing power and storage at the sensor suite, it is desirable to extract the covariance structure from a single pass over the data stream and a small number of measurements. In this paper, we explore a quadratic random measurement model which imposes a minimal memory requirement and low computational complexity during the sampling process, and is shown to be optimal in preserving lowdimensional covariance structures. Specifically, four popular structural assumptions of covariance matrices, namely low rank, Toeplitz low rank, sparsity, jointly rankone and sparse structure, are investigated. We show that a covariance matrix with either structure can be perfectly recovered from a nearoptimal number of subGaussian quadratic measurements, via efficient convex relaxation algorithms for the respective structure. The proposed algorithm has a variety of potential applications in streaming data processing, highfrequency wireless communication, phase space tomography in optics, noncoherent subspace detection, etc. Our method admits universally accurate covariance estimation in the absence of noise, as soon as the number of measurements exceeds the theoretic sampling limits. We also demonstrate the robustness of this approach against noise and imperfect structural assumptions. Our analysis is established upon a novel notion called the mixednorm restricted isometry property (RIP`2/`1), as well as the conventional RIP`2/`2 for nearisotropic and bounded measurements. Besides, our results improve upon bestknown phase retrieval (including both dense and sparse signals) guarantees using PhaseLift with a significantly simpler approach. 1
Statistical guarantees for the EM algorithm: From population to samplebased analysis
, 2014
"... We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
(Show Context)
We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to updates based on a finite set of samples. First, we characterize the domain of attraction of any global maximizer of the population likelihood. This characterization is based on a novel view of the EM updates as a perturbed form of likelihood ascent, or in parallel, of the gradient EM updates as a perturbed form of standard gradient ascent. Leveraging this characterization, we then provide nonasymptotic guarantees on the EM and gradient EM algorithms when applied to a finite set of samples. We develop consequences of our general theory for three canonical examples of incompletedata problems: mixture of Gaussians, mixture of regressions, and linear regression with covariates missing completely at random. In each case, our theory guarantees that with a suitable initialization, a relatively small number of EM (or gradient EM) steps will yield (with high probability) an estimate that is within statistical error of the MLE. We provide simulations to confirm this theoretically predicted behavior. 1
A convergent gradient descent algorithm for rank minimization and semidefinite programming from random linear measurements. arXiv preprint arXiv:1506.06081
, 2015
"... Abstract We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With O(r 3 κ 2 n log n) random measurements of a positive semidefinite n×n matrix of rank r and cond ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Abstract We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With O(r 3 κ 2 n log n) random measurements of a positive semidefinite n×n matrix of rank r and condition number κ, our method is guaranteed to converge linearly to the global optimum.
Solving Random Quadratic Systems of Equations is nearly as easy as . . .
, 2015
"... We consider the fundamental problem of solving quadratic systems of equations in n variables, where yi = 〈ai,x〉2, i = 1,...,m and x ∈ Rn is unknown. We propose a novel method, which starting with an initial guess computed by means of a spectral method, proceeds by minimizing a nonconvex functional ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We consider the fundamental problem of solving quadratic systems of equations in n variables, where yi = 〈ai,x〉2, i = 1,...,m and x ∈ Rn is unknown. We propose a novel method, which starting with an initial guess computed by means of a spectral method, proceeds by minimizing a nonconvex functional as in the Wirtinger flow approach [11]. There are several key distinguishing features, most notably, a distinct objective functional and novel update rules, which operate in an adaptive fashion and drop terms bearing too much influence on the search direction. These careful selection rules provide a tighter initial guess, better descent directions, and thus enhanced practical performance. On the theoretical side, we prove that for certain unstructured models of quadratic systems, our algorithms return the correct solution in linear time, i.e. in time proportional to reading the data {ai} and {yi} as soon as the ratio m/n between the number of equations and unknowns exceeds a fixed numerical constant. We extend the theory to deal with noisy systems in which we only have yi ≈ 〈ai,x〉2 and prove that our algorithms achieve a statistical accuracy, which is nearly unimprovable. We complement our theoretical study with numerical examples showing that solving random quadratic systems is both computationally and statistically not much harder than solving linear systems of the same size—hence the title of this paper. For instance, we
On some provably correct cases of variational inference for topic models.
 In NIPS,
, 2015
"... Abstract Variational inference is an efficient, popular heuristic used in the context of latent variable models. We provide the first analysis of instances where variational inference algorithms converge to the global optimum, in the setting of topic models. Our initializations are natural, one of ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract Variational inference is an efficient, popular heuristic used in the context of latent variable models. We provide the first analysis of instances where variational inference algorithms converge to the global optimum, in the setting of topic models. Our initializations are natural, one of them being used in LDAc, the most popular implementation of variational inference. In addition to providing intuition into why this heuristic might work in practice, the multiplicative, rather than additive nature of the variational inference updates forces us to use nonstandard proof arguments, which we believe might be of general theoretical interest.
Eigenwords: Spectral Word Embeddings
, 2015
"... Abstract Spectral learning algorithms have recently become popular in datarich domains, driven in part by recent advances in large scale randomized SVD, and in spectral estimation of Hidden Markov Models. Extensions of these methods lead to statistical estimation algorithms which are not only fast ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract Spectral learning algorithms have recently become popular in datarich domains, driven in part by recent advances in large scale randomized SVD, and in spectral estimation of Hidden Markov Models. Extensions of these methods lead to statistical estimation algorithms which are not only fast, scalable, and useful on real data sets, but are also provably correct. Following this line of research, we propose four fast and scalable spectral algorithms for learning word embeddings low dimensional real vectors (called Eigenwords) that capture the "meaning" of words from their context. All the proposed algorithms harness the multiview nature of text data i.e. the left and right context of each word, are fast to train and have strong theoretical properties. Some of the variants also have lower sample complexity and hence higher statistical power for rare words. We provide theory which establishes relationships between these algorithms and optimality criteria for the estimates they provide. We also perform thorough qualitative and quantitative evaluation of Eigenwords showing that simple linear approaches give performance comparable to or superior than the stateoftheart nonlinear deep learning based methods.