Results 1 
5 of
5
Lowrank Solutions of Linear Matrix Equations via Procrustes Flow
, 2015
"... In this paper we study the problem of recovering an lowrank positive semidefinite matrix from linear measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate obtained by a thresholding scheme followed by gradient descent on a nonconvex objective. We show that as ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
In this paper we study the problem of recovering an lowrank positive semidefinite matrix from linear measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate obtained by a thresholding scheme followed by gradient descent on a nonconvex objective. We show that as long as the measurements obey a standard restricted isometry property, our algorithm converges to the unknown matrix at a geometric rate. In the case of Gaussian measurements, such convergence occurs for a n×n matrix of rank r when the number of measurements exceeds a constant times nr. 1
Fast stochastic algorithms for svd and pca: Convergence properties and convexity
 CoRR
, 2015
"... Abstract We study the convergence properties of the VRPCA algorithm introduced by ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract We study the convergence properties of the VRPCA algorithm introduced by
Provable Efficient Online Matrix Completion via Nonconvex Stochastic Gradient Descent
"... Abstract Matrix completion, where we wish to recover a low rank matrix by observing a few entries from it, is a widely studied problem in both theory and practice with wide applications. Most of the provable algorithms so far on this problem have been restricted to the offline setting where they pr ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Matrix completion, where we wish to recover a low rank matrix by observing a few entries from it, is a widely studied problem in both theory and practice with wide applications. Most of the provable algorithms so far on this problem have been restricted to the offline setting where they provide an estimate of the unknown matrix using all observations simultaneously. However, in many applications, the online version, where we observe one entry at a time and dynamically update our estimate, is more appealing. While existing algorithms are efficient for the offline setting, they could be highly inefficient for the online setting. In this paper, we propose the first provable, efficient online algorithm for matrix completion. Our algorithm starts from an initial estimate of the matrix and then performs nonconvex stochastic gradient descent (SGD). After every observation, it performs a fast update involving only one row of two tall matrices, giving near linear total runtime. Our algorithm can be naturally used in the offline setting as well, where it gives competitive sample complexity and runtime to state of the art algorithms. Our proofs introduce a general framework to show that SGD updates tend to stay away from saddle surfaces and could be of broader interests to other nonconvex problems.
Convergence of Stochastic Gradient Descent for PCA
"... Abstract We consider the problem of principal component analysis (PCA) in a streaming stochastic setting, where our goal is to find a direction of approximate maximal variance, based on a stream of i.i.d. data points in R d . A simple and computationally cheap algorithm for this is stochastic gradi ..."
Abstract
 Add to MetaCart
Abstract We consider the problem of principal component analysis (PCA) in a streaming stochastic setting, where our goal is to find a direction of approximate maximal variance, based on a stream of i.i.d. data points in R d . A simple and computationally cheap algorithm for this is stochastic gradient descent (SGD), which incrementally updates its estimate based on each new data point. However, due to the nonconvex nature of the problem, analyzing its performance has been a challenge. In particular, existing guarantees rely on a nontrivial eigengap assumption on the covariance matrix, which is intuitively unnecessary. In this paper, we provide (to the best of our knowledge) the first eigengapfree convergence guarantees for SGD in the context of PCA. This also partially resolves an open problem posed in
A Geometric Analysis of Phase Retrieval
"... Abstract Can we recover a complex signal from its Fourier magnitudes? More generally, given a set of m measurements, y k = a * k x for k = 1, . . . , m, is it possible to recover x ∈ C n (i.e., lengthn complex vector)? This generalized phase retrieval (GPR) problem is a fundamental task in vario ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Can we recover a complex signal from its Fourier magnitudes? More generally, given a set of m measurements, y k = a * k x for k = 1, . . . , m, is it possible to recover x ∈ C n (i.e., lengthn complex vector)? This generalized phase retrieval (GPR) problem is a fundamental task in various disciplines, and has been the subject of much recent investigation. Natural nonconvex heuristics often work remarkably well for GPR in practice, but lack clear theoretical explanations. In this paper, we take a step towards bridging this gap. We prove that when the measurement vectors a k 's are generic (i.i.d. complex Gaussian) and the number of measurements is large enough (m ≥ Cn log 3 n), with high probability, a natural leastsquares formulation for GPR has the following benign geometric structure: (1) there are no spurious local minimizers, and all global minimizers are equal to the target signal x, up to a global phase; and (2) the objective function has a negative curvature around each saddle point. This structure allows a number of iterative optimization methods to efficiently find a global minimizer, without special initialization. To corroborate the claim, we describe and analyze a secondorder trustregion algorithm.