Results 1  10
of
24
Guaranteed matrix completion via nonconvex factorization
, 2014
"... Matrix factorization is a popular approach for largescale matrix completion and constitutes a basic component of many solutions for Netflix Prize competition. In this approach, the unknown lowrank matrix is expressed as the product of two much smaller matrices so that the lowrank property is auto ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
Matrix factorization is a popular approach for largescale matrix completion and constitutes a basic component of many solutions for Netflix Prize competition. In this approach, the unknown lowrank matrix is expressed as the product of two much smaller matrices so that the lowrank property is automatically fulfilled. The resulting optimization problem, even with huge size, can be solved (to stationary points) very efficiently through standard optimization algorithms such as alternating minimization and stochastic gradient descent (SGD). However, due to the nonconvexity caused by the factorization model, there is a limited theoretical understanding of whether these algorithms will generate a good solution. In this paper, we establish a theoretical guarantee for the factorization based formulation to correctly recover the underlying lowrank matrix. In particular, we show that under similar conditions to those in previous works, many standard optimization algorithms converge to the global optima of the factorization based formulation, thus recovering the true lowrank matrix. To the best of our knowledge, our result is the first one that provides recovery guarantee for many standard algorithms such as gradient descent, SGD and block coordinate gradient descent. Our result also applies to alternating minimization, and a notable difference from previous studies on alternating minimization is that we do not need the resampling scheme (i.e. using independent samples in each iteration).
Global convergence of stochastic gradient descent for some nonconvex matrix problems. arXiv preprint arXiv:1411.1134,
, 2014
"... Abstract Stochastic gradient descent (SGD) on a lowrank factorization ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Abstract Stochastic gradient descent (SGD) on a lowrank factorization
Lowrank Solutions of Linear Matrix Equations via Procrustes Flow
, 2015
"... In this paper we study the problem of recovering an lowrank positive semidefinite matrix from linear measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate obtained by a thresholding scheme followed by gradient descent on a nonconvex objective. We show that as ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
In this paper we study the problem of recovering an lowrank positive semidefinite matrix from linear measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate obtained by a thresholding scheme followed by gradient descent on a nonconvex objective. We show that as long as the measurements obey a standard restricted isometry property, our algorithm converges to the unknown matrix at a geometric rate. In the case of Gaussian measurements, such convergence occurs for a n×n matrix of rank r when the number of measurements exceeds a constant times nr. 1
A convergent gradient descent algorithm for rank minimization and semidefinite programming from random linear measurements. arXiv preprint arXiv:1506.06081
, 2015
"... Abstract We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With O(r 3 κ 2 n log n) random measurements of a positive semidefinite n×n matrix of rank r and cond ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Abstract We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With O(r 3 κ 2 n log n) random measurements of a positive semidefinite n×n matrix of rank r and condition number κ, our method is guaranteed to converge linearly to the global optimum.
Sharp timedata tradeoffs for linear inverse problems
 In preparation
, 2015
"... In this paper we characterize sharp timedata tradeoffs for optimization problems used for solving linear inverse problems. We focus on the minimization of a leastsquares objective subject to a constraint defined as the sublevel set of a penalty function. We present a unified convergence analysis ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
In this paper we characterize sharp timedata tradeoffs for optimization problems used for solving linear inverse problems. We focus on the minimization of a leastsquares objective subject to a constraint defined as the sublevel set of a penalty function. We present a unified convergence analysis of the gradient projection algorithm applied to such problems. We sharply characterize the convergence rate associated with a wide variety of random measurement ensembles in terms of the number of measurements and structural complexity of the signal with respect to the chosen penalty function. The results apply to both convex and nonconvex constraints, demonstrating that a linear convergence rate is attainable even though the least squares objective is not strongly convex in these settings. When specialized to Gaussian measurements our results show that such linear convergence occurs when the number of measurements is merely 4 times the minimal number required to recover the desired signal at all (a.k.a. the phase transition). We also achieve a slower but geometric rate of convergence precisely above the phase transition point. Extensive numerical results suggest that the derived rates exactly match the empirical performance.
Guarantees of Riemannian Optimization for Low Rank Matrix Recovery
, 2015
"... We establish theoretical recovery guarantees of a family of Riemannian optimization algorithms for low rank matrix recovery, which is about recovering an m × n rank r matrix from p < mn number of linear measurements. The algorithms are first interpreted as the iterative hard thresholding algorit ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
We establish theoretical recovery guarantees of a family of Riemannian optimization algorithms for low rank matrix recovery, which is about recovering an m × n rank r matrix from p < mn number of linear measurements. The algorithms are first interpreted as the iterative hard thresholding algorithms with subspace projections. Then based on this connection, we prove that if the restricted isometry constant R3r of the sensing operator is less than Cκ/ r where Cκ depends on the condition number of the matrix, the Riemannian gradient descent method and a restarted variant of the Riemannian conjugate gradient method are guaranteed to converge to the measured rank r matrix provided they are initialized by one step hard thresholding. Empirical evaluation shows that the algorithms are able to recover a low rank matrix from nearly the minimum number of measurements necessary.
Solving Random Quadratic Systems of Equations is nearly as easy as . . .
, 2015
"... We consider the fundamental problem of solving quadratic systems of equations in n variables, where yi = 〈ai,x〉2, i = 1,...,m and x ∈ Rn is unknown. We propose a novel method, which starting with an initial guess computed by means of a spectral method, proceeds by minimizing a nonconvex functional ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We consider the fundamental problem of solving quadratic systems of equations in n variables, where yi = 〈ai,x〉2, i = 1,...,m and x ∈ Rn is unknown. We propose a novel method, which starting with an initial guess computed by means of a spectral method, proceeds by minimizing a nonconvex functional as in the Wirtinger flow approach [11]. There are several key distinguishing features, most notably, a distinct objective functional and novel update rules, which operate in an adaptive fashion and drop terms bearing too much influence on the search direction. These careful selection rules provide a tighter initial guess, better descent directions, and thus enhanced practical performance. On the theoretical side, we prove that for certain unstructured models of quadratic systems, our algorithms return the correct solution in linear time, i.e. in time proportional to reading the data {ai} and {yi} as soon as the ratio m/n between the number of equations and unknowns exceeds a fixed numerical constant. We extend the theory to deal with noisy systems in which we only have yi ≈ 〈ai,x〉2 and prove that our algorithms achieve a statistical accuracy, which is nearly unimprovable. We complement our theoretical study with numerical examples showing that solving random quadratic systems is both computationally and statistically not much harder than solving linear systems of the same size—hence the title of this paper. For instance, we
Gradient Descent Only Converges to Minimizers
, 2016
"... Abstract We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory. ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.
Solving systems of phaseless equations via Kaczmarz methods: A proof of concept study
, 2015
"... We study the Kaczmarz methods for solving systems of phaseless equations, i.e., the generalized phase retrieval problem. The methods extend the Kaczmarz methods for solving systems of linear equations by integrating a phase selection heuristic in each iteration and overall have the same per iterati ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We study the Kaczmarz methods for solving systems of phaseless equations, i.e., the generalized phase retrieval problem. The methods extend the Kaczmarz methods for solving systems of linear equations by integrating a phase selection heuristic in each iteration and overall have the same per iteration computational complexity. Extensive empirical performance comparisons establish the computational advantages of the Kaczmarz methods over other stateoftheart phase retrieval algorithms both in terms of the number of measurements needed for successful recovery and in terms of computation time. Preliminary convergence analysis is presented for the randomized Kaczmarz methods.