Results 1  10
of
16
Guaranteed matrix completion via nonconvex factorization
, 2014
"... Matrix factorization is a popular approach for largescale matrix completion and constitutes a basic component of many solutions for Netflix Prize competition. In this approach, the unknown lowrank matrix is expressed as the product of two much smaller matrices so that the lowrank property is auto ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
Matrix factorization is a popular approach for largescale matrix completion and constitutes a basic component of many solutions for Netflix Prize competition. In this approach, the unknown lowrank matrix is expressed as the product of two much smaller matrices so that the lowrank property is automatically fulfilled. The resulting optimization problem, even with huge size, can be solved (to stationary points) very efficiently through standard optimization algorithms such as alternating minimization and stochastic gradient descent (SGD). However, due to the nonconvexity caused by the factorization model, there is a limited theoretical understanding of whether these algorithms will generate a good solution. In this paper, we establish a theoretical guarantee for the factorization based formulation to correctly recover the underlying lowrank matrix. In particular, we show that under similar conditions to those in previous works, many standard optimization algorithms converge to the global optima of the factorization based formulation, thus recovering the true lowrank matrix. To the best of our knowledge, our result is the first one that provides recovery guarantee for many standard algorithms such as gradient descent, SGD and block coordinate gradient descent. Our result also applies to alternating minimization, and a notable difference from previous studies on alternating minimization is that we do not need the resampling scheme (i.e. using independent samples in each iteration).
On the power of adaptivity in matrix completion and approximation
, 2014
"... We consider the related tasks of matrix completion and matrix approximation from missing data and propose adaptive sampling procedures for both problems. We show that adaptive sampling allows one to eliminate standard incoherence assumptions on the matrix row space that are necessary for passive sam ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
We consider the related tasks of matrix completion and matrix approximation from missing data and propose adaptive sampling procedures for both problems. We show that adaptive sampling allows one to eliminate standard incoherence assumptions on the matrix row space that are necessary for passive sampling procedures. For exact recovery of a lowrank matrix, our algorithm judiciously selects a few columns to observe in full and, with few additional measurements, projects the remaining columns onto their span. This algorithm exactly recovers an n × n rank r matrix using O(nrµ0 log2(r)) observations, where µ0 is a coherence parameter on the column space of the matrix. In addition to completely eliminating any row space assumptions that have pervaded the literature, this algorithm enjoys a better sample complexity than any existing matrix completion algorithm. To certify that this improvement is due to adaptive sampling, we establish that row space coherence is necessary for passive sampling algorithms to achieve nontrivial sample complexity bounds. For constructing a lowrank approximation to a highrank input matrix, we propose a simple algorithm that thresholds the singular values of a zerofilled version of the input matrix. The algorithm computes an approximation that is nearly as good as the best rankr approximation using O(nrµ log2(n)) samples, where µ is a slightly different coherence parameter on the matrix columns. Again we eliminate assumptions on the row space. 1
Tighter lowrank approximation via sampling the leveraged element
 CoRR
"... Abstract In this work, we propose a new randomized algorithm for computing a lowrank approximation to a given matrix. Taking an approach different from existing literature, our method first involves a specific biased sampling, with an element being chosen based on the leverage scores of its row an ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Abstract In this work, we propose a new randomized algorithm for computing a lowrank approximation to a given matrix. Taking an approach different from existing literature, our method first involves a specific biased sampling, with an element being chosen based on the leverage scores of its row and column, and then involves weighted alternating minimization over the factored form of the intended lowrank matrix, to minimize error only on these samples. Our method can leverage input sparsity, yet produce approximations in spectral (as opposed to the Frobenius) norm; this combines the best aspects of otherwise disparate current results, but with a dependence on the condition number κ = σ 1 /σ r . In particular we require O(nnz(M ) + ) computations to generate a rankr approximation to M in spectral norm. In contrast, the best existing method requires O(nnz(M ) + nr 2 4 ) time to compute an approximation in Frobenius norm. Besides the tightness in spectral norm, we have a better dependence on the error . Our method is naturally and highly parallelizable. This approach also leads to a new method to directly compute a lowrank approximation (in efficient factored form) to the product of two given matrices; it computes a small random set of entries of the product, and then executes weighted alternating minimization (as before) on these.
Optimal Testing for Planted Satisfiability Problems
"... Abstract. We study the problem of detecting planted solutions in a random satisfiability formula. Adopting the formalism of hypothesis testing in statistical analysis, we describe the minimax optimal rates of detection. Our analysis relies on the study of the number of satisfying assignments, for w ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract. We study the problem of detecting planted solutions in a random satisfiability formula. Adopting the formalism of hypothesis testing in statistical analysis, we describe the minimax optimal rates of detection. Our analysis relies on the study of the number of satisfying assignments, for which we prove new results. We also address algorithmic issues, and give a computationally efficient test with optimal statistical performance. This result is compared to an averagecase hypothesis on the hardness of refuting satisfiability of random formulas.
A Characterization of Deterministic Sampling Patterns for LowRank Matrix Completion
, 2015
"... ..."
Article A LowRank Matrix Recovery Approach for Energy Efficient EEG Acquisition for a Wireless Body Area Network
, 2014
"... sensors ..."
(Show Context)
CUR Algorithm for Partially Observed Matrices
"... CUR matrix decomposition computes the low rank approximation of a given matrix by using the actual rows and columns of the matrix. It has been a very useful tool for handling large matrices. One limitation with the existing algorithms for CURmatrix decomposition is that they cannot deal with entrie ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
CUR matrix decomposition computes the low rank approximation of a given matrix by using the actual rows and columns of the matrix. It has been a very useful tool for handling large matrices. One limitation with the existing algorithms for CURmatrix decomposition is that they cannot deal with entries in a partially observed matrix, while incomplete matrices are found in many real world applications. In this work, we alleviate this limitation by developing a CUR decomposition algorithm for partially observed matrices. In particular, the proposed algorithm computes the low rank approximation of the target matrix based on (i) the randomly sampled rows and columns, and (ii) a subset of observed entries that are randomly sampled from the matrix. Our analysis shows the error bound, measured by spectral norm, for the proposed algorithm when the target matrix is of full rank. We also show that only O(nr ln r) observed entries are needed by the proposed algorithm to perfectly recover a rank r matrix of size n × n, which improves the sample complexity of the existing algorithms for matrix completion. Empirical studies on both synthetic and realworld datasets verify our theoretical claims and demonstrate the effectiveness of the proposed algorithm.
Completing any lowrank matrix, provably
 ArXiv:1306.2979
, 2013
"... Abstract Matrix completion, i.e., the exact and provable recovery of a lowrank matrix from a small subset of its elements, is currently only known to be possible if the matrix satisfies a restrictive structural constraintknown as incoherenceon its row and column spaces. In these cases, the subse ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract Matrix completion, i.e., the exact and provable recovery of a lowrank matrix from a small subset of its elements, is currently only known to be possible if the matrix satisfies a restrictive structural constraintknown as incoherenceon its row and column spaces. In these cases, the subset of elements is assumed to be sampled uniformly at random. In this paper, we show that any rankr nbyn matrix can be exactly recovered from as few as O(nr log 2 n) randomly chosen elements, provided this random choice is made according to a specific biased distribution suitably dependent on the coherence structure of the matrix: the probability of any element being sampled should be at least a constant times the sum of the leverage scores of the corresponding row and column. Moreover, we prove that this specific form of sampling is nearly necessary, in a natural precise sense; this implies that many other perhaps more intuitive sampling schemes fail. We further establish three ways to use the above result for the setting when leverage scores are not known a priori. (a) We describe a provablycorrect sampling strategy for the case when only the column space is incoherent and no assumption or knowledge of the row space is required. (b) We propose a twophase sampling procedure for general matrices that first samples to estimate leverage scores followed by sampling for exact recovery. These two approaches assume control over the sampling procedure. (c) By using our main theorem in a reverse direction, we provide an analysis showing the advantages of the (empirically successful) weighted nuclear/tracenorm minimization approach over the vanilla unweighted formulation given nonuniformly distributed observed elements. This approach does not require controlled sampling or knowledge of the leverage scores.
Noisy Tensor Completion via the SumofSquares Hierarchy
, 2016
"... Abstract In the noisy tensor completion problem we observe m entries (whose location is chosen uniformly at random) from an unknown n 1 × n 2 × n 3 tensor T . We assume that T is entrywise close to being rank r. Our goal is to fill in its missing entries using as few observations as possible. Let ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract In the noisy tensor completion problem we observe m entries (whose location is chosen uniformly at random) from an unknown n 1 × n 2 × n 3 tensor T . We assume that T is entrywise close to being rank r. Our goal is to fill in its missing entries using as few observations as possible. Let n = max(n 1 , n 2 , n 3 ). We show that if m = n 3/2 r then there is a polynomial time algorithm based on the sixth level of the sumofsquares hierarchy for completing it. Our estimate agrees with almost all of T 's entries almost exactly and works even when our observations are corrupted by noise. This is also the first algorithm for tensor completion that works in the overcomplete case when r > n, and in fact it works all the way up to r = n 3/2− . Our proofs are short and simple and are based on establishing a new connection between noisy tensor completion (through the language of Rademacher complexity) and the task of refuting random constant satisfaction problems. This connection seems to have gone unnoticed even in the context of matrix completion. Furthermore, we use this connection to show matching lower bounds. Our main technical result is in characterizing the Rademacher complexity of the sequence of norms that arise in the sumofsquares relaxations to the tensor nuclear norm. These results point to an interesting new direction: Can we explore computational vs. sample complexity tradeoffs through the sumofsquares hierarchy?