Results 1 
4 of
4
Randomized block krylov methods for stronger and faster approximate singular value decomposition.
 In Advances in Neural Information Processing Systems 28 (NIPS),
, 2015
"... Abstract Since being analyzed by Rokhlin, Szlam, and Tygert [1] and popularized by Halko, Martinsson, and Tropp [2], randomized Simultaneous Power Iteration has become the method of choice for approximate singular value decomposition. It is more accurate than simpler sketching algorithms, yet stil ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract Since being analyzed by Rokhlin, Szlam, and Tygert [1] and popularized by Halko, Martinsson, and Tropp [2], randomized Simultaneous Power Iteration has become the method of choice for approximate singular value decomposition. It is more accurate than simpler sketching algorithms, yet still converges quickly for any matrix, independently of singular value gaps. AfterÕ(1/ ) iterations, it gives a lowrank approximation within (1 + ) of optimal for spectral norm error. We give the first provable runtime improvement on Simultaneous Iteration: a simple randomized block Krylov method, closely related to the classic Block Lanczos algorithm, gives the same guarantees in justÕ(1/ √ ) iterations and performs substantially better experimentally. Despite their long history, our analysis is the first of a Krylov subspace method that does not depend on singular value gaps, which are unreliable in practice. Furthermore, while it is a simple accuracy benchmark, even (1 + ) error for spectral norm lowrank approximation does not imply that an algorithm returns high quality principal components, a major issue for data applications. We address this problem for the first time by showing that both Block Krylov Iteration and a minor modification of Simultaneous Iteration give nearly optimal PCA for any matrix. This result further justifies their strength over noniterative sketching methods. Finally, we give insight beyond the worst case, justifying why both algorithms can run much faster in practice than predicted. We clarify how simple techniques can take advantage of common matrix properties to significantly improve runtime.
Low Rank Approximation using Error Correcting Coding Matrices
"... Abstract Lowrank matrix approximation is an integral component of tools such as principal component analysis (PCA), as well as is an important instrument used in applications like web search, text mining and computer vision, e.g., face recognition. Recently, randomized algorithms were proposed to ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Lowrank matrix approximation is an integral component of tools such as principal component analysis (PCA), as well as is an important instrument used in applications like web search, text mining and computer vision, e.g., face recognition. Recently, randomized algorithms were proposed to effectively construct low rank approximations of large matrices. In this paper, we show how matrices from error correcting codes can be used to find such low rank approximations. The benefits of using these code matrices are the following: (i) They are easy to generate and they reduce randomness significantly. (ii) Code matrices have low coherence and have a better chance of preserving the geometry of an entire subspace of vectors; (iii) Unlike Fourier transforms or Hadamard matrices, which require sampling O(k log k) columns for a rankk approximation, the log factor is not necessary in the case of code matrices. (iv) Under certain conditions, the approximation errors can be better and the singular values obtained can be more accurate, than those obtained using Gaussian random matrices and other structured random matrices.
Spectral Gap Error Bounds for Improving CUR Matrix Decomposition and the Nyström Method
"... The CUR matrix decomposition and the related Nyström method build lowrank approximations of data matrices by selecting a small number of representative rows and columns of the data. Here, we introduce novel spectral gap error bounds that judiciously exploit the potentially rapid spectrum decay i ..."
Abstract
 Add to MetaCart
(Show Context)
The CUR matrix decomposition and the related Nyström method build lowrank approximations of data matrices by selecting a small number of representative rows and columns of the data. Here, we introduce novel spectral gap error bounds that judiciously exploit the potentially rapid spectrum decay in the input matrix, a most common occurrence in machine learning and data analysis. Our error bounds are much tighter than existing ones for matrices with rapid spectrum decay, and they justify the use of a constant amount of oversampling relative to the rank parameter k, i.e, when the number of columns/rows is ` = k + O(1). We demonstrate our analysis on a novel deterministic algorithm, StableCUR, which additionally eliminates a previously unrecognized source of potential instability in CUR decompositions. While our algorithm accepts any method of row and column selection, we implement it with a recent column selection scheme with strong singular value bounds. Empirical results on various classes of real world data matrices demonstrate that our algorithm is as efficient as, and often outperforms, competing algorithms.
‖A−QcBkQTr ‖2F ≤ ‖A−Qc
"... First we prove a useful theorem, which is similar to [3] theorem 3.5. Theorem S.1. Let Qc be an m × c columnorthonormal matrix matrix. Let Qr be a n × r columnorthonormal matrix. Let Bk be the rankk truncated SVD of QTc AQr. We have: min rank(B)≤k,B∈Rc×r ∥∥A−QcBQTr ∥∥2F = ∥∥A−QcBkQTr ∥∥2F (S.1) I ..."
Abstract
 Add to MetaCart
(Show Context)
First we prove a useful theorem, which is similar to [3] theorem 3.5. Theorem S.1. Let Qc be an m × c columnorthonormal matrix matrix. Let Qr be a n × r columnorthonormal matrix. Let Bk be the rankk truncated SVD of QTc AQr. We have: min rank(B)≤k,B∈Rc×r ∥∥A−QcBQTr ∥∥2F = ∥∥A−QcBkQTr ∥∥2F (S.1) In addition:∥∥A−QcBkQTr ∥∥2F ≤ ‖A−Ak‖2F + ∥∥(I −QcQTc)Ak∥∥2F + ∥∥Ak (I −QrQTr)∥∥2F