Results 1  10
of
16
BLENDENPIK: SUPERCHARGING LAPACK'S LEASTSQUARES SOLVER
"... Several innovative randomsampling and randommixing techniques for solving problems in linear algebra have been proposed in the last decade, but they have not yet made a significant impact on numerical linear algebra. We show that by using an high quality implementation of one of these techniques w ..."
Abstract

Cited by 38 (4 self)
 Add to MetaCart
(Show Context)
Several innovative randomsampling and randommixing techniques for solving problems in linear algebra have been proposed in the last decade, but they have not yet made a significant impact on numerical linear algebra. We show that by using an high quality implementation of one of these techniques we obtain a solver that performs extremely well in the traditional yardsticks of numerical linear algebra: it is significantly faster than highperformance implementations of existing stateoftheart algorithms, and it is numerically backward stable. More speci cally, we describe a leastsquare solver for dense highly overdetermined systems that achieves residuals similar to those of direct QR factorization based solvers (lapack), outperforms lapack by large factors, and scales significantly better than any QRbased solver.
Improved matrix algorithms via the subsampled randomized Hadamard transform
 SIAM J. Matrix Analysis Applications
"... Abstract. Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the subsampled randomized Hadamard transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral norms, of an SRHTbased lowrank matrix approxim ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the subsampled randomized Hadamard transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral norms, of an SRHTbased lowrank matrix approximation technique introduced by Woolfe, Liberty, Rohklin, and Tygert. We establish a slightly better Frobenius norm error bound than is currently available, and a much sharper spectral norm error bound (in the presence of reasonable decay of the singular values). Along the way, we produce several results on matrix operations with SRHTs (such as approximate matrix multiplication) that may be of independent interest. Our approach builds upon Tropp’s in “Improved Analysis of the
Accelerated lowrank visual recovery by random projection
 in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR
"... Exact recovery from contaminated visual data plays an important role in various tasks. By assuming the observed data matrix as the addition of a lowrank matrix and a sparse matrix, theoretic guarantee exists under mild conditions for exact data recovery. Practically matrix nuclear norm is adopted ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
Exact recovery from contaminated visual data plays an important role in various tasks. By assuming the observed data matrix as the addition of a lowrank matrix and a sparse matrix, theoretic guarantee exists under mild conditions for exact data recovery. Practically matrix nuclear norm is adopted as a convex surrogate of the nonconvex matrix rank function to encourage lowrank property and serves as the major component of recentlyproposed Robust Principal Component Analysis (RPCA). Recent endeavors have focused on enhancing the scalability of RPCA to largescale datasets, especially mitigating the computational burden of frequent largescale Singular Value Decomposition (SVD) inherent with the nuclear norm optimization. In our proposed scheme, the nuclear norm of an auxiliary matrix is minimized instead, which is related to the original lowrank matrix by random projection. By design, the modified optimization entails SVD on matrices of much smaller scale, as compared to the original optimization problem. Theoretic analysis well justifies the proposed scheme, along with greatly reduced optimization complexity. Both qualitative and quantitative studies are provided on various computer vision benchmarks to validate its effectiveness, including facial shadow removal, surveillance background modeling and largescale image tag transduction. It is also highlighted that the proposed solution can serve as a general principal to accelerate many other nuclear norm oriented problems in numerous tasks. 1.
The effect of coherence on sampling from matrices with orthonormal columns, and preconditioned least squares problems. arXiv preprint arXiv:1203.4809
, 2012
"... ar ..."
(Show Context)
Efficient Nonnegative Matrix Factorization with Random Projections
"... The recent years have witnessed a surge of interests in Nonnegative Matrix Factorization (NMF) in data mining and machine learning fields. Despite its elegant theory and empirical success, one of the limitations of NMF based algorithms is that it needs to store the whole data matrix in the entire pr ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
The recent years have witnessed a surge of interests in Nonnegative Matrix Factorization (NMF) in data mining and machine learning fields. Despite its elegant theory and empirical success, one of the limitations of NMF based algorithms is that it needs to store the whole data matrix in the entire process, which requires expensive storage and computation costs when the data set is large and highdimensional. In this paper, we propose to apply the random projection techniques to accelerate the NMF process. Both theoretical analysis and experimental validations will be presented to demonstrate the effectiveness of the proposed strategy. 1
Nearoptimal coresets for leastsquares regression
 IEEE Transactions on Information Theory
, 2013
"... Abstract—We study the (constrained) leastsquares regression as well asmultiple response leastsquares regression and ask the question of whether a subset of the data, a coreset, suffices to compute a good approximate solution to the regression. We give deterministic, loworder polynomialtime alg ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract—We study the (constrained) leastsquares regression as well asmultiple response leastsquares regression and ask the question of whether a subset of the data, a coreset, suffices to compute a good approximate solution to the regression. We give deterministic, loworder polynomialtime algorithms to construct such coresets with approximation guarantees, together with lower bounds indicating that there is not much room for improvement upon our results. Index Terms—Least mean square algorithms, machine learning algorithms, regression analysis. I.
Efficient Dimensionality Reduction for Canonical Correlation Analysis
"... We present a fast algorithm for approximate Canonical Correlation Analysis (CCA). Given a pair of tallandthin matrices, the proposed algorithm first employs a randomized dimensionality reduction transform to reduce the size of the input matrices, and then applies any standard CCA algorithm to the ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
We present a fast algorithm for approximate Canonical Correlation Analysis (CCA). Given a pair of tallandthin matrices, the proposed algorithm first employs a randomized dimensionality reduction transform to reduce the size of the input matrices, and then applies any standard CCA algorithm to the new pair of matrices. The algorithm computes an approximate CCA to the original pair of matrices with provable guarantees, while requiring asymptotically less operations than the stateoftheart exact algorithms. 1.
Approximate spectral clustering via randomized sketching. preprint arXiv:1311.2854
, 2013
"... Spectral clustering is arguably one of the most important algorithms in data mining and machine intelligence; however, its computational complexity makes it a challenge to use it for large scale data analysis. Recently, several approximation algorithms for spectral clustering have been developed in ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Spectral clustering is arguably one of the most important algorithms in data mining and machine intelligence; however, its computational complexity makes it a challenge to use it for large scale data analysis. Recently, several approximation algorithms for spectral clustering have been developed in order to alleviate the relevant costs, but theoretical results are lacking. In this paper, we present a novel approximation algorithm for spectral clustering with strong theoretical evidence of its performance. Our algorithm is based on approximating the eigenvectors of the Laplacian matrix using random projections, a.k.a randomized sketching. Our experimental results demonstrate that the proposed approximation algorithm compares remarkably well to the exact algorithm. 1