Results 11  20
of
239
Paved with good intentions: Analysis of a randomized Kaczmarz method
, 2012
"... ABSTRACT. The block Kaczmarz method is an iterative scheme for solving overdetermined leastsquares problems. At each step, the algorithm projects the current iterate onto the solution space of a subset of the constraints. This paper describes a block Kaczmarz algorithm that uses a randomized contro ..."
Abstract

Cited by 20 (5 self)
 Add to MetaCart
ABSTRACT. The block Kaczmarz method is an iterative scheme for solving overdetermined leastsquares problems. At each step, the algorithm projects the current iterate onto the solution space of a subset of the constraints. This paper describes a block Kaczmarz algorithm that uses a randomized control scheme to choose the subset at each step. This algorithm is the first block Kaczmarz method with an (expected) linear rate of convergence that can be expressed in terms of the geometric properties of the matrix and its submatrices. The analysis reveals that the algorithm is most effective when it is given a good row paving of the matrix, a partition of the rows into wellconditioned blocks. The operator theory literature provides detailed information about the existence and construction of good row pavings. Together, these results yield an efficient block Kaczmarz scheme that applies to many overdetermined leastsquares problem. 1.
An algorithm for the principal component analysis of large data sets
 IEEE 13th International Conference on Data Mining ©2013 IEEE DOI 10.1109/ICDM.2013.155
, 2011
"... Abstract. Recently popularized randomized methods for principal component analysis (PCA) efficiently and reliably produce nearly optimal accuracy — even on parallel processors — unlike the classical (deterministic) alternatives. We adapt one of these randomized methods for use with data sets that ar ..."
Abstract

Cited by 18 (2 self)
 Add to MetaCart
(Show Context)
Abstract. Recently popularized randomized methods for principal component analysis (PCA) efficiently and reliably produce nearly optimal accuracy — even on parallel processors — unlike the classical (deterministic) alternatives. We adapt one of these randomized methods for use with data sets that are too large to be stored in randomaccess memory (RAM). (The traditional terminology is that our procedure works efficiently outofcore.) We illustrate the performance of the algorithm via several numerical examples. For example, we report on the PCA of a data set stored on disk that is so large that less than a hundredth of it can fit in our computer’s RAM.
The JohnsonLindenstrauss Transform itself preserves differential privacy
 In IEEE 53rd Annual Symposium on Foundations of Computer Science (FOCS
, 2012
"... ar ..."
(Show Context)
Improved matrix algorithms via the subsampled randomized Hadamard transform
 SIAM J. Matrix Analysis Applications
"... Abstract. Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the subsampled randomized Hadamard transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral norms, of an SRHTbased lowrank matrix approxim ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the subsampled randomized Hadamard transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral norms, of an SRHTbased lowrank matrix approximation technique introduced by Woolfe, Liberty, Rohklin, and Tygert. We establish a slightly better Frobenius norm error bound than is currently available, and a much sharper spectral norm error bound (in the presence of reasonable decay of the singular values). Along the way, we produce several results on matrix operations with SRHTs (such as approximate matrix multiplication) that may be of independent interest. Our approach builds upon Tropp’s in “Improved Analysis of the
Hierarchical interpolative factorization for elliptic operators: differential equations
 Comm. Pure Appl. Math
"... This paper introduces the hierarchical interpolative factorization for integral equations (HIFIE) associated with elliptic problems in two and three dimensions. This factorization takes the form of an approximate generalized LU decomposition that permits the efficient application of the discretiz ..."
Abstract

Cited by 16 (5 self)
 Add to MetaCart
(Show Context)
This paper introduces the hierarchical interpolative factorization for integral equations (HIFIE) associated with elliptic problems in two and three dimensions. This factorization takes the form of an approximate generalized LU decomposition that permits the efficient application of the discretized operator and its inverse. HIFIE is based on the recursive skeletonization algorithm but incorporates a novel combination of two key features: (1) a matrix factorization framework for sparsifying structured dense matrices and (2) a recursive dimensional reduction strategy to decrease the cost. Thus, higherdimensional problems are effectively mapped to one dimension, and we conjecture that constructing, applying, and inverting the factorization all have linear or quasilinear complexity. Numerical experiments support this claim and further demonstrate the performance of our algorithm as a generalized fast multipole method, direct solver, and preconditioner. HIFIE is compatible with geometric adaptivity and can handle both boundary and volume problems. MATLAB ® codes are freely available.
LSRN: A parallel iterative solver for strongly over or underdetermined systems
, 2011
"... Abstract. We describe a parallel iterative least squares solver named LSRN that is based on random normal projection. LSRN computes the minlength solution to minx∈Rn ‖Ax − b‖2, where A ∈ Rm×n with m n or m n, and where A may be rankdeficient. Tikhonov regularization may also be included. Since A ..."
Abstract

Cited by 16 (6 self)
 Add to MetaCart
(Show Context)
Abstract. We describe a parallel iterative least squares solver named LSRN that is based on random normal projection. LSRN computes the minlength solution to minx∈Rn ‖Ax − b‖2, where A ∈ Rm×n with m n or m n, and where A may be rankdeficient. Tikhonov regularization may also be included. Since A is only involved in matrixmatrix and matrixvector multiplications, it can be a dense or sparse matrix or a linear operator, and LSRN automatically speeds up when A is sparse or a fast linear operator. The preconditioning phase consists of a random normal projection, which is embarrassingly parallel, and a singular value decomposition of size dγmin(m,n)e × min(m,n), where γ is moderately larger than 1, e.g., γ = 2. We prove that the preconditioned system is wellconditioned, with a strong concentration result on the extreme singular values, and hence that the number of iterations is fully predictable when we apply LSQR or the Chebyshev semiiterative method. As we demonstrate, the Chebyshev method is particularly efficient for solving large problems on clusters with high communication cost. Numerical results demonstrate that on a sharedmemory machine, LSRN outperforms LAPACK’s DGELSD on large dense problems, and MATLAB’s backslash (SuiteSparseQR) on sparse problems. Further experiments demonstrate that LSRN scales well on an
Improving CUR Matrix Decomposition and the Nyström Approximation via Adaptive Sampling
"... The CUR matrix decomposition and the Nyström approximation are two important lowrank matrix approximation techniques. The Nyström method approximates a symmetric positive semidefinite matrix in terms of a small number of its columns, while CUR approximates an arbitrary data matrix by a small number ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
The CUR matrix decomposition and the Nyström approximation are two important lowrank matrix approximation techniques. The Nyström method approximates a symmetric positive semidefinite matrix in terms of a small number of its columns, while CUR approximates an arbitrary data matrix by a small number of its columns and rows. Thus, CUR decomposition can be regarded as an extension of the Nyström approximation. In this paper we establish a more general error bound for the adaptive column/row sampling algorithm, based on which we propose more accurate CUR and Nyström algorithms with expected relativeerror bounds. The proposed CUR and Nyström algorithms also have low time complexity and can avoid maintaining the whole data matrix in RAM. In addition, we give theoretical analysis for the lower error bounds of the standard Nyström method and the ensemble Nyström method. The main theoretical results established in this paper are novel, and our analysis makes no special assumption on the data matrices.
Extremescale UQ for Bayesian inverse problems governed by PDEs
 in SC12: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
, 2012
"... Abstract—Quantifying uncertainties in largescale simulations has emerged as the central challenge facing CS&E. When the simulations require supercomputers, and uncertain parameter dimensions are large, conventional UQ methods fail. Here we address uncertainty quantification for largescale inve ..."
Abstract

Cited by 16 (10 self)
 Add to MetaCart
(Show Context)
Abstract—Quantifying uncertainties in largescale simulations has emerged as the central challenge facing CS&E. When the simulations require supercomputers, and uncertain parameter dimensions are large, conventional UQ methods fail. Here we address uncertainty quantification for largescale inverse problems in a Bayesian inference framework: given data and model uncertainties, find the pdf describing parameter uncertainties. To overcome the curse of dimensionality of conventional methods, we exploit the fact that the data are typically informative about lowdimensional manifolds of parameter space to construct low rank approximations of the covariance matrix of the posterior pdf via a matrixfree randomized method. We obtain a method that scales independently of the forward problem dimension, the uncertain parameter dimension, the data dimension, and the number of cores. We apply the method to the Bayesian solution of an inverse problem in 3D global seismic wave propagation with over one million uncertain earth model parameters, 630 million wave propagation unknowns, on up to 262K cores, for which we obtain a factor of over 2000 reduction in problem dimension. This makes UQ tractable for the inverse problem.
Tail bounds for all eigenvalues of a sum of random matrices
, 2011
"... This work introduces the minimax Laplace transform method, a modification of the cumulantbased matrix Laplace transform method developed in [Tro11c] that yields both upper and lower bounds on each eigenvalue of a sum of random selfadjoint matrices. This machinery is used to derive eigenvalue ana ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
This work introduces the minimax Laplace transform method, a modification of the cumulantbased matrix Laplace transform method developed in [Tro11c] that yields both upper and lower bounds on each eigenvalue of a sum of random selfadjoint matrices. This machinery is used to derive eigenvalue analogs of the classical Chernoff, Bennett, and Bernstein bounds. Two examples demonstrate the efficacy of the minimax Laplace transform. The first concerns the effects of column sparsification on the spectrum of a matrix with orthonormal rows. Here, the behavior of the singular values can be described in terms of coherencelike quantities. The second example addresses the question of relative accuracy in the estimation of eigenvalues of the covariance matrix of a random process. Standard results on the convergence of sample covariance matrices provide bounds on the number of samples needed to obtain relative accuracy in the spectral norm, but these results only guarantee relative accuracy in the estimate of the maximum eigenvalue. The minimax Laplace transform argument establishes that if the lowest eigenvalues decay sufficiently fast, Ω(ε−2κ2` ` log p) samples, where κ ` = λ1(C)/λ`(C), are sufficient to ensure that the dominant ` eigenvalues of the covariance matrix of a N (0,C) random vector are estimated to within a factor of 1 ± ε with high probability.