Results 1  10
of
20
Inexact Coordinate Descent: Complexity and Preconditioning
, 2013
"... In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a parti ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is solved exactly. In his work we relax this requirement, and allow for the subproblem to be solved inexactly, leading to an inexact block coordinate descent method. Our approach incorporates the best known results for exact updates as a special case. Moreover, these theoretical guarantees are complemented by practical considerations: the use of iterative techniques to determine the update as well as the use of preconditioning for further acceleration.
Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm
 ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS
, 2014
"... ..."
(Show Context)
A.: Randomized block kaczmarz method with projection for solving least squares
, 2014
"... ABSTRACT. The Kaczmarz method is an iterative method for solving overcomplete linear systems of equations Ax = b. The randomized version of the Kaczmarz method put forth by Strohmer and Vershynin iteratively projects onto a randomly chosen solution space given by a single row of the matrix A and con ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
ABSTRACT. The Kaczmarz method is an iterative method for solving overcomplete linear systems of equations Ax = b. The randomized version of the Kaczmarz method put forth by Strohmer and Vershynin iteratively projects onto a randomly chosen solution space given by a single row of the matrix A and converges exponentially in expectation to the solution of a consistent system. In this paper we analyze two block versions of the method each with a randomized projection, that converge in expectation to the least squares solution of inconsistent systems. Our approach utilizes a paving of the matrix A to guarantee exponential convergence, and suggests that paving yields a significant improvement in performance in certain regimes. The proposed method is an extension of the block Kaczmarz method analyzed by Needell and Tropp and the Randomized Extended Kaczmarz method of Zouzias and Freris. The contribution is thus twofold; unlike the standard Kaczmarz method, our methods converge to the leastsquares solution of inconsistent systems, and by using appropriate blocks of the matrix this convergence can be significantly accelerated. Numerical experiments suggest that the proposed algorithm can indeed lead to advantages in practice. 1.
A fast randomized Kaczmarz algorithm for sparse solutions of consistent linear systems,” http://arxiv.org/abs/1305.3803
, 2013
"... ar ..."
Solving systems of phaseless equations via Kaczmarz methods: A proof of concept study
, 2015
"... We study the Kaczmarz methods for solving systems of phaseless equations, i.e., the generalized phase retrieval problem. The methods extend the Kaczmarz methods for solving systems of linear equations by integrating a phase selection heuristic in each iteration and overall have the same per iterati ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We study the Kaczmarz methods for solving systems of phaseless equations, i.e., the generalized phase retrieval problem. The methods extend the Kaczmarz methods for solving systems of linear equations by integrating a phase selection heuristic in each iteration and overall have the same per iteration computational complexity. Extensive empirical performance comparisons establish the computational advantages of the Kaczmarz methods over other stateoftheart phase retrieval algorithms both in terms of the number of measurements needed for successful recovery and in terms of computation time. Preliminary convergence analysis is presented for the randomized Kaczmarz methods.
Extending kaczmarz algorithm with soft constraints for user interface layout
 In Proceedings of 25th International Conference on Tools with Artificial Intelligence (ICTAI
, 2013
"... ABSTRACT. The Kaczmarz method is an iterative method for solving large systems of equations that projects iterates orthogonally onto the solution space of each equation. In contrast to direct methods such as Gaussian elimination or QRfactorization, this algorithm is efficient for problems with spa ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
ABSTRACT. The Kaczmarz method is an iterative method for solving large systems of equations that projects iterates orthogonally onto the solution space of each equation. In contrast to direct methods such as Gaussian elimination or QRfactorization, this algorithm is efficient for problems with sparse matrices, as they appear in constraintbased user interface (UI) layout specifications. However, the Kaczmarz method as described in the literature has its limitations: it considers only equality constraints and does not support soft constraints, which makes it inapplicable to the UI layout problem. In this paper we extend the Kaczmarz method for solving specifications containing soft constraints, using the prioritized IIS detection algorithm. Furthermore, the performance and convergence of the proposed algorithms are evaluated empirically using randomly generated UI layout specifications of various sizes. The results show that these methods offer improvements in performance over standard methods like Matlab’s LINPROG, a wellknown efficient linear programming solver. 1.
A NOTE ON COLUMN SUBSET SELECTION
, 2013
"... ABSTRACT. Given a matrix U, using a deterministic method, we extract a "large " submatrix of Ũ (whose columns are obtained by normalizing those of U) and estimate its smallest and largest singular value. We apply this result to the study of contact points of the unit ball with its maximal ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
ABSTRACT. Given a matrix U, using a deterministic method, we extract a "large " submatrix of Ũ (whose columns are obtained by normalizing those of U) and estimate its smallest and largest singular value. We apply this result to the study of contact points of the unit ball with its maximal volume ellipsoid. We consider also the paving problem and give a deterministic algorithm to partition a matrix into almost isometric blocks recovering previous results of BourgainTzafriri and Tropp. Finally, we partially answer a question raised by Naor about finding an algorithm in the spirit of BatsonSpielmanSrivastava’s work to extract a "large " square submatrix of "small" norm.
Specialization in Signal Processing and System Identification
, 2014
"... On some sparsity related problems and the randomized Kaczmarz algorithm ..."