Results 1  10
of
25
SUPERLINEAR CONVERGENCE OF CONJUGATE GRADIENTS
, 2001
"... We give a theoretical explanation for superlinear convergence behavior observed while solving large symmetric systems of equations using the conjugate gradient method or other Krylov subspace methods. We present a new bound on the relative error after n iterations. This bound is valid in an asympto ..."
Abstract

Cited by 24 (6 self)
 Add to MetaCart
We give a theoretical explanation for superlinear convergence behavior observed while solving large symmetric systems of equations using the conjugate gradient method or other Krylov subspace methods. We present a new bound on the relative error after n iterations. This bound is valid in an asymptotic sense when the size N of the system grows together with the number of iterations. The bound depends on the asymptotic eigenvalue distribution and on the ratio n/N. Under appropriate conditions we show that the bound is asymptotically sharp. Our findings are related to some recent results concerning asymptotics of discrete orthogonal polynomials. An important tool in our investigations is a constrained energy problem in logarithmic potential theory. The new asymptotic bounds for the rate of convergence are illustrated by discussing Toeplitz systems as well as a model problem stemming from the discretization of the Poisson equation.
Convergence analysis of Krylov subspace iterations with methods from potential theory
 SIAM Review
"... Abstract. Krylov subspace iterations are among the bestknown and most widely used numerical methods for solving linear systems of equations and for computing eigenvalues of large matrices. These methods are polynomial methods whose convergence behavior is related to the behavior of polynomials on t ..."
Abstract

Cited by 16 (2 self)
 Add to MetaCart
(Show Context)
Abstract. Krylov subspace iterations are among the bestknown and most widely used numerical methods for solving linear systems of equations and for computing eigenvalues of large matrices. These methods are polynomial methods whose convergence behavior is related to the behavior of polynomials on the spectrum of the matrix. This leads to an extremal problem in polynomial approximation theory: how small can a monic polynomial of a given degree be on the spectrum? This survey gives an introduction to a recently developed technique to analyze this extremal problem in the case of symmetric matrices. It is based on global information on the spectrum in the sense that the eigenvalues are assumed to be distributed according to a certain measure. Then depending on the number of iterations, the Lanczos method for the calculation of eigenvalues finds those eigenvalues that lie in a certain region, which is characterized by means of a constrained equilibrium problem from potential theory. The same constrained equilibrium problem also describes the superlinear convergence of conjugate gradients and other iterative methods for solving linear systems. Key words. Krylov subspace iterations, Ritz values, eigenvalue distribution, equilibrium measure, contrained equilibrium, potential theory AMS subject classifications. 15A18, 31A05, 31A15, 65F15 1. Introduction. Krylov
Coclustering documents and words using bipartite isoperimetric graph partitioning
 In ICDM
, 2006
"... In this paper, we present a novel graph theoretic approach to the problem of documentword coclustering. In our approach, documents and words are modeled as the two vertices of a bipartite graph. We then propose Isoperimetric Coclustering Algorithm (ICA) a new method for partitioning the docum ..."
Abstract

Cited by 16 (4 self)
 Add to MetaCart
(Show Context)
In this paper, we present a novel graph theoretic approach to the problem of documentword coclustering. In our approach, documents and words are modeled as the two vertices of a bipartite graph. We then propose Isoperimetric Coclustering Algorithm (ICA) a new method for partitioning the documentword bipartite graph. ICA requires a simple solution to a sparse system of linear equations instead of the eigenvalue or SVD problem in the popular spectral coclustering approach. Our extensive experiments performed on publicly available datasets demonstrate the advantages of ICA over spectral approach in terms of the quality, efficiency and stability in partitioning the documentword bipartite graph. 1
Superlinear CG Convergence for Special RightHand Sides
 Electronic Transactions on Numerical Analysis
"... Recently, we gave a theoretical explanation for superlinear convergence behavior observed while solving large symmetric systems of equations using the Conjugate Gradient method. Roughly speaking, one may observe superlinear convergence while solving a sequence of (symmetric positive definite) linear ..."
Abstract

Cited by 14 (1 self)
 Add to MetaCart
Recently, we gave a theoretical explanation for superlinear convergence behavior observed while solving large symmetric systems of equations using the Conjugate Gradient method. Roughly speaking, one may observe superlinear convergence while solving a sequence of (symmetric positive definite) linear systems if the asymptotic eigenvalue distribution of the sequence of the corresponding matrices of coefficients is far from an equilibrium distribution. However, it is well known that the convergence of the Conjugate Gradient or other Krylov subspace methods does not only depend on the spectrum but also on the righthand side of the underlying system and the starting vector. In this paper we present a family of examples based on the discretization via finite differences of the one dimensional Poisson problem where the asymptotic distribution equals an equilibrium distribution but one may as well observe superlinear convergence according to the particular choice of the righthand sides. Our findings are related to some recent results concerning asymptotics of discrete orthogonal polynomials. An important tool in our investigations is a constrained energy problem in logarithmic potential theory, where an additional external field is used being related to our particular righthand sides. Key words. Superlinear convergence, Conjugate gradients, Krylov subspace methods, Logarithmic potential theory. AMS subject classifications. 65F10, 65E05, 31A99, 41A10. 1.
The LanczosRitz values appearing in an orthogonal similarity reduction of a matrix into semiseparable form
, 2003
"... ..."
Convergence of the isometric Arnoldi process
, 2003
"... It is well known that the performance of eigenvalue algorithms such as the Lanczos and the Arnoldi method depends on the distribution of eigenvalues. Under fairly general assumptions we characterize the region of good convergence for the Isometric Arnoldi Process. We also determine bounds for the ra ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
It is well known that the performance of eigenvalue algorithms such as the Lanczos and the Arnoldi method depends on the distribution of eigenvalues. Under fairly general assumptions we characterize the region of good convergence for the Isometric Arnoldi Process. We also determine bounds for the rate of convergence and we prove sharpness of these bounds. The distribution of isometric Ritz values is obtained as the minimizer of an extremal problem. We use techniques from logarithmic potential theory in proving these results.
On the convergence of rational Ritz values
 SIAM J. Matrix Anal. Appl
, 2010
"... Abstract. Ruhe’s rational Krylov method is a popular tool for approximating eigenvalues of a given matrix, though its convergence behavior is far from being fully understood. Under fairly general assumptions we characterize in an asymptotic sense which eigenvalues of a Hermitian matrix are approxima ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Ruhe’s rational Krylov method is a popular tool for approximating eigenvalues of a given matrix, though its convergence behavior is far from being fully understood. Under fairly general assumptions we characterize in an asymptotic sense which eigenvalues of a Hermitian matrix are approximated by rational Ritz values and how fast this approximation takes place. Our main tool is a constrained extremal problem from logarithmic potential theory, where an additional external field is required for taking into account the poles of the underlying rational Krylov space. Several examples illustrate our analytic results. Key words. Rational Krylov, Ritz values, orthogonal rational functions, logarithmic potential theory. AMS subject classifications. 15A18, 31A05, 31A15, 65F15 1. Introduction. In order to approximate parts of the spectrum Λ(A) of a Hermitian matrix A ∈ CN×N, a widely used approach is to project A onto an ndimensional subspace of CN, with n being small compared to N. Given a matrix Vn ∈ CN×n with orthonormal columns, the eigenvalues of the projected counterpart V ∗nAVn ∈ Cn×n are called Ritz values of order n. These Ritz values are often good
Sharpness in rates of convergence for CG and symmetric Lanczos methods
, 2005
"... Conjugate Gradient (CG) method is often used to solve a positive definite linear system Ax = b. Existing bounds suggest that the residual of the kth approximate solution by CG goes to zero like [( κ − 1)/(√κ+ 1)]k, where κ ≡ κ(A) = ‖A‖2‖A−1‖2 is A’s spectral condition number. It is wellknown that ..."
Abstract

Cited by 7 (6 self)
 Add to MetaCart
(Show Context)
Conjugate Gradient (CG) method is often used to solve a positive definite linear system Ax = b. Existing bounds suggest that the residual of the kth approximate solution by CG goes to zero like [( κ − 1)/(√κ+ 1)]k, where κ ≡ κ(A) = ‖A‖2‖A−1‖2 is A’s spectral condition number. It is wellknown that for a given positive definite linear system, CG may converge (much) faster, known as superlinear convergence. The question is “do the existing bounds tell the correct convergence rate in general?”. An affirmative answer is given here by examples whose CG solutions have errors comparable to the error bounds for all k. A similar question for the convergence rate of Lanczos algorithm for symmetric eigenvalue problems is addressed and answered firmly, too. Conceivably examples devised here may be good testing problems for linear system and eigensystem solvers. 1This report is available on the web at
A note on the convergence of Ritz values for sequences of matrices
, 2000
"... While discussing the convergence of the Lanczos method, Trefethen and Bau observed a relationship with electric charge distributions, and claimed that the Lanczos iteration tends to converge to eigenvalues in regions of "too little charge" for an equilibrium distribution. Recently, Kuij ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
While discussing the convergence of the Lanczos method, Trefethen and Bau observed a relationship with electric charge distributions, and claimed that the Lanczos iteration tends to converge to eigenvalues in regions of "too little charge" for an equilibrium distribution. Recently, Kuijlaars found a theoretical justification for this phenomenon by considering the Lanczos method applied to a suitable sequence of matrices with similar spectra which may occur for instance in the discretization of PDEs while varying the parameter of discretization. The aim of the present note is to improve the result of Kuijlaars: we obtain a better rate of convergence under weaker regularity assumptions, and show that this new rate of convergence is sharp. Key words: Lanczos method, Convergence of Ritz values, Logarithmic potential theory. Subject Classifications: AMS(MOS): 65F10, 65E05, 31A99, 41A10. 1 Introduction In order to approximate eigenvalues of large real symmetric matrices A of orde...
SpectrumAdapted Tight Graph Wavelet and VertexFrequency Frames
, 2013
"... We consider the problem of designing spectral graph filters for the construction of dictionaries of atoms that can be used to efficiently represent signals residing on weighted graphs. While the filters used in previous spectral graph wavelet constructions are only adapted to the length of the spect ..."
Abstract

Cited by 7 (5 self)
 Add to MetaCart
We consider the problem of designing spectral graph filters for the construction of dictionaries of atoms that can be used to efficiently represent signals residing on weighted graphs. While the filters used in previous spectral graph wavelet constructions are only adapted to the length of the spectrum, the filters proposed in this paper are adapted to the distribution of graph Laplacian eigenvalues, and therefore lead to atoms with better discriminatory power. Our approach is to first characterize a family of systems of uniformly translated kernels in the graph spectral domain that give rise to tight frames of atoms generated via generalized translation on the graph. We then warp the uniform translates with a function that approximates the cumulative spectral density function of the graph Laplacian eigenvalues. We use this approach to construct computationally efficient, spectrumadapted, tight vertexfrequency and graph wavelet frames. We give numerous examples of the resulting spectrumadapted graph filters, and also present an illustrative example of vertexfrequency analysis using the proposed construction.