A new method for automatic indexing and retrieval is described. The approach is to take advantage of implicit higher-order structure in the association of terms with documents (“semantic structure”) in order to improve the detection of relevant documents on the basis of terms found in queries. The particular technique used is singular-value decomposition, in which a large term by document matrix is decomposed into a set of ca. 100 or-thogonal factors from which the original matrix can be approximated by linear combination. Documents are represented by ca. 100 item vectors of factor weights. Queries are represented as pseudo-document vectors formed from weighted combinations of terms, and documents with supra-threshold cosine values are re-turned. initial tests find this completely automatic method for retrieval to be promising.

How do people know as much as they do with as little information as they get? The problem takes many forms; learning vocabulary from text is an especially dramatic and convenient case for research. A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena. By inducing global knowledge indirectly from local co-occurrence data in a large body of representative text, LSA acquired knowledge about the full vocabulary of English at a comparable rate to schoolchildren. LSA uses no prior linguistic or perceptual similarity knowledge; it is based solely on a general mathematical learning method that achieves powerful inductive effects by extracting the right number of dimensions (e.g., 300) to represent objects and contexts. Relations to other theories, phenomena, and problems are sketched. Prologue "How much do we know at any time? Much more, or so I believe, than we know we know!" —Agatha Christie, The Moving Finger A typical American seventh grader knows the meaning of

SVDPACKC comprises four numerical (iterative) methods for computing the singular value decomposition (SVD) of large sparse matrices using ANSI C. This software package implements Lanczos and subspace iteration-based methods for determining several of the largest singular triplets (singular values and corresponding left- and right-singular vectors) for large sparse matrices. The package has been ported to a variety of machines ranging from supercomputers to workstations: CRAY Y-MP, IBM RS/6000-550, DEC 5000100, HP 9000-750, SPARCstation 2, and Macintosh II/fx. This document (i) explains each algorithm in some detail, (ii) explains the input parameters for each program, (iii) explains how to compile/execute each program, and (iv) illustrates the performance of each method when we compute lower rank approximations to sparse term-document matrices from information retrieval applications. A user-friendly software interface to the package for UNIX-based systems and the Macintosh II/fx is als...

Low rank approximation of large and/or sparse matrices is important in many applications. We show that good low rank matrix approximations can be directly obtained from the Lanczos bidiagonalization process without computing singular value decomposition. We also demonstrate that a so-called one-sided reorthogonalization process can be used to maintain adequate level of orthogonality among the Lanczos vectors and produce accurate low rank approximations. This technique reduces the computational cost of the Lanczos bidiagonalization process. We illustrate the efficiency and applicability of our algorithm using numerical examples from several applications areas.

Abstract. We discuss a new method for the iterative computation of a portion of the singular values and vectors of a large sparse matrix. Similar to the Jacobi–Davidson method for the eigenvalue problem, we compute in each step a correction by (approximately) solving a correction equation. We give a few variants of this Jacobi–Davidson SVD (JDSVD) method with their theoretical properties. It is shown that the JDSVD can be seen as an accelerated (inexact) Newton scheme. We experimentally compare the method with some other iterative SVD methods. Key words. Jacobi–Davidson, singular value decomposition (SVD), singular values, singular vectors, norm, augmented matrix, correction equation, (inexact) accelerated Newton, improving singular values AMS subject classifications. 65F15 (65F35) PII. S1064827500372973

. Iterative methods based on Lanczos bidiagonalization with full reorthogonalization (LBDR) are considered for solving large scale discrete ill-posed linear least squares problems of the form min x kAx \Gamma bk 2 . Methods for regularization in the Krylov subspaces are discussed which use generalized cross validation (GCV) for determining the regularization parameter. These methods have the advantage that no a priori information about the noise level is required. To improve convergence of the Lanczos process we apply a variant of the implicitly restarted Lanczos algorithm by Sorenson using zero shifts. Although this restarted method simply corresponds to using LBDR with a starting vector (AA T ) p b, it is shown that carrying out the process implicitly is essential for numerical stability. An LBDR algorithm is presented which incorporates implicit restarts to ensure that the global minimum of the CGV curve corresponds to a minimum on the curve for the truncated SVD solution. Nume...

. In this paper, we present four numerical methods for computing the singular value decomposition (SVD) of large sparse matrices on a multiprocessor architecture. We particularly emphasize Lanczos and subspace iteration-based methods for determining several of the largest singular triplets (singular values and corresponding left- and right-singular vectors) for sparse matrices arising from two practical applications: information retrieval and seismic reflection tomography. The target architectures for our implementations of such methods are the Cray-2S/4-128 and Alliant FX/80. The sparse SVD problem is well motivated by recent information-retrieval techniques in which dominant singular values and their corresponding singular vectors of large sparse term-document matrices are desired, and by nonlinear inverse problems from seismic tomography applications in which approximate pseudo-inverses of large sparse Jacobian matrices are needed. It is hoped that this research will advance the dev...

this memory is statically allocated, whereas on the Alliant FX/80 it is dynamically allocated as needed. On the Cray-2S/4128, the vector z would be both retrieved from and written to core memory. However, on the Alliant FX/80, z may be fetched and held in the 512 kilobyte cache. Since memory accesses from the cache (fast local memory) can almost twice as fast as those from the larger globally-shared memory, we achieve an overall higher computational rate for multiplication by A

. A unifying framework for methods employed in the approximation of the resolvent norm of nonnormal matrices is presented. This formulation uses specific transfer functions, and it provides new information about the approximation properties of these methods and their application in computing the pseudospectrum of matrices. Key words. Resolvent norm, transfer function, Arnoldi iteration, pseudospectrum. AMS subject classification. 65F15. 1. Introduction. We now know that the analysis of matrix-dependent algorithms is considerably more complicated when nonnormal matrices are involved; see for example [5]. In particular, several studies indicate that the eigenvalues of the matrix in question often provide insufficient or even misleading information [20]. This has been the motivation behind recent research on more reliable indicators as well as on methods for their practical computation. Several studies concur that a better accordance between theory and practice can be achieved by using ...

this paper reflect the use of 2-cyclic iteration matrices as defined in Equation (2). The three main steps that constitute the CSI-MSVD algorithm are: 1. calculation of the CSI-iterate using Equations (26) and (27), 2. calculation of the new moments for the current iterate, and 3. updating the bidiagonal matrix and approximating the eigenvalues of the two-cyclic iteration matrix through the QR-iteration. Figure 2 shows the dependencies involved in the steps of the above outlined procedure. The pipelined nature of the computation indicates that Steps 1, 2, and 3 described could be carried out concurrently. For example, the computation of the anti-diagonal elements OE 13 ; OE 22 ; OE 15 ; OE 24 ; OE 33 (shown in the box labeled PHI in Figure 2) could be overlapped with the computation of the iterates ¸