Magnus R. Hestenes. Inversion of Matrices by Biorthogonalization and Related Results. Journal of the Society for Industrial and Applied Mathematics, 6(1):51-90, March 1958.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....to apply Jacobi s method implicitly, at each stage postmultiplying A by the Jacobi transformation defined by Jacobi s method for the symmetric eigenproblem applied to A A. This idea is encapsulated in the one sided Jacobi algorithm (from the right) which was first proposed by Hestenes [30]. Algorithm 5.1 (one sided Jacobi) This algorithm computes the SVD of , m n, by the one sided Jacobi algorithm. done rot = true; V = I while done rot = true done rot = false for j = 1: n Gamma 1 for k = j 1: n h jj = A( j) A( j) h kj = A( j) h kk = A( k) if jh ....

Magnus R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. Soc. Indust. Appl. Math., 6(1):51--90, 1958.

....of convergence for the parallel ordering was given by Park and Luk [60] Theoretically, the SVD of a matrix M may be computed from the eigenvalue decomposition of M T M . However, the numerical difficulties associated with the computation of M T M makes the approach impractical. Hestenes [43] suggested a one sided approach, which applies the Jacobi method implicitly, to overcome the difficulty in the formation of M T M . A discussion on the relative difficulties and accuracies of the different approaches to computing the SVD, and the justification for a systolic array can be found ....

....and requires O(m nS) time, where S denotes the number of sweeps. The array is extended to a rectangular configuration with 1 2 mn O(m) processors for the computation of singular vectors. The systolic array of Brent Luk [5] uses the one sided orthogonalization method due to Hestenes [43] on a linearly connected mesh of O(n) processors and requires O(mn log n) steps to compute a singular value. Atwo dimensional array, with O(mn) processors and a non planar interconnection structure, that requires O(n log m) time was also proposed. The method is quadratically convergent and the ....

[Article contains additional citation context not shown here]

M. R. Hestenes. Inversion of Matrices by Biorthogonalization and Related Results. J. Soc. Indust. Appl. Math, 6:51--90, 1958.

....by Brent and Luk [8] which requires O(mn log n) time and O(n) processors. Theoretically, the SVD of a matrix A may be computed from the eigenvalue decomposition of A H A.However, the numerical difficulties associated with the computation of A H A makes the approach impractical. Hestenes [61] suggested a one sided approach, which applies the Jacobi method implicitly,toovercome the difficulty in the formation of A H A. A discussion on the relative difficulties and accuracies of the different approaches to computing the SVD, and the justification for a systolic array can be found in ....

....and requires O(m nS) time, where S denotes the number of sweeps. The array is extended to a rectangular configuration with 1 2 mn O(m) processors for the computation of singular vectors. The systolic array of Brent Luk [8] uses the one sided orthogonalization method due to Hestenes [61] on a linearly connected mesh of O(n) processors and requires O(mn log n) steps to compute a singular value. A two dimensional array, with O(mn) processors and a non planar interconnection structure, that requires O(n log m) time was also proposed. The method is quadratically convergent and ....

[Article contains additional citation context not shown here]

M. R. Hestenes. Inversion of Matrices by Biorthogonalization and Related Results. J. Soc. Indust. Appl. Math, 6:51--90, 1958.

..... 3. Singular Value Decomposition of Sparse Matrices. Before presenting methods for computing the sparse singular value decomposition, we note that classical methods for determining the SVD of dense matrices: the Golub Kahan Reinsch method ( 19] 21] and Jacobi like SVD methods ( 5] 6] and [25]) are not optimal for large sparse matrices. Since these methods apply orthogonal transformations (Householder or Givens) directly to the sparse matrix A, they incur excessive fill in and thereby require tremendous amounts of memory. Moreover, it would be necessary to provide enough computer ....

M. R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. Soc. Indust. Appl. Math., 6:51--90, 1958.

....complex operation in the subspacebased method is the SVD of Y and hence it also gives rise to the dominant term in the expression for error due to fixed point implementation of the method. We will assume that the Frobenius norm of the matrix Y is bounded by unity. Using Hestenes method for SVD [3, 8], we get : Y = XV 0 , where X = U Sigma, and Y = U SigmaV 0 is the SVD of Y. The unitary matrices, U and V , contain the left and right singular vectors and Sigma is the diagonal matrix of singular values. For a fixed point implementation, instead of X and V , we get X and V , where ....

M. R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. SIAM, 6:51--90, 1958.

....so that computation time can be reduced by distributing the work among a number of processing units. This paper proposes a parallel scheme for SVD updating that can be implemented on a fixed size array of off the shelf processors. 2 Algorithm description Our algorithm is based on Hestenes method [3] for computing the SVD. In the Hestenes method, starting with any given matrix A, an orthogonal matrix V is built such that AV has orthogonal columns. Thus, AV = US, where U has orthonormal columns and S is non negative and diagonal. The SVD of A is A = USV 0 . Let us denote US by the single ....

....following equations : Yn = fiY n Gamma1 y 0 n : 8) Using equation 7 with 8, Yn = fiWn Gamma1 y 0 n Vn Gamma1 V 0 n Gamma1 : 9) Let us denote the first matrix on the right hand side of equation 9 by Xn . The columns Xn is orthogonalized using r sweeps of Hestenes method [3]. So we get : Yn = Wn V 0 n V 0 n Gamma1 = WnV 0 n ; 10) 1 where Vn = Vn Gamma1 Vn . Simulations of application of this method to the frequency estimation problem (refer section 4) show that a good enough approximation is obtained after just one sweep of Hestenes method, for ....

[Article contains additional citation context not shown here]

M. R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. SIAM, 6:51--90, 1958.

....they observe that there are orderings for which no proof of convergence exists. As an example, the Brent Luk ordering [16] for even values of n is cited. The same trends are noticeable in computing the SVD, CSD, GSVD, QRD, and Schur decompositions. Here, algorithms first suggested by Hestenes in [78] and Kogbetliantz in [89] were developed and adapted for the new parallel machines. Again, the main motivation for selection (as in the case of Jacobi) was the potential speedup available due to the high degree of parallelism of these algorithms [39, 101, 134] 1 In summary, the cyclic Jacobi ....

Hestenes M.R., Inversion of matrices by biorthogonalization and related results, J. SIAM , Vol 6, 1958.

....and two sided. The two sided Jacobi algorithms are computationally more expensive than the one sided algorithms, and not so suitable for vector pipeline computing. Thus, to achieve efficient parallel SVD computation the best approach may be to adopt the Hestenes one sided transformation method [13] as advocated in [4, 5] The Hestenes method generates an orthogonal matrix V such that AV = H; where the columns of H are orthogonal. The nonzero columns H of H are then normalised so that H = U r Sigma r with U T r U r = I r , Sigma r = diag(oe 1 ; Delta Delta Delta ; oe r ) and r ....

....It can be proved that the value of b (k) ii is increased and the value of b (k) jj is decreased after a plane rotation operation if b (k) ii b (k) jj . Otherwise, b (k) ii is decreased and b (k) jj is increased. Rotation Algorithm 2 The second algorithm, introduced by Hestenes [13], is the same as the Algorithm 1 except that the columns a (k) i and a (k) j are to be swapped if ka (k) i k ka (k) j k for i j before the orthogonalization of the two columns. Therefore, we always have b (k 1) ii b (k 1) jj . When the cyclic ordering by rows is applied, the ....

M. R. Hestenes, "Inversion of matrices by biorthogonalization and related results", J. Soc. Indust. Appl. Math., 6, 1958, pp. 51-90.

....with diagonal elements, Phi oe 2 1 ; oe 2 2 ; Delta Delta Delta ; oe 2 N Psi . Moreover, U = Q J(i; j; S(i; j; OE) and V = Q J(i; j; The fact that the OSVD of A (almost) corresponds to an eigenvalue decomposition of A T A (or AA T ) has been utilized by Hestenes [16] to compute the OSVD of a rectangular matrix by the Jacobi algorithm, without forming the matrix product. P. M. Rands Jensen 4 Rotation Schemes for Jacobi like Algorithms 8 For each rotation pair, i; j) a i and a j are orthogonalized. Forming [a i a j ] T [a i a j ] and applying Jacobi one ....

M. R. Hestenes. Inversion of Matrices by Biorthogonalization and Related Results. Journal, SIAM, 6, 1959. P. M. Rands Jensen References 16

.... matrix, and Sigma is an n Theta n non negative diagonal matrix, say Sigma = diag(oe 1 ; Delta Delta Delta ; oe n ) There are various ways to compute the SVD [6] To achieve efficient parallel SVD computation the best approach may be to adopt the Hestenes one sided transformation method [7] as advocated in [2] The Hestenes method generates an orthogonal matrix V such that AV = H; where the columns of H are orthogonal. The nonzero columns H of H are then normalised so that H = U r Sigma r with U T r U r = I r , Sigma r = diag(oe 1 ; Delta Delta Delta ; oe r ) and r n ....

M. R. Hestenes, "Inversion of matrices by biorthogonalization and related results", J. Soc. Indust. Appl. Math., 6, 1958, pp. 51-90.

....control of redundant manipulators. While the arguably best general algorithm for calculating the SVD is the Golub Reinsch algorithm [14] for this application one would like to use an algorithm that is more amenable to parallelization [21] Thus the algorithm used here is based on Givens rotations [16], 25] 29] which are orthogonal transformations of the form Q = 2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 4 1 Delta Delta . Delta Delta 1 Delta Delta Delta Delta Delta cos(OE) Delta Delta Delta Gamma sin(OE) Delta Delta Delta Delta 1 Delta Delta . Delta Delta 1 Delta ....

M. R. Hestenes, "Inversion of matrices by biorthogonalization and related results," Journal of the Society for Industrial and Applied Mathematics, vol. 6, no. 1, pp. 51--90, 1958.

.... matrix, and Sigma is an n Theta n nonnegative diagonal matrix, say Sigma = diag(oe 1 ; Delta Delta Delta ; oe n ) There are various ways to compute the SVD [2] To achieve efficient parallel SVD computation the best approach may be to adopt the Hestenes one sided transformation method [3] as advocated in [1] The Hestenes method generates an orthogonal matrix V such that AV = H; where the columns of H are orthogonal. The nonzero columns H of H are then normalised so that H = U r Sigma r with U T r U r = I r , Sigma r = diag(oe 1 ; Delta Delta Delta ; oe r ) and r n ....

M. R. Hestenes, "Inversion of matrices by biorthogonalization and related results", J. Soc. Indust. Appl. Math., 6, 1958, pp. 51--90.

....To avoid this complexity increase, one sided or implicit Jacobi algorithms have been proposed. We discuss the implementation of Veseli c and Hari [23] One sided Jacobi algorithms have their origin in the algorithm of Hestenes for the computation of the singular value decomposition of a matrix [15]. Let us assume, that A = A T is positive definite. This is theoretically no restriction as any symmetric matrix can be made positive definite by an appropriate spectral shift. This shift may however worsen the relative accuracy of the computed results [6] When A is positive definite, its ....

M. Hestenes. Inversion of matrices by biorthogonalization and related results. J. Soc. Indust. Appl. Math., 6:51--90, 1958.

....and two sided. The two sided Jacobi algorithms are computationally more expensive than the one sided algorithms, and not so suitable for vector pipeline computing. Thus, to achieve efficient parallel SVD computation the best approach may be to adopt the Hestenes one sided transformation method [13] as advocated in [4, 5] The Hestenes method generates an orthogonal matrix V such that AV = H; where the columns of H are orthogonal. The nonzero columns H of H are then normalised so that H = U r Sigma r To appear in J. Parallel and Distributed Computing. Received Oct 5, 1994; revised ....

.... b (k) jj . Otherwise, b (k 1) ii b (k) ii and b (k 1) jj b (k) jj . Thus the larger column norm is always further increased and the smaller one further decreased when orthogonalising two columns using this algorithm. Rotation Algorithm 2 The second algorithm, introduced by Hestenes [13], is the same as the Algorithm 1 except that the columns a (k) i and a (k) j are to be swapped if ka (k) i k ka (k) j k for i j before the orthogonalisation of the two columns. From (11) and (12) we always have b (k 1) ii b (k 1) jj . When the cyclic ordering is applied, ....

M. R. Hestenes, "Inversion of matrices by biorthogonalization and related results", J. Soc. Indust. Appl. Math., 6, 1958, pp. 51-90.

.... for computing the SVD can be grouped in at least 3 groups [19] ffl Serial QR based which are sequential in nature [20,12] ffl Jacobi like which all are extensions of the serial Jacobi algorithm for computing the eigenvalues of square and symmetric matrices and which are inherent parallel [21,22,12]. ffl Other methods such as power methods [23] Due to the inherent parallel nature of Jacobi like algorithms most interest has been paid to these in association with real time DSP applications; this is also done here. Referring to (2.7) 2.9) it is seen that we only need the right singular ....

....done here. Referring to (2.7) 2.9) it is seen that we only need the right singular vectors to compute the DW TLS solution. However, for selecting the parsimonious order we also need the product of the left singular vectors and the singular values. This makes a column oriented Hestenes algorithm [22] the most appropriate choice; cf. 19] for a description of various Jacobi like algorithms. In Hestenes algorithms all column pairs, i; j) of A are mutually orthogonalised by iteratively applying planar rotations, J T (i; j; as A (k) A (k Gamma1) J(i; j; 3.2) c fl P. M. Rands ....

M. R. Hestenes, "Inversion of Matrices by Biorthogonalization and Related Results," Journal, SIAM, vol. 6, 1959.

....D = 0 B B B p 1 0 Delta Delta Delta 0 0 p 2 Delta Delta Delta 0 . 0 0 Delta Delta Delta pn 1 C C C A : This algorithm is essentially due to L. Fox, see Ch. 6 of [25] Closely related methods have also been considered by Hestenes and Stiefel [35, pp. 426 427] [34] and by Stewart [52] A more general treatment is given in the recent paper [14] Geometrically, the procedure can be regarded as a generalized Gram Schmidt orthogonalization with oblique projections and nonstandard inner products, see [6] 14] Several observations regarding this algorithm are in ....

M. R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. Soc. Indust. Appl. Math., 6:51--90, 1958.

....m by n matrix A, where m AE n, we perform a block generalization of Householder s reduction ( 4] A = QR; 2) where Q is m by n and has orthogonal columns. For rank deficient matrices A (i.e. r n) the n by n upper triangular matrix R will be singular. Applying the one sided Jacobi method ([12], 13] 15] to the n by n upper triangular matrix R (which may be singular) from the block Householder factorization of A, we determine an n by r matrix V having orthogonal columns, as a product of plane rotations so that R V = Q = q 1 ; q 2 ; q r ) 3) and q T i q j = oe ....

M. R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. Soc. Indust. Appl. Math., 6:51--90, 1958.

....of the matrix is fulfilled. The above scheme has been implemented on many parallel computers [10, 12] Our interest was focused on an algorithm described by R.P. Brent and F.T. Luk [1] This systolic version of SVD algorithm uses a one sided orthogonalisation method developed by Hestenes 1 [8] and was originally implemented on the ILLIAC IV parallel computer. In this paper we describe several optimisations for the parallel memory hierarchy (register, cache, main memory and the external processor memory levels) of Hestenes implementation of SVD algorithm. We implemented this algorithm ....

M.R. Hestenes. Inversion of matrices by biorthogonalization and related results. J. Soc. Indust. Appl. Math., 6:51--90, 1958.

....preceded by a one sided reduction [11] followed by the QR algorithm [28, 30, 63] It is difficult to implement this Golub Kahan Reinsch algorithm efficiently on a parallel machine. It is much simpler (though perhaps less efficient) to use a one sided orthogonalization method due to Hestenes [37]. The idea is to generate an orthogonal matrix V such that AV has orthogonal columns. Normalizing the Euclidean length of each nonnull column to unity, we get AV = U Sigma (6:2) As a null column of U is always associated with a zero diagonal element of Sigma, there is no essential ....

M. R. Hestenes, "Inversion of matrices by biorthogonalization and related results ", J. SIAM 6 (1958), 51-90.

....will become of paramount importance. 3 Algorithms Before presenting algorithms for computing the sparse singular value decomposition, we note that classical methods for determining the SVD of dense matrices: the Golub Kahan Reinsch method ( 15] 18] and Jacobi like SVD methods ( 2] [20]) are not optimal for large sparse matrices. Since these methods apply orthogonal transformations (Householder or Givens) directly to the sparse matrix A, they incur excessive fill in and thereby require tremendous amounts of memory. Another drawback to these methods for computing the SVD of dense ....

Hestenes, M. R. Inversion of matrices by biorthogonalization and related results. Journal of the Society for Industrial and Applied Mathematics 6 (1958), 51--90.

No context found.

Magnus R. Hestenes. Inversion of Matrices by Biorthogonalization and Related Results. Journal of the Society for Industrial and Applied Mathematics, 6(1):51-90, March 1958.

No context found.

M. R. Hestenes, "Inversion of Matrices by Biorthogonalization and Related Results," Journal, SIAM, vol. 6, 1959.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC