J. H. Wilkinson, "Error Analysis of Direct Methods of Matrix Inversion," J. ACM, 10 (1961), pp. 281--330.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... If nonexact arithmetic is used, as occurs in practice, all of these methods suffer from numerical instability, due to their use of the Cayley Hamilton theorem and to involving the evaluation of the characteristic polynomial of the input matrix, which is an unstable computation (see the analysis in [17, 18]) Two methods for parallel matrix inversion over arbitrary fields of constants are known. In [1] the authors observed that the results of [19] and [20] combined can be used to parallelize the sequential Gaussian elimination algorithm for matrix inversion. This yielded an algorithm for INVERT ....

J. H. Wilkinson, Error analysis of direct methods of matrix inversion. J. ACM 8, 281-330 (1961).

....process for solving linear systems, finding LU and similar factorizations, and computing determinants. It has found a key position in the kernel of many numerical methods (although some have questioned this [23] An excellent detailed presentation of Wilkinson s work on Gaussian elimination [27, 28], a cornerstone of modern numerical analysis, appears in Section 21 of [6] A more complete overview, with comprehensive references up to the late 1980 s, can be found in Chapter 3 of [7] 2.1 Naked Gaussian Elimination In its basic form, Gaussian elimination is a sequence of transformations to ....

....; The beauty of complete pivoting is that it both avoids the problem of zero diagonal elements and also permits bounds to be derived. For example, since the maximum element is always swapped into a kk , it is always the case that j ik j 1. It leads to the following theorem of Wilkinson [27, 28]: Theorem 1 Suppose A is an n Theta n nonsingular matrix. and t digit, base fi floating point arithmetic is used, where fi 1=n. Then the matrices U computed by Gaussian elimination with pivoting in floating point with unit roundoff u satisfy U = A E; where E is a matrix of roundoff ....

J.H. Wilkinson, "Error analysis of direct methods of matrix inversion", J. ACM 8, 281--330, 1961.

....Growth Factors, and Complete Pivoting Our original motivation behind this paper was to get more exact bounds for the error in Gaussian Elimination. The roundoff error in the floating point implementation of Gaussian Elimination has a well known backwards error bound: Theorem 5 (Wilkinson [39, 40]) Suppose A is an n Theta n nonsingular matrix. Then the matrices b L and b U computed by Gaussian Elimination with pivoting in floating point with unit roundoff u satisfy b L b U = A E; where E is a matrix of roundoff errors such that, if b a ij is the computed floating point approximation ....

.... n, and putting this together with the fact that max i a ii = max i;j ja ij j when A is symmetric positive definite, derived the bound j max Wilkinson proved that use of pivoting in Gaussian Elimination yields tighter bounds on roundoff error, notably his famous bound for complete pivoting [39]: ij j k 1=3 Delta Delta Delta k 1= k Gamma1) j 1=2 ja ij j C k (1=4) ln k Wilkinson complained that this bound appears quite loose and a tighter bound may be possible. Although it seems very difficult to improve on Wilkinson s bound, we can come up with the following ....

J.H. Wilkinson, "Error analysis of direct methods of matrix inversion", J. ACM 8, 281--330, 1961.

....the University of Manchester, Department of Mathematics, Manchester M13 9PL UK If we let L = Ln Gamma1 ; U = An Gamma1 ; we obtain the decomposition of A given by (1.1) Equation (1.7) assures that L = ij ) where j ij j 1 for all i and j and that the same holds for each L k . Wilkinson [6] showed that the stability of GEPP depends upon the growth factor max (i;j) ja ij max (i;j) ja ij j He also showed that max A2 n Thetan A = 2 and that i n = max i A = 2 The decomposition (1.1) is stable for a particular A if i A = max is a modest value. For our ....

....; 2; 1; 1; 1) For this vector kF (L k ) xk 2 =kxk 2 ae n o(2 ) 3. Wilkinson s Example Revisited and Conclusions. Let A = a ij ) 2 25 Theta25 be the matrix a ij = 0 if 1 i j 1 if i = j or i = 1 Gamma1 if i j. This is a famous example due to Wilkinson [6]. We then let the P L U factorization by GEPP be given by (1.1) and the orthgonal factorization of A be given by (1.10) ae A = 1:2370 Theta 10 However, the growth factor ae Q for the orthogonal factor Q in (1.10) is ae Q = 1:1185 Theta 10 ae A : We note that 25 6 Theta 25 ....

J. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

....sparse matrices, it is also used to reduce fill in in the factor matrices L and U, see Duff et al. 9] Mostly, partial pivoting is performed, and complete pivoting is seldom used. For large systems, however, numerical stability can only be guaranteed with complete pivoting, as shown by Wilkinson [18]. Complete pivoting requires about 50 more operations (namely comparisons) than partial pivoting. Moreover, it requires a substantial communication overhead on distributed parallel systems. It is possible, however, to combine partial and complete pivoting as follows. Partial pivoting is ....

.... with complete pivoting [1] 9] Moreover, the partial pivoting with monitoring requires only little extra overhead on a distributed memory system, as appears from experiments by Hoffmann Potma [12] The results of error analysis of Gaussian elimination, established by Wilkinson, is well known [18] [19] We briefly mention these results in a form suitable for our subsequent analysis of Gauss Huard, similar to the formulation of Golub Van Loan [8] Gaussian elimination to solve system (2.1) satisfies the following theorem. Note that we use the notations mentioned in section 1. 2.5. ....

[Article contains additional citation context not shown here]

J.H. Wilkinson, Error analysis of direct methods of matrix inversion, J. ACM 8 (1961), 281-330.

....the rth pass through the outer loop. Define the element growth factor # by # = max r max ij B (r) ij max ij B (0) ij . It is easy to show that max ij r ij # # #. Wilkinson s classical analysis of the growth factor for Gaussian elimination with complete pivoting (GCP) [13] carries over with only trivial modifications to the above algorithm. It can be shown that for a nonsingular 2n 2n skewsymmetric matrix B, # # # (2n)4 1 6 1 2 8 1 3 (2n) 1 (n 1) 3.1) This bound has the same order of magnitude as Wilkinson s classical bound on the element growth ....

....only trivial modifications to the above algorithm. It can be shown that for a nonsingular 2n 2n skewsymmetric matrix B, # # # (2n)4 1 6 1 2 8 1 3 (2n) 1 (n 1) 3. 1) This bound has the same order of magnitude as Wilkinson s classical bound on the element growth factor for GCP [13]. Mountains of numerical evidence accumulated over the years have shown that the GCP bound is quite pessimistic. For example, for matrices of dimension n = 100 and n = 200 the bound guarantees that the growth factor is no greater than 3571 and 28298, respectively. However, numerical experience ....

J. WILKINSON, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

....to show that 2 n 1 and the bound is reachable, see [5, p.49] Even though usually behaves like n or less, Foster [8] has found an example which plausibly could arise in practice and for GAUSSIAN ELIMINATION WITH ROOK PIVOTING 3 which can grow exponentially. For complete pivoting, in [21] it is shown that 2 p nn ln(n) 4 . No one has been able to nd an example where the growth factor for complete pivoting is bigger than, for example, 2n. So complete pivoting has better numerical stability than partial pivoting. The main disadvantage with complete pivoting is that it ....

J. H. Wilkinson, Error analysis of direct methods for matrix inversion, J. Soc. Indust. Appl. Math., 10 (1962), pp. 162-195.

.... x to Ax = b satisfies b A x = b b, where b A is close to A and b b is close to b. Definition 2.1 says that, for stability, the computed solution has to be the exact solution of a problem which is close to the original problem. This is the classical backward stability of Wilkinson [79, 80, 81]. We interpret close to mean close in the relative sense in some norm, i.e. k b A Gamma Ak=kAk = On ( k b b Gamma bk=kbk = On ( Note that the matrix b A is not required to be in the class A. For example, A might be the class of nonsingular Toeplitz matrices, but b A is not ....

....[34, 35, 48, 76] Stability does not imply that the computed solution x is close to the exact solution x, unless the problem is well conditioned. Provided is sufficiently small, stability implies that k x Gamma xk=kxk = On ( 2. 1) For more precise results, see Bunch [15] and Wilkinson [79]. As an example, consider the method of Gaussian elimination. Wilkinson [79] shows that k b A Gamma Ak=kAk = On (g ) where g = g(n) is the growth factor . g depends on whether partial or complete pivoting is used. In practice g is usually moderate, even for partial pivoting. However, a ....

[Article contains additional citation context not shown here]

J. H. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach. 8 (1961), 281--330.

....feasible. Let g(n; A) max i;j;k ja (k) ij j=ja (0) 11 j denote the growth associated with GE on a CP A and g(n) supf g(n; A) A 2 R n Thetan g. The problem of determining g(n) for various values of n is called the growth problem. The determination of g(n) remains a mystery. Wilkinson in [8] proved that g(n) n 2 3 1=2 : n 1=n Gamma1 ] 1=2 = f(n) In Table 1 there are values of f(n) for representative values of n. Department of Mathematics, National Technical University of Athens, Zografou 15773, Athens, Greece. y Department of Mathematics, University of Athens, ....

J. H. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), 281-330.

....of this kind, causing instability for all but small dimensions. In the 1950s, Wilkinson developed a beautiful theory based on backward error analysis that, while it explained a great deal about Gaussian elimination, confirmed that for certain matrices, exponential instability does indeed occur [17]. He showed that amplification of rounding errors by factors on the order of kL Gamma1 k may take place, and that for certain matrices, kL Gamma1 k is of order 2 n . Thus for certain matrices, rounding errors are amplified by O(2 n ) causing a catastrophic loss of n bits of precision. ....

J.H. Wilkinson, Error Analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach. 8 (1961), 281-330.

....end of the rth pass through the outer loop. Define the element growth factor fl by fl = max r max ij j B (r) ij j max ij j B (0) ij j : It is easy to show that max ij j r ij j p fl. Wilkinson s classical analysis of the growth factor for Gaussian elimination with complete pivoting (GCP) [13] carries over with only trivial modifications to Algorithm 1. It can be shown that for a nonsingular 2n Theta 2n skew symmetric matrix B, fl q (2n)4 1 6 1=2 8 1=3 Delta Delta Delta (2n) 1= n Gamma1) 3.1) This bound has the same order of magnitude as Wilkinson s classical bound on ....

....to Algorithm 1. It can be shown that for a nonsingular 2n Theta 2n skew symmetric matrix B, fl q (2n)4 1 6 1=2 8 1=3 Delta Delta Delta (2n) 1= n Gamma1) 3. 1) This bound has the same order of magnitude as Wilkinson s classical bound on the element growth factor for GCP [13]. Mountains of numerical evidence accumulated over the years have shown that the GCP bound is quite pessimistic. For example, for matrices of dimension n = 100 and n = 200 the bound guarantees that the growth factor is no greater than 3571 and 28298, respectively. However, numerical experience ....

J. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

....residual. Since computing the actual residual is an expensive and useless (as far as advancing the solution goes) operation, it is one that we would like to avoid. In this section we shall give bounds on the size of the error being propagated. The analysis of this section follows the analysis in [52]. Define the spectral norm of A as k A k= q ae(AA T ) Define the Frobenius norm of A as k A k F = q P i P j a 2 ij . Recall that the iteration proceeds by first solving Mz k 1 = r k and then computing r k 1 = Nz k 1 . Let e s k 1 be the error in the solve step which we define as e ....

....E k 1 = r k 1 Gamma Tr k Using our definitions from above we have E k 1 = Nz k 1 e m k 1 Gamma Tr k = NM Gamma1 e s k 1 NM Gamma1 r k e m k 1 Gamma Tr k = Te s k 1 e m k 1 : 3.28) Thus our task remains to bound e s k 1 and e m k 1 . Our analysis is derived from [52]. Define jAj as jAj = ja ij j. We begin with the solve step. Consider the solution of the equation Mz = r where M is lower triangular. The solution proceeds at each step by computing the value of z i using z 1 ; z 2 ; z n . Therefore z i = f l r i Gamma m i;1 z 1 Gamma m i;2 z 2 ....

[Article contains additional citation context not shown here]

J. H. Wilkinson. Error analysis of direct methods of matrix inversion. JACM, 8:281--330, 1961.

....the importance of a systematic analysis in numerical linear algebra that included the effects of rounding errors as well as a problem s sensitivity to variations in the data. The difficulty with Gaussian Elimination was that occasionally an ill conditioned problem would be solved. Wilkinson [6] was able to express this relationship in a simple manner: Theorem 3: For A nonsingular, K(A) jk A k Delta k A Gamma1 k, and kA Gamma Ak kAk 1 K(A) then A is nonsingular and k x Gamma x k k x k K(A) 1 Gamma K(A) kA Gamma Ak kAk k A Gamma A k k A k k b ....

J.H. Wilkinson, "Error analysis of direct methods of matrix inversion," J. Assoc. Comput. Mach., vol. 8, pp. 281--330, 1961.

....to J = 1, and a poor quality to J signi cantly larger than 1. De nition 1.3 A reliable algorithm is said to be optimal when J 1. In the Numerical Software literature, the property of an algorithm that we have just de ned as optimal reliability is referred to as backward stability since [22]. It entails computability in nite precision ( 5] When the data are known with an accuracy of the order of (i.e. k Ak kAk for instance) then the backward error should be compared to ; this is important when is signi cantly larger than machine precision. Such a situation is frequent ....

J. H. Wilkinson. Error analysis of direct methods of matrix inversion. J. Assoc. Comput. Mach., 8:281-330, July 1961.

....es v (k 1) S S (k) u (k) and we want to relate the perturbation S (k) with the inner backward errors. We will denote by u (k) i the approximate solution of the i th local system in (4) at step k of the outer process. Taking the viewpoint of the backward error analysis [14], we consider that u (k) i is the exact solution of a perturbed local system (A ii A (k) ii ) u (k) i = A i u (k) From now on, the notation k : k stands for any subordinate norm. The smallest quantity 4 achievable for A (k) ii = kA ii k u (k) i ) is ....

J. H. Wilkinson. Error analysis of direct methods of matrix inversion. J. Assoc. Comput. Mach., 8:281-330, July 1961. 20

....solution of a linear system Ax = b using the block LU factors. In Section 3 we show that block LU factorization is stable if A is block diagonally dominant by columns; this generalizes the known results that Gaussian elimination without pivoting is stable for column diagonally dominant matrices [29] and that block LU factorization is stable for block tridiagonal matrices that are block diagonally dominant by columns [27] We also show that for a general matrix A the backward error is bounded by a product involving (A) and the growth factor ae n for Gaussian elimination without pivoting on ....

....with respect to a subordinate matrix norm in (3.1) Then A has a block LU factorization, and all the Schur complements arising in Algorithm BLU have the same kind of diagonal dominance as A. Proof. The proof is a generalization of the corresponding result for point diagonal dominance [16, p. 20] [29]. We consider the case of block diagonal dominance by columns; the proof for row wise diagonal dominance is analogous. Let A (2) U 11 U 12 0 S denote the matrix obtained from A after one step of Algorithm BLU. For 2 j m we have m X i=2 i6=j kA (2) ij k = m X i=2 i6=j kA ij ....

[Article contains additional citation context not shown here]

J. H. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

....1993. y Department of Mathematics, University of Manchester, Manchester, M13 9PL, UK. na.nhigham na net.ornl.gov 1 2 DIRECT SEARCH IN MATRIX COMPUTATIONS Our aims and techniques can be illustrated using the example of Gaussian elimination (GE) Wilkinson s classic backward error analysis [59] shows that the stability of the process for A 2 IR n Thetan is determined by the size of the growth factor ae n (A) max i;j;k ja (k) ij j max i;j ja ij j ; where the a (k) ij are the intermediate elements generated during the elimination. For a given pivoting strategy we would ....

....tol = 10 Gamma3 . 2 All numbers quoted are rounded to the number of significant figures shown. N. J. HIGHAM 3 for which ae 4 (C) 7:939. Note that this matrix is not of the form A = 2 6 4 1 1 Gamma1 1 1 Gamma1 Gamma1 1 1 Gamma1 Gamma1 Gamma1 1 3 7 5 identified by Wilkinson [59] as yielding the maximum possible growth ae n = 2 n Gamma1 for partial pivoting. The whole set of matrices A 2 IR n Thetan for which ae n (A) 2 n Gamma1 is described in [34] and C is one of these matrices, modulo the convergence tolerance. These examples, and others presented below, ....

J. H. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

....evidence in support of GE, commenting that in our practical experience on matrices of orders up to the twentieth, some of them very ill conditioned, the errors were in fact quite small . A major breakthrough in the error analysis of GE came with Wilkinson s pioneering backward error analysis [48, 49]. Wilkinson showed that with partial or complete pivoting the computed solution b x satisfies (A E)bx = b; 3.1) where kEk1 ae n p(n)ukAk1 : 3.2) Here, p is a cubic polynomial and the growth factor ae n is defined by ae n = ae n (A) max i;j;k ja (k) ij j max i;j ja ij j ; where the ....

....various distributions of random matrices the average growth factor (normalized by the standard deviation of the initial matrix elements) is close to n 2=3 for partial pivoting and n 1=2 for complete pivoting. For certain classes of matrix special bounds are known for the growth factor (see [48], 50, pp. 218 220] and [38, p. 158] ffl If A is diagonally dominant by columns (ja jj j P i6=j ja ij j for all j) then ae n (A) ae p n (A) 2 (and no row interchanges are performed with partial pivoting) ffl If A is tridiagonal then ae p n (A) 2 and if A is upper Hessenberg then ....

J.H. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

....propagate rounding errors in different ways. The performance issues are fairly well understood. The purpose of this work is to investigate the numerical stability properties of the methods, with a view to guiding the choice of inversion method in LAPACK. Existing error analysis, such as that in [18, 20] and [11] is applicable to two of the methods considered here (Method 1 and Method A) We believe our analysis for the other methods to be new. A secondary aim of this work is to use matrix inversion as a vehicle for illustrating some important principles in error analysis. Our strategy is to ....

J.H. Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comput. Mach., 8 (1961), pp. 281--330.

No context found.

J. H. Wilkinson, " Error analysis of direct methods of matrix inversion ", " J. Assoc. Comput. Mach. ", 8, pp. 281 - 330, ( 1961 ).

No context found.

J. H. Wilkinson, "Error Analysis of Direct Methods of Matrix Inversion," J. ACM, 10 (1961), pp. 281--330.

No context found.

J.H. Wilkinson, "Error analysis of direct methods of matrix inversion", J. ACM 8, 281--330, 1961.

No context found.

J.H.Wilkinson, Error analysis of direct methods of matrix inversion, J. Assoc. Comp. Machinery 8 (1961), 281--330. 9

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC