M. Arioli, I.S. Duff and P.P.M. de Rijk, On the Augmented System Approach to Sparse Least-Squares Problems, Numer. Math., 55(1989) 667--684.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....A T 0 x 0 (2.31 0 1 O 1 ( where = 1111= 1 The parameter s usually unknown before the computation. ff s not small enough, the result may not be acceptable. On the other hand, if is too small, the amount of fill in will be too large to be accepted. Experimental results are presented in [1]. which is the scaled form. This method was proposed by Barrels et al. and later considered for the sparse case by Hachtel. BjiSrck used it in a study of iterative refinement for least square solutions [7] Numerical experiments on this method and the comparison with other methods have been ....

....which is the scaled form. This method was proposed by Barrels et al. and later considered for the sparse case by Hachtel. BjiSrck used it in a study of iterative refinement for least square solutions [7] Numerical experiments on this method and the comparison with other methods have been done [1] [20] If the pivots are chosen from the diagonal when Cholesky factorization is applied to the augmented system, then after m steps, the reduced system is exactly the normal equations. Expressing the least squares problems in augmented system form with scaling allows more flexibility in pivoting ....

M. Arioli, I. Duff, and P. P.M. De Rijk. On the augmented system approach to sparse least-squares problems. Numerische Mathematik, 55:667 684, 1989.

....backward error is that it is insensitive to the scaling of the system: if Ax = b is scaled to (S 1 AS 2 ) S Gamma1 2 x) S 1 b, where S 1 and S 2 are diagonal, and y is scaled to S Gamma1 2 y, then remains unchanged. Recent work that makes use of componentwise backward error includes [1, 2, 13, 15, 16]. There are situations where even the componentwise backward error is not entirely appropriate, because it does not respect any structure (other than sparsity) in A or b. For example, if A is a Toeplitz matrix and j(y) and (y) are small, it does not necessarily follow that y solves a nearby ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk, On the augmented system approach to sparse least-squares problems, Numer. Math., 55 (1989), pp. 667--684.

....and Sun observe [71, p. 159] b r = b Gamma Ay is computable and jsj jbrj ffl(f Ejyj) and using this bound in (4.6) makes only a second order change. A componentwise bound of the form (4. 6) was first derived by Bjorck in 1988 and variations of it have been given by Arioli, Duff and de Rijk [3], Bjorck [11] and Higham [42] 12 NICHOLAS J. HIGHAM Apart from its increased sharpness over (4.1) the bound (4.6) has better scaling properties. It is not invariant under row or column scalings, but it is less sensitive to these scalings than (4.1) In [42] we examined the famous Longley test ....

....: Thus, as would be expected from known backward error analysis, b xQR is a backward stable solution but b xNE is not. In this example it makes little difference whether or not we perturb b. Componentwise backward error for the LS problem has been investigated by Arioli, Duff and de Rijk [3], Bjorck [11] and Higham [42] The simplest approach is to apply the componentwise backward error E;f (y) of (2.6) to the augmented system (4.2) setting E = 0 EA E T A 0 so as not to perturb the diagonal blocks I and 0 of the augmented system coefficient matrix. However, this approach ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk, On the augmented system approach to sparse least-squares problems, Numer. Math. 55 (1989), 667--684.

....and requires less storage than that of [Kiw89] unless mL p 2n) In fact no algorithm seems to be competitive with our method in terms of workspace and work per iteration, especially in large scale applications. Alternative implementations of our algorithm might use the techniques of [ADdR89, Bjo87, Bjo88, Bjo90, BjP92, GGM 84, Hig91, Pan90] These papers also discuss stability questions relevant to our approach. In general, the accuracy of matrix decompositions may be increased by using (partial) orthogonal factors and or iterative refinement (related to reorthogonalization) ....

Arioli, M., Duff I.S., de Rijk, P.P.M. (1989): On the augmented system approach to sparse least-squares problems. Numer. Math. 55, 667--684

.... obtain a good approximation q of q that solves (35) In our implementation we set D = dI k ; where d = 100 Theta max j=1;2; n f Theta j g; 41) and solve (37) for p using the normal equations system (39) and compute q from (38) Any IPM code that employs the augmented system solver [2] to compute Newton direction would of course solve (36) directly. 5 Numerical results The warm start approach proposed in this paper has been tested in the context of the primaldual cutting plane method of Gondzio and Sarkissian [12] The reader interested in more detail in this particular ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk. "On the Augmented System Approach to Sparse Least-- Squares Problems", Numerische Mathematik 55 (1989) 667--684.

....sequence of linear systems by factoring H Gamma I is used to determine few eigenvectors of Hermitian matrix H. In optimization the so called augmented systems (or the Karush Kuhn Tucker 3 systems) of the form A B B 0 are used in several cases: in unconstrained least squares problems [2] where the augmented system approach has better numerical properties than the normal equation approach, in constrained least squares problems [14] and in general quadratic programming [17, 33] The last application naturally extends to the minimization of general function with linear constraints, ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk, On the augmented system approach to sparse least--squares problems, Numer. Math., 55:667--684, (1989).

....which is the scaled form. This method was proposed by Bartels et al. and later considered for the sparse case by Hachtel. Bjorck used it in a study of iterative refinement for least square solutions [7] Numerical experiments on this method and the comparison with other methods have been done [1] [20] If the pivots are chosen from the diagonal when Cholesky factorization is applied to the augmented system, then after m steps, the reduced system is exactly the normal equations. Expressing the least squares problems in augmented system form with scaling allows more flexibility in pivoting ....

....n where ff = oe nkrk2 kxk2 ) 1=2 . The parameter ff is usually unknown before the computation. If ff is not small enough, the result may not be acceptable. On the other hand, if ff is too small, the amount of fill in will be too large to be accepted. Experimental results are presented in [1]. 2.5 Iterative methods As in solving large sparse linear systems, an iterative method is a very attractive alternative to direct methods for solving large sparse linear least squares problems. There are basically two classes of methods. The first class contains those iterative methods for ....

M. Arioli, I. Duff, and P. P. M. De Rijk. On the augmented system approach to sparse least-squares problems. Numerische Mathematik, 55:667--684, 1989.

....ff. A simple analysis shows that if ff is chosen equal to the smallest singular value of A, the condition number of the augmented system is approximately equal to 1.6 times the condition number of A, and is about the same as the condition number of A when A is ill conditioned. Arioli et al. [2] demonstrated that conditioning can be greatly improved by a scaling similar to that of Bjorck; they suggested an automatic technique for selecting the best scaling factor ff. Unfortunately, both Bjorck and Arioli et al. use a condition number estimator to calculate ff, which would make a tensor ....

....pseudo random numbers. The a ijk are uniformly distributed random integers in [0,3] and with likelihood p (p = min(100 Gamma[ 200 n ] 90) these values are randomly reset to zero. The parameters c ik and e i are in [ 100,100] and [ 10,10] respectively, the initial vector x 0 has elements in [1,2], and the index l = 8 is chosen. The solution of this type of problem is not known a priori, and indeed different local solutions may exist. The size of the residuals at the solution is typically large and is determined mainly by the bounds [ 10,10] on e i and can be changed by varying these ....

M. Arioli, I. S. Duff, and P. P. M. Rijk. On the augmented system approach to sparse least-squares problems. Numer. Math., 55:667--684, 1989.

.... invertible preconditioner for an iterative method, as e.g. conjugate gradients algorithm (implemented successfully for network problems [65, 62] direct methods [18] that compute sparse symmetric factorization (Cholesky decomposition of the positive definite system AD 2 A T or Bunch Parlett [13, 7] decomposition of the indefinite augmented system 2 6 4 D Gamma2 A T A 0 3 7 5) are the methods of choice. Computing projections onto affine spaces seems crucial for the efficiency of any interior point algorithm. We shall thus discuss it in detail in Section 3 that addresses also other ....

....T ) Gamma1 ; r = c Gamma X Gamma1 ( e Gamma XZe) S Gamma1 ( e Gamma STe) Gamma S Gamma1 T u ; 10) h = b : The matrix in the augmented system (9) is sparse, symmetric but indefinite. This system of linear equations can be solved directly by the Bunch Parlett factorization [13, 7] or, after multiplying the first equation by AD 2 and substituting from the second equation,the system can be reduced to sparse, symmetric and positive definite normal equations system (AD 2 A T ) Deltay = AD 2 r h: 11) Advantages of both approaches will be discussed in the next ....

[Article contains additional citation context not shown here]

Arioli, M., Duff, I.S. and de Rijk, P.P.M. (1989). On the Augmented System Approach to Sparse Least--Squares Problems, Numerische Mathematik 55, 667--684.

....IPM codes use a direct approach [19] to solve the Newton equation system. The alternative, iterative methods has not been used as much due to difficulties in choosing a preconditioner. There are two competitive direct approaches for solving the Newton equations: the augmented system approach [6, 7] and the normal equations approach. The former requires factorization of a symmetric indefinite matrix, the latter works with a smaller positive definite matrix. In Section 4, we discuss both these approaches in detail, analyse their advantages and point out some difficulties that arise in their ....

....This motivated several researchers to pay special attention to the augmented system form of the Newton equations which allows more freedom in the pivot choice. 4. 2 The augmented system approach The augmented system approach is an old and well understood technique to solve a least squares problem [6, 7, 11, 19]. It consists in the application of the Bunch Parlett [13] factorization to the symmetric indefinite matrix GammaD Gamma2 A T A 0 # = LL T ; 31) where is an indefinite block diagonal matrix with 1 Theta 1 and 2 Theta 2 blocks. In contrast to the normal equations approach in which ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk. On the augmented system approach to sparse least-squares problems. Numer. Math., 55:667--684, 1989.

....lead to a sparser (and cheaper) factorization. Unfortunately, it is not possible, in general, to separate the reordering for sparsity from the numerical factorization of H since this matrix is not positive definite. Following the classical approach, one should use the Bunch Parlett factorization [7, 6] that chooses between 1 Theta 1 and (possibly indefinite) 2 Theta 2 pivots. The Bunch Parlett factorization [10] like any other factorization that combines dynamic reordering with the numerical operations, is usually more expensive than its counterpart in which these operations are separated. ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk, On the augmented system approach to sparse least-squares problems, Numerische Mathematik, 55 (1989), pp. 667--684.

.... the details of how the products Bx and B x are computed, making it of general applicability (to mimic the LINPACK estimator, take B = A and use an LU factorization to compute the products with B) The key to the widespread use of Hager s method was the observation by Arioli, Demmel and Duff [3] that the method can be used to estimate k jA jd k1 for any given nonnegative vector d, which could, for example, be a matrix vector product. Writing D = diag(d) and e = 1; 1; 1] we have jd k1 = k jA jDe k1 = k jA Dje k1 = k jA Dj k1 = kA Dk1 : 2.2) Since kBk1 = ....

M. Arioli, I. S. Duff, and P. P. M. de Rijk. On the augmented system approach to sparse least-squares problems. Numer. Math., 55:667--684, 1989.

....section. For the solution of least squares problems, the matrix H is diagonal and the vector c on the right hand side of (5.1) is zero. The beauty here is that, by suitable choice of 2 Theta 2 pivots, small entries on the diagonal of the (1,1) block in (5.1) do not cause stability problems. Arioli, Duff and de Rijk (1989) have used the analysis of Arioli et al. 1989) and have conducted several experiments to support the viability of using 2 Theta 2 pivots. Bjorck (1992) provides a detailed error analysis for this approach. Further computational experience in using a scaling on the (1,1) block is presented in ....

....the matrix H is diagonal and the vector c on the right hand side of (5.1) is zero. The beauty here is that, by suitable choice of 2 Theta 2 pivots, small entries on the diagonal of the (1,1) block in (5.1) do not cause stability problems. Arioli, Duff and de Rijk (1989) have used the analysis of Arioli et al. 1989) and have conducted several experiments to support the viability of using 2 Theta 2 pivots. Bjorck (1992) provides a detailed error analysis for this approach. Further computational experience in using a scaling on the (1,1) block is presented in Duff (1994) At first glance, QR methods do not ....

Arioli, M., Duff, I. S. and de Rijk, P. P. M. (1989), `On the augmented systems approach to sparse least-squares problems', Numer. Math. 55, 667--684.

No context found.

M. Arioli, I.S. Duff and P.P.M. de Rijk, On the Augmented System Approach to Sparse Least-Squares Problems, Numer. Math., 55(1989) 667--684.

No context found.

Arioli, M., Duff, I. S., and Rijk, P. P. M. On the augmented system approach to sparse least-squares problem. Numer. Math., 55:667--684, 1989.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC