S. Hossain and T. Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33--48, 1998.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....in new variants of the graph coloring problem. The literature on matrix estimation using the relatively more recent technique AD is less extensive. Examples of graph theoretic studies in this area include the work on Jacobian matrix estimation by Coleman and Verma [5] and by Hossain and Steihaug [10]. In both the FD and AD cases, a graph theoretic approach has proved to be superior to a matrix oriented one for several reasons. The approach often provides better insight into the problem and the resulting graph problem becomes easier to analyze thereby leading to good algorithms. The first ....

....and columns of the matrix and use the partition which yields the minimum number of groups. However, such an approach may not be satisfactory for matrices with a few dense rows and columns. For such matrices considering a combined row and column partition may be worthwhile. Hossain and Steihaug [10] studied this approach and formalized the resulting matrix partition problem and its equivalent graph problem. Note that by considering the vectors d 1 , d 2 , d p as the p columns of the n p matrix D, Problem 2.1 can also be written as: given the structure of A 5 find a matrix D ....

[Article contains additional citation context not shown here]

A.K.M S. Hossain and T. Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33--48, 1998.

....for estimating these matrices via finite di#erences (FD) or automatic di#erentiation (AD) are needed. It is known that the problem of minimizing the number of function evaluations (or AD passes) required in the computation of these matrices can be formulated as variants of graph coloring problems [1, 2, 3, 9, 13]. The particular coloring problem di#ers with the optimization context: whether the Jacobian or the Hessian matrix is to be computed; whether a direct or a substitution method is employed; and whether only columns, or only rows, or both columns and rows are to be used to evaluate the matrix ....

A.K.M S. Hossain and T. Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33-- 48, 1998.

....AD passes) ## This author s research was supported by NSF grant DMS 9807172, DOE ASCI level2 subcontract B347882 from Lawrence Livermore National Lab; and by DOE SCIDAC grant DE FC02 01ER25476. required in the computation of these matrices can be formulated as variants of graph coloring problems [1 3, 8, 11]. The particular coloring problem di#ers with the optimization context: whether the Jacobian or the Hessian matrix is to be computed; whether a direct or a substitution method is employed; and whether only columns, or only rows, or both columns and rows are to be used to evaluate the matrix ....

A.K.M. S. Hossain and T. Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33--48, 1998.

....the Hessian matrix H is the main task. For large problems the computation of J or H by a straightforward application of either mode of AD can be unacceptably expensive. Recently, techniques for the e#cient determination of sparse Jacobian matrices J , via AD, have been developed [AMB 94,CV98c,SH95] e.g. the bi coloring approach of Coleman and Verma [CV98c] as discussed in Chapter 2. Unfortunately, not all large systems exhibit sparse Jacobian or Hessian matrices. In this section, we demonstrate the exploitation of structure in large scale applications and show how it is possible to ....

.... structure for computation of gradients using AD [CJ97] Griewank and others have looked at Newsam and Ramsdell approach to compute the sparse Jacobian matrix [NR83, GUG96,GU96] There has been work on using both modes of automatic di#erentiation to exploit sparsity in Jacobian matrices [CV98c,SH95] Automatic di#erentiation tools The proceedings of the Breckenridge conference in 1991 [GC91] contains an article by David Juedes of then existing software packages [Jue91] There has been phenomenal development in AD tools since then involving both improvement of then existing tools and new AD ....

T. Steihaug and S. Hossain. Computing a sparse jacobian matrix by rows and columns. Technical Report, Department of Computer Science, University of Bergen, 1995.

....than nz r . For some practical problems the heuristics return increasing coloring numbers with increasing dimension, while nz r remains constant. In contrast to the original (CPR) algorithm graph coloring achieves results that are essentially independent of column permutation. Recently Steihaug [17] and Coleman [6] proposed a major extension and improvement built upon vector Jacobian products as in (f3) The principle is to look also at structurally independent rows, performing a coloring of the row incidence graph and then exploiting (f3) For matrices with an arrow like structure that have ....

T. Steihaug and S. Hossain, Computing a sparse Jacobian Matrix by Rows and Columns, University of Bergen, Department of Computer Science, 1995.

....of this paper is to show how it is possible to dramatically lower the cost of computing J by exploiting structure and sparsity in the application of AD. Recently, techniques for the efficient determination of sparse Jacobian matrices J , via AD, have been developed [Averick1994a] Coleman1995a] [Hossain1995a]. The bi coloring approach of Coleman and Verma [Coleman1995a] as discussed in Section 2, rests on the observation that is is usually possible to define thin matrices V; W such that the nonzero This research was partially supported by the Applied Mathematical Sciences Research Program ....

.... B ffi x ffi y 1 ffi y 2 1 C A = 0 B 0 0 GammaF (x) 1 C A ; 3) where JE = 2 6 4 Gamma J I 0 A x y 2 GammaI A 0 0 Gamma J 3 7 5 : 3 Here is a key point: the extended Jacobian matrix JE is sparse and clearly sparse AD techniques, e.g. Averick1994a] Coleman1995a] [Hossain1995a], can be applied with respect to FE (x; y) 0 B y 1 Gamma F (x) A(x)y 2 Gamma y 1 Gamma F (y 2 ) 1 C A : to efficiently determine JE . For example, the work required by the bi coloring technique developed in [Coleman1995a] is Delta (FE ) Delta (F ) where is a ....

[Article contains additional citation context not shown here]

A. K. M. Hossain and T. Steihaug, Computing a sparse Jacobian matrix by rows and columns, Technical Report 109, Department of Informatics, University of Bergen,, Bergen, June 1995.

....we analyze the problem of the estimation of a Jacobian matrix from a graph theoretic view point. In particular, we show how the known sparsity structure can be exploited in computing rows and columns. Methods for computing the Jacobian matrix by partitioning rows and columns have been studied in [11, 12]. The main results of the paper is given in Section 2. We begin Section 2 by considering the partitioning of the rows and the columns. We give examples where either row or column partitioning alone may not be able to take full advantage of the known sparsity. A more general partitioning problem is ....

Steihaug, Trond and Hossain, A. K. M. S. , Computing a Sparse Jacobian Matrix by Rows and Columns, Report 109, Department of Informatics, University of Bergen, June 1995.

....expensive. The purpose of this paper is to show how it is possible to dramatically lower the cost of computing J by exploiting structure and sparsity in the application of AD. Recently, techniques for the efficient determination of sparse Jacobian matrices J , via AD, have been developed [1, 9, 13]. The bi coloring approach of Coleman and Verma [9] as discussed in x2, rests on the observation that is is usually possible to define thin matrices V; W such that the nonzero elements of J can be readily extracted from the pair To appear: Proceedings of the Second SIAM International ....

.... Newton equations: JE 0 B ffi x ffi y 1 ffi y 2 1 C A = 0 B 0 0 GammaF (x) 1 C A ; 5) where JE = 2 6 4 Gamma J I 0 A x y 2 GammaI A 0 0 Gamma J 3 7 5 : 6) 3 Here is a key point: the extended Jacobian matrix JE is sparse and clearly sparse ADtechniques, e.g. [1, 9, 13], can be applied with respect to FE (x; y) 0 B y 1 Gamma F (x) A(x)y 2 Gamma y 1 Gamma F (y 2 ) 1 C A : 7) to efficiently determine JE . For example, the work required by the bi coloring technique developed in [9] is Delta (FE ) Delta (F ) where is a bi chromatic ....

[Article contains additional citation context not shown here]

A. K. M. Hossain and T. Steihaug, Computing a sparse Jacobian matrix by rows and columns, Tech. Rep. 109, Department of Informatics, University of Bergen, June 1995.

....r . For some practical problems the heuristics return increasing coloring numbers with increasing dimension, while nz r remains constant. In contrast to the original CPR algorithm, graph coloring achieves results that are essentially independent of column permutation. Recently Steihaug and Hossain [18] and Coleman and Cai [7] proposed a major extension and improvement built upon vector Jacobian products as in (f3) The principle is to look also at structurally independent rows, perform a coloring of the row incidence graph, and then exploit (f3) For matrices with an arrow like structure that ....

....principle is to look also at structurally independent rows, perform a coloring of the row incidence graph, and then exploit (f3) For matrices with an arrow like structure that have only a few (nearly) dense rows and columns, both techniques alone are impracticable. Therefore, the authors of both [18] and [7] suggested a combination of the two techniques that allows the treatment of a larger class of sparse matrices. In this paper we extend the method of G. N. Newsam and J. D. Ramsdell (NR) which is discussed in the next section. Unlike its competitors, the NR approach is not bounded by the ....

T. Steihaug and S. Hossain, Computing a sparse Jacobian Matrix by Rows and Columns, University of Bergen, Department of Computer Science, 1995.

....of both forward and reverse modes, may not be possible. There is current research activity on reducing the space requirements of the reverse mode of automatic differentiation, e.g. 12] We note that an independent proposal regarding sparse Jacobian calculation is made by Hossain and Steihaug [15]: a graph theoretic interpretation of the direct determination problem is given and an algorithm based on this interpretation is provided. In this paper we proffer a new direct method and we also propose a substitution method, both based directly on the Jacobian structure. We compare our direct ....

....OE(j) 4. Every path of 3 edges uses at least at least 3 colors. The smallest number for which graph G b is bipartite path p colorable is denoted by p (G b ) Figure 2.4 shows a valid bipartite path p coloring. Numbers adjacent to the vertices denote colors. We note that Hossain and Steihaug [15] define a similar concept. However, their definition of path p coloring does not allow for the uncolor assignment , i.e. OE(i) 0. Consequently, a technique to remove empty groups is needed [15] We are now in position to state the graph analogy to the concept of a bi partition consistent with ....

[Article contains additional citation context not shown here]

A. K. M. Hossain and T. Steihaug, Computing a sparse Jacobian matrix by rows and columns, Tech. Report 109, Department of Informatics, University of Bergen, June 1995.

.... Newton equations, JE 0 ffi x ffi y 1 ffi y 2 1 A = 0 0 0 GammaF (x) 1 A ; 4) where JE = 2 4 Gamma J I 0 A x y 2 GammaI A Gamma J x 0 Gamma J y 3 5 : 5) The point here is that the extended Jacobian matrix JE is sparse and clearly sparse AD techniques, e.g. [1, 6, 9] can be applied with respect to FE (x; y) 0 y 1 Gamma F (x) A(x)y 2 Gamma y 1 Gamma F (x; y 2 ) 1 A (6) 3 to efficiently determine JE . For example, the work required by the bi coloring technique developed in [6] is of order Delta (FE ) Delta (F ) where is a ....

A. K. M. Hossain and T. Steihaug, Computing a sparse Jacobian matrix by rows and columns, Tech. Report 109, Department of Informatics, University of Bergen, Bergen, June 1995.

....[9] SparseLib is a library of C classes for the efiqcient storage and operation of sparse matrices. 4 Concluding remarks This paper describes an application of graph coloring ideas in the numerical determination of large sparse derivative matrices. Other related work can be found in [2, 5, 6, 13]. We also note that the procedure for the determination of Jacobian matrices as outlined in Page 2 can be given in a more general way [14, 15] and a different graph coloring formulation of the problem is currently under investigation. ....

A. S. Hossain and T. Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33-48, 1998.

....method can save many AD passes. This assertion is supported by numerical examples. 1 Introduction To determine a sparse Jacobian matrix, one can compute a partition of the columns and then use finite di#erences (FD) or automatic di#erentiation (AD) to obtain the nonzeros. In a direct method [1, 5, 7, 12], the nonzero entries corresponding to a group of columns can be read o# the di#erence approximations or a forward pass directly, i.e. without any other arithmetic operations. In an indirect method such as a substitution method the unknowns are ordered such that the ordering leads to a triangular ....

....number of AD passes. Substitution methods have been investigated mainly in the context of symmetric matrices [3, 10, 14] In this chapter we extend the results in [11] and present numerical experiments. The techniques presented here assume that a column (row) partition or a complete direct cover [12] or bipartition [6] is available. Using the sparsity information we define systems of linear equations that can be solved by substitution. If the number of groups in the partition is larger than the maximum number of nonzeros in any row of the Jacobian matrix, then the proposed techniques reduce ....

[Article contains additional citation context not shown here]

A.K.M. Shahadat Hossain and Trond Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33--48, 1998. 10 Shahadat Hossain and Trond Steihaug

No context found.

S. Hossain and T. Steihaug. Computing a sparse Jacobian matrix by rows and columns. Optimization Methods and Software, 10:33--48, 1998.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC