Christian H. Bischof, Peyvand M. Khademi, Ali Bouaricha, and Alan Carle. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation. Technical Report Preprint MCS-P519-0595, Mathematics and Computer Science Division, Argonne National Laboratory, August 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....solved by applying compiler technology to the user specified function code: dependencies of code structures and code variables are determined and the non zeros in the Jacobian matrix are detected. The automatic sparse code generation approach is supported by a C language package called SparsLinC [8, 11]. The SparsLinC library exploits sparsity dynamically. Non zero entries have been detected and calculated at automatic code derivation time. When invoked with the SparsLinC approach, ADIFOR allocates for every active variable an integer type variable that is a pointer to the sparse representation ....

Christian H. Bischof, Peyvand M. Khademi, Ali Bouaricha, and Alan Carle. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation. Technical Report Preprint MCS-P519-0595, Mathematics and Computer Science Division, Argonne National Laboratory, August 1995.

..... Thus, these techniques require at least ae 2 M function evaluations to estimate the Hessian matrix. 3 Computing Gradients We now outline the techniques that we use for computing the gradients of partially separable functions. For additional information on the techniques in this section, see [5, 8]. Computing the gradient of a partially separable function so that the bounds (2.2) and (2.3) are satisfied is based on the observation, due to Andreas Griewank, that if f : R n R is partially separable, then f(x) f E (x) T e; where e 2 R m is the vector of all ones, and hence rf(x) ....

C. Bischof, P. Khademi, A. Bouaricha, and A. Carle, Efficient computation of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation, Optim. Methods Software, 7 (1996), pp. 1--39.

....linear or nonlinear the code is or how often storage is overwritten. Moreover, this upper bound may be grossly pessimistic. In particular, by exploiting sparsity inherent in many largescale optimization problems, gradients of such problems can be efficiently computed with forward mode based tools [5, 7]. More in depth information on automatic differentiation can be found in [4, 14] and an overview of currently available AD tools is provided at URL http: www.mcs.anl.gov Projects autodiff AD Tools. In this article, we show how the reverse and forward modes of automatic differentiation can be ....

....time steps whose Jacobian they are responsible for. The mypid( function returns the unique ID number of a particular process between 0 and P Gamma 1. Suitable invocations of g H to exploit the sparsity of the timestep Jacobian are detailed for this particular example in [11] and, in general, in [2, 7]. The scheme outlined in Figure 3 allows us to generate all timestep Jacobians in a parallel fashion. The additional floating point and memory complexity is a fixed multiple of that of the complexity for H, depending on the particular stencil, but not the size of the grid. We also note that a ....

Christian Bischof, Peyvand Khademi, Ali Bouaricha, and Alan Carle. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation. Optimization Methods and Software, 7(1):1--39, July 1996.

....entries. In this case, SparsLinC provides a much more suitable implementation than the dense loops shown in Figures 6 and 7. SparsLinC, which is written in ANSI C, was originally developed in the context of the ADIFOR project and has been successfully employed in large scale nonlinear modeling [10, 16, 21, 18]. Since SparsLinC employs dynamic data structures, from a user s perspective it allows the exploitation of derivative sparsity without any a priori knowledge of the sparsity structure in a transparent fashion. ffl ADIntrinsics: The ADIntrinsics system provides (1) a reasonable default behavior ....

Christian Bischof, Peyvand Khademi, Ali Bouaricha, and Alan Carle. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation. Optimization Methods and Software, 7(1):1--39, July 1996.

.... Omega M 10p 2p Omega M 12p on the need to represent nonzero derivative information. Certainly, the memory needed for representing the sparse Jacobian matrix has a lower bound of nnz(f 0 (x) Beyond this, SparsLinC requires additional memory for internal representations as explained in [8]. The first column in Tables 1 3 shows the memory required for running the original function. Memory requirements for the hand coded MINPACK 2 gradient codes are not shown separately, but are always between a factor of 1.5 2 times the memory requirements of the corresponding function. The next ....

....owing to the code dependent effects of vectorization, as already discussed. We also note the large variation in Omega T for the Sparse AD results on the SPARC 10. This results from the way SparsLinC exploits the particular sparsity characteristics of each problem (this issue is explored in [8]) Finally, we note that the performance of Sparse AD degrades on vector computers, as a result of pervasive use of indirect addressing and lack of vector instructions, though this performance could be improved through the use of hardwaresupported gather scatter instructions. Table 6 also compares ....

BISCHOF, C., KHADEMI, P., BOUARICHA, A., and CARLE, A., 1996. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation, Optimization Methods and Software 7(1), 1--39.

.... all partial derivatives of H are evaluated at (Z(t) Z(t Gamma 1) W ) Typically, the matrices H Z(t) and H Z(t Gamma 1) are sparse due to the local nature of the stencil employed in H, and one can exploit this fact to compute them inexpensively using forward mode based AD tools [1, 7, 9]. Thus, if we use the aforementioned AD tools to (cheaply) compute H Z(t) and H Z(t Gamma 1) and then form d Z(t 1) d X through a series of matrixmatrix multiplications, we may well come out ahead. This article is structured as follows. In the next section, we review the ....

....of J at roughly half the cost compared with that induced by setting S to a 4 Theta 4 identity. In general, the groups of columns that can be grouped together can be identified via a graph coloring approach. This compressed Jacobian approach has been used successfully in large scale optimization [1, 7]. An alternative approach to exploiting sparsity is to employ sparse data structures for the derivative objects in the AD generated code. The SparsLinC library, which is integrated with the ADIFOR and ADIC tools, provides this functionality, and has been successfully employed in large scale ....

[Article contains additional citation context not shown here]

Christian Bischof, Peyvand Khademi, Ali Bouaricha, and Alan Carle. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation. Optimization Methods and Software, 7(1):1--39, July 1996.

....entries. In this case, SparsLinC provides a much more suitable implementation than the dense loops shown in Figures 6 and 7. SparsLinC, which is written in ANSI C, was originally developed in the context of the ADIFOR project and has been successfully employed in large scale nonlinear modeling [7, 12, 17]. Since SparsLinC employs dynamic data structures, from a user s perspective it allows the exploitation of derivative sparsity without any a priori knowledge of the sparsity structure in a transparent fashion. ffl ADIntrinsics: The ADIntrinsics system provides (1) a reasonable default behavior ....

Christian Bischof, Peyvand Khademi, Ali Bouaricha, and Alan Carle. Efficient computation of gradients and Jacobians by transparent exploitation of sparsity in automatic differentiation. Optimization Methods and Software, 7(1):1--39, July 1996.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC