P. Hovland, C. Bischof, D. Spiegelman, and M. Casella, Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics, Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... The Computational Differentiation (CD) known also as Automatic or Algorithmic Differentiation, which is the subject of this article. Computational Differentiation is a well established research and engineering domain, and the number of theoretical and applied papers is impressive. See, e.g. [4, 5, 9, 11, 12], and especially the bibliography collected by George Corliss [8] The CD algorithms are numerical, but they yield results as exact as the numerical evaluation of symbolic derivatives. Unfortunately relatively little has been written about functional programming in this context. It seems that the ....

Hovland Paul, Bischof Christian, Spiegelman Donna, Cosella Mario, Efficient derivative codes through automatic differentiation and interface contraction: an application in biostatistics, SIAM J. on Sci. Comp. 18, (1997), pp. 1056--1066.

....of Automatic or Algorithmic Differentiation (which makes the bibliographic search somewhat irritating. This is the main subject of this article. The Computational Differentiation is a very well established research and engineering domain, and the number of references is impressive. See e.g. [2, 3, 4, 5], and especially [6] Unfortunately relatively little has been written about the functional programming in this context. Perhaps it was considered to be too easy . In fact it seems that the practical aspects of the domain tend to concentrate the implementors attention to Fortran, C , and ....

P. Hovland, C. Bischof, D. Spiegelman, M. Cosella, Efficient Derivative Codes through Automatic Differentiation and Interface Contraction: an Application in Biostatistics, SIAM J. on Sci. Comp. 18, (1997), pp. 1056--1066.

....Likelihood Optimization at the Harvard School of Public Health Mario Casella and Donna Spiegelman, in the Departments of Epidemiology and Biostatistics at the Harvard School of Public Health, are using ADIFOR 2. 0 to perform maximum likelihood optimization on problems in nutritional epidemiology [22]. Two likelihood functions were considered, one with 17 parameters, the other with 33 parameters. For the 17 parameter case, analytic derivatives had been constructed by hand over a period of two years. Initial results using the derivative code generated by ADIFOR 2.0 exhibited roughly a linear ....

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella, Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics, Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995.

....derivatives faster than divided difference approximations (see the references in the ADIFOR 2. 0 paper (Bischof et al. 1994) and there are examples where the availability of fully accurate derivatives was essential for numerical robustness and convergence (see, for example, the papers by Hovland et al. 1995), Eberhard (1996) and Ibsais and Ajjarapu (1996) By taking AD considerations into account in the development of their software, library developers can easily develop sensitivity enhanced versions of their codes using AD tools. Some needed features (such as intrinsics handling) are already ....

Hovland, P., Bischof, C., Spiegelman, D., and Casella, M. (1995). Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory. To appear in SIAM J. Scientific Computing.

....eliminate the nodes above node s to compute s= a; b; c) then eliminate the nodes below node s to compute (x; y; z) s, and finally eliminate s to arrive at (a; b; c) x; y; z) only 17 multiplications are required. Iri [20] referred to such a situation as a vertex cut ; Hovland et al. [19] called it an interface contraction. This small example illustrates three important principles. Graph Partitioning: The identification of program sections whose derivatives with respect to input and output variables should be computed out of context, that is, ignoring the surrounding ....

....the ADIFOR ADIC fashion at a lower level. A concrete example of stencil level partitioning is given in the next section. 4. Moving up further in the program structure, we observe that subroutine boundaries may be likely places for interface contraction. For example, in the application described in [19], a subroutine that computed a scalar output value from two scalar input values was called in a loop. By considering this subroutine as its own derivative partition, computing its associated two element gradient using ADIFOR, and then chaining this resulting derivative back into the remainder of ....

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella, Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics, Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995. To appear in SIAM J. Scientific Computing.

....eliminate the nodes above node s to compute s= a; b; c) then eliminate the nodes below node s to compute (x; y; z) s, and finally eliminate s to arrive at (a; b; c) x; y; z) only 17 multiplications are required. Iri [20] referred to such a situation as a vertex cut ; Hovland et al. [19] called it an interface contraction. This small example illustrates three important principles. Graph Partitioning: The identification of program sections whose derivatives with respect to input and output variables should be computed out of context, that is, ignoring the surrounding ....

....the ADIFOR ADIC fashion at a lower level. A concrete example of stencil level partitioning is given in the next section. 4. Moving up further in the program structure, we observe that subroutine boundaries may be likely places for interface contraction. For example, in the application described in [19], a subroutine that computed a scalar output value from two scalar input values was called in a loop. By considering this subroutine as its own derivative partition, computing its associated two element gradient using ADIFOR, and then chaining this resulting derivative back into the remainder of ....

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella, Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics, Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995. To appear in SIAM J. Scientific Computing.

....of abstraction. At the simplest, the forward mode works at the scope of a single binary operation. By expanding the scope to a higher level, such as an assignment statement, a loop body or a subroutine, it is possible to decrease the amount of work necessary to propagate derivatives, as shown in [7, 9]. The preaccumulation technique computes the gradient and Hessian of the variable on the left side of the assignment statement in two steps. Assume that for the statement z = f(x1 ; x2 ; xN ) we have rx i and r 2 x i ; i = 1; Delta Delta Delta N , the global gradient and Hessian of x ....

Hovland, P., Bischof, C., Spiegelman, D., and Casella, M. Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995. To appear in SIAM J. Scientific Computing 18-4 (July 97).

....suggests that implementations combining both approaches will be the promising approach in the long term. Lastly, the associativity of the chain rule of differential calculus leaves great leeway in how derivatives are actually computed. New approaches are slowly emerging (see, for example, [9]) but much remains to be explored. 6. ....

Paul Hovland, Christian Bischof, Donna Spiegelman, and Mario Casella. Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995.

....turnaround time for derivative computations by spawning several independent subprocesses computing parts of the desired gradient or Jacobian. The seed matrix also provides a powerful mechanism for decreasing the computational complexity of derivative codes through judicious use of the chain rule [8, 21]. The running time and storage requirements of the ADIFOR generated code are roughly proportional to the numbers of columns of S, which equals the g p variable in the sample code above. ADIFOR has been successfully applied to codes from various domains of science. Experiences with meteorological ....

Paul Hovland, Christian Bischof, Donna Spiegelman, and Mario Casella. Efficient derivative codes through automatic differentiation and interface contraction and an application in biostatistics. in preparation.

....of abstraction. At the simplest, the forward mode works at the scope of a single binary operation. By expanding the scope to a higher level, such as an assignment statement, a loop body or a subroutine, it is possible to decrease the amount of work necessary to propagate derivatives, as shown in [8, 10]. A preaccumulation technique we employ in our work computes the gradient and Hessian of the variable on the left side of the assignment statement in two steps. Assume that for the statement z = f(x1 ; x2 ; xN ) we have rx i and r 2 x i ; i = 1; Delta Delta Delta N , the global ....

Hovland, P., Bischof, C., Spiegelman, D., and Casella, M. Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995. To appear in SIAM J. Scientific Computing 18, 4 (July 97).

....implicit method. The computation of the Jacobian can be accelerated by employing the technique of Averick et al. [2] to compute a compressed Jacobian using dense derivative vectors. An additional small improvement can be achieved through the use of the interface contraction technique described in [11]. Finally, the nonlinearity of the function Psi(u) Automatic Differentiation and Navier Stokes Computations 15 can be handled better if we solve the system of nonlinear equations directly rather than first linearizing the system. 5.1.1. Compressed Jacobian Since the Jacobian being computed is ....

....we use SparsLinC to compute a sparse Jacobian. Because the mesh does not change, subsequent iterations have the same sparsity pattern. By coloring the Jacobian matrix from the first iteration, we are able to compute a compressed Jacobian for subsequent iterations. 5.1.2. Interface Contraction In [11], Hovland et al. describe a heuristic for reducing the cost of computing derivatives based on the observation that the number of parameters passed to a subroutine is usually quite small. If the number of such variables is small relative to the number of independent variables, it is referred to as ....

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella. Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. SIAM Journal on Scientific Computing, 18(4):1056--1066, July 1997.

....Likelihood Optimization at the Harvard School of Public Health Mario Casella and Donna Spiegelman, in the Departments of Epidemiology and Biostatistics at the Harvard School of Public Health, are using ADIFOR 2. 0 to perform maximum likelihood optimization on problems in nutritional epidemiology [22]. Two likelihood functions were considered, one with 17 parameters, the other with 33 parameters. For the 17 parameter case, analytic derivatives had been constructed by hand over a period of two years. Initial results using the derivative code generated by ADIFOR 2.0 exhibited roughly a linear ....

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella, Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics, Preprint MCS-P491-0195, Mathematics and Computer Science Division, Argonne National Laboratory, 1995.

No context found.

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella. Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. SIAM Journal on Scientific Computing, 18(4):1056--1066, 1997.

No context found.

P. Hovland, C. Bischof, D. Spiegelman, and M. Casella. Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. SIAM Journal on Scientific Computing, 18(4):1056--1066, 1997.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC