Keywords: adjoint code, NAGWare Fortran compiler.

In this paper we report on recent advances made in the development of the first Fortran compiler that provides intrinsic support for computing derivatives. We focus on the automatic generation of an intraprocedural adjoint code. Technical details of the modifications made to the internal representation as well as case studies are presented. For example, the new feature allows for the computation of large gradients at a computational cost that is independent of their sizes. Numerous numerical algorithms, derivative-based optimization algorithms in particular, will benefit both from the convenience of the approach and from the efficiency of the intrinsic derivative code.

Consider the computation of the gradient of the following function that is due to Griewank [2].

The classical approach to numerical differentiation uses finite difference quotients to approximate the values of derivatives. Centered finite differences defined by

are known to exhibit second-order accuracy as opposed to only first order accuracy obtained when using forward or backward differences.

The computational complexity of the accumulation of the full gradient by finite difference approximation is O(n) Every single input needs to be perturbed which turns out to be infeasible for large n. The solution comes in the form of reverse mode automatic differentiation [1]. It builds the basis for semantic source transformation algorithms that generate adjoint code for the computation of

automatically. The adjoint input vector becomes equal to the gradient when setting A single evaluation of the adjoint code is sufficient (compared with at least evaluations of the original code when finite differences are used).

We describe a research prototype of the NAGWare Fortran compiler that provides intrinsic support for the generation of adjoint code. The functionality is currently limited to the intraprocedural level. A comparison of the runtime behavior of adjoint code for Equation (86) compared with finite difference approximation of the gradient is shown in Figure 1. This figure alone provides enough justification for the considerable development effort that adding adjoint capabilities to a compiler represents.

1

A. Griewank, Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation, Frontiers in Applied Mathematics, SIAM, 2000.

2

A. Griewank, Generalized descent for global optimization, Journal of Optimization Theory and Applications 34, 11-39, 1981. doi:10.1007/BF00933356

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £105 +P&P)