Shared Infrastructure for Source Transformation Automatic Differentiation

Research topic and goals

Our goal is to pursue development of Algorithmic Differentiation (AD) tools, to enhance their applicability to HPC codes. We focus on the adjoint mode of AD, which is the most effective way of obtaining gradients of large HPC simulations. We focus on AD tools by source transformation because of the efficiency of the code they produce. One of our goals is to implement cooperation at the algorithmic level between OpenAD, the AD tool developed at ANL and Tapenade, the AD tool developed at INRIA. Another goal is to develop and maintain AMPI, and open-source library wrapped around MPI that implements the adjoint operations corresponding to the most commonly used MPI communication primitives.

Results for 2016/2017

We have continued to develop a runtime library called ADMM to allow source-transformation adjoint AD of input codes that contain the operations pointers and dynamic allocation of memory. This is the first library of its kind for source transformation AD. Before this, dynamic allocation and pointer usage were unsupported in adjoint-mode-source-transformation AD. The library ensures that any memory accessed in the reverse computation exists through restoration (if needed) and contains the same value as it would have in the forward computation. Pointer variables used to access memory in the forward computation are updated to access the corresponding memory in the reverse computation. We have tested this method with a medium sized code. After an initial version the library was developed in 2016 that targeted codes written in C we have worked on refinements to the library and extended it to be target codes written in Fortran.

We have continued to develop the ability to interoperate OpenAD from Argonne and Tapenade from INRIA(Narayanan and Hascoet 2016). Interoperation between the OpenAD and Tapenade is possible because they share the same global architecture i.e. a front-end which parses and builds an internal representation, followed by an static data-flow analysis stage, then actual building of the differentiated program still in internal form, and finally a back-end that outputs this differentiated internal form into new source files. The interoperation uses Tapenade for parsing the code and analysis. It uses OpenAD’s XAIFBooster module for code transformation. The code to be differentiated is written into the XML Abstract Interface Form (XAIF) by Tapenade which is subsequently transformed by XAIFBooster. The transformed XAIF is converted back to source code by Tapenade. We have tested the new tool flow on various test cases from OpenAD’s regression testsuite.

Visits and meetings

Frequent e-mail exchanges. Laurent Hascoët (INRIA) visited Argonne and Sri Hari Krishna Narayanan (ANL) visited INRIA.

Impact and publications

In the last year, we have presented one extended abstract and a poster in AD2016, submitted a talk in USNCCM, and have another publication in preparation.

  1. Narayanan, Sri Hari Krishna, and Laurent Hascoet. 2016. “Interfacing OpenAD And Tapenade.”
    @article{NarayananHascoet2016,
      author = {Narayanan, Sri Hari Krishna and Hascoet, Laurent},
      title = {Interfacing OpenAD and Tapenade},
      journal = {},
      year = {2016},
      volume = {},
      number = {},
      pages = {},
      notes = {submitted, also available as {ANL/MCS-P5588-0316}}
    }
    

Future plans

We plan to continue to integrate Tapenade and OpenAD infrastructures. Additionally, we plan to explore other topics in the field of algorithmic differentiation.

References

  1. Goldberg, Daniel, Sri Hari Krishna Narayanan, Laurent Hasc oet, and Jean Utke. 2016. “An Optimized Treatment for Algorithmic Differentiation of an Important g Laciological Fixed-Point Problem.”
    @article{GoldbergEtAl2016,
      title = {An optimized treatment for algorithmic differentiation of an important g
          laciological fixed-point problem},
      journal = {},
      volume = {},
      number = {},
      pages = {},
      year = {2016},
      note = {submitted, also available as ANL/MCS-P5516-0116.},
      issn = {},
      author = {Goldberg, Daniel and Narayanan, Sri Hari Krishna and oet, Laurent Hasc and Utke, Jean}
    }