Algorithmic Differentiation of Pragma-Defined Parallel Regions (PDF)

Numerical programs often use parallel programming techniques such as OpenMP to compute the program's output values as efficient as possible. In addition, derivative values of these output values with respect to certain input values play a crucial role. To achieve code that computes not only the output values simultaneously but also the derivative values, this work introduces several source-to-source transformation rules. These rules are based on a technique called algorithmic differentiation. The main focus of this work lies on the important reverse mode of algorithmic differentiation. The inherent data-flow reversal of the reverse mode must be handled properly during the transformation. The first part of the work examines the transformations in a very general way since pragma-based parallel regions occur in many different kinds such as OpenMP, OpenACC, and Intel Phi. The second part describes the transformation rules of the most important OpenMP constructs.

Contents

• Introduction with Examples from Numerical Optimization
• Algorithmic Differentiation by Source Transformation
• Transformation rules for Parallel Code Regions (e.g. OpenMP 3.1)
• Static Program Analysis

Target Groups

• Lecturers and students of computer science
• Computer scientists, engineers, mathematicians and numerical analysts

The Author