The Art of High Performance Computing for Computational Science, Vol. 1 (PDF)

Starting with an introduction to the recent trends in computer architecture and parallel processing, Chapter 1 explains the basic knowledge of speedup programs with simple examples of numerical computing. Chapters 2 - 4 detail the basics of parallel programming, the message passing interface (MPI), and OpenMP and discuss hybrid parallelization techniques. Showing an actual example of adaptation, Chapter 5 gives an overview of performance tuning and communication optimizations. To deal with dense matrix calculations, Chapter 6 details the basics and practice of linear algebra calculation libraries BLAS and LAPACK, including some examples that can be easily reproduced by readers using free software. Focusing on sparse matrix calculations, Chapter 7 explains high performance algorithms for numerical linear algebra. Chapter 8 introduces the fast Fourier transform in large-scale systems from the basics. Chapter 9 explains optimization and related topics such as debug methods and version control systems. Chapter 10 discusses techniques for increasing computation accuracy as an essential topic in numerical calculation.

This is the first of the two volumes that grew out of a series of lectures in the K computer project in Japan. The second volume will focus on advanced techniques and examples of applications in materials science.