Parallel Scientific Computation (PDF)

This is the first text explaining how to use the bulk synchronous parallel (BSP) model and the freely available BSPlib communication library in parallel algorithm design and parallel programming. Aimed at graduate students and researchers in mathematics, physics and computer science, the main topics treated in the book are core topics in the area of scientific computation and many additional topics are treated in numerous exercises. An appendix on themessage-passing interface (MPI) discusses how to program using the MPI communication library. MPI equivalents of all the programs are also presented. The main topics treated in the book are core in the area of scientific computation: solving dense linear systems by Gaussian elimination, computing fast Fourier transforms, and solving sparse linear systems by iterative methods. Each topic is treated in depth, starting from the problem formulation and a sequential algorithm, through a parallel algorithm and its analysis, to a complete parallel program written in C and BSPlib, and experimental results obtained using this program on a parallelcomputer. Additional topics treated in the exercises include: data compression, random number generation, cryptography, eigensystem solving, 3D and Strassen matrix multiplication, wavelets and image compression, fast cosine transform, decimals of pi, simulated annealing, and molecular dynamics. The book contains five small but complete example programs written in BSPlib which illustrate the methods taught. The appendix on MPI discusses how to program in a structured, bulk synchronous parallel style using the MPI communication library. It presents MPI equivalents of all the programs in the book. The complete programs of the book and their driver programs are freely available online in the packages BSPedupack and MPIedupack.