Preface

In the summer of 1997, I was terrified. Instead of working as an intern in my major (microelectronic engineering), the best job I could find was at a research laboratory devoted to high-speed signal processing. My job was to program the two-dimensional fast Fourier transform (FFT) using C and the Message Passing Interface (MPI), and get it running as quickly as possible. The good news was that the lab had sixteen brand new SPARCstations. The bad news was that I knew absolutely nothing about MPI or the FFT.

Thanks to books purchased from a strange new site called Amazon.com, I managed to understand the basics of MPI: the application deploys one set of instructions to multiple computers, and each processor accesses data according to its ID. As each processor finishes its task, it sends its output to the processor whose ID equals 0.

It took me time to grasp the finer details of MPI (blocking versus nonblocking data transfer, synchronous versus asynchronous communication), but as I worked more with the language, I fell in love with distributed computing. I loved the fact that I could get sixteen monstrous computers to process data in lockstep, working together like athletes on a playing field. I felt like a choreographer arranging a dance or a composer writing a symphony for an orchestra. By the end of the internship, I coded multiple versions of the 2-D FFT in MPI, but the lab’s researchers decided that network latency made the computation impractical.