1 Why parallel computing

Parallel computing is the execution of many operations at a single instance in time. Fully exploiting parallel computing does not happen automatically. It requires some effort from the programmer. First, you must identify and expose the potential for parallelism in an application. Potential parallelism, or concurrency, means that you certify that it is safe to conduct the operations in any order as the resources become available on the system. For parallel computing there is an additional requirement; these operations must occur at the same time. For this to happen, you must also properly leverage the resources to execute them simultaneously. Parallel computing introduces new concerns that are not present in a serial world. A change in thought process is needed to adapt to the additional complexities of parallel execution, but with practice, it begins to be second nature. This book begins your journey on how to access the power of parallel.

1.1       Why should you learn about parallel computing?

1.1.1   What are the potential benefits of parallel computing?

1.1.2   Parallel computing cautions

1.2       The fundamental laws of parallel computing

1.2.1   The limit to parallel computing: Amdahl’s Law

1.2.2   Breaking through the parallel limit: Gustafson-Barsis’s Law

1.3       How does parallel computing work?

1.3.1   Walk through a sample application

1.3.2   A hardware model for today’s heterogeneous parallel systems

1.3.3   The application/software model for today’s heterogeneous parallel systems

1.4       Categorizing parallel approaches

1.5       Parallel Strategies

1.6       Parallel speedup vs comparative speedups: two different measures

1.7       What will you learn in this book?

1.7.1   Exercises

1.8       Summary