12 Advanced parallelism with teams, events, and collectives

 

This chapter covers

  • Forming teams of parallel images for different tasks
  • Synchronizing execution by posting and waiting for events
  • Exchanging data across images using collectives

Parallel programming is ubiquitous in many applications in science and engineering, such as aerodynamics, weather and ocean prediction, and machine learning. Parallel programming lets you distribute work between many CPUs, allowing the program to finish sooner. Distributing the work also reduces the amount of memory needed by the program, so parallelism allows running large programs that otherwise wouldn’t fit into the memory of a single computer. Fortran is natively parallel, which means that the syntax used to express parallel programs is built into the language itself.

In chapter 7, your first foray into parallel Fortran programming was through coarrays. They allowed you to distribute the work among multiple CPUs, exchange data between them, and perform the computations faster. In this chapter, we’ll take it a step further and explore three new parallel concepts: teams, events, and collectives. We’ll use these new features toward the final implementation of the tsunami simulator that we’ve been developing in this book.

12.1 From coarrays to teams, events, and collectives

12.2 Grouping images into teams with common tasks

12.2.1 Teams in the tsunami simulator

12.2.2 Forming new teams

12.2.3 Changing execution between teams

12.2.4 Synchronizing teams and exchanging data

12.3 Posting and waiting for events

12.3.1 A push notification example

12.3.2 Posting an event

12.3.3 Waiting for an event

12.3.4 Counting event posts

12.4 Distributed computing using collectives

12.4.1 Computing the minimum and maximum of distributed arrays

12.4.2 Collective subroutines syntax

12.4.3 Broadcasting values to other images

12.5 Answer key

12.5.1 Exercise 1: Hunters and gatherers

12.5.2 Exercise 2: Tsunami time step logging using events

12.5.3 Exercise 3: Calculating the global mean of water height

12.6 New Fortran elements, at a glance

12.7 Further reading

sitemap