Imagine planning a trip with friends. One person books accommodation while another chooses a route. They can work at the same time, but each may need information from the other. Concurrent tasks face the same problem: communication lets them exchange needed data and coordinate when work can proceed. A task that depends on another task’s result may have to wait for it. section 13.2 section 5.0
In a sequential program, its parts can access the program’s data. Once work is split into tasks running on different processors or cores, needed data may belong to another task. Designing communication means deciding what data must move, when tasks must coordinate, and what channel they will use. Even a simple division of work can create a complicated pattern of exchanges. section 13.5
Two timing choices affect how much useful work tasks can do while communicating:
- Synchronous communication makes a task wait for an operation to complete before it continues. That wait provides a synchronization point, but it can leave tasks idle during data exchange. section 13.1 section 13.11
- Asynchronous communication lets a task request an operation and do other work while completion is pending. It suits long-running tasks that need not block, though sending and receiving data still consume processing time. section 13.1 section 13.11
A matrix task needs inputs
Suppose each task calculates one element of matrix C from a row of A and a column of B. Each task must have those inputs available. If tasks are combined to calculate whole rows of C, they need more of B. Giving every task a copy of B can use substantial memory; shared access is another option because the inputs are only read. The choice is not simply whether tasks can communicate, but what each arrangement costs. section 13.5
Passing data between cluster workers
In a word-count job, map and reduce tasks may run on different machines without shared context. Map output must reach the reduce tasks. Message passing is one option; storing intermediate data on shared network-attached storage is another. The amount of data, waiting time, and processing overhead all matter when choosing how to transfer it. section 13.11
Communication is a design step, not an automatic benefit of concurrency. After partitioning work, identify the data and coordination each task needs. Then consider grouping related tasks or giving them separate datasets to reduce exchanges and synchronization. Copies also have a cost, so compare that cost with the communication they avoid. section 13.2 section 13.5
