Parallelism means executing multiple tasks simultaneously on different processors or other execution units. It can shorten work that would otherwise run sequentially, but it requires work that can actually overlap. Concurrency is broader: tasks can make progress by taking turns on one processor without executing simultaneously. section 9.1 section 9.3
Figure 9.1 Concurrency versus parallelism. Concurrent execution (on the left) interleaves thread execution on a single core, which provides an illusion of work happening at the same time because all threads make some progress quickly. In contrast, parallel execution (on the right) executes the threads simultaneously on different cores, providing true parallel execution.
The two timelines distinguish interleaved execution on one core from simultaneous execution on different cores. section 9.1
First identify independent work. Task parallelism assigns different tasks to processing units; data parallelism applies the same operation to different portions of data. A product page, for example, can fetch specifications, inventory, pricing, and reviews independently. Its completion time then approaches that of the slowest operation, plus coordination overhead, rather than the sum of all four times. section 9.3.2 section 9.7
Dependencies set a harder limit than processor count. Amdahl’s law estimates speedup from the fraction of a fixed-size problem that can run in parallel and the number of execution units. In a 50 ms order request, parsing takes 5 ms, database lookup 15 ms, and calculation and insertion 30 ms. The later stages depend on earlier results, so parallelizing the lookup targets only 30% of total time. Two execution units yield about 1.2× theoretical speedup; adding more cannot remove the sequential work. section 2.1.2
Check costs before adding more parallel work:
- At a join point, completion waits for the slowest task. Fanout can therefore magnify rare slow responses: with 100 subrequests and a one-in-100 chance that each takes one second, about 63% of requests exceed one second. section 2.2
- Shared locks can erase gains. A contended mutex makes threads wait; one counter benchmark rises from roughly 7 ns with one thread to roughly 5 µs with 100. section 8.2.1
- Task placement matters. Processor affinity can avoid thread migration and preserve cache locality; on NUMA systems, local memory placement can also reduce access latency. section 9.3.2
Figure 2.5 Tail latency illustrated. The diagram shows the probability of service latency being more than 1 second (y axis) as a function of parallel fanout count (x axis) depending on the probability of one fanout request being slow.
The fanout curve shows why a small probability of a slow subrequest can become a substantial service-level tail-latency risk. section 2.2
Measure overlap without mistaking it for parallel execution
A Rust micro-benchmark runs ten simulated 10 ms delays sequentially, then collects them as futures and awaits them with Tokio’s join_all. The measured times are about 130 ms and 13 ms. This demonstrates the benefit of overlapping waits; because the simulated work sleeps, it does not establish a tenfold speedup for CPU-bound work across processors. Benchmark the workload you intend to improve. section 9.6 section 9.1
Parallelism can improve throughput without improving each item’s latency. In a pipelined laundry example, washing the next load while drying the previous one raises throughput from 0.6 to 1 load per hour, while the stated per-load latency rises from 90 to 120 minutes. Choose the metric—latency or throughput—before treating additional overlap as a win. section 1.5
