Parallelization executes two or more independent tasks at the same time on separate processing units. Concurrency can exist on one CPU core through task switching, but parallelism requires multiple cores or equivalent processing capacity. section 1.3.2 section 1.3.3
Use the distinction to choose an appropriate execution model:
- Treat I/O-bound work as concurrent rather than truly parallel in Python: the GIL can be released during I/O, but only one piece of Python code executes at a time. section 1.5.1 section 1.8
- Use processes for CPU-bound work when sufficient CPU cores make parallel execution worth the overhead. section 6.4.3 section 6.7
- Split large data into chunks, map a computation over those chunks, and reduce the partial results; protect shared state with locks when necessary. section 6.4.3 section 6.7
Parallel word-frequency counting
A large word-frequency file can be partitioned into chunks, processed by separate workers, and merged through a reduce operation. Process pools distribute the CPU-bound counting and reduction work, while asyncio.gather waits for the worker results. section 6.4.3 section 6.7
Figure 1.1 With concurrency, we have multiple tasks happening at the same time, but only one we’re actively doing at a given point in time. With parallelism, we have multiple tasks happening and are actively doing more than one simultaneously.
Use the figure to compare concurrency—multiple tasks in progress with one active at a time—with parallelism—multiple tasks actively executing simultaneously. section 1.3.2
