Parallelism runs independent workloads concurrently instead of processing them one after another. It reduces elapsed time when tasks can proceed without depending on each other. section 1.4 section 9.6
Sequential processing handles one item at a time; parallel processing fans out independent work and gathers the results. The pattern suits researching ten companies, reviewing twenty pull requests, or processing document batches, but not tasks that depend on earlier results. section 1.4 section 9.6
Use parallelism systematically:
- Identify whether each item is independent before enabling concurrent execution. section 9.6
- Fan out work with the asyncio.gather pattern, optionally limiting concurrency with a batch size. section 9.5
- Compare sequential and parallel execution by measuring total time and completion rates. section 9.5
- Watch rate limits and retain explicit handling for completed, failed, and pending items. section 9.6
Parallel task processing
Process five independent text-transformation tasks with retry logic, then run the same workload in parallel with a batch size of three. Compare execution time, completion rates, and permanently failed items. section 9.5
