Concurrency lets multiple tasks make progress over the same period. When one task waits for network or disk I/O, another can run; on a single CPU core, the system can instead interleave tasks by switching between them. Parallelism is narrower: tasks execute simultaneously on different processing resources. Interleaving can improve responsiveness without making CPU-intensive work finish faster. section 9.1 section 9.0
Figure 9.1 Concurrency versus parallelism. Concurrent execution (on the left) interleaves thread execution on a single core, which provides an illusion of work happening at the same time because all threads make some progress quickly. In contrast, parallel execution (on the right) executes the threads simultaneously on different cores, providing true parallel execution.
Compare interleaved execution on one core with simultaneous execution on separate cores. section 9.1
Concurrency is also a count of work in progress. Little’s law relates average concurrency C, throughput T, and average latency L: C = T × L. At 1,000 requests per second and 50 milliseconds average latency, about 50 requests are in the system at once. The equation describes a relationship, not a promise that adding requests will lower latency: growing queues can increase waiting time. section 2.1.1
Choose an execution model around the work and its costs:
- Threads provide independent execution streams. An OS can switch between them on one core or run them on different cores, but shared resources require synchronization, and switching has overhead. section 9.2.1 section 9.7
- Coroutines provide suspendable, user-space-managed execution; event-driven designs use event loops and non-blocking operations. These offer alternatives when tasks frequently wait. section 9.7
- Use data or task parallelism when independent work can actually run simultaneously. Neither form should be confused with interleaving on one processor. section 9.7 section 9.1
Measure a waiting workload
For a controlled comparison in Rust, make each task await a 10-millisecond sleep. Await ten tasks one after another for a sequential baseline; then collect ten futures and await them together with Tokio’s join_all. The reported measurements are about 129.8 milliseconds sequentially and 12.9 milliseconds concurrently. Because the workload is an artificial wait, this result demonstrates overlapping delays—not an equivalent speedup for CPU-intensive calculations. section 9.6
More concurrency is not automatically better. Contended locks can leave threads waiting and add context-switching costs; database transactions also need concurrency control to preserve consistency. In an asynchronous service, monitor active tasks, thread-pool utilization, and queue depths alongside queue-waiting and processing times. If every worker stays busy while queues grow, investigate pool capacity and task duration before raising a limit. section 8.2.1 section 9.7 section 10.7.2
