Parallelism is the simultaneous execution of separate computations on multiple processing units. A program can divide work into tasks—such as serving a request, searching a file, calculating a matrix row, or rendering a frame—but those tasks run in parallel only when the execution environment has multiple units available. On a multicore machine, different tasks may run at the same instant on different CPUs. section 2.4 section 2.1
Parallelism differs from concurrency. Concurrency structures program instructions as separate tasks and identifies their boundaries and synchronization points. Parallelism is what happens when an executing concurrent program can perform several of those tasks at once. A concurrent program on one processor can still appear simultaneous because the operating system rapidly switches threads, saving and restoring each job’s process context block; this is interleaving, not true CPU parallelism. section 2.4 section 2.1
When judging whether work is actually parallel, check the execution setting:
- Multiple CPUs or cores can execute jobs simultaneously, while sharing main memory and a bus interface; each core has its own CPU and at least one cache. section 2.1
- A single processor can time-share tasks. Frequent scheduler switches improve responsiveness and create an impression of simultaneous work without parallel execution. section 2.1 section 2.4
- Waiting for I/O complicates a narrow CPU-only view: one thread may write to disk while another executes CPU instructions. section 2.4
Matrix rows on a multicore system
Treat each row of a matrix multiplication as a separate concurrent task. On a multicore system, more than one row calculation may proceed at the same time. On a single processor, the same task structure may instead be interleaved by time sharing. The task decomposition remains useful, but observed parallel execution depends on the hardware and environment. section 2.4
In Go, goroutines are lightweight units of concurrent execution. Create goroutines for work that should proceed concurrently, then use channels when goroutines must exchange information or synchronize. The runtime and available hardware handle the mechanics of parallelism, so the same correct concurrent design can scale differently across environments rather than requiring manual resource allocation for every task. section 1.5.1
More processors do not automatically mean a proportional speedup. Gustafson’s law emphasizes that practical workloads can grow with available resources: if the serial portion stays constant while the problem grows, speedup can continue to rise as additional parallel resources are kept busy. Otherwise, surplus resources can be assigned to other work, such as improving software quality or accuracy. section 1.6.2
Compare speedup with processor count under Gustafson’s law, where keeping extra resources busy and growing the problem can support continued scaling. section 1.6.2
