Context switching lets multiple threads or processes share one CPU core. The operating system stops the current thread, saves execution state such as registers, memory mappings, and stack pointers, then loads another thread’s state. Each thread makes progress, though only one runs on that core at a time. section 2.3.4 section 9.2.1
A switch costs roughly microseconds of CPU work. Its effects can last longer: kernel activity may displace application data from caches, and the incoming thread may miss in the cache as it resumes. Frequent switches can therefore produce unpredictable latency spikes. Scheduling delay is related but distinct: a ready thread must also wait for the OS to run it. section 2.3.4 section 7.5.1
Figure 9.2 The OS runs multiple threads at the same time (but not simultaneously) on a single CPU core by context-switching between them. In this example, every thread is run for exactly 4 milliseconds, and then the OS switches to another thread. In real-world systems, a thread might block to wait for I/O, for example, causing the OS to switch to another thread more quickly.
Three threads take turns on one core in four-millisecond slices. The fixed interval is illustrative; a real scheduler may switch sooner when a thread blocks for I/O. section 9.2.1
For latency-sensitive workloads, reduce unnecessary competition for CPU time before adopting more elaborate concurrency designs:
- Keep the thread count low. More runnable threads can mean more scheduling delays and switches. section 7.5.1
- Consider CPU affinity to confine application threads to selected CPUs; higher thread priority can reduce scheduling delay, but it does not remove switch costs. section 7.5.1
- For a more specialized design, use at most one pinned application thread per core and manage concurrency with coroutines or futures and promises. This reduces reliance on OS thread scheduling but adds architectural complexity. section 7.5.1
