Cache thrashing is the repeated eviction and reloading of useful CPU-cache data when competing or changing working sets prevent reuse. The processor keeps fetching data from memory instead of benefiting from recently loaded cache lines. section 9.2.1 section 7.4.5
Thread context switches can trigger thrashing because each thread brings a different working set. The direct switch may take hundreds of nanoseconds, but cache disruption can raise the effective cost to a microsecond or more and dominate execution when many threads are managed. section 9.2.1
Reduce cache thrashing by aligning data access with cache capacity and locality:
- Process large datasets in batches small enough to remain useful in CPU caches rather than repeatedly scanning the entire dataset. section 7.4.5
- Measure the working-set size and access pattern before choosing a memory strategy; data that fits in L1 can run far faster than data served from DRAM. section 7.4.5
- For NUMA systems, place a CPU’s working set on nearby physical memory; Linux mbind() and topology tools such as hwloc support this placement. section 7.4.5
- Prefer user-space concurrency or hardware parallelism when excessive kernel-thread switching makes cache disruption dominate useful execution. section 9.2.1
Figure 9.2 The OS runs multiple threads at the same time (but not simultaneously) on a single CPU core by context-switching between them. In this example, every thread is run for exactly 4 milliseconds, and then the OS switches to another thread. In real-world systems, a thread might block to wait for I/O, for example, causing the OS to switch to another thread more quickly.
Use the context-switching timeline to visualize how one CPU core alternates between threads and repeatedly changes its active working set. section 9.2.1
