liveBook · Manning

Processor affinity

As explained in Quantization and Fast Inference

CPU affinity is a policy or mechanism that restricts a process or thread to specified processor units.

Processor affinity pins a process or thread to selected CPUs. Placement matters: forcing work onto physical cores without their SMT siblings can sharply reduce throughput, while using SMT siblings can add useful parallel work. section 10.5.2

Pinned thread placement

On an 11-core, 22-vCPU host, placing 11 threads on physical cores 0–10 and a 12th on CPU 11 reproduced a throughput dip to 15.63 tok/s. Forcing 22 threads onto CPUs 0–10 instead reduced throughput to 9.98 tok/s. section 10.5.2

How Quantization and Fast Inference covers it

How deep it goes
Measures specific pinning outcomes and relates them to SMT sibling use and cache contention.
Examples
Compares pinned placements on an 11-core, 22-vCPU host, reporting throughput of 15.63 and 9.98 tok/s.
What you can do after
Evaluate how CPU placement choices affect throughput when threads use physical cores and SMT siblings.
How it connects
Connects thread count and processor affinity to simultaneous multithreading and L1/L2 cache contention.

Also explained in

Compare how 2 books explain processor affinity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.