Thermal throttling is a workload-dependent loss of sustained inference throughput when thermal limits constrain device operation. Short-burst latency can hide it; steady-state throughput is the more relevant measure for sustained workloads. section 11.3.4
Measured outcomes vary with the workload:
- On a fanless M3 MacBook Air, EfficientNet-Lite0 with Core ML palettize_4bit on the ANE fell from about 2,660 to 2,360 images per second over five minutes, an 11% soft-throttle. section 11.3.4
- Llama-3.2-1B INT8 decode on Pixel stayed near 23.8 tokens per second, consistent with a memory-bandwidth ceiling that did not move with temperature. section 11.3.4
- A 300-second M3 vision loop ended faster than it began: the light CPU-heavy workload did not reach its thermal envelope, while Core ML warm-up slowed the early measurements. section 12.2
For sustained workloads, compare throughput in the first and last 30 seconds of a thermally representative run. Interpret the result alongside workload and device conditions; a flat or rising rate does not show that heavier workloads will avoid throttling. section 11.3.4 section 12.2
