Energy per inference is the electrical energy used to complete one model inference, commonly expressed in joules. For edge deployments, it can be a more useful cost axis than mean power: the limiting budget depends on the workload. section 11.6
Power and energy constrain different workloads. Whisper-tiny on the M3 Neural Engine draws about 5.7 W, yet takes 30 times longer per inference than EfficientNet; for sequential ASR, energy per inference can become the binding constraint. Apply latency and power budgets together, then identify which axis actually limits the workload. section 11.1 section 11.6
Keep comparisons on a fair axis
On the same L4 server, AWQ INT4 roughly doubles peak throughput over FP16 and reduces energy from 45.4 to 24.5 mJ per token. Do not directly compare that server figure with the device estimate of 16.8 mJ per inference: the units describe different workloads, and one value is sensor-grade while the other is estimated. section 12.3.4 section 12.3.4
