A timing call changes the run
A C loop timed with `timespec_get(&t[0], TIME_UTC)` before it and `timespec_get(&t[1], TIME_UTC)` after it includes the cost of measuring. The calls may also change how the compiler optimizes the surrounding code. section 16.4
Bias is the deviation measurement introduces into the result it is meant to report. A timing call takes time, may require registers to be saved, and can affect scheduling or caches. TAKEAWAY: instrumentation changes both compile-time and run-time behavior. Assess that effect before treating a measured duration as the cost of the code alone. section 16.4
Table 16.1 Comparison of measurements
| Loop | Sec per iteration | Difference | Gain/loss | Conclusive | |
|---|---|---|---|---|---|
| 0 | volatile loop |
1.97⋅10-09 | |||
| 1 | volatile store |
3.66⋅10-10 | -1.60⋅10-09 | -81% | Yes |
| 2 | Opaque addition | 3.45⋅10-10 | -2.10⋅10-11 | -6% | No |
| 3 | Plus timespec_get |
1.77⋅10-08 | 1.74⋅10-08 | +5043% | Yes |
| 4 | Plus mean | 2.62⋅10-08 | 8.5⋅10-09 | +48% | Yes |
| 5 | Plus variance | 2.90⋅10-08 | 2.8⋅10-09 | +11% | Yes |
| 6 | Plus skew | 3.17⋅10-08 | 2.7⋅10-09 | +9% | Yes |
The comparison separates loop costs from the much larger per-iteration costs of adding `timespec_get` and statistical collection. section 16.4
Repeat experiments and collect statistics, including the mean and standard deviation, to help assess results. Repetition does not make the measurement apparatus disappear. section 16.4
