A performance claim about running code needs observed results, not just an argument about what ought to be fast. Algorithm selection calls for a separate complexity assessment, supported by proof rather than speculation. Once code exists, measure its execution—and remember that the measurement is part of the execution you observe. section 16.4
Timing a loop
A C timing experiment places timespec_get(&t[0], TIME_UTC) before a loop and timespec_get(&t[1], TIME_UTC) after it. One test uses a volatile loop counter so that the otherwise empty loop is actually executed; another writes each iteration’s value to a volatile object. The timing calls delimit different work, but the volatile operations also define what work the compiler must retain. Do not interpret these tests as timings of an ordinary, unobservable empty loop. section 16.4
TAKEAWAY: All measurements introduce bias. A timing call takes time, may require the compiler to save registers or abandon optimization assumptions, and may affect scheduling or caches through an operating-system call. Instrumentation can therefore change both compile-time decisions and run-time behavior, not merely add a fixed cost to subtract. section 16.4
A measured comparison needs more than one elapsed-time value:
- Repeat the code snippets and collect elapsed-time differences between successive timestamps. section 16.4
- Report variation alongside the mean; the timing example prints an unbiased relative standard deviation as a percentage. section 16.4
- Inspect the compiled loops when their measured behavior is surprising: the GCC and Clang assembly listings show what the compiler retained. section 16.4
Table 16.1 Comparison of measurements
| Loop | Sec per iteration | Difference | Gain/loss | Conclusive | |
|---|---|---|---|---|---|
| 0 | volatile loop |
1.97⋅10-09 | |||
| 1 | volatile store |
3.66⋅10-10 | -1.60⋅10-09 | -81% | Yes |
| 2 | Opaque addition | 3.45⋅10-10 | -2.10⋅10-11 | -6% | No |
| 3 | Plus timespec_get |
1.77⋅10-08 | 1.74⋅10-08 | +5043% | Yes |
| 4 | Plus mean | 2.62⋅10-08 | 8.5⋅10-09 | +48% | Yes |
| 5 | Plus variance | 2.90⋅10-08 | 2.8⋅10-09 | +11% | Yes |
| 6 | Plus skew | 3.17⋅10-08 | 2.7⋅10-09 | +9% | Yes |
The comparison reports seconds per iteration, differences, percentage gains or losses, and whether each result is conclusive. Its opaque-addition comparison shows a reported 6% gain marked “No”: a numerical difference alone is not a supported improvement claim. section 16.4
TAKEAWAY: Accompany performance-improvement claims with thorough measurements and statistics, without trading away correctness. In the demonstrated setup, measurement and statistics-collection overhead can be made negligible; that is a result to check, not a reason to ignore instrumentation. section 16.5 section 16.4
