Suppose a code fragment is timed 20 times. The mean is 0.6 seconds, but individual timings range from 0.1 to 1.3 seconds. A mean alone hides that variation. section 16.4
The sample’s standard deviation, σ(S), is 0.31 seconds. It measures departure from the ideal of identical timings. Because 0.31 is substantial beside the 0.6-second mean, these measurements give little grounds for a confident performance claim. Variation could come from the computation, the measurement, or both. section 16.4
Listing 16.5 Printing time stats with stats_mean and stats_rsdev_unbiased
for (unsigned i = 0; i < loops; i++) {
double mean = stats_mean(&statistic[i]);
double rsdev = stats_rsdev_unbiased(&statistic[i]);
printf("loop %u: E(t) (sec):\t%5.2e ± %4.02f%%,\tloop body %5.2e\n",
i, mean, 100.0*rsdev, mean/iterations);
}
The C listing prints each loop’s mean timing alongside an unbiased relative standard deviation, expressed as a percentage. section 16.4
TAKEAWAY: Repeat timings and inspect their spread before treating an average as representative. Measurement itself can change compile-time and run-time behavior. section 16.4
