The observer effect can distort a benchmark when measuring a function changes the state that affects its performance. Reusing the same matrix lets CPU-cache behavior influence later iterations, so equivalent calculations can appear dramatically different. section 11.8.4
Reusing benchmark data
Two Go functions sum the first eight columns of matrices with 512 and 513 columns. Although their loops are equivalent, the reused-data benchmark reports about 15,073 ns/op for 512 columns and 7,358 ns/op for 513 columns. The surprising gap arises because the matrices interact differently with CPU caches, not because the functions process different numbers of columns.
Mitigate the distortion by changing the benchmark setup:
- Re-create the data used by a CPU-bound function inside each iteration, and stop the timer while preparing it. section 11.10
- Interpret micro-benchmark results cautiously when the benchmark machine differs from the machine running the application. section 11.10
- Ensure the tested function produces a side effect so compiler optimizations cannot make the measurement misleading. section 11.10
Corrected measurement
Creating a fresh 512-column matrix for every timed operation removes the original cache-state bias: the results become roughly 33,547 ns/op for 512 columns and 35,507 ns/op for 513 columns.
