liveBook · Manning

Benchmark

As explained in 100 Go Mistakes and How to Avoid Them

A benchmark is a controlled measurement of software execution under specified workloads or inputs.

A benchmark measures execution under a specified workload. It helps test a performance claim, but a precise-looking ns/op result does not guarantee that the intended work was measured. Setup time, machine activity, and compiler optimizations can all produce misleading comparisons. section 11.8 section 11.8.2

func BenchmarkFoo(b *testing.B) {
    for i := 0; i < b.N; i++ {
        foo()
    }
}

A Go benchmark starts with Benchmark and accepts *testing.B. Put the operation being measured inside the b.N loop. Go adjusts b.N to approach the requested benchmark time, which defaults to one second and can be changed with -benchtime. section 11.8

Do not include expensive preparation in the measurement by accident. For one-time setup, call b.ResetTimer() after setup and before the loop; this clears elapsed time and allocation counters. For setup needed on every iteration, stop the timer before setup and restart it before the operation under test. Pausing a long setup around a very fast operation can make the overall run take much longer than the requested benchmark time. section 11.8.1

func BenchmarkFoo(b *testing.B) {
    for i := 0; i < b.N; i++ {
        b.StopTimer()                #1
        expensiveSetup()
        b.StartTimer()               #2
        functionUnderTest()
    }
}

This pattern excludes per-iteration setup from the timed operation. Use b.ResetTimer() instead when setup occurs only once before the loop. section 11.8.1

A result that changes when tests change order

Separate micro-benchmarks made atomic.StoreInt64 appear faster than atomic.StoreInt32. Reversing their order made atomic.StoreInt32 appear faster. Machine activity, power management, thermal scaling, and instruction-cache alignment can affect tiny measurements. Increase -benchtime or run repeated measurements with -count; benchstat can compare the resulting statistics. In the repeated example, both operations measured 5.10 ns/op ± 1%. Treat a result cautiously when the application will run on a different system. section 11.8.2 section 11.10

An operation that disappears

A benchmark that discarded popcnt(uint64(i)) reported 0.2858 ns/op. After inlining, the compiler could remove the unused, side-effect-free calculation. Assigning each result to a local variable and assigning the final value to a global variable produced 1.993 ns/op in the example. Keep the result observable without adding a global write to every iteration. section 11.8.3

var global uint64                         #1
 
func BenchmarkPopcnt2(b *testing.B) {
    var v uint64                          #2
    for i := 0; i < b.N; i++ {
        v = popcnt(uint64(i))             #3
    }
    global = v                            #4
}

The local variable receives each result; the global variable receives the final one after the loop. This prevents the calculation from disappearing while limiting work inside the timed loop. section 11.8.3

Use benchmarks to inform a choice, not to override its context. Converting one million slice elements was much slower when the output slice grew without a known capacity. Allocating the full length and assigning by index was about 4% faster than reserving capacity and appending, but append was preferred in a non-performance-sensitive function because it kept indexing simpler. Match the workload and weigh measured gains against readability. section 3.5

How 100 Go Mistakes and How to Avoid Them covers it

How deep it goes
Explains benchmark loops, timer control, repeated micro-benchmarks, and compiler-optimization traps; cautions that deployment conditions may differ.
Examples
Compares slice conversion, atomic stores, and a population-count function using ns/op outputs and corrected benchmark patterns.
What it assumes
Assumes familiarity with Go functions, loops, and slices while introducing *testing.B and b.N from a minimal example.
What you can do after
Practice/perform Go benchmarks that exclude unintended setup, retain observable results, and interpret small differences cautiously.
How it connects
Connects benchmarks to slice allocation, compiler optimization, timer control, and micro-benchmark statistics.

Also explained in

Compare how 23 books explain benchmark →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of 100 Go Mistakes and How to Avoid Them. The section links point to the parts of the book it draws on.