liveBook · Manning

Benchmark

As explained in Latency

A benchmark is a repeatable performance measurement procedure applied to a defined workload or operation.

A benchmark is a repeatable performance measurement of a defined workload or operation. A microbenchmark focuses on a small operation, such as calculating a Fibonacci number or accessing a lock. Comparing implementations on the same workload helps quantify an optimization rather than relying on complexity alone. section 7.7 section 8.2.1

A practical measurement workflow:

  • Define the operation and workload, including input size and concurrency. Establish a baseline before changing the implementation. section 7.7 section 9.6
  • Run repeated measurements with a tool such as Criterion. Examine the timing range, outliers, and any warnings about sample collection. section 7.7 section 9.6
  • Change the implementation, rerun the benchmark, and compare results for the defined workload. If more explanation is needed, use CPU profiling and a flame graph to see where time is spent. section 7.7

Eliminating Fibonacci work

For fib_recursive(10), one Criterion run measured approximately 158 ns. A memoized version measured approximately 44 ns, and an iterative version approximately 5 ns. These results compare implementations for a specific input; the recursive algorithm’s exponential complexity also matters as the input grows. section 7.7

Listing 7.2 Fibonacci microbenchmark
use criterion::{criterion_group, criterion_main, Criterion};
use fibonacci::{fib_iterative, fib_memoization, fib_recursive};

fn bench(c: &mut Criterion) {
    let mut group = c.benchmark_group("fibonacci");
    group.bench_function("recursive", |b| {
        b.iter(|| {
            let _ = fib_recursive(10);
        });
    });
}
criterion_group!(benches, bench);
criterion_main!(benches);

The Criterion benchmark group repeatedly measures the recursive Fibonacci call; run it with cargo bench. section 7.7

Controlled concurrency

A simulated workload waits for 10 ms per task. For ten tasks, one run measured about 130 ms sequentially and 13 ms concurrently. The controlled delay makes the comparison reproducible, but it does not include the variability of actual I/O or CPU workloads. section 9.6

Workload conditions can dominate the result. In a shared-counter benchmark, mutex lock and unlock time increased from roughly 7 ns with one thread to 5.3 µs with 100 threads, illustrating the cost of contention. Report the thread count along with the timing rather than treating a single measurement as universal. section 8.2.1

How Latency covers it

How deep it goes
Walks through benchmark setup, repeated timing, comparison, and CPU profiling, with sampling warnings and outliers.
Examples
Measures Fibonacci implementations, sequential versus concurrent delayed tasks, and shared-counter locks at different thread counts.
What you can do after
Apply it via Criterion to measure and compare small Rust workloads, and use CPU profiling to investigate runtime.
How it connects
Connects benchmarking to algorithmic complexity, memoization, concurrency, lock contention, and CPU profiling.

Also explained in

Compare how 24 books explain benchmark →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Latency. The section links point to the parts of the book it draws on.