liveBook · Manning

Benchmark

As explained in Latency

A benchmark is a repeatable performance measurement procedure applied to a defined workload or operation.

A benchmark repeatedly measures a defined operation or workload so that performance choices can be compared. Keep the operation and inputs explicit: timing a Fibonacci calculation answers a different question from timing ten tasks or a contended lock. section 7.7 section 9.6 section 8.2.1

A practical measurement loop is:

  • Establish a baseline. In Criterion, put the operation inside a benchmark function and run `cargo bench`. Criterion repeats it enough times to build statistical confidence; inspect the reported time range and outliers. section 7.7
  • Change one implementation while retaining the workload. For Fibonacci at n = 10, the reported times are about 157.71 ns recursively, 44.208 ns with memoization, and 5.0690 ns iteratively. section 7.7
  • Investigate the result rather than stopping at the number. A Criterion benchmark can use CPU profiling to produce a flame graph showing time spent in each function. section 7.7
Listing 7.2 Fibonacci microbenchmark
use criterion::{criterion_group, criterion_main, Criterion};
use fibonacci::{fib_iterative, fib_memoization, fib_recursive};

fn bench(c: &mut Criterion) {
    let mut group = c.benchmark_group("fibonacci");
    group.bench_function("recursive", |b| {
        b.iter(|| {
            let _ = fib_recursive(10);
        });
    });
}
criterion_group!(benches, bench);
criterion_main!(benches);

The Criterion setup names a benchmark group and times a specific call, `fib_recursive(10)`, inside `b.iter`. section 7.7

Concurrency under a controlled delay

With ten independent tasks each awaiting a simulated 10 ms delay, sequential execution measured about 129.82 ms and concurrent execution about 12.924 ms. The artificial delay makes the comparison easier to reproduce and reason about, but does not reproduce the variability of actual I/O or CPU workloads. The benchmark also warned that its target time was too short for 100 samples. section 9.6

Workload shape can reverse an easy conclusion. Mutex lock-and-unlock measurements rise from about 7.4 ns with one thread to 5.28 µs with 100 threads updating a shared counter. A queue comparison likewise measures a particular 128-element producer–consumer workload, not every possible queue use. Report the operation, input size, and concurrency level alongside any result. section 8.2.1 section 8.6

How Latency covers it

How deep it goes
Shows Criterion setup, repeated measurements, reported ranges, and profiling; warns through controlled workloads that measured results depend on what is timed.
Examples
Measures Fibonacci implementations, delayed sequential and concurrent tasks, contended locks, and producer–consumer queues.
What it assumes
Assumes enough Rust to read functions and benchmark closures; specialized examples also use asynchronous tasks or shared-state synchronization.
What you can do after
Apply it via Criterion and `cargo bench` to compare defined Rust operations and inspect their measured runtime differences.
How it connects
Connects benchmarks to algorithmic complexity, memoization, concurrency, mutex contention, and CPU profiling.

Also explained in

Compare how 23 books explain benchmark →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Latency. The section links point to the parts of the book it draws on.