liveBook · Manning

Confidence interval

As explained in Refactoring to Rust

A confidence interval is a statistical range that expresses uncertainty around an estimated quantity at a specified confidence level.

A confidence interval expresses uncertainty around an estimated benchmark time. Criterion reports a configurable interval, defaulting to 95%, with lower and upper bounds around its best estimate. section 6.5

Reading Criterion output

For the u8 benchmark, Criterion reports time: [257.13 ps 261.71 ps 266.79 ps]. The first value is the lower bound, the last is the upper bound, and the middle value is Criterion’s best estimate. section 6.5

Figure 6.4 Anatomy of Criterion’s command-line output
figure

Use the command-line output anatomy to identify the confidence interval’s bounds and central estimate in benchmark results. section 6.5

How Refactoring to Rust covers it

How deep it goes
Defines interval bounds and the central estimate, but does not explain statistical derivation.
Examples
Uses Criterion’s u8 and u128 timing lines as concrete interval examples.
What you can do after
Evaluate benchmark precision by reading the lower bound, estimate, and upper bound together.
How it connects
Connects confidence intervals with benchmarking, Criterion, execution time, and performance measurement.

Also explained in

Compare how 10 books explain confidence interval →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Refactoring to Rust. The section links point to the parts of the book it draws on.