A benchmark harness runs comparable cases repeatedly and records measurements so alternative implementations can be evaluated fairly. For a Python-to-Rust migration, it can compare pure Python, pure Rust, and Rust called through Python while keeping surrounding I/O unchanged. section 6.2 section 6.5
A practical harness typically:
- Uses Criterion to warm up benchmarks, collect samples, and report timing ranges rather than relying on a single measurement. section 6.5
- Places benchmark code in a separate crate when the main crate is a PyO3 extension module and linking restrictions apply. section 6.5
- Keeps Criterion and benchmark-only PyO3 configuration in dev-dependencies, preventing them from entering final libraries or executables. section 6.5
- Uses black_box around measured work so the benchmark exercises the intended computation. section 6.5
Comparing Python with a Rust extension
A benchmark can prepare one JSON input, import both Python's json module and the rust_json extension, and execute each implementation through Python under Criterion. A sample run reports pure Python at 25.415–25.842 microseconds and the Rust extension at 21.746–22.314 microseconds. The comparison preserves the same input and surrounding call path, but the result remains meaningful only when the cases and measurement setup are equivalent. section 6.5
Listing 6.14 Benchmarking pure Python vs. a Rust extension module
use criterion::{
black_box, criterion_group, criterion_main, Bencher, Criterion,
};
use pyo3::prelude::*;
use pyo3::types::PyDict;
criterion_main!(python_vs_rust);
criterion_group!(python_vs_rust, bench_fn);
fn bench_py(b: &mut Bencher, code: &str, input: &str) {
Python::with_gil(|py| {
let locals = PyDict::new(py);
locals.set_item("json", py.import("json").unwrap()).unwrap();
locals
.set_item("rust_json", py.import("rust_json").unwrap())
.unwrap();
locals.set_item("INPUT", input).unwrap();
b.iter(|| black_box(py.run(code, None, Some(&locals)).unwrap()));
});
}
fn bench_fn(c: &mut Criterion) {
let input = r#"{"name": "lily", "value": 42}"#;
c.bench_function("pure python", |b| {
bench_py(
b,
"
value = json.loads(INPUT)
s = value['value'] + len(value['name'])
",
input,
);
});
c.bench_function("rust extension library", |b| {
bench_py(b, "s = rust_json.sum(INPUT)", input);
});
}
Inspect the complete Criterion and PyO3 example for benchmarking pure Python against a Rust extension module. section 6.5
