PyO3 connects Rust and Python without requiring a wholesale rewrite. A Python program can keep its input loop and call a Rust function for a performance-sensitive operation. PyO3 also works in the other direction: Rust can run Python code through bindings to the interpreter. section 6.2 section 6.4
Move JSON parsing behind a Python function
Suppose Python reads one JSON record per line and adds each record’s result to a total. Keep that loop in Python. In Rust, write a function that parses a record containing a name and numeric value, then returns the value plus the name’s length. Mark the function with #[pyfunction]; create a #[pymodule] function and register it with m.add_function(wrap_pyfunction!(sum, m)?)?. Python can then import the module and call rust_json.sum(line). This example uses serde_json::from_str(input).unwrap(), so it does not demonstrate error handling for invalid JSON. section 6.2 section 6.4
Listing 6.10 rust_json extension module that works
use pyo3::prelude::*;
#[derive(Debug, serde::Deserialize)]
struct Data {
name: String,
value: i32,
}
#[pyfunction] #1
fn sum(input: &str) -> i32 {
let parsed: Data = serde_json::from_str(input).unwrap();
parsed.name.len() as i32 + parsed.value
}
#[pymodule]
fn rust_json(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(sum, m)?)?; #2
Ok(())
}
The complete extension-module listing puts the Rust calculation beside the two PyO3 annotations and the function-registration call. section 6.4
To make that function importable from Python:
- Configure a Rust library crate as a cdylib and enable PyO3’s extension-module feature in Cargo.toml. section 6.4
- Install maturin, activate a Python virtual environment, and run maturin develop to compile and install the module for local use. section 6.4
- Import the module from Python and call its registered function. Compiling Rust code alone does not make the module available to Python. section 6.4
Measure the boundary rather than assuming Rust will make every call dramatically faster. A Criterion benchmark uses PyO3 to run both Python’s JSON calculation and a call to the Rust extension. In the shown run, their reported central times were 25.623 µs and 21.987 µs, respectively. For benchmarking Python from Rust, enable PyO3’s auto-initialize feature instead of extension-module; the benchmark lives in a separate crate because of extension-module linking restrictions. section 6.5
PyO3 can also help when Python threads are limited by the global interpreter lock (GIL). A Rust function exposed to Python can call py.allow_threads around its calculation, releasing the GIL so other threads can run. The Mandelbrot example submits that function through a Python ThreadPoolExecutor; the GIL release is an explicit change, not a consequence of merely writing the function in Rust. section 8.6
Listing 8.11 lib.rs: Adding a thread-safe function
#[pyfunction]
fn mandelbrot_fast(
py: Python<'_>,
size: u32,
path: &str,
range_x0: f64,
range_y0: f64,
range_x1: f64,
range_y1: f64,
) {
py.allow_threads(|| mandelbrot_func(size, path, range_x0, range_y0,
range_x1, range_y1)) #1
}
#[pymodule]
fn mandelbrot(_py: Python, m: &PyModule) -> PyResult<()> {
...
m.add_function(wrap_pyfunction!(mandelbrot_fast, m)?)?;
Ok(())
}
The thread-safe wrapper shows where py.allow_threads surrounds the Rust calculation and how the new function is registered with the Python module. section 8.6
