The global interpreter lock (GIL) lets only one thread at a time run code through the Python interpreter within a process. It protects interpreter resources from simultaneous access, but it also limits parallel execution of Python work across threads. A thread pool alone therefore may not speed up a CPU-heavy Python task. section 8.5 section 8.6
A lock shared by threads
Think of the GIL as a single hall pass: one thread holds it while using interpreter resources; another must wait its turn. This makes the safety trade-off easier to see. Adding more threads does not add more passes. section 8.5
The lock matters explicitly when Rust interacts with Python. Most Python operations require it to be held. PyO3’s `Python::with_gil` supplies a handle for operations such as creating a `PyDict`, importing modules, and running Python code. A `Python` parameter on a PyO3 extension module likewise indicates that the GIL is held. section 6.5 section 6.4
use criterion::Bencher;
use pyo3::prelude::*;
use pyo3::types::PyDict;
fn bench_py(b: &mut Bencher, code: &str, input: &str) {
Python::with_gil(|py| {
let locals = PyDict::new(py);
locals.set_item("json", py.import("json").unwrap()).unwrap();
locals
.set_item("rust_json", py.import("rust_json").unwrap())
.unwrap();
locals.set_item("INPUT", input).unwrap();
b.iter(|| black_box(py.run(code, None, Some(&locals)).unwrap()));
});
}
The benchmark helper acquires the GIL before creating Python locals and executing Python code. Notice how the `py` handle is passed to `PyDict::new` and used for imports and `py.run`. section 6.5
For CPU-heavy work, a targeted Rust refactor offers a different route from adding Python threads. Keep the Python-facing function and move the computation into Rust; PyO3’s `py.allow_threads` releases the GIL around that work so other threads can run. The work inside the closure must be thread-safe. This is a change to a bottleneck, not a reason to rewrite an entire application. section 8.6
Listing 8.11 lib.rs: Adding a thread-safe function
#[pyfunction]
fn mandelbrot_fast(
py: Python<'_>,
size: u32,
path: &str,
range_x0: f64,
range_y0: f64,
range_x1: f64,
range_y1: f64,
) {
py.allow_threads(|| mandelbrot_func(size, path, range_x0, range_y0,
range_x1, range_y1)) #1
}
#[pymodule]
fn mandelbrot(_py: Python, m: &PyModule) -> PyResult<()> {
...
m.add_function(wrap_pyfunction!(mandelbrot_fast, m)?)?;
Ok(())
}
The Mandelbrot wrapper places the Rust computation inside `py.allow_threads`. Compare that deliberate release with `Python::with_gil`: one permits other threads to run during Rust work, while the other obtains the lock for Python operations. section 8.6 section 6.5
