liveBook · Manning

Python extension module

As explained in Refactoring to Rust

A Python extension module is an importable Python module implemented partly or entirely in compiled native code.

A Python extension module is an importable module compiled against Python’s C/C++ libraries rather than written in Python. Rust can provide the implementation while preserving a normal Python module API, with substantially faster execution than ordinary Python modules. section 6.4

A minimal Rust extension-module workflow is:

  • Add PyO3 with its extension-module feature to Cargo.toml. section 6.4
  • Convert the executable crate into a library crate by replacing main.rs with lib.rs and removing main. section 6.4
  • Declare the module with #[pymodule], expose functions with #[pyfunction], and register them using wrap_pyfunction!. section 6.4
  • Build and install the module into an activated virtual environment with maturin, then import it from Python. section 1.9 section 6.4

Expose a Rust function

A Rust function can deserialize JSON, calculate a result, and become callable from Python after PyO3 registration: rust_json.sum('{ "name": "Rachelle Ferguson", "value": 948129 }') returns 948146. A Python program can then process JSON Lines through the extension and accumulate the results. section 6.4

[package]
name = "rust_json"
version = "0.1.0"
edition = "2018"

[lib]  #1
crate-type = ["cdylib"]

[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
pyo3 = { version = "0.14", features = ["extension-module"] }  #2

Use the Cargo configuration to inspect the library crate type, serde dependencies, and PyO3 extension-module feature. section 6.4

How Refactoring to Rust covers it

How deep it goes
Explains definition, crate conversion, PyO3 registration, installation, and import, but focuses on functions rather than classes or global variables.
Examples
Uses a rust_json module, a JSON-deserializing sum function, interactive imports, and JSON Lines processing from standard input.
What it assumes
Assumes a Rust Cargo project and requires Python 3, virtualenv, pip, and suitable development tooling for the integration workflow.
What you can do after
Apply it via PyO3 and maturin to build, install, import, and call a Rust-backed Python module.
How it connects
Connects Python modules with Rust libraries, PyO3, Cargo, maturin, virtualenv, and Python’s C/C++ libraries.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Refactoring to Rust. The section links point to the parts of the book it draws on.