6 Integrating with dynamic languages
This chapter covers
- Writing Rust code that can be easily called from Python
- Calling Python code from Rust
- Benchmarking Rust code with Criterion
So far, we have devoted a lot of time to Rust fundamentals and C FFI. This chapter will more directly cover how we can integrate Rust code into dynamic programming languages and reap huge performance benefits from it.
6.1 Data processing in Python
Let’s imagine we are working on a Python program that aggregates some newline-separated JSON data. Here is our input data file; let’s call it data.jsonl
:
{ "name": "Stokes Baker", "value": 954832 } { "name": "Joseph Solomon", "value": 279836 } { "name": "Gonzalez Koch", "value": 140431 } { "name": "Parrish Waters", "value": 490411 } { "name": "Sharlene Nunez", "value": 889667 } { "name": "Meadows David", "value": 892040 } { "name": "Whitley Mendoza", "value": 965462 } { "name": "Santiago Hood", "value": 280041 } { "name": "Carver Caldwell", "value": 632926 } { "name": "Tara Patterson", "value": 678175 }
Our program calculates the total sum of each of the "value"
entries and the sum of the length of all of the "name"
strings. This process is relatively straightforward in normal Python code. Let’s save this in a file called main.py
.
Listing 6.1 Python program to aggregate JSON lines
import sys import json s = 0 for line in sys.stdin: value = json.loads(line) s += value['value'] s += len(value['name']) print(s)
Let’s run it to see what we get: