6 Integrating with dynamic languages

 

This chapter covers

  • Writing Rust code that can be easily called from Python
  • Calling Python code from Rust
  • Benchmarking Rust code with Criterion

So far, we have devoted a lot of time to Rust fundamentals and C FFI. This chapter will more directly cover how we can integrate Rust code into dynamic programming languages and reap huge performance benefits from it.

6.1 Data processing in Python

Let’s imagine we are working on a Python program that aggregates some newline-separated JSON data. Here is our input data file; let’s call it data.jsonl:

{ "name": "Stokes Baker", "value": 954832 }
{ "name": "Joseph Solomon", "value": 279836 }
{ "name": "Gonzalez Koch", "value": 140431 }
{ "name": "Parrish Waters", "value": 490411 }
{ "name": "Sharlene Nunez", "value": 889667 }
{ "name": "Meadows David", "value": 892040 }
{ "name": "Whitley Mendoza", "value": 965462 }
{ "name": "Santiago Hood", "value": 280041 }
{ "name": "Carver Caldwell", "value": 632926 }
{ "name": "Tara Patterson", "value": 678175 }

Our program calculates the total sum of each of the "value" entries and the sum of the length of all of the "name" strings. This process is relatively straightforward in normal Python code. Let’s save this in a file called main.py.

Listing 6.1 Python program to aggregate JSON lines
import sys
import json

s = 0

for line in sys.stdin:
  value = json.loads(line)
  s += value['value']
  s += len(value['name'])

print(s)

Let’s run it to see what we get:

6.2 Planning the move

6.3 JSON Parsing

6.4 Writing a Python extension module in Rust

6.5 Benchmarking in Rust

6.6 Optimized builds

Summary