ExecuTorch is a PyTorch ecosystem library for running models efficiently on constrained devices, including mobile phones and embedded systems. It addresses devices that cannot support PyTorch’s full operation set by providing a runtime adapted to their limits. section 17.6
The deployment workflow follows three main steps:
- Capture the model’s computational graph with torch.export. section 17.6
- Convert the exported program into ExecuTorch’s Edge IR format for efficient device execution. section 17.6
- Package the converted program, load it in a mobile application, and execute inference with the ExecuTorch runtime. section 17.6
Exporting and running a model
A typical export uses export(model, example_input), converts the result with to_edge, and produces a serialized ExecuTorch program with to_executorch(). The saved package can then be loaded with _load_for_executorch(model_path) and invoked through executorch_module.forward([input_data]). section 17.6
Listing 17.13 Model export to ExecuTorch
import torch
from torch.export import export
from executorch.exir import to_edge
exported_program = export(model, example_input) #1
edge_program = to_edge(exported_program) #2
executorch_program = edge_program.to_executorch() #3
with open(output_path, "wb") as f:
f.write(executorch_program.buffer)
return executorch_program
Shows the complete conversion path from an exported PyTorch graph to a serialized ExecuTorch package. section 17.6
Listing 17.14 Running inference with an ExecuTorch model
from executorch.extension.pybindings.portable_lib import _load_for_executorch executorch_module = _load_for_executorch(model_path) outputs = executorch_module.forward([input_data])
Shows how to load a serialized package and invoke inference through the portable ExecuTorch runtime. section 17.6
