liveBook · Manning

AOTInductor

As explained in Deep Learning with PyTorch, Second Edition

PyTorch's ahead-of-time compiler that turns exported graphs into optimized executable code.

AOTInductor (AOTI) is PyTorch’s ahead-of-time compiler for running inference from an exported computation graph. It compiles the graph into optimized executable code and packages the result for later loading. section 17.2.2

A typical deployment workflow is:

  • Export a PyTorch model with torch.export, producing a graph whose parameters become inputs and whose layers become lower-level torch.ops.aten operations. section 17.2.2
  • Compile and package the exported program with torch._inductor.aoti_compile_and_package. section 17.2.2
  • Load the package with torch._inductor.aoti_load_package and call the resulting model for inference. section 17.2.2

Compile and reload an exported model

output_path = torch._inductor.aoti_compile_and_package(exported_program, package_path="model.pt2") model = torch._inductor.aoti_load_package("model.pt2") model(dummy_input) The compiled model is optimized for the input shapes used during export, so later inputs must match those shapes. section 17.2.2

How Deep Learning with PyTorch, Second Edition covers it

How deep it goes
Explains export, graph lowering, packaging, loading, and fixed-shape execution, but does not detail compiler internals.
Examples
Uses a linear-ReLU model, an exported graph listing, and compile-load code for a model.pt2 package.
What it assumes
Assumes a PyTorch nn.Module and an exported program produced with torch.export.
What you can do after
Apply it via torch._inductor to compile, package, load, and execute an exported PyTorch inference model.
How it connects
Connects PyTorch models with torch.export, exported programs, ATen operations, and compiled-model packaging.
Cover of Deep Learning with PyTorch, Second Edition
liveBookmanning.com
Read Deep Learning with PyTorch, Second Edition on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with PyTorch, Second Edition. The section links point to the parts of the book it draws on.