liveBook · Manning

Trajectory

As explained in LLMs in Production

Model tracing is the capture of operations invoked during a model execution to construct an intermediate computational representation.

Tracing captures the operations invoked during a model run and turns them into an intermediate representation, such as TorchScript, that can support later graph optimization and compilation. section 6.1.1

Trace a model input

A GPT-2 example uses torch.jit.trace with tokenized text. For a model accepting images and text, tracing with text alone can prune image operations from the resulting representation. section 6.1.1

Figure 6.2 An example of an unoptimized network compared to the same network optimized using graph optimization. CBR is a NVIDIA fused layer kernel that simply stands for Convolution, Bias, and ReLU. See the following NVIDIA blog post for reference: https://mng.bz/PNvw.
figure

Tracing can make later graph optimization easier; this figure contrasts an unoptimized network with an optimized one. section 6.1.1

How LLMs in Production covers it

How deep it goes
Defines tracing and demonstrates torch.jit.trace on a tokenized GPT-2 input.
Examples
Uses tokenized GPT-2 input and a CLIP example where text-only tracing prunes image operations.
What you can do after
Apply tracing with torch.jit.trace to produce a TorchScript representation for TensorRT compilation.
How it connects
Links tracing to TorchScript, graph optimization, TensorRT compilation, and inference.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 19 books explain trajectory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.