Intermediate representation
An intermediate representation is a structured internal form of a program, model, or information used between processing stages for analysis, transformation, or generation.
LLMs in Production
An intermediate representation is a machine-readable form of a program or model that enables translation and optimization between source frameworks and target hardware.
- How it is used
- Uses an IR to bridge model development frameworks and hardware-specific compilation for deployment efficiency.
- How deep it goes
- Explains the conversion rationale and demonstrates tracing and compilation, while deferring detailed kernel-tuning implementation.
- Examples
- Uses GPT-2 tracing, TensorRT compilation settings, ONNX export, and an optimized-versus-unoptimized network comparison.
- What you can do after
- Apply it via PyTorch, Torch-TensorRT, or Optimum to export and compile models for selected hardware.
Read how LLMs in Production explains intermediate representation →
Rearchitecting LLMs
An intermediate representation is a vector-valued activation produced inside a neural model between its input and output layers.
- How it is used
- Aligns teacher and student hidden states during distillation, adding layer-level signals alongside task and output-logit losses.
- How deep it goes
- Explains mapped-layer cosine alignment and extends it to changes between consecutive layers.
- Examples
- Compares student layer 5 with teacher layer 9, then aligns changes across mapped consecutive layers.
- What you can do after
- Build it from scratch by computing mapped hidden-state cosine loss and consecutive-layer delta alignment.
Read how Rearchitecting LLMs explains intermediate representation →
GPU Programming with Triton
An intermediate representation is a structured compiler artifact that expresses a program between its source form and its final executable form.
- How it is used
- Use compiler IRs to trace how a Triton kernel changes through compilation, especially when investigating performance or surprising behavior.
- How deep it goes
- Trace Triton IR, Triton GPU IR, LLVM IR, PTX, and binary outputs, but stop short of detailing every representation.
- Examples
- Uses a compiled Triton kernel’s `asm` attribute to retrieve intermediate forms.
- What you can do after
- Diagnose kernel behavior or performance by retrieving compiled stages through Triton’s `asm` attribute.
Read how GPU Programming with Triton explains intermediate representation →
Also covered in
- Machines that ThinkIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- PythonPython is a high-level, general-purpose, dynamically typed programming language with readable syntax and support for multiple programming paradigms.
Related concepts
- bytecodeincludes · Bytecode is an intermediate instruction representation produced from source code for execution or further compilation by a virtual machine or language runtime.
- code compilationproduced by · Code compilation is the transformation of source code or another formal representation into executable code or an intermediate or lower-level representation for execution or further processing.
- pseudocodeincludes · Pseudocode is informal, language-independent notation that expresses an algorithm’s logic, structure, and steps without requiring executable syntax.
- ONNXalternative to · ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.
- Tritonproduced by · Triton is a programming language and compiler for writing and compiling high-performance GPU kernels.
- feature alignmentused by · Feature alignment is a distillation technique that matches intermediate representations produced by corresponding blocks of teacher and student models.
Questions about intermediate representation
What is intermediate representation?
An intermediate representation is a structured internal form of a program, model, or information used between processing stages for analysis, transformation, or generation.
Which Manning books explain intermediate representation?
LLMs in Production, Rearchitecting LLMs and GPU Programming with Triton.
What should I know before learning intermediate representation?
Manning books that teach intermediate representation usually introduce language model and Python first.
What is the difference between intermediate representation and ONNX?
An intermediate representation is a structured internal form of a program, model, or information used between processing stages for analysis, transformation, or generation. ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.
What kinds of intermediate representation are there?
Kinds of intermediate representation include bytecode and pseudocode.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


