How deeply does LLMs in Production cover ONNX?

Explains ONNX as an intermediate representation and motivates its conversion architecture, but stops short of representation internals.

ONNX

As explained in LLMs in Production

ONNX is an open model format and ecosystem for representing neural-network models across different frameworks, tools, and hardware environments.

ONNX acts as a practical middle layer between model frameworks and hardware-specific deployment. Instead of writing conversions for every framework–device pairing, developers export to an intermediate representation that vendors can compile for GPUs, TPUs, CPUs, and other hardware. section 6.1.1

The main production trade-off is simpler interoperability rather than eliminating deployment work:

Frameworks such as PyTorch and TensorFlow export models to ONNX or other intermediate representations. section 6.1.1
Hardware vendors then convert the representation into machine code optimized for their devices. section 6.1.1
Export and compilation still require compatible tooling and expertise in both model architecture and hardware architecture. section 6.1.1

Exporting a model

A command-line workflow can export a Hugging Face model to ONNX and request optimization: `optimum-cli export onnx --model WizardLM/WizardCoder-1B-V1.0 ./models_onnx --optimize O1`.

How LLMs in Production covers it

liveBook manning.com

Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Domain-Specific Small Language ModelsA serialized machine-learning model represented in the ONNX format for portable execution outside the training framework.
liveBook manning.com

Server-Side WebAssemblyONNX is an open model representation and interoperability standard for exchanging machine-learning models between frameworks and runtimes.
liveBook manning.com

Machine Learning System DesignONNX is an open format and interoperability standard for representing machine learning models across frameworks and runtimes.
liveBook manning.com

Deep Learning with Python, Third EditionAn open format and runtime ecosystem for representing and executing trained machine-learning models outside the original training framework.
liveBook manning.com

Deep Learning with R, Third EditionONNX is an open format and ecosystem for representing and exchanging machine-learning models across frameworks and runtimes.
liveBook manning.com

Deep Learning with PyTorch, Second EditionA vendor-neutral standard and file format for representing neural network computations so models can be executed in different runtimes and devices.

Compare how 7 books explain ONNX →

Related concepts

ONNX Runtimeused by · ONNX Runtime is an open-source, cross-platform inference engine for executing and optimizing machine-learning models represented in the ONNX format.
model exportproduced by · Model export is the process of converting a trained machine-learning model into a portable representation suitable for another format, framework, runtime, or deployment environment.
TensorFlow Servingalternative to · TensorFlow Serving is an open-source system for deploying trained machine-learning models and exposing them through production inference APIs.
artifact packagingused by · Artifact packaging is the preparation of a trained model and its associated files and requirements as a deployable unit.
Optimumproduced by · Optimum is a Hugging Face library for optimizing and deploying machine-learning models across inference engines and hardware platforms.
language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.