ONNX
ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.
Domain-Specific Small Language Models
A serialized machine-learning model represented in the ONNX format for portable execution outside the training framework.
- How it is used
- Enables model inference across frameworks and hardware platforms without tying deployment to the training framework.
- How deep it goes
- Defines the format through typed tensors and operators, then works through graph construction, export, validation, and CPU/GPU execution.
- Examples
- Uses a MatMul–Add linear-regression graph and exported BERT and GPT-2 models to illustrate construction and inference.
- What you can do after
- Build it from scratch with ONNX helpers, or apply it via PyTorch export and ONNX Runtime to validate and run models.
Read how Domain-Specific Small Language Models explains ONNX →
Server-Side WebAssembly
ONNX is an open model representation and interoperability standard for exchanging machine-learning models between frameworks and runtimes.
- How it is used
- Enables pretrained image-classification models to load into a wasi-nn component for inference.
- How deep it goes
- Explains ONNX through a worked graph-loading and tensor-inference implementation, not model training or format internals.
- Examples
- Uses SqueezeNet 1.1, a 224×224 cat image, labels, tensor preprocessing, softmax ranking, and Persian-cat output.
- What you can do after
- Apply it via wasi-nn to load an ONNX image model, submit a tensor, and read ranked classifications.
LLMs in Production
ONNX is an open model format and ecosystem for representing neural-network models across different frameworks, tools, and hardware environments.
- How it is used
- Bridges model development frameworks and hardware-specific compilation during deployment.
- How deep it goes
- Explains ONNX as an intermediate representation and motivates its conversion architecture, but stops short of representation internals.
- Examples
- Uses an Optimum CLI command to export WizardCoder to ONNX with O1 optimization.
- What you can do after
- Apply it via Optimum to export a model into an optimized ONNX deployment artifact.
Fight Fraud with Machine Learning
ONNX is an open standard for representing machine-learning models to support interoperability between frameworks and runtimes.
- How it is used
- Enables trained deep-learning models to move across framework ecosystems and into optimized inference deployment.
- How deep it goes
- Defines ONNX through conversion and runtime use, without detailing its file structure or conversion implementation.
- Examples
- Uses PyTorch-to-TensorFlow conversion and ONNX-session text detection on receipt images.
- What you can do after
- Apply it via ONNX sessions or conversion workflows to reuse models and run inference across supported environments.
Quantization and Fast Inference
ONNX is an open standard for representing machine-learning models as portable computational graphs across frameworks and runtimes.
- How it is used
- Bridges framework-specific model training and deployment by providing a portable graph format for runtimes outside the training framework.
- How deep it goes
- Defines directed acyclic tensor graphs, standardized opsets, and Protocol Buffers serialization, while distinguishing the format from execution engines.
- Examples
- Compares ResNet-18 and BERT quantization, including MatMul-only versus broader operator coverage and resulting compression.
- What you can do after
- Apply it by exporting a PyTorch ResNet-18 to opset 17, then producing a statically quantized ONNX artifact with ONNX Runtime.
Machine Learning System Design
ONNX is an open format and interoperability standard for representing machine learning models across frameworks and runtimes.
- How it is used
- Supports model portability between training frameworks and production inference systems.
- How deep it goes
- Explains conversion, runtime support, optimization tools, and deployment tradeoffs without detailing the graph specification.
- Examples
- Uses paired text and image encoders, static graph exports, inference parameters, metrics reports, and S3-stored artifacts.
- What you can do after
- Operate cross-framework inference through ONNX Runtime or hardware-specific backends, while weighing flexibility against peak performance.
Deep Learning with Python, Third Edition
An open format and runtime ecosystem for representing and executing trained machine-learning models outside the original training framework.
- How it is used
- Deploys trained models for remote inference when serving outside Python is useful.
- How deep it goes
- Shows export and runtime execution, but defers detailed optimization and implementation internals.
- Examples
- Uses Keras export, ONNX Runtime session loading, and prediction execution as the concrete workflow.
- What you can do after
- Operate exported models through ONNX Runtime for production prediction.
Read how Deep Learning with Python, Third Edition explains ONNX →
Deep Learning with R, Third Edition
ONNX is an open format and ecosystem for representing and exchanging machine-learning models across frameworks and runtimes.
- How it is used
- Supports deploying trained models for prediction when applications need serving outside R or Python.
- How deep it goes
- Explains the export-and-runtime workflow but stops short of detailing ONNX internals or optimization techniques.
- Examples
- Uses Keras export_savedmodel(), onnxruntime InferenceSession, and prediction calls as the working deployment example.
- What you can do after
- Operate/configure ONNX serving by exporting a Keras model and running predictions through onnxruntime.
Read how Deep Learning with R, Third Edition explains ONNX →
Deep Learning with PyTorch, Second Edition
A vendor-neutral standard and file format for representing neural network computations so models can be executed in different runtimes and devices.
- How it is used
- Supports moving PyTorch models into compatible runtimes, applications, cloud services, embedded devices, and specialized hardware.
- How deep it goes
- Explains operation serialization, export tracing, dummy inputs, and runtime execution, but stops short of detailed ONNX internals.
- Examples
- Uses a simple exported model, a random tensor with shape one-by-ten, an ONNX file, and an ONNX Runtime prediction.
- What you can do after
- Apply it via torch.onnx.export and ONNX Runtime to export a model and execute it outside PyTorch.
Read how Deep Learning with PyTorch, Second Edition explains ONNX →
Also covered in
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- computation graphA computation graph is a directed graph whose nodes represent computational operations or values and whose edges represent dependencies between them.
- convolutional layerA convolutional layer is a neural-network layer that applies shared learned filters to local regions of structured input to produce feature representations.
- CPUA central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
Related concepts
- ONNX Runtimeused by · ONNX Runtime is an open-source, cross-platform runtime for executing and optimizing machine-learning models in the ONNX format.
- TensorRTused by · TensorRT is NVIDIA’s software development kit for optimizing and executing machine-learning inference on NVIDIA hardware.
- torch.onnx.exportproduced by · torch.onnx.export is a PyTorch API that exports a PyTorch model to the ONNX format.
- computation graphuses · A computation graph is a directed graph whose nodes represent computational operations or values and whose edges represent dependencies between them.
- model exportproduced by · Model export is the process of transforming a machine-learning model into a representation suitable for use in another format, framework, runtime, or deployment environment.
Questions about ONNX
What is ONNX?
ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.
Which Manning books explain ONNX?
Domain-Specific Small Language Models, Server-Side WebAssembly, LLMs in Production, Fight Fraud with Machine Learning, Quantization and Fast Inference, Machine Learning System Design, Deep Learning with Python, Third Edition, Deep Learning with R, Third Edition and 1 more.
What should I know before learning ONNX?
Manning books that teach ONNX usually introduce language model, computation graph, convolutional layer and CPU first.
What is ONNX a kind of?
ONNX is a kind of serialization format. A serialization format defines how structured data is encoded for storage or transmission and later reconstructed.
What is ONNX also called?
ONNX is also called Open Neural Network Exchange.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.








