liveBook · Manning

ONNX

Also called Open Neural Network Exchange

ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.

How 9 Manning books explain it

Domain-Specific Small Language Models

A serialized machine-learning model represented in the ONNX format for portable execution outside the training framework.

How it is used
Enables model inference across frameworks and hardware platforms without tying deployment to the training framework.
How deep it goes
Defines the format through typed tensors and operators, then works through graph construction, export, validation, and CPU/GPU execution.
Examples
Uses a MatMul–Add linear-regression graph and exported BERT and GPT-2 models to illustrate construction and inference.
What you can do after
Build it from scratch with ONNX helpers, or apply it via PyTorch export and ONNX Runtime to validate and run models.

Read how Domain-Specific Small Language Models explains ONNX →

Server-Side WebAssembly

ONNX is an open model representation and interoperability standard for exchanging machine-learning models between frameworks and runtimes.

How it is used
Enables pretrained image-classification models to load into a wasi-nn component for inference.
How deep it goes
Explains ONNX through a worked graph-loading and tensor-inference implementation, not model training or format internals.
Examples
Uses SqueezeNet 1.1, a 224×224 cat image, labels, tensor preprocessing, softmax ranking, and Persian-cat output.
What you can do after
Apply it via wasi-nn to load an ONNX image model, submit a tensor, and read ranked classifications.

Read how Server-Side WebAssembly explains ONNX →

LLMs in Production

ONNX is an open model format and ecosystem for representing neural-network models across different frameworks, tools, and hardware environments.

How it is used
Bridges model development frameworks and hardware-specific compilation during deployment.
How deep it goes
Explains ONNX as an intermediate representation and motivates its conversion architecture, but stops short of representation internals.
Examples
Uses an Optimum CLI command to export WizardCoder to ONNX with O1 optimization.
What you can do after
Apply it via Optimum to export a model into an optimized ONNX deployment artifact.

Read how LLMs in Production explains ONNX →

Fight Fraud with Machine Learning

ONNX is an open standard for representing machine-learning models to support interoperability between frameworks and runtimes.

How it is used
Enables trained deep-learning models to move across framework ecosystems and into optimized inference deployment.
How deep it goes
Defines ONNX through conversion and runtime use, without detailing its file structure or conversion implementation.
Examples
Uses PyTorch-to-TensorFlow conversion and ONNX-session text detection on receipt images.
What you can do after
Apply it via ONNX sessions or conversion workflows to reuse models and run inference across supported environments.

Read how Fight Fraud with Machine Learning explains ONNX →

Quantization and Fast Inference

ONNX is an open standard for representing machine-learning models as portable computational graphs across frameworks and runtimes.

How it is used
Bridges framework-specific model training and deployment by providing a portable graph format for runtimes outside the training framework.
How deep it goes
Defines directed acyclic tensor graphs, standardized opsets, and Protocol Buffers serialization, while distinguishing the format from execution engines.
Examples
Compares ResNet-18 and BERT quantization, including MatMul-only versus broader operator coverage and resulting compression.
What you can do after
Apply it by exporting a PyTorch ResNet-18 to opset 17, then producing a statically quantized ONNX artifact with ONNX Runtime.

Read how Quantization and Fast Inference explains ONNX →

Machine Learning System Design

ONNX is an open format and interoperability standard for representing machine learning models across frameworks and runtimes.

How it is used
Supports model portability between training frameworks and production inference systems.
How deep it goes
Explains conversion, runtime support, optimization tools, and deployment tradeoffs without detailing the graph specification.
Examples
Uses paired text and image encoders, static graph exports, inference parameters, metrics reports, and S3-stored artifacts.
What you can do after
Operate cross-framework inference through ONNX Runtime or hardware-specific backends, while weighing flexibility against peak performance.

Read how Machine Learning System Design explains ONNX →

Deep Learning with Python, Third Edition

An open format and runtime ecosystem for representing and executing trained machine-learning models outside the original training framework.

How it is used
Deploys trained models for remote inference when serving outside Python is useful.
How deep it goes
Shows export and runtime execution, but defers detailed optimization and implementation internals.
Examples
Uses Keras export, ONNX Runtime session loading, and prediction execution as the concrete workflow.
What you can do after
Operate exported models through ONNX Runtime for production prediction.

Read how Deep Learning with Python, Third Edition explains ONNX →

Deep Learning with R, Third Edition

ONNX is an open format and ecosystem for representing and exchanging machine-learning models across frameworks and runtimes.

How it is used
Supports deploying trained models for prediction when applications need serving outside R or Python.
How deep it goes
Explains the export-and-runtime workflow but stops short of detailing ONNX internals or optimization techniques.
Examples
Uses Keras export_savedmodel(), onnxruntime InferenceSession, and prediction calls as the working deployment example.
What you can do after
Operate/configure ONNX serving by exporting a Keras model and running predictions through onnxruntime.

Read how Deep Learning with R, Third Edition explains ONNX →

Deep Learning with PyTorch, Second Edition

A vendor-neutral standard and file format for representing neural network computations so models can be executed in different runtimes and devices.

How it is used
Supports moving PyTorch models into compatible runtimes, applications, cloud services, embedded devices, and specialized hardware.
How deep it goes
Explains operation serialization, export tracing, dummy inputs, and runtime execution, but stops short of detailed ONNX internals.
Examples
Uses a simple exported model, a random tensor with shape one-by-ten, an ONNX file, and an ONNX Runtime prediction.
What you can do after
Apply it via torch.onnx.export and ONNX Runtime to export a model and execute it outside PyTorch.

Read how Deep Learning with PyTorch, Second Edition explains ONNX →

Also covered in

What to know first

Related concepts

Questions about ONNX

What is ONNX?

ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.

Which Manning books explain ONNX?

Domain-Specific Small Language Models, Server-Side WebAssembly, LLMs in Production, Fight Fraud with Machine Learning, Quantization and Fast Inference, Machine Learning System Design, Deep Learning with Python, Third Edition, Deep Learning with R, Third Edition and 1 more.

What should I know before learning ONNX?

Manning books that teach ONNX usually introduce language model, computation graph, convolutional layer and CPU first.

What is ONNX a kind of?

ONNX is a kind of serialization format. A serialization format defines how structured data is encoded for storage or transmission and later reconstructed.

What is ONNX also called?

ONNX is also called Open Neural Network Exchange.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.