ONNX
ONNX is an open standard for representing machine-learning models in a portable format that supports exchange and execution across frameworks and runtimes.
Domain-Specific Small Language Models
A serialized machine-learning model represented in the ONNX format for portable execution outside the training framework.
- How it is used
- Positions ONNX as a portability layer for exporting, optimizing, quantizing, and deploying models across frameworks and hardware.
- How deep it goes
- Defines graphs, operators, tensors, serialization, runtimes, validation, optimization, and IOBinding, then applies them to transformer deployment.
- Examples
- Demonstrates linear regression graphs, BERT question answering, GPT-2 generation, CUDA execution, IOBinding, and 8-bit classification.
- What you can do after
- Build and operate portable model graphs with ONNX Runtime, validate outputs, select CPU or CUDA execution, optimize graphs, and quantize models.
Read how Domain-Specific Small Language Models explains ONNX →
Server-Side WebAssembly
ONNX is an open model representation and interoperability standard for exchanging machine-learning models between frameworks and runtimes.
- How it is used
- Enables pretrained image-classification models to load into a wasi-nn component for inference.
- How deep it goes
- Explains ONNX through a worked graph-loading and tensor-inference implementation, not model training or format internals.
- Examples
- Uses SqueezeNet 1.1, a 224×224 cat image, labels, tensor preprocessing, softmax ranking, and Persian-cat output.
- What you can do after
- Apply it via wasi-nn to load an ONNX image model, submit a tensor, and read ranked classifications.
LLMs in Production
ONNX is an open model format and ecosystem for representing neural-network models across different frameworks, tools, and hardware environments.
- How it is used
- Bridges model development frameworks and hardware-specific compilation during deployment.
- How deep it goes
- Explains ONNX as an intermediate representation and motivates its conversion architecture, but stops short of representation internals.
- Examples
- Uses an Optimum CLI command to export WizardCoder to ONNX with O1 optimization.
- What you can do after
- Apply it via Optimum to export a model into an optimized ONNX deployment artifact.
Machine Learning System Design
ONNX is an open format and interoperability standard for representing machine learning models across frameworks and runtimes.
- How it is used
- Supports model portability between training frameworks and production inference systems.
- How deep it goes
- Explains conversion, runtime support, optimization tools, and deployment tradeoffs without detailing the graph specification.
- Examples
- Uses paired text and image encoders, static graph exports, inference parameters, metrics reports, and S3-stored artifacts.
- What you can do after
- Operate cross-framework inference through ONNX Runtime or hardware-specific backends, while weighing flexibility against peak performance.
Deep Learning with Python, Third Edition
An open format and runtime ecosystem for representing and executing trained machine-learning models outside the original training framework.
- How it is used
- Deploys trained models for remote inference when serving outside Python is useful.
- How deep it goes
- Shows export and runtime execution, but defers detailed optimization and implementation internals.
- Examples
- Uses Keras export, ONNX Runtime session loading, and prediction execution as the concrete workflow.
- What you can do after
- Operate exported models through ONNX Runtime for production prediction.
Read how Deep Learning with Python, Third Edition explains ONNX →
Deep Learning with R, Third Edition
ONNX is an open format and ecosystem for representing and exchanging machine-learning models across frameworks and runtimes.
- How it is used
- Supports deploying trained models for prediction when applications need serving outside R or Python.
- How deep it goes
- Explains the export-and-runtime workflow but stops short of detailing ONNX internals or optimization techniques.
- Examples
- Uses Keras export_savedmodel(), onnxruntime InferenceSession, and prediction calls as the working deployment example.
- What you can do after
- Operate/configure ONNX serving by exporting a Keras model and running predictions through onnxruntime.
Read how Deep Learning with R, Third Edition explains ONNX →
Deep Learning with PyTorch, Second Edition
A vendor-neutral standard and file format for representing neural network computations so models can be executed in different runtimes and devices.
- How it is used
- Supports moving PyTorch models into compatible runtimes, applications, cloud services, embedded devices, and specialized hardware.
- How deep it goes
- Explains operation serialization, export tracing, dummy inputs, and runtime execution, but stops short of detailed ONNX internals.
- Examples
- Uses a simple exported model, a random tensor with shape one-by-ten, an ONNX file, and an ONNX Runtime prediction.
- What you can do after
- Apply it via torch.onnx.export and ONNX Runtime to export a model and execute it outside PyTorch.
Read how Deep Learning with PyTorch, Second Edition explains ONNX →
Related concepts
- ONNX Runtimeused by · ONNX Runtime is an open-source, cross-platform inference engine for executing and optimizing machine-learning models represented in the ONNX format.
- model exportproduced by · Model export is the process of converting a trained machine-learning model into a portable representation suitable for another format, framework, runtime, or deployment environment.
- TensorFlow Servingalternative to · TensorFlow Serving is an open-source system for deploying trained machine-learning models and exposing them through production inference APIs.
- artifact packagingused by · Artifact packaging is the preparation of a trained model and its associated files and requirements as a deployable unit.
- Optimumproduced by · Optimum is a Hugging Face library for optimizing and deploying machine-learning models across inference engines and hardware platforms.
- language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.