ONNX

As explained in Server-Side WebAssembly

ONNX is an open model representation and interoperability standard for exchanging machine-learning models between frameworks and runtimes.

ONNX, or Open Neural Network Exchange, is an open format for representing machine-learning models. Its practical value is portability: a model can move between frameworks and tools instead of being tied to the environment in which it was prepared. section 5.2.1

For inference, an ONNX model is loaded as a graph, given tensor input, and asked to compute output. Image data becomes a tensor whose dimensions represent batch, color channels, width, and height; the resulting scores can be converted to probabilities and matched with classification labels. section 5.2.1 section 5.2.2

Run an ONNX image classifier through wasi-nn

Load `squeezenet1.1-7.onnx` with `GraphEncoding::Onnx` and a CPU execution target. Resize and normalize a 224×224 image into a floating-point tensor, run `compute()`, then apply softmax and print the three highest-probability labels. In the demonstrated run, a cat image is classified as a Persian cat with probability 0.9488756. section 5.2.2

Figure 5.2 wasi_nn_example application schematic

Follow the application schematic to see how image input, model inference, and classification output fit together. section 5.2.1

ONNX is one of three experimental wasi-nn backends supported by Wasmtime at the stated point in time, alongside OpenVINO and WinML. The component uses generated bindings for the wasi-nn interface; obtaining that interface is manual because it is not yet packaged as an OCI artifact. section 5.2.2

How Server-Side WebAssembly covers it

How deep it goes

Explains ONNX through a worked graph-loading and tensor-inference implementation, not model training or format internals.

Examples

Uses SqueezeNet 1.1, a 224×224 cat image, labels, tensor preprocessing, softmax ranking, and Persian-cat output.

What it assumes

Introduces machine-learning models, tensors, labels, and softmax from zero before implementing inference.

What you can do after

Apply it via wasi-nn to load an ONNX image model, submit a tensor, and read ranked classifications.

How it connects

Connects ONNX to machine-learning models, tensors, image classification, softmax, wasi-nn, Wasmtime, OpenVINO, and WinML.

liveBookmanning.com

Read Server-Side WebAssembly on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Domain-Specific Small Language ModelsA serialized machine-learning model represented in the ONNX format for portable execution outside the training framework.
• 

  liveBookmanning.com
  LLMs in ProductionONNX is an open model format and ecosystem for representing neural-network models across different frameworks, tools, and hardware environments.
• 

  liveBookmanning.com
  Machine Learning System DesignONNX is an open format and interoperability standard for representing machine learning models across frameworks and runtimes.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionAn open format and runtime ecosystem for representing and executing trained machine-learning models outside the original training framework.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionONNX is an open format and ecosystem for representing and exchanging machine-learning models across frameworks and runtimes.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA vendor-neutral standard and file format for representing neural network computations so models can be executed in different runtimes and devices.

Compare how 7 books explain ONNX →

Related concepts

• ONNX Runtimeused by · ONNX Runtime is an open-source, cross-platform inference engine for executing and optimizing machine-learning models represented in the ONNX format.
• model exportproduced by · Model export is the process of converting a trained machine-learning model into a portable representation suitable for another format, framework, runtime, or deployment environment.
• TensorFlow Servingalternative to · TensorFlow Serving is an open-source system for deploying trained machine-learning models and exposing them through production inference APIs.
• artifact packagingused by · Artifact packaging is the preparation of a trained model and its associated files and requirements as a deployable unit.
• Optimumproduced by · Optimum is a Hugging Face library for optimizing and deploying machine-learning models across inference engines and hardware platforms.
• language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.

This explainer was generated by Manning's knowledge graph from the text of Server-Side WebAssembly. The section links point to the parts of the book it draws on.