Model export

As explained in Machine Learning System Design

Model conversion is the transformation of a trained model into a representation or format supported by another execution framework or deployment environment.

Model export converts a trained model into a representation another inference framework or deployment environment can run. It separates research flexibility from production requirements: a model may be trained in PyTorch, then exported for iOS, backend, server, or mobile inference. section 15.3.1 section 15.6

Treat export as a deployment and verification step:

• Choose an intermediate or target format according to latency, throughput, scalability, cost, reliability, flexibility, security, privacy, and platform constraints. section 15.3.1 section 15.6
• Use ONNX as a common interframework language when training and inference environments differ; ONNX also supplies runtime and conversion tools. section 15.3.1
• Test sensitive conversion paths by comparing predictions from the converted model with predictions from the original model. section 10.5
• Accept that a general format can balance flexibility and performance without matching a custom solution for one model, hardware target, and usage pattern. section 15.3.1

One model, multiple targets

A model trained with PyTorch can be converted for iOS inference with CoreML and backend inference with ONNX. The conversion test checks that both converted models preserve the original results. section 10.5

Figure 15.3 A wide spectrum of tools with their focus on either research or production serving goals

Use the tool spectrum to compare research-oriented frameworks with production-serving engines and deployment formats. section 15.3.1

How Machine Learning System Design covers it

How deep it goes

Explains formats, runtimes, hardware backends, and output validation, but does not detail conversion implementation.

Examples

Uses PyTorch-to-CoreML iOS conversion, PyTorch-to-ONNX backend conversion, and ONNX Runtime, OpenVINO, TensorRT, and mobile targets.

What it assumes

Assumes a trained model and target inference environment; teaches the research-flexibility versus production-performance tradeoff.

What you can do after

Apply it by converting models across frameworks and checking converted outputs against the original model.

How it connects

Connects model conversion with inference optimization, model serving, ONNX Runtime, and hardware-specific inference.

liveBookmanning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Related concepts

• ONNXproduces · ONNX is an open standard for representing machine-learning models in a portable format that supports exchange and execution across frameworks and runtimes.
• inference optimizationused by · Inference optimization comprises methods that improve a trained model’s serving efficiency by reducing latency, computational cost, or resource use.
• frameworkuses · A software framework is a reusable structure that provides components, conventions, services, or extension points for building applications or other software systems.
• pipelinecontrasts with · A pipeline is an ordered or coordinated sequence of connected processing stages or tasks that passes data, work, or results toward an outcome.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.