liveBook · Manning

Conversion pipeline

As explained in Quantization and Fast Inference

A conversion pipeline is a sequence of processing steps that transforms a model into a format or implementation suitable for a target runtime or accelerator.

A conversion pipeline turns a model into a version suited to a target runtime or accelerator. For small-box deployments, build and validate a separate pipeline for each vendor: calibration can produce different activation ranges and shift accuracy even when vendors use the same model and representative data. section 11.4

Account for these deployment costs:

  • Maintain vendor-specific conversion scripts and runtime binaries. section 11.4
  • Repeat INT8 calibration for each vendor compilation; calibration ranges can differ and accuracy can shift by 1–3 percentage points. section 11.4
  • Check accelerator operator support; unsupported models may need an SBC CPU-only path. section 11.4

One model, four targets

Deploying the same EfficientNet to Jetson, OpenVINO, Coral, and Hailo requires four conversion scripts and four runtime binaries, with calibration discipline repeated for each vendor. section 11.4

How Quantization and Fast Inference covers it

How deep it goes
Explains per-vendor INT8 calibration drift and accuracy effects, but does not detail conversion scripts.
Examples
Uses EfficientNet deployments across Jetson, OpenVINO, Coral, and Hailo to illustrate separate pipelines and calibration.
What it assumes
Assumes a developer-supplied representative dataset for vendor calibration and requires checking whether the model fits the accelerator operator set.
What you can do after
Build it from scratch by creating and validating a conversion pipeline for each vendor target.
How it connects
Connects model conversion to INT8 calibration, representative datasets, runtime selection, and accelerator operator support.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.