A deployment surface—the target device and operating environment—sets which inference tier and runtime are viable. Choose the tier from the hardware and workload, not from the model alone. section 11.6
Match the deployment surface to the available execution path:
- Phones use platform paths such as TFLite or LiteRT-LM, Core ML, and MLX. section 11.6
- Small Linux boards commonly rely on LiteRT with XNNPACK on the ARM CPU; Coral or Hailo accelerators add their own operator-support limits. section 11.4
- Fixed, throughput-bound installations can use a small-box tier and vendor SDK, trading flexibility for throughput and accepting calibration drift. section 11.6
An attached accelerator does not guarantee that the whole graph runs on it: unsupported operators can force CPU fallback and reduce throughput. Measure end-to-end latency, since preprocessing and postprocessing can outweigh inference. section 11.4 section 11.6
