liveBook · Manning

Phone tier

As explained in Quantization and Fast Inference

The phone tier is the deployment class comprising smartphones with integrated processing hardware and mobile-device constraints.

The phone tier is a deployment choice determined by the target surface, not the model. A phone SoC combines a CPU, GPU, and integrated NPU; its runtime and portability options differ from SBC and small-box deployments. section 11.6 section 11.4

Phone deployments use TFLite and LiteRT-LM or Core ML and MLX. Runtime choice matters: on Apple silicon, the compute-unit hint can outweigh weight-variant choice, and the runtime may silently override it. Measure end-to-end latency, since preprocessing and postprocessing can exceed inference time. section 11.6

Phone-tier devices

The Pixel 10 Pro and iPhone 16 are phone-tier examples. For a Pixel audio pipeline, a Kotlin log-mel STFT can take longer than the Whisper-tiny encoder it feeds; a NEON-tuned FFT can reduce more latency than an inference-side change. section 11.4 section 11.6

How Quantization and Fast Inference covers it

How deep it goes
Defines phone hardware and runtime choices, then contrasts deployment tiers without detailing phone-specific conversion steps.
Examples
Uses the Pixel 10 Pro, iPhone 16, and a Pixel Whisper-tiny pipeline with log-mel STFT preprocessing.
What you can do after
Operate/configure phone deployments by selecting a platform runtime and reporting end-to-end p95 latency.
How it connects
Connects phone deployment to TFLite, LiteRT-LM, Core ML, MLX, end-to-end latency, and preprocessing.

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.