liveBook · Manning

.litertlm bundle

As explained in Quantization and Fast Inference

A LiteRT-LM bundle is a model artifact in the FlatBuffers family that packages a transformer model with runtime metadata.

A .litertlm bundle packages a transformer model with metadata needed by LiteRT-LM. It belongs to the FlatBuffers family and serves as the unit of dispatch. section 11.2.3

The bundle includes three runtime metadata groups:

  • Tokenizer configuration: vocabulary, special-token IDs, and normalization rules. section 11.2.3
  • KV-cache layout: per-layer key/value dimensions, maximum sequence length, and cache-management policy. section 11.2.3
  • A default sampling configuration. section 11.2.3

Unlike a .tflite file, which represents an op graph dispatched through an Interpreter, a .litertlm bundle is dispatched through LiteRT-LM. The app can stream token deltas through sendMessageAsync as a Kotlin Flow. section 11.2.3

How Quantization and Fast Inference covers it

How deep it goes
Specifies tokenizer vocabulary and rules, per-layer KV-cache dimensions and bounds, and a default sampling configuration.
Examples
Uses Llama-3.2-1B-Instruct as an Android deployment example and distinguishes .litertlm bundles from .tflite and older .task artifacts.
How it connects
Links tokenization and key-value caching to LiteRT-LM's transformer runtime through required bundle metadata.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.