liveBook · Manning

Precision

As explained in Domain-Specific Small Language Models

A numeric representation setting that determines how many bits are used to store or compute model values during serving.

Precision sets how many bits represent or compute LLM values during serving. Production work compares precision formats with smaller-precision formats produced through quantization. section 6.0

Treat precision as a production inference decision:

  • Compare full- and smaller-precision formats when optimizing LLM inference performance. section 6.0
  • Consider quantization techniques and libraries as implementation paths rather than treating precision as an isolated numeric choice. section 6.0

How Domain-Specific Small Language Models covers it

How deep it goes
Distinguishes precision formats from smaller-precision quantization and names multiple techniques and libraries without implementation detail.
Examples
Uses precision formats, smaller-precision formats, and quantization techniques and libraries as concrete options.
What it assumes
Assumes familiarity with core ONNX concepts and capabilities before addressing quantization.
What you can do after
Evaluate quantization as a way to address LLM inference performance.
How it connects
Connects precision to LLM inference, quantization, and ONNX.
Cover of Domain-Specific Small Language Models
liveBookmanning.com
Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Also explained in

Compare how 9 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.