liveBook · Manning

Fixed-point representation

As explained in Quantization and Fast Inference

Fixed-point representation encodes numerical values with a fixed scaling position between integer and fractional components.

Fixed-point representation stores integers but interprets them using a fixed scale. It provides a way to represent fractional values on hardware that works with integers, including embedded DSPs and AI accelerators. In neural-network deployment, integer representations can reduce memory, arithmetic cost, and energy, subject to accuracy and hardware constraints. section 2.1 section 2.0

A stored integer represents RealValue = Integer × S. In binary Q-format, S is a power of two. Signed Q3.4 uses one sign bit, three integer bits, and four fractional bits, so S = 2⁻⁴ = 1/16. Adjacent integer codes therefore represent values 1/16 apart. section 2.1

Encoding 1.5 in Q3.4

Divide 1.5 by 1/16 to obtain the stored integer 24, or 0001 1000 in binary. Reading the bits around the implied binary point gives 1 + 1/2 = 1.5. section 2.1

Figure 2.1 The Q3.4 fixed-point format divides 8 bits into a sign bit, 3 integer bits, and 4 fractional bits. Here, the bit pattern 0 001 1000 encodes +1.5 — the integer bits 001 contribute 1, and the leading fractional bit 1000 contributes 2-¹ = 0.5.

The bit layout shows the sign, integer, and fractional fields for the 1.5 example. section 2.1

Signed Q-format uses two's complement for the stored integer. In eight bits, 1111 1111 has signed integer value −1, not 255. Applying the Q3.4 scale gives −1 × 1/16 = −0.0625, the representable negative value closest to zero. section 2.1

The bit allocation determines the range–resolution tradeoff. With a fixed eight-bit container, assigning more bits to the integer part increases range but makes the spacing between values coarser; assigning more fractional bits does the reverse. Signed Q3.4 represents integer codes −128 through 127, corresponding to −8 through 7.9375. section 2.1

Figure 2.3 The range-versus-precision tradeoff in Q-format. For a fixed 8-bit container, every bit allocated to the integer part doubles the representable range but halves the precision (step size). Moving the binary point left (more fractional bits) gives finer resolution at the cost of a narrower range; moving it right does the reverse. There is no free lunch: the total information capacity is fixed by the bit-width.

The figure compares range and resolution as the binary point moves within an eight-bit container. section 2.1

Arithmetic and saturation

In Q3.4, 1.5 and 0.5 have raw integers 24 and 8; their product is 0.75, with raw integer 12. A saturating implementation clamps 10.0 to the maximum 7.9375. It likewise clamps 5.0 + 5.0 to 7.9375 rather than representing the exact sum. section 2.1

Choose the fractional-bit allocation for the values and operations the target hardware must handle: finer steps improve resolution but leave less range before saturation. Fixed-point scaling also provides a starting point for affine quantization, which uses scale and zero-point parameters to map neural-network values onto integer levels. section 2.1 section 2.0

How Quantization and Fast Inference covers it

How deep it goes
Explains Q-format scaling, signed decoding, range and precision, and arithmetic with a hardware-behavior simulation.
Examples
Works through Q3.4 encodings of 1.5 and −0.0625, plus multiplication and saturation in a FixedPoint simulation.
What you can do after
Evaluate range, resolution, and saturation in a fixed-point format, and understand their implications for quantization accuracy.
How it connects
Connects fixed-point representation to two's complement, quantization error, and affine quantization with scale and zero point.

Also explained in

Compare how 2 books explain fixed-point representation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.