INT4 stores weights using four-bit integers, reducing memory use and latency at the cost of a measured quality drop. Symmetric INT4 spaces its positive values uniformly using a scale factor. section 8.3.1 section 12.4
Decide whether INT4 fits the model and deployment:
- Assess architecture, model size, target bit-width, and acceptable accuracy loss when judging whether post-training quantization is sufficient. section 4.1
- Validate the deployed runtime: compare only formats actually measured, rather than treating configured alternatives as tested results. section 12.3
INT4 and FP4 level spacing
From 0 to 2, INT4 has 3 non-negative values while FP4 has 5. Across 0 to 6, both have 8; FP4 concentrates more levels near zero. section 8.3.1
Figure 8.7 Representable values in [0, 6]. FP4 E2M1 and INT4 symmetric each have 8 non-negative values (including zero), but FP4 clusters them toward the origin while INT4 spaces them uniformly. FP8 E4M3 (bottom row) packs 78 values into the same interval—roughly 10× the resolution of either 4-bit format.

Compare how symmetric INT4 spaces values uniformly while FP4 clusters them toward zero. section 8.3.1


