FP4 stores each weight in 4 bits, halving the weight storage of 8-bit formats. Its limited representable values make scaling granularity critical: per-tensor scaling can save memory but severely damage model quality. section 8.3 section 8.6
E2M1 uses one sign bit, two exponent bits, and one mantissa bit with exponent bias 1. It reserves no bit patterns for infinity or NaN; its positive values are 0, 0.5, 1, 1.5, 2, 3, 4, and 6. Unlike uniformly spaced INT4, FP4 places more values near zero. section 8.3.1 section 8.3.1
Choose scaling and hardware together:
- Prefer blockwise scaling over per-tensor scaling. On OPT-6.7B, block-16 FP32 scales brought perplexity to 16.49 from a BF16 baseline of 16.26; per-tensor FP4 reached 26.09. section 8.3.4
- On Blackwell, NVFP4 uses FP8 scales per 16-value block plus a tensor-level FP32 scale; MXFP4 uses power-of-two scales per 32-value block. Earlier GPUs require dequantization before matmul, so FP4 offers no throughput advantage over FP8 there. section 8.3.4
Table 8.2 Complete FP4 E2M1 positive value table
|
Bits
|
Layout (S_EE_M)
|
Exponent field
|
Mantissa field
|
Type
|
Value
|
|
0000
|
0_00_0
|
0
|
0
|
Zero
|
0.0
|
|
0001
|
0_00_1
|
0
|
1
|
Subnormal
|
0.5
|
|
0010
|
0_01_0
|
1
|
0
|
Normal
|
1.0
|
|
0011
|
0_01_1
|
1
|
1
|
Normal
|
1.5
|
|
0100
|
0_10_0
|
2
|
0
|
Normal
|
2.0
|
|
0101
|
0_10_1
|
2
|
1
|
Normal
|
3.0
|
|
0110
|
0_11_0
|
3
|
0
|
Normal
|
4.0
|
|
0111
|
0_11_1
|
3
|
1
|
Normal
|
6.0
|
Inspect the complete E2M1 positive-value table, including bit patterns and normal versus subnormal values. section 8.3.1

