liveBook · Manning

Normal distribution

As explained in Quantization and Fast Inference

A normal distribution is a continuous probability distribution that is symmetric about its mean and has a bell-shaped density.

A normal distribution is symmetric around its mean and has a bell-shaped density. NF4 uses its quantiles to place quantization levels more densely where neural-network weights tend to cluster. section 8.4

Figure 8.12 NF4, INT4 uniform, and FP4 E2M1 quantization levels in [-1, 1]. NF4 (top row) clusters its levels near the center, matching the normal distribution's density. INT4 (middle) spaces levels uniformly. FP4 (bottom) clusters near zero via its exponential spacing. NF4 and FP4 each place 7 levels in [-0.3, 0.3]; INT4 places only 4.

Compare how NF4, uniform INT4, and FP4 place quantization levels near zero. section 8.4

How Quantization and Fast Inference covers it

How deep it goes
Explains normal quantiles, a 16-level lookup table, and its asymmetric split into negative levels, zero, and positive levels.
Examples
Compares NF4's near-zero levels with uniform INT4 and FP4, and gives a concrete 16-value NF4 codebook.
What you can do after
Understand why NF4 concentrates quantization levels near zero to reduce error for weights that cluster there.
How it connects
Links normal quantiles to NF4 quantization, neural-network weights, and QLoRA fine-tuning.

Also explained in

Compare how 19 books explain normal distribution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.