NormalFloat4 (NF4) is a four-bit data type for quantizing neural-network weights that approximately follow a Gaussian distribution. It places quantization levels by equal-sized distribution bins, concentrating levels where weights are densest and reducing information loss from outliers. section 3.5.2
NF4 supports memory-efficient QLoRA training by:
- Storing frozen base-model weights at four-bit precision instead of 16-bit precision. section 3.5.2
- Combining weight quantization with double quantization of scaling factors and zero points. section 3.5.2
- Leaving trainable LoRA adapters at 16-bit precision while the base model remains quantized. section 3.5.2
NF4 configuration
A Transformers and BitsAndBytes configuration enables four-bit loading, NF4 quantization, double quantization, and bfloat16 computation: load_in_4bit=True, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4", and bnb_4bit_compute_dtype=torch.bfloat16.





