NormalFloat4 (NF4) stores weights in 4 bits using a fixed, nonuniform codebook matched to approximately normally distributed neural-network weights. Its practical use is QLoRA fine-tuning: a frozen NF4 base model saves memory while trainable adapters learn the task. section 8.1 section 8.4
Unlike FP4, NF4 has no exponent or mantissa. Its 16 lookup values are derived from normal-distribution quantiles, with denser spacing near zero and sparser spacing in the tails. The codebook has seven negative values, zero, and eight positive values. INT4 instead uses uniformly spaced levels; both NF4 and INT4 use 4 bits per weight. section 8.1 section 8.4
{-1.00, -0.70, -0.53, -0.39, -0.28, -0.18, -0.09, 0.00,
0.08, 0.16, 0.25, 0.34, 0.44, 0.56, 0.72, 1.00}The codebook shows the nonuniform spacing and the asymmetric allocation around zero. section 8.4
Scaling granularity matters alongside the codebook. In a reconstruction comparison on OPT-6.7B fc1 layers, NF4 with blocks of 64 averaged 2.64×10⁻⁶ MSE; INT4 with the same group size had 1.44× higher MSE. FP4 with blocks of 16 was close at 2.65×10⁻⁶, but used four times finer blocks. These results measure weight reconstruction, not end-task quality. section 8.4 section 8.6
Figure 8.14 Weight reconstruction MSE across formats on OPT-6.7B fc1 layers (log scale). NF4 block-64 (shortest solid bars) achieves the lowest 4-bit MSE at 2.64×10⁻⁶ average. INT4 group-64 (diagonal hatch, same block size) is 1.44× worse. INT4 group-128 (GPTQ/AWQ default) is 1.73× worse. FP4 block-16 (backslash hatch) matches NF4 at 2.65×10⁻⁶ but uses 4× finer blocks. FP8 E4M3 per-tensor (vertical hatch, shortest bars) remains 12× better than any 4-bit format.

Compare reconstruction error across formats and scaling granularities. section 8.4
NF4 has no dedicated hardware acceleration. In the described QLoRA path, bitsandbytes CUDA kernels dequantize weights before BF16 matrix multiplication. Choose NF4 for fine-tuning memory pressure rather than assuming a native 4-bit throughput gain. section 8.6
Llama-2-7B QLoRA
Configure bitsandbytes with 4-bit loading, quantization type "nf4", BF16 compute, and double quantization; prepare the model for k-bit training and attach LoRA adapters. On H100, the NF4 base occupied 3.60 GB, and the measured training peak was 12.31 GB with 8-bit optimizer states, gradients, and activations. section 8.4
Listing 8.1 Loading Llama-2-7B in NF4 with QLoRA adapters
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4", #A
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True, #B
)
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-2-7b-hf",
quantization_config=bnb_config,
device_map="auto",
)
lora_config = LoraConfig(
r=16, lora_alpha=32,
target_modules=["q_proj", "k_proj", "v_proj", #C
"o_proj", "gate_proj",
"up_proj", "down_proj"],
lora_dropout=0.05, bias="none",
task_type="CAUSAL_LM",
)
model = get_peft_model(
prepare_model_for_kbit_training(model), lora_config
)
The configuration specifies NF4 loading, BF16 compute, double quantization, and adapter targets. section 8.4





