NormalFloat4
NormalFloat4 (NF4) is a nonuniform four-bit numerical format with representable levels derived from a normal distribution, designed to represent approximately normally distributed values.
Quantization and Fast Inference
NF4 is a 4-bit NormalFloat format that provides nonuniform representable levels suited to normally distributed neural-network weights.
- How it is used
- Uses NF4 to reduce frozen base-model memory during QLoRA fine-tuning on consumer GPUs.
- How deep it goes
- Explains codebook construction, blockwise scaling, reconstruction error, hardware constraints, and a concrete adapter configuration.
- Examples
- Compares OPT-6.7B weight reconstruction errors and configures Llama-2-7B QLoRA with a measured training-memory breakdown.
- What you can do after
- Apply NF4 via bitsandbytes and LoRA to configure a memory-efficient fine-tuning workflow.
Read how Quantization and Fast Inference explains NormalFloat4 →
Rearchitecting LLMs
4-bit NormalFloat quantization represents model weights with a four-bit NormalFloat numerical encoding tailored to the distribution of language-model weights.
- How it is used
- Reduces pretrained-model weight memory during fine-tuning and can improve inference throughput when weight traffic limits decoding.
- How deep it goes
- Explains level placement, compares uniform quantization, and distinguishes storage from computation without detailing the quantization algorithm.
- Examples
- Uses weight-distribution comparisons and T4/L4 inference results to illustrate quantization distortion and hardware-dependent throughput.
- What you can do after
- Operate/configure it by setting bnb_4bit_quant_type="nf4" for QLoRA fine-tuning.
Building Reliable AI Systems
NF4 is a four-bit quantization data type with quantization bins distributed according to a normal distribution.
- How it is used
- Uses NF4 to reduce weight memory in QLoRA fine-tuning on limited hardware.
- How deep it goes
- Explains bin distribution, mixed-precision compute, and configuration without specifying numerical bin boundaries.
- Examples
- Configures NF4 for a Llama 2 7B customer-support assistant.
- What you can do after
- Apply it via BitsAndBytesConfig to load and prepare a model for k-bit training.
Read how Building Reliable AI Systems explains NormalFloat4 →
Building Specialized AI Systems
NF4 is a four-bit numerical data type designed to represent neural-network weights whose values approximately follow a Gaussian distribution.
- How it is used
- Reduces memory requirements when fine-tuning large models with QLoRA while retaining trainable adapter precision.
- How deep it goes
- Explains Gaussian-aware quantile placement, memory trade-offs, and configuration, but defers numerical encoding details.
- Examples
- Uses a BitsAndBytesConfig for NF4 loading, double quantization, and bfloat16 computation.
- What you can do after
- Apply it via BitsAndBytesConfig when loading a model for four-bit QLoRA training.
Read how Building Specialized AI Systems explains NormalFloat4 →
Transformers in Action
NF4 is a four-bit numerical data type optimized for representing neural-network weights with an approximately normal distribution.
- How it is used
- Supports memory-efficient QLoRA fine-tuning by storing normally distributed model weights in a four-bit format.
- How deep it goes
- Explains NF4’s distributional rationale, quantile quantization, double quantization, and storage-computation type separation.
- Examples
- Uses Falcon 7B loading with bitsandbytes and a configuration enabling NF4, double quantization, and bfloat16 computation.
- What you can do after
- Operate/configure four-bit model loading with bitsandbytes using NF4 storage, double quantization, and bfloat16 computation.
LLM Customization and Fine-Tuning
A four-bit quantization format designed for neural-network weights that are approximately normally distributed.
- How it is used
- Uses NF4 to quantize base-model weights for QLoRA fine-tuning under GPU memory constraints.
- How deep it goes
- Defines NF4’s distribution-specific precision advantage but does not derive its quantization scheme.
- Examples
- Configures NF4 with bitsandbytes in a 4-bit model-loading example.
- What you can do after
- Configure NF4 for 4-bit base-model loading in a QLoRA training pipeline.
Read how LLM Customization and Fine-Tuning explains NormalFloat4 →
Also covered in
What to know first
- foundation modelA foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
- low-rank adaptationLow-rank adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
- bfloat16bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
Related concepts
- QLoRAused by · QLoRA is a parameter-efficient fine-tuning method that combines a quantized pretrained model with trainable low-rank adapters.
- standard normal distributionuses · A standard normal distribution is a normal probability distribution with mean zero and standard deviation one.
- uniform quantizationalternative to · Uniform quantization maps numerical values to discrete representable levels spaced at equal intervals across a range.
- memory usageaddresses · Memory usage is the amount of computer memory occupied, allocated, or required by a program, system, data structure, or computation during operation.
- normal distributionuses · A normal distribution is a continuous probability distribution with a symmetric, bell-shaped density determined by its mean and variance.
Questions about NormalFloat4
What is NormalFloat4?
NormalFloat4 (NF4) is a nonuniform four-bit numerical format with representable levels derived from a normal distribution, designed to represent approximately normally distributed values.
Which Manning books explain NormalFloat4?
Quantization and Fast Inference, Rearchitecting LLMs, Building Reliable AI Systems, Building Specialized AI Systems, Transformers in Action and LLM Customization and Fine-Tuning.
What should I know before learning NormalFloat4?
Manning books that teach NormalFloat4 usually introduce foundation model, low-rank adaptation, bfloat16 and fine-tuning first.
What is the difference between NormalFloat4 and uniform quantization?
NormalFloat4 (NF4) is a nonuniform four-bit numerical format with representable levels derived from a normal distribution, designed to represent approximately normally distributed values. Uniform quantization maps numerical values to discrete representable levels spaced at equal intervals across a range.
What is NormalFloat4 also called?
NormalFloat4 is also called nf4.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





