liveBook · Manning

NormalFloat4

Also called nf4

NormalFloat4 (NF4) is a nonuniform four-bit numerical format with representable levels derived from a normal distribution, designed to represent approximately normally distributed values.

How 6 Manning books explain it

Quantization and Fast Inference

NF4 is a 4-bit NormalFloat format that provides nonuniform representable levels suited to normally distributed neural-network weights.

How it is used
Uses NF4 to reduce frozen base-model memory during QLoRA fine-tuning on consumer GPUs.
How deep it goes
Explains codebook construction, blockwise scaling, reconstruction error, hardware constraints, and a concrete adapter configuration.
Examples
Compares OPT-6.7B weight reconstruction errors and configures Llama-2-7B QLoRA with a measured training-memory breakdown.
What you can do after
Apply NF4 via bitsandbytes and LoRA to configure a memory-efficient fine-tuning workflow.

Read how Quantization and Fast Inference explains NormalFloat4 →

Rearchitecting LLMs

4-bit NormalFloat quantization represents model weights with a four-bit NormalFloat numerical encoding tailored to the distribution of language-model weights.

How it is used
Reduces pretrained-model weight memory during fine-tuning and can improve inference throughput when weight traffic limits decoding.
How deep it goes
Explains level placement, compares uniform quantization, and distinguishes storage from computation without detailing the quantization algorithm.
Examples
Uses weight-distribution comparisons and T4/L4 inference results to illustrate quantization distortion and hardware-dependent throughput.
What you can do after
Operate/configure it by setting bnb_4bit_quant_type="nf4" for QLoRA fine-tuning.

Read how Rearchitecting LLMs explains NormalFloat4 →

Building Reliable AI Systems

NF4 is a four-bit quantization data type with quantization bins distributed according to a normal distribution.

How it is used
Uses NF4 to reduce weight memory in QLoRA fine-tuning on limited hardware.
How deep it goes
Explains bin distribution, mixed-precision compute, and configuration without specifying numerical bin boundaries.
Examples
Configures NF4 for a Llama 2 7B customer-support assistant.
What you can do after
Apply it via BitsAndBytesConfig to load and prepare a model for k-bit training.

Read how Building Reliable AI Systems explains NormalFloat4 →

Building Specialized AI Systems

NF4 is a four-bit numerical data type designed to represent neural-network weights whose values approximately follow a Gaussian distribution.

How it is used
Reduces memory requirements when fine-tuning large models with QLoRA while retaining trainable adapter precision.
How deep it goes
Explains Gaussian-aware quantile placement, memory trade-offs, and configuration, but defers numerical encoding details.
Examples
Uses a BitsAndBytesConfig for NF4 loading, double quantization, and bfloat16 computation.
What you can do after
Apply it via BitsAndBytesConfig when loading a model for four-bit QLoRA training.

Read how Building Specialized AI Systems explains NormalFloat4 →

Transformers in Action

NF4 is a four-bit numerical data type optimized for representing neural-network weights with an approximately normal distribution.

How it is used
Supports memory-efficient QLoRA fine-tuning by storing normally distributed model weights in a four-bit format.
How deep it goes
Explains NF4’s distributional rationale, quantile quantization, double quantization, and storage-computation type separation.
Examples
Uses Falcon 7B loading with bitsandbytes and a configuration enabling NF4, double quantization, and bfloat16 computation.
What you can do after
Operate/configure four-bit model loading with bitsandbytes using NF4 storage, double quantization, and bfloat16 computation.

Read how Transformers in Action explains NormalFloat4 →

LLM Customization and Fine-Tuning

A four-bit quantization format designed for neural-network weights that are approximately normally distributed.

How it is used
Uses NF4 to quantize base-model weights for QLoRA fine-tuning under GPU memory constraints.
How deep it goes
Defines NF4’s distribution-specific precision advantage but does not derive its quantization scheme.
Examples
Configures NF4 with bitsandbytes in a 4-bit model-loading example.
What you can do after
Configure NF4 for 4-bit base-model loading in a QLoRA training pipeline.

Read how LLM Customization and Fine-Tuning explains NormalFloat4 →

Also covered in

What to know first

Related concepts

Questions about NormalFloat4

What is NormalFloat4?

NormalFloat4 (NF4) is a nonuniform four-bit numerical format with representable levels derived from a normal distribution, designed to represent approximately normally distributed values.

Which Manning books explain NormalFloat4?

Quantization and Fast Inference, Rearchitecting LLMs, Building Reliable AI Systems, Building Specialized AI Systems, Transformers in Action and LLM Customization and Fine-Tuning.

What should I know before learning NormalFloat4?

Manning books that teach NormalFloat4 usually introduce foundation model, low-rank adaptation, bfloat16 and fine-tuning first.

What is the difference between NormalFloat4 and uniform quantization?

NormalFloat4 (NF4) is a nonuniform four-bit numerical format with representable levels derived from a normal distribution, designed to represent approximately normally distributed values. Uniform quantization maps numerical values to discrete representable levels spaced at equal intervals across a range.

What is NormalFloat4 also called?

NormalFloat4 is also called nf4.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.