liveBook · Manning

FP4

FP4 is a family of four-bit floating-point formats for representing numerical values at reduced precision.

How 2 Manning books explain it

Quantization and Fast Inference

A family of 4-bit floating-point formats for reduced-precision numerical representation.

How it is used
Compresses model weights for low-memory inference while trading numerical resolution against storage and hardware throughput.
How deep it goes
Defines E2M1 encoding and compares scaling configurations with measured model quality and memory use.
Examples
Compares per-tensor, block-16, and block-32 FP4 inference on OPT-6.7B using WikiText-2 perplexity and estimated model size.
What you can do after
Apply blockwise scaling and match FP4 deployment to GPU support to balance model quality, storage, and throughput.

Read how Quantization and Fast Inference explains FP4 →

CUDA for LLMs

FP4 is a four-bit floating-point numerical format used for low-precision representation and computation.

How it is used
Reduces precision for deep-learning computation, especially inference, to raise tensor-core throughput and ease memory-bandwidth pressure.
How deep it goes
Defines FP4 fields and block scaling, then sketches CUTLASS use; defers detailed implementation and calibration complexities.
Examples
Uses MXFP4 and NVFP4 block scales, a CUTLASS GEMM configuration, and an 8192³ NVFP4 benchmark.
What you can do after
Apply FP4 through CUTLASS to configure and benchmark a Blackwell SM100 GEMM.

Read how CUDA for LLMs explains FP4 →

What to know first

Related concepts

Questions about FP4

What is FP4?

FP4 is a family of four-bit floating-point formats for representing numerical values at reduced precision.

Which Manning books explain FP4?

Quantization and Fast Inference and CUDA for LLMs.

What should I know before learning FP4?

Manning books that teach FP4 usually introduce bfloat16, latency, floating-point operations per second and throughput first.

What is the difference between FP4 and int4 quantization?

FP4 is a family of four-bit floating-point formats for representing numerical values at reduced precision. INT4 quantization represents numerical values using 4-bit integers, reducing storage requirements while limiting numerical precision.

What is FP4 a kind of?

FP4 is a kind of low-bit floating-point format. A low-bit floating-point format is a floating-point representation that uses substantially fewer bits than conventional formats such as FP32.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.