Dequantization
Dequantization maps values from a quantized numerical representation to a higher-precision representation, typically using quantization parameters.
CUDA for LLMs
Dequantization is converting quantized integer representations back into a higher-precision numerical representation.
- How it is used
- Reconstructs approximate floating-point values from compressed model weights and activations for computation.
- How deep it goes
- Works through symmetric, asymmetric, and packed integer CUDA kernels, plus NormalFloat4 lookup-based reconstruction.
- Examples
- Uses four INT8 weights, a skewed activation range, packed INT4 values, and NormalFloat4 blocks.
- What you can do after
- Build dequantization kernels for symmetric INT8, asymmetric UINT8, and packed INT4 representations.
Build a DeepSeek Model (From Scratch)
Dequantization is the transformation of quantized values back into a higher-precision numerical representation using quantization scales.
- How it is used
- Restores low-precision matrix results before they enter the next neural-network layer.
- How deep it goes
- Explains scaling-factor multiplication, Tensor Core handoff, FP32 accumulation, and BF16 storage without implementation code.
- Examples
- Uses independently scaled activation chunks, weight blocks, and periodically promoted matrix-multiplication partial sums.
- What you can do after
- Operate an FP8 computation path that restores output magnitude while retaining high-precision accumulation.
Read how Build a DeepSeek Model (From Scratch) explains dequantization →
What to know first
- bfloat16bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
- matrix-matrix multiplicationMatrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
- mean squared errorMean squared error is the arithmetic mean of the squared differences between corresponding predicted and reference numerical values.
- model weightA model weight is a learned numerical parameter that influences a machine-learning model’s computations and behavior.
Related concepts
- quantization scaleuses · A quantization scale is a multiplicative factor relating quantized values or steps to the numerical values they represent.
- bitsandbytesimplemented by · bitsandbytes is an open-source library providing memory-efficient optimizers and low-precision operations, including quantization, for machine-learning models.
- gpu coreuses · A CUDA core is a relatively simple arithmetic and logic execution unit in an NVIDIA GPU that processes instructions for individual threads.
- dynamic quantizationused by · Dynamic quantization is a method that determines or adjusts quantization parameters at runtime from the values being processed, often quantizing activations during inference.
Questions about dequantization
What is dequantization?
Dequantization maps values from a quantized numerical representation to a higher-precision representation, typically using quantization parameters.
Which Manning books explain dequantization?
CUDA for LLMs and Build a DeepSeek Model (From Scratch).
What should I know before learning dequantization?
Manning books that teach dequantization usually introduce bfloat16, matrix-matrix multiplication, mean squared error and model weight first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

