liveBook · Manning

Mixed precision

Mixed precision is a computational technique that uses different numerical precisions for different values or operations to balance efficiency with accuracy and stability.

How 4 Manning books explain it

CUDA for LLMs

A mixed-precision strategy assigns different numerical precision formats to different parts of a computation or model.

How it is used
Uses lower-precision matrix operations to increase tensor-core throughput while preserving stability through higher-precision accumulation.
How deep it goes
Traces precision choices through tensor-core instructions, WMMA fragments, CUTLASS types, and model layers, but defers full production deployment details.
Examples
Demonstrates FP16 or BF16 inputs with FP32 accumulation in WMMA and CUTLASS GEMM configurations.
What you can do after
Apply mixed precision through CUDA WMMA or CUTLASS configurations and choose layer-specific formats for inference.

Read how CUDA for LLMs explains mixed precision →

Domain-Specific Small Language Models

Using reduced-precision numeric formats for some computations or weights to reduce memory use, often with limited loss in performance.

How it is used
Reduces model memory use and computation costs while balancing numerical stability, performance, and hardware limits.
How deep it goes
Details floating-point formats and a mixed-precision training workflow, then compares inference tradeoffs without specifying implementation steps.
Examples
Compares GPT-Neo inference memory estimates in FP32 and FP16 and describes mixed-precision weight, gradient, and update operations.
What you can do after
Operate within GPU memory limits by choosing FP16 for a pretrained model when needed.

Read how Domain-Specific Small Language Models explains mixed precision →

Deep Learning with R, Third Edition

Mixed-precision training is model training that uses numerical values with different floating-point precisions to balance computational efficiency and numerical accuracy.

How it is used
Speeds model training on GPUs or TPUs while preserving precision-sensitive weight updates.
How deep it goes
Explains dtype roles, unstable operations, loss scaling, and Keras configuration without deriving hardware-level implementation details.
Examples
Uses a mixed_float16 mini-GPT configuration, float32 layer opt-outs, and fixed or automatic optimizer loss scaling.
What you can do after
Operate/configure Keras mixed precision and protect training with float32 exceptions or loss scaling.

Read how Deep Learning with R, Third Edition explains mixed precision →

Deep Learning with Python, Third Edition

A training technique that uses lower-precision numeric types for some computations while keeping others in higher precision to balance speed, memory use, and numerical stability.

How it is used
Uses lower-precision computation to speed GPU or TPU training while preserving full precision for stability-sensitive updates.
How deep it goes
Explains layer dtypes, gradient casting, unstable operations, and loss scaling, but does not derive floating-point behavior mathematically.
Examples
Demonstrates Keras dtype policies, float32 opt-outs, fixed or automatic loss scaling, and a mini-GPT configuration.
What you can do after
Apply mixed precision in Keras, configure loss scaling, and protect numerically unstable layers with float32.

Read how Deep Learning with Python, Third Edition explains mixed precision →

Related concepts

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.