Float16
float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
Quantization and Fast Inference
A 16-bit floating-point format with 5 exponent bits and 10 fraction bits.
- How it is used
- Uses FP16 as a precision choice for training and as a baseline for evaluating quantized inference.
- How deep it goes
- Explains exponent and fraction tradeoffs, numerical range, and relative energy costs without detailing FP16 encoding or implementation.
- Examples
- Illustrates floating-point rounding with a four-fraction-bit toy format and compares FP16 with AWQ four-bit serving on an L4.
- What you can do after
- Enables evaluation of FP16 precision, range, and energy tradeoffs when choosing training or inference formats.
CUDA for LLMs
FP16 is a 16-bit floating-point numerical format commonly used in machine learning computation.
- How it is used
- Uses FP16 in GEMM to reduce data movement and exploit higher hardware throughput for deep-learning computation.
- How deep it goes
- Defines FP16’s bit layout and performance tradeoffs, then shows mixed-precision GEMM configuration without detailing accuracy measurement.
- Examples
- Uses a naive CUDA GEMM, a WMMA tensor-core multiply, and CUTLASS type definitions as concrete examples.
- What you can do after
- Apply FP16 through CUDA WMMA or CUTLASS, pairing 16-bit inputs with 32-bit accumulation for GEMM.
Deep Learning with JAX
A 16-bit floating-point numeric type used as a lower-precision data type for deep learning and inference.
- How it is used
- Deploys lower-precision floating-point values for deep learning and inference workloads.
- How deep it goes
- Defines float16 and demonstrates construction, dtype inspection, and promotion behavior without developing training internals.
- Examples
- Uses range-based JAX arrays, dtype inspection, and mixed bfloat16-plus-float16 addition.
- What you can do after
- Apply it via jax.numpy by constructing float16 arrays and checking how mixed-type operations promote.
Domain-Specific Small Language Models
- How it is used
- Uses FP16 as a lower-memory arithmetic option in mixed-precision model training and inference comparisons.
- How deep it goes
- Defines FP16's bit allocation and outlines mixed-precision training steps and inference tradeoffs.
- Examples
- Compares training the same LLM in FP32 and FP16, then contrasts mixed-precision inference.
- What you can do after
- Evaluate memory, numerical-stability, and performance tradeoffs when selecting FP16 or mixed precision.
Read how Domain-Specific Small Language Models explains float16 →
Also covered in
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
- Sutskever's List
- LLMs in Production
- Local Image GenerationIn early access (MEAP)
- Deep Learning with R, Third Edition
- Deep Learning with Python, Third Edition
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- gradientA gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.
- throughputThroughput is the rate at which a system processes or completes work, data, or requests over a specified period.
- activationAn activation is a numerical value or tensor produced by a neural-network unit, layer, or operation during a forward computation.
Related concepts
- mixed precisionused by · Mixed precision is a computational technique that uses different numerical precisions for values or operations within a computation.
- FP32contrasts with · FP32 is a 32-bit single-precision floating-point format for representing approximate real numbers.
- bfloat16contrasts with · bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
- LLM.int8()used by · LLM.int8() is a mixed-precision quantization method for large language models that performs most computation in INT8 while handling activation outliers at higher precision.
- int4 quantizationcontrasts with · INT4 quantization represents numerical values using 4-bit integers, reducing storage requirements while limiting numerical precision.
Questions about float16
What is float16?
float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
Which Manning books explain float16?
Quantization and Fast Inference, CUDA for LLMs, Deep Learning with JAX and Domain-Specific Small Language Models.
What should I know before learning float16?
Manning books that teach float16 usually introduce language model, gradient, throughput and activation first.
What is the difference between float16 and FP32?
float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats. FP32 is a 32-bit single-precision floating-point format for representing approximate real numbers.
What is float16 also called?
Float16 is also called FP16, half precision or 16-bit floating point.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



