Bfloat16
bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
Deep Learning with JAX
A 16-bit floating-point format with a float32-like exponent range and reduced significand precision, commonly used for deep learning.
- How it is used
- Uses bfloat16 for lower-precision deep-learning arrays and inference-oriented storage savings.
- How deep it goes
- Defines the format, demonstrates construction and storage, and tests promotion with float16; it stops short of training details.
- Examples
- Demonstrates a ten-element bfloat16 array, its 20-byte size, and additions with bfloat16 and float16.
- What you can do after
- Operate JAX arrays with bfloat16 and predict result dtypes when combining bfloat16 and float16 operands.
Deep Learning with R, Third Edition
bfloat16 is a 16-bit floating-point data type with an 8-bit exponent and a 7-bit mantissa, providing greater numerical range but lower precision than float16.
- How it is used
- Supports lower-memory, potentially faster inference and computation on hardware optimized for reduced-precision operations.
- How deep it goes
- Explains bit allocation and range-versus-resolution trade-offs, but stops short of detailed floating-point implementation.
- Examples
- Uses a float16 comparison table and recommends testing both formats on the target device.
- What you can do after
- Evaluate bfloat16 against float16 when selecting an inference precision for a newer GPU or TPU.
Read how Deep Learning with R, Third Edition explains bfloat16 →
Domain-Specific Small Language Models
A 16-bit floating-point format with a wider exponent range and reduced mantissa precision than FP16, used in machine learning.
- How it is used
- Uses reduced-precision arithmetic to lower memory requirements and support efficient model training and inference.
- How deep it goes
- Defines the bit layout, compares BF16 with FP16, and notes hardware checks without detailing numerical algorithms.
- Examples
- Compares BF16 with FP16 and demonstrates GPU support through torch.cuda.is_bf16_supported().
- What you can do after
- Evaluate whether supported hardware can use BF16 while balancing dynamic range, precision, memory, and inference performance.
Read how Domain-Specific Small Language Models explains bfloat16 →
Related concepts
- float16contrasts with · Float16 is a 16-bit floating-point numeric format that represents values with less precision and storage than single-precision formats such as float32.
- mixed precisionused by · Mixed precision is a computational technique that uses multiple numerical precisions for different values or operations to balance resource efficiency and numerical accuracy or stability.
- QLoRAused by · QLoRA is a parameter-efficient fine-tuning method that quantizes a pretrained model’s weights while training low-rank adapters to reduce memory requirements.
- FP32uses · FP32 is the IEEE 754 binary32 single-precision floating-point format, which represents approximate real numbers using 32 bits.
- language modelconfigures · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.