Underflow
Underflow is a numerical condition in which a computed result falls below the representable range of its numeric type, potentially losing precision or becoming zero.
100 Java Mistakes and How to Avoid Them
- How it is used
- Flags a floating-point failure mode when calculations use extremely small values.
- How deep it goes
- Explains intermediate multiplication underflow through a focused hypotenuse example and a library alternative.
- Examples
- Uses 3e-200 and 4e-200 inputs whose manual hypotenuse computation yields zero instead of 5e-200.
- What you can do after
- Evaluate/understand when an extremely small floating-point calculation can collapse to zero and choose Math.hypot.
Read how 100 Java Mistakes and How to Avoid Them explains underflow →
Build a DeepSeek Model (From Scratch)
Underflow is the condition in which a numerical value is too small in magnitude for a format to represent its detail accurately.
- How it is used
- Explains numerical failures that constrain FP8 quantization, reduced-precision formats, and low-precision matrix multiplication.
- How deep it goes
- Connects underflow to exponent range, mantissa precision, accumulation registers, scaling factors, and mitigation strategies.
- Examples
- Uses FP16 detail loss, thousands of FP8 products, stale scaling factors, and Tensor Core promotion as concrete cases.
- What you can do after
- Diagnose underflow by examining format range, accumulator precision, and whether quantization scales match current tensor magnitudes.
Read how Build a DeepSeek Model (From Scratch) explains underflow →
Also covered in
What to know first
- floating-point arithmeticFloating-point arithmetic is numerical computation using finite-precision representations of real numbers, which can introduce rounding and range errors.
- bfloat16bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
- integerAn integer is a whole-number mathematical value, including negative numbers, zero, and positive numbers, with no fractional part.
- softmaxSoftmax is a function that exponentiates each value in a score vector and normalizes the results to produce positive values summing to one, forming a probability distribution.
Related concepts
- Math.hypotaddressed by · Math.hypot is a Java static method that returns the Euclidean hypotenuse of two arguments with numerical handling designed to reduce overflow and underflow.
- online quantizationaddressed by · Online quantization calculates quantization scaling factors from the current tensor or batch during processing rather than from historical statistics.
- numerical overflowcontrasts with · Numerical overflow occurs when an arithmetic result falls outside the range representable by its numeric type or format.
Questions about underflow
What is underflow?
Underflow is a numerical condition in which a computed result falls below the representable range of its numeric type, potentially losing precision or becoming zero.
Which Manning books explain underflow?
100 Java Mistakes and How to Avoid Them and Build a DeepSeek Model (From Scratch).
What should I know before learning underflow?
Manning books that teach underflow usually introduce floating-point arithmetic, bfloat16, integer and softmax first.
What is the difference between underflow and numerical overflow?
Underflow is a numerical condition in which a computed result falls below the representable range of its numeric type, potentially losing precision or becoming zero. Numerical overflow occurs when an arithmetic result falls outside the range representable by its numeric type or format.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

