liveBook · Manning

Dynamic range

Dynamic range is the span between the smallest and largest magnitudes that a numerical representation or computation can represent effectively.

How 2 Manning books explain it

Build a DeepSeek Model (From Scratch)

Dynamic range is the span between the smallest and largest magnitudes that a numerical representation or computation can represent effectively.

How it is used
Frames numerical-format selection and quantization around representable magnitude, memory, and precision trade-offs.
How deep it goes
Explains exponent allocation, format comparisons, overflow, underflow, and blockwise range management.
Examples
Compares FP32, FP16, BF16, E4M3, and E5M2, then quantizes an outlier-containing vector.
What you can do after
Evaluate numerical formats and choose local scaling strategies that reduce overflow and precision loss.

Read how Build a DeepSeek Model (From Scratch) explains dynamic range →

Quantization and Fast Inference

Dynamic range is the span of magnitudes that a numerical representation can encode.

How it is used
Uses activation ranges to guide fixed-point binary-point placement during neural-network quantization.
How deep it goes
Explains the fixed-point range–precision tradeoff with Q-format failures, then motivates floating-point without detailing IEEE-754 encoding.
Examples
Contrasts Q0.7 saturation at 5.2 with Q7.0 rounding a 0.001 gradient to zero.
What you can do after
Apply calibration to measured activation ranges to choose a fixed-point binary point.

Read how Quantization and Fast Inference explains dynamic range →

Questions about dynamic range

What is dynamic range?

Dynamic range is the span between the smallest and largest magnitudes that a numerical representation or computation can represent effectively.

Which Manning books explain dynamic range?

Build a DeepSeek Model (From Scratch) and Quantization and Fast Inference.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.