liveBook · Manning

Precision

As explained in Modern C, Third Edition

Precision is the number of significant binary digits available to represent a floating-point value in the C standard's restricted sense.

A floating-point value

In a simplified binary model, let the sign be −1, the exponent −2, and the mantissa bits 101. The value is −1 × 2⁻² × (2⁻¹ + 2⁻³) = −5/32 = −0.15625. The precision p is the number of mantissa bits available, not the exponent or the value’s sign. section 5.7.8

Precision is the number of significant digits available to represent a value. In the simplified binary floating-point model, a value has a sign, an exponent, and p mantissa bits. The exponent moves the point at which the binary expansion is cut off, so the cutoff depends on the value’s magnitude. section 5.7.8

The precision and exponent bounds depend on the type and implementation. For double, DBL_MANT_DIG gives the mantissa precision; DBL_MIN_EXP and DBL_MAX_EXP give exponent bounds. A typical value for DBL_MANT_DIG is 53, but C constrains rather than universally fixes type precision. section 5.7.8 section 5.2

Precision is not the same as the accuracy of a floating-point computation. Intermediate results can be cut off, and calculations involving very different magnitudes are especially susceptible to the effects of limited representation. Algebraic rearrangements that are valid in exact arithmetic need not give the same computed result. section 5.7.8 section 5.8

The term also applies to integers. For an unsigned integer type, precision p is the number of binary value bits and equals its width; its values range from 0 through 2ᵖ − 1. For example, with p = 16, the value 240 has bits 4, 5, 6, and 7 set. section 5.7.1

TAKEAWAY: Treat precision as a property of the type’s representation, not a guarantee that a computation is accurate. Check the relevant type macros instead of assuming a fixed precision across implementations. section 5.7.8 section 5.8

How Modern C, Third Edition covers it

How deep it goes
Works through a binary floating-point representation and an unsigned integer example, using a simplified floating-point model.
Examples
Calculates −5/32 from sign, exponent, and mantissa bits, and identifies the set bits of 240 in a 16-bit unsigned type.
What you can do after
Evaluate representable values and distinguish type precision from computational accuracy.
How it connects
Connects precision to floating-point accuracy, exponent bounds, integer width, and bitwise operations.

Also explained in

Compare how 10 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Modern C, Third Edition. The section links point to the parts of the book it draws on.