liveBook · Manning

Precision

As explained in 100 Go Mistakes and How to Avoid Them

Precision is the degree to which a numerical representation or computation preserves significant detail.

Precision measures how much significant detail a numerical representation preserves. Floating-point types store approximations because infinitely many real values must fit into a finite number of bits. section 3.3

The surprising multiplication result

A developer may expect float32 multiplication of 1.0001 by itself to print 1.00020001. Instead, it commonly prints 1.0002 because the stored value is already an approximation. Increasing the representation’s available detail can improve accuracy, but cannot make floating-point arithmetic exact. section 3.3

Go floating-point values follow IEEE-754: sign, exponent, and mantissa bits determine the stored approximation. float32 allocates fewer mantissa bits than float64, so its representation can lose more detail. section 3.3

Precision affects more than the initial conversion:

  • Avoid treating == as a universally reliable floating-point comparison; compare the difference instead. section 3.3

How 100 Go Mistakes and How to Avoid Them covers it

How deep it goes
Explains finite-bit approximations, IEEE-754 fields, comparison risks, and accumulation order, but defers conversion mechanics.
Examples
Uses float32 multiplication, IEEE-754 representation, accumulation-order functions, and numeric result comparisons.
What you can do after
Diagnose surprising floating-point results and choose comparison and evaluation-order strategies more carefully.
How it connects
Connects precision with floating-point arithmetic, IEEE-754, mantissas, exponents, comparisons, and rounding.

Also explained in

Compare how 20 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of 100 Go Mistakes and How to Avoid Them. The section links point to the parts of the book it draws on.