liveBook · Manning

Precision

As explained in Learn Go with Pocket-Sized Projects

Numeric precision is the degree of detail or significant accuracy with which a numerical value is represented or computed.

Precision describes the detail with which a number is represented. In floating-point arithmetic, it limits how many significant decimal digits can be relied on. In a fixed-precision decimal, it can instead mean the number of digits after the decimal point. section 6.2.1 section 6.2.4

Go’s IEEE-754 float32 and float64 provide six and 15 guaranteed decimal digits, respectively. Some fractions, such as 0.625, are exactly representable; most decimal fractions are not. As a result, arithmetic can lose small changes or produce a value close to, but not exactly, the expected result. section 6.2.1 section 6.2.3

A payment lost to precision

With float32, subtracting 1 from 100,000,000 leaves the stored result at 100,000,000: the change is too small relative to the balance’s precision. Similarly, math.Sin(math.Pi) returns a small nonzero float64 rather than exactly zero. For an approximate result, compare the difference against a tolerance suitable for the calculation instead of relying on ==. section 6.2.3

For money, fixed precision avoids introducing floating-point imprecision at input. A Decimal can store integer subunits and a decimal-place count: 1.52 becomes 152 with precision 2. Parse the input string directly into that representation, and enforce size limits because integer storage is finite. section 6.2.2 section 6.2.4

Listing 6.6 decimal.go: Decimal struct implementation
// Decimal can represent a floating-point number with a fixed precision.
// example: 
// 1.52 = 152 * 10^(-2) will be stored as {152, 2} #1
type Decimal struct {
    // subunits is the amount of subunits. 
    // Multiply it by the precision to get the real value
    subunits int64 #2
    // Number of "subunits" in a unit, expressed as a power of 10.
    precision byte #3
}

The Decimal fields store the integer subunits and the power-of-ten scale separately. section 6.2.4

When applying precision to currency conversion:

  • Reject an amount whose decimal precision exceeds the currency’s supported precision: 19.8875 has four decimal places, but Canadian dollars use two. section 6.2.6
  • Multiply the amount and exchange-rate subunits, adding their precision counts. Adjust the result’s scale to the target currency by multiplying or dividing by a power of ten. section 6.3.1
  • Check supported size and precision limits, and test scale changes, large values, and small rates. The shown scale reduction uses integer division, so it truncates rather than implementing a general rounding policy. section 6.3.1 section 6.3.2
  • Format the fractional part using the currency’s precision: two decimal places display 12.5 as 12.50; zero precision requires no decimal point. section 6.4.3

The converter is a tutorial project, not a tool for real-life transactions. section 6.0

How Learn Go with Pocket-Sized Projects covers it

How deep it goes
Explains floating-point limitations and implements fixed-precision parsing, conversion, validation, and formatting; the converter remains a tutorial.
Examples
Illustrates lost subtraction, approximate sine, decimal subunits, currency precision checks, and exchange-rate test cases.
What you can do after
Builds a bounded fixed-precision decimal representation and applies it to a tutorial currency converter.
How it connects
Connects precision to floating-point arithmetic, fixed-point representation, currency conversion, validation, and string formatting.

Also explained in

Compare how 10 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Learn Go with Pocket-Sized Projects. The section links point to the parts of the book it draws on.