liveBook · Manning

Floating-point arithmetic

As explained in Deep Learning with PyTorch, Second Edition

Numeric values represented approximately in binary floating-point form, used here as the tensor element type for loaded data.

Floating-point numbers give neural networks a practical way to encode real-world data and transform it through successive representations. Inputs, intermediate features, and outputs are typically stored as numeric values in tensors. section 3.1 section 3.0

In PyTorch, floating-point values are used to represent:

  • Image information, including pixel values and learned features such as edges, textures, ears, noses, and eyes. section 3.1 section 4.8
  • Numerical columns from tables, such as measurements that neural networks process through matrix multiplication and nonlinear functions. section 4.3
  • Intermediate representations whose values depend on both the input and the preceding layer’s weights. section 3.1

Choosing a tensor numeric type

Neural-network computation typically uses float32. Float64 generally increases memory use and computation time without improving model accuracy, while float16 can reduce memory use with a minor accuracy effect. section 3.6.2 section 3.6.2

How Deep Learning with PyTorch, Second Edition covers it

How deep it goes
Defines floating-point representations and compares common precisions, but does not develop floating-point arithmetic internals.
Examples
Uses image pixels, learned edges and textures, image structures, and numerical table columns as concrete representations.
What it assumes
Introduces tensor-based numeric representation before requiring familiarity with neural-network data handling.
What you can do after
Understand how PyTorch encodes data and selects float32, float16, or float64 for neural-network computation.
How it connects
Connects floating-point values with tensors, neural networks, intermediate representations, data encoding, and data types.
Cover of Deep Learning with PyTorch, Second Edition
liveBookmanning.com
Read Deep Learning with PyTorch, Second Edition on liveBook Get the book at manning.com

Also explained in

Compare how 20 books explain floating-point arithmetic →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with PyTorch, Second Edition. The section links point to the parts of the book it draws on.