liveBook · Manning

Abstraction

As explained in Machines that Think

Abstraction is the formation of a more general representation by omitting incidental details while preserving relevant structure.

Abstraction forms a more general representation by retaining useful structure and leaving out details. What counts as useful depends on the task: a reduction can make a problem manageable while also removing information needed in another setting. section 3.3.5 section 5.2.1

In neural networks, abstraction takes the form of compression. A model condenses examples into patterns shaped by what occurs together in its training data. These patterns let it respond to unfamiliar inputs without finding an exact match, but they can include incidental correlations rather than defining features. section 3.3.5

Human abstraction typically draws on multiple sources of information and expectations about the world to identify what stays the same as circumstances change. People can also learn misleading associations, especially with limited or noisy feedback. A model’s consistent behavior on new inputs can likewise be useful without establishing that it understands what its patterns represent. section 3.3.5

From biological to artificial neurons

An artificial neuron reduces a biological neuron to a computational role. Connections become adjustable strengths, timing becomes simplified activation updates, and biochemical processes and most cellular structure are omitted. The reduction made networks expressible mathematically and suitable for training on digital hardware; its value lay in useful behavior, not biological fidelity. section 5.2.1

Figure 5.1 Early neural network research replaced the biological neuron with a simplified computational unit that transforms inputs into outputs through a small set of mathematical operations. This abstraction made it possible to analyze, train, and scale networks using digital computation.

The figure contrasts a biological neuron with a simplified computational unit, showing the reduction that makes network analysis, training, and scaling possible. section 5.2.1

Language is another abstraction: it compresses experience into symbols while leaving out much sensory detail. A language-only model can learn descriptions of a cat without seeing it move or hearing it. This distinction matters for tasks that depend on perception or grounding. Training on text paired with images, audio, or video widens the evidence available to a model, but does not guarantee understanding. section 4.5.1 section 4.5.1

Generalizing beyond training examples is a form of abstraction, even without awareness. Whether such behavior counts as thinking depends partly on whether intelligence is judged by consistent performance or by understanding what the performance concerns. section 1.6.2 section 3.3.5

How Machines that Think covers it

How deep it goes
Compares compression, human abstraction, and simplified artificial neurons without giving a mathematical implementation.
Examples
Uses a computational neuron and a language model’s description of a cat to illustrate different reductions.
What you can do after
Evaluate/understand how useful generalization can coexist with incidental correlations and limits on grounding or understanding.
How it connects
Connects abstraction to generalization, neural networks, grounding, and multimodality.

Also explained in

Compare how 24 books explain abstraction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.