liveBook · Manning

Neural network

As explained in Machines that Think

A computational model composed of interconnected units with parameters that can be learned to process patterns.

A neural network is a computational model that turns inputs into outputs through many small, interconnected computations. Each unit receives numerical inputs, adjusts their relative influence, and produces a signal for further transformation. Arranged in successive stages, these computations can build increasingly complex representations of patterns in data. section 3.1 section 3.0

Unlike a system whose rules are specified in advance, a neural network learns parameters from examples. Training adjusts connection strengths to reduce a numerical measure of error. In a typical training process, the network makes predictions on batches of examples, compares them with desired answers, and updates its parameters over repeated passes through the data, called epochs. section 1.4.4 section 3.2.4 section 5.4.1

Learning visual features

A network processing images can learn successive representations, from edges to shapes to objects. Rather than requiring every feature to be specified beforehand, it learns transformations that help perform the task. section 1.4.3 section 3.1.6

The aim is generalization: extracting patterns that support useful responses to new data. A network that relies too closely on its training examples can instead become brittle when faced with unfamiliar cases. Different architectures arrange the same basic components to suit different structures in data, including spatial relationships and sequences. section 3.3 section 3.0 section 3.4

Neural networks were loosely inspired by biological learning, but their mechanisms are shaped by mathematical and engineering constraints. Their success became practical as data, computing power, and training methods improved; graphics processors helped by efficiently performing repetitive matrix operations. section 5.0 section 1.4.3

The same distributed parameters that make complex patterns possible also make individual decisions difficult to interpret. This black box problem complicates debugging, bias detection, and accountability, particularly in high-stakes settings. Observable fluency or responsiveness is not, by itself, evidence that a network understands or learns in the same way a person does. section 1.4.4 section 5.4.6

Figure 5.2 Human learning unfolds through interaction with the world, where actions produce consequences that differ in what they cost and directly shape future behavior, with negative outcomes often exerting stronger influence than positive ones. Neural networks instead learn by minimizing numerical loss functions during training, adjusting parameters through optimization rather than through lived outcomes.

The figure contrasts learning from lived consequences with parameter adjustment through numerical loss. section 5.4.1

How Machines that Think covers it

How deep it goes
Explains interconnected computations, parameter updates, batches, and generalization without a code implementation.
Examples
Uses visual features from edges to objects and word relationships in mathematical space to illustrate learned representations.
What you can do after
Evaluate and understand neural-network behavior by distinguishing generalization from memorization and observable performance from underlying mechanism.
How it connects
Connects neural networks to generalization, learned representations, architecture, interpretability, and human cognition.

Also explained in

Compare how 11 books explain neural network →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.