Learning changes how a system responds through experience or data. In machine learning, training develops this capability by adjusting a model’s parameters from examples, rather than specifying every rule in advance. The goal is to capture patterns that apply to new cases, not merely reproduce the training data. section 1.4.1 section 3.3 section 4.6.1
Neural networks build representations through successive stages of computation. During training, they make predictions, compare them with desired outputs, and adjust parameters to reduce error. Examples are typically processed in batches over multiple passes, or epochs. Smaller batches produce more variable updates; larger batches give steadier feedback but require more memory. section 3.0 section 3.2.4 section 4.6.1
Training differs from inference, when a trained model uses its fixed parameters to produce outputs. A correction in a conversation can change a response without changing the underlying model. New experience generally affects parameters only if it is captured and used in further training; online learning applies such updates incrementally during operation. section 4.6.1 section 5.4.4 section 9.4.4
What a model learns depends on its objective. Language-model training rewards plausible continuation of token sequences, not independent verification of facts. A fluent answer can therefore extend a weak association rather than express reliable knowledge. Training on images, audio, and text can strengthen associations across modalities, but association alone does not establish causation. section 6.1.2 section 4.5.4
Learning from a falling glass
A child can develop an expectation that an unsupported glass will fall after relatively few examples. Language models generally acquire regularities through much larger datasets before deployment. The comparison is not a simple measure of efficiency: children begin with perceptual and other learning biases, whereas a model starting from randomly initialized parameters must acquire its priors through training. section 9.4.4 section 9.5.5
Figure 5.2 Human learning unfolds through interaction with the world, where actions produce consequences that differ in what they cost and directly shape future behavior, with negative outcomes often exerting stronger influence than positive ones. Neural networks instead learn by minimizing numerical loss functions during training, adjusting parameters through optimization rather than through lived outcomes.

The figure contrasts learning shaped by lived consequences with neural-network training driven by numerical loss. section 5.4.1
Large-scale training also has organizational consequences. More parameters, data, and optimization steps demand sustained computation, while data collection and preparation add substantial work. At the frontier, specialized infrastructure and coordination can restrict who can train a model from end to end, even when the methods are publicly known. section 4.6.1 section 4.7.1































