A parameter is a learned numerical weight that helps determine how a model transforms inputs into outputs. In a language model, parameters encode statistical regularities as a web of constraints on how features interact and which patterns are treated as similar. They do not store sentences, rules, or facts literally. section 4.2.1 section 6.4.1
The number of parameters is a visible measure of model size and affects representational capacity. Wider layers can represent more features at each step, while deeper stacks can combine and transform features across layers. More capacity gives a model room to distinguish more situations, but a higher parameter count alone does not determine capability. section 4.2.1
Training data influences which structures a model encounters, while parameters determine how much structure it can represent. Performance depends on balancing model size with data and computation; increasing all three tends to improve results, though additional gains diminish. section 4.2.2
During ordinary inference, learned weights remain fixed. They transform representations of the current input, influencing which continuations are more likely. A prompt can change the response without changing the parameters: this in-context adaptation is temporary, not accumulated knowledge. Online learning can update parameters as new data arrives, but those updates are still training. section 5.4.4 section 6.4.1
Figure 5.4 Human cognition continuously updates through interaction with the world, so learning and action form a single loop. Artificial neural networks separate learning from use: parameters are updated during training, after which the trained model performs inference without incorporating new experience.

The figure contrasts parameter updates during training with fixed-parameter inference. section 5.4.4
This distinction matters for reliability. If a fact is neither encoded in the parameters nor supplied in the active input, it cannot influence the answer. Recent events, omitted documents, and earlier exchanges therefore need to be supplied as input when they are otherwise unavailable. section 6.4.1








































