A feature is a representation of a meaningful distinction in input. Neural networks can learn features from examples rather than being explicitly told what to detect, building increasingly abstract representations as patterns combine. section 1.4.3 section 4.2.1
In language models, learned weights constrain how features interact and which distinctions remain separate. More parameters, wider layers, and deeper stacks increase representational capacity, but do not guarantee performance; weights encode patterns, not sentences or facts stored literally. section 4.2.1
From simple patterns to abstraction
A neural network may develop features that progress from edges to shapes and objects, or from letters to words and meaning. section 1.4.3













