Emergent behavior is a capacity that appears at the level of an organized system, rather than in its components considered individually. It depends on the parts, but becomes visible through their interactions. A neuron, for example, does not perceive or remember on its own; a nervous system can produce both capacities. The precise definition of emergence remains debated. section 9.3.1
Figure 9.6 Emergence as system-level behavior. A single neuron does not perceive, remember, or think, and a single bee does not organize the colony. Yet through interaction at scale, neurons can give rise to cognition and bees can produce coordinated collective behavior. In both cases, the emergent property depends on the components but becomes visible only through their organization and interaction.

Neurons and bees illustrate how interaction can produce capacities at the level of a nervous system or colony. section 9.3.1
In large language models, emergent abilities have been described as capabilities absent in smaller models but present in larger ones, making them difficult to predict from smaller-scale performance. Performance can appear to jump from near-random guessing to useful results. Examples include instruction following, in-context learning, symbolic manipulation, and multi-step reasoning. section 9.3.1
Scale alone does not explain these changes. Model behavior depends on the interaction of data, architecture, training, and conditions of use. Several processes may contribute: section 9.3.2
Potential contributors include:
- Pattern accumulation and interaction: learned regularities from different contexts can combine into reusable signals. section 9.3.2
- Intermediate representations: models can develop structures corresponding to roles, procedures, analogies, relations, and task formats. section 9.3.2
- Coordination and thresholds: an ability may depend on many representations acting together, becoming visible only when enough supporting structures align. section 9.3.2
An apparent threshold can reflect gradual underlying changes. Grokking is a related but distinct phenomenon: generalization can improve suddenly within the same model as training continues, rather than across models of different scales. Conditions of use also matter: organizing generation differently can make a previously difficult problem solvable without changing the model itself. section 9.3.2 section 9.2.3
Emergence explains how surprising capacities can arise without being explicitly programmed, but it does not establish what those capacities mean. Token generation remains the mechanism behind language-model output; judgment also requires examining whether behavior is reliable, flexible, and transferable. Even then, the question of whether the system displays intelligence remains open. section 9.3.3 section 9.6





