liveBook · Manning

Emergent behavior

As explained in Machines that Think

Higher-level system behavior arising from interactions among components without being explicitly programmed.

Emergent behavior is a capacity that appears at the level of an organized system, rather than in its components considered individually. It depends on the parts, but becomes visible through their interactions. A neuron, for example, does not perceive or remember on its own; a nervous system can produce both capacities. The precise definition of emergence remains debated. section 9.3.1

Figure 9.6 Emergence as system-level behavior. A single neuron does not perceive, remember, or think, and a single bee does not organize the colony. Yet through interaction at scale, neurons can give rise to cognition and bees can produce coordinated collective behavior. In both cases, the emergent property depends on the components but becomes visible only through their organization and interaction.

Neurons and bees illustrate how interaction can produce capacities at the level of a nervous system or colony. section 9.3.1

In large language models, emergent abilities have been described as capabilities absent in smaller models but present in larger ones, making them difficult to predict from smaller-scale performance. Performance can appear to jump from near-random guessing to useful results. Examples include instruction following, in-context learning, symbolic manipulation, and multi-step reasoning. section 9.3.1

Scale alone does not explain these changes. Model behavior depends on the interaction of data, architecture, training, and conditions of use. Several processes may contribute: section 9.3.2

Potential contributors include:

  • Pattern accumulation and interaction: learned regularities from different contexts can combine into reusable signals. section 9.3.2
  • Intermediate representations: models can develop structures corresponding to roles, procedures, analogies, relations, and task formats. section 9.3.2
  • Coordination and thresholds: an ability may depend on many representations acting together, becoming visible only when enough supporting structures align. section 9.3.2

An apparent threshold can reflect gradual underlying changes. Grokking is a related but distinct phenomenon: generalization can improve suddenly within the same model as training continues, rather than across models of different scales. Conditions of use also matter: organizing generation differently can make a previously difficult problem solvable without changing the model itself. section 9.3.2 section 9.2.3

Emergence explains how surprising capacities can arise without being explicitly programmed, but it does not establish what those capacities mean. Token generation remains the mechanism behind language-model output; judgment also requires examining whether behavior is reliable, flexible, and transferable. Even then, the question of whether the system displays intelligence remains open. section 9.3.3 section 9.6

How Machines that Think covers it

How deep it goes
Explains system-level interactions and possible threshold dynamics while noting that emergence’s precise definition remains debated.
Examples
Uses neurons, bee colonies, and larger models’ instruction following to illustrate system-level capacities.
What you can do after
Evaluate claims about emergent intelligence by distinguishing observable competence from underlying mechanism.
How it connects
Links emergence to complex systems, in-context learning, threshold dynamics, and grokking.

Also explained in

Compare how 6 books explain emergent behavior →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.