Emergent behavior
System-level behavior or capability that arises from interactions among components or from a system’s scale or complexity, rather than being specified as an isolated outcome.
Natural Language Processing in Action, Second Edition
An emergent capability is a behavior or performance characteristic that appears to arise only when a system reaches a particular scale or complexity.
- How it is used
- Frames surprising LLM capabilities as claims to evaluate against scale, benchmarks, reliability, and social consequences.
- How deep it goes
- Defines emergence, relates it to parameters and data, compares scaling patterns, and stops short of establishing general intelligence.
- Examples
- Uses LLM-size plots, emergence-by-task data, and a Llama arithmetic answer that contradicts its own explanation.
- What you can do after
- Evaluate/understand emergence claims by comparing scale, benchmark trends, model size, and observed reliability.
Read how Natural Language Processing in Action, Second Edition explains emergent behavior →
Introduction to Generative AI, Second Edition
An emergent property is a capability or behavior that appears when a system reaches sufficient scale or complexity and is not apparent in smaller versions.
- How it is used
- Uses emergence to frame unexpected capabilities in large language models and collective risks in multi-agent systems.
- How deep it goes
- Defines qualitative changes and interaction-driven patterns, and examines a measurement-based challenge without resolving the emergence debate.
- Examples
- Illustrates zero-shot learning with a vegetarian menu and group-level behavior with pricing agents that raise prices together.
- What you can do after
- Evaluate and understand claims of emergence while recognizing measurement uncertainty and collective agent risks.
Read how Introduction to Generative AI, Second Edition explains emergent behavior →
Machines that Think
Higher-level system behavior arising from interactions among components without being explicitly programmed.
- How it is used
- Uses emergence to examine reasoning-like capabilities and judgments about intelligence in language models.
- How deep it goes
- Explains system-level interactions and possible threshold dynamics while noting that emergence’s precise definition remains debated.
- Examples
- Uses neurons, bee colonies, and larger models’ instruction following to illustrate system-level capacities.
- What you can do after
- Evaluate claims about emergent intelligence by distinguishing observable competence from underlying mechanism.
Sutskever's List
A capability that appears as a model is scaled or otherwise developed and is not readily apparent in smaller or less capable versions.
- How it is used
- Describes capabilities that become visible as models scale, framing GPT-3’s in-context learning as a turning point in hyperscale.
- How deep it goes
- Distinguishes smooth training-loss trends from abrupt benchmark gains and regime changes, but offers no universal rule for predicting specific capabilities.
- Examples
- Uses GPT-3, multistep reasoning, multidigit arithmetic, translation, code generation, and benchmark score fluctuations as examples.
- What you can do after
- Evaluate apparent capability jumps by comparing benchmark scores with training-loss trends and recognizing that evaluations sample only part of possible inputs.
Generative AI in Action
An emergent property is a capability or behavior that arises from a system's components or scale without being explicitly programmed as an individual feature.
- How it is used
- Explains why LLMs can generalize from prompts to logic solving, code analysis, and other capabilities not individually programmed.
- How deep it goes
- Defines implicit emergence, demonstrates scale thresholds, and stops short of explaining the exact internal logic producing each behavior.
- Examples
- Uses proportional reasoning, step-by-step math, C-to-Python translation, and a scaling curve as concrete demonstrations.
- What you can do after
- Apply it via prompts to solve expressed problems and translate unfamiliar code, while recognizing scale-dependent and unanticipated behavior.
Read how Generative AI in Action explains emergent behavior →
Grokking AI Algorithms, Second Edition
A collective pattern or capability that arises from interactions among simpler individual agents.
- How it is used
- Frames swarm intelligence as collective problem solving produced by many simple individuals interacting.
- How deep it goes
- Explains pheromone reinforcement, shortest-path convergence, and flocking rules through concrete biological examples.
- Examples
- Uses ant colonies, an asymmetric bridge experiment, bird flocks, and a bee swarm as examples.
- What you can do after
- Evaluate how simple local behaviors can produce coordinated groups and inspire optimization algorithms.
Read how Grokking AI Algorithms, Second Edition explains emergent behavior →
Also covered in
- The Coder CafeIn early access (MEAP)
- LLM Customization and Fine-TuningIn early access (MEAP)
- Building Specialized AI SystemsIn early access (MEAP)
- Build a Large Language Model (From Scratch)
- Inside AI
- Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
- Timeless Algorithms, The Foundational IdeasIn early access (MEAP)
- LLM Evaluation and Alignment, The Foundational IdeasIn early access (MEAP)
- Think Distributed Systems
- Microservices Patterns, Second EditionIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- transformer architectureA transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- GPT-4GPT-4 is a multimodal large language model developed by OpenAI that processes text and image inputs and generates text.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
Related concepts
- language modelproduced by · A language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- indirect messagingproduces · Indirect messaging is a communication pattern in which a sender and receiver exchange information through an intermediary or event channel rather than direct calls.
- multi-agent systemproduced by · A multi-agent system is a computational system composed of multiple autonomous or semi-autonomous agents that interact or coordinate to perform tasks or pursue goals.
- behaviorkind of · Behavior is the observable actions, responses, outputs, or effects of a system or object under specified conditions or interactions.
- scaling lawcontrasts with · A scaling law is an empirical relationship describing how a system’s properties change as its scale or resource levels vary.
Questions about emergent behavior
What is emergent behavior?
System-level behavior or capability that arises from interactions among components or from a system’s scale or complexity, rather than being specified as an isolated outcome.
Which Manning books explain emergent behavior?
Natural Language Processing in Action, Second Edition, Introduction to Generative AI, Second Edition, Machines that Think, Sutskever's List, Generative AI in Action and Grokking AI Algorithms, Second Edition.
What should I know before learning emergent behavior?
Manning books that teach emergent behavior usually introduce language model, transformer architecture, GPT-4 and accuracy first.
What is the difference between emergent behavior and scaling law?
System-level behavior or capability that arises from interactions among components or from a system’s scale or complexity, rather than being specified as an isolated outcome. A scaling law is an empirical relationship describing how a system’s properties change as its scale or resource levels vary.
What is emergent behavior a kind of?
Emergent behavior is a kind of behavior. Behavior is the observable actions, responses, outputs, or effects of a system or object under specified conditions or interactions.
What is emergent behavior part of?
Emergent behavior is part of complex systems science. Complex systems science is the interdisciplinary study of systems whose interacting parts produce collective behavior that cannot be explained by examining parts in isolation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





