Diminishing returns
Diminishing returns is a pattern in which successive increases in an input produce progressively smaller incremental gains in an outcome.
Vibe Engineering
Diminishing returns is a pattern in which each additional increase in an input produces a smaller incremental improvement in outcome.
- How it is used
- Guides decisions about model scaling, context size, and when to stop optimizing AI-assisted development costs.
- How deep it goes
- Explains declining marginal gains through benchmark comparisons and an optimization ladder, without deriving a mathematical model.
- Examples
- Compares query-history benchmarks and a sequence of AI-assisted coding cost reductions.
- What you can do after
- Evaluate whether additional context or optimization effort is justified by its incremental benefit and cost.
Timeless Algorithms, The Foundational Ideas
The principle that successive increases in an input produce progressively smaller incremental gains in an outcome.
- How it is used
- Uses diminishing returns to frame how scaling resources affect loss and to assess whether larger investments are worthwhile.
- How deep it goes
- Explains power-law improvement and its limits, including conditional predictions and resource, infrastructure, and economic constraints.
- Examples
- Illustrates flattening loss curves and describes diminishing time savings from adding processors.
- What you can do after
- Evaluate whether expected loss reductions justify the rising resource and operating costs of further scaling.
Read how Timeless Algorithms, The Foundational Ideas explains diminishing returns →
How Large Language Models Work
Diminishing returns is the pattern in which additional investment or effort produces progressively smaller gains in an outcome.
- How it is used
- Limits claims that LLMs can improve indefinitely through repeated self-generated training.
- How deep it goes
- Explains S-curve plateaus, degraded iterative training, and hardware constraints without developing a quantitative model.
- Examples
- Uses repeated model training, transistor growth, computing performance, GPU infrastructure, and iterative technological techniques.
- What you can do after
- Evaluate claims about boundless LLM improvement by checking for new information, new techniques, and system-level constraints.
Read how How Large Language Models Work explains diminishing returns →
Grokking Statistics
Diminishing returns is a pattern in which each additional input produces a progressively smaller increase in output.
- How it is used
- Uses capability growth and data-collection costs to judge whether a system’s operating domain is too broad.
- How deep it goes
- Explains the pattern through an S-curve, inflection point, coverage targets, and rescoping.
- Examples
- Uses golf photographs, a family life story, system features, and capability coverage as examples.
- What you can do after
- Evaluate capability growth, recognize an inflection point, and rescope an overly broad operating domain.
Also covered in
- Applied Reinforcement LearningIn early access (MEAP)
- Everyday Data Visualization
- The Art of Unit Testing 3e
- Statistics Slam Dunk
- LLM Customization and Fine-TuningIn early access (MEAP)
- Writing for Developers: Blogs that get read
- Learn PowerShell in a Month of Lunches: Fourth Edition, Covers Windows, Linux, and macOS
- Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- performancePerformance is a measure of how effectively a person, process, model, or system achieves its objectives, including qualities such as output quality, speed, and resource efficiency.
- tool useTool use is the capability of an AI system or agent to request external software operations through defined interfaces to obtain information or perform actions.
Related concepts
- parallelismproduced by · Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially.
- scaling lawcontrasts with · A scaling law is an empirical relationship describing how a system’s properties change as its scale or resource levels vary.
- accuracymeasures · Accuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- fine-tuningaddressed by · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- power-law relationshipproduced by · A power-law relationship is a mathematical relationship in which one quantity is proportional to another quantity raised to a fixed exponent.
Questions about diminishing returns
What is diminishing returns?
Diminishing returns is a pattern in which successive increases in an input produce progressively smaller incremental gains in an outcome.
Which Manning books explain diminishing returns?
Vibe Engineering, Timeless Algorithms, The Foundational Ideas, How Large Language Models Work and Grokking Statistics.
What should I know before learning diminishing returns?
Manning books that teach diminishing returns usually introduce language model, fine-tuning, performance and tool use first.
What is the difference between diminishing returns and parallel foreach?
Diminishing returns is a pattern in which successive increases in an input produce progressively smaller incremental gains in an outcome. Parallel foreach is a technique that processes multiple input items concurrently through a foreach operation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



