liveBook · Manning

Scale

As explained in Machines that Think

The magnitude of a model and its associated data and computational resources.

Scale is the magnitude of a model and the data and computational resources associated with it. In large language models, it encompasses parameter counts, data exposure, and the architectural and computational conditions needed for training. Scale helps explain both capabilities and limitations; it is not simply a matter of refining an existing model. section 4.2

The conditions that enable learning at scale do not necessarily make a model practical to deploy. A model may need to be reshaped, constrained, or compressed to operate within real-world limits. Usable capability therefore depends on more than successful large-scale training. section 4.3

Cost also shapes which ideas are tested. New architectures and training strategies are often tried on smaller models first; only promising candidates justify evaluation at frontier scale. Large experiments require planning and validation, leaving less room for rapid iteration or speculative exploration. section 4.6.4

At the frontier, expertise and published methods are not enough to guarantee practical access. Full-scale training requires sustained compute, suitable data, specialized infrastructure, and coordination. Data must be varied and usable as well as abundant; access to proprietary, specialized, or newly generated data can become an advantage. section 4.7.1 section 4.7.2

This creates an uneven field: model capabilities can spread widely while control over their underlying development remains concentrated. Smaller teams can still contribute through research, adaptation, integration, and applications, but often work downstream of decisions made by those able to sustain full-scale training. Scale amplifies capability while also shaping who can influence it. section 4.7.1 section 4.7.5

How Machines that Think covers it

How deep it goes
Explains training, deployment, research, and participation constraints at an overview level.
Examples
Contrasts testing ideas on smaller models with evaluation at frontier scale and identifies GPUs and TPUs as specialized hardware.
What you can do after
Evaluate how resource requirements affect model usability, research choices, and participation in development.
How it connects
Connects scale to model parameters, training data, model compression, infrastructure, and deployment.

Also explained in

Compare how 2 books explain scale →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.