Parameter count
Parameter count is the number of numerical parameters in a computational model, typically the learned parameters of a machine-learning model.
Rearchitecting LLMs
Parameter count is the number of learned scalar parameters in a machine-learning model.
- How it is used
- Establishes model size before architecture changes as part of a baseline inspection.
- How deep it goes
- Computes the total by summing each parameter tensor’s element count, alongside a layer count and printed architecture.
- Examples
- Counts Gemma 3 at 268,098,176 parameters, described as a 270M-parameter model.
- What you can do after
- Evaluate model scale by summing numel() across model.parameters().
Deep Learning with R, Third Edition
Model size is the amount of structural and parameter capacity in a machine learning model, determined by features such as layer and unit counts.
- How it is used
- Uses model size to balance sufficient capacity against overfitting on a given dataset.
- How deep it goes
- Describes an iterative validation-loss workflow but gives no formula for calculating parameter counts.
- Examples
- Compares IMDb review classifiers with four, 16, and 512 units in each hidden layer.
- What you can do after
- Evaluate candidate model sizes by comparing validation loss as layers or layer sizes increase.
Read how Deep Learning with R, Third Edition explains parameter count →
Build a DeepSeek Model (From Scratch)
Parameter count is the number of learned numerical parameters in a machine-learning model.
- How it is used
- Quantifies model scale when comparing dense-feed-forward costs and teacher–student size differences.
- How deep it goes
- Compares parameter totals and architectural implications, but supplies no general procedure for counting parameters.
- Examples
- Uses dense transformer FFNs and a 26,000- versus 8,800-parameter teacher–student pair.
- What you can do after
- Evaluate model-size trade-offs by relating parameter totals to dense computation costs and teacher–student differences.
Read how Build a DeepSeek Model (From Scratch) explains parameter count →
Build Applications with Local AI Models on a Mac
Model size is a measure of a language model's scale, commonly expressed by its number of learned parameters.
- How it is used
- Guides local model selection by weighing quality, speed, and available memory.
- How deep it goes
- Defines learned parameters and compares size tradeoffs, including total versus active parameters in a mixture-of-experts model.
- Examples
- Compares Gemma 3 size tags and five MLX Whisper variants with measured transcription speeds.
- What you can do after
- Evaluate/understand model options by parameter count, task suitability, and approximate resource requirements.
Read how Build Applications with Local AI Models on a Mac explains parameter count →
Software Testing with Generative AI
The number of learned statistical parameters, or weights, in a machine-learning model.
- How it is used
- Uses parameter count to compare language models and estimate whether their resource demands may justify their value.
- How deep it goes
- Defines parameters as statistical weights and connects their quantity to performance and cost, stopping short of model internals.
- Examples
- Uses 175-billion-parameter and 1-trillion-parameter language models as comparison examples.
- What you can do after
- Evaluate/understand model scale as a rough performance and cost indicator without treating it as proof of quality.
Read how Software Testing with Generative AI explains parameter count →
Timeless Algorithms, The Foundational Ideas
Model size is the scale of a model's structure or parameterization.
- How it is used
- Measures model size as one resource for comparing performance across training conditions.
- How deep it goes
- Defines parameter count as capacity and describes declining held-out loss, but limits the finding to tested settings and supported training conditions.
- Examples
- Compares more than 400 transformer language models using next-token prediction and held-out cross-entropy.
- What you can do after
- Evaluate how held-out loss changes with parameter count while accounting for training data and compute.
Read how Timeless Algorithms, The Foundational Ideas explains parameter count →
Also covered in
- AI Agents in Action
- How Large Language Models Work
- Sutskever's List
- Retrieval Augmented Generation, The Foundational IdeasIn early access (MEAP)
- LLMs in Production
- The Design of Web APIs, Second Edition
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- neural networkA neural network is a computational model composed of interconnected, parameterized units or transformations that learn to map inputs to outputs.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- LlamaLlama is a family of large language models developed by Meta.
Related concepts
- parametermeasures · A parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
- computeproduces · Compute is the processing capacity, resources, or computational work involved in executing software and other computational workloads.
- generalizationcontrasts with · Generalization is the extension of learned patterns, abstractions, or conclusions from observed cases to new cases beyond those from which they were derived.
- memory bandwidthcontrasts with · Memory bandwidth is the rate at which data can be transferred between a memory system and a processor or other computing component.
Questions about parameter count
What is parameter count?
Parameter count is the number of numerical parameters in a computational model, typically the learned parameters of a machine-learning model.
Which Manning books explain parameter count?
Rearchitecting LLMs, Deep Learning with R, Third Edition, Build a DeepSeek Model (From Scratch), Build Applications with Local AI Models on a Mac, Software Testing with Generative AI and Timeless Algorithms, The Foundational Ideas.
What should I know before learning parameter count?
Manning books that teach parameter count usually introduce language model, neural network, accuracy and Llama first.
What is the difference between parameter count and generalization?
Parameter count is the number of numerical parameters in a computational model, typically the learned parameters of a machine-learning model. Generalization is the extension of learned patterns, abstractions, or conclusions from observed cases to new cases beyond those from which they were derived.
What is parameter count also called?
Parameter count is also called model size.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





