liveBook · Manning

Parameter count

As explained in Rearchitecting LLMs

Parameter count is the number of learned scalar parameters in a machine-learning model.

Parameter count is the number of learned scalar parameters in a model. Inspecting it before modification establishes the model’s size; the total can be computed by summing the element counts of its parameter tensors. section 2.2

original_params= sum(p.numel() for p in model.parameters())
print(f"Total parameters: {original_params}")
print(f"Original layers: {len(model.model.layers)}")
print("=" * 20)
print(model)

The code sums parameter-tensor element counts and prints the model’s layer count and architecture. section 2.2

How Rearchitecting LLMs covers it

How deep it goes
Computes the total by summing each parameter tensor’s element count, alongside a layer count and printed architecture.
Examples
Counts Gemma 3 at 268,098,176 parameters, described as a 270M-parameter model.
What you can do after
Evaluate model scale by summing numel() across model.parameters().
How it connects
Links parameter count to model architecture and layer count during inspection.

Also explained in

Compare how 6 books explain parameter count →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.