A model weight is one statistical value contributing to an LLM’s computations. A model’s parameter count describes how many such weights it contains; each provides part of the model’s statistical puzzle. section 2.5.1
When comparing models, model weights help frame:
- Scale: parameter counts may range from 175 billion to 1 trillion. section 2.5.1
- Cost: larger parameter counts generally make models more expensive to run. section 2.5.1
- Expected capability: more parameters may support better performance, but parameter count alone does not determine quality. section 2.5.1
