liveBook · Manning

Parameter

As explained in Context Engineering

A model parameter is a learned numerical value, such as a weight or bias, that determines how a model transforms its inputs.

A model parameter is a learned weight or bias that encodes patterns from training data and helps determine how an LLM transforms one token sequence into the next. Models are often named by parameter count, such as 8B or 70B. section 1.2.1

Parameters differ from hyperparameters in how they are obtained and used:

  • Parameters are learned during training; hyperparameters are configuration values chosen by practitioners, including inference-time settings that affect token generation. section 1.2.1
  • Parameter count alone does not establish model quality or computational cost; mixture-of-experts architectures may activate only part of the model for an input. section 1.2.1

How Context Engineering covers it

How deep it goes
Defines weights and biases, contrasts hyperparameters, and cautions against treating parameter count as a quality or compute measure.
Examples
Uses weights, biases, and 8B or 70B parameter counts as examples.
What you can do after
Understand what model parameters encode and distinguish learned values from practitioner-chosen configuration.
How it connects
Connects parameters to hyperparameters, tokens, training data, and mixture-of-experts architectures.

Also explained in

Compare how 41 books explain parameter →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.