liveBook · Manning

Parameter

As explained in Building Specialized AI Systems

A learned numerical value that controls a model's computation.

A parameter is a learned numerical value that controls how a neural network transforms inputs into outputs. In large language models, parameters are weights and biases; weights are stored in matrices, and together billions or trillions of values encode learned knowledge and capabilities. section 1.1.1 section 3.1.2

Parameters begin as random numbers when training a model from scratch. Training repeatedly measures error with a loss function, then uses gradient descent to adjust parameters toward outputs that better match the targets. section 3.1.2

Fine-tuning changes parameters differently from initial training:

  • It starts with a pretrained model rather than a blank network, retaining broad capabilities while adapting behavior for a task or domain. section 1.4.1
  • It uses curated prompt-completion pairs, such as a refund request paired with a policy-aligned customer-service reply, to steer desired outputs. section 1.4.1
  • It typically applies lower learning rates, making selective adjustments that preserve foundational knowledge while adding specialized information. section 1.4.1

Sentiment-analysis update

If a model assigns only 30% probability to the correct label, positive, cross-entropy loss penalizes its uncertainty. Gradient descent uses that error signal to adjust the parameters so future predictions can become more accurate. section 3.1.2

Figure 3.5 Gradient descent principle illustrated with a single weight variable. For LLMs, the same principle applies to billions of parameters.

A single-weight diagram makes the update principle visible before extending it to the billions of parameters in an LLM. section 3.1.2

How Building Specialized AI Systems covers it

How deep it goes
Defines parameters as weights and biases, then works through loss-guided gradient-descent updates without detailing full optimization internals.
Examples
Uses trillion-scale LLMs, refund-response prompt-completion pairs, and a positive sentiment prediction with 30% probability.
What it assumes
Introduces parameters from random initialization and explains their training role before assuming knowledge of gradient descent mechanics.
What you can do after
Evaluate/understand how loss-guided updates and careful fine-tuning alter model behavior without rebuilding a model from scratch.
How it connects
Connects parameters to neural networks, weights, biases, loss functions, gradient descent, fine-tuning, and transfer learning.

Also explained in

Compare how 41 books explain parameter →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Specialized AI Systems. The section links point to the parts of the book it draws on.