A parameter is a learned numerical value that controls how a neural network transforms inputs into outputs. In large language models, parameters are weights and biases; weights are stored in matrices, and together billions or trillions of values encode learned knowledge and capabilities. section 1.1.1 section 3.1.2
Parameters begin as random numbers when training a model from scratch. Training repeatedly measures error with a loss function, then uses gradient descent to adjust parameters toward outputs that better match the targets. section 3.1.2
Fine-tuning changes parameters differently from initial training:
- It starts with a pretrained model rather than a blank network, retaining broad capabilities while adapting behavior for a task or domain. section 1.4.1
- It uses curated prompt-completion pairs, such as a refund request paired with a policy-aligned customer-service reply, to steer desired outputs. section 1.4.1
- It typically applies lower learning rates, making selective adjustments that preserve foundational knowledge while adding specialized information. section 1.4.1
Sentiment-analysis update
If a model assigns only 30% probability to the correct label, positive, cross-entropy loss penalizes its uncertainty. Gradient descent uses that error signal to adjust the parameters so future predictions can become more accurate. section 3.1.2
Figure 3.5 Gradient descent principle illustrated with a single weight variable. For LLMs, the same principle applies to billions of parameters.

A single-weight diagram makes the update principle visible before extending it to the billions of parameters in an LLM. section 3.1.2








































